Commit Graph

7851 Commits

Author SHA1 Message Date
Nicholas Tindle
704e2959ba fix(backend): add file extension from Content-Type header for URLs
When downloading files from URLs without file extensions (like Unsplash
images), detect the extension from the Content-Type response header.
For example, `image/jpeg` becomes `.jpg`.

Also updates FileStoreBlock description to clarify behavior in both
CoPilot (saves to workspace) and graphs (outputs data URI).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 03:53:25 -06:00
Nicholas Tindle
cc95f1dbd6 fix(blocks): FileStoreBlock now saves to workspace in CoPilot
Change FileStoreBlock to use `for_block_output` return format instead
of `for_local_processing`. This makes files automatically save to the
user's workspace when running in CoPilot, while maintaining backward
compatibility for graph executions (returns data URI).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 03:40:04 -06:00
Nicholas Tindle
7357c26c00 fix(workspace): add trailing slash to session path prefix
Prevents session path prefix collisions where "/sessions/abc" could
incorrectly match "/sessions/abc123" when using startswith filtering.
Now uses "/sessions/abc/" to ensure exact session boundary matching.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 03:28:48 -06:00
Nicholas Tindle
5d82a42b49 refactor(backend): extract shared GCS utilities from storage modules
Consolidate duplicated GCS code from workspace_storage.py and
cloud_storage.py into a new gcs_utils.py module. Extracts path parsing,
fresh-session downloads, and signed URL generation into reusable
utilities while preserving existing authorization and logging behavior.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 03:28:19 -06:00
Nicholas Tindle
d2fca0adbd fix(workspace): add proper error logging to file download fallback
Log exceptions with context (file.id, storagePath) when signed URL
generation fails, and wrap fallback streaming in try/except to avoid
silently swallowing errors.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 03:24:53 -06:00
Nicholas Tindle
0acf868b18 fix(backend): clean up orphaned storage files on DB errors
- Add _get_effective_path() helper to deduplicate path resolution logic
  between list_files and get_file_count methods
- Add broader exception handling in write_file to clean up storage files
  when create_workspace_file fails with non-UniqueViolationError errors
- Fix test mock to include required graph_version attribute

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 03:13:15 -06:00
Nicholas Tindle
d81e97b881 fix(chat): use workspace:// reference for large file downloads
Return workspace:// format instead of calling get_download_url so
the frontend urlTransform can properly add the proxy prefix.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 02:57:36 -06:00
Nicholas Tindle
ef3dfb8af4 test(executor): add graph_version to mock objects
Fixes test failures after changing ExecutionContext to use
graph_exec.graph_version instead of the parameter.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 02:55:00 -06:00
Nicholas Tindle
eb22180e6d Merge branch 'dev' into user-workspace 2026-01-28 02:13:07 -06:00
Nicholas Tindle
f3d8d953f5 style(chat): fix import ordering in workspace_tools
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 02:12:55 -06:00
Nicholas Tindle
2b0afc348e fix(workspace): use sanitized filename for non-ASCII Content-Disposition
The RFC5987 encoding path was using the original filename instead of
the sanitized version, bypassing CR/LF/null byte removal.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 02:10:30 -06:00
Nicholas Tindle
e3a389ba00 fix(chat): use configurable max_file_size_mb in tool description
The description was hardcoded to "100MB" but the actual limit is
configurable via Config().max_file_size_mb.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 02:09:34 -06:00
Nicholas Tindle
f7c59b00d8 fix(executor): use graph_exec.graph_version instead of parameter in ExecutionContext
The graph_version parameter can be None, but graph_exec.graph_version
always has the concrete version from create_graph_execution.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 02:00:37 -06:00
Nicholas Tindle
1feed23475 fix(blocks): use for_external_api when base_64=True in AgentFileInputBlock
Same fix as FileStoreBlock - for_block_output can return workspace://
in CoPilot, for_external_api guarantees data URI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 01:53:04 -06:00
Nicholas Tindle
3a295b3192 fix(blocks): use for_external_api when base_64=True in FileStoreBlock
for_block_output can return workspace:// in CoPilot, violating the
"Produce Base64 Output" promise. for_external_api guarantees data URI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 01:51:48 -06:00
Nicholas Tindle
4872eb3ccd fix(backend): handle orphaned storage on retry failure and add session_id
- Wrap overwrite retry in try/except to clean up storage file if retry fails
- Pass source_session_id to write_file in store_media_file for proper tracking

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 01:45:15 -06:00
Nicholas Tindle
f5d7b3f618 fix(backend): use single UUID for storage and database file records
Previously, write_file() generated a UUID for storage paths and let Prisma
auto-generate a separate UUID for the database record. This caused download
URLs to return 404 because the storage layer extracted the wrong ID.

Now the same UUID is used for both, fixing the download URL issue.

Also consolidates MAX_FILE_SIZE_BYTES into Config.max_file_size_mb setting
for consistent configuration across file.py, workspace.py, and workspace_tools.py.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 01:42:57 -06:00
Nicholas Tindle
de9ef2366e fix(platform): address workspace file handling issues
Backend fixes:
- Extract _create_streaming_response helper to deduplicate Response creation
- Fix orphaned storage file when race condition occurs with overwrite=False
- Fix soft-delete breaking unique path constraint by modifying path on delete
- Normalize path prefixes in list_files/get_file_count with include_all_sessions
- Remove unused exists() method from storage backend abstraction

Frontend fixes:
- Remove unnecessary transformUrl wrapper, use resolveWorkspaceUrl directly

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 01:25:19 -06:00
Nicholas Tindle
0953983944 feat(platform): disable onboarding redirects and add $5 signup bonus (#11862)
Disable automatic onboarding redirects on signup/login while keeping the
checklist/wallet functional. Users now receive $5 (500 credits) on their
first visit to /copilot.

### Changes 🏗️

- **Frontend**: `shouldShowOnboarding()` now returns `false`, disabling
auto-redirects to `/onboarding`
- **Backend**: Added `VISIT_COPILOT` onboarding step with 500 credit
($5) reward
- **Frontend**: Copilot page automatically completes `VISIT_COPILOT`
step on mount
- **Database**: Migration to add `VISIT_COPILOT` to `OnboardingStep`
enum

NOTE: /onboarding/1-welcome -> /library now as shouldShowOnboardin is
always false

Users land directly on `/copilot` after signup/login and receive $5
invisibly (not shown in checklist UI).

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] New user signup (email/password) → lands on `/copilot`, wallet
shows 500 credits
- [x] Verified credits are only granted once (idempotent via onboarding
reward mechanism)
- [x] Existing user login (already granted flag set) → lands on
`/copilot`, no duplicate credits
  - [x] Checklist/wallet remains functional

#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

No configuration changes required.

---

OPEN-2967

🤖 Generated with [Claude Code](https://claude.ai/code)


<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Introduces a new onboarding step and adjusts onboarding flow.
> 
> - Adds `VISIT_COPILOT` onboarding step (+500 credits) with DB enum
migration and API/type updates
> - Copilot page auto-completes `VISIT_COPILOT` on mount to grant the
welcome bonus
> - Changes `/onboarding/enabled` to require user context and return
`false` when `CHAT` feature is enabled (skips legacy onboarding)
> - Wallet now refreshes credits on any onboarding `step_completed`
notification; confetti limited to visible tasks
> - Test flows updated to accept redirects to `copilot`/`library` and
verify authenticated state
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
ec5a5a4dfd. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <ntindle@users.noreply.github.com>
2026-01-28 07:22:46 +00:00
Nicholas Tindle
c4d83505c0 fix(backend): fix remaining block test failures
- Fix invalid base64 padding in FalVideoGeneratorBlock test mock
- Add data URI mock to BannerbearTextOverlayBlock to avoid HTTP 404

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 01:21:40 -06:00
Nicholas Tindle
28e4da5b13 fix(backend): use data URIs in block tests to avoid HTTP requests
Block tests were failing because store_media_file tried to download
mock URLs that don't exist. Changed test mocks to return data URIs
instead, and updated test_output to use lambda validators that accept
either workspace:// refs or data: URIs.

Affected blocks:
- AIImageCustomizerBlock
- FluxKontextBlock
- FalVideoGeneratorBlock
- AIShortformVideoCreatorBlock
- AIProductAdvertCreatorBlock
- AIScreenshotToVideoBlock
- TalkingHeadBlock

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 01:11:12 -06:00
Nicholas Tindle
72b1542a43 refactor(backend): remove unused workspace_file_exists function
This function was defined but never imported or called anywhere.
Callers can use get_workspace_file_by_path() directly and check
if the result is None.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 01:02:46 -06:00
Nicholas Tindle
5bbc3d55f0 fix(frontend): correct path length check for workspace download requests
The isWorkspaceDownloadRequest function was checking for path.length >= 4,
but the pattern api/workspace/files/{id}/download has 5 segments. This
allowed malformed requests missing the file_id to be incorrectly routed
through the binary download handler.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 01:00:07 -06:00
Nicholas Tindle
87814bcdcb docs(blocks): update multimedia block docs for automatic output format
Update manual sections in Add Audio To Video and Loop Video blocks
to reflect that output format is now automatically determined
(workspace:// in CoPilot, data URIs in graph executions) instead
of requiring a manual output_return_type parameter.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:55:23 -06:00
Nicholas Tindle
85c229dd6c fix(frontend): exclude video refs from image rendering in chat
Video workspace references (video_url, video_out, etc.) were being
incorrectly rendered as <img> tags. Added video keyword check to
exclude them before defaulting to image rendering.

TODO: Replace keyword matching with MIME type encoded in workspace ref
(e.g., workspace://abc123#video/mp4) for robust media type detection.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:50:13 -06:00
Nicholas Tindle
270586751b style(backend): format workspace routes and rest_api imports
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:40:12 -06:00
Nicholas Tindle
fa1afd6a6d fix(backend): replace assert with ValueError in FileReadBlock
Asserts can be stripped with -O flag. Use explicit ValueError for
graph_exec_id validation to ensure consistent error handling.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:39:35 -06:00
Nicholas Tindle
af2bcd900a fix(backend): remove dead output_return_type from media blocks
The output_return_type field was defined in Input but never wired up to
store_media_file. The code always used for_block_output. Removed the
misleading field from LoopVideoBlock and AddAudioToVideoBlock.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:39:27 -06:00
Nicholas Tindle
8f7204484d fix(backend): handle race condition in WorkspaceManager.write_file
- Wrap create_workspace_file in try/except for UniqueViolationError
- On conflict with overwrite=True, delete existing and retry
- Remove unused file_exists method (dead code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:39:14 -06:00
Nicholas Tindle
5af4c60b8e fix(backend): use upsert in get_or_create_workspace to prevent race condition
Concurrent first-time requests for the same user could both find no workspace
and attempt to create, causing unique constraint violation. Using upsert
handles this atomically.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:39:01 -06:00
Nicholas Tindle
49b67ccd94 fix(backend): resolve circular import in file.py and remove deprecated params
- Defer WorkspaceManager import to inside store_media_file() to break circular import
- Remove deprecated return_content and save_to_workspace parameters (no callers)
- Make return_format a required parameter
- Update tests to use return_format

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:38:51 -06:00
Nicholas Tindle
efb2e2792d fix(backend): resolve circular import in workspace_storage.py
Defer sanitize_filename import to inside _build_file_path() to break
the circular import chain: workspace_storage → file → workspace → data → blocks → file

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:38:40 -06:00
Nicholas Tindle
d51d811497 fix(backend): security fixes and dead code removal
- routes.py: Sanitize filename in Content-Disposition header to prevent
  header injection (RFC5987 encoding for non-ASCII)
- http.py: Replace assert with explicit ValueError for graph_exec_id check
  (asserts can be stripped with -O)
- workspace.py: Remove unused functions (get_workspace_by_id,
  hard_delete_workspace_file, update_workspace_file)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:14:22 -06:00
Nicholas Tindle
83f93d00f4 fix(backend): add shutdown hook for workspace storage
Add shutdown_workspace_storage() to properly close GCS aiohttp sessions
during application shutdown. Follows the same pattern as cloud_storage.py.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:09:04 -06:00
Nicholas Tindle
c132b6dfa5 fix(backend): prevent path traversal in local workspace storage
- Sanitize filenames using sanitize_filename() before building paths
- Add is_relative_to() check after path resolution for defense in depth
- Replace string comparison with is_relative_to() in _parse_storage_path()
  for robust path containment on case-insensitive filesystems

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:04:53 -06:00
Nicholas Tindle
7eb7b7186f fix: lint 2026-01-28 00:01:24 -06:00
Nicholas Tindle
4b58eac877 fix(backend): prevent race condition in concurrent node execution context
Use model_copy() instead of mutating shared ExecutionContext to prevent
race conditions when multiple nodes execute concurrently. Each node now
gets its own isolated copy with correct node_id and node_exec_id values.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:01:05 -06:00
Nicholas Tindle
bae6be915f fix(backend): replace graph_exec_id or "" fallbacks with asserts
The empty string fallback was dead code since store_media_file() validates
graph_exec_id before these lines execute. Replace with explicit asserts
for clearer failure if assumptions are ever violated.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:39:56 -06:00
Nicholas Tindle
8f16d583a4 chore(frontend): update openapi.json after workspace route cleanup
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:33:03 -06:00
Nicholas Tindle
0b8c671a27 chore(backend): remove unused workspace REST API routes
Keep only GET /files/{file_id}/download which is used by frontend chat
to render workspace:// images. Remove 10 unused endpoints and models.py.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:31:02 -06:00
Nicholas Tindle
cb074b0076 refactor(backend): extract shared download logic into helper function
Both download_file and download_file_by_path now use _create_file_download_response()
to eliminate ~40 lines of duplicated download handling code.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:27:23 -06:00
Nicholas Tindle
f29dd34f51 fix(backend): include path filter in workspace file count
get_file_count() now accepts path parameter to match list_files() filtering,
fixing pagination totals when filtering by path prefix.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:25:06 -06:00
Nicholas Tindle
581dc337f2 chore(backend): remove unused UploadFileRequest model
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:23:11 -06:00
Nicholas Tindle
f8b041fd63 fix(backend): respect session scoping in workspace file count
get_file_count() now uses the same session scoping logic as list_files(),
ensuring consistent totals when include_all_sessions is false.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:22:27 -06:00
Nicholas Tindle
56248ae7b7 chore(backend): remove unused WORKSPACE_FILE_INFO enum value
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:02:37 -06:00
Nicholas Tindle
bec0157f9e Update migration to retain 'search' column
Removed the dropping of the 'search' column and its associated index from the migration script.
2026-01-27 23:01:19 -06:00
Nicholas Tindle
57f44e166a fix(backend): update HTTP block tests for execution_context
Update SendAuthenticatedWebRequestBlock to use execution_context
instead of separate graph_exec_id/user_id parameters, matching
the parent class signature.

Update test_http.py to pass execution_context to all test calls.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 22:59:52 -06:00
Nicholas Tindle
2c678f2658 refactor(backend): rename return_format options for clarity and add auto-fallback
Rename store_media_file() return_format options to make intent clear:
- "local_path" -> "for_local_processing" (ffmpeg, MoviePy, PIL)
- "data_uri" -> "for_external_api" (Replicate, OpenAI APIs)
- "workspace_ref" -> "for_block_output" (auto-adapts to context)

The "for_block_output" format now gracefully handles both contexts:
- CoPilot (has workspace): returns workspace:// reference
- Graph execution (no workspace): falls back to data URI

This prevents blocks from failing in graph execution while still
providing workspace persistence in CoPilot.

Also adds documentation to CLAUDE.md, new_blocks.md, and
block-sdk-guide.md explaining when to use each format.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 21:53:09 -06:00
Nicholas Tindle
669e33d709 chore: remove IDEAS.md from tracking
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 21:15:17 -06:00
Nicholas Tindle
953e7a5afb refactor(backend): replace return_content/save_to_workspace with return_format
Simplify store_media_file API with a single return_format parameter:

- "local_path": Return relative path (for local processing like MoviePy)
- "data_uri": Return base64 data URI (for external APIs like Replicate)
- "workspace_ref": Save to workspace and return workspace://id (for CoPilot)

This replaces the confusing combination of return_content and save_to_workspace
parameters. The old parameters are deprecated but still work via a compatibility
layer.

Updated all blocks to use the new explicit return_format parameter:
- Local processing: return_format="local_path"
- External APIs: return_format="data_uri"
- CoPilot outputs: return_format="workspace_ref"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 21:12:29 -06:00