Compare commits

...

39 Commits

Author SHA1 Message Date
Siddharth Ganesan
8b87a07508 Fix lint 2026-02-04 16:51:25 -08:00
Siddharth Ganesan
728463ace7 Fix stream buffer 2026-02-04 16:51:25 -08:00
Siddharth Ganesan
8cea43d926 LUAAAA 2026-02-04 16:51:25 -08:00
Siddharth Ganesan
2198a6caae Fix edge issue 2026-02-04 16:51:25 -08:00
Siddharth Ganesan
0ba8c0ad29 Streaming seems to work but copilot is dumb 2026-02-04 16:51:25 -08:00
Siddharth Ganesan
a9e2f4a82e Things are broken 2026-02-04 16:51:25 -08:00
Siddharth Ganesan
1e99f45590 Fix abort 2026-02-04 16:51:25 -08:00
Siddharth Ganesan
1ec0ec4c1a Streaming 2026-02-04 16:51:25 -08:00
Siddharth Ganesan
2ce78e8c60 Checkpoint 2026-02-04 16:51:24 -08:00
Siddharth Ganesan
ea759b2d00 BROKEN 2026-02-04 16:51:24 -08:00
Siddharth Ganesan
09d7fc671f Fix 2026-02-04 16:51:24 -08:00
Siddharth Ganesan
ea210a56a8 Improvement 2026-02-04 16:51:24 -08:00
Siddharth Ganesan
c05b70be1e mcp v1 2026-02-04 16:51:24 -08:00
Siddharth Ganesan
413c53208e Add mcp 2026-02-04 16:51:24 -08:00
Siddharth Ganesan
b8ccd71423 Stuff 2026-02-04 16:51:24 -08:00
Siddharth Ganesan
8f556684a6 Ss tests 2026-02-04 16:51:24 -08:00
Siddharth Ganesan
a391019995 Basic ss tes 2026-02-04 16:51:24 -08:00
Siddharth Ganesan
d6179e7691 v1 2026-02-04 16:51:23 -08:00
Siddharth Ganesan
e68e653d5c v0 2026-02-04 16:51:23 -08:00
Waleed
36ec68d93e fix(serializer): validate required fields for blocks without tools (#3137) 2026-02-04 16:47:18 -08:00
Vikhyath Mondreti
fce566cc2f fix(mistral): restore mistral configs for v2 version (#3138) 2026-02-04 16:03:04 -08:00
Siddharth Ganesan
1933e1aad5 improvement(openai): migrate to responses api (#3135)
* Migrate openai to use responses api

* Consolidate azure

* Fix streaming

* Bug fixes

* Bug fixes

* Fix responseformat

* Refactor

* Fix bugs

* Fix

* Fix azure openai response format with tool calls

* Fixes

* Fixes

* Fix temp
2026-02-04 15:50:18 -08:00
Waleed
793adda986 fix(limits): updated rate limiter to match execution timeouts, adjusted timeouts fallback to be free plan (#3136)
* fix(limits): updated rate limiter to match execution timeouts, adjusted timeouts fallback to be free plan

* upgrade turborepo
2026-02-04 15:31:53 -08:00
Waleed
8d846c5983 feat(async-jobs): async execution with job queue backends (#3134)
* feat(async-jobs): async execution with job queue backends

* added migration

* remove unused envvar, remove extraneous comments

* ack comment

* same for db

* added dedicated async envvars for timeouts, updated helm

* updated comment

* ack comment

* migrated routes to be more restful

* ack comments

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 14:52:33 -08:00
Vikhyath Mondreti
362f4c2918 improvement(timeouts): sync to 50 min, self-hosted maxed out (#3133)
* improvement(timeouts): sync to 50 min, self-hosted maxed out

* update env vars
2026-02-04 11:27:41 -08:00
Waleed
c77e351067 fix(providers): correct tool calling message format across all providers (#3132)
* fix(providers): correct tool calling message format across all providers

* fix(bedrock): correct timestamp char count in comment

* chore(gemini): remove dead executeToolCall function

* remove unused var
2026-02-04 11:02:49 -08:00
Vikhyath Mondreti
a627faabe7 feat(timeouts): execution timeout limits (#3120)
* feat(timeouts): execution timeout limits

* fix type issues

* add to docs

* update stale exec cleanup route

* update more callsites

* update tests

* address bugbot comments

* remove import expression

* support streaming and async paths'

* fix streaming path

* add hitl and workflow handler

* make sync path match

* consolidate

* timeout errors

* validation errors typed

* import order

* Merge staging into feat/timeout-lims

Resolved conflicts:
- stt/route.ts: Keep both execution timeout and security imports
- textract/parse/route.ts: Keep both execution timeout and validation imports
- use-workflow-execution.ts: Keep cancellation console entry from feature branch
- input-validation.ts: Remove server functions (moved to .server.ts in staging)
- tools/index.ts: Keep execution timeout, use .server import for security

* make run from block consistent

* revert console update change

* fix subflow errors

* clean up base 64 cache correctly

* update docs

* consolidate workflow execution and run from block hook code

* remove unused constant

* fix cleanup base64 sse

* fix run from block tracespan
2026-02-04 10:26:36 -08:00
Vikhyath Mondreti
f811594875 improvement(rooms): redis client closed should fail with indicator (#3115)
* improvement(rooms): redis client closed should fail fast

* bugbot comment

* consolidate
2026-02-03 23:48:46 -08:00
Emir Karabeg
0bc245b7a9 feat(note-block): note block preview newlines (#3127)
* feat(note-block): add single newline support in preview

Add remark-breaks plugin to the note block markdown renderer to convert
single newlines into line breaks. This fixes the issue where users had
to use double newlines (\n\n) to create visible line breaks in the
note block preview.

Co-authored-by: Emir Karabeg <emir-karabeg@users.noreply.github.com>

* Revert "feat(note-block): add single newline support in preview"

This reverts commit 049b42502a.

* feat(note-block): add single newline support in preview

Add a preprocessor function that converts single newlines to markdown
hard breaks (two trailing spaces + newline) before rendering. This
ensures that when users press Enter in the note block editor, the
line break shows up in the preview.

The function preserves:
- Double newlines (paragraph breaks)
- Code block formatting (fenced and inline)

Co-authored-by: Emir Karabeg <emir-karabeg@users.noreply.github.com>

* refactor(note-block): simplify comments

Co-authored-by: Emir Karabeg <emir-karabeg@users.noreply.github.com>

* added remark-breaks to allow single new line

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Emir Karabeg <emir-karabeg@users.noreply.github.com>
Co-authored-by: waleed <walif6@gmail.com>
2026-02-03 20:30:39 -08:00
Waleed
0a08ac03b9 fix(import): preserve workflow colors during import (#3130)
* fix(import): preserve workflow colors during import

* cleanup
2026-02-03 20:18:08 -08:00
Waleed
7977ac88ca fix(editor): block rename applies to correct block when selection changes (#3129) 2026-02-03 20:06:03 -08:00
Vikhyath Mondreti
5b0c2156e0 improvement(files): pass user file objects around consistently (#3119)
* improvement(collab): do not refetch active workflow id

* progress on files

* more integrations

* separate server and client logic

* consolidate more code

* fix integrations

* fix types

* consolidate more code

* fix tests

* fix more bugbot comments

* fix type check

* fix circular impport

* address more bugbot comments

* fix ocr integrations

* fix typing

* remove leftover type

* address bugbot comment

* fix file block adv mode

* fix

* normalize file input

* fix v2 blocmks for ocr

* fix for v2 versions

* fix more v2 blocks

* update single file blocks

* make interface simpler

* cleanup fireflies

* remove file only annotation

* accept all types

* added wand to ssh block

* user files should be passed through

* improve docs

* fix slack to include successful execs

* fix dropbox upload file

* fix sendgrid

* fix dropbox

* fix

* fix

* update skills

* fix uploaded file

---------

Co-authored-by: waleed <walif6@gmail.com>
2026-02-03 19:50:23 -08:00
Waleed
4db6e556b7 feat(canvas): added the ability to lock blocks (#3102)
* feat(canvas): added the ability to lock blocks

* unlock duplicates of locked blocks

* fix(duplicate): place duplicate outside locked container

When duplicating a block that's inside a locked loop/parallel,
the duplicate is now placed outside the container since nothing
should be added to a locked container.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(duplicate): unlock all blocks when duplicating workflow

- Server-side workflow duplication now sets locked: false for all blocks
- regenerateWorkflowStateIds also unlocks blocks for templates
- Client-side regenerateBlockIds already handled this (for paste/import)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix code block disabled state, allow unlock from editor

* fix(lock): address code review feedback

- Fix toggle enabled using first toggleable block, not first block
- Delete button now checks isParentLocked
- Lock button now has disabled state
- Editor lock icon distinguishes block vs parent lock state

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(lock): prevent unlocking blocks inside locked containers

- Editor: can't unlock block if parent container is locked
- Action bar: can't unlock block if parent container is locked
- Shows "Parent container is locked" tooltip in both cases

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(lock): ensure consistent behavior across all UIs

Block Menu, Editor, Action Bar now all have identical behavior:
- Enable/Disable: disabled when locked OR parent locked
- Flip Handles: disabled when locked OR parent locked
- Delete: disabled when locked OR parent locked
- Remove from Subflow: disabled when locked OR parent locked
- Lock: always available for admins
- Unlock: disabled when parent is locked (unlock parent first)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(enable): consistent behavior - can't enable if parent disabled

Same pattern as lock: must enable parent container first before
enabling children inside it.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs(quick-reference): add lock block action

Added documentation for the lock/unlock block feature (admin only).
Note: Image placeholder added, pending actual screenshot.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* remove prefix square brackets in error notif

* add lock block image

* fix(block-menu): paste should not be disabled for locked selection

Paste creates new blocks, doesn't modify selected ones. Changed from
disableEdit (includes lock state) to !userCanEdit (permission only),
matching the Duplicate action behavior.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(workflow): extract block deletion protection into shared utility

Extract duplicated block protection logic from workflow.tsx into
a reusable filterProtectedBlocks helper in utils/block-protection-utils.ts.
This ensures consistent behavior between context menu delete and
keyboard delete operations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(workflow): extend block protection utilities for edge protection

Add isEdgeProtected, filterUnprotectedEdges, and hasProtectedBlocks
utilities. Refactor workflow.tsx to use these helpers for:
- onEdgesChange edge removal filtering
- onConnect connection prevention
- onNodeDragStart drag prevention
- Keyboard edge deletion
- Block menu disableEdit calculation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(lock): address review comments for lock feature

1. Store batchToggleEnabled now uses continue to skip locked blocks
   entirely, matching database operation behavior

2. Copilot add operation now checks if parent container is locked
   before adding nested nodes (defensive check for consistency)

3. Remove unused filterUnprotectedEdges function

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(copilot): add lock checks for insert and extract operations

- insert_into_subflow: Check if existing block being moved is locked
- extract_from_subflow: Check if block or parent subflow is locked

These operations now match the UI behavior where locked blocks
cannot be moved into/out of containers.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(lock): prevent duplicates inside locked containers via regenerateBlockIds

1. regenerateBlockIds now checks if existing parent is locked before
   keeping the block inside it. If parent is locked, the duplicate
   is placed outside (parentId cleared) instead of creating an
   inconsistent state.

2. Remove unnecessary effectivePermissions.canAdmin and potentialParentId
   from onNodeDragStart dependency array.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(lock): fix toggle locked target state and draggable check

1. BATCH_TOGGLE_LOCKED now uses first block from blocksToToggle set
   instead of blockIds[0], matching BATCH_TOGGLE_ENABLED pattern.
   Also added early exit if blocksToToggle is empty.

2. Blocks inside locked containers are now properly non-draggable.
   Changed draggable check from !block.locked to use isBlockProtected()
   which checks both block lock and parent container lock.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(copilot): check parent lock in edit and delete operations

Both edit and delete operations now check if the block's parent
container is locked, not just if the block itself is locked. This
ensures consistent behavior with the UI which uses isBlockProtected
utility that checks both direct lock and parent lock.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(socket): add server-side lock validation and admin-only permissions

1. BATCH_TOGGLE_LOCKED now requires admin role - non-admin users with
   write role can no longer bypass UI restriction via direct socket
   messages

2. BATCH_REMOVE_BLOCKS now validates lock status server-side - filters
   out protected blocks (locked or inside locked parent) before deletion

3. Remove duplicate/outdated comment in regenerateBlockIds

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(socket): update permission test for admin-only lock toggle

batch-toggle-locked is now admin-only, so write role should be denied.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(undo-redo): use consistent target state for toggle redo

The redo logic for BATCH_TOGGLE_ENABLED and BATCH_TOGGLE_LOCKED was
incorrectly computing each block's new state as !previousStates[blockId].
However, the store's batchToggleEnabled/batchToggleLocked set ALL blocks
to the SAME target state based on the first block's previous state.

Now redo computes targetState = !previousStates[firstBlockId] and applies
it to all blocks, matching the store's behavior.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(socket): add comprehensive lock validation across operations

Based on audit findings, adds lock validation to multiple operations:

1. BATCH_TOGGLE_HANDLES - now skips locked/protected blocks at:
   - Store layer (batchToggleHandles)
   - Collaborative hook (collaborativeBatchToggleBlockHandles)
   - Server socket handler

2. BATCH_ADD_BLOCKS - server now filters blocks being added to
   locked parent containers

3. BATCH_UPDATE_PARENT - server now:
   - Skips protected blocks (locked or inside locked container)
   - Prevents moving blocks into locked containers

All validations use consistent isProtected() helper that checks both
direct lock and parent container lock.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(workflow): use pre-computed lock state from contextMenuBlocks

contextMenuBlocks already has locked and isParentLocked properties
computed in use-canvas-context-menu.ts, so there's no need to look
up blocks again via hasProtectedBlocks.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(lock): add lock validation to block rename operations

Defense-in-depth: although the UI disables rename for locked blocks,
the collaborative layer and server now also validate locks.

- collaborativeUpdateBlockName: checks if block is locked or inside
  locked container before attempting rename
- UPDATE_NAME server handler: checks lock status and parent lock
  before performing database update

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* added defense in depth for renaming locked blocks

* fix(socket): add server-side lock validation for edges and subblocks

Defense-in-depth: adds lock checks to server-side handlers that were
previously relying only on client-side validation.

Edge operations (ADD, REMOVE, BATCH_ADD, BATCH_REMOVE):
- Check if source or target blocks are protected before modifying edges

Subblock updates:
- Check if parent block is protected before updating subblock values

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(lock): fetch parent blocks for edge protection checks and consistent tooltip

- Fixed edge operations to fetch parent blocks before checking lock status
  - Previously, isBlockProtected checked if parent was locked, but the parent
    wasn't in blocksById because only source/target blocks were fetched
  - Now fetches parent blocks for all four edge operations: ADD, REMOVE,
    BATCH_ADD_EDGES, BATCH_REMOVE_EDGES
- Fixed tooltip inconsistency: changed "Run previous blocks first" to
  "Run upstream blocks first" in action-bar to match workflow.tsx

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* updated tooltip text for run from block

* fix(lock): add lock check to duplicate button and clean up drag handler

- Added lock check to duplicate button in action bar to prevent
  duplicating locked blocks (consistent with other edit operations)
- Removed ineffective early return in onNodeDragStart since the
  `draggable` property on nodes already prevents dragging protected
  blocks - the early return was misleading as it couldn't actually
  stop a drag operation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(lock): use disableEdit for duplicate in block menu

Changed duplicate menu item to use disableEdit (which includes lock
check) instead of !userCanEdit for consistency with action bar and
other edit operations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 19:15:04 -08:00
Waleed
4ba22527b6 improvement(tag-dropdown): removed custom styling on tag dropdown popover, fixed execution ordering in terminal and loops entries (#3126)
* improvement(tag-dropdown): removeed custom styling on tag dropdown popover, fixed execution ordering in terminal and loops entries

* ack pr comments

* handle old records
2026-02-03 18:32:40 -08:00
Waleed
c51f266ad7 fix(logs): use formatDuration utility and align file cards styling (#3125) 2026-02-03 15:26:09 -08:00
Waleed
4ca00810b2 fix(http): serialize nested objects in form-urlencoded body (#3124) 2026-02-03 10:58:01 -08:00
Waleed
710bf75bca fix(sidebar): right-click replaces selection, reset popover hover state (#3123)
* fix(sidebar): right-click replaces selection, reset popover hover state

* fix(queries): add userId to superuser query key for cache isolation
2026-02-03 10:09:03 -08:00
Waleed
f21fe2309c fix(formatting): consolidate duration formatting into shared utility (#3118)
* fix(formatting): consolidate duration formatting into shared utility

* fix(formatting): preserve original precision and rounding behavior

* fix(logs): add precision to logs list duration formatting

* fix(formatting): use parseFloat to preserve fractional milliseconds

* feat(ee): add enterprise modules (#3121)

* fix(formatting): return null for missing values, strip trailing zeros
2026-02-02 23:57:08 -08:00
Waleed
9c3fd1f7af feat(ee): add enterprise modules (#3121) 2026-02-02 23:40:18 -08:00
468 changed files with 43225 additions and 7393 deletions

View File

@@ -183,6 +183,109 @@ export const {ServiceName}Block: BlockConfig = {
} }
``` ```
## File Input Handling
When your block accepts file uploads, use the basic/advanced mode pattern with `normalizeFileInput`.
### Basic/Advanced File Pattern
```typescript
// Basic mode: Visual file upload
{
id: 'uploadFile',
title: 'File',
type: 'file-upload',
canonicalParamId: 'file', // Both map to 'file' param
placeholder: 'Upload file',
mode: 'basic',
multiple: false,
required: true,
condition: { field: 'operation', value: 'upload' },
},
// Advanced mode: Reference from other blocks
{
id: 'fileRef',
title: 'File',
type: 'short-input',
canonicalParamId: 'file', // Both map to 'file' param
placeholder: 'Reference file (e.g., {{file_block.output}})',
mode: 'advanced',
required: true,
condition: { field: 'operation', value: 'upload' },
},
```
**Critical constraints:**
- `canonicalParamId` must NOT match any subblock's `id` in the same block
- Values are stored under subblock `id`, not `canonicalParamId`
### Normalizing File Input in tools.config
Use `normalizeFileInput` to handle all input variants:
```typescript
import { normalizeFileInput } from '@/blocks/utils'
tools: {
access: ['service_upload'],
config: {
tool: (params) => {
// Check all field IDs: uploadFile (basic), fileRef (advanced), fileContent (legacy)
const normalizedFile = normalizeFileInput(
params.uploadFile || params.fileRef || params.fileContent,
{ single: true }
)
if (normalizedFile) {
params.file = normalizedFile
}
return `service_${params.operation}`
},
},
}
```
**Why this pattern?**
- Values come through as `params.uploadFile` or `params.fileRef` (the subblock IDs)
- `canonicalParamId` only controls UI/schema mapping, not runtime values
- `normalizeFileInput` handles JSON strings from advanced mode template resolution
### File Input Types in `inputs`
Use `type: 'json'` for file inputs:
```typescript
inputs: {
uploadFile: { type: 'json', description: 'Uploaded file (UserFile)' },
fileRef: { type: 'json', description: 'File reference from previous block' },
// Legacy field for backwards compatibility
fileContent: { type: 'string', description: 'Legacy: base64 encoded content' },
}
```
### Multiple Files
For multiple file uploads:
```typescript
{
id: 'attachments',
title: 'Attachments',
type: 'file-upload',
multiple: true, // Allow multiple files
maxSize: 25, // Max size in MB per file
acceptedTypes: 'image/*,application/pdf,.doc,.docx',
}
// In tools.config:
const normalizedFiles = normalizeFileInput(
params.attachments || params.attachmentRefs,
// No { single: true } - returns array
)
if (normalizedFiles) {
params.files = normalizedFiles
}
```
## Condition Syntax ## Condition Syntax
Controls when a field is shown based on other field values. Controls when a field is shown based on other field values.

View File

@@ -457,7 +457,230 @@ You can usually find this in the service's brand/press kit page, or copy it from
Paste the SVG code here and I'll convert it to a React component. Paste the SVG code here and I'll convert it to a React component.
``` ```
## Common Gotchas ## File Handling
When your integration handles file uploads or downloads, follow these patterns to work with `UserFile` objects consistently.
### What is a UserFile?
A `UserFile` is the standard file representation in Sim:
```typescript
interface UserFile {
id: string // Unique identifier
name: string // Original filename
url: string // Presigned URL for download
size: number // File size in bytes
type: string // MIME type (e.g., 'application/pdf')
base64?: string // Optional base64 content (if small file)
key?: string // Internal storage key
context?: object // Storage context metadata
}
```
### File Input Pattern (Uploads)
For tools that accept file uploads, **always route through an internal API endpoint** rather than calling external APIs directly. This ensures proper file content retrieval.
#### 1. Block SubBlocks for File Input
Use the basic/advanced mode pattern:
```typescript
// Basic mode: File upload UI
{
id: 'uploadFile',
title: 'File',
type: 'file-upload',
canonicalParamId: 'file', // Maps to 'file' param
placeholder: 'Upload file',
mode: 'basic',
multiple: false,
required: true,
condition: { field: 'operation', value: 'upload' },
},
// Advanced mode: Reference from previous block
{
id: 'fileRef',
title: 'File',
type: 'short-input',
canonicalParamId: 'file', // Same canonical param
placeholder: 'Reference file (e.g., {{file_block.output}})',
mode: 'advanced',
required: true,
condition: { field: 'operation', value: 'upload' },
},
```
**Critical:** `canonicalParamId` must NOT match any subblock `id`.
#### 2. Normalize File Input in Block Config
In `tools.config.tool`, use `normalizeFileInput` to handle all input variants:
```typescript
import { normalizeFileInput } from '@/blocks/utils'
tools: {
config: {
tool: (params) => {
// Normalize file from basic (uploadFile), advanced (fileRef), or legacy (fileContent)
const normalizedFile = normalizeFileInput(
params.uploadFile || params.fileRef || params.fileContent,
{ single: true }
)
if (normalizedFile) {
params.file = normalizedFile
}
return `{service}_${params.operation}`
},
},
}
```
#### 3. Create Internal API Route
Create `apps/sim/app/api/tools/{service}/{action}/route.ts`:
```typescript
import { createLogger } from '@sim/logger'
import { NextResponse, type NextRequest } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { FileInputSchema, type RawFileInput } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
const logger = createLogger('{Service}UploadAPI')
const RequestSchema = z.object({
accessToken: z.string(),
file: FileInputSchema.optional().nullable(),
// Legacy field for backwards compatibility
fileContent: z.string().optional().nullable(),
// ... other params
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
return NextResponse.json({ success: false, error: 'Unauthorized' }, { status: 401 })
}
const body = await request.json()
const data = RequestSchema.parse(body)
let fileBuffer: Buffer
let fileName: string
// Prefer UserFile input, fall back to legacy base64
if (data.file) {
const userFiles = processFilesToUserFiles([data.file as RawFileInput], requestId, logger)
if (userFiles.length === 0) {
return NextResponse.json({ success: false, error: 'Invalid file' }, { status: 400 })
}
const userFile = userFiles[0]
fileBuffer = await downloadFileFromStorage(userFile, requestId, logger)
fileName = userFile.name
} else if (data.fileContent) {
// Legacy: base64 string (backwards compatibility)
fileBuffer = Buffer.from(data.fileContent, 'base64')
fileName = 'file'
} else {
return NextResponse.json({ success: false, error: 'File required' }, { status: 400 })
}
// Now call external API with fileBuffer
const response = await fetch('https://api.{service}.com/upload', {
method: 'POST',
headers: { Authorization: `Bearer ${data.accessToken}` },
body: new Uint8Array(fileBuffer), // Convert Buffer for fetch
})
// ... handle response
}
```
#### 4. Update Tool to Use Internal Route
```typescript
export const {service}UploadTool: ToolConfig<Params, Response> = {
id: '{service}_upload',
// ...
params: {
file: { type: 'file', required: false, visibility: 'user-or-llm' },
fileContent: { type: 'string', required: false, visibility: 'hidden' }, // Legacy
},
request: {
url: '/api/tools/{service}/upload', // Internal route
method: 'POST',
body: (params) => ({
accessToken: params.accessToken,
file: params.file,
fileContent: params.fileContent,
}),
},
}
```
### File Output Pattern (Downloads)
For tools that return files, use `FileToolProcessor` to store files and return `UserFile` objects.
#### In Tool transformResponse
```typescript
import { FileToolProcessor } from '@/executor/utils/file-tool-processor'
transformResponse: async (response, context) => {
const data = await response.json()
// Process file outputs to UserFile objects
const fileProcessor = new FileToolProcessor(context)
const file = await fileProcessor.processFileData({
data: data.content, // base64 or buffer
mimeType: data.mimeType,
filename: data.filename,
})
return {
success: true,
output: { file },
}
}
```
#### In API Route (for complex file handling)
```typescript
// Return file data that FileToolProcessor can handle
return NextResponse.json({
success: true,
output: {
file: {
data: base64Content,
mimeType: 'application/pdf',
filename: 'document.pdf',
},
},
})
```
### Key Helpers Reference
| Helper | Location | Purpose |
|--------|----------|---------|
| `normalizeFileInput` | `@/blocks/utils` | Normalize file params in block config |
| `processFilesToUserFiles` | `@/lib/uploads/utils/file-utils` | Convert raw inputs to UserFile[] |
| `downloadFileFromStorage` | `@/lib/uploads/utils/file-utils.server` | Get file Buffer from UserFile |
| `FileToolProcessor` | `@/executor/utils/file-tool-processor` | Process tool output files |
| `isUserFile` | `@/lib/core/utils/user-file` | Type guard for UserFile objects |
| `FileInputSchema` | `@/lib/uploads/utils/file-schemas` | Zod schema for file validation |
### Common Gotchas
1. **OAuth serviceId must match** - The `serviceId` in oauth-input must match the OAuth provider configuration 1. **OAuth serviceId must match** - The `serviceId` in oauth-input must match the OAuth provider configuration
2. **Tool IDs are snake_case** - `stripe_create_payment`, not `stripeCreatePayment` 2. **Tool IDs are snake_case** - `stripe_create_payment`, not `stripeCreatePayment`
@@ -465,3 +688,5 @@ Paste the SVG code here and I'll convert it to a React component.
4. **Alphabetical ordering** - Keep imports and registry entries alphabetically sorted 4. **Alphabetical ordering** - Keep imports and registry entries alphabetically sorted
5. **Required can be conditional** - Use `required: { field: 'op', value: 'create' }` instead of always true 5. **Required can be conditional** - Use `required: { field: 'op', value: 'create' }` instead of always true
6. **DependsOn clears options** - When a dependency changes, selector options are refetched 6. **DependsOn clears options** - When a dependency changes, selector options are refetched
7. **Never pass Buffer directly to fetch** - Convert to `new Uint8Array(buffer)` for TypeScript compatibility
8. **Always handle legacy file params** - Keep hidden `fileContent` params for backwards compatibility

View File

@@ -195,6 +195,52 @@ import { {service}WebhookTrigger } from '@/triggers/{service}'
{service}_webhook: {service}WebhookTrigger, {service}_webhook: {service}WebhookTrigger,
``` ```
## File Handling
When integrations handle file uploads/downloads, use `UserFile` objects consistently.
### File Input (Uploads)
1. **Block subBlocks:** Use basic/advanced mode pattern with `canonicalParamId`
2. **Normalize in block config:** Use `normalizeFileInput` from `@/blocks/utils`
3. **Internal API route:** Create route that uses `downloadFileFromStorage` to get file content
4. **Tool routes to internal API:** Don't call external APIs directly with files
```typescript
// In block tools.config:
import { normalizeFileInput } from '@/blocks/utils'
const normalizedFile = normalizeFileInput(
params.uploadFile || params.fileRef || params.fileContent,
{ single: true }
)
if (normalizedFile) params.file = normalizedFile
```
### File Output (Downloads)
Use `FileToolProcessor` in tool `transformResponse` to store files:
```typescript
import { FileToolProcessor } from '@/executor/utils/file-tool-processor'
const processor = new FileToolProcessor(context)
const file = await processor.processFileData({
data: base64Content,
mimeType: 'application/pdf',
filename: 'doc.pdf',
})
```
### Key Helpers
| Helper | Location | Purpose |
|--------|----------|---------|
| `normalizeFileInput` | `@/blocks/utils` | Normalize file params in block config |
| `processFilesToUserFiles` | `@/lib/uploads/utils/file-utils` | Convert raw inputs to UserFile[] |
| `downloadFileFromStorage` | `@/lib/uploads/utils/file-utils.server` | Get Buffer from UserFile |
| `FileToolProcessor` | `@/executor/utils/file-tool-processor` | Process tool output files |
## Checklist ## Checklist
- [ ] Look up API docs for the service - [ ] Look up API docs for the service
@@ -207,3 +253,5 @@ import { {service}WebhookTrigger } from '@/triggers/{service}'
- [ ] Register block in `blocks/registry.ts` - [ ] Register block in `blocks/registry.ts`
- [ ] (Optional) Create triggers in `triggers/{service}/` - [ ] (Optional) Create triggers in `triggers/{service}/`
- [ ] (Optional) Register triggers in `triggers/registry.ts` - [ ] (Optional) Register triggers in `triggers/registry.ts`
- [ ] (If file uploads) Create internal API route with `downloadFileFromStorage`
- [ ] (If file uploads) Use `normalizeFileInput` in block config

View File

@@ -193,6 +193,52 @@ import { {service}WebhookTrigger } from '@/triggers/{service}'
{service}_webhook: {service}WebhookTrigger, {service}_webhook: {service}WebhookTrigger,
``` ```
## File Handling
When integrations handle file uploads/downloads, use `UserFile` objects consistently.
### File Input (Uploads)
1. **Block subBlocks:** Use basic/advanced mode pattern with `canonicalParamId`
2. **Normalize in block config:** Use `normalizeFileInput` from `@/blocks/utils`
3. **Internal API route:** Create route that uses `downloadFileFromStorage` to get file content
4. **Tool routes to internal API:** Don't call external APIs directly with files
```typescript
// In block tools.config:
import { normalizeFileInput } from '@/blocks/utils'
const normalizedFile = normalizeFileInput(
params.uploadFile || params.fileRef || params.fileContent,
{ single: true }
)
if (normalizedFile) params.file = normalizedFile
```
### File Output (Downloads)
Use `FileToolProcessor` in tool `transformResponse` to store files:
```typescript
import { FileToolProcessor } from '@/executor/utils/file-tool-processor'
const processor = new FileToolProcessor(context)
const file = await processor.processFileData({
data: base64Content,
mimeType: 'application/pdf',
filename: 'doc.pdf',
})
```
### Key Helpers
| Helper | Location | Purpose |
|--------|----------|---------|
| `normalizeFileInput` | `@/blocks/utils` | Normalize file params in block config |
| `processFilesToUserFiles` | `@/lib/uploads/utils/file-utils` | Convert raw inputs to UserFile[] |
| `downloadFileFromStorage` | `@/lib/uploads/utils/file-utils.server` | Get Buffer from UserFile |
| `FileToolProcessor` | `@/executor/utils/file-tool-processor` | Process tool output files |
## Checklist ## Checklist
- [ ] Look up API docs for the service - [ ] Look up API docs for the service
@@ -205,3 +251,5 @@ import { {service}WebhookTrigger } from '@/triggers/{service}'
- [ ] Register block in `blocks/registry.ts` - [ ] Register block in `blocks/registry.ts`
- [ ] (Optional) Create triggers in `triggers/{service}/` - [ ] (Optional) Create triggers in `triggers/{service}/`
- [ ] (Optional) Register triggers in `triggers/registry.ts` - [ ] (Optional) Register triggers in `triggers/registry.ts`
- [ ] (If file uploads) Create internal API route with `downloadFileFromStorage`
- [ ] (If file uploads) Use `normalizeFileInput` in block config

View File

@@ -265,6 +265,23 @@ Register in `blocks/registry.ts` (alphabetically).
**dependsOn:** `['field']` or `{ all: ['a'], any: ['b', 'c'] }` **dependsOn:** `['field']` or `{ all: ['a'], any: ['b', 'c'] }`
**File Input Pattern (basic/advanced mode):**
```typescript
// Basic: file-upload UI
{ id: 'uploadFile', type: 'file-upload', canonicalParamId: 'file', mode: 'basic' },
// Advanced: reference from other blocks
{ id: 'fileRef', type: 'short-input', canonicalParamId: 'file', mode: 'advanced' },
```
In `tools.config.tool`, normalize with:
```typescript
import { normalizeFileInput } from '@/blocks/utils'
const file = normalizeFileInput(params.uploadFile || params.fileRef, { single: true })
if (file) params.file = file
```
For file uploads, create an internal API route (`/api/tools/{service}/upload`) that uses `downloadFileFromStorage` to get file content from `UserFile` objects.
### 3. Icon (`components/icons.tsx`) ### 3. Icon (`components/icons.tsx`)
```typescript ```typescript
@@ -293,3 +310,5 @@ Register in `triggers/registry.ts`.
- [ ] Create block in `blocks/blocks/{service}.ts` - [ ] Create block in `blocks/blocks/{service}.ts`
- [ ] Register block in `blocks/registry.ts` - [ ] Register block in `blocks/registry.ts`
- [ ] (Optional) Create and register triggers - [ ] (Optional) Create and register triggers
- [ ] (If file uploads) Create internal API route with `downloadFileFromStorage`
- [ ] (If file uploads) Use `normalizeFileInput` in block config

View File

@@ -213,6 +213,25 @@ Different subscription plans have different usage limits:
| **Team** | $40/seat (pooled, adjustable) | 300 sync, 2,500 async | | **Team** | $40/seat (pooled, adjustable) | 300 sync, 2,500 async |
| **Enterprise** | Custom | Custom | | **Enterprise** | Custom | Custom |
## Execution Time Limits
Workflows have maximum execution time limits based on your subscription plan:
| Plan | Sync Execution | Async Execution |
|------|----------------|-----------------|
| **Free** | 5 minutes | 10 minutes |
| **Pro** | 50 minutes | 90 minutes |
| **Team** | 50 minutes | 90 minutes |
| **Enterprise** | 50 minutes | 90 minutes |
**Sync executions** run immediately and return results directly. These are triggered via the API with `async: false` (default) or through the UI.
**Async executions** (triggered via API with `async: true`, webhooks, or schedules) run in the background. Async time limits are up to 2x the sync limit, capped at 90 minutes.
<Callout type="info">
If a workflow exceeds its time limit, it will be terminated and marked as failed with a timeout error. Design long-running workflows to use async execution or break them into smaller workflows.
</Callout>
## Billing Model ## Billing Model
Sim uses a **base subscription + overage** billing model: Sim uses a **base subscription + overage** billing model:

View File

@@ -0,0 +1,168 @@
---
title: Passing Files
---
import { Callout } from 'fumadocs-ui/components/callout'
import { Tab, Tabs } from 'fumadocs-ui/components/tabs'
Sim makes it easy to work with files throughout your workflows. Blocks can receive files, process them, and pass them to other blocks seamlessly.
## File Objects
When blocks output files (like Gmail attachments, generated images, or parsed documents), they return a standardized file object:
```json
{
"name": "report.pdf",
"url": "https://...",
"base64": "JVBERi0xLjQK...",
"type": "application/pdf",
"size": 245678
}
```
You can access any of these properties when referencing files from previous blocks.
## The File Block
The **File block** is the universal entry point for files in your workflows. It accepts files from any source and outputs standardized file objects that work with all integrations.
**Inputs:**
- **Uploaded files** - Drag and drop or select files directly
- **External URLs** - Any publicly accessible file URL
- **Files from other blocks** - Pass files from Gmail attachments, Slack downloads, etc.
**Outputs:**
- A list of `UserFile` objects with consistent structure (`name`, `url`, `base64`, `type`, `size`)
- `combinedContent` - Extracted text content from all files (for documents)
**Example usage:**
```
// Get all files from the File block
<file.files>
// Get the first file
<file.files[0]>
// Get combined text content from parsed documents
<file.combinedContent>
```
The File block automatically:
- Detects file types from URLs and extensions
- Extracts text from PDFs, CSVs, and documents
- Generates base64 encoding for binary files
- Creates presigned URLs for secure access
Use the File block when you need to normalize files from different sources before passing them to other blocks like Vision, STT, or email integrations.
## Passing Files Between Blocks
Reference files from previous blocks using the tag dropdown. Click in any file input field and type `<` to see available outputs.
**Common patterns:**
```
// Single file from a block
<gmail.attachments[0]>
// Pass the whole file object
<file_parser.files[0]>
// Access specific properties
<gmail.attachments[0].name>
<gmail.attachments[0].base64>
```
Most blocks accept the full file object and extract what they need automatically. You don't need to manually extract `base64` or `url` in most cases.
## Triggering Workflows with Files
When calling a workflow via API that expects file input, include files in your request:
<Tabs items={['Base64', 'URL']}>
<Tab value="Base64">
```bash
curl -X POST "https://sim.ai/api/workflows/YOUR_WORKFLOW_ID/execute" \
-H "Content-Type: application/json" \
-H "x-api-key: YOUR_API_KEY" \
-d '{
"document": {
"name": "report.pdf",
"base64": "JVBERi0xLjQK...",
"type": "application/pdf"
}
}'
```
</Tab>
<Tab value="URL">
```bash
curl -X POST "https://sim.ai/api/workflows/YOUR_WORKFLOW_ID/execute" \
-H "Content-Type: application/json" \
-H "x-api-key: YOUR_API_KEY" \
-d '{
"document": {
"name": "report.pdf",
"url": "https://example.com/report.pdf",
"type": "application/pdf"
}
}'
```
</Tab>
</Tabs>
The workflow's Start block should have an input field configured to receive the file parameter.
## Receiving Files in API Responses
When a workflow outputs files, they're included in the response:
```json
{
"success": true,
"output": {
"generatedFile": {
"name": "output.png",
"url": "https://...",
"base64": "iVBORw0KGgo...",
"type": "image/png",
"size": 34567
}
}
}
```
Use `url` for direct downloads or `base64` for inline processing.
## Blocks That Work with Files
**File inputs:**
- **File** - Parse documents, images, and text files
- **Vision** - Analyze images with AI models
- **Mistral Parser** - Extract text from PDFs
**File outputs:**
- **Gmail** - Email attachments
- **Slack** - Downloaded files
- **TTS** - Generated audio files
- **Video Generator** - Generated videos
- **Image Generator** - Generated images
**File storage:**
- **Supabase** - Upload/download from storage
- **S3** - AWS S3 operations
- **Google Drive** - Drive file operations
- **Dropbox** - Dropbox file operations
<Callout type="info">
Files are automatically available to downstream blocks. The execution engine handles all file transfer and format conversion.
</Callout>
## Best Practices
1. **Use file objects directly** - Pass the full file object rather than extracting individual properties. Blocks handle the conversion automatically.
2. **Check file types** - Ensure the file type matches what the receiving block expects. The Vision block needs images, the File block handles documents.
3. **Consider file size** - Large files increase execution time. For very large files, consider using storage blocks (S3, Supabase) for intermediate storage.

View File

@@ -1,3 +1,3 @@
{ {
"pages": ["index", "basics", "api", "logging", "costs"] "pages": ["index", "basics", "files", "api", "logging", "costs"]
} }

View File

@@ -180,6 +180,11 @@ A quick lookup for everyday actions in the Sim workflow editor. For keyboard sho
<td>Right-click → **Enable/Disable**</td> <td>Right-click → **Enable/Disable**</td>
<td><ActionImage src="/static/quick-reference/disable-block.png" alt="Disable block" /></td> <td><ActionImage src="/static/quick-reference/disable-block.png" alt="Disable block" /></td>
</tr> </tr>
<tr>
<td>Lock/Unlock a block</td>
<td>Hover block → Click lock icon (Admin only)</td>
<td><ActionImage src="/static/quick-reference/lock-block.png" alt="Lock block" /></td>
</tr>
<tr> <tr>
<td>Toggle handle orientation</td> <td>Toggle handle orientation</td>
<td>Right-click → **Toggle Handles**</td> <td>Right-click → **Toggle Handles**</td>

View File

@@ -11,7 +11,7 @@ import { BlockInfoCard } from "@/components/ui/block-info-card"
/> />
{/* MANUAL-CONTENT-START:intro */} {/* MANUAL-CONTENT-START:intro */}
The [Pulse](https://www.pulseapi.com/) tool enables seamless extraction of text and structured content from a wide variety of documents—including PDFs, images, and Office files—using state-of-the-art OCR (Optical Character Recognition) powered by Pulse. Designed for automated agentic workflows, Pulse Parser makes it easy to unlock valuable information trapped in unstructured documents and integrate the extracted content directly into your workflow. The [Pulse](https://www.runpulse.com) tool enables seamless extraction of text and structured content from a wide variety of documents—including PDFs, images, and Office files—using state-of-the-art OCR (Optical Character Recognition) powered by Pulse. Designed for automated agentic workflows, Pulse Parser makes it easy to unlock valuable information trapped in unstructured documents and integrate the extracted content directly into your workflow.
With Pulse, you can: With Pulse, you can:

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

View File

@@ -1,6 +1,6 @@
import { redirect } from 'next/navigation' import { redirect } from 'next/navigation'
import { getEnv, isTruthy } from '@/lib/core/config/env' import { getEnv, isTruthy } from '@/lib/core/config/env'
import SSOForm from '@/app/(auth)/sso/sso-form' import SSOForm from '@/ee/sso/components/sso-form'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'

View File

@@ -11,7 +11,7 @@ import {
Database, Database,
DollarSign, DollarSign,
HardDrive, HardDrive,
Workflow, Timer,
} from 'lucide-react' } from 'lucide-react'
import { useRouter } from 'next/navigation' import { useRouter } from 'next/navigation'
import { cn } from '@/lib/core/utils/cn' import { cn } from '@/lib/core/utils/cn'
@@ -44,7 +44,7 @@ interface PricingTier {
const FREE_PLAN_FEATURES: PricingFeature[] = [ const FREE_PLAN_FEATURES: PricingFeature[] = [
{ icon: DollarSign, text: '$20 usage limit' }, { icon: DollarSign, text: '$20 usage limit' },
{ icon: HardDrive, text: '5GB file storage' }, { icon: HardDrive, text: '5GB file storage' },
{ icon: Workflow, text: 'Public template access' }, { icon: Timer, text: '5 min execution limit' },
{ icon: Database, text: 'Limited log retention' }, { icon: Database, text: 'Limited log retention' },
{ icon: Code2, text: 'CLI/SDK Access' }, { icon: Code2, text: 'CLI/SDK Access' },
] ]

View File

@@ -16,7 +16,7 @@ import {
import { checkHybridAuth } from '@/lib/auth/hybrid' import { checkHybridAuth } from '@/lib/auth/hybrid'
import { getBrandConfig } from '@/lib/branding/branding' import { getBrandConfig } from '@/lib/branding/branding'
import { acquireLock, getRedisClient, releaseLock } from '@/lib/core/config/redis' import { acquireLock, getRedisClient, releaseLock } from '@/lib/core/config/redis'
import { validateExternalUrl } from '@/lib/core/security/input-validation' import { validateUrlWithDNS } from '@/lib/core/security/input-validation.server'
import { SSE_HEADERS } from '@/lib/core/utils/sse' import { SSE_HEADERS } from '@/lib/core/utils/sse'
import { getBaseUrl } from '@/lib/core/utils/urls' import { getBaseUrl } from '@/lib/core/utils/urls'
import { markExecutionCancelled } from '@/lib/execution/cancellation' import { markExecutionCancelled } from '@/lib/execution/cancellation'
@@ -1119,7 +1119,7 @@ async function handlePushNotificationSet(
) )
} }
const urlValidation = validateExternalUrl( const urlValidation = await validateUrlWithDNS(
params.pushNotificationConfig.url, params.pushNotificationConfig.url,
'Push notification URL' 'Push notification URL'
) )

View File

@@ -7,8 +7,14 @@ import { z } from 'zod'
import { getSession } from '@/lib/auth' import { getSession } from '@/lib/auth'
import { generateChatTitle } from '@/lib/copilot/chat-title' import { generateChatTitle } from '@/lib/copilot/chat-title'
import { getCopilotModel } from '@/lib/copilot/config' import { getCopilotModel } from '@/lib/copilot/config'
import { SIM_AGENT_API_URL_DEFAULT, SIM_AGENT_VERSION } from '@/lib/copilot/constants' import { SIM_AGENT_VERSION } from '@/lib/copilot/constants'
import { COPILOT_MODEL_IDS, COPILOT_REQUEST_MODES } from '@/lib/copilot/models' import { COPILOT_MODEL_IDS, COPILOT_REQUEST_MODES } from '@/lib/copilot/models'
import { orchestrateCopilotStream } from '@/lib/copilot/orchestrator'
import {
createStreamEventWriter,
resetStreamBuffer,
setStreamMeta,
} from '@/lib/copilot/orchestrator/stream-buffer'
import { import {
authenticateCopilotRequestSessionOnly, authenticateCopilotRequestSessionOnly,
createBadRequestResponse, createBadRequestResponse,
@@ -21,13 +27,12 @@ import type { CopilotProviderConfig } from '@/lib/copilot/types'
import { env } from '@/lib/core/config/env' import { env } from '@/lib/core/config/env'
import { CopilotFiles } from '@/lib/uploads' import { CopilotFiles } from '@/lib/uploads'
import { createFileContent } from '@/lib/uploads/utils/file-utils' import { createFileContent } from '@/lib/uploads/utils/file-utils'
import { resolveWorkflowIdForUser } from '@/lib/workflows/utils'
import { tools } from '@/tools/registry' import { tools } from '@/tools/registry'
import { getLatestVersionTools, stripVersionSuffix } from '@/tools/utils' import { getLatestVersionTools, stripVersionSuffix } from '@/tools/utils'
const logger = createLogger('CopilotChatAPI') const logger = createLogger('CopilotChatAPI')
const SIM_AGENT_API_URL = env.SIM_AGENT_API_URL || SIM_AGENT_API_URL_DEFAULT
const FileAttachmentSchema = z.object({ const FileAttachmentSchema = z.object({
id: z.string(), id: z.string(),
key: z.string(), key: z.string(),
@@ -40,7 +45,8 @@ const ChatMessageSchema = z.object({
message: z.string().min(1, 'Message is required'), message: z.string().min(1, 'Message is required'),
userMessageId: z.string().optional(), // ID from frontend for the user message userMessageId: z.string().optional(), // ID from frontend for the user message
chatId: z.string().optional(), chatId: z.string().optional(),
workflowId: z.string().min(1, 'Workflow ID is required'), workflowId: z.string().optional(),
workflowName: z.string().optional(),
model: z.enum(COPILOT_MODEL_IDS).optional().default('claude-4.5-opus'), model: z.enum(COPILOT_MODEL_IDS).optional().default('claude-4.5-opus'),
mode: z.enum(COPILOT_REQUEST_MODES).optional().default('agent'), mode: z.enum(COPILOT_REQUEST_MODES).optional().default('agent'),
prefetch: z.boolean().optional(), prefetch: z.boolean().optional(),
@@ -100,7 +106,8 @@ export async function POST(req: NextRequest) {
message, message,
userMessageId, userMessageId,
chatId, chatId,
workflowId, workflowId: providedWorkflowId,
workflowName,
model, model,
mode, mode,
prefetch, prefetch,
@@ -113,6 +120,20 @@ export async function POST(req: NextRequest) {
contexts, contexts,
commands, commands,
} = ChatMessageSchema.parse(body) } = ChatMessageSchema.parse(body)
// Resolve workflowId - if not provided, use first workflow or find by name
const resolved = await resolveWorkflowIdForUser(
authenticatedUserId,
providedWorkflowId,
workflowName
)
if (!resolved) {
return createBadRequestResponse(
'No workflows found. Create a workflow first or provide a valid workflowId.'
)
}
const workflowId = resolved.workflowId
// Ensure we have a consistent user message ID for this request // Ensure we have a consistent user message ID for this request
const userMessageIdToUse = userMessageId || crypto.randomUUID() const userMessageIdToUse = userMessageId || crypto.randomUUID()
try { try {
@@ -465,77 +486,53 @@ export async function POST(req: NextRequest) {
}) })
} catch {} } catch {}
const simAgentResponse = await fetch(`${SIM_AGENT_API_URL}/api/chat-completion-streaming`, { if (stream) {
method: 'POST', const streamId = userMessageIdToUse
headers: { let eventWriter: ReturnType<typeof createStreamEventWriter> | null = null
'Content-Type': 'application/json', let clientDisconnected = false
...(env.COPILOT_API_KEY ? { 'x-api-key': env.COPILOT_API_KEY } : {}),
},
body: JSON.stringify(requestPayload),
})
if (!simAgentResponse.ok) {
if (simAgentResponse.status === 401 || simAgentResponse.status === 402) {
// Rethrow status only; client will render appropriate assistant message
return new NextResponse(null, { status: simAgentResponse.status })
}
const errorText = await simAgentResponse.text().catch(() => '')
logger.error(`[${tracker.requestId}] Sim agent API error:`, {
status: simAgentResponse.status,
error: errorText,
})
return NextResponse.json(
{ error: `Sim agent API error: ${simAgentResponse.statusText}` },
{ status: simAgentResponse.status }
)
}
// If streaming is requested, forward the stream and update chat later
if (stream && simAgentResponse.body) {
// Create user message to save
const userMessage = {
id: userMessageIdToUse, // Consistent ID used for request and persistence
role: 'user',
content: message,
timestamp: new Date().toISOString(),
...(fileAttachments && fileAttachments.length > 0 && { fileAttachments }),
...(Array.isArray(contexts) && contexts.length > 0 && { contexts }),
...(Array.isArray(contexts) &&
contexts.length > 0 && {
contentBlocks: [{ type: 'contexts', contexts: contexts as any, timestamp: Date.now() }],
}),
}
// Create a pass-through stream that captures the response
const transformedStream = new ReadableStream({ const transformedStream = new ReadableStream({
async start(controller) { async start(controller) {
const encoder = new TextEncoder() const encoder = new TextEncoder()
let assistantContent = ''
const toolCalls: any[] = []
let buffer = ''
const isFirstDone = true
let responseIdFromStart: string | undefined
let responseIdFromDone: string | undefined
// Track tool call progress to identify a safe done event
const announcedToolCallIds = new Set<string>()
const startedToolExecutionIds = new Set<string>()
const completedToolExecutionIds = new Set<string>()
let lastDoneResponseId: string | undefined
let lastSafeDoneResponseId: string | undefined
// Send chatId as first event await resetStreamBuffer(streamId)
if (actualChatId) { await setStreamMeta(streamId, { status: 'active', userId: authenticatedUserId })
const chatIdEvent = `data: ${JSON.stringify({ eventWriter = createStreamEventWriter(streamId)
type: 'chat_id',
chatId: actualChatId, const shouldFlushEvent = (event: Record<string, any>) =>
})}\n\n` event.type === 'tool_call' ||
controller.enqueue(encoder.encode(chatIdEvent)) event.type === 'tool_result' ||
logger.debug(`[${tracker.requestId}] Sent initial chatId event to client`) event.type === 'tool_error' ||
event.type === 'subagent_end' ||
event.type === 'structured_result' ||
event.type === 'subagent_result' ||
event.type === 'done' ||
event.type === 'error'
const pushEvent = async (event: Record<string, any>) => {
if (!eventWriter) return
const entry = await eventWriter.write(event)
if (shouldFlushEvent(event)) {
await eventWriter.flush()
}
const payload = {
...event,
eventId: entry.eventId,
streamId,
}
try {
if (!clientDisconnected) {
controller.enqueue(encoder.encode(`data: ${JSON.stringify(payload)}\n\n`))
}
} catch {
clientDisconnected = true
await eventWriter.flush()
}
}
if (actualChatId) {
await pushEvent({ type: 'chat_id', chatId: actualChatId })
} }
// Start title generation in parallel if needed
if (actualChatId && !currentChat?.title && conversationHistory.length === 0) { if (actualChatId && !currentChat?.title && conversationHistory.length === 0) {
generateChatTitle(message) generateChatTitle(message)
.then(async (title) => { .then(async (title) => {
@@ -547,311 +544,64 @@ export async function POST(req: NextRequest) {
updatedAt: new Date(), updatedAt: new Date(),
}) })
.where(eq(copilotChats.id, actualChatId!)) .where(eq(copilotChats.id, actualChatId!))
await pushEvent({ type: 'title_updated', title })
const titleEvent = `data: ${JSON.stringify({
type: 'title_updated',
title: title,
})}\n\n`
controller.enqueue(encoder.encode(titleEvent))
logger.info(`[${tracker.requestId}] Generated and saved title: ${title}`)
} }
}) })
.catch((error) => { .catch((error) => {
logger.error(`[${tracker.requestId}] Title generation failed:`, error) logger.error(`[${tracker.requestId}] Title generation failed:`, error)
}) })
} else {
logger.debug(`[${tracker.requestId}] Skipping title generation`)
} }
// Forward the sim agent stream and capture assistant response
const reader = simAgentResponse.body!.getReader()
const decoder = new TextDecoder()
try { try {
while (true) { const result = await orchestrateCopilotStream(requestPayload, {
const { done, value } = await reader.read() userId: authenticatedUserId,
if (done) { workflowId,
break chatId: actualChatId,
} autoExecuteTools: true,
interactive: true,
// Decode and parse SSE events for logging and capturing content onEvent: async (event) => {
const decodedChunk = decoder.decode(value, { stream: true }) await pushEvent(event)
buffer += decodedChunk },
const lines = buffer.split('\n')
buffer = lines.pop() || '' // Keep incomplete line in buffer
for (const line of lines) {
if (line.trim() === '') continue // Skip empty lines
if (line.startsWith('data: ') && line.length > 6) {
try {
const jsonStr = line.slice(6)
// Check if the JSON string is unusually large (potential streaming issue)
if (jsonStr.length > 50000) {
// 50KB limit
logger.warn(`[${tracker.requestId}] Large SSE event detected`, {
size: jsonStr.length,
preview: `${jsonStr.substring(0, 100)}...`,
})
}
const event = JSON.parse(jsonStr)
// Log different event types comprehensively
switch (event.type) {
case 'content':
if (event.data) {
assistantContent += event.data
}
break
case 'reasoning':
logger.debug(
`[${tracker.requestId}] Reasoning chunk received (${(event.data || event.content || '').length} chars)`
)
break
case 'tool_call':
if (!event.data?.partial) {
toolCalls.push(event.data)
if (event.data?.id) {
announcedToolCallIds.add(event.data.id)
}
}
break
case 'tool_generating':
if (event.toolCallId) {
startedToolExecutionIds.add(event.toolCallId)
}
break
case 'tool_result':
if (event.toolCallId) {
completedToolExecutionIds.add(event.toolCallId)
}
break
case 'tool_error':
logger.error(`[${tracker.requestId}] Tool error:`, {
toolCallId: event.toolCallId,
toolName: event.toolName,
error: event.error,
success: event.success,
})
if (event.toolCallId) {
completedToolExecutionIds.add(event.toolCallId)
}
break
case 'start':
if (event.data?.responseId) {
responseIdFromStart = event.data.responseId
}
break
case 'done':
if (event.data?.responseId) {
responseIdFromDone = event.data.responseId
lastDoneResponseId = responseIdFromDone
// Mark this done as safe only if no tool call is currently in progress or pending
const announced = announcedToolCallIds.size
const completed = completedToolExecutionIds.size
const started = startedToolExecutionIds.size
const hasToolInProgress = announced > completed || started > completed
if (!hasToolInProgress) {
lastSafeDoneResponseId = responseIdFromDone
}
}
break
case 'error':
break
default:
}
// Emit to client: rewrite 'error' events into user-friendly assistant message
if (event?.type === 'error') {
try {
const displayMessage: string =
(event?.data && (event.data.displayMessage as string)) ||
'Sorry, I encountered an error. Please try again.'
const formatted = `_${displayMessage}_`
// Accumulate so it persists to DB as assistant content
assistantContent += formatted
// Send as content chunk
try {
controller.enqueue(
encoder.encode(
`data: ${JSON.stringify({ type: 'content', data: formatted })}\n\n`
)
)
} catch (enqueueErr) {
reader.cancel()
break
}
// Then close this response cleanly for the client
try {
controller.enqueue(
encoder.encode(`data: ${JSON.stringify({ type: 'done' })}\n\n`)
)
} catch (enqueueErr) {
reader.cancel()
break
}
} catch {}
// Do not forward the original error event
} else {
// Forward original event to client
try {
controller.enqueue(encoder.encode(`data: ${jsonStr}\n\n`))
} catch (enqueueErr) {
reader.cancel()
break
}
}
} catch (e) {
// Enhanced error handling for large payloads and parsing issues
const lineLength = line.length
const isLargePayload = lineLength > 10000
if (isLargePayload) {
logger.error(
`[${tracker.requestId}] Failed to parse large SSE event (${lineLength} chars)`,
{
error: e,
preview: `${line.substring(0, 200)}...`,
size: lineLength,
}
)
} else {
logger.warn(
`[${tracker.requestId}] Failed to parse SSE event: "${line.substring(0, 200)}..."`,
e
)
}
}
} else if (line.trim() && line !== 'data: [DONE]') {
logger.debug(`[${tracker.requestId}] Non-SSE line from sim agent: "${line}"`)
}
}
}
// Process any remaining buffer
if (buffer.trim()) {
logger.debug(`[${tracker.requestId}] Processing remaining buffer: "${buffer}"`)
if (buffer.startsWith('data: ')) {
try {
const jsonStr = buffer.slice(6)
const event = JSON.parse(jsonStr)
if (event.type === 'content' && event.data) {
assistantContent += event.data
}
// Forward remaining event, applying same error rewrite behavior
if (event?.type === 'error') {
const displayMessage: string =
(event?.data && (event.data.displayMessage as string)) ||
'Sorry, I encountered an error. Please try again.'
const formatted = `_${displayMessage}_`
assistantContent += formatted
try {
controller.enqueue(
encoder.encode(
`data: ${JSON.stringify({ type: 'content', data: formatted })}\n\n`
)
)
controller.enqueue(
encoder.encode(`data: ${JSON.stringify({ type: 'done' })}\n\n`)
)
} catch (enqueueErr) {
reader.cancel()
}
} else {
try {
controller.enqueue(encoder.encode(`data: ${jsonStr}\n\n`))
} catch (enqueueErr) {
reader.cancel()
}
}
} catch (e) {
logger.warn(`[${tracker.requestId}] Failed to parse final buffer: "${buffer}"`)
}
}
}
// Log final streaming summary
logger.info(`[${tracker.requestId}] Streaming complete summary:`, {
totalContentLength: assistantContent.length,
toolCallsCount: toolCalls.length,
hasContent: assistantContent.length > 0,
toolNames: toolCalls.map((tc) => tc?.name).filter(Boolean),
}) })
// NOTE: Messages are saved by the client via update-messages endpoint with full contentBlocks. if (currentChat && result.conversationId) {
// Server only updates conversationId here to avoid overwriting client's richer save. await db
if (currentChat) { .update(copilotChats)
// Persist only a safe conversationId to avoid continuing from a state that expects tool outputs .set({
const previousConversationId = currentChat?.conversationId as string | undefined updatedAt: new Date(),
const responseId = lastSafeDoneResponseId || previousConversationId || undefined conversationId: result.conversationId,
})
if (responseId) { .where(eq(copilotChats.id, actualChatId!))
await db
.update(copilotChats)
.set({
updatedAt: new Date(),
conversationId: responseId,
})
.where(eq(copilotChats.id, actualChatId!))
logger.info(
`[${tracker.requestId}] Updated conversationId for chat ${actualChatId}`,
{
updatedConversationId: responseId,
}
)
}
} }
await eventWriter.close()
await setStreamMeta(streamId, { status: 'complete', userId: authenticatedUserId })
} catch (error) { } catch (error) {
logger.error(`[${tracker.requestId}] Error processing stream:`, error) logger.error(`[${tracker.requestId}] Orchestration error:`, error)
await eventWriter.close()
// Send an error event to the client before closing so it knows what happened await setStreamMeta(streamId, {
try { status: 'error',
const errorMessage = userId: authenticatedUserId,
error instanceof Error && error.message === 'terminated' error: error instanceof Error ? error.message : 'Stream error',
? 'Connection to AI service was interrupted. Please try again.' })
: 'An unexpected error occurred while processing the response.' await pushEvent({
const encoder = new TextEncoder() type: 'error',
data: {
// Send error as content so it shows in the chat displayMessage: 'An unexpected error occurred while processing the response.',
controller.enqueue( },
encoder.encode( })
`data: ${JSON.stringify({ type: 'content', data: `\n\n_${errorMessage}_` })}\n\n`
)
)
// Send done event to properly close the stream on client
controller.enqueue(encoder.encode(`data: ${JSON.stringify({ type: 'done' })}\n\n`))
} catch (enqueueError) {
// Stream might already be closed, that's ok
logger.warn(
`[${tracker.requestId}] Could not send error event to client:`,
enqueueError
)
}
} finally { } finally {
try { controller.close()
controller.close() }
} catch { },
// Controller might already be closed async cancel() {
} clientDisconnected = true
if (eventWriter) {
await eventWriter.flush()
} }
}, },
}) })
const response = new Response(transformedStream, { return new Response(transformedStream, {
headers: { headers: {
'Content-Type': 'text/event-stream', 'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache', 'Cache-Control': 'no-cache',
@@ -859,43 +609,31 @@ export async function POST(req: NextRequest) {
'X-Accel-Buffering': 'no', 'X-Accel-Buffering': 'no',
}, },
}) })
logger.info(`[${tracker.requestId}] Returning streaming response to client`, {
duration: tracker.getDuration(),
chatId: actualChatId,
headers: {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'keep-alive',
},
})
return response
} }
// For non-streaming responses const nonStreamingResult = await orchestrateCopilotStream(requestPayload, {
const responseData = await simAgentResponse.json() userId: authenticatedUserId,
logger.info(`[${tracker.requestId}] Non-streaming response from sim agent:`, { workflowId,
chatId: actualChatId,
autoExecuteTools: true,
interactive: true,
})
const responseData = {
content: nonStreamingResult.content,
toolCalls: nonStreamingResult.toolCalls,
model: selectedModel,
provider: providerConfig?.provider || env.COPILOT_PROVIDER || 'openai',
}
logger.info(`[${tracker.requestId}] Non-streaming response from orchestrator:`, {
hasContent: !!responseData.content, hasContent: !!responseData.content,
contentLength: responseData.content?.length || 0, contentLength: responseData.content?.length || 0,
model: responseData.model, model: responseData.model,
provider: responseData.provider, provider: responseData.provider,
toolCallsCount: responseData.toolCalls?.length || 0, toolCallsCount: responseData.toolCalls?.length || 0,
hasTokens: !!responseData.tokens,
}) })
// Log tool calls if present
if (responseData.toolCalls?.length > 0) {
responseData.toolCalls.forEach((toolCall: any) => {
logger.info(`[${tracker.requestId}] Tool call in response:`, {
id: toolCall.id,
name: toolCall.name,
success: toolCall.success,
result: `${JSON.stringify(toolCall.result).substring(0, 200)}...`,
})
})
}
// Save messages if we have a chat // Save messages if we have a chat
if (currentChat && responseData.content) { if (currentChat && responseData.content) {
const userMessage = { const userMessage = {
@@ -947,6 +685,9 @@ export async function POST(req: NextRequest) {
.set({ .set({
messages: updatedMessages, messages: updatedMessages,
updatedAt: new Date(), updatedAt: new Date(),
...(nonStreamingResult.conversationId
? { conversationId: nonStreamingResult.conversationId }
: {}),
}) })
.where(eq(copilotChats.id, actualChatId!)) .where(eq(copilotChats.id, actualChatId!))
} }
@@ -998,10 +739,7 @@ export async function GET(req: NextRequest) {
try { try {
const { searchParams } = new URL(req.url) const { searchParams } = new URL(req.url)
const workflowId = searchParams.get('workflowId') const workflowId = searchParams.get('workflowId')
const chatId = searchParams.get('chatId')
if (!workflowId) {
return createBadRequestResponse('workflowId is required')
}
// Get authenticated user using consolidated helper // Get authenticated user using consolidated helper
const { userId: authenticatedUserId, isAuthenticated } = const { userId: authenticatedUserId, isAuthenticated } =
@@ -1010,6 +748,47 @@ export async function GET(req: NextRequest) {
return createUnauthorizedResponse() return createUnauthorizedResponse()
} }
// If chatId is provided, fetch a single chat
if (chatId) {
const [chat] = await db
.select({
id: copilotChats.id,
title: copilotChats.title,
model: copilotChats.model,
messages: copilotChats.messages,
planArtifact: copilotChats.planArtifact,
config: copilotChats.config,
createdAt: copilotChats.createdAt,
updatedAt: copilotChats.updatedAt,
})
.from(copilotChats)
.where(and(eq(copilotChats.id, chatId), eq(copilotChats.userId, authenticatedUserId)))
.limit(1)
if (!chat) {
return NextResponse.json({ success: false, error: 'Chat not found' }, { status: 404 })
}
const transformedChat = {
id: chat.id,
title: chat.title,
model: chat.model,
messages: Array.isArray(chat.messages) ? chat.messages : [],
messageCount: Array.isArray(chat.messages) ? chat.messages.length : 0,
planArtifact: chat.planArtifact || null,
config: chat.config || null,
createdAt: chat.createdAt,
updatedAt: chat.updatedAt,
}
logger.info(`Retrieved chat ${chatId}`)
return NextResponse.json({ success: true, chat: transformedChat })
}
if (!workflowId) {
return createBadRequestResponse('workflowId or chatId is required')
}
// Fetch chats for this user and workflow // Fetch chats for this user and workflow
const chats = await db const chats = await db
.select({ .select({

View File

@@ -0,0 +1,130 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import {
getStreamMeta,
readStreamEvents,
type StreamMeta,
} from '@/lib/copilot/orchestrator/stream-buffer'
import { authenticateCopilotRequestSessionOnly } from '@/lib/copilot/request-helpers'
import { SSE_HEADERS } from '@/lib/core/utils/sse'
const logger = createLogger('CopilotChatStreamAPI')
const POLL_INTERVAL_MS = 250
const MAX_STREAM_MS = 10 * 60 * 1000
function encodeEvent(event: Record<string, any>): Uint8Array {
return new TextEncoder().encode(`data: ${JSON.stringify(event)}\n\n`)
}
export async function GET(request: NextRequest) {
const { userId: authenticatedUserId, isAuthenticated } =
await authenticateCopilotRequestSessionOnly()
if (!isAuthenticated || !authenticatedUserId) {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const url = new URL(request.url)
const streamId = url.searchParams.get('streamId') || ''
const fromParam = url.searchParams.get('from') || '0'
const fromEventId = Number(fromParam || 0)
// If batch=true, return buffered events as JSON instead of SSE
const batchMode = url.searchParams.get('batch') === 'true'
const toParam = url.searchParams.get('to')
const toEventId = toParam ? Number(toParam) : undefined
if (!streamId) {
return NextResponse.json({ error: 'streamId is required' }, { status: 400 })
}
const meta = (await getStreamMeta(streamId)) as StreamMeta | null
logger.info('[Resume] Stream lookup', {
streamId,
fromEventId,
toEventId,
batchMode,
hasMeta: !!meta,
metaStatus: meta?.status,
})
if (!meta) {
return NextResponse.json({ error: 'Stream not found' }, { status: 404 })
}
if (meta.userId && meta.userId !== authenticatedUserId) {
return NextResponse.json({ error: 'Unauthorized' }, { status: 403 })
}
// Batch mode: return all buffered events as JSON
if (batchMode) {
const events = await readStreamEvents(streamId, fromEventId)
const filteredEvents = toEventId ? events.filter((e) => e.eventId <= toEventId) : events
logger.info('[Resume] Batch response', {
streamId,
fromEventId,
toEventId,
eventCount: filteredEvents.length,
})
return NextResponse.json({
success: true,
events: filteredEvents,
status: meta.status,
})
}
const startTime = Date.now()
const stream = new ReadableStream({
async start(controller) {
let lastEventId = Number.isFinite(fromEventId) ? fromEventId : 0
const flushEvents = async () => {
const events = await readStreamEvents(streamId, lastEventId)
if (events.length > 0) {
logger.info('[Resume] Flushing events', {
streamId,
fromEventId: lastEventId,
eventCount: events.length,
})
}
for (const entry of events) {
lastEventId = entry.eventId
const payload = {
...entry.event,
eventId: entry.eventId,
streamId: entry.streamId,
}
controller.enqueue(encodeEvent(payload))
}
}
try {
await flushEvents()
while (Date.now() - startTime < MAX_STREAM_MS) {
const currentMeta = await getStreamMeta(streamId)
if (!currentMeta) break
await flushEvents()
if (currentMeta.status === 'complete' || currentMeta.status === 'error') {
break
}
if (request.signal.aborted) {
break
}
await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL_MS))
}
} catch (error) {
logger.warn('Stream replay failed', {
streamId,
error: error instanceof Error ? error.message : String(error),
})
} finally {
controller.close()
}
},
})
return new Response(stream, { headers: SSE_HEADERS })
}

View File

@@ -21,6 +21,7 @@ const UpdateCreatorProfileSchema = z.object({
name: z.string().min(1, 'Name is required').max(100, 'Max 100 characters').optional(), name: z.string().min(1, 'Name is required').max(100, 'Max 100 characters').optional(),
profileImageUrl: z.string().optional().or(z.literal('')), profileImageUrl: z.string().optional().or(z.literal('')),
details: CreatorProfileDetailsSchema.optional(), details: CreatorProfileDetailsSchema.optional(),
verified: z.boolean().optional(), // Verification status (super users only)
}) })
// Helper to check if user has permission to manage profile // Helper to check if user has permission to manage profile
@@ -97,11 +98,29 @@ export async function PUT(request: NextRequest, { params }: { params: Promise<{
return NextResponse.json({ error: 'Profile not found' }, { status: 404 }) return NextResponse.json({ error: 'Profile not found' }, { status: 404 })
} }
// Check permissions // Verification changes require super user permission
const canEdit = await hasPermission(session.user.id, existing[0]) if (data.verified !== undefined) {
if (!canEdit) { const { verifyEffectiveSuperUser } = await import('@/lib/templates/permissions')
logger.warn(`[${requestId}] User denied permission to update profile: ${id}`) const { effectiveSuperUser } = await verifyEffectiveSuperUser(session.user.id)
return NextResponse.json({ error: 'Access denied' }, { status: 403 }) if (!effectiveSuperUser) {
logger.warn(`[${requestId}] Non-super user attempted to change creator verification: ${id}`)
return NextResponse.json(
{ error: 'Only super users can change verification status' },
{ status: 403 }
)
}
}
// For non-verified updates, check regular permissions
const hasNonVerifiedUpdates =
data.name !== undefined || data.profileImageUrl !== undefined || data.details !== undefined
if (hasNonVerifiedUpdates) {
const canEdit = await hasPermission(session.user.id, existing[0])
if (!canEdit) {
logger.warn(`[${requestId}] User denied permission to update profile: ${id}`)
return NextResponse.json({ error: 'Access denied' }, { status: 403 })
}
} }
const updateData: any = { const updateData: any = {
@@ -111,6 +130,7 @@ export async function PUT(request: NextRequest, { params }: { params: Promise<{
if (data.name !== undefined) updateData.name = data.name if (data.name !== undefined) updateData.name = data.name
if (data.profileImageUrl !== undefined) updateData.profileImageUrl = data.profileImageUrl if (data.profileImageUrl !== undefined) updateData.profileImageUrl = data.profileImageUrl
if (data.details !== undefined) updateData.details = data.details if (data.details !== undefined) updateData.details = data.details
if (data.verified !== undefined) updateData.verified = data.verified
const updated = await db const updated = await db
.update(templateCreators) .update(templateCreators)

View File

@@ -1,113 +0,0 @@
import { db } from '@sim/db'
import { templateCreators } from '@sim/db/schema'
import { createLogger } from '@sim/logger'
import { eq } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server'
import { getSession } from '@/lib/auth'
import { generateRequestId } from '@/lib/core/utils/request'
import { verifyEffectiveSuperUser } from '@/lib/templates/permissions'
const logger = createLogger('CreatorVerificationAPI')
export const revalidate = 0
// POST /api/creators/[id]/verify - Verify a creator (super users only)
export async function POST(request: NextRequest, { params }: { params: Promise<{ id: string }> }) {
const requestId = generateRequestId()
const { id } = await params
try {
const session = await getSession()
if (!session?.user?.id) {
logger.warn(`[${requestId}] Unauthorized verification attempt for creator: ${id}`)
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
// Check if user is a super user
const { effectiveSuperUser } = await verifyEffectiveSuperUser(session.user.id)
if (!effectiveSuperUser) {
logger.warn(`[${requestId}] Non-super user attempted to verify creator: ${id}`)
return NextResponse.json({ error: 'Only super users can verify creators' }, { status: 403 })
}
// Check if creator exists
const existingCreator = await db
.select()
.from(templateCreators)
.where(eq(templateCreators.id, id))
.limit(1)
if (existingCreator.length === 0) {
logger.warn(`[${requestId}] Creator not found for verification: ${id}`)
return NextResponse.json({ error: 'Creator not found' }, { status: 404 })
}
// Update creator verified status to true
await db
.update(templateCreators)
.set({ verified: true, updatedAt: new Date() })
.where(eq(templateCreators.id, id))
logger.info(`[${requestId}] Creator verified: ${id} by super user: ${session.user.id}`)
return NextResponse.json({
message: 'Creator verified successfully',
creatorId: id,
})
} catch (error) {
logger.error(`[${requestId}] Error verifying creator ${id}`, error)
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
}
}
// DELETE /api/creators/[id]/verify - Unverify a creator (super users only)
export async function DELETE(
request: NextRequest,
{ params }: { params: Promise<{ id: string }> }
) {
const requestId = generateRequestId()
const { id } = await params
try {
const session = await getSession()
if (!session?.user?.id) {
logger.warn(`[${requestId}] Unauthorized unverification attempt for creator: ${id}`)
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
// Check if user is a super user
const { effectiveSuperUser } = await verifyEffectiveSuperUser(session.user.id)
if (!effectiveSuperUser) {
logger.warn(`[${requestId}] Non-super user attempted to unverify creator: ${id}`)
return NextResponse.json({ error: 'Only super users can unverify creators' }, { status: 403 })
}
// Check if creator exists
const existingCreator = await db
.select()
.from(templateCreators)
.where(eq(templateCreators.id, id))
.limit(1)
if (existingCreator.length === 0) {
logger.warn(`[${requestId}] Creator not found for unverification: ${id}`)
return NextResponse.json({ error: 'Creator not found' }, { status: 404 })
}
// Update creator verified status to false
await db
.update(templateCreators)
.set({ verified: false, updatedAt: new Date() })
.where(eq(templateCreators.id, id))
logger.info(`[${requestId}] Creator unverified: ${id} by super user: ${session.user.id}`)
return NextResponse.json({
message: 'Creator unverified successfully',
creatorId: id,
})
} catch (error) {
logger.error(`[${requestId}] Error unverifying creator ${id}`, error)
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
}
}

View File

@@ -1,13 +1,16 @@
import { db } from '@sim/db' import { asyncJobs, db } from '@sim/db'
import { workflowExecutionLogs } from '@sim/db/schema' import { workflowExecutionLogs } from '@sim/db/schema'
import { createLogger } from '@sim/logger' import { createLogger } from '@sim/logger'
import { and, eq, lt, sql } from 'drizzle-orm' import { and, eq, inArray, lt, sql } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { verifyCronAuth } from '@/lib/auth/internal' import { verifyCronAuth } from '@/lib/auth/internal'
import { JOB_RETENTION_HOURS, JOB_STATUS } from '@/lib/core/async-jobs'
import { getMaxExecutionTimeout } from '@/lib/core/execution-limits'
const logger = createLogger('CleanupStaleExecutions') const logger = createLogger('CleanupStaleExecutions')
const STALE_THRESHOLD_MINUTES = 30 const STALE_THRESHOLD_MS = getMaxExecutionTimeout() + 5 * 60 * 1000
const STALE_THRESHOLD_MINUTES = Math.ceil(STALE_THRESHOLD_MS / 60000)
const MAX_INT32 = 2_147_483_647 const MAX_INT32 = 2_147_483_647
export async function GET(request: NextRequest) { export async function GET(request: NextRequest) {
@@ -78,12 +81,102 @@ export async function GET(request: NextRequest) {
logger.info(`Stale execution cleanup completed. Cleaned: ${cleaned}, Failed: ${failed}`) logger.info(`Stale execution cleanup completed. Cleaned: ${cleaned}, Failed: ${failed}`)
// Clean up stale async jobs (stuck in processing)
let asyncJobsMarkedFailed = 0
try {
const staleAsyncJobs = await db
.update(asyncJobs)
.set({
status: JOB_STATUS.FAILED,
completedAt: new Date(),
error: `Job terminated: stuck in processing for more than ${STALE_THRESHOLD_MINUTES} minutes`,
updatedAt: new Date(),
})
.where(
and(eq(asyncJobs.status, JOB_STATUS.PROCESSING), lt(asyncJobs.startedAt, staleThreshold))
)
.returning({ id: asyncJobs.id })
asyncJobsMarkedFailed = staleAsyncJobs.length
if (asyncJobsMarkedFailed > 0) {
logger.info(`Marked ${asyncJobsMarkedFailed} stale async jobs as failed`)
}
} catch (error) {
logger.error('Failed to clean up stale async jobs:', {
error: error instanceof Error ? error.message : String(error),
})
}
// Clean up stale pending jobs (never started, e.g., due to server crash before startJob())
let stalePendingJobsMarkedFailed = 0
try {
const stalePendingJobs = await db
.update(asyncJobs)
.set({
status: JOB_STATUS.FAILED,
completedAt: new Date(),
error: `Job terminated: stuck in pending state for more than ${STALE_THRESHOLD_MINUTES} minutes (never started)`,
updatedAt: new Date(),
})
.where(
and(eq(asyncJobs.status, JOB_STATUS.PENDING), lt(asyncJobs.createdAt, staleThreshold))
)
.returning({ id: asyncJobs.id })
stalePendingJobsMarkedFailed = stalePendingJobs.length
if (stalePendingJobsMarkedFailed > 0) {
logger.info(`Marked ${stalePendingJobsMarkedFailed} stale pending jobs as failed`)
}
} catch (error) {
logger.error('Failed to clean up stale pending jobs:', {
error: error instanceof Error ? error.message : String(error),
})
}
// Delete completed/failed jobs older than retention period
const retentionThreshold = new Date(Date.now() - JOB_RETENTION_HOURS * 60 * 60 * 1000)
let asyncJobsDeleted = 0
try {
const deletedJobs = await db
.delete(asyncJobs)
.where(
and(
inArray(asyncJobs.status, [JOB_STATUS.COMPLETED, JOB_STATUS.FAILED]),
lt(asyncJobs.completedAt, retentionThreshold)
)
)
.returning({ id: asyncJobs.id })
asyncJobsDeleted = deletedJobs.length
if (asyncJobsDeleted > 0) {
logger.info(
`Deleted ${asyncJobsDeleted} old async jobs (retention: ${JOB_RETENTION_HOURS}h)`
)
}
} catch (error) {
logger.error('Failed to delete old async jobs:', {
error: error instanceof Error ? error.message : String(error),
})
}
return NextResponse.json({ return NextResponse.json({
success: true, success: true,
found: staleExecutions.length, executions: {
cleaned, found: staleExecutions.length,
failed, cleaned,
thresholdMinutes: STALE_THRESHOLD_MINUTES, failed,
thresholdMinutes: STALE_THRESHOLD_MINUTES,
},
asyncJobs: {
staleProcessingMarkedFailed: asyncJobsMarkedFailed,
stalePendingMarkedFailed: stalePendingJobsMarkedFailed,
oldDeleted: asyncJobsDeleted,
staleThresholdMinutes: STALE_THRESHOLD_MINUTES,
retentionHours: JOB_RETENTION_HOURS,
},
}) })
} catch (error) { } catch (error) {
logger.error('Error in stale execution cleanup job:', error) logger.error('Error in stale execution cleanup job:', error)

View File

@@ -6,7 +6,11 @@ import { createLogger } from '@sim/logger'
import binaryExtensionsList from 'binary-extensions' import binaryExtensionsList from 'binary-extensions'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { checkHybridAuth } from '@/lib/auth/hybrid' import { checkHybridAuth } from '@/lib/auth/hybrid'
import { secureFetchWithPinnedIP, validateUrlWithDNS } from '@/lib/core/security/input-validation' import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { sanitizeUrlForLog } from '@/lib/core/utils/logging'
import { isSupportedFileType, parseFile } from '@/lib/file-parsers' import { isSupportedFileType, parseFile } from '@/lib/file-parsers'
import { isUsingCloudStorage, type StorageContext, StorageService } from '@/lib/uploads' import { isUsingCloudStorage, type StorageContext, StorageService } from '@/lib/uploads'
import { uploadExecutionFile } from '@/lib/uploads/contexts/execution' import { uploadExecutionFile } from '@/lib/uploads/contexts/execution'
@@ -19,6 +23,7 @@ import {
getMimeTypeFromExtension, getMimeTypeFromExtension,
getViewerUrl, getViewerUrl,
inferContextFromKey, inferContextFromKey,
isInternalFileUrl,
} from '@/lib/uploads/utils/file-utils' } from '@/lib/uploads/utils/file-utils'
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
import { verifyFileAccess } from '@/app/api/files/authorization' import { verifyFileAccess } from '@/app/api/files/authorization'
@@ -215,7 +220,7 @@ async function parseFileSingle(
} }
} }
if (filePath.includes('/api/files/serve/')) { if (isInternalFileUrl(filePath)) {
return handleCloudFile(filePath, fileType, undefined, userId, executionContext) return handleCloudFile(filePath, fileType, undefined, userId, executionContext)
} }
@@ -246,7 +251,7 @@ function validateFilePath(filePath: string): { isValid: boolean; error?: string
return { isValid: false, error: 'Invalid path: tilde character not allowed' } return { isValid: false, error: 'Invalid path: tilde character not allowed' }
} }
if (filePath.startsWith('/') && !filePath.startsWith('/api/files/serve/')) { if (filePath.startsWith('/') && !isInternalFileUrl(filePath)) {
return { isValid: false, error: 'Path outside allowed directory' } return { isValid: false, error: 'Path outside allowed directory' }
} }
@@ -420,7 +425,7 @@ async function handleExternalUrl(
return parseResult return parseResult
} catch (error) { } catch (error) {
logger.error(`Error handling external URL ${url}:`, error) logger.error(`Error handling external URL ${sanitizeUrlForLog(url)}:`, error)
return { return {
success: false, success: false,
error: `Error fetching URL: ${(error as Error).message}`, error: `Error fetching URL: ${(error as Error).message}`,

View File

@@ -1,7 +1,7 @@
import { createLogger } from '@sim/logger' import { createLogger } from '@sim/logger'
import { runs } from '@trigger.dev/sdk'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { checkHybridAuth } from '@/lib/auth/hybrid' import { checkHybridAuth } from '@/lib/auth/hybrid'
import { getJobQueue, JOB_STATUS } from '@/lib/core/async-jobs'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { createErrorResponse } from '@/app/api/workflows/utils' import { createErrorResponse } from '@/app/api/workflows/utils'
@@ -15,8 +15,6 @@ export async function GET(
const requestId = generateRequestId() const requestId = generateRequestId()
try { try {
logger.debug(`[${requestId}] Getting status for task: ${taskId}`)
const authResult = await checkHybridAuth(request, { requireWorkflowId: false }) const authResult = await checkHybridAuth(request, { requireWorkflowId: false })
if (!authResult.success || !authResult.userId) { if (!authResult.success || !authResult.userId) {
logger.warn(`[${requestId}] Unauthorized task status request`) logger.warn(`[${requestId}] Unauthorized task status request`)
@@ -25,76 +23,60 @@ export async function GET(
const authenticatedUserId = authResult.userId const authenticatedUserId = authResult.userId
const run = await runs.retrieve(taskId) const jobQueue = await getJobQueue()
const job = await jobQueue.getJob(taskId)
logger.debug(`[${requestId}] Task ${taskId} status: ${run.status}`) if (!job) {
return createErrorResponse('Task not found', 404)
const payload = run.payload as any
if (payload?.workflowId) {
const { verifyWorkflowAccess } = await import('@/socket/middleware/permissions')
const accessCheck = await verifyWorkflowAccess(authenticatedUserId, payload.workflowId)
if (!accessCheck.hasAccess) {
logger.warn(`[${requestId}] User ${authenticatedUserId} denied access to task ${taskId}`, {
workflowId: payload.workflowId,
})
return createErrorResponse('Access denied', 403)
}
logger.debug(`[${requestId}] User ${authenticatedUserId} has access to task ${taskId}`)
} else {
if (payload?.userId && payload.userId !== authenticatedUserId) {
logger.warn(
`[${requestId}] User ${authenticatedUserId} attempted to access task ${taskId} owned by ${payload.userId}`
)
return createErrorResponse('Access denied', 403)
}
if (!payload?.userId) {
logger.warn(
`[${requestId}] Task ${taskId} has no ownership information in payload. Denying access for security.`
)
return createErrorResponse('Access denied', 403)
}
} }
const statusMap = { if (job.metadata?.workflowId) {
QUEUED: 'queued', const { verifyWorkflowAccess } = await import('@/socket/middleware/permissions')
WAITING_FOR_DEPLOY: 'queued', const accessCheck = await verifyWorkflowAccess(
EXECUTING: 'processing', authenticatedUserId,
RESCHEDULED: 'processing', job.metadata.workflowId as string
FROZEN: 'processing', )
COMPLETED: 'completed', if (!accessCheck.hasAccess) {
CANCELED: 'cancelled', logger.warn(`[${requestId}] Access denied to workflow ${job.metadata.workflowId}`)
FAILED: 'failed', return createErrorResponse('Access denied', 403)
CRASHED: 'failed', }
INTERRUPTED: 'failed', } else if (job.metadata?.userId && job.metadata.userId !== authenticatedUserId) {
SYSTEM_FAILURE: 'failed', logger.warn(`[${requestId}] Access denied to user ${job.metadata.userId}`)
EXPIRED: 'failed', return createErrorResponse('Access denied', 403)
} as const } else if (!job.metadata?.userId && !job.metadata?.workflowId) {
logger.warn(`[${requestId}] Access denied to job ${taskId}`)
return createErrorResponse('Access denied', 403)
}
const mappedStatus = statusMap[run.status as keyof typeof statusMap] || 'unknown' const mappedStatus = job.status === JOB_STATUS.PENDING ? 'queued' : job.status
const response: any = { const response: any = {
success: true, success: true,
taskId, taskId,
status: mappedStatus, status: mappedStatus,
metadata: { metadata: {
startedAt: run.startedAt, startedAt: job.startedAt,
}, },
} }
if (mappedStatus === 'completed') { if (job.status === JOB_STATUS.COMPLETED) {
response.output = run.output // This contains the workflow execution results response.output = job.output
response.metadata.completedAt = run.finishedAt response.metadata.completedAt = job.completedAt
response.metadata.duration = run.durationMs if (job.startedAt && job.completedAt) {
response.metadata.duration = job.completedAt.getTime() - job.startedAt.getTime()
}
} }
if (mappedStatus === 'failed') { if (job.status === JOB_STATUS.FAILED) {
response.error = run.error response.error = job.error
response.metadata.completedAt = run.finishedAt response.metadata.completedAt = job.completedAt
response.metadata.duration = run.durationMs if (job.startedAt && job.completedAt) {
response.metadata.duration = job.completedAt.getTime() - job.startedAt.getTime()
}
} }
if (mappedStatus === 'processing' || mappedStatus === 'queued') { if (job.status === JOB_STATUS.PROCESSING || job.status === JOB_STATUS.PENDING) {
response.estimatedDuration = 180000 // 3 minutes max from our config response.estimatedDuration = 180000
} }
return NextResponse.json(response) return NextResponse.json(response)

View File

@@ -0,0 +1,413 @@
import {
type CallToolResult,
ErrorCode,
type InitializeResult,
isJSONRPCNotification,
isJSONRPCRequest,
type JSONRPCError,
type JSONRPCMessage,
type JSONRPCResponse,
type ListToolsResult,
type RequestId,
} from '@modelcontextprotocol/sdk/types.js'
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { checkHybridAuth } from '@/lib/auth/hybrid'
import { getCopilotModel } from '@/lib/copilot/config'
import { orchestrateSubagentStream } from '@/lib/copilot/orchestrator/subagent'
import {
executeToolServerSide,
prepareExecutionContext,
} from '@/lib/copilot/orchestrator/tool-executor'
import { DIRECT_TOOL_DEFS, SUBAGENT_TOOL_DEFS } from '@/lib/copilot/tools/mcp/definitions'
const logger = createLogger('CopilotMcpAPI')
export const dynamic = 'force-dynamic'
/**
* MCP Server instructions that guide LLMs on how to use the Sim copilot tools.
* This is included in the initialize response to help external LLMs understand
* the workflow lifecycle and best practices.
*/
const MCP_SERVER_INSTRUCTIONS = `
## Sim Workflow Copilot - Usage Guide
You are interacting with Sim's workflow automation platform. These tools orchestrate specialized AI agents that build workflows. Follow these guidelines carefully.
---
## Platform Knowledge
Sim is a workflow automation platform. Workflows are visual pipelines of blocks.
### Block Types
**Core Logic:**
- **Agent** - The heart of Sim (LLM block with tools, memory, structured output, knowledge bases)
- **Function** - JavaScript code execution
- **Condition** - If/else branching
- **Router** - AI-powered content-based routing
- **Loop** - While/do-while iteration
- **Parallel** - Simultaneous execution
- **API** - HTTP requests
**Integrations (3rd Party):**
- OAuth: Slack, Gmail, Google Calendar, Sheets, Outlook, Linear, GitHub, Notion
- API: Stripe, Twilio, SendGrid, any REST API
### The Agent Block
The Agent block is the core of intelligent workflows:
- **Tools** - Add integrations, custom tools, web search to give it capabilities
- **Memory** - Multi-turn conversations with persistent context
- **Structured Output** - JSON schema for reliable parsing
- **Knowledge Bases** - RAG-powered document retrieval
**Design principle:** Put tools INSIDE agents rather than using standalone tool blocks.
### Triggers
| Type | Description |
|------|-------------|
| Manual/Chat | User sends message in UI (start block: input, files, conversationId) |
| API | REST endpoint with custom input schema |
| Webhook | External services POST to trigger URL |
| Schedule | Cron-based (hourly, daily, weekly) |
### Deployments
| Type | Trigger | Use Case |
|------|---------|----------|
| API | Start block | REST endpoint for programmatic access |
| Chat | Start block | Managed chat UI with auth options |
| MCP | Start block | Expose as MCP tool for AI agents |
| General | Schedule/Webhook | Activate triggers to run automatically |
**Undeployed workflows only run in the builder UI.**
### Variable Syntax
Reference outputs from previous blocks: \`<blockname.field>\`
Reference environment variables: \`{{ENV_VAR_NAME}}\`
Rules:
- Block names must be lowercase, no spaces, no special characters
- Use dot notation for nested fields: \`<blockname.field.subfield>\`
---
## Workflow Lifecycle
1. **Create**: For NEW workflows, FIRST call create_workflow to get a workflowId
2. **Plan**: Use copilot_plan with the workflowId to plan the workflow
3. **Edit**: Use copilot_edit with the workflowId AND the plan to build the workflow
4. **Deploy**: ALWAYS deploy after building using copilot_deploy before testing/running
5. **Test**: Use copilot_test to verify the workflow works correctly
6. **Share**: Provide the user with the workflow URL after completion
---
## CRITICAL: Always Pass workflowId
- For NEW workflows: Call create_workflow FIRST, then use the returned workflowId
- For EXISTING workflows: Pass the workflowId to all copilot tools
- copilot_plan, copilot_edit, copilot_deploy, copilot_test, copilot_debug all REQUIRE workflowId
---
## CRITICAL: How to Handle Plans
The copilot_plan tool returns a structured plan object. You MUST:
1. **Do NOT modify the plan**: Pass the plan object EXACTLY as returned to copilot_edit
2. **Do NOT interpret or summarize the plan**: The edit agent needs the raw plan data
3. **Pass the plan in the context.plan field**: \`{ "context": { "plan": <plan_object> } }\`
4. **Include ALL plan data**: Block configurations, connections, credentials, everything
Example flow:
\`\`\`
1. copilot_plan({ request: "build a workflow...", workflowId: "abc123" })
-> Returns: { "plan": { "blocks": [...], "connections": [...], ... } }
2. copilot_edit({
workflowId: "abc123",
message: "Execute the plan",
context: { "plan": <EXACT plan object from step 1> }
})
\`\`\`
**Why this matters**: The plan contains technical details (block IDs, field mappings, API schemas) that the edit agent needs verbatim. Summarizing or rephrasing loses critical information.
---
## CRITICAL: Error Handling
**If the user says "doesn't work", "broke", "failed", "error" → ALWAYS use copilot_debug FIRST.**
Don't guess. Don't plan. Debug first to find the actual problem.
---
## Important Rules
- ALWAYS deploy a workflow before attempting to run or test it
- Workflows must be deployed to have an "active deployment" for execution
- After building, call copilot_deploy with the appropriate deployment type (api, chat, or mcp)
- Return the workflow URL to the user so they can access it in Sim
---
## Quick Operations (use direct tools)
- list_workflows, list_workspaces, list_folders, get_workflow: Fast database queries
- create_workflow: Create new workflow and get workflowId (CALL THIS FIRST for new workflows)
- create_folder: Create new resources
## Workflow Building (use copilot tools)
- copilot_plan: Plan workflow changes (REQUIRES workflowId) - returns a plan object
- copilot_edit: Execute the plan (REQUIRES workflowId AND plan from copilot_plan)
- copilot_deploy: Deploy workflows (REQUIRES workflowId)
- copilot_test: Test workflow execution (REQUIRES workflowId)
- copilot_debug: Diagnose errors (REQUIRES workflowId) - USE THIS FIRST for issues
`
function createResponse(id: RequestId, result: unknown): JSONRPCResponse {
return {
jsonrpc: '2.0',
id,
result: result as JSONRPCResponse['result'],
}
}
function createError(id: RequestId, code: ErrorCode | number, message: string): JSONRPCError {
return {
jsonrpc: '2.0',
id,
error: { code, message },
}
}
export async function GET() {
return NextResponse.json({
name: 'copilot-subagents',
version: '1.0.0',
protocolVersion: '2024-11-05',
capabilities: { tools: {} },
})
}
export async function POST(request: NextRequest) {
try {
const auth = await checkHybridAuth(request, { requireWorkflowId: false })
if (!auth.success || !auth.userId) {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const body = (await request.json()) as JSONRPCMessage
if (isJSONRPCNotification(body)) {
return new NextResponse(null, { status: 202 })
}
if (!isJSONRPCRequest(body)) {
return NextResponse.json(
createError(0, ErrorCode.InvalidRequest, 'Invalid JSON-RPC message'),
{ status: 400 }
)
}
const { id, method, params } = body
switch (method) {
case 'initialize': {
const result: InitializeResult = {
protocolVersion: '2024-11-05',
capabilities: { tools: {} },
serverInfo: { name: 'sim-copilot', version: '1.0.0' },
instructions: MCP_SERVER_INSTRUCTIONS,
}
return NextResponse.json(createResponse(id, result))
}
case 'ping':
return NextResponse.json(createResponse(id, {}))
case 'tools/list':
return handleToolsList(id)
case 'tools/call':
return handleToolsCall(
id,
params as { name: string; arguments?: Record<string, unknown> },
auth.userId
)
default:
return NextResponse.json(
createError(id, ErrorCode.MethodNotFound, `Method not found: ${method}`),
{ status: 404 }
)
}
} catch (error) {
logger.error('Error handling MCP request', { error })
return NextResponse.json(createError(0, ErrorCode.InternalError, 'Internal error'), {
status: 500,
})
}
}
async function handleToolsList(id: RequestId): Promise<NextResponse> {
const directTools = DIRECT_TOOL_DEFS.map((tool) => ({
name: tool.name,
description: tool.description,
inputSchema: tool.inputSchema,
}))
const subagentTools = SUBAGENT_TOOL_DEFS.map((tool) => ({
name: tool.name,
description: tool.description,
inputSchema: tool.inputSchema,
}))
const result: ListToolsResult = {
tools: [...directTools, ...subagentTools],
}
return NextResponse.json(createResponse(id, result))
}
async function handleToolsCall(
id: RequestId,
params: { name: string; arguments?: Record<string, unknown> },
userId: string
): Promise<NextResponse> {
const args = params.arguments || {}
// Check if this is a direct tool (fast, no LLM)
const directTool = DIRECT_TOOL_DEFS.find((tool) => tool.name === params.name)
if (directTool) {
return handleDirectToolCall(id, directTool, args, userId)
}
// Check if this is a subagent tool (uses LLM orchestration)
const subagentTool = SUBAGENT_TOOL_DEFS.find((tool) => tool.name === params.name)
if (subagentTool) {
return handleSubagentToolCall(id, subagentTool, args, userId)
}
return NextResponse.json(
createError(id, ErrorCode.MethodNotFound, `Tool not found: ${params.name}`),
{ status: 404 }
)
}
async function handleDirectToolCall(
id: RequestId,
toolDef: (typeof DIRECT_TOOL_DEFS)[number],
args: Record<string, unknown>,
userId: string
): Promise<NextResponse> {
try {
const execContext = await prepareExecutionContext(userId, (args.workflowId as string) || '')
const toolCall = {
id: crypto.randomUUID(),
name: toolDef.toolId,
status: 'pending' as const,
params: args as Record<string, any>,
startTime: Date.now(),
}
const result = await executeToolServerSide(toolCall, execContext)
const response: CallToolResult = {
content: [
{
type: 'text',
text: JSON.stringify(result.output ?? result, null, 2),
},
],
isError: !result.success,
}
return NextResponse.json(createResponse(id, response))
} catch (error) {
logger.error('Direct tool execution failed', { tool: toolDef.name, error })
return NextResponse.json(
createError(id, ErrorCode.InternalError, `Tool execution failed: ${error}`),
{ status: 500 }
)
}
}
async function handleSubagentToolCall(
id: RequestId,
toolDef: (typeof SUBAGENT_TOOL_DEFS)[number],
args: Record<string, unknown>,
userId: string
): Promise<NextResponse> {
const requestText =
(args.request as string) ||
(args.message as string) ||
(args.error as string) ||
JSON.stringify(args)
const context = (args.context as Record<string, unknown>) || {}
if (args.plan && !context.plan) {
context.plan = args.plan
}
const { model } = getCopilotModel('chat')
const result = await orchestrateSubagentStream(
toolDef.agentId,
{
message: requestText,
workflowId: args.workflowId,
workspaceId: args.workspaceId,
context,
model,
// Signal to the copilot backend that this is a headless request
// so it can enforce workflowId requirements on tools
headless: true,
},
{
userId,
workflowId: args.workflowId as string | undefined,
workspaceId: args.workspaceId as string | undefined,
}
)
// When a respond tool (plan_respond, edit_respond, etc.) was used,
// return only the structured result - not the full result with all internal tool calls.
// This provides clean output for MCP consumers.
let responseData: unknown
if (result.structuredResult) {
responseData = {
success: result.structuredResult.success ?? result.success,
type: result.structuredResult.type,
summary: result.structuredResult.summary,
data: result.structuredResult.data,
}
} else if (result.error) {
responseData = {
success: false,
error: result.error,
errors: result.errors,
}
} else {
// Fallback: return content if no structured result
responseData = {
success: result.success,
content: result.content,
}
}
const response: CallToolResult = {
content: [
{
type: 'text',
text: JSON.stringify(responseData, null, 2),
},
],
isError: !result.success,
}
return NextResponse.json(createResponse(id, response))
}

View File

@@ -21,6 +21,7 @@ import { and, eq } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { checkHybridAuth } from '@/lib/auth/hybrid' import { checkHybridAuth } from '@/lib/auth/hybrid'
import { generateInternalToken } from '@/lib/auth/internal' import { generateInternalToken } from '@/lib/auth/internal'
import { getMaxExecutionTimeout } from '@/lib/core/execution-limits'
import { getBaseUrl } from '@/lib/core/utils/urls' import { getBaseUrl } from '@/lib/core/utils/urls'
const logger = createLogger('WorkflowMcpServeAPI') const logger = createLogger('WorkflowMcpServeAPI')
@@ -264,7 +265,7 @@ async function handleToolsCall(
method: 'POST', method: 'POST',
headers, headers,
body: JSON.stringify({ input: params.arguments || {}, triggerType: 'mcp' }), body: JSON.stringify({ input: params.arguments || {}, triggerType: 'mcp' }),
signal: AbortSignal.timeout(600000), // 10 minute timeout signal: AbortSignal.timeout(getMaxExecutionTimeout()),
}) })
const executeResult = await response.json() const executeResult = await response.json()

View File

@@ -1,5 +1,8 @@
import { createLogger } from '@sim/logger' import { createLogger } from '@sim/logger'
import type { NextRequest } from 'next/server' import type { NextRequest } from 'next/server'
import { getHighestPrioritySubscription } from '@/lib/billing/core/plan'
import { getExecutionTimeout } from '@/lib/core/execution-limits'
import type { SubscriptionPlan } from '@/lib/core/rate-limiter/types'
import { getParsedBody, withMcpAuth } from '@/lib/mcp/middleware' import { getParsedBody, withMcpAuth } from '@/lib/mcp/middleware'
import { mcpService } from '@/lib/mcp/service' import { mcpService } from '@/lib/mcp/service'
import type { McpTool, McpToolCall, McpToolResult } from '@/lib/mcp/types' import type { McpTool, McpToolCall, McpToolResult } from '@/lib/mcp/types'
@@ -7,7 +10,6 @@ import {
categorizeError, categorizeError,
createMcpErrorResponse, createMcpErrorResponse,
createMcpSuccessResponse, createMcpSuccessResponse,
MCP_CONSTANTS,
validateStringParam, validateStringParam,
} from '@/lib/mcp/utils' } from '@/lib/mcp/utils'
@@ -171,13 +173,16 @@ export const POST = withMcpAuth('read')(
arguments: args, arguments: args,
} }
const userSubscription = await getHighestPrioritySubscription(userId)
const executionTimeout = getExecutionTimeout(
userSubscription?.plan as SubscriptionPlan | undefined,
'sync'
)
const result = await Promise.race([ const result = await Promise.race([
mcpService.executeTool(userId, serverId, toolCall, workspaceId), mcpService.executeTool(userId, serverId, toolCall, workspaceId),
new Promise<never>((_, reject) => new Promise<never>((_, reject) =>
setTimeout( setTimeout(() => reject(new Error('Tool execution timeout')), executionTimeout)
() => reject(new Error('Tool execution timeout')),
MCP_CONSTANTS.EXECUTION_TIMEOUT
)
), ),
]) ])

View File

@@ -29,7 +29,7 @@ import { hasWorkspaceAdminAccess } from '@/lib/workspaces/permissions/utils'
import { import {
InvitationsNotAllowedError, InvitationsNotAllowedError,
validateInvitationsAllowed, validateInvitationsAllowed,
} from '@/executor/utils/permission-check' } from '@/ee/access-control/utils/permission-check'
const logger = createLogger('OrganizationInvitations') const logger = createLogger('OrganizationInvitations')

View File

@@ -1,10 +1,9 @@
import { db, workflowDeploymentVersion, workflowSchedule } from '@sim/db' import { db, workflowDeploymentVersion, workflowSchedule } from '@sim/db'
import { createLogger } from '@sim/logger' import { createLogger } from '@sim/logger'
import { tasks } from '@trigger.dev/sdk'
import { and, eq, isNull, lt, lte, not, or, sql } from 'drizzle-orm' import { and, eq, isNull, lt, lte, not, or, sql } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { verifyCronAuth } from '@/lib/auth/internal' import { verifyCronAuth } from '@/lib/auth/internal'
import { isTriggerDevEnabled } from '@/lib/core/config/feature-flags' import { getJobQueue, shouldExecuteInline } from '@/lib/core/async-jobs'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { executeScheduleJob } from '@/background/schedule-execution' import { executeScheduleJob } from '@/background/schedule-execution'
@@ -55,72 +54,67 @@ export async function GET(request: NextRequest) {
logger.debug(`[${requestId}] Successfully queried schedules: ${dueSchedules.length} found`) logger.debug(`[${requestId}] Successfully queried schedules: ${dueSchedules.length} found`)
logger.info(`[${requestId}] Processing ${dueSchedules.length} due scheduled workflows`) logger.info(`[${requestId}] Processing ${dueSchedules.length} due scheduled workflows`)
if (isTriggerDevEnabled) { const jobQueue = await getJobQueue()
const triggerPromises = dueSchedules.map(async (schedule) => {
const queueTime = schedule.lastQueuedAt ?? queuedAt
try { const queuePromises = dueSchedules.map(async (schedule) => {
const payload = { const queueTime = schedule.lastQueuedAt ?? queuedAt
scheduleId: schedule.id,
workflowId: schedule.workflowId,
blockId: schedule.blockId || undefined,
cronExpression: schedule.cronExpression || undefined,
lastRanAt: schedule.lastRanAt?.toISOString(),
failedCount: schedule.failedCount || 0,
now: queueTime.toISOString(),
scheduledFor: schedule.nextRunAt?.toISOString(),
}
const handle = await tasks.trigger('schedule-execution', payload) const payload = {
logger.info( scheduleId: schedule.id,
`[${requestId}] Queued schedule execution task ${handle.id} for workflow ${schedule.workflowId}` workflowId: schedule.workflowId,
) blockId: schedule.blockId || undefined,
return handle cronExpression: schedule.cronExpression || undefined,
} catch (error) { lastRanAt: schedule.lastRanAt?.toISOString(),
logger.error( failedCount: schedule.failedCount || 0,
`[${requestId}] Failed to trigger schedule execution for workflow ${schedule.workflowId}`, now: queueTime.toISOString(),
error scheduledFor: schedule.nextRunAt?.toISOString(),
) }
return null
}
})
await Promise.allSettled(triggerPromises) try {
const jobId = await jobQueue.enqueue('schedule-execution', payload, {
logger.info(`[${requestId}] Queued ${dueSchedules.length} schedule executions to Trigger.dev`) metadata: { workflowId: schedule.workflowId },
} else {
const directExecutionPromises = dueSchedules.map(async (schedule) => {
const queueTime = schedule.lastQueuedAt ?? queuedAt
const payload = {
scheduleId: schedule.id,
workflowId: schedule.workflowId,
blockId: schedule.blockId || undefined,
cronExpression: schedule.cronExpression || undefined,
lastRanAt: schedule.lastRanAt?.toISOString(),
failedCount: schedule.failedCount || 0,
now: queueTime.toISOString(),
scheduledFor: schedule.nextRunAt?.toISOString(),
}
void executeScheduleJob(payload).catch((error) => {
logger.error(
`[${requestId}] Direct schedule execution failed for workflow ${schedule.workflowId}`,
error
)
}) })
logger.info( logger.info(
`[${requestId}] Queued direct schedule execution for workflow ${schedule.workflowId} (Trigger.dev disabled)` `[${requestId}] Queued schedule execution task ${jobId} for workflow ${schedule.workflowId}`
) )
})
await Promise.allSettled(directExecutionPromises) if (shouldExecuteInline()) {
void (async () => {
try {
await jobQueue.startJob(jobId)
const output = await executeScheduleJob(payload)
await jobQueue.completeJob(jobId, output)
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error)
logger.error(
`[${requestId}] Schedule execution failed for workflow ${schedule.workflowId}`,
{ jobId, error: errorMessage }
)
try {
await jobQueue.markJobFailed(jobId, errorMessage)
} catch (markFailedError) {
logger.error(`[${requestId}] Failed to mark job as failed`, {
jobId,
error:
markFailedError instanceof Error
? markFailedError.message
: String(markFailedError),
})
}
}
})()
}
} catch (error) {
logger.error(
`[${requestId}] Failed to queue schedule execution for workflow ${schedule.workflowId}`,
error
)
}
})
logger.info( await Promise.allSettled(queuePromises)
`[${requestId}] Queued ${dueSchedules.length} direct schedule executions (Trigger.dev disabled)`
) logger.info(`[${requestId}] Queued ${dueSchedules.length} schedule executions`)
}
return NextResponse.json({ return NextResponse.json({
message: 'Scheduled workflow executions processed', message: 'Scheduled workflow executions processed',

View File

@@ -1,101 +0,0 @@
import { db } from '@sim/db'
import { templates } from '@sim/db/schema'
import { createLogger } from '@sim/logger'
import { eq } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server'
import { getSession } from '@/lib/auth'
import { generateRequestId } from '@/lib/core/utils/request'
import { verifyEffectiveSuperUser } from '@/lib/templates/permissions'
const logger = createLogger('TemplateApprovalAPI')
export const revalidate = 0
/**
* POST /api/templates/[id]/approve - Approve a template (super users only)
*/
export async function POST(request: NextRequest, { params }: { params: Promise<{ id: string }> }) {
const requestId = generateRequestId()
const { id } = await params
try {
const session = await getSession()
if (!session?.user?.id) {
logger.warn(`[${requestId}] Unauthorized template approval attempt for ID: ${id}`)
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const { effectiveSuperUser } = await verifyEffectiveSuperUser(session.user.id)
if (!effectiveSuperUser) {
logger.warn(`[${requestId}] Non-super user attempted to approve template: ${id}`)
return NextResponse.json({ error: 'Only super users can approve templates' }, { status: 403 })
}
const existingTemplate = await db.select().from(templates).where(eq(templates.id, id)).limit(1)
if (existingTemplate.length === 0) {
logger.warn(`[${requestId}] Template not found for approval: ${id}`)
return NextResponse.json({ error: 'Template not found' }, { status: 404 })
}
await db
.update(templates)
.set({ status: 'approved', updatedAt: new Date() })
.where(eq(templates.id, id))
logger.info(`[${requestId}] Template approved: ${id} by super user: ${session.user.id}`)
return NextResponse.json({
message: 'Template approved successfully',
templateId: id,
})
} catch (error) {
logger.error(`[${requestId}] Error approving template ${id}`, error)
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
}
}
/**
* DELETE /api/templates/[id]/approve - Unapprove a template (super users only)
*/
export async function DELETE(
_request: NextRequest,
{ params }: { params: Promise<{ id: string }> }
) {
const requestId = generateRequestId()
const { id } = await params
try {
const session = await getSession()
if (!session?.user?.id) {
logger.warn(`[${requestId}] Unauthorized template rejection attempt for ID: ${id}`)
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const { effectiveSuperUser } = await verifyEffectiveSuperUser(session.user.id)
if (!effectiveSuperUser) {
logger.warn(`[${requestId}] Non-super user attempted to reject template: ${id}`)
return NextResponse.json({ error: 'Only super users can reject templates' }, { status: 403 })
}
const existingTemplate = await db.select().from(templates).where(eq(templates.id, id)).limit(1)
if (existingTemplate.length === 0) {
logger.warn(`[${requestId}] Template not found for rejection: ${id}`)
return NextResponse.json({ error: 'Template not found' }, { status: 404 })
}
await db
.update(templates)
.set({ status: 'rejected', updatedAt: new Date() })
.where(eq(templates.id, id))
logger.info(`[${requestId}] Template rejected: ${id} by super user: ${session.user.id}`)
return NextResponse.json({
message: 'Template rejected successfully',
templateId: id,
})
} catch (error) {
logger.error(`[${requestId}] Error rejecting template ${id}`, error)
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
}
}

View File

@@ -1,55 +0,0 @@
import { db } from '@sim/db'
import { templates } from '@sim/db/schema'
import { createLogger } from '@sim/logger'
import { eq } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server'
import { getSession } from '@/lib/auth'
import { generateRequestId } from '@/lib/core/utils/request'
import { verifyEffectiveSuperUser } from '@/lib/templates/permissions'
const logger = createLogger('TemplateRejectionAPI')
export const revalidate = 0
/**
* POST /api/templates/[id]/reject - Reject a template (super users only)
*/
export async function POST(request: NextRequest, { params }: { params: Promise<{ id: string }> }) {
const requestId = generateRequestId()
const { id } = await params
try {
const session = await getSession()
if (!session?.user?.id) {
logger.warn(`[${requestId}] Unauthorized template rejection attempt for ID: ${id}`)
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const { effectiveSuperUser } = await verifyEffectiveSuperUser(session.user.id)
if (!effectiveSuperUser) {
logger.warn(`[${requestId}] Non-super user attempted to reject template: ${id}`)
return NextResponse.json({ error: 'Only super users can reject templates' }, { status: 403 })
}
const existingTemplate = await db.select().from(templates).where(eq(templates.id, id)).limit(1)
if (existingTemplate.length === 0) {
logger.warn(`[${requestId}] Template not found for rejection: ${id}`)
return NextResponse.json({ error: 'Template not found' }, { status: 404 })
}
await db
.update(templates)
.set({ status: 'rejected', updatedAt: new Date() })
.where(eq(templates.id, id))
logger.info(`[${requestId}] Template rejected: ${id} by super user: ${session.user.id}`)
return NextResponse.json({
message: 'Template rejected successfully',
templateId: id,
})
} catch (error) {
logger.error(`[${requestId}] Error rejecting template ${id}`, error)
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
}
}

View File

@@ -106,6 +106,7 @@ const updateTemplateSchema = z.object({
creatorId: z.string().optional(), // Creator profile ID creatorId: z.string().optional(), // Creator profile ID
tags: z.array(z.string()).max(10, 'Maximum 10 tags allowed').optional(), tags: z.array(z.string()).max(10, 'Maximum 10 tags allowed').optional(),
updateState: z.boolean().optional(), // Explicitly request state update from current workflow updateState: z.boolean().optional(), // Explicitly request state update from current workflow
status: z.enum(['approved', 'rejected', 'pending']).optional(), // Status change (super users only)
}) })
// PUT /api/templates/[id] - Update a template // PUT /api/templates/[id] - Update a template
@@ -131,7 +132,7 @@ export async function PUT(request: NextRequest, { params }: { params: Promise<{
) )
} }
const { name, details, creatorId, tags, updateState } = validationResult.data const { name, details, creatorId, tags, updateState, status } = validationResult.data
const existingTemplate = await db.select().from(templates).where(eq(templates.id, id)).limit(1) const existingTemplate = await db.select().from(templates).where(eq(templates.id, id)).limit(1)
@@ -142,21 +143,44 @@ export async function PUT(request: NextRequest, { params }: { params: Promise<{
const template = existingTemplate[0] const template = existingTemplate[0]
if (!template.creatorId) { // Status changes require super user permission
logger.warn(`[${requestId}] Template ${id} has no creator, denying update`) if (status !== undefined) {
return NextResponse.json({ error: 'Access denied' }, { status: 403 }) const { verifyEffectiveSuperUser } = await import('@/lib/templates/permissions')
const { effectiveSuperUser } = await verifyEffectiveSuperUser(session.user.id)
if (!effectiveSuperUser) {
logger.warn(`[${requestId}] Non-super user attempted to change template status: ${id}`)
return NextResponse.json(
{ error: 'Only super users can change template status' },
{ status: 403 }
)
}
} }
const { verifyCreatorPermission } = await import('@/lib/templates/permissions') // For non-status updates, verify creator permission
const { hasPermission, error: permissionError } = await verifyCreatorPermission( const hasNonStatusUpdates =
session.user.id, name !== undefined ||
template.creatorId, details !== undefined ||
'admin' creatorId !== undefined ||
) tags !== undefined ||
updateState
if (!hasPermission) { if (hasNonStatusUpdates) {
logger.warn(`[${requestId}] User denied permission to update template ${id}`) if (!template.creatorId) {
return NextResponse.json({ error: permissionError || 'Access denied' }, { status: 403 }) logger.warn(`[${requestId}] Template ${id} has no creator, denying update`)
return NextResponse.json({ error: 'Access denied' }, { status: 403 })
}
const { verifyCreatorPermission } = await import('@/lib/templates/permissions')
const { hasPermission, error: permissionError } = await verifyCreatorPermission(
session.user.id,
template.creatorId,
'admin'
)
if (!hasPermission) {
logger.warn(`[${requestId}] User denied permission to update template ${id}`)
return NextResponse.json({ error: permissionError || 'Access denied' }, { status: 403 })
}
} }
const updateData: any = { const updateData: any = {
@@ -167,6 +191,7 @@ export async function PUT(request: NextRequest, { params }: { params: Promise<{
if (details !== undefined) updateData.details = details if (details !== undefined) updateData.details = details
if (tags !== undefined) updateData.tags = tags if (tags !== undefined) updateData.tags = tags
if (creatorId !== undefined) updateData.creatorId = creatorId if (creatorId !== undefined) updateData.creatorId = creatorId
if (status !== undefined) updateData.status = status
if (updateState && template.workflowId) { if (updateState && template.workflowId) {
const { verifyWorkflowAccess } = await import('@/socket/middleware/permissions') const { verifyWorkflowAccess } = await import('@/socket/middleware/permissions')

View File

@@ -4,6 +4,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { createA2AClient, extractTextContent, isTerminalState } from '@/lib/a2a/utils' import { createA2AClient, extractTextContent, isTerminalState } from '@/lib/a2a/utils'
import { checkHybridAuth } from '@/lib/auth/hybrid' import { checkHybridAuth } from '@/lib/auth/hybrid'
import { validateUrlWithDNS } from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -95,6 +96,14 @@ export async function POST(request: NextRequest) {
if (validatedData.files && validatedData.files.length > 0) { if (validatedData.files && validatedData.files.length > 0) {
for (const file of validatedData.files) { for (const file of validatedData.files) {
if (file.type === 'url') { if (file.type === 'url') {
const urlValidation = await validateUrlWithDNS(file.data, 'fileUrl')
if (!urlValidation.isValid) {
return NextResponse.json(
{ success: false, error: urlValidation.error },
{ status: 400 }
)
}
const filePart: FilePart = { const filePart: FilePart = {
kind: 'file', kind: 'file',
file: { file: {

View File

@@ -3,7 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { createA2AClient } from '@/lib/a2a/utils' import { createA2AClient } from '@/lib/a2a/utils'
import { checkHybridAuth } from '@/lib/auth/hybrid' import { checkHybridAuth } from '@/lib/auth/hybrid'
import { validateExternalUrl } from '@/lib/core/security/input-validation' import { validateUrlWithDNS } from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -40,7 +40,7 @@ export async function POST(request: NextRequest) {
const body = await request.json() const body = await request.json()
const validatedData = A2ASetPushNotificationSchema.parse(body) const validatedData = A2ASetPushNotificationSchema.parse(body)
const urlValidation = validateExternalUrl(validatedData.webhookUrl, 'Webhook URL') const urlValidation = await validateUrlWithDNS(validatedData.webhookUrl, 'Webhook URL')
if (!urlValidation.isValid) { if (!urlValidation.isValid) {
logger.warn(`[${requestId}] Invalid webhook URL`, { error: urlValidation.error }) logger.warn(`[${requestId}] Invalid webhook URL`, { error: urlValidation.error })
return NextResponse.json( return NextResponse.json(

View File

@@ -92,6 +92,9 @@ export async function POST(request: NextRequest) {
formData.append('comment', comment) formData.append('comment', comment)
} }
// Add minorEdit field as required by Confluence API
formData.append('minorEdit', 'false')
const response = await fetch(url, { const response = await fetch(url, {
method: 'POST', method: 'POST',
headers: { headers: {

View File

@@ -4,6 +4,7 @@ import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { validateNumericId } from '@/lib/core/security/input-validation' import { validateNumericId } from '@/lib/core/security/input-validation'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
@@ -15,7 +16,7 @@ const DiscordSendMessageSchema = z.object({
botToken: z.string().min(1, 'Bot token is required'), botToken: z.string().min(1, 'Bot token is required'),
channelId: z.string().min(1, 'Channel ID is required'), channelId: z.string().min(1, 'Channel ID is required'),
content: z.string().optional().nullable(), content: z.string().optional().nullable(),
files: z.array(z.any()).optional().nullable(), files: RawFileInputArraySchema.optional().nullable(),
}) })
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {
@@ -101,6 +102,12 @@ export async function POST(request: NextRequest) {
logger.info(`[${requestId}] Processing ${validatedData.files.length} file(s)`) logger.info(`[${requestId}] Processing ${validatedData.files.length} file(s)`)
const userFiles = processFilesToUserFiles(validatedData.files, requestId, logger) const userFiles = processFilesToUserFiles(validatedData.files, requestId, logger)
const filesOutput: Array<{
name: string
mimeType: string
data: string
size: number
}> = []
if (userFiles.length === 0) { if (userFiles.length === 0) {
logger.warn(`[${requestId}] No valid files to upload, falling back to text-only`) logger.warn(`[${requestId}] No valid files to upload, falling back to text-only`)
@@ -137,6 +144,12 @@ export async function POST(request: NextRequest) {
logger.info(`[${requestId}] Downloading file ${i}: ${userFile.name}`) logger.info(`[${requestId}] Downloading file ${i}: ${userFile.name}`)
const buffer = await downloadFileFromStorage(userFile, requestId, logger) const buffer = await downloadFileFromStorage(userFile, requestId, logger)
filesOutput.push({
name: userFile.name,
mimeType: userFile.type || 'application/octet-stream',
data: buffer.toString('base64'),
size: buffer.length,
})
const blob = new Blob([new Uint8Array(buffer)], { type: userFile.type }) const blob = new Blob([new Uint8Array(buffer)], { type: userFile.type })
formData.append(`files[${i}]`, blob, userFile.name) formData.append(`files[${i}]`, blob, userFile.name)
@@ -173,6 +186,7 @@ export async function POST(request: NextRequest) {
message: data.content, message: data.content,
data: data, data: data,
fileCount: userFiles.length, fileCount: userFiles.length,
files: filesOutput,
}, },
}) })
} catch (error) { } catch (error) {

View File

@@ -0,0 +1,132 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { httpHeaderSafeJson } from '@/lib/core/utils/validation'
import { FileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles, type RawFileInput } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
export const dynamic = 'force-dynamic'
const logger = createLogger('DropboxUploadAPI')
const DropboxUploadSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'),
path: z.string().min(1, 'Destination path is required'),
file: FileInputSchema.optional().nullable(),
// Legacy field for backwards compatibility
fileContent: z.string().optional().nullable(),
fileName: z.string().optional().nullable(),
mode: z.enum(['add', 'overwrite']).optional().nullable(),
autorename: z.boolean().optional().nullable(),
mute: z.boolean().optional().nullable(),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized Dropbox upload attempt: ${authResult.error}`)
return NextResponse.json(
{ success: false, error: authResult.error || 'Authentication required' },
{ status: 401 }
)
}
logger.info(`[${requestId}] Authenticated Dropbox upload request via ${authResult.authType}`)
const body = await request.json()
const validatedData = DropboxUploadSchema.parse(body)
let fileBuffer: Buffer
let fileName: string
// Prefer UserFile input, fall back to legacy base64 string
if (validatedData.file) {
// Process UserFile input
const userFiles = processFilesToUserFiles(
[validatedData.file as RawFileInput],
requestId,
logger
)
if (userFiles.length === 0) {
return NextResponse.json({ success: false, error: 'Invalid file input' }, { status: 400 })
}
const userFile = userFiles[0]
logger.info(`[${requestId}] Downloading file: ${userFile.name} (${userFile.size} bytes)`)
fileBuffer = await downloadFileFromStorage(userFile, requestId, logger)
fileName = userFile.name
} else if (validatedData.fileContent) {
// Legacy: base64 string input (backwards compatibility)
logger.info(`[${requestId}] Using legacy base64 content input`)
fileBuffer = Buffer.from(validatedData.fileContent, 'base64')
fileName = validatedData.fileName || 'file'
} else {
return NextResponse.json({ success: false, error: 'File is required' }, { status: 400 })
}
// Determine final path
let finalPath = validatedData.path
if (finalPath.endsWith('/')) {
finalPath = `${finalPath}${fileName}`
}
logger.info(`[${requestId}] Uploading to Dropbox: ${finalPath} (${fileBuffer.length} bytes)`)
const dropboxApiArg = {
path: finalPath,
mode: validatedData.mode || 'add',
autorename: validatedData.autorename ?? true,
mute: validatedData.mute ?? false,
}
const response = await fetch('https://content.dropboxapi.com/2/files/upload', {
method: 'POST',
headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
'Content-Type': 'application/octet-stream',
'Dropbox-API-Arg': httpHeaderSafeJson(dropboxApiArg),
},
body: new Uint8Array(fileBuffer),
})
const data = await response.json()
if (!response.ok) {
const errorMessage = data.error_summary || data.error?.message || 'Failed to upload file'
logger.error(`[${requestId}] Dropbox API error:`, { status: response.status, data })
return NextResponse.json({ success: false, error: errorMessage }, { status: response.status })
}
logger.info(`[${requestId}] File uploaded successfully to ${data.path_display}`)
return NextResponse.json({
success: true,
output: {
file: data,
},
})
} catch (error) {
if (error instanceof z.ZodError) {
logger.warn(`[${requestId}] Validation error:`, error.errors)
return NextResponse.json(
{ success: false, error: error.errors[0]?.message || 'Validation failed' },
{ status: 400 }
)
}
logger.error(`[${requestId}] Unexpected error:`, error)
return NextResponse.json(
{ success: false, error: error instanceof Error ? error.message : 'Unknown error' },
{ status: 500 }
)
}
}

View File

@@ -0,0 +1,195 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request'
export const dynamic = 'force-dynamic'
const logger = createLogger('GitHubLatestCommitAPI')
interface GitHubErrorResponse {
message?: string
}
interface GitHubCommitResponse {
sha: string
html_url: string
commit: {
message: string
author: { name: string; email: string; date: string }
committer: { name: string; email: string; date: string }
}
author?: { login: string; avatar_url: string; html_url: string }
committer?: { login: string; avatar_url: string; html_url: string }
stats?: { additions: number; deletions: number; total: number }
files?: Array<{
filename: string
status: string
additions: number
deletions: number
changes: number
patch?: string
raw_url?: string
blob_url?: string
}>
}
const GitHubLatestCommitSchema = z.object({
owner: z.string().min(1, 'Owner is required'),
repo: z.string().min(1, 'Repo is required'),
branch: z.string().optional().nullable(),
apiKey: z.string().min(1, 'API key is required'),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized GitHub latest commit attempt: ${authResult.error}`)
return NextResponse.json(
{
success: false,
error: authResult.error || 'Authentication required',
},
{ status: 401 }
)
}
const body = await request.json()
const validatedData = GitHubLatestCommitSchema.parse(body)
const { owner, repo, branch, apiKey } = validatedData
const baseUrl = `https://api.github.com/repos/${owner}/${repo}`
const commitUrl = branch ? `${baseUrl}/commits/${branch}` : `${baseUrl}/commits/HEAD`
logger.info(`[${requestId}] Fetching latest commit from GitHub`, { owner, repo, branch })
const urlValidation = await validateUrlWithDNS(commitUrl, 'commitUrl')
if (!urlValidation.isValid) {
return NextResponse.json({ success: false, error: urlValidation.error }, { status: 400 })
}
const response = await secureFetchWithPinnedIP(commitUrl, urlValidation.resolvedIP!, {
method: 'GET',
headers: {
Accept: 'application/vnd.github.v3+json',
Authorization: `Bearer ${apiKey}`,
'X-GitHub-Api-Version': '2022-11-28',
},
})
if (!response.ok) {
const errorData = (await response.json().catch(() => ({}))) as GitHubErrorResponse
logger.error(`[${requestId}] GitHub API error`, {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ success: false, error: errorData.message || `GitHub API error: ${response.status}` },
{ status: 400 }
)
}
const data = (await response.json()) as GitHubCommitResponse
const content = `Latest commit: "${data.commit.message}" by ${data.commit.author.name} on ${data.commit.author.date}. SHA: ${data.sha}`
const files = data.files || []
const fileDetailsWithContent = []
for (const file of files) {
const fileDetail: Record<string, any> = {
filename: file.filename,
additions: file.additions,
deletions: file.deletions,
changes: file.changes,
status: file.status,
raw_url: file.raw_url,
blob_url: file.blob_url,
patch: file.patch,
content: undefined,
}
if (file.status !== 'removed' && file.raw_url) {
try {
const rawUrlValidation = await validateUrlWithDNS(file.raw_url, 'rawUrl')
if (rawUrlValidation.isValid) {
const contentResponse = await secureFetchWithPinnedIP(
file.raw_url,
rawUrlValidation.resolvedIP!,
{
headers: {
Authorization: `Bearer ${apiKey}`,
'X-GitHub-Api-Version': '2022-11-28',
},
}
)
if (contentResponse.ok) {
fileDetail.content = await contentResponse.text()
}
}
} catch (error) {
logger.warn(`[${requestId}] Failed to fetch content for ${file.filename}:`, error)
}
}
fileDetailsWithContent.push(fileDetail)
}
logger.info(`[${requestId}] Latest commit fetched successfully`, {
sha: data.sha,
fileCount: files.length,
})
return NextResponse.json({
success: true,
output: {
content,
metadata: {
sha: data.sha,
html_url: data.html_url,
commit_message: data.commit.message,
author: {
name: data.commit.author.name,
login: data.author?.login || 'Unknown',
avatar_url: data.author?.avatar_url || '',
html_url: data.author?.html_url || '',
},
committer: {
name: data.commit.committer.name,
login: data.committer?.login || 'Unknown',
avatar_url: data.committer?.avatar_url || '',
html_url: data.committer?.html_url || '',
},
stats: data.stats
? {
additions: data.stats.additions,
deletions: data.stats.deletions,
total: data.stats.total,
}
: undefined,
files: fileDetailsWithContent.length > 0 ? fileDetailsWithContent : undefined,
},
},
})
} catch (error) {
logger.error(`[${requestId}] Error fetching GitHub latest commit:`, error)
return NextResponse.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error occurred',
},
{ status: 500 }
)
}
}

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
import { import {
@@ -28,7 +29,7 @@ const GmailDraftSchema = z.object({
replyToMessageId: z.string().optional().nullable(), replyToMessageId: z.string().optional().nullable(),
cc: z.string().optional().nullable(), cc: z.string().optional().nullable(),
bcc: z.string().optional().nullable(), bcc: z.string().optional().nullable(),
attachments: z.array(z.any()).optional().nullable(), attachments: RawFileInputArraySchema.optional().nullable(),
}) })
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
import { import {
@@ -28,7 +29,7 @@ const GmailSendSchema = z.object({
replyToMessageId: z.string().optional().nullable(), replyToMessageId: z.string().optional().nullable(),
cc: z.string().optional().nullable(), cc: z.string().optional().nullable(),
bcc: z.string().optional().nullable(), bcc: z.string().optional().nullable(),
attachments: z.array(z.any()).optional().nullable(), attachments: RawFileInputArraySchema.optional().nullable(),
}) })
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {

View File

@@ -0,0 +1,252 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request'
import type { GoogleDriveFile, GoogleDriveRevision } from '@/tools/google_drive/types'
import {
ALL_FILE_FIELDS,
ALL_REVISION_FIELDS,
DEFAULT_EXPORT_FORMATS,
GOOGLE_WORKSPACE_MIME_TYPES,
} from '@/tools/google_drive/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('GoogleDriveDownloadAPI')
/** Google API error response structure */
interface GoogleApiErrorResponse {
error?: {
message?: string
code?: number
status?: string
}
}
/** Google Drive revisions list response */
interface GoogleDriveRevisionsResponse {
revisions?: GoogleDriveRevision[]
nextPageToken?: string
}
const GoogleDriveDownloadSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'),
fileId: z.string().min(1, 'File ID is required'),
mimeType: z.string().optional().nullable(),
fileName: z.string().optional().nullable(),
includeRevisions: z.boolean().optional().default(true),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized Google Drive download attempt: ${authResult.error}`)
return NextResponse.json(
{
success: false,
error: authResult.error || 'Authentication required',
},
{ status: 401 }
)
}
const body = await request.json()
const validatedData = GoogleDriveDownloadSchema.parse(body)
const {
accessToken,
fileId,
mimeType: exportMimeType,
fileName,
includeRevisions,
} = validatedData
const authHeader = `Bearer ${accessToken}`
logger.info(`[${requestId}] Getting file metadata from Google Drive`, { fileId })
const metadataUrl = `https://www.googleapis.com/drive/v3/files/${fileId}?fields=${ALL_FILE_FIELDS}&supportsAllDrives=true`
const metadataUrlValidation = await validateUrlWithDNS(metadataUrl, 'metadataUrl')
if (!metadataUrlValidation.isValid) {
return NextResponse.json(
{ success: false, error: metadataUrlValidation.error },
{ status: 400 }
)
}
const metadataResponse = await secureFetchWithPinnedIP(
metadataUrl,
metadataUrlValidation.resolvedIP!,
{
headers: { Authorization: authHeader },
}
)
if (!metadataResponse.ok) {
const errorDetails = (await metadataResponse
.json()
.catch(() => ({}))) as GoogleApiErrorResponse
logger.error(`[${requestId}] Failed to get file metadata`, {
status: metadataResponse.status,
error: errorDetails,
})
return NextResponse.json(
{ success: false, error: errorDetails.error?.message || 'Failed to get file metadata' },
{ status: 400 }
)
}
const metadata = (await metadataResponse.json()) as GoogleDriveFile
const fileMimeType = metadata.mimeType
let fileBuffer: Buffer
let finalMimeType = fileMimeType
if (GOOGLE_WORKSPACE_MIME_TYPES.includes(fileMimeType)) {
const exportFormat = exportMimeType || DEFAULT_EXPORT_FORMATS[fileMimeType] || 'text/plain'
finalMimeType = exportFormat
logger.info(`[${requestId}] Exporting Google Workspace file`, {
fileId,
mimeType: fileMimeType,
exportFormat,
})
const exportUrl = `https://www.googleapis.com/drive/v3/files/${fileId}/export?mimeType=${encodeURIComponent(exportFormat)}&supportsAllDrives=true`
const exportUrlValidation = await validateUrlWithDNS(exportUrl, 'exportUrl')
if (!exportUrlValidation.isValid) {
return NextResponse.json(
{ success: false, error: exportUrlValidation.error },
{ status: 400 }
)
}
const exportResponse = await secureFetchWithPinnedIP(
exportUrl,
exportUrlValidation.resolvedIP!,
{ headers: { Authorization: authHeader } }
)
if (!exportResponse.ok) {
const exportError = (await exportResponse
.json()
.catch(() => ({}))) as GoogleApiErrorResponse
logger.error(`[${requestId}] Failed to export file`, {
status: exportResponse.status,
error: exportError,
})
return NextResponse.json(
{
success: false,
error: exportError.error?.message || 'Failed to export Google Workspace file',
},
{ status: 400 }
)
}
const arrayBuffer = await exportResponse.arrayBuffer()
fileBuffer = Buffer.from(arrayBuffer)
} else {
logger.info(`[${requestId}] Downloading regular file`, { fileId, mimeType: fileMimeType })
const downloadUrl = `https://www.googleapis.com/drive/v3/files/${fileId}?alt=media&supportsAllDrives=true`
const downloadUrlValidation = await validateUrlWithDNS(downloadUrl, 'downloadUrl')
if (!downloadUrlValidation.isValid) {
return NextResponse.json(
{ success: false, error: downloadUrlValidation.error },
{ status: 400 }
)
}
const downloadResponse = await secureFetchWithPinnedIP(
downloadUrl,
downloadUrlValidation.resolvedIP!,
{ headers: { Authorization: authHeader } }
)
if (!downloadResponse.ok) {
const downloadError = (await downloadResponse
.json()
.catch(() => ({}))) as GoogleApiErrorResponse
logger.error(`[${requestId}] Failed to download file`, {
status: downloadResponse.status,
error: downloadError,
})
return NextResponse.json(
{ success: false, error: downloadError.error?.message || 'Failed to download file' },
{ status: 400 }
)
}
const arrayBuffer = await downloadResponse.arrayBuffer()
fileBuffer = Buffer.from(arrayBuffer)
}
const canReadRevisions = metadata.capabilities?.canReadRevisions === true
if (includeRevisions && canReadRevisions) {
try {
const revisionsUrl = `https://www.googleapis.com/drive/v3/files/${fileId}/revisions?fields=revisions(${ALL_REVISION_FIELDS})&pageSize=100`
const revisionsUrlValidation = await validateUrlWithDNS(revisionsUrl, 'revisionsUrl')
if (revisionsUrlValidation.isValid) {
const revisionsResponse = await secureFetchWithPinnedIP(
revisionsUrl,
revisionsUrlValidation.resolvedIP!,
{ headers: { Authorization: authHeader } }
)
if (revisionsResponse.ok) {
const revisionsData = (await revisionsResponse.json()) as GoogleDriveRevisionsResponse
metadata.revisions = revisionsData.revisions
logger.info(`[${requestId}] Fetched file revisions`, {
fileId,
revisionCount: metadata.revisions?.length || 0,
})
}
}
} catch (error) {
logger.warn(`[${requestId}] Error fetching revisions, continuing without them`, { error })
}
}
const resolvedName = fileName || metadata.name || 'download'
logger.info(`[${requestId}] File downloaded successfully`, {
fileId,
name: resolvedName,
size: fileBuffer.length,
mimeType: finalMimeType,
})
const base64Data = fileBuffer.toString('base64')
return NextResponse.json({
success: true,
output: {
file: {
name: resolvedName,
mimeType: finalMimeType,
data: base64Data,
size: fileBuffer.length,
},
metadata,
},
})
} catch (error) {
logger.error(`[${requestId}] Error downloading Google Drive file:`, error)
return NextResponse.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error occurred',
},
{ status: 500 }
)
}
}

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils' import { processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
import { import {
@@ -20,7 +21,7 @@ const GOOGLE_DRIVE_API_BASE = 'https://www.googleapis.com/upload/drive/v3/files'
const GoogleDriveUploadSchema = z.object({ const GoogleDriveUploadSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'), accessToken: z.string().min(1, 'Access token is required'),
fileName: z.string().min(1, 'File name is required'), fileName: z.string().min(1, 'File name is required'),
file: z.any().optional().nullable(), file: RawFileInputSchema.optional().nullable(),
mimeType: z.string().optional().nullable(), mimeType: z.string().optional().nullable(),
folderId: z.string().optional().nullable(), folderId: z.string().optional().nullable(),
}) })

View File

@@ -0,0 +1,131 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request'
import { enhanceGoogleVaultError } from '@/tools/google_vault/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('GoogleVaultDownloadExportFileAPI')
const GoogleVaultDownloadExportFileSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'),
bucketName: z.string().min(1, 'Bucket name is required'),
objectName: z.string().min(1, 'Object name is required'),
fileName: z.string().optional().nullable(),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized Google Vault download attempt: ${authResult.error}`)
return NextResponse.json(
{
success: false,
error: authResult.error || 'Authentication required',
},
{ status: 401 }
)
}
const body = await request.json()
const validatedData = GoogleVaultDownloadExportFileSchema.parse(body)
const { accessToken, bucketName, objectName, fileName } = validatedData
const bucket = encodeURIComponent(bucketName)
const object = encodeURIComponent(objectName)
const downloadUrl = `https://storage.googleapis.com/storage/v1/b/${bucket}/o/${object}?alt=media`
logger.info(`[${requestId}] Downloading file from Google Vault`, { bucketName, objectName })
const urlValidation = await validateUrlWithDNS(downloadUrl, 'downloadUrl')
if (!urlValidation.isValid) {
return NextResponse.json(
{ success: false, error: enhanceGoogleVaultError(urlValidation.error || 'Invalid URL') },
{ status: 400 }
)
}
const downloadResponse = await secureFetchWithPinnedIP(downloadUrl, urlValidation.resolvedIP!, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
if (!downloadResponse.ok) {
const errorText = await downloadResponse.text().catch(() => '')
const errorMessage = `Failed to download file: ${errorText || downloadResponse.statusText}`
logger.error(`[${requestId}] Failed to download Vault export file`, {
status: downloadResponse.status,
error: errorText,
})
return NextResponse.json(
{ success: false, error: enhanceGoogleVaultError(errorMessage) },
{ status: 400 }
)
}
const contentType = downloadResponse.headers.get('content-type') || 'application/octet-stream'
const disposition = downloadResponse.headers.get('content-disposition') || ''
const match = disposition.match(/filename\*=UTF-8''([^;]+)|filename="([^"]+)"/)
let resolvedName = fileName
if (!resolvedName) {
if (match?.[1]) {
try {
resolvedName = decodeURIComponent(match[1])
} catch {
resolvedName = match[1]
}
} else if (match?.[2]) {
resolvedName = match[2]
} else if (objectName) {
const parts = objectName.split('/')
resolvedName = parts[parts.length - 1] || 'vault-export.bin'
} else {
resolvedName = 'vault-export.bin'
}
}
const arrayBuffer = await downloadResponse.arrayBuffer()
const buffer = Buffer.from(arrayBuffer)
logger.info(`[${requestId}] Vault export file downloaded successfully`, {
name: resolvedName,
size: buffer.length,
mimeType: contentType,
})
return NextResponse.json({
success: true,
output: {
file: {
name: resolvedName,
mimeType: contentType,
data: buffer.toString('base64'),
size: buffer.length,
},
},
})
} catch (error) {
logger.error(`[${requestId}] Error downloading Google Vault export file:`, error)
return NextResponse.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error occurred',
},
{ status: 500 }
)
}
}

View File

@@ -1,7 +1,10 @@
import { createLogger } from '@sim/logger' import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { validateImageUrl } from '@/lib/core/security/input-validation' import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
const logger = createLogger('ImageProxyAPI') const logger = createLogger('ImageProxyAPI')
@@ -26,7 +29,7 @@ export async function GET(request: NextRequest) {
return new NextResponse('Missing URL parameter', { status: 400 }) return new NextResponse('Missing URL parameter', { status: 400 })
} }
const urlValidation = validateImageUrl(imageUrl) const urlValidation = await validateUrlWithDNS(imageUrl, 'imageUrl')
if (!urlValidation.isValid) { if (!urlValidation.isValid) {
logger.warn(`[${requestId}] Blocked image proxy request`, { logger.warn(`[${requestId}] Blocked image proxy request`, {
url: imageUrl.substring(0, 100), url: imageUrl.substring(0, 100),
@@ -38,7 +41,8 @@ export async function GET(request: NextRequest) {
logger.info(`[${requestId}] Proxying image request for: ${imageUrl}`) logger.info(`[${requestId}] Proxying image request for: ${imageUrl}`)
try { try {
const imageResponse = await fetch(imageUrl, { const imageResponse = await secureFetchWithPinnedIP(imageUrl, urlValidation.resolvedIP!, {
method: 'GET',
headers: { headers: {
'User-Agent': 'User-Agent':
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36',
@@ -64,14 +68,14 @@ export async function GET(request: NextRequest) {
const contentType = imageResponse.headers.get('content-type') || 'image/jpeg' const contentType = imageResponse.headers.get('content-type') || 'image/jpeg'
const imageBlob = await imageResponse.blob() const imageArrayBuffer = await imageResponse.arrayBuffer()
if (imageBlob.size === 0) { if (imageArrayBuffer.byteLength === 0) {
logger.error(`[${requestId}] Empty image blob received`) logger.error(`[${requestId}] Empty image received`)
return new NextResponse('Empty image received', { status: 404 }) return new NextResponse('Empty image received', { status: 404 })
} }
return new NextResponse(imageBlob, { return new NextResponse(imageArrayBuffer, {
headers: { headers: {
'Content-Type': contentType, 'Content-Type': contentType,
'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Origin': '*',

View File

@@ -0,0 +1,121 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
import { getJiraCloudId } from '@/tools/jira/utils'
const logger = createLogger('JiraAddAttachmentAPI')
export const dynamic = 'force-dynamic'
const JiraAddAttachmentSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'),
domain: z.string().min(1, 'Domain is required'),
issueKey: z.string().min(1, 'Issue key is required'),
files: RawFileInputArraySchema,
cloudId: z.string().optional().nullable(),
})
export async function POST(request: NextRequest) {
const requestId = `jira-attach-${Date.now()}`
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
return NextResponse.json(
{ success: false, error: authResult.error || 'Unauthorized' },
{ status: 401 }
)
}
const body = await request.json()
const validatedData = JiraAddAttachmentSchema.parse(body)
const userFiles = processFilesToUserFiles(validatedData.files, requestId, logger)
if (userFiles.length === 0) {
return NextResponse.json(
{ success: false, error: 'No valid files provided for upload' },
{ status: 400 }
)
}
const cloudId =
validatedData.cloudId ||
(await getJiraCloudId(validatedData.domain, validatedData.accessToken))
const formData = new FormData()
const filesOutput: Array<{ name: string; mimeType: string; data: string; size: number }> = []
for (const file of userFiles) {
const buffer = await downloadFileFromStorage(file, requestId, logger)
filesOutput.push({
name: file.name,
mimeType: file.type || 'application/octet-stream',
data: buffer.toString('base64'),
size: buffer.length,
})
const blob = new Blob([new Uint8Array(buffer)], {
type: file.type || 'application/octet-stream',
})
formData.append('file', blob, file.name)
}
const url = `https://api.atlassian.com/ex/jira/${cloudId}/rest/api/3/issue/${validatedData.issueKey}/attachments`
const response = await fetch(url, {
method: 'POST',
headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
'X-Atlassian-Token': 'no-check',
},
body: formData,
})
if (!response.ok) {
const errorText = await response.text()
logger.error(`[${requestId}] Jira attachment upload failed`, {
status: response.status,
statusText: response.statusText,
error: errorText,
})
return NextResponse.json(
{
success: false,
error: `Failed to upload attachments: ${response.statusText}`,
},
{ status: response.status }
)
}
const attachments = await response.json()
const attachmentIds = Array.isArray(attachments)
? attachments.map((attachment) => attachment.id).filter(Boolean)
: []
return NextResponse.json({
success: true,
output: {
ts: new Date().toISOString(),
issueKey: validatedData.issueKey,
attachmentIds,
files: filesOutput,
},
})
} catch (error) {
if (error instanceof z.ZodError) {
return NextResponse.json(
{ success: false, error: 'Invalid request data', details: error.errors },
{ status: 400 }
)
}
logger.error(`[${requestId}] Jira attachment upload error`, error)
return NextResponse.json(
{ success: false, error: error instanceof Error ? error.message : 'Internal server error' },
{ status: 500 }
)
}
}

View File

@@ -2,9 +2,11 @@ import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { secureFetchWithValidation } from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { uploadFilesForTeamsMessage } from '@/tools/microsoft_teams/server-utils'
import type { GraphApiErrorResponse, GraphChatMessage } from '@/tools/microsoft_teams/types'
import { resolveMentionsForChannel, type TeamsMention } from '@/tools/microsoft_teams/utils' import { resolveMentionsForChannel, type TeamsMention } from '@/tools/microsoft_teams/utils'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -16,7 +18,7 @@ const TeamsWriteChannelSchema = z.object({
teamId: z.string().min(1, 'Team ID is required'), teamId: z.string().min(1, 'Team ID is required'),
channelId: z.string().min(1, 'Channel ID is required'), channelId: z.string().min(1, 'Channel ID is required'),
content: z.string().min(1, 'Message content is required'), content: z.string().min(1, 'Message content is required'),
files: z.array(z.any()).optional().nullable(), files: RawFileInputArraySchema.optional().nullable(),
}) })
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {
@@ -53,93 +55,12 @@ export async function POST(request: NextRequest) {
fileCount: validatedData.files?.length || 0, fileCount: validatedData.files?.length || 0,
}) })
const attachments: any[] = [] const { attachments, filesOutput } = await uploadFilesForTeamsMessage({
if (validatedData.files && validatedData.files.length > 0) { rawFiles: validatedData.files || [],
const rawFiles = validatedData.files accessToken: validatedData.accessToken,
logger.info(`[${requestId}] Processing ${rawFiles.length} file(s) for upload to OneDrive`) requestId,
logger,
const userFiles = processFilesToUserFiles(rawFiles, requestId, logger) })
for (const file of userFiles) {
try {
logger.info(`[${requestId}] Uploading file to Teams: ${file.name} (${file.size} bytes)`)
const buffer = await downloadFileFromStorage(file, requestId, logger)
const uploadUrl =
'https://graph.microsoft.com/v1.0/me/drive/root:/TeamsAttachments/' +
encodeURIComponent(file.name) +
':/content'
logger.info(`[${requestId}] Uploading to Teams: ${uploadUrl}`)
const uploadResponse = await fetch(uploadUrl, {
method: 'PUT',
headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
'Content-Type': file.type || 'application/octet-stream',
},
body: new Uint8Array(buffer),
})
if (!uploadResponse.ok) {
const errorData = await uploadResponse.json().catch(() => ({}))
logger.error(`[${requestId}] Teams upload failed:`, errorData)
throw new Error(
`Failed to upload file to Teams: ${errorData.error?.message || 'Unknown error'}`
)
}
const uploadedFile = await uploadResponse.json()
logger.info(`[${requestId}] File uploaded to Teams successfully`, {
id: uploadedFile.id,
webUrl: uploadedFile.webUrl,
})
const fileDetailsUrl = `https://graph.microsoft.com/v1.0/me/drive/items/${uploadedFile.id}?$select=id,name,webDavUrl,eTag,size`
const fileDetailsResponse = await fetch(fileDetailsUrl, {
headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
},
})
if (!fileDetailsResponse.ok) {
const errorData = await fileDetailsResponse.json().catch(() => ({}))
logger.error(`[${requestId}] Failed to get file details:`, errorData)
throw new Error(
`Failed to get file details: ${errorData.error?.message || 'Unknown error'}`
)
}
const fileDetails = await fileDetailsResponse.json()
logger.info(`[${requestId}] Got file details`, {
webDavUrl: fileDetails.webDavUrl,
eTag: fileDetails.eTag,
})
const attachmentId = fileDetails.eTag?.match(/\{([a-f0-9-]+)\}/i)?.[1] || fileDetails.id
attachments.push({
id: attachmentId,
contentType: 'reference',
contentUrl: fileDetails.webDavUrl,
name: file.name,
})
logger.info(`[${requestId}] Created attachment reference for ${file.name}`)
} catch (error) {
logger.error(`[${requestId}] Failed to process file ${file.name}:`, error)
throw new Error(
`Failed to process file "${file.name}": ${error instanceof Error ? error.message : 'Unknown error'}`
)
}
}
logger.info(
`[${requestId}] All ${attachments.length} file(s) uploaded and attachment references created`
)
}
let messageContent = validatedData.content let messageContent = validatedData.content
let contentType: 'text' | 'html' = 'text' let contentType: 'text' | 'html' = 'text'
@@ -197,17 +118,21 @@ export async function POST(request: NextRequest) {
const teamsUrl = `https://graph.microsoft.com/v1.0/teams/${encodeURIComponent(validatedData.teamId)}/channels/${encodeURIComponent(validatedData.channelId)}/messages` const teamsUrl = `https://graph.microsoft.com/v1.0/teams/${encodeURIComponent(validatedData.teamId)}/channels/${encodeURIComponent(validatedData.channelId)}/messages`
const teamsResponse = await fetch(teamsUrl, { const teamsResponse = await secureFetchWithValidation(
method: 'POST', teamsUrl,
headers: { {
'Content-Type': 'application/json', method: 'POST',
Authorization: `Bearer ${validatedData.accessToken}`, headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${validatedData.accessToken}`,
},
body: JSON.stringify(messageBody),
}, },
body: JSON.stringify(messageBody), 'teamsUrl'
}) )
if (!teamsResponse.ok) { if (!teamsResponse.ok) {
const errorData = await teamsResponse.json().catch(() => ({})) const errorData = (await teamsResponse.json().catch(() => ({}))) as GraphApiErrorResponse
logger.error(`[${requestId}] Microsoft Teams API error:`, errorData) logger.error(`[${requestId}] Microsoft Teams API error:`, errorData)
return NextResponse.json( return NextResponse.json(
{ {
@@ -218,7 +143,7 @@ export async function POST(request: NextRequest) {
) )
} }
const responseData = await teamsResponse.json() const responseData = (await teamsResponse.json()) as GraphChatMessage
logger.info(`[${requestId}] Teams channel message sent successfully`, { logger.info(`[${requestId}] Teams channel message sent successfully`, {
messageId: responseData.id, messageId: responseData.id,
attachmentCount: attachments.length, attachmentCount: attachments.length,
@@ -237,6 +162,7 @@ export async function POST(request: NextRequest) {
url: responseData.webUrl || '', url: responseData.webUrl || '',
attachmentCount: attachments.length, attachmentCount: attachments.length,
}, },
files: filesOutput,
}, },
}) })
} catch (error) { } catch (error) {

View File

@@ -2,9 +2,11 @@ import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { secureFetchWithValidation } from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { uploadFilesForTeamsMessage } from '@/tools/microsoft_teams/server-utils'
import type { GraphApiErrorResponse, GraphChatMessage } from '@/tools/microsoft_teams/types'
import { resolveMentionsForChat, type TeamsMention } from '@/tools/microsoft_teams/utils' import { resolveMentionsForChat, type TeamsMention } from '@/tools/microsoft_teams/utils'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -15,7 +17,7 @@ const TeamsWriteChatSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'), accessToken: z.string().min(1, 'Access token is required'),
chatId: z.string().min(1, 'Chat ID is required'), chatId: z.string().min(1, 'Chat ID is required'),
content: z.string().min(1, 'Message content is required'), content: z.string().min(1, 'Message content is required'),
files: z.array(z.any()).optional().nullable(), files: RawFileInputArraySchema.optional().nullable(),
}) })
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {
@@ -51,93 +53,12 @@ export async function POST(request: NextRequest) {
fileCount: validatedData.files?.length || 0, fileCount: validatedData.files?.length || 0,
}) })
const attachments: any[] = [] const { attachments, filesOutput } = await uploadFilesForTeamsMessage({
if (validatedData.files && validatedData.files.length > 0) { rawFiles: validatedData.files || [],
const rawFiles = validatedData.files accessToken: validatedData.accessToken,
logger.info(`[${requestId}] Processing ${rawFiles.length} file(s) for upload to Teams`) requestId,
logger,
const userFiles = processFilesToUserFiles(rawFiles, requestId, logger) })
for (const file of userFiles) {
try {
logger.info(`[${requestId}] Uploading file to Teams: ${file.name} (${file.size} bytes)`)
const buffer = await downloadFileFromStorage(file, requestId, logger)
const uploadUrl =
'https://graph.microsoft.com/v1.0/me/drive/root:/TeamsAttachments/' +
encodeURIComponent(file.name) +
':/content'
logger.info(`[${requestId}] Uploading to Teams: ${uploadUrl}`)
const uploadResponse = await fetch(uploadUrl, {
method: 'PUT',
headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
'Content-Type': file.type || 'application/octet-stream',
},
body: new Uint8Array(buffer),
})
if (!uploadResponse.ok) {
const errorData = await uploadResponse.json().catch(() => ({}))
logger.error(`[${requestId}] Teams upload failed:`, errorData)
throw new Error(
`Failed to upload file to Teams: ${errorData.error?.message || 'Unknown error'}`
)
}
const uploadedFile = await uploadResponse.json()
logger.info(`[${requestId}] File uploaded to Teams successfully`, {
id: uploadedFile.id,
webUrl: uploadedFile.webUrl,
})
const fileDetailsUrl = `https://graph.microsoft.com/v1.0/me/drive/items/${uploadedFile.id}?$select=id,name,webDavUrl,eTag,size`
const fileDetailsResponse = await fetch(fileDetailsUrl, {
headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
},
})
if (!fileDetailsResponse.ok) {
const errorData = await fileDetailsResponse.json().catch(() => ({}))
logger.error(`[${requestId}] Failed to get file details:`, errorData)
throw new Error(
`Failed to get file details: ${errorData.error?.message || 'Unknown error'}`
)
}
const fileDetails = await fileDetailsResponse.json()
logger.info(`[${requestId}] Got file details`, {
webDavUrl: fileDetails.webDavUrl,
eTag: fileDetails.eTag,
})
const attachmentId = fileDetails.eTag?.match(/\{([a-f0-9-]+)\}/i)?.[1] || fileDetails.id
attachments.push({
id: attachmentId,
contentType: 'reference',
contentUrl: fileDetails.webDavUrl,
name: file.name,
})
logger.info(`[${requestId}] Created attachment reference for ${file.name}`)
} catch (error) {
logger.error(`[${requestId}] Failed to process file ${file.name}:`, error)
throw new Error(
`Failed to process file "${file.name}": ${error instanceof Error ? error.message : 'Unknown error'}`
)
}
}
logger.info(
`[${requestId}] All ${attachments.length} file(s) uploaded and attachment references created`
)
}
let messageContent = validatedData.content let messageContent = validatedData.content
let contentType: 'text' | 'html' = 'text' let contentType: 'text' | 'html' = 'text'
@@ -194,17 +115,21 @@ export async function POST(request: NextRequest) {
const teamsUrl = `https://graph.microsoft.com/v1.0/chats/${encodeURIComponent(validatedData.chatId)}/messages` const teamsUrl = `https://graph.microsoft.com/v1.0/chats/${encodeURIComponent(validatedData.chatId)}/messages`
const teamsResponse = await fetch(teamsUrl, { const teamsResponse = await secureFetchWithValidation(
method: 'POST', teamsUrl,
headers: { {
'Content-Type': 'application/json', method: 'POST',
Authorization: `Bearer ${validatedData.accessToken}`, headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${validatedData.accessToken}`,
},
body: JSON.stringify(messageBody),
}, },
body: JSON.stringify(messageBody), 'teamsUrl'
}) )
if (!teamsResponse.ok) { if (!teamsResponse.ok) {
const errorData = await teamsResponse.json().catch(() => ({})) const errorData = (await teamsResponse.json().catch(() => ({}))) as GraphApiErrorResponse
logger.error(`[${requestId}] Microsoft Teams API error:`, errorData) logger.error(`[${requestId}] Microsoft Teams API error:`, errorData)
return NextResponse.json( return NextResponse.json(
{ {
@@ -215,7 +140,7 @@ export async function POST(request: NextRequest) {
) )
} }
const responseData = await teamsResponse.json() const responseData = (await teamsResponse.json()) as GraphChatMessage
logger.info(`[${requestId}] Teams message sent successfully`, { logger.info(`[${requestId}] Teams message sent successfully`, {
messageId: responseData.id, messageId: responseData.id,
attachmentCount: attachments.length, attachmentCount: attachments.length,
@@ -233,6 +158,7 @@ export async function POST(request: NextRequest) {
url: responseData.webUrl || '', url: responseData.webUrl || '',
attachmentCount: attachments.length, attachmentCount: attachments.length,
}, },
files: filesOutput,
}, },
}) })
} catch (error) { } catch (error) {

View File

@@ -2,15 +2,17 @@ import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { getBaseUrl } from '@/lib/core/utils/urls'
import { StorageService } from '@/lib/uploads'
import { import {
extractStorageKey, secureFetchWithPinnedIP,
inferContextFromKey, validateUrlWithDNS,
isInternalFileUrl, } from '@/lib/core/security/input-validation.server'
} from '@/lib/uploads/utils/file-utils' import { generateRequestId } from '@/lib/core/utils/request'
import { verifyFileAccess } from '@/app/api/files/authorization' import { FileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { isInternalFileUrl, processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils'
import {
downloadFileFromStorage,
resolveInternalFileUrl,
} from '@/lib/uploads/utils/file-utils.server'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -18,7 +20,9 @@ const logger = createLogger('MistralParseAPI')
const MistralParseSchema = z.object({ const MistralParseSchema = z.object({
apiKey: z.string().min(1, 'API key is required'), apiKey: z.string().min(1, 'API key is required'),
filePath: z.string().min(1, 'File path is required'), filePath: z.string().min(1, 'File path is required').optional(),
fileData: FileInputSchema.optional(),
file: FileInputSchema.optional(),
resultType: z.string().optional(), resultType: z.string().optional(),
pages: z.array(z.number()).optional(), pages: z.array(z.number()).optional(),
includeImageBase64: z.boolean().optional(), includeImageBase64: z.boolean().optional(),
@@ -49,66 +53,140 @@ export async function POST(request: NextRequest) {
const body = await request.json() const body = await request.json()
const validatedData = MistralParseSchema.parse(body) const validatedData = MistralParseSchema.parse(body)
const fileData = validatedData.file || validatedData.fileData
const filePath = typeof fileData === 'string' ? fileData : validatedData.filePath
if (!fileData && (!filePath || filePath.trim() === '')) {
return NextResponse.json(
{
success: false,
error: 'File input is required',
},
{ status: 400 }
)
}
logger.info(`[${requestId}] Mistral parse request`, { logger.info(`[${requestId}] Mistral parse request`, {
filePath: validatedData.filePath, hasFileData: Boolean(fileData),
isWorkspaceFile: isInternalFileUrl(validatedData.filePath), filePath,
isWorkspaceFile: filePath ? isInternalFileUrl(filePath) : false,
userId, userId,
}) })
let fileUrl = validatedData.filePath const mistralBody: any = {
model: 'mistral-ocr-latest',
}
if (isInternalFileUrl(validatedData.filePath)) { if (fileData && typeof fileData === 'object') {
const rawFile = fileData
let userFile
try { try {
const storageKey = extractStorageKey(validatedData.filePath) userFile = processSingleFileToUserFile(rawFile, requestId, logger)
const context = inferContextFromKey(storageKey)
const hasAccess = await verifyFileAccess(
storageKey,
userId,
undefined, // customConfig
context, // context
false // isLocal
)
if (!hasAccess) {
logger.warn(`[${requestId}] Unauthorized presigned URL generation attempt`, {
userId,
key: storageKey,
context,
})
return NextResponse.json(
{
success: false,
error: 'File not found',
},
{ status: 404 }
)
}
fileUrl = await StorageService.generatePresignedDownloadUrl(storageKey, context, 5 * 60)
logger.info(`[${requestId}] Generated presigned URL for ${context} file`)
} catch (error) { } catch (error) {
logger.error(`[${requestId}] Failed to generate presigned URL:`, error)
return NextResponse.json( return NextResponse.json(
{ {
success: false, success: false,
error: 'Failed to generate file access URL', error: error instanceof Error ? error.message : 'Failed to process file',
}, },
{ status: 500 } { status: 400 }
) )
} }
} else if (validatedData.filePath?.startsWith('/')) {
const baseUrl = getBaseUrl()
fileUrl = `${baseUrl}${validatedData.filePath}`
}
const mistralBody: any = { let mimeType = userFile.type
model: 'mistral-ocr-latest', if (!mimeType || mimeType === 'application/octet-stream') {
document: { const filename = userFile.name?.toLowerCase() || ''
type: 'document_url', if (filename.endsWith('.pdf')) {
document_url: fileUrl, mimeType = 'application/pdf'
}, } else if (filename.endsWith('.png')) {
mimeType = 'image/png'
} else if (filename.endsWith('.jpg') || filename.endsWith('.jpeg')) {
mimeType = 'image/jpeg'
} else if (filename.endsWith('.gif')) {
mimeType = 'image/gif'
} else if (filename.endsWith('.webp')) {
mimeType = 'image/webp'
} else {
mimeType = 'application/pdf'
}
}
let base64 = userFile.base64
if (!base64) {
const buffer = await downloadFileFromStorage(userFile, requestId, logger)
base64 = buffer.toString('base64')
}
const base64Payload = base64.startsWith('data:')
? base64
: `data:${mimeType};base64,${base64}`
// Mistral API uses different document types for images vs documents
const isImage = mimeType.startsWith('image/')
if (isImage) {
mistralBody.document = {
type: 'image_url',
image_url: base64Payload,
}
} else {
mistralBody.document = {
type: 'document_url',
document_url: base64Payload,
}
}
} else if (filePath) {
let fileUrl = filePath
const isInternalFilePath = isInternalFileUrl(filePath)
if (isInternalFilePath) {
const resolution = await resolveInternalFileUrl(filePath, userId, requestId, logger)
if (resolution.error) {
return NextResponse.json(
{
success: false,
error: resolution.error.message,
},
{ status: resolution.error.status }
)
}
fileUrl = resolution.fileUrl || fileUrl
} else if (filePath.startsWith('/')) {
logger.warn(`[${requestId}] Invalid internal path`, {
userId,
path: filePath.substring(0, 50),
})
return NextResponse.json(
{
success: false,
error: 'Invalid file path. Only uploaded files are supported for internal paths.',
},
{ status: 400 }
)
} else {
const urlValidation = await validateUrlWithDNS(fileUrl, 'filePath')
if (!urlValidation.isValid) {
return NextResponse.json(
{
success: false,
error: urlValidation.error,
},
{ status: 400 }
)
}
}
const imageExtensions = ['.png', '.jpg', '.jpeg', '.gif', '.webp', '.avif']
const pathname = new URL(fileUrl).pathname.toLowerCase()
const isImageUrl = imageExtensions.some((ext) => pathname.endsWith(ext))
if (isImageUrl) {
mistralBody.document = {
type: 'image_url',
image_url: fileUrl,
}
} else {
mistralBody.document = {
type: 'document_url',
document_url: fileUrl,
}
}
} }
if (validatedData.pages) { if (validatedData.pages) {
@@ -124,15 +202,34 @@ export async function POST(request: NextRequest) {
mistralBody.image_min_size = validatedData.imageMinSize mistralBody.image_min_size = validatedData.imageMinSize
} }
const mistralResponse = await fetch('https://api.mistral.ai/v1/ocr', { const mistralEndpoint = 'https://api.mistral.ai/v1/ocr'
method: 'POST', const mistralValidation = await validateUrlWithDNS(mistralEndpoint, 'Mistral API URL')
headers: { if (!mistralValidation.isValid) {
'Content-Type': 'application/json', logger.error(`[${requestId}] Mistral API URL validation failed`, {
Accept: 'application/json', error: mistralValidation.error,
Authorization: `Bearer ${validatedData.apiKey}`, })
}, return NextResponse.json(
body: JSON.stringify(mistralBody), {
}) success: false,
error: 'Failed to reach Mistral API',
},
{ status: 502 }
)
}
const mistralResponse = await secureFetchWithPinnedIP(
mistralEndpoint,
mistralValidation.resolvedIP!,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
Accept: 'application/json',
Authorization: `Bearer ${validatedData.apiKey}`,
},
body: JSON.stringify(mistralBody),
}
)
if (!mistralResponse.ok) { if (!mistralResponse.ok) {
const errorText = await mistralResponse.text() const errorText = await mistralResponse.text()

View File

@@ -0,0 +1,177 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request'
export const dynamic = 'force-dynamic'
/** Microsoft Graph API error response structure */
interface GraphApiError {
error?: {
code?: string
message?: string
}
}
/** Microsoft Graph API drive item metadata response */
interface DriveItemMetadata {
id?: string
name?: string
folder?: Record<string, unknown>
file?: {
mimeType?: string
}
}
const logger = createLogger('OneDriveDownloadAPI')
const OneDriveDownloadSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'),
fileId: z.string().min(1, 'File ID is required'),
fileName: z.string().optional().nullable(),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized OneDrive download attempt: ${authResult.error}`)
return NextResponse.json(
{
success: false,
error: authResult.error || 'Authentication required',
},
{ status: 401 }
)
}
const body = await request.json()
const validatedData = OneDriveDownloadSchema.parse(body)
const { accessToken, fileId, fileName } = validatedData
const authHeader = `Bearer ${accessToken}`
logger.info(`[${requestId}] Getting file metadata from OneDrive`, { fileId })
const metadataUrl = `https://graph.microsoft.com/v1.0/me/drive/items/${fileId}`
const metadataUrlValidation = await validateUrlWithDNS(metadataUrl, 'metadataUrl')
if (!metadataUrlValidation.isValid) {
return NextResponse.json(
{ success: false, error: metadataUrlValidation.error },
{ status: 400 }
)
}
const metadataResponse = await secureFetchWithPinnedIP(
metadataUrl,
metadataUrlValidation.resolvedIP!,
{
headers: { Authorization: authHeader },
}
)
if (!metadataResponse.ok) {
const errorDetails = (await metadataResponse.json().catch(() => ({}))) as GraphApiError
logger.error(`[${requestId}] Failed to get file metadata`, {
status: metadataResponse.status,
error: errorDetails,
})
return NextResponse.json(
{ success: false, error: errorDetails.error?.message || 'Failed to get file metadata' },
{ status: 400 }
)
}
const metadata = (await metadataResponse.json()) as DriveItemMetadata
if (metadata.folder && !metadata.file) {
logger.error(`[${requestId}] Attempted to download a folder`, {
itemId: metadata.id,
itemName: metadata.name,
})
return NextResponse.json(
{
success: false,
error: `Cannot download folder "${metadata.name}". Please select a file instead.`,
},
{ status: 400 }
)
}
const mimeType = metadata.file?.mimeType || 'application/octet-stream'
logger.info(`[${requestId}] Downloading file from OneDrive`, { fileId, mimeType })
const downloadUrl = `https://graph.microsoft.com/v1.0/me/drive/items/${fileId}/content`
const downloadUrlValidation = await validateUrlWithDNS(downloadUrl, 'downloadUrl')
if (!downloadUrlValidation.isValid) {
return NextResponse.json(
{ success: false, error: downloadUrlValidation.error },
{ status: 400 }
)
}
const downloadResponse = await secureFetchWithPinnedIP(
downloadUrl,
downloadUrlValidation.resolvedIP!,
{
headers: { Authorization: authHeader },
}
)
if (!downloadResponse.ok) {
const downloadError = (await downloadResponse.json().catch(() => ({}))) as GraphApiError
logger.error(`[${requestId}] Failed to download file`, {
status: downloadResponse.status,
error: downloadError,
})
return NextResponse.json(
{ success: false, error: downloadError.error?.message || 'Failed to download file' },
{ status: 400 }
)
}
const arrayBuffer = await downloadResponse.arrayBuffer()
const fileBuffer = Buffer.from(arrayBuffer)
const resolvedName = fileName || metadata.name || 'download'
logger.info(`[${requestId}] File downloaded successfully`, {
fileId,
name: resolvedName,
size: fileBuffer.length,
mimeType,
})
const base64Data = fileBuffer.toString('base64')
return NextResponse.json({
success: true,
output: {
file: {
name: resolvedName,
mimeType,
data: base64Data,
size: fileBuffer.length,
},
},
})
} catch (error) {
logger.error(`[${requestId}] Error downloading OneDrive file:`, error)
return NextResponse.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error occurred',
},
{ status: 500 }
)
}
}

View File

@@ -4,7 +4,9 @@ import * as XLSX from 'xlsx'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { validateMicrosoftGraphId } from '@/lib/core/security/input-validation' import { validateMicrosoftGraphId } from '@/lib/core/security/input-validation'
import { secureFetchWithValidation } from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { import {
getExtensionFromMimeType, getExtensionFromMimeType,
processSingleFileToUserFile, processSingleFileToUserFile,
@@ -29,12 +31,33 @@ const ExcelValuesSchema = z.union([
const OneDriveUploadSchema = z.object({ const OneDriveUploadSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'), accessToken: z.string().min(1, 'Access token is required'),
fileName: z.string().min(1, 'File name is required'), fileName: z.string().min(1, 'File name is required'),
file: z.any().optional(), file: RawFileInputSchema.optional(),
folderId: z.string().optional().nullable(), folderId: z.string().optional().nullable(),
mimeType: z.string().nullish(), mimeType: z.string().nullish(),
values: ExcelValuesSchema.optional().nullable(), values: ExcelValuesSchema.optional().nullable(),
conflictBehavior: z.enum(['fail', 'replace', 'rename']).optional().nullable(),
}) })
/** Microsoft Graph DriveItem response */
interface OneDriveFileData {
id: string
name: string
size: number
webUrl: string
createdDateTime: string
lastModifiedDateTime: string
file?: { mimeType: string }
parentReference?: { id: string; path: string }
'@microsoft.graph.downloadUrl'?: string
}
/** Microsoft Graph Excel range response */
interface ExcelRangeData {
address?: string
addressLocal?: string
values?: unknown[][]
}
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {
const requestId = generateRequestId() const requestId = generateRequestId()
@@ -88,25 +111,9 @@ export async function POST(request: NextRequest) {
) )
} }
let fileToProcess
if (Array.isArray(rawFile)) {
if (rawFile.length === 0) {
return NextResponse.json(
{
success: false,
error: 'No file provided',
},
{ status: 400 }
)
}
fileToProcess = rawFile[0]
} else {
fileToProcess = rawFile
}
let userFile let userFile
try { try {
userFile = processSingleFileToUserFile(fileToProcess, requestId, logger) userFile = processSingleFileToUserFile(rawFile, requestId, logger)
} catch (error) { } catch (error) {
return NextResponse.json( return NextResponse.json(
{ {
@@ -179,14 +186,23 @@ export async function POST(request: NextRequest) {
uploadUrl = `${MICROSOFT_GRAPH_BASE}/me/drive/root:/${encodeURIComponent(fileName)}:/content` uploadUrl = `${MICROSOFT_GRAPH_BASE}/me/drive/root:/${encodeURIComponent(fileName)}:/content`
} }
const uploadResponse = await fetch(uploadUrl, { // Add conflict behavior if specified (defaults to replace by Microsoft Graph API)
method: 'PUT', if (validatedData.conflictBehavior) {
headers: { uploadUrl += `?@microsoft.graph.conflictBehavior=${validatedData.conflictBehavior}`
Authorization: `Bearer ${validatedData.accessToken}`, }
'Content-Type': mimeType,
const uploadResponse = await secureFetchWithValidation(
uploadUrl,
{
method: 'PUT',
headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
'Content-Type': mimeType,
},
body: fileBuffer,
}, },
body: new Uint8Array(fileBuffer), 'uploadUrl'
}) )
if (!uploadResponse.ok) { if (!uploadResponse.ok) {
const errorText = await uploadResponse.text() const errorText = await uploadResponse.text()
@@ -200,7 +216,7 @@ export async function POST(request: NextRequest) {
) )
} }
const fileData = await uploadResponse.json() const fileData = (await uploadResponse.json()) as OneDriveFileData
let excelWriteResult: any | undefined let excelWriteResult: any | undefined
const shouldWriteExcelContent = const shouldWriteExcelContent =
@@ -209,8 +225,11 @@ export async function POST(request: NextRequest) {
if (shouldWriteExcelContent) { if (shouldWriteExcelContent) {
try { try {
let workbookSessionId: string | undefined let workbookSessionId: string | undefined
const sessionResp = await fetch( const sessionUrl = `${MICROSOFT_GRAPH_BASE}/me/drive/items/${encodeURIComponent(
`${MICROSOFT_GRAPH_BASE}/me/drive/items/${encodeURIComponent(fileData.id)}/workbook/createSession`, fileData.id
)}/workbook/createSession`
const sessionResp = await secureFetchWithValidation(
sessionUrl,
{ {
method: 'POST', method: 'POST',
headers: { headers: {
@@ -218,11 +237,12 @@ export async function POST(request: NextRequest) {
'Content-Type': 'application/json', 'Content-Type': 'application/json',
}, },
body: JSON.stringify({ persistChanges: true }), body: JSON.stringify({ persistChanges: true }),
} },
'sessionUrl'
) )
if (sessionResp.ok) { if (sessionResp.ok) {
const sessionData = await sessionResp.json() const sessionData = (await sessionResp.json()) as { id?: string }
workbookSessionId = sessionData?.id workbookSessionId = sessionData?.id
} }
@@ -231,14 +251,19 @@ export async function POST(request: NextRequest) {
const listUrl = `${MICROSOFT_GRAPH_BASE}/me/drive/items/${encodeURIComponent( const listUrl = `${MICROSOFT_GRAPH_BASE}/me/drive/items/${encodeURIComponent(
fileData.id fileData.id
)}/workbook/worksheets?$select=name&$orderby=position&$top=1` )}/workbook/worksheets?$select=name&$orderby=position&$top=1`
const listResp = await fetch(listUrl, { const listResp = await secureFetchWithValidation(
headers: { listUrl,
Authorization: `Bearer ${validatedData.accessToken}`, {
...(workbookSessionId ? { 'workbook-session-id': workbookSessionId } : {}), method: 'GET',
headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
...(workbookSessionId ? { 'workbook-session-id': workbookSessionId } : {}),
},
}, },
}) 'listUrl'
)
if (listResp.ok) { if (listResp.ok) {
const listData = await listResp.json() const listData = (await listResp.json()) as { value?: Array<{ name?: string }> }
const firstSheetName = listData?.value?.[0]?.name const firstSheetName = listData?.value?.[0]?.name
if (firstSheetName) { if (firstSheetName) {
sheetName = firstSheetName sheetName = firstSheetName
@@ -297,15 +322,19 @@ export async function POST(request: NextRequest) {
)}')/range(address='${encodeURIComponent(computedRangeAddress)}')` )}')/range(address='${encodeURIComponent(computedRangeAddress)}')`
) )
const excelWriteResponse = await fetch(url.toString(), { const excelWriteResponse = await secureFetchWithValidation(
method: 'PATCH', url.toString(),
headers: { {
Authorization: `Bearer ${validatedData.accessToken}`, method: 'PATCH',
'Content-Type': 'application/json', headers: {
...(workbookSessionId ? { 'workbook-session-id': workbookSessionId } : {}), Authorization: `Bearer ${validatedData.accessToken}`,
'Content-Type': 'application/json',
...(workbookSessionId ? { 'workbook-session-id': workbookSessionId } : {}),
},
body: JSON.stringify({ values: processedValues }),
}, },
body: JSON.stringify({ values: processedValues }), 'excelWriteUrl'
}) )
if (!excelWriteResponse || !excelWriteResponse.ok) { if (!excelWriteResponse || !excelWriteResponse.ok) {
const errorText = excelWriteResponse ? await excelWriteResponse.text() : 'no response' const errorText = excelWriteResponse ? await excelWriteResponse.text() : 'no response'
@@ -320,7 +349,7 @@ export async function POST(request: NextRequest) {
details: errorText, details: errorText,
} }
} else { } else {
const writeData = await excelWriteResponse.json() const writeData = (await excelWriteResponse.json()) as ExcelRangeData
const addr = writeData.address || writeData.addressLocal const addr = writeData.address || writeData.addressLocal
const v = writeData.values || [] const v = writeData.values || []
excelWriteResult = { excelWriteResult = {
@@ -328,21 +357,25 @@ export async function POST(request: NextRequest) {
updatedRange: addr, updatedRange: addr,
updatedRows: Array.isArray(v) ? v.length : undefined, updatedRows: Array.isArray(v) ? v.length : undefined,
updatedColumns: Array.isArray(v) && v[0] ? v[0].length : undefined, updatedColumns: Array.isArray(v) && v[0] ? v[0].length : undefined,
updatedCells: Array.isArray(v) && v[0] ? v.length * (v[0] as any[]).length : undefined, updatedCells: Array.isArray(v) && v[0] ? v.length * v[0].length : undefined,
} }
} }
if (workbookSessionId) { if (workbookSessionId) {
try { try {
const closeResp = await fetch( const closeUrl = `${MICROSOFT_GRAPH_BASE}/me/drive/items/${encodeURIComponent(
`${MICROSOFT_GRAPH_BASE}/me/drive/items/${encodeURIComponent(fileData.id)}/workbook/closeSession`, fileData.id
)}/workbook/closeSession`
const closeResp = await secureFetchWithValidation(
closeUrl,
{ {
method: 'POST', method: 'POST',
headers: { headers: {
Authorization: `Bearer ${validatedData.accessToken}`, Authorization: `Bearer ${validatedData.accessToken}`,
'workbook-session-id': workbookSessionId, 'workbook-session-id': workbookSessionId,
}, },
} },
'closeSessionUrl'
) )
if (!closeResp.ok) { if (!closeResp.ok) {
const closeText = await closeResp.text() const closeText = await closeResp.text()

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
@@ -18,7 +19,7 @@ const OutlookDraftSchema = z.object({
contentType: z.enum(['text', 'html']).optional().nullable(), contentType: z.enum(['text', 'html']).optional().nullable(),
cc: z.string().optional().nullable(), cc: z.string().optional().nullable(),
bcc: z.string().optional().nullable(), bcc: z.string().optional().nullable(),
attachments: z.array(z.any()).optional().nullable(), attachments: RawFileInputArraySchema.optional().nullable(),
}) })
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
@@ -20,7 +21,7 @@ const OutlookSendSchema = z.object({
bcc: z.string().optional().nullable(), bcc: z.string().optional().nullable(),
replyToMessageId: z.string().optional().nullable(), replyToMessageId: z.string().optional().nullable(),
conversationId: z.string().optional().nullable(), conversationId: z.string().optional().nullable(),
attachments: z.array(z.any()).optional().nullable(), attachments: RawFileInputArraySchema.optional().nullable(),
}) })
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {
@@ -95,14 +96,14 @@ export async function POST(request: NextRequest) {
if (attachments.length > 0) { if (attachments.length > 0) {
const totalSize = attachments.reduce((sum, file) => sum + file.size, 0) const totalSize = attachments.reduce((sum, file) => sum + file.size, 0)
const maxSize = 4 * 1024 * 1024 // 4MB const maxSize = 3 * 1024 * 1024 // 3MB - Microsoft Graph API limit for inline attachments
if (totalSize > maxSize) { if (totalSize > maxSize) {
const sizeMB = (totalSize / (1024 * 1024)).toFixed(2) const sizeMB = (totalSize / (1024 * 1024)).toFixed(2)
return NextResponse.json( return NextResponse.json(
{ {
success: false, success: false,
error: `Total attachment size (${sizeMB}MB) exceeds Outlook's limit of 4MB per request`, error: `Total attachment size (${sizeMB}MB) exceeds Microsoft Graph API limit of 3MB per request`,
}, },
{ status: 400 } { status: 400 }
) )

View File

@@ -0,0 +1,165 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request'
import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('PipedriveGetFilesAPI')
interface PipedriveFile {
id?: number
name?: string
url?: string
}
interface PipedriveApiResponse {
success: boolean
data?: PipedriveFile[]
error?: string
}
const PipedriveGetFilesSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'),
deal_id: z.string().optional().nullable(),
person_id: z.string().optional().nullable(),
org_id: z.string().optional().nullable(),
limit: z.string().optional().nullable(),
downloadFiles: z.boolean().optional().default(false),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized Pipedrive get files attempt: ${authResult.error}`)
return NextResponse.json(
{
success: false,
error: authResult.error || 'Authentication required',
},
{ status: 401 }
)
}
const body = await request.json()
const validatedData = PipedriveGetFilesSchema.parse(body)
const { accessToken, deal_id, person_id, org_id, limit, downloadFiles } = validatedData
const baseUrl = 'https://api.pipedrive.com/v1/files'
const queryParams = new URLSearchParams()
if (deal_id) queryParams.append('deal_id', deal_id)
if (person_id) queryParams.append('person_id', person_id)
if (org_id) queryParams.append('org_id', org_id)
if (limit) queryParams.append('limit', limit)
const queryString = queryParams.toString()
const apiUrl = queryString ? `${baseUrl}?${queryString}` : baseUrl
logger.info(`[${requestId}] Fetching files from Pipedrive`, { deal_id, person_id, org_id })
const urlValidation = await validateUrlWithDNS(apiUrl, 'apiUrl')
if (!urlValidation.isValid) {
return NextResponse.json({ success: false, error: urlValidation.error }, { status: 400 })
}
const response = await secureFetchWithPinnedIP(apiUrl, urlValidation.resolvedIP!, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
Accept: 'application/json',
},
})
const data = (await response.json()) as PipedriveApiResponse
if (!data.success) {
logger.error(`[${requestId}] Pipedrive API request failed`, { data })
return NextResponse.json(
{ success: false, error: data.error || 'Failed to fetch files from Pipedrive' },
{ status: 400 }
)
}
const files = data.data || []
const downloadedFiles: Array<{
name: string
mimeType: string
data: string
size: number
}> = []
if (downloadFiles) {
for (const file of files) {
if (!file?.url) continue
try {
const fileUrlValidation = await validateUrlWithDNS(file.url, 'fileUrl')
if (!fileUrlValidation.isValid) continue
const downloadResponse = await secureFetchWithPinnedIP(
file.url,
fileUrlValidation.resolvedIP!,
{
method: 'GET',
headers: { Authorization: `Bearer ${accessToken}` },
}
)
if (!downloadResponse.ok) continue
const arrayBuffer = await downloadResponse.arrayBuffer()
const buffer = Buffer.from(arrayBuffer)
const extension = getFileExtension(file.name || '')
const mimeType =
downloadResponse.headers.get('content-type') || getMimeTypeFromExtension(extension)
const fileName = file.name || `pipedrive-file-${file.id || Date.now()}`
downloadedFiles.push({
name: fileName,
mimeType,
data: buffer.toString('base64'),
size: buffer.length,
})
} catch (error) {
logger.warn(`[${requestId}] Failed to download file ${file.id}:`, error)
}
}
}
logger.info(`[${requestId}] Pipedrive files fetched successfully`, {
fileCount: files.length,
downloadedCount: downloadedFiles.length,
})
return NextResponse.json({
success: true,
output: {
files,
downloadedFiles: downloadedFiles.length > 0 ? downloadedFiles : undefined,
total_items: files.length,
success: true,
},
})
} catch (error) {
logger.error(`[${requestId}] Error fetching Pipedrive files:`, error)
return NextResponse.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error occurred',
},
{ status: 500 }
)
}
}

View File

@@ -2,15 +2,14 @@ import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { getBaseUrl } from '@/lib/core/utils/urls'
import { StorageService } from '@/lib/uploads'
import { import {
extractStorageKey, secureFetchWithPinnedIP,
inferContextFromKey, validateUrlWithDNS,
isInternalFileUrl, } from '@/lib/core/security/input-validation.server'
} from '@/lib/uploads/utils/file-utils' import { generateRequestId } from '@/lib/core/utils/request'
import { verifyFileAccess } from '@/app/api/files/authorization' import { RawFileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { isInternalFileUrl } from '@/lib/uploads/utils/file-utils'
import { resolveFileInputToUrl } from '@/lib/uploads/utils/file-utils.server'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -18,7 +17,8 @@ const logger = createLogger('PulseParseAPI')
const PulseParseSchema = z.object({ const PulseParseSchema = z.object({
apiKey: z.string().min(1, 'API key is required'), apiKey: z.string().min(1, 'API key is required'),
filePath: z.string().min(1, 'File path is required'), filePath: z.string().optional(),
file: RawFileInputSchema.optional(),
pages: z.string().optional(), pages: z.string().optional(),
extractFigure: z.boolean().optional(), extractFigure: z.boolean().optional(),
figureDescription: z.boolean().optional(), figureDescription: z.boolean().optional(),
@@ -51,50 +51,30 @@ export async function POST(request: NextRequest) {
const validatedData = PulseParseSchema.parse(body) const validatedData = PulseParseSchema.parse(body)
logger.info(`[${requestId}] Pulse parse request`, { logger.info(`[${requestId}] Pulse parse request`, {
fileName: validatedData.file?.name,
filePath: validatedData.filePath, filePath: validatedData.filePath,
isWorkspaceFile: isInternalFileUrl(validatedData.filePath), isWorkspaceFile: validatedData.filePath ? isInternalFileUrl(validatedData.filePath) : false,
userId, userId,
}) })
let fileUrl = validatedData.filePath const resolution = await resolveFileInputToUrl({
file: validatedData.file,
filePath: validatedData.filePath,
userId,
requestId,
logger,
})
if (isInternalFileUrl(validatedData.filePath)) { if (resolution.error) {
try { return NextResponse.json(
const storageKey = extractStorageKey(validatedData.filePath) { success: false, error: resolution.error.message },
const context = inferContextFromKey(storageKey) { status: resolution.error.status }
)
}
const hasAccess = await verifyFileAccess(storageKey, userId, undefined, context, false) const fileUrl = resolution.fileUrl
if (!fileUrl) {
if (!hasAccess) { return NextResponse.json({ success: false, error: 'File input is required' }, { status: 400 })
logger.warn(`[${requestId}] Unauthorized presigned URL generation attempt`, {
userId,
key: storageKey,
context,
})
return NextResponse.json(
{
success: false,
error: 'File not found',
},
{ status: 404 }
)
}
fileUrl = await StorageService.generatePresignedDownloadUrl(storageKey, context, 5 * 60)
logger.info(`[${requestId}] Generated presigned URL for ${context} file`)
} catch (error) {
logger.error(`[${requestId}] Failed to generate presigned URL:`, error)
return NextResponse.json(
{
success: false,
error: 'Failed to generate file access URL',
},
{ status: 500 }
)
}
} else if (validatedData.filePath?.startsWith('/')) {
const baseUrl = getBaseUrl()
fileUrl = `${baseUrl}${validatedData.filePath}`
} }
const formData = new FormData() const formData = new FormData()
@@ -119,13 +99,36 @@ export async function POST(request: NextRequest) {
formData.append('chunk_size', String(validatedData.chunkSize)) formData.append('chunk_size', String(validatedData.chunkSize))
} }
const pulseResponse = await fetch('https://api.runpulse.com/extract', { const pulseEndpoint = 'https://api.runpulse.com/extract'
method: 'POST', const pulseValidation = await validateUrlWithDNS(pulseEndpoint, 'Pulse API URL')
headers: { if (!pulseValidation.isValid) {
'x-api-key': validatedData.apiKey, logger.error(`[${requestId}] Pulse API URL validation failed`, {
}, error: pulseValidation.error,
body: formData, })
}) return NextResponse.json(
{
success: false,
error: 'Failed to reach Pulse API',
},
{ status: 502 }
)
}
const pulsePayload = new Response(formData)
const contentType = pulsePayload.headers.get('content-type') || 'multipart/form-data'
const bodyBuffer = Buffer.from(await pulsePayload.arrayBuffer())
const pulseResponse = await secureFetchWithPinnedIP(
pulseEndpoint,
pulseValidation.resolvedIP!,
{
method: 'POST',
headers: {
'x-api-key': validatedData.apiKey,
'Content-Type': contentType,
},
body: bodyBuffer,
}
)
if (!pulseResponse.ok) { if (!pulseResponse.ok) {
const errorText = await pulseResponse.text() const errorText = await pulseResponse.text()

View File

@@ -2,15 +2,14 @@ import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { getBaseUrl } from '@/lib/core/utils/urls'
import { StorageService } from '@/lib/uploads'
import { import {
extractStorageKey, secureFetchWithPinnedIP,
inferContextFromKey, validateUrlWithDNS,
isInternalFileUrl, } from '@/lib/core/security/input-validation.server'
} from '@/lib/uploads/utils/file-utils' import { generateRequestId } from '@/lib/core/utils/request'
import { verifyFileAccess } from '@/app/api/files/authorization' import { RawFileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { isInternalFileUrl } from '@/lib/uploads/utils/file-utils'
import { resolveFileInputToUrl } from '@/lib/uploads/utils/file-utils.server'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -18,7 +17,8 @@ const logger = createLogger('ReductoParseAPI')
const ReductoParseSchema = z.object({ const ReductoParseSchema = z.object({
apiKey: z.string().min(1, 'API key is required'), apiKey: z.string().min(1, 'API key is required'),
filePath: z.string().min(1, 'File path is required'), filePath: z.string().optional(),
file: RawFileInputSchema.optional(),
pages: z.array(z.number()).optional(), pages: z.array(z.number()).optional(),
tableOutputFormat: z.enum(['html', 'md']).optional(), tableOutputFormat: z.enum(['html', 'md']).optional(),
}) })
@@ -47,56 +47,30 @@ export async function POST(request: NextRequest) {
const validatedData = ReductoParseSchema.parse(body) const validatedData = ReductoParseSchema.parse(body)
logger.info(`[${requestId}] Reducto parse request`, { logger.info(`[${requestId}] Reducto parse request`, {
fileName: validatedData.file?.name,
filePath: validatedData.filePath, filePath: validatedData.filePath,
isWorkspaceFile: isInternalFileUrl(validatedData.filePath), isWorkspaceFile: validatedData.filePath ? isInternalFileUrl(validatedData.filePath) : false,
userId, userId,
}) })
let fileUrl = validatedData.filePath const resolution = await resolveFileInputToUrl({
file: validatedData.file,
filePath: validatedData.filePath,
userId,
requestId,
logger,
})
if (isInternalFileUrl(validatedData.filePath)) { if (resolution.error) {
try { return NextResponse.json(
const storageKey = extractStorageKey(validatedData.filePath) { success: false, error: resolution.error.message },
const context = inferContextFromKey(storageKey) { status: resolution.error.status }
)
}
const hasAccess = await verifyFileAccess( const fileUrl = resolution.fileUrl
storageKey, if (!fileUrl) {
userId, return NextResponse.json({ success: false, error: 'File input is required' }, { status: 400 })
undefined, // customConfig
context, // context
false // isLocal
)
if (!hasAccess) {
logger.warn(`[${requestId}] Unauthorized presigned URL generation attempt`, {
userId,
key: storageKey,
context,
})
return NextResponse.json(
{
success: false,
error: 'File not found',
},
{ status: 404 }
)
}
fileUrl = await StorageService.generatePresignedDownloadUrl(storageKey, context, 5 * 60)
logger.info(`[${requestId}] Generated presigned URL for ${context} file`)
} catch (error) {
logger.error(`[${requestId}] Failed to generate presigned URL:`, error)
return NextResponse.json(
{
success: false,
error: 'Failed to generate file access URL',
},
{ status: 500 }
)
}
} else if (validatedData.filePath?.startsWith('/')) {
const baseUrl = getBaseUrl()
fileUrl = `${baseUrl}${validatedData.filePath}`
} }
const reductoBody: Record<string, unknown> = { const reductoBody: Record<string, unknown> = {
@@ -104,8 +78,13 @@ export async function POST(request: NextRequest) {
} }
if (validatedData.pages && validatedData.pages.length > 0) { if (validatedData.pages && validatedData.pages.length > 0) {
// Reducto API expects page_range as an object with start/end, not an array
const pages = validatedData.pages
reductoBody.settings = { reductoBody.settings = {
page_range: validatedData.pages, page_range: {
start: Math.min(...pages),
end: Math.max(...pages),
},
} }
} }
@@ -115,15 +94,34 @@ export async function POST(request: NextRequest) {
} }
} }
const reductoResponse = await fetch('https://platform.reducto.ai/parse', { const reductoEndpoint = 'https://platform.reducto.ai/parse'
method: 'POST', const reductoValidation = await validateUrlWithDNS(reductoEndpoint, 'Reducto API URL')
headers: { if (!reductoValidation.isValid) {
'Content-Type': 'application/json', logger.error(`[${requestId}] Reducto API URL validation failed`, {
Accept: 'application/json', error: reductoValidation.error,
Authorization: `Bearer ${validatedData.apiKey}`, })
}, return NextResponse.json(
body: JSON.stringify(reductoBody), {
}) success: false,
error: 'Failed to reach Reducto API',
},
{ status: 502 }
)
}
const reductoResponse = await secureFetchWithPinnedIP(
reductoEndpoint,
reductoValidation.resolvedIP!,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
Accept: 'application/json',
Authorization: `Bearer ${validatedData.apiKey}`,
},
body: JSON.stringify(reductoBody),
}
)
if (!reductoResponse.ok) { if (!reductoResponse.ok) {
const errorText = await reductoResponse.text() const errorText = await reductoResponse.text()

View File

@@ -4,6 +4,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils' import { processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
@@ -17,7 +18,7 @@ const S3PutObjectSchema = z.object({
region: z.string().min(1, 'Region is required'), region: z.string().min(1, 'Region is required'),
bucketName: z.string().min(1, 'Bucket name is required'), bucketName: z.string().min(1, 'Bucket name is required'),
objectKey: z.string().min(1, 'Object key is required'), objectKey: z.string().min(1, 'Object key is required'),
file: z.any().optional().nullable(), file: RawFileInputSchema.optional().nullable(),
content: z.string().optional().nullable(), content: z.string().optional().nullable(),
contentType: z.string().optional().nullable(), contentType: z.string().optional().nullable(),
acl: z.string().optional().nullable(), acl: z.string().optional().nullable(),

View File

@@ -0,0 +1,188 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
export const dynamic = 'force-dynamic'
const logger = createLogger('SendGridSendMailAPI')
const SendGridSendMailSchema = z.object({
apiKey: z.string().min(1, 'API key is required'),
from: z.string().min(1, 'From email is required'),
fromName: z.string().optional().nullable(),
to: z.string().min(1, 'To email is required'),
toName: z.string().optional().nullable(),
subject: z.string().optional().nullable(),
content: z.string().optional().nullable(),
contentType: z.string().optional().nullable(),
cc: z.string().optional().nullable(),
bcc: z.string().optional().nullable(),
replyTo: z.string().optional().nullable(),
replyToName: z.string().optional().nullable(),
templateId: z.string().optional().nullable(),
dynamicTemplateData: z.any().optional().nullable(),
attachments: RawFileInputArraySchema.optional().nullable(),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized SendGrid send attempt: ${authResult.error}`)
return NextResponse.json(
{ success: false, error: authResult.error || 'Authentication required' },
{ status: 401 }
)
}
logger.info(`[${requestId}] Authenticated SendGrid send request via ${authResult.authType}`)
const body = await request.json()
const validatedData = SendGridSendMailSchema.parse(body)
logger.info(`[${requestId}] Sending SendGrid email`, {
to: validatedData.to,
subject: validatedData.subject || '(template)',
hasAttachments: !!(validatedData.attachments && validatedData.attachments.length > 0),
attachmentCount: validatedData.attachments?.length || 0,
})
// Build personalizations
const personalizations: Record<string, unknown> = {
to: [
{ email: validatedData.to, ...(validatedData.toName && { name: validatedData.toName }) },
],
}
if (validatedData.cc) {
personalizations.cc = [{ email: validatedData.cc }]
}
if (validatedData.bcc) {
personalizations.bcc = [{ email: validatedData.bcc }]
}
if (validatedData.templateId && validatedData.dynamicTemplateData) {
personalizations.dynamic_template_data =
typeof validatedData.dynamicTemplateData === 'string'
? JSON.parse(validatedData.dynamicTemplateData)
: validatedData.dynamicTemplateData
}
// Build mail body
const mailBody: Record<string, unknown> = {
personalizations: [personalizations],
from: {
email: validatedData.from,
...(validatedData.fromName && { name: validatedData.fromName }),
},
subject: validatedData.subject,
}
if (validatedData.templateId) {
mailBody.template_id = validatedData.templateId
} else {
mailBody.content = [
{
type: validatedData.contentType || 'text/plain',
value: validatedData.content,
},
]
}
if (validatedData.replyTo) {
mailBody.reply_to = {
email: validatedData.replyTo,
...(validatedData.replyToName && { name: validatedData.replyToName }),
}
}
// Process attachments from UserFile objects
if (validatedData.attachments && validatedData.attachments.length > 0) {
const rawAttachments = validatedData.attachments
logger.info(`[${requestId}] Processing ${rawAttachments.length} attachment(s)`)
const userFiles = processFilesToUserFiles(rawAttachments, requestId, logger)
if (userFiles.length > 0) {
const sendGridAttachments = await Promise.all(
userFiles.map(async (file) => {
try {
logger.info(
`[${requestId}] Downloading attachment: ${file.name} (${file.size} bytes)`
)
const buffer = await downloadFileFromStorage(file, requestId, logger)
return {
content: buffer.toString('base64'),
filename: file.name,
type: file.type || 'application/octet-stream',
disposition: 'attachment',
}
} catch (error) {
logger.error(`[${requestId}] Failed to download attachment ${file.name}:`, error)
throw new Error(
`Failed to download attachment "${file.name}": ${error instanceof Error ? error.message : 'Unknown error'}`
)
}
})
)
mailBody.attachments = sendGridAttachments
}
}
// Send to SendGrid
const response = await fetch('https://api.sendgrid.com/v3/mail/send', {
method: 'POST',
headers: {
Authorization: `Bearer ${validatedData.apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify(mailBody),
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
const errorMessage =
errorData.errors?.[0]?.message || errorData.message || 'Failed to send email'
logger.error(`[${requestId}] SendGrid API error:`, { status: response.status, errorData })
return NextResponse.json({ success: false, error: errorMessage }, { status: response.status })
}
const messageId = response.headers.get('X-Message-Id')
logger.info(`[${requestId}] Email sent successfully`, { messageId })
return NextResponse.json({
success: true,
output: {
success: true,
messageId: messageId || undefined,
to: validatedData.to,
subject: validatedData.subject || '',
},
})
} catch (error) {
if (error instanceof z.ZodError) {
logger.warn(`[${requestId}] Validation error:`, error.errors)
return NextResponse.json(
{ success: false, error: error.errors[0]?.message || 'Validation failed' },
{ status: 400 }
)
}
logger.error(`[${requestId}] Unexpected error:`, error)
return NextResponse.json(
{ success: false, error: error instanceof Error ? error.message : 'Unknown error' },
{ status: 500 }
)
}
}

View File

@@ -4,6 +4,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils'
import { createSftpConnection, getSftp, isPathSafe, sanitizePath } from '@/app/api/tools/sftp/utils' import { createSftpConnection, getSftp, isPathSafe, sanitizePath } from '@/app/api/tools/sftp/utils'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -111,6 +112,8 @@ export async function POST(request: NextRequest) {
const buffer = Buffer.concat(chunks) const buffer = Buffer.concat(chunks)
const fileName = path.basename(remotePath) const fileName = path.basename(remotePath)
const extension = getFileExtension(fileName)
const mimeType = getMimeTypeFromExtension(extension)
let content: string let content: string
if (params.encoding === 'base64') { if (params.encoding === 'base64') {
@@ -124,6 +127,12 @@ export async function POST(request: NextRequest) {
return NextResponse.json({ return NextResponse.json({
success: true, success: true,
fileName, fileName,
file: {
name: fileName,
mimeType,
data: buffer.toString('base64'),
size: buffer.length,
},
content, content,
size: buffer.length, size: buffer.length,
encoding: params.encoding, encoding: params.encoding,

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
import { import {
@@ -26,14 +27,7 @@ const UploadSchema = z.object({
privateKey: z.string().nullish(), privateKey: z.string().nullish(),
passphrase: z.string().nullish(), passphrase: z.string().nullish(),
remotePath: z.string().min(1, 'Remote path is required'), remotePath: z.string().min(1, 'Remote path is required'),
files: z files: RawFileInputArraySchema.optional().nullable(),
.union([z.array(z.any()), z.string(), z.number(), z.null(), z.undefined()])
.transform((val) => {
if (Array.isArray(val)) return val
if (val === null || val === undefined || val === '') return undefined
return undefined
})
.nullish(),
fileContent: z.string().nullish(), fileContent: z.string().nullish(),
fileName: z.string().nullish(), fileName: z.string().nullish(),
overwrite: z.boolean().default(true), overwrite: z.boolean().default(true),

View File

@@ -2,9 +2,12 @@ import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { secureFetchWithValidation } from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
import type { MicrosoftGraphDriveItem } from '@/tools/onedrive/types'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -16,7 +19,7 @@ const SharepointUploadSchema = z.object({
driveId: z.string().optional().nullable(), driveId: z.string().optional().nullable(),
folderPath: z.string().optional().nullable(), folderPath: z.string().optional().nullable(),
fileName: z.string().optional().nullable(), fileName: z.string().optional().nullable(),
files: z.array(z.any()).optional().nullable(), files: RawFileInputArraySchema.optional().nullable(),
}) })
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {
@@ -79,18 +82,23 @@ export async function POST(request: NextRequest) {
let effectiveDriveId = validatedData.driveId let effectiveDriveId = validatedData.driveId
if (!effectiveDriveId) { if (!effectiveDriveId) {
logger.info(`[${requestId}] No driveId provided, fetching default drive for site`) logger.info(`[${requestId}] No driveId provided, fetching default drive for site`)
const driveResponse = await fetch( const driveUrl = `https://graph.microsoft.com/v1.0/sites/${validatedData.siteId}/drive`
`https://graph.microsoft.com/v1.0/sites/${validatedData.siteId}/drive`, const driveResponse = await secureFetchWithValidation(
driveUrl,
{ {
method: 'GET',
headers: { headers: {
Authorization: `Bearer ${validatedData.accessToken}`, Authorization: `Bearer ${validatedData.accessToken}`,
Accept: 'application/json', Accept: 'application/json',
}, },
} },
'driveUrl'
) )
if (!driveResponse.ok) { if (!driveResponse.ok) {
const errorData = await driveResponse.json().catch(() => ({})) const errorData = (await driveResponse.json().catch(() => ({}))) as {
error?: { message?: string }
}
logger.error(`[${requestId}] Failed to get default drive:`, errorData) logger.error(`[${requestId}] Failed to get default drive:`, errorData)
return NextResponse.json( return NextResponse.json(
{ {
@@ -101,7 +109,7 @@ export async function POST(request: NextRequest) {
) )
} }
const driveData = await driveResponse.json() const driveData = (await driveResponse.json()) as { id: string }
effectiveDriveId = driveData.id effectiveDriveId = driveData.id
logger.info(`[${requestId}] Using default drive: ${effectiveDriveId}`) logger.info(`[${requestId}] Using default drive: ${effectiveDriveId}`)
} }
@@ -145,34 +153,87 @@ export async function POST(request: NextRequest) {
logger.info(`[${requestId}] Uploading to: ${uploadUrl}`) logger.info(`[${requestId}] Uploading to: ${uploadUrl}`)
const uploadResponse = await fetch(uploadUrl, { const uploadResponse = await secureFetchWithValidation(
method: 'PUT', uploadUrl,
headers: { {
Authorization: `Bearer ${validatedData.accessToken}`, method: 'PUT',
'Content-Type': userFile.type || 'application/octet-stream', headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
'Content-Type': userFile.type || 'application/octet-stream',
},
body: buffer,
}, },
body: new Uint8Array(buffer), 'uploadUrl'
}) )
if (!uploadResponse.ok) { if (!uploadResponse.ok) {
const errorData = await uploadResponse.json().catch(() => ({})) const errorData = await uploadResponse.json().catch(() => ({}))
logger.error(`[${requestId}] Failed to upload file ${fileName}:`, errorData) logger.error(`[${requestId}] Failed to upload file ${fileName}:`, errorData)
if (uploadResponse.status === 409) { if (uploadResponse.status === 409) {
logger.warn(`[${requestId}] File ${fileName} already exists, attempting to replace`) // File exists - retry with conflict behavior set to replace
logger.warn(`[${requestId}] File ${fileName} already exists, retrying with replace`)
const replaceUrl = `${uploadUrl}?@microsoft.graph.conflictBehavior=replace`
const replaceResponse = await secureFetchWithValidation(
replaceUrl,
{
method: 'PUT',
headers: {
Authorization: `Bearer ${validatedData.accessToken}`,
'Content-Type': userFile.type || 'application/octet-stream',
},
body: buffer,
},
'replaceUrl'
)
if (!replaceResponse.ok) {
const replaceErrorData = (await replaceResponse.json().catch(() => ({}))) as {
error?: { message?: string }
}
logger.error(`[${requestId}] Failed to replace file ${fileName}:`, replaceErrorData)
return NextResponse.json(
{
success: false,
error: replaceErrorData.error?.message || `Failed to replace file: ${fileName}`,
},
{ status: replaceResponse.status }
)
}
const replaceData = (await replaceResponse.json()) as {
id: string
name: string
webUrl: string
size: number
createdDateTime: string
lastModifiedDateTime: string
}
logger.info(`[${requestId}] File replaced successfully: ${fileName}`)
uploadedFiles.push({
id: replaceData.id,
name: replaceData.name,
webUrl: replaceData.webUrl,
size: replaceData.size,
createdDateTime: replaceData.createdDateTime,
lastModifiedDateTime: replaceData.lastModifiedDateTime,
})
continue continue
} }
return NextResponse.json( return NextResponse.json(
{ {
success: false, success: false,
error: errorData.error?.message || `Failed to upload file: ${fileName}`, error:
(errorData as { error?: { message?: string } }).error?.message ||
`Failed to upload file: ${fileName}`,
}, },
{ status: uploadResponse.status } { status: uploadResponse.status }
) )
} }
const uploadData = await uploadResponse.json() const uploadData = (await uploadResponse.json()) as MicrosoftGraphDriveItem
logger.info(`[${requestId}] File uploaded successfully: ${fileName}`) logger.info(`[${requestId}] File uploaded successfully: ${fileName}`)
uploadedFiles.push({ uploadedFiles.push({

View File

@@ -0,0 +1,170 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request'
export const dynamic = 'force-dynamic'
const logger = createLogger('SlackDownloadAPI')
const SlackDownloadSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'),
fileId: z.string().min(1, 'File ID is required'),
fileName: z.string().optional().nullable(),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized Slack download attempt: ${authResult.error}`)
return NextResponse.json(
{
success: false,
error: authResult.error || 'Authentication required',
},
{ status: 401 }
)
}
logger.info(`[${requestId}] Authenticated Slack download request via ${authResult.authType}`, {
userId: authResult.userId,
})
const body = await request.json()
const validatedData = SlackDownloadSchema.parse(body)
const { accessToken, fileId, fileName } = validatedData
logger.info(`[${requestId}] Getting file info from Slack`, { fileId })
const infoResponse = await fetch(`https://slack.com/api/files.info?file=${fileId}`, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
if (!infoResponse.ok) {
const errorDetails = await infoResponse.json().catch(() => ({}))
logger.error(`[${requestId}] Failed to get file info from Slack`, {
status: infoResponse.status,
statusText: infoResponse.statusText,
error: errorDetails,
})
return NextResponse.json(
{
success: false,
error: errorDetails.error || 'Failed to get file info',
},
{ status: 400 }
)
}
const data = await infoResponse.json()
if (!data.ok) {
logger.error(`[${requestId}] Slack API returned error`, { error: data.error })
return NextResponse.json(
{
success: false,
error: data.error || 'Slack API error',
},
{ status: 400 }
)
}
const file = data.file
const resolvedFileName = fileName || file.name || 'download'
const mimeType = file.mimetype || 'application/octet-stream'
const urlPrivate = file.url_private
if (!urlPrivate) {
return NextResponse.json(
{
success: false,
error: 'File does not have a download URL',
},
{ status: 400 }
)
}
const urlValidation = await validateUrlWithDNS(urlPrivate, 'urlPrivate')
if (!urlValidation.isValid) {
return NextResponse.json(
{
success: false,
error: urlValidation.error,
},
{ status: 400 }
)
}
logger.info(`[${requestId}] Downloading file from Slack`, {
fileId,
fileName: resolvedFileName,
mimeType,
})
const downloadResponse = await secureFetchWithPinnedIP(urlPrivate, urlValidation.resolvedIP!, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
if (!downloadResponse.ok) {
logger.error(`[${requestId}] Failed to download file content`, {
status: downloadResponse.status,
statusText: downloadResponse.statusText,
})
return NextResponse.json(
{
success: false,
error: 'Failed to download file content',
},
{ status: 400 }
)
}
const arrayBuffer = await downloadResponse.arrayBuffer()
const fileBuffer = Buffer.from(arrayBuffer)
logger.info(`[${requestId}] File downloaded successfully`, {
fileId,
name: resolvedFileName,
size: fileBuffer.length,
mimeType,
})
const base64Data = fileBuffer.toString('base64')
return NextResponse.json({
success: true,
output: {
file: {
name: resolvedFileName,
mimeType,
data: base64Data,
size: fileBuffer.length,
},
},
})
} catch (error) {
logger.error(`[${requestId}] Error downloading Slack file:`, error)
return NextResponse.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error occurred',
},
{ status: 500 }
)
}
}

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { sendSlackMessage } from '../utils' import { sendSlackMessage } from '../utils'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -16,7 +17,7 @@ const SlackSendMessageSchema = z
userId: z.string().optional().nullable(), userId: z.string().optional().nullable(),
text: z.string().min(1, 'Message text is required'), text: z.string().min(1, 'Message text is required'),
thread_ts: z.string().optional().nullable(), thread_ts: z.string().optional().nullable(),
files: z.array(z.any()).optional().nullable(), files: RawFileInputArraySchema.optional().nullable(),
}) })
.refine((data) => data.channel || data.userId, { .refine((data) => data.channel || data.userId, {
message: 'Either channel or userId is required', message: 'Either channel or userId is required',

View File

@@ -1,6 +1,8 @@
import type { Logger } from '@sim/logger' import type { Logger } from '@sim/logger'
import { secureFetchWithValidation } from '@/lib/core/security/input-validation.server'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
import type { ToolFileData } from '@/tools/types'
/** /**
* Sends a message to a Slack channel using chat.postMessage * Sends a message to a Slack channel using chat.postMessage
@@ -70,9 +72,10 @@ export async function uploadFilesToSlack(
accessToken: string, accessToken: string,
requestId: string, requestId: string,
logger: Logger logger: Logger
): Promise<string[]> { ): Promise<{ fileIds: string[]; files: ToolFileData[] }> {
const userFiles = processFilesToUserFiles(files, requestId, logger) const userFiles = processFilesToUserFiles(files, requestId, logger)
const uploadedFileIds: string[] = [] const uploadedFileIds: string[] = []
const uploadedFiles: ToolFileData[] = []
for (const userFile of userFiles) { for (const userFile of userFiles) {
logger.info(`[${requestId}] Uploading file: ${userFile.name}`) logger.info(`[${requestId}] Uploading file: ${userFile.name}`)
@@ -100,10 +103,14 @@ export async function uploadFilesToSlack(
logger.info(`[${requestId}] Got upload URL for ${userFile.name}, file_id: ${urlData.file_id}`) logger.info(`[${requestId}] Got upload URL for ${userFile.name}, file_id: ${urlData.file_id}`)
const uploadResponse = await fetch(urlData.upload_url, { const uploadResponse = await secureFetchWithValidation(
method: 'POST', urlData.upload_url,
body: new Uint8Array(buffer), {
}) method: 'POST',
body: buffer,
},
'uploadUrl'
)
if (!uploadResponse.ok) { if (!uploadResponse.ok) {
logger.error(`[${requestId}] Failed to upload file data: ${uploadResponse.status}`) logger.error(`[${requestId}] Failed to upload file data: ${uploadResponse.status}`)
@@ -112,9 +119,16 @@ export async function uploadFilesToSlack(
logger.info(`[${requestId}] File data uploaded successfully`) logger.info(`[${requestId}] File data uploaded successfully`)
uploadedFileIds.push(urlData.file_id) uploadedFileIds.push(urlData.file_id)
// Only add to uploadedFiles after successful upload to keep arrays in sync
uploadedFiles.push({
name: userFile.name,
mimeType: userFile.type || 'application/octet-stream',
data: buffer.toString('base64'),
size: buffer.length,
})
} }
return uploadedFileIds return { fileIds: uploadedFileIds, files: uploadedFiles }
} }
/** /**
@@ -124,7 +138,8 @@ export async function completeSlackFileUpload(
uploadedFileIds: string[], uploadedFileIds: string[],
channel: string, channel: string,
text: string, text: string,
accessToken: string accessToken: string,
threadTs?: string | null
): Promise<{ ok: boolean; files?: any[]; error?: string }> { ): Promise<{ ok: boolean; files?: any[]; error?: string }> {
const response = await fetch('https://slack.com/api/files.completeUploadExternal', { const response = await fetch('https://slack.com/api/files.completeUploadExternal', {
method: 'POST', method: 'POST',
@@ -136,6 +151,7 @@ export async function completeSlackFileUpload(
files: uploadedFileIds.map((id) => ({ id })), files: uploadedFileIds.map((id) => ({ id })),
channel_id: channel, channel_id: channel,
initial_comment: text, initial_comment: text,
...(threadTs && { thread_ts: threadTs }),
}), }),
}) })
@@ -217,7 +233,13 @@ export async function sendSlackMessage(
logger: Logger logger: Logger
): Promise<{ ): Promise<{
success: boolean success: boolean
output?: { message: any; ts: string; channel: string; fileCount?: number } output?: {
message: any
ts: string
channel: string
fileCount?: number
files?: ToolFileData[]
}
error?: string error?: string
}> { }> {
const { accessToken, text, threadTs, files } = params const { accessToken, text, threadTs, files } = params
@@ -249,10 +271,15 @@ export async function sendSlackMessage(
// Process files // Process files
logger.info(`[${requestId}] Processing ${files.length} file(s)`) logger.info(`[${requestId}] Processing ${files.length} file(s)`)
const uploadedFileIds = await uploadFilesToSlack(files, accessToken, requestId, logger) const { fileIds, files: uploadedFiles } = await uploadFilesToSlack(
files,
accessToken,
requestId,
logger
)
// No valid files uploaded - send text-only // No valid files uploaded - send text-only
if (uploadedFileIds.length === 0) { if (fileIds.length === 0) {
logger.warn(`[${requestId}] No valid files to upload, sending text-only message`) logger.warn(`[${requestId}] No valid files to upload, sending text-only message`)
const data = await postSlackMessage(accessToken, channel, text, threadTs) const data = await postSlackMessage(accessToken, channel, text, threadTs)
@@ -264,8 +291,8 @@ export async function sendSlackMessage(
return { success: true, output: formatMessageSuccessResponse(data, text) } return { success: true, output: formatMessageSuccessResponse(data, text) }
} }
// Complete file upload // Complete file upload with thread support
const completeData = await completeSlackFileUpload(uploadedFileIds, channel, text, accessToken) const completeData = await completeSlackFileUpload(fileIds, channel, text, accessToken, threadTs)
if (!completeData.ok) { if (!completeData.ok) {
logger.error(`[${requestId}] Failed to complete upload:`, completeData.error) logger.error(`[${requestId}] Failed to complete upload:`, completeData.error)
@@ -282,7 +309,8 @@ export async function sendSlackMessage(
message: fileMessage, message: fileMessage,
ts: fileMessage.ts, ts: fileMessage.ts,
channel, channel,
fileCount: uploadedFileIds.length, fileCount: fileIds.length,
files: uploadedFiles,
}, },
} }
} }

View File

@@ -4,6 +4,7 @@ import nodemailer from 'nodemailer'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
@@ -28,7 +29,7 @@ const SmtpSendSchema = z.object({
cc: z.string().optional().nullable(), cc: z.string().optional().nullable(),
bcc: z.string().optional().nullable(), bcc: z.string().optional().nullable(),
replyTo: z.string().optional().nullable(), replyTo: z.string().optional().nullable(),
attachments: z.array(z.any()).optional().nullable(), attachments: RawFileInputArraySchema.optional().nullable(),
}) })
export async function POST(request: NextRequest) { export async function POST(request: NextRequest) {

View File

@@ -5,6 +5,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import type { Client, SFTPWrapper } from 'ssh2' import type { Client, SFTPWrapper } from 'ssh2'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils'
import { createSSHConnection, sanitizePath } from '@/app/api/tools/ssh/utils' import { createSSHConnection, sanitizePath } from '@/app/api/tools/ssh/utils'
const logger = createLogger('SSHDownloadFileAPI') const logger = createLogger('SSHDownloadFileAPI')
@@ -79,6 +80,16 @@ export async function POST(request: NextRequest) {
}) })
}) })
// Check file size limit (50MB to prevent memory exhaustion)
const maxSize = 50 * 1024 * 1024
if (stats.size > maxSize) {
const sizeMB = (stats.size / (1024 * 1024)).toFixed(2)
return NextResponse.json(
{ error: `File size (${sizeMB}MB) exceeds download limit of 50MB` },
{ status: 400 }
)
}
// Read file content // Read file content
const content = await new Promise<Buffer>((resolve, reject) => { const content = await new Promise<Buffer>((resolve, reject) => {
const chunks: Buffer[] = [] const chunks: Buffer[] = []
@@ -96,6 +107,8 @@ export async function POST(request: NextRequest) {
}) })
const fileName = path.basename(remotePath) const fileName = path.basename(remotePath)
const extension = getFileExtension(fileName)
const mimeType = getMimeTypeFromExtension(extension)
// Encode content as base64 for binary safety // Encode content as base64 for binary safety
const base64Content = content.toString('base64') const base64Content = content.toString('base64')
@@ -104,6 +117,12 @@ export async function POST(request: NextRequest) {
return NextResponse.json({ return NextResponse.json({
downloaded: true, downloaded: true,
file: {
name: fileName,
mimeType,
data: base64Content,
size: stats.size,
},
content: base64Content, content: base64Content,
fileName: fileName, fileName: fileName,
remotePath: remotePath, remotePath: remotePath,

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { env } from '@/lib/core/config/env' import { env } from '@/lib/core/config/env'
import { validateUrlWithDNS } from '@/lib/core/security/input-validation.server'
import { isSensitiveKey, REDACTED_MARKER } from '@/lib/core/security/redaction' import { isSensitiveKey, REDACTED_MARKER } from '@/lib/core/security/redaction'
import { ensureZodObject, normalizeUrl } from '@/app/api/tools/stagehand/utils' import { ensureZodObject, normalizeUrl } from '@/app/api/tools/stagehand/utils'
@@ -123,6 +124,10 @@ export async function POST(request: NextRequest) {
const variablesObject = processVariables(params.variables) const variablesObject = processVariables(params.variables)
const startUrl = normalizeUrl(rawStartUrl) const startUrl = normalizeUrl(rawStartUrl)
const urlValidation = await validateUrlWithDNS(startUrl, 'startUrl')
if (!urlValidation.isValid) {
return NextResponse.json({ error: urlValidation.error }, { status: 400 })
}
logger.info('Starting Stagehand agent process', { logger.info('Starting Stagehand agent process', {
rawStartUrl, rawStartUrl,

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { env } from '@/lib/core/config/env' import { env } from '@/lib/core/config/env'
import { validateUrlWithDNS } from '@/lib/core/security/input-validation.server'
import { ensureZodObject, normalizeUrl } from '@/app/api/tools/stagehand/utils' import { ensureZodObject, normalizeUrl } from '@/app/api/tools/stagehand/utils'
const logger = createLogger('StagehandExtractAPI') const logger = createLogger('StagehandExtractAPI')
@@ -51,6 +52,10 @@ export async function POST(request: NextRequest) {
const params = validationResult.data const params = validationResult.data
const { url: rawUrl, instruction, selector, provider, apiKey, schema } = params const { url: rawUrl, instruction, selector, provider, apiKey, schema } = params
const url = normalizeUrl(rawUrl) const url = normalizeUrl(rawUrl)
const urlValidation = await validateUrlWithDNS(url, 'url')
if (!urlValidation.isValid) {
return NextResponse.json({ error: urlValidation.error }, { status: 400 })
}
logger.info('Starting Stagehand extraction process', { logger.info('Starting Stagehand extraction process', {
rawUrl, rawUrl,

View File

@@ -2,7 +2,16 @@ import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { extractAudioFromVideo, isVideoFile } from '@/lib/audio/extractor' import { extractAudioFromVideo, isVideoFile } from '@/lib/audio/extractor'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { DEFAULT_EXECUTION_TIMEOUT_MS } from '@/lib/core/execution-limits'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { getMimeTypeFromExtension, isInternalFileUrl } from '@/lib/uploads/utils/file-utils'
import {
downloadFileFromStorage,
resolveInternalFileUrl,
} from '@/lib/uploads/utils/file-utils.server'
import type { UserFile } from '@/executor/types' import type { UserFile } from '@/executor/types'
import type { TranscriptSegment } from '@/tools/stt/types' import type { TranscriptSegment } from '@/tools/stt/types'
@@ -45,6 +54,7 @@ export async function POST(request: NextRequest) {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
} }
const userId = authResult.userId
const body: SttRequestBody = await request.json() const body: SttRequestBody = await request.json()
const { const {
provider, provider,
@@ -72,13 +82,25 @@ export async function POST(request: NextRequest) {
let audioMimeType: string let audioMimeType: string
if (body.audioFile) { if (body.audioFile) {
if (Array.isArray(body.audioFile) && body.audioFile.length !== 1) {
return NextResponse.json({ error: 'audioFile must be a single file' }, { status: 400 })
}
const file = Array.isArray(body.audioFile) ? body.audioFile[0] : body.audioFile const file = Array.isArray(body.audioFile) ? body.audioFile[0] : body.audioFile
logger.info(`[${requestId}] Processing uploaded file: ${file.name}`) logger.info(`[${requestId}] Processing uploaded file: ${file.name}`)
audioBuffer = await downloadFileFromStorage(file, requestId, logger) audioBuffer = await downloadFileFromStorage(file, requestId, logger)
audioFileName = file.name audioFileName = file.name
audioMimeType = file.type // file.type may be missing if the file came from a block that doesn't preserve it
// Infer from filename extension as fallback
const ext = file.name.split('.').pop()?.toLowerCase() || ''
audioMimeType = file.type || getMimeTypeFromExtension(ext)
} else if (body.audioFileReference) { } else if (body.audioFileReference) {
if (Array.isArray(body.audioFileReference) && body.audioFileReference.length !== 1) {
return NextResponse.json(
{ error: 'audioFileReference must be a single file' },
{ status: 400 }
)
}
const file = Array.isArray(body.audioFileReference) const file = Array.isArray(body.audioFileReference)
? body.audioFileReference[0] ? body.audioFileReference[0]
: body.audioFileReference : body.audioFileReference
@@ -86,18 +108,54 @@ export async function POST(request: NextRequest) {
audioBuffer = await downloadFileFromStorage(file, requestId, logger) audioBuffer = await downloadFileFromStorage(file, requestId, logger)
audioFileName = file.name audioFileName = file.name
audioMimeType = file.type
const ext = file.name.split('.').pop()?.toLowerCase() || ''
audioMimeType = file.type || getMimeTypeFromExtension(ext)
} else if (body.audioUrl) { } else if (body.audioUrl) {
logger.info(`[${requestId}] Downloading from URL: ${body.audioUrl}`) logger.info(`[${requestId}] Downloading from URL: ${body.audioUrl}`)
const response = await fetch(body.audioUrl) let audioUrl = body.audioUrl.trim()
if (audioUrl.startsWith('/') && !isInternalFileUrl(audioUrl)) {
return NextResponse.json(
{
error: 'Invalid file path. Only uploaded files are supported for internal paths.',
},
{ status: 400 }
)
}
if (isInternalFileUrl(audioUrl)) {
if (!userId) {
return NextResponse.json(
{ error: 'Authentication required for internal file access' },
{ status: 401 }
)
}
const resolution = await resolveInternalFileUrl(audioUrl, userId, requestId, logger)
if (resolution.error) {
return NextResponse.json(
{ error: resolution.error.message },
{ status: resolution.error.status }
)
}
audioUrl = resolution.fileUrl || audioUrl
}
const urlValidation = await validateUrlWithDNS(audioUrl, 'audioUrl')
if (!urlValidation.isValid) {
return NextResponse.json({ error: urlValidation.error }, { status: 400 })
}
const response = await secureFetchWithPinnedIP(audioUrl, urlValidation.resolvedIP!, {
method: 'GET',
})
if (!response.ok) { if (!response.ok) {
throw new Error(`Failed to download audio from URL: ${response.statusText}`) throw new Error(`Failed to download audio from URL: ${response.statusText}`)
} }
const arrayBuffer = await response.arrayBuffer() const arrayBuffer = await response.arrayBuffer()
audioBuffer = Buffer.from(arrayBuffer) audioBuffer = Buffer.from(arrayBuffer)
audioFileName = body.audioUrl.split('/').pop() || 'audio_file' audioFileName = audioUrl.split('/').pop() || 'audio_file'
audioMimeType = response.headers.get('content-type') || 'audio/mpeg' audioMimeType = response.headers.get('content-type') || 'audio/mpeg'
} else { } else {
return NextResponse.json( return NextResponse.json(
@@ -149,7 +207,9 @@ export async function POST(request: NextRequest) {
translateToEnglish, translateToEnglish,
model, model,
body.prompt, body.prompt,
body.temperature body.temperature,
audioMimeType,
audioFileName
) )
transcript = result.transcript transcript = result.transcript
segments = result.segments segments = result.segments
@@ -162,7 +222,8 @@ export async function POST(request: NextRequest) {
language, language,
timestamps, timestamps,
diarization, diarization,
model model,
audioMimeType
) )
transcript = result.transcript transcript = result.transcript
segments = result.segments segments = result.segments
@@ -252,7 +313,9 @@ async function transcribeWithWhisper(
translate?: boolean, translate?: boolean,
model?: string, model?: string,
prompt?: string, prompt?: string,
temperature?: number temperature?: number,
mimeType?: string,
fileName?: string
): Promise<{ ): Promise<{
transcript: string transcript: string
segments?: TranscriptSegment[] segments?: TranscriptSegment[]
@@ -261,8 +324,11 @@ async function transcribeWithWhisper(
}> { }> {
const formData = new FormData() const formData = new FormData()
const blob = new Blob([new Uint8Array(audioBuffer)], { type: 'audio/mpeg' }) // Use actual MIME type and filename if provided
formData.append('file', blob, 'audio.mp3') const actualMimeType = mimeType || 'audio/mpeg'
const actualFileName = fileName || 'audio.mp3'
const blob = new Blob([new Uint8Array(audioBuffer)], { type: actualMimeType })
formData.append('file', blob, actualFileName)
formData.append('model', model || 'whisper-1') formData.append('model', model || 'whisper-1')
if (language && language !== 'auto') { if (language && language !== 'auto') {
@@ -279,10 +345,11 @@ async function transcribeWithWhisper(
formData.append('response_format', 'verbose_json') formData.append('response_format', 'verbose_json')
// OpenAI API uses array notation for timestamp_granularities
if (timestamps === 'word') { if (timestamps === 'word') {
formData.append('timestamp_granularities', 'word') formData.append('timestamp_granularities[]', 'word')
} else if (timestamps === 'sentence') { } else if (timestamps === 'sentence') {
formData.append('timestamp_granularities', 'segment') formData.append('timestamp_granularities[]', 'segment')
} }
const endpoint = translate ? 'translations' : 'transcriptions' const endpoint = translate ? 'translations' : 'transcriptions'
@@ -325,7 +392,8 @@ async function transcribeWithDeepgram(
language?: string, language?: string,
timestamps?: 'none' | 'sentence' | 'word', timestamps?: 'none' | 'sentence' | 'word',
diarization?: boolean, diarization?: boolean,
model?: string model?: string,
mimeType?: string
): Promise<{ ): Promise<{
transcript: string transcript: string
segments?: TranscriptSegment[] segments?: TranscriptSegment[]
@@ -357,7 +425,7 @@ async function transcribeWithDeepgram(
method: 'POST', method: 'POST',
headers: { headers: {
Authorization: `Token ${apiKey}`, Authorization: `Token ${apiKey}`,
'Content-Type': 'audio/mpeg', 'Content-Type': mimeType || 'audio/mpeg',
}, },
body: new Uint8Array(audioBuffer), body: new Uint8Array(audioBuffer),
}) })
@@ -513,7 +581,8 @@ async function transcribeWithAssemblyAI(
audio_url: upload_url, audio_url: upload_url,
} }
if (model === 'best' || model === 'nano') { // AssemblyAI supports 'best', 'slam-1', or 'universal' for speech_model
if (model === 'best' || model === 'slam-1' || model === 'universal') {
transcriptRequest.speech_model = model transcriptRequest.speech_model = model
} }
@@ -568,7 +637,8 @@ async function transcribeWithAssemblyAI(
let transcript: any let transcript: any
let attempts = 0 let attempts = 0
const maxAttempts = 60 // 5 minutes with 5-second intervals const pollIntervalMs = 5000
const maxAttempts = Math.ceil(DEFAULT_EXECUTION_TIMEOUT_MS / pollIntervalMs)
while (attempts < maxAttempts) { while (attempts < maxAttempts) {
const statusResponse = await fetch(`https://api.assemblyai.com/v2/transcript/${id}`, { const statusResponse = await fetch(`https://api.assemblyai.com/v2/transcript/${id}`, {

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { FileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils' import { processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
@@ -16,7 +17,7 @@ const SupabaseStorageUploadSchema = z.object({
bucket: z.string().min(1, 'Bucket name is required'), bucket: z.string().min(1, 'Bucket name is required'),
fileName: z.string().min(1, 'File name is required'), fileName: z.string().min(1, 'File name is required'),
path: z.string().optional().nullable(), path: z.string().optional().nullable(),
fileData: z.any(), fileData: FileInputSchema,
contentType: z.string().optional().nullable(), contentType: z.string().optional().nullable(),
upsert: z.boolean().optional().default(false), upsert: z.boolean().optional().default(false),
}) })

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputArraySchema } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
import { convertMarkdownToHTML } from '@/tools/telegram/utils' import { convertMarkdownToHTML } from '@/tools/telegram/utils'
@@ -14,7 +15,7 @@ const logger = createLogger('TelegramSendDocumentAPI')
const TelegramSendDocumentSchema = z.object({ const TelegramSendDocumentSchema = z.object({
botToken: z.string().min(1, 'Bot token is required'), botToken: z.string().min(1, 'Bot token is required'),
chatId: z.string().min(1, 'Chat ID is required'), chatId: z.string().min(1, 'Chat ID is required'),
files: z.array(z.any()).optional().nullable(), files: RawFileInputArraySchema.optional().nullable(),
caption: z.string().optional().nullable(), caption: z.string().optional().nullable(),
}) })
@@ -93,6 +94,14 @@ export async function POST(request: NextRequest) {
logger.info(`[${requestId}] Uploading document: ${userFile.name}`) logger.info(`[${requestId}] Uploading document: ${userFile.name}`)
const buffer = await downloadFileFromStorage(userFile, requestId, logger) const buffer = await downloadFileFromStorage(userFile, requestId, logger)
const filesOutput = [
{
name: userFile.name,
mimeType: userFile.type || 'application/octet-stream',
data: buffer.toString('base64'),
size: buffer.length,
},
]
logger.info(`[${requestId}] Downloaded file: ${buffer.length} bytes`) logger.info(`[${requestId}] Downloaded file: ${buffer.length} bytes`)
@@ -135,6 +144,7 @@ export async function POST(request: NextRequest) {
output: { output: {
message: 'Document sent successfully', message: 'Document sent successfully',
data: data.result, data: data.result,
files: filesOutput,
}, },
}) })
} catch (error) { } catch (error) {

View File

@@ -3,19 +3,19 @@ import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { DEFAULT_EXECUTION_TIMEOUT_MS } from '@/lib/core/execution-limits'
import { validateAwsRegion, validateS3BucketName } from '@/lib/core/security/input-validation'
import { import {
validateAwsRegion, secureFetchWithPinnedIP,
validateExternalUrl, validateUrlWithDNS,
validateS3BucketName, } from '@/lib/core/security/input-validation.server'
} from '@/lib/core/security/input-validation'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { StorageService } from '@/lib/uploads' import { RawFileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { isInternalFileUrl, processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils'
import { import {
extractStorageKey, downloadFileFromStorage,
inferContextFromKey, resolveInternalFileUrl,
isInternalFileUrl, } from '@/lib/uploads/utils/file-utils.server'
} from '@/lib/uploads/utils/file-utils'
import { verifyFileAccess } from '@/app/api/files/authorization'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
export const maxDuration = 300 // 5 minutes for large multi-page PDF processing export const maxDuration = 300 // 5 minutes for large multi-page PDF processing
@@ -35,6 +35,7 @@ const TextractParseSchema = z
region: z.string().min(1, 'AWS region is required'), region: z.string().min(1, 'AWS region is required'),
processingMode: z.enum(['sync', 'async']).optional().default('sync'), processingMode: z.enum(['sync', 'async']).optional().default('sync'),
filePath: z.string().optional(), filePath: z.string().optional(),
file: RawFileInputSchema.optional(),
s3Uri: z.string().optional(), s3Uri: z.string().optional(),
featureTypes: z featureTypes: z
.array(z.enum(['TABLES', 'FORMS', 'QUERIES', 'SIGNATURES', 'LAYOUT'])) .array(z.enum(['TABLES', 'FORMS', 'QUERIES', 'SIGNATURES', 'LAYOUT']))
@@ -50,6 +51,20 @@ const TextractParseSchema = z
path: ['region'], path: ['region'],
}) })
} }
if (data.processingMode === 'async' && !data.s3Uri) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: 'S3 URI is required for multi-page processing (s3://bucket/key)',
path: ['s3Uri'],
})
}
if (data.processingMode !== 'async' && !data.file && !data.filePath) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: 'File input is required for single-page processing',
path: ['filePath'],
})
}
}) })
function getSignatureKey( function getSignatureKey(
@@ -111,7 +126,14 @@ function signAwsRequest(
} }
async function fetchDocumentBytes(url: string): Promise<{ bytes: string; contentType: string }> { async function fetchDocumentBytes(url: string): Promise<{ bytes: string; contentType: string }> {
const response = await fetch(url) const urlValidation = await validateUrlWithDNS(url, 'Document URL')
if (!urlValidation.isValid) {
throw new Error(urlValidation.error || 'Invalid document URL')
}
const response = await secureFetchWithPinnedIP(url, urlValidation.resolvedIP!, {
method: 'GET',
})
if (!response.ok) { if (!response.ok) {
throw new Error(`Failed to fetch document: ${response.statusText}`) throw new Error(`Failed to fetch document: ${response.statusText}`)
} }
@@ -205,8 +227,8 @@ async function pollForJobCompletion(
useAnalyzeDocument: boolean, useAnalyzeDocument: boolean,
requestId: string requestId: string
): Promise<Record<string, unknown>> { ): Promise<Record<string, unknown>> {
const pollIntervalMs = 5000 // 5 seconds between polls const pollIntervalMs = 5000
const maxPollTimeMs = 180000 // 3 minutes maximum polling time const maxPollTimeMs = DEFAULT_EXECUTION_TIMEOUT_MS
const maxAttempts = Math.ceil(maxPollTimeMs / pollIntervalMs) const maxAttempts = Math.ceil(maxPollTimeMs / pollIntervalMs)
const getTarget = useAnalyzeDocument const getTarget = useAnalyzeDocument
@@ -318,8 +340,8 @@ export async function POST(request: NextRequest) {
logger.info(`[${requestId}] Textract parse request`, { logger.info(`[${requestId}] Textract parse request`, {
processingMode, processingMode,
filePath: validatedData.filePath?.substring(0, 50), hasFile: Boolean(validatedData.file),
s3Uri: validatedData.s3Uri?.substring(0, 50), hasS3Uri: Boolean(validatedData.s3Uri),
featureTypes, featureTypes,
userId, userId,
}) })
@@ -414,90 +436,89 @@ export async function POST(request: NextRequest) {
}) })
} }
if (!validatedData.filePath) { let bytes = ''
return NextResponse.json( let contentType = 'application/octet-stream'
{ let isPdf = false
success: false,
error: 'File path is required for single-page processing',
},
{ status: 400 }
)
}
let fileUrl = validatedData.filePath if (validatedData.file) {
let userFile
const isInternalFilePath = validatedData.filePath && isInternalFileUrl(validatedData.filePath)
if (isInternalFilePath) {
try { try {
const storageKey = extractStorageKey(validatedData.filePath) userFile = processSingleFileToUserFile(validatedData.file, requestId, logger)
const context = inferContextFromKey(storageKey)
const hasAccess = await verifyFileAccess(storageKey, userId, undefined, context, false)
if (!hasAccess) {
logger.warn(`[${requestId}] Unauthorized presigned URL generation attempt`, {
userId,
key: storageKey,
context,
})
return NextResponse.json(
{
success: false,
error: 'File not found',
},
{ status: 404 }
)
}
fileUrl = await StorageService.generatePresignedDownloadUrl(storageKey, context, 5 * 60)
logger.info(`[${requestId}] Generated presigned URL for ${context} file`)
} catch (error) { } catch (error) {
logger.error(`[${requestId}] Failed to generate presigned URL:`, error)
return NextResponse.json( return NextResponse.json(
{ {
success: false, success: false,
error: 'Failed to generate file access URL', error: error instanceof Error ? error.message : 'Failed to process file',
},
{ status: 500 }
)
}
} else if (validatedData.filePath?.startsWith('/')) {
// Reject arbitrary absolute paths that don't contain /api/files/serve/
logger.warn(`[${requestId}] Invalid internal path`, {
userId,
path: validatedData.filePath.substring(0, 50),
})
return NextResponse.json(
{
success: false,
error: 'Invalid file path. Only uploaded files are supported for internal paths.',
},
{ status: 400 }
)
} else {
const urlValidation = validateExternalUrl(fileUrl, 'Document URL')
if (!urlValidation.isValid) {
logger.warn(`[${requestId}] SSRF attempt blocked`, {
userId,
url: fileUrl.substring(0, 100),
error: urlValidation.error,
})
return NextResponse.json(
{
success: false,
error: urlValidation.error,
}, },
{ status: 400 } { status: 400 }
) )
} }
const buffer = await downloadFileFromStorage(userFile, requestId, logger)
bytes = buffer.toString('base64')
contentType = userFile.type || 'application/octet-stream'
isPdf = contentType.includes('pdf') || userFile.name?.toLowerCase().endsWith('.pdf')
} else if (validatedData.filePath) {
let fileUrl = validatedData.filePath
const isInternalFilePath = isInternalFileUrl(fileUrl)
if (isInternalFilePath) {
const resolution = await resolveInternalFileUrl(fileUrl, userId, requestId, logger)
if (resolution.error) {
return NextResponse.json(
{
success: false,
error: resolution.error.message,
},
{ status: resolution.error.status }
)
}
fileUrl = resolution.fileUrl || fileUrl
} else if (fileUrl.startsWith('/')) {
logger.warn(`[${requestId}] Invalid internal path`, {
userId,
path: fileUrl.substring(0, 50),
})
return NextResponse.json(
{
success: false,
error: 'Invalid file path. Only uploaded files are supported for internal paths.',
},
{ status: 400 }
)
} else {
const urlValidation = await validateUrlWithDNS(fileUrl, 'Document URL')
if (!urlValidation.isValid) {
logger.warn(`[${requestId}] SSRF attempt blocked`, {
userId,
url: fileUrl.substring(0, 100),
error: urlValidation.error,
})
return NextResponse.json(
{
success: false,
error: urlValidation.error,
},
{ status: 400 }
)
}
}
const fetched = await fetchDocumentBytes(fileUrl)
bytes = fetched.bytes
contentType = fetched.contentType
isPdf = contentType.includes('pdf') || fileUrl.toLowerCase().endsWith('.pdf')
} else {
return NextResponse.json(
{
success: false,
error: 'File input is required for single-page processing',
},
{ status: 400 }
)
} }
const { bytes, contentType } = await fetchDocumentBytes(fileUrl)
// Track if this is a PDF for better error messaging
const isPdf = contentType.includes('pdf') || fileUrl.toLowerCase().endsWith('.pdf')
const uri = '/' const uri = '/'
let textractBody: Record<string, unknown> let textractBody: Record<string, unknown>

View File

@@ -2,6 +2,7 @@ import { createLogger } from '@sim/logger'
import type { NextRequest } from 'next/server' import type { NextRequest } from 'next/server'
import { NextResponse } from 'next/server' import { NextResponse } from 'next/server'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { DEFAULT_EXECUTION_TIMEOUT_MS } from '@/lib/core/execution-limits'
import { validateAlphanumericId } from '@/lib/core/security/input-validation' import { validateAlphanumericId } from '@/lib/core/security/input-validation'
import { getBaseUrl } from '@/lib/core/utils/urls' import { getBaseUrl } from '@/lib/core/utils/urls'
import { StorageService } from '@/lib/uploads' import { StorageService } from '@/lib/uploads'
@@ -60,7 +61,7 @@ export async function POST(request: NextRequest) {
text, text,
model_id: modelId, model_id: modelId,
}), }),
signal: AbortSignal.timeout(60000), signal: AbortSignal.timeout(DEFAULT_EXECUTION_TIMEOUT_MS),
}) })
if (!response.ok) { if (!response.ok) {

View File

@@ -0,0 +1,250 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request'
import { getExtensionFromMimeType } from '@/lib/uploads/utils/file-utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('TwilioGetRecordingAPI')
interface TwilioRecordingResponse {
sid?: string
call_sid?: string
duration?: string
status?: string
channels?: number
source?: string
price?: string
price_unit?: string
uri?: string
error_code?: number
message?: string
error_message?: string
}
interface TwilioErrorResponse {
message?: string
}
interface TwilioTranscription {
transcription_text?: string
status?: string
price?: string
price_unit?: string
}
interface TwilioTranscriptionsResponse {
transcriptions?: TwilioTranscription[]
}
const TwilioGetRecordingSchema = z.object({
accountSid: z.string().min(1, 'Account SID is required'),
authToken: z.string().min(1, 'Auth token is required'),
recordingSid: z.string().min(1, 'Recording SID is required'),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized Twilio get recording attempt: ${authResult.error}`)
return NextResponse.json(
{
success: false,
error: authResult.error || 'Authentication required',
},
{ status: 401 }
)
}
const body = await request.json()
const validatedData = TwilioGetRecordingSchema.parse(body)
const { accountSid, authToken, recordingSid } = validatedData
if (!accountSid.startsWith('AC')) {
return NextResponse.json(
{
success: false,
error: `Invalid Account SID format. Account SID must start with "AC" (you provided: ${accountSid.substring(0, 2)}...)`,
},
{ status: 400 }
)
}
const twilioAuth = Buffer.from(`${accountSid}:${authToken}`).toString('base64')
logger.info(`[${requestId}] Getting recording info from Twilio`, { recordingSid })
const infoUrl = `https://api.twilio.com/2010-04-01/Accounts/${accountSid}/Recordings/${recordingSid}.json`
const infoUrlValidation = await validateUrlWithDNS(infoUrl, 'infoUrl')
if (!infoUrlValidation.isValid) {
return NextResponse.json({ success: false, error: infoUrlValidation.error }, { status: 400 })
}
const infoResponse = await secureFetchWithPinnedIP(infoUrl, infoUrlValidation.resolvedIP!, {
method: 'GET',
headers: { Authorization: `Basic ${twilioAuth}` },
})
if (!infoResponse.ok) {
const errorData = (await infoResponse.json().catch(() => ({}))) as TwilioErrorResponse
logger.error(`[${requestId}] Twilio API error`, {
status: infoResponse.status,
error: errorData,
})
return NextResponse.json(
{ success: false, error: errorData.message || `Twilio API error: ${infoResponse.status}` },
{ status: 400 }
)
}
const data = (await infoResponse.json()) as TwilioRecordingResponse
if (data.error_code) {
return NextResponse.json({
success: false,
output: {
success: false,
error: data.message || data.error_message || 'Failed to retrieve recording',
},
error: data.message || data.error_message || 'Failed to retrieve recording',
})
}
const baseUrl = 'https://api.twilio.com'
const mediaUrl = data.uri ? `${baseUrl}${data.uri.replace('.json', '')}` : undefined
let transcriptionText: string | undefined
let transcriptionStatus: string | undefined
let transcriptionPrice: string | undefined
let transcriptionPriceUnit: string | undefined
let file:
| {
name: string
mimeType: string
data: string
size: number
}
| undefined
try {
const transcriptionUrl = `https://api.twilio.com/2010-04-01/Accounts/${accountSid}/Transcriptions.json?RecordingSid=${data.sid}`
logger.info(`[${requestId}] Checking for transcriptions`)
const transcriptionUrlValidation = await validateUrlWithDNS(
transcriptionUrl,
'transcriptionUrl'
)
if (transcriptionUrlValidation.isValid) {
const transcriptionResponse = await secureFetchWithPinnedIP(
transcriptionUrl,
transcriptionUrlValidation.resolvedIP!,
{
method: 'GET',
headers: { Authorization: `Basic ${twilioAuth}` },
}
)
if (transcriptionResponse.ok) {
const transcriptionData =
(await transcriptionResponse.json()) as TwilioTranscriptionsResponse
if (transcriptionData.transcriptions && transcriptionData.transcriptions.length > 0) {
const transcription = transcriptionData.transcriptions[0]
transcriptionText = transcription.transcription_text
transcriptionStatus = transcription.status
transcriptionPrice = transcription.price
transcriptionPriceUnit = transcription.price_unit
logger.info(`[${requestId}] Transcription found`, {
status: transcriptionStatus,
textLength: transcriptionText?.length,
})
}
}
}
} catch (error) {
logger.warn(`[${requestId}] Failed to fetch transcription:`, error)
}
if (mediaUrl) {
try {
const mediaUrlValidation = await validateUrlWithDNS(mediaUrl, 'mediaUrl')
if (mediaUrlValidation.isValid) {
const mediaResponse = await secureFetchWithPinnedIP(
mediaUrl,
mediaUrlValidation.resolvedIP!,
{
method: 'GET',
headers: { Authorization: `Basic ${twilioAuth}` },
}
)
if (mediaResponse.ok) {
const contentType =
mediaResponse.headers.get('content-type') || 'application/octet-stream'
const extension = getExtensionFromMimeType(contentType) || 'dat'
const arrayBuffer = await mediaResponse.arrayBuffer()
const buffer = Buffer.from(arrayBuffer)
const fileName = `${data.sid || recordingSid}.${extension}`
file = {
name: fileName,
mimeType: contentType,
data: buffer.toString('base64'),
size: buffer.length,
}
}
}
} catch (error) {
logger.warn(`[${requestId}] Failed to download recording media:`, error)
}
}
logger.info(`[${requestId}] Twilio recording fetched successfully`, {
recordingSid: data.sid,
hasFile: !!file,
hasTranscription: !!transcriptionText,
})
return NextResponse.json({
success: true,
output: {
success: true,
recordingSid: data.sid,
callSid: data.call_sid,
duration: data.duration ? Number.parseInt(data.duration, 10) : undefined,
status: data.status,
channels: data.channels,
source: data.source,
mediaUrl,
file,
price: data.price,
priceUnit: data.price_unit,
uri: data.uri,
transcriptionText,
transcriptionStatus,
transcriptionPrice,
transcriptionPriceUnit,
},
})
} catch (error) {
logger.error(`[${requestId}] Error fetching Twilio recording:`, error)
return NextResponse.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error occurred',
},
{ status: 500 }
)
}
}

View File

@@ -1,6 +1,7 @@
import { createLogger } from '@sim/logger' import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { getMaxExecutionTimeout } from '@/lib/core/execution-limits'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
import type { UserFile } from '@/executor/types' import type { UserFile } from '@/executor/types'
import type { VideoRequestBody } from '@/tools/video/types' import type { VideoRequestBody } from '@/tools/video/types'
@@ -326,11 +327,12 @@ async function generateWithRunway(
logger.info(`[${requestId}] Runway task created: ${taskId}`) logger.info(`[${requestId}] Runway task created: ${taskId}`)
const maxAttempts = 120 // 10 minutes with 5-second intervals const pollIntervalMs = 5000
const maxAttempts = Math.ceil(getMaxExecutionTimeout() / pollIntervalMs)
let attempts = 0 let attempts = 0
while (attempts < maxAttempts) { while (attempts < maxAttempts) {
await sleep(5000) // Poll every 5 seconds await sleep(pollIntervalMs)
const statusResponse = await fetch(`https://api.dev.runwayml.com/v1/tasks/${taskId}`, { const statusResponse = await fetch(`https://api.dev.runwayml.com/v1/tasks/${taskId}`, {
headers: { headers: {
@@ -370,7 +372,7 @@ async function generateWithRunway(
attempts++ attempts++
} }
throw new Error('Runway generation timed out after 10 minutes') throw new Error('Runway generation timed out')
} }
async function generateWithVeo( async function generateWithVeo(
@@ -429,11 +431,12 @@ async function generateWithVeo(
logger.info(`[${requestId}] Veo operation created: ${operationName}`) logger.info(`[${requestId}] Veo operation created: ${operationName}`)
const maxAttempts = 60 // 5 minutes with 5-second intervals const pollIntervalMs = 5000
const maxAttempts = Math.ceil(getMaxExecutionTimeout() / pollIntervalMs)
let attempts = 0 let attempts = 0
while (attempts < maxAttempts) { while (attempts < maxAttempts) {
await sleep(5000) await sleep(pollIntervalMs)
const statusResponse = await fetch( const statusResponse = await fetch(
`https://generativelanguage.googleapis.com/v1beta/${operationName}`, `https://generativelanguage.googleapis.com/v1beta/${operationName}`,
@@ -485,7 +488,7 @@ async function generateWithVeo(
attempts++ attempts++
} }
throw new Error('Veo generation timed out after 5 minutes') throw new Error('Veo generation timed out')
} }
async function generateWithLuma( async function generateWithLuma(
@@ -541,11 +544,12 @@ async function generateWithLuma(
logger.info(`[${requestId}] Luma generation created: ${generationId}`) logger.info(`[${requestId}] Luma generation created: ${generationId}`)
const maxAttempts = 120 // 10 minutes const pollIntervalMs = 5000
const maxAttempts = Math.ceil(getMaxExecutionTimeout() / pollIntervalMs)
let attempts = 0 let attempts = 0
while (attempts < maxAttempts) { while (attempts < maxAttempts) {
await sleep(5000) await sleep(pollIntervalMs)
const statusResponse = await fetch( const statusResponse = await fetch(
`https://api.lumalabs.ai/dream-machine/v1/generations/${generationId}`, `https://api.lumalabs.ai/dream-machine/v1/generations/${generationId}`,
@@ -592,7 +596,7 @@ async function generateWithLuma(
attempts++ attempts++
} }
throw new Error('Luma generation timed out after 10 minutes') throw new Error('Luma generation timed out')
} }
async function generateWithMiniMax( async function generateWithMiniMax(
@@ -658,14 +662,13 @@ async function generateWithMiniMax(
logger.info(`[${requestId}] MiniMax task created: ${taskId}`) logger.info(`[${requestId}] MiniMax task created: ${taskId}`)
// Poll for completion (6-10 minutes typical) const pollIntervalMs = 5000
const maxAttempts = 120 // 10 minutes with 5-second intervals const maxAttempts = Math.ceil(getMaxExecutionTimeout() / pollIntervalMs)
let attempts = 0 let attempts = 0
while (attempts < maxAttempts) { while (attempts < maxAttempts) {
await sleep(5000) await sleep(pollIntervalMs)
// Query task status
const statusResponse = await fetch( const statusResponse = await fetch(
`https://api.minimax.io/v1/query/video_generation?task_id=${taskId}`, `https://api.minimax.io/v1/query/video_generation?task_id=${taskId}`,
{ {
@@ -743,7 +746,7 @@ async function generateWithMiniMax(
attempts++ attempts++
} }
throw new Error('MiniMax generation timed out after 10 minutes') throw new Error('MiniMax generation timed out')
} }
// Helper function to strip subpaths from Fal.ai model IDs for status/result endpoints // Helper function to strip subpaths from Fal.ai model IDs for status/result endpoints
@@ -861,11 +864,12 @@ async function generateWithFalAI(
// Get base model ID (without subpath) for status and result endpoints // Get base model ID (without subpath) for status and result endpoints
const baseModelId = getBaseModelId(falModelId) const baseModelId = getBaseModelId(falModelId)
const maxAttempts = 96 // 8 minutes with 5-second intervals const pollIntervalMs = 5000
const maxAttempts = Math.ceil(getMaxExecutionTimeout() / pollIntervalMs)
let attempts = 0 let attempts = 0
while (attempts < maxAttempts) { while (attempts < maxAttempts) {
await sleep(5000) await sleep(pollIntervalMs)
const statusResponse = await fetch( const statusResponse = await fetch(
`https://queue.fal.run/${baseModelId}/requests/${requestIdFal}/status`, `https://queue.fal.run/${baseModelId}/requests/${requestIdFal}/status`,
@@ -938,7 +942,7 @@ async function generateWithFalAI(
attempts++ attempts++
} }
throw new Error('Fal.ai generation timed out after 8 minutes') throw new Error('Fal.ai generation timed out')
} }
function getVideoDimensions( function getVideoDimensions(

View File

@@ -1,10 +1,20 @@
import { GoogleGenAI } from '@google/genai'
import { createLogger } from '@sim/logger' import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils' import { RawFileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' import { isInternalFileUrl, processSingleFileToUserFile } from '@/lib/uploads/utils/file-utils'
import {
downloadFileFromStorage,
resolveInternalFileUrl,
} from '@/lib/uploads/utils/file-utils.server'
import { convertUsageMetadata, extractTextContent } from '@/providers/google/utils'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -13,8 +23,8 @@ const logger = createLogger('VisionAnalyzeAPI')
const VisionAnalyzeSchema = z.object({ const VisionAnalyzeSchema = z.object({
apiKey: z.string().min(1, 'API key is required'), apiKey: z.string().min(1, 'API key is required'),
imageUrl: z.string().optional().nullable(), imageUrl: z.string().optional().nullable(),
imageFile: z.any().optional().nullable(), imageFile: RawFileInputSchema.optional().nullable(),
model: z.string().optional().default('gpt-4o'), model: z.string().optional().default('gpt-5.2'),
prompt: z.string().optional().nullable(), prompt: z.string().optional().nullable(),
}) })
@@ -39,6 +49,7 @@ export async function POST(request: NextRequest) {
userId: authResult.userId, userId: authResult.userId,
}) })
const userId = authResult.userId
const body = await request.json() const body = await request.json()
const validatedData = VisionAnalyzeSchema.parse(body) const validatedData = VisionAnalyzeSchema.parse(body)
@@ -77,18 +88,72 @@ export async function POST(request: NextRequest) {
) )
} }
const buffer = await downloadFileFromStorage(userFile, requestId, logger) let base64 = userFile.base64
let bufferLength = 0
const base64 = buffer.toString('base64') if (!base64) {
const buffer = await downloadFileFromStorage(userFile, requestId, logger)
base64 = buffer.toString('base64')
bufferLength = buffer.length
}
const mimeType = userFile.type || 'image/jpeg' const mimeType = userFile.type || 'image/jpeg'
imageSource = `data:${mimeType};base64,${base64}` imageSource = `data:${mimeType};base64,${base64}`
logger.info(`[${requestId}] Converted image to base64 (${buffer.length} bytes)`) if (bufferLength > 0) {
logger.info(`[${requestId}] Converted image to base64 (${bufferLength} bytes)`)
}
}
let imageUrlValidation: Awaited<ReturnType<typeof validateUrlWithDNS>> | null = null
if (imageSource && !imageSource.startsWith('data:')) {
if (imageSource.startsWith('/') && !isInternalFileUrl(imageSource)) {
return NextResponse.json(
{
success: false,
error: 'Invalid file path. Only uploaded files are supported for internal paths.',
},
{ status: 400 }
)
}
if (isInternalFileUrl(imageSource)) {
if (!userId) {
return NextResponse.json(
{
success: false,
error: 'Authentication required for internal file access',
},
{ status: 401 }
)
}
const resolution = await resolveInternalFileUrl(imageSource, userId, requestId, logger)
if (resolution.error) {
return NextResponse.json(
{
success: false,
error: resolution.error.message,
},
{ status: resolution.error.status }
)
}
imageSource = resolution.fileUrl || imageSource
}
imageUrlValidation = await validateUrlWithDNS(imageSource, 'imageUrl')
if (!imageUrlValidation.isValid) {
return NextResponse.json(
{
success: false,
error: imageUrlValidation.error,
},
{ status: 400 }
)
}
} }
const defaultPrompt = 'Please analyze this image and describe what you see in detail.' const defaultPrompt = 'Please analyze this image and describe what you see in detail.'
const prompt = validatedData.prompt || defaultPrompt const prompt = validatedData.prompt || defaultPrompt
const isClaude = validatedData.model.startsWith('claude-3') const isClaude = validatedData.model.startsWith('claude-')
const isGemini = validatedData.model.startsWith('gemini-')
const apiUrl = isClaude const apiUrl = isClaude
? 'https://api.anthropic.com/v1/messages' ? 'https://api.anthropic.com/v1/messages'
: 'https://api.openai.com/v1/chat/completions' : 'https://api.openai.com/v1/chat/completions'
@@ -106,6 +171,72 @@ export async function POST(request: NextRequest) {
let requestBody: any let requestBody: any
if (isGemini) {
let base64Payload = imageSource
if (!base64Payload.startsWith('data:')) {
const urlValidation =
imageUrlValidation || (await validateUrlWithDNS(base64Payload, 'imageUrl'))
if (!urlValidation.isValid) {
return NextResponse.json({ success: false, error: urlValidation.error }, { status: 400 })
}
const response = await secureFetchWithPinnedIP(base64Payload, urlValidation.resolvedIP!, {
method: 'GET',
})
if (!response.ok) {
return NextResponse.json(
{ success: false, error: 'Failed to fetch image for Gemini' },
{ status: 400 }
)
}
const contentType =
response.headers.get('content-type') || validatedData.imageFile?.type || 'image/jpeg'
const arrayBuffer = await response.arrayBuffer()
const base64 = Buffer.from(arrayBuffer).toString('base64')
base64Payload = `data:${contentType};base64,${base64}`
}
const base64Marker = ';base64,'
const markerIndex = base64Payload.indexOf(base64Marker)
if (!base64Payload.startsWith('data:') || markerIndex === -1) {
return NextResponse.json(
{ success: false, error: 'Invalid base64 image format' },
{ status: 400 }
)
}
const rawMimeType = base64Payload.slice('data:'.length, markerIndex)
const mediaType = rawMimeType.split(';')[0] || 'image/jpeg'
const base64Data = base64Payload.slice(markerIndex + base64Marker.length)
if (!base64Data) {
return NextResponse.json(
{ success: false, error: 'Invalid base64 image format' },
{ status: 400 }
)
}
const ai = new GoogleGenAI({ apiKey: validatedData.apiKey })
const geminiResponse = await ai.models.generateContent({
model: validatedData.model,
contents: [
{
role: 'user',
parts: [{ text: prompt }, { inlineData: { mimeType: mediaType, data: base64Data } }],
},
],
})
const content = extractTextContent(geminiResponse.candidates?.[0])
const usage = convertUsageMetadata(geminiResponse.usageMetadata)
return NextResponse.json({
success: true,
output: {
content,
model: validatedData.model,
tokens: usage.totalTokenCount || undefined,
},
})
}
if (isClaude) { if (isClaude) {
if (imageSource.startsWith('data:')) { if (imageSource.startsWith('data:')) {
const base64Match = imageSource.match(/^data:([^;]+);base64,(.+)$/) const base64Match = imageSource.match(/^data:([^;]+);base64,(.+)$/)
@@ -172,7 +303,7 @@ export async function POST(request: NextRequest) {
], ],
}, },
], ],
max_tokens: 1000, max_completion_tokens: 1000,
} }
} }

View File

@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid' import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { RawFileInputSchema } from '@/lib/uploads/utils/file-schemas'
import { import {
getFileExtension, getFileExtension,
getMimeTypeFromExtension, getMimeTypeFromExtension,
@@ -19,7 +20,7 @@ const WORDPRESS_COM_API_BASE = 'https://public-api.wordpress.com/wp/v2/sites'
const WordPressUploadSchema = z.object({ const WordPressUploadSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'), accessToken: z.string().min(1, 'Access token is required'),
siteId: z.string().min(1, 'Site ID is required'), siteId: z.string().min(1, 'Site ID is required'),
file: z.any().optional().nullable(), file: RawFileInputSchema.optional().nullable(),
filename: z.string().optional().nullable(), filename: z.string().optional().nullable(),
title: z.string().optional().nullable(), title: z.string().optional().nullable(),
caption: z.string().optional().nullable(), caption: z.string().optional().nullable(),

View File

@@ -0,0 +1,216 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import {
secureFetchWithPinnedIP,
validateUrlWithDNS,
} from '@/lib/core/security/input-validation.server'
import { generateRequestId } from '@/lib/core/utils/request'
import { getExtensionFromMimeType } from '@/lib/uploads/utils/file-utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('ZoomGetRecordingsAPI')
interface ZoomRecordingFile {
id?: string
meeting_id?: string
recording_start?: string
recording_end?: string
file_type?: string
file_extension?: string
file_size?: number
play_url?: string
download_url?: string
status?: string
recording_type?: string
}
interface ZoomRecordingsResponse {
uuid?: string
id?: string | number
account_id?: string
host_id?: string
topic?: string
type?: number
start_time?: string
duration?: number
total_size?: number
recording_count?: number
share_url?: string
recording_files?: ZoomRecordingFile[]
}
interface ZoomErrorResponse {
message?: string
code?: number
}
const ZoomGetRecordingsSchema = z.object({
accessToken: z.string().min(1, 'Access token is required'),
meetingId: z.string().min(1, 'Meeting ID is required'),
includeFolderItems: z.boolean().optional(),
ttl: z.number().optional(),
downloadFiles: z.boolean().optional().default(false),
})
export async function POST(request: NextRequest) {
const requestId = generateRequestId()
try {
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
logger.warn(`[${requestId}] Unauthorized Zoom get recordings attempt: ${authResult.error}`)
return NextResponse.json(
{
success: false,
error: authResult.error || 'Authentication required',
},
{ status: 401 }
)
}
const body = await request.json()
const validatedData = ZoomGetRecordingsSchema.parse(body)
const { accessToken, meetingId, includeFolderItems, ttl, downloadFiles } = validatedData
const baseUrl = `https://api.zoom.us/v2/meetings/${encodeURIComponent(meetingId)}/recordings`
const queryParams = new URLSearchParams()
if (includeFolderItems != null) {
queryParams.append('include_folder_items', String(includeFolderItems))
}
if (ttl) {
queryParams.append('ttl', String(ttl))
}
const queryString = queryParams.toString()
const apiUrl = queryString ? `${baseUrl}?${queryString}` : baseUrl
logger.info(`[${requestId}] Fetching recordings from Zoom`, { meetingId })
const urlValidation = await validateUrlWithDNS(apiUrl, 'apiUrl')
if (!urlValidation.isValid) {
return NextResponse.json({ success: false, error: urlValidation.error }, { status: 400 })
}
const response = await secureFetchWithPinnedIP(apiUrl, urlValidation.resolvedIP!, {
method: 'GET',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${accessToken}`,
},
})
if (!response.ok) {
const errorData = (await response.json().catch(() => ({}))) as ZoomErrorResponse
logger.error(`[${requestId}] Zoom API error`, {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ success: false, error: errorData.message || `Zoom API error: ${response.status}` },
{ status: 400 }
)
}
const data = (await response.json()) as ZoomRecordingsResponse
const files: Array<{
name: string
mimeType: string
data: string
size: number
}> = []
if (downloadFiles && Array.isArray(data.recording_files)) {
for (const file of data.recording_files) {
if (!file?.download_url) continue
try {
const fileUrlValidation = await validateUrlWithDNS(file.download_url, 'downloadUrl')
if (!fileUrlValidation.isValid) continue
const downloadResponse = await secureFetchWithPinnedIP(
file.download_url,
fileUrlValidation.resolvedIP!,
{
method: 'GET',
headers: { Authorization: `Bearer ${accessToken}` },
}
)
if (!downloadResponse.ok) continue
const contentType =
downloadResponse.headers.get('content-type') || 'application/octet-stream'
const arrayBuffer = await downloadResponse.arrayBuffer()
const buffer = Buffer.from(arrayBuffer)
const extension =
file.file_extension?.toString().toLowerCase() ||
getExtensionFromMimeType(contentType) ||
'dat'
const fileName = `zoom-recording-${file.id || file.recording_start || Date.now()}.${extension}`
files.push({
name: fileName,
mimeType: contentType,
data: buffer.toString('base64'),
size: buffer.length,
})
} catch (error) {
logger.warn(`[${requestId}] Failed to download recording file:`, error)
}
}
}
logger.info(`[${requestId}] Zoom recordings fetched successfully`, {
recordingCount: data.recording_files?.length || 0,
downloadedCount: files.length,
})
return NextResponse.json({
success: true,
output: {
recording: {
uuid: data.uuid,
id: data.id,
account_id: data.account_id,
host_id: data.host_id,
topic: data.topic,
type: data.type,
start_time: data.start_time,
duration: data.duration,
total_size: data.total_size,
recording_count: data.recording_count,
share_url: data.share_url,
recording_files: (data.recording_files || []).map((file: ZoomRecordingFile) => ({
id: file.id,
meeting_id: file.meeting_id,
recording_start: file.recording_start,
recording_end: file.recording_end,
file_type: file.file_type,
file_extension: file.file_extension,
file_size: file.file_size,
play_url: file.play_url,
download_url: file.download_url,
status: file.status,
recording_type: file.recording_type,
})),
},
files: files.length > 0 ? files : undefined,
},
})
} catch (error) {
logger.error(`[${requestId}] Error fetching Zoom recordings:`, error)
return NextResponse.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error occurred',
},
{ status: 500 }
)
}
}

View File

@@ -0,0 +1,116 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { z } from 'zod'
import { getCopilotModel } from '@/lib/copilot/config'
import { SIM_AGENT_VERSION } from '@/lib/copilot/constants'
import { COPILOT_REQUEST_MODES } from '@/lib/copilot/models'
import { orchestrateCopilotStream } from '@/lib/copilot/orchestrator'
import { resolveWorkflowIdForUser } from '@/lib/workflows/utils'
import { authenticateV1Request } from '@/app/api/v1/auth'
const logger = createLogger('CopilotHeadlessAPI')
const RequestSchema = z.object({
message: z.string().min(1, 'message is required'),
workflowId: z.string().optional(),
workflowName: z.string().optional(),
chatId: z.string().optional(),
mode: z.enum(COPILOT_REQUEST_MODES).optional().default('agent'),
model: z.string().optional(),
autoExecuteTools: z.boolean().optional().default(true),
timeout: z.number().optional().default(300000),
})
/**
* POST /api/v1/copilot/chat
* Headless copilot endpoint for server-side orchestration.
*
* workflowId is optional - if not provided:
* - If workflowName is provided, finds that workflow
* - Otherwise uses the user's first workflow as context
* - The copilot can still operate on any workflow using list_user_workflows
*/
export async function POST(req: NextRequest) {
const auth = await authenticateV1Request(req)
if (!auth.authenticated || !auth.userId) {
return NextResponse.json(
{ success: false, error: auth.error || 'Unauthorized' },
{ status: 401 }
)
}
try {
const body = await req.json()
const parsed = RequestSchema.parse(body)
const defaults = getCopilotModel('chat')
const selectedModel = parsed.model || defaults.model
// Resolve workflow ID
const resolved = await resolveWorkflowIdForUser(
auth.userId,
parsed.workflowId,
parsed.workflowName
)
if (!resolved) {
return NextResponse.json(
{
success: false,
error: 'No workflows found. Create a workflow first or provide a valid workflowId.',
},
{ status: 400 }
)
}
// Transform mode to transport mode (same as client API)
// build and agent both map to 'agent' on the backend
const effectiveMode = parsed.mode === 'agent' ? 'build' : parsed.mode
const transportMode = effectiveMode === 'build' ? 'agent' : effectiveMode
// Always generate a chatId - required for artifacts system to work with subagents
const chatId = parsed.chatId || crypto.randomUUID()
const requestPayload = {
message: parsed.message,
workflowId: resolved.workflowId,
userId: auth.userId,
stream: true,
streamToolCalls: true,
model: selectedModel,
mode: transportMode,
messageId: crypto.randomUUID(),
version: SIM_AGENT_VERSION,
headless: true, // Enable cross-workflow operations via workflowId params
chatId,
}
const result = await orchestrateCopilotStream(requestPayload, {
userId: auth.userId,
workflowId: resolved.workflowId,
chatId,
autoExecuteTools: parsed.autoExecuteTools,
timeout: parsed.timeout,
interactive: false,
})
return NextResponse.json({
success: result.success,
content: result.content,
toolCalls: result.toolCalls,
chatId: result.chatId || chatId, // Return the chatId for conversation continuity
conversationId: result.conversationId,
error: result.error,
})
} catch (error) {
if (error instanceof z.ZodError) {
return NextResponse.json(
{ success: false, error: 'Invalid request', details: error.errors },
{ status: 400 }
)
}
logger.error('Headless copilot request failed', {
error: error instanceof Error ? error.message : String(error),
})
return NextResponse.json({ success: false, error: 'Internal server error' }, { status: 500 })
}
}

View File

@@ -3,7 +3,6 @@ import { userStats, workflow } from '@sim/db/schema'
import { createLogger } from '@sim/logger' import { createLogger } from '@sim/logger'
import { eq, sql } from 'drizzle-orm' import { eq, sql } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import OpenAI, { AzureOpenAI } from 'openai'
import { getBYOKKey } from '@/lib/api-key/byok' import { getBYOKKey } from '@/lib/api-key/byok'
import { getSession } from '@/lib/auth' import { getSession } from '@/lib/auth'
import { logModelUsage } from '@/lib/billing/core/usage-log' import { logModelUsage } from '@/lib/billing/core/usage-log'
@@ -12,6 +11,7 @@ import { env } from '@/lib/core/config/env'
import { getCostMultiplier, isBillingEnabled } from '@/lib/core/config/feature-flags' import { getCostMultiplier, isBillingEnabled } from '@/lib/core/config/feature-flags'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { verifyWorkspaceMembership } from '@/app/api/workflows/utils' import { verifyWorkspaceMembership } from '@/app/api/workflows/utils'
import { extractResponseText, parseResponsesUsage } from '@/providers/openai/utils'
import { getModelPricing } from '@/providers/utils' import { getModelPricing } from '@/providers/utils'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -28,18 +28,6 @@ const openaiApiKey = env.OPENAI_API_KEY
const useWandAzure = azureApiKey && azureEndpoint && azureApiVersion const useWandAzure = azureApiKey && azureEndpoint && azureApiVersion
const client = useWandAzure
? new AzureOpenAI({
apiKey: azureApiKey,
apiVersion: azureApiVersion,
endpoint: azureEndpoint,
})
: openaiApiKey
? new OpenAI({
apiKey: openaiApiKey,
})
: null
if (!useWandAzure && !openaiApiKey) { if (!useWandAzure && !openaiApiKey) {
logger.warn( logger.warn(
'Neither Azure OpenAI nor OpenAI API key found. Wand generation API will not function.' 'Neither Azure OpenAI nor OpenAI API key found. Wand generation API will not function.'
@@ -202,20 +190,18 @@ export async function POST(req: NextRequest) {
} }
let isBYOK = false let isBYOK = false
let activeClient = client let activeOpenAIKey = openaiApiKey
let byokApiKey: string | null = null
if (workspaceId && !useWandAzure) { if (workspaceId && !useWandAzure) {
const byokResult = await getBYOKKey(workspaceId, 'openai') const byokResult = await getBYOKKey(workspaceId, 'openai')
if (byokResult) { if (byokResult) {
isBYOK = true isBYOK = true
byokApiKey = byokResult.apiKey activeOpenAIKey = byokResult.apiKey
activeClient = new OpenAI({ apiKey: byokResult.apiKey })
logger.info(`[${requestId}] Using BYOK OpenAI key for wand generation`) logger.info(`[${requestId}] Using BYOK OpenAI key for wand generation`)
} }
} }
if (!activeClient) { if (!useWandAzure && !activeOpenAIKey) {
logger.error(`[${requestId}] AI client not initialized. Missing API key.`) logger.error(`[${requestId}] AI client not initialized. Missing API key.`)
return NextResponse.json( return NextResponse.json(
{ success: false, error: 'Wand generation service is not configured.' }, { success: false, error: 'Wand generation service is not configured.' },
@@ -276,17 +262,18 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
) )
const apiUrl = useWandAzure const apiUrl = useWandAzure
? `${azureEndpoint}/openai/deployments/${wandModelName}/chat/completions?api-version=${azureApiVersion}` ? `${azureEndpoint?.replace(/\/$/, '')}/openai/v1/responses?api-version=${azureApiVersion}`
: 'https://api.openai.com/v1/chat/completions' : 'https://api.openai.com/v1/responses'
const headers: Record<string, string> = { const headers: Record<string, string> = {
'Content-Type': 'application/json', 'Content-Type': 'application/json',
'OpenAI-Beta': 'responses=v1',
} }
if (useWandAzure) { if (useWandAzure) {
headers['api-key'] = azureApiKey! headers['api-key'] = azureApiKey!
} else { } else {
headers.Authorization = `Bearer ${byokApiKey || openaiApiKey}` headers.Authorization = `Bearer ${activeOpenAIKey}`
} }
logger.debug(`[${requestId}] Making streaming request to: ${apiUrl}`) logger.debug(`[${requestId}] Making streaming request to: ${apiUrl}`)
@@ -296,11 +283,10 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
headers, headers,
body: JSON.stringify({ body: JSON.stringify({
model: useWandAzure ? wandModelName : 'gpt-4o', model: useWandAzure ? wandModelName : 'gpt-4o',
messages: messages, input: messages,
temperature: 0.2, temperature: 0.2,
max_tokens: 10000, max_output_tokens: 10000,
stream: true, stream: true,
stream_options: { include_usage: true },
}), }),
}) })
@@ -327,16 +313,29 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
return return
} }
let finalUsage: any = null
let usageRecorded = false
const recordUsage = async () => {
if (usageRecorded || !finalUsage) {
return
}
usageRecorded = true
await updateUserStatsForWand(session.user.id, finalUsage, requestId, isBYOK)
}
try { try {
let buffer = '' let buffer = ''
let chunkCount = 0 let chunkCount = 0
let finalUsage: any = null let activeEventType: string | undefined
while (true) { while (true) {
const { done, value } = await reader.read() const { done, value } = await reader.read()
if (done) { if (done) {
logger.info(`[${requestId}] Stream completed. Total chunks: ${chunkCount}`) logger.info(`[${requestId}] Stream completed. Total chunks: ${chunkCount}`)
await recordUsage()
controller.enqueue(encoder.encode(`data: ${JSON.stringify({ done: true })}\n\n`)) controller.enqueue(encoder.encode(`data: ${JSON.stringify({ done: true })}\n\n`))
controller.close() controller.close()
break break
@@ -348,47 +347,90 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
buffer = lines.pop() || '' buffer = lines.pop() || ''
for (const line of lines) { for (const line of lines) {
if (line.startsWith('data: ')) { const trimmed = line.trim()
const data = line.slice(6).trim() if (!trimmed) {
continue
}
if (data === '[DONE]') { if (trimmed.startsWith('event:')) {
logger.info(`[${requestId}] Received [DONE] signal`) activeEventType = trimmed.slice(6).trim()
continue
}
if (finalUsage) { if (!trimmed.startsWith('data:')) {
await updateUserStatsForWand(session.user.id, finalUsage, requestId, isBYOK) continue
}
const data = trimmed.slice(5).trim()
if (data === '[DONE]') {
logger.info(`[${requestId}] Received [DONE] signal`)
await recordUsage()
controller.enqueue(
encoder.encode(`data: ${JSON.stringify({ done: true })}\n\n`)
)
controller.close()
return
}
let parsed: any
try {
parsed = JSON.parse(data)
} catch (parseError) {
logger.debug(`[${requestId}] Skipped non-JSON line: ${data.substring(0, 100)}`)
continue
}
const eventType = parsed?.type ?? activeEventType
if (
eventType === 'response.error' ||
eventType === 'error' ||
eventType === 'response.failed'
) {
throw new Error(parsed?.error?.message || 'Responses stream error')
}
if (
eventType === 'response.output_text.delta' ||
eventType === 'response.output_json.delta'
) {
let content = ''
if (typeof parsed.delta === 'string') {
content = parsed.delta
} else if (parsed.delta && typeof parsed.delta.text === 'string') {
content = parsed.delta.text
} else if (parsed.delta && parsed.delta.json !== undefined) {
content = JSON.stringify(parsed.delta.json)
} else if (parsed.json !== undefined) {
content = JSON.stringify(parsed.json)
} else if (typeof parsed.text === 'string') {
content = parsed.text
}
if (content) {
chunkCount++
if (chunkCount === 1) {
logger.info(`[${requestId}] Received first content chunk`)
} }
controller.enqueue( controller.enqueue(
encoder.encode(`data: ${JSON.stringify({ done: true })}\n\n`) encoder.encode(`data: ${JSON.stringify({ chunk: content })}\n\n`)
) )
controller.close()
return
} }
}
try { if (eventType === 'response.completed') {
const parsed = JSON.parse(data) const usage = parseResponsesUsage(parsed?.response?.usage ?? parsed?.usage)
const content = parsed.choices?.[0]?.delta?.content if (usage) {
finalUsage = {
if (content) { prompt_tokens: usage.promptTokens,
chunkCount++ completion_tokens: usage.completionTokens,
if (chunkCount === 1) { total_tokens: usage.totalTokens,
logger.info(`[${requestId}] Received first content chunk`)
}
controller.enqueue(
encoder.encode(`data: ${JSON.stringify({ chunk: content })}\n\n`)
)
} }
logger.info(
if (parsed.usage) { `[${requestId}] Received usage data: ${JSON.stringify(finalUsage)}`
finalUsage = parsed.usage
logger.info(
`[${requestId}] Received usage data: ${JSON.stringify(parsed.usage)}`
)
}
} catch (parseError) {
logger.debug(
`[${requestId}] Skipped non-JSON line: ${data.substring(0, 100)}`
) )
} }
} }
@@ -401,6 +443,12 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
stack: streamError?.stack, stack: streamError?.stack,
}) })
try {
await recordUsage()
} catch (usageError) {
logger.warn(`[${requestId}] Failed to record usage after stream error`, usageError)
}
const errorData = `data: ${JSON.stringify({ error: 'Streaming failed', done: true })}\n\n` const errorData = `data: ${JSON.stringify({ error: 'Streaming failed', done: true })}\n\n`
controller.enqueue(encoder.encode(errorData)) controller.enqueue(encoder.encode(errorData))
controller.close() controller.close()
@@ -424,8 +472,6 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
message: error?.message || 'Unknown error', message: error?.message || 'Unknown error',
code: error?.code, code: error?.code,
status: error?.status, status: error?.status,
responseStatus: error?.response?.status,
responseData: error?.response?.data ? safeStringify(error.response.data) : undefined,
stack: error?.stack, stack: error?.stack,
useWandAzure, useWandAzure,
model: useWandAzure ? wandModelName : 'gpt-4o', model: useWandAzure ? wandModelName : 'gpt-4o',
@@ -440,14 +486,43 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
} }
} }
const completion = await activeClient.chat.completions.create({ const apiUrl = useWandAzure
model: useWandAzure ? wandModelName : 'gpt-4o', ? `${azureEndpoint?.replace(/\/$/, '')}/openai/v1/responses?api-version=${azureApiVersion}`
messages: messages, : 'https://api.openai.com/v1/responses'
temperature: 0.3,
max_tokens: 10000, const headers: Record<string, string> = {
'Content-Type': 'application/json',
'OpenAI-Beta': 'responses=v1',
}
if (useWandAzure) {
headers['api-key'] = azureApiKey!
} else {
headers.Authorization = `Bearer ${activeOpenAIKey}`
}
const response = await fetch(apiUrl, {
method: 'POST',
headers,
body: JSON.stringify({
model: useWandAzure ? wandModelName : 'gpt-4o',
input: messages,
temperature: 0.2,
max_output_tokens: 10000,
}),
}) })
const generatedContent = completion.choices[0]?.message?.content?.trim() if (!response.ok) {
const errorText = await response.text()
const apiError = new Error(
`API request failed: ${response.status} ${response.statusText} - ${errorText}`
)
;(apiError as any).status = response.status
throw apiError
}
const completion = await response.json()
const generatedContent = extractResponseText(completion.output)?.trim()
if (!generatedContent) { if (!generatedContent) {
logger.error( logger.error(
@@ -461,8 +536,18 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
logger.info(`[${requestId}] Wand generation successful`) logger.info(`[${requestId}] Wand generation successful`)
if (completion.usage) { const usage = parseResponsesUsage(completion.usage)
await updateUserStatsForWand(session.user.id, completion.usage, requestId, isBYOK) if (usage) {
await updateUserStatsForWand(
session.user.id,
{
prompt_tokens: usage.promptTokens,
completion_tokens: usage.completionTokens,
total_tokens: usage.totalTokens,
},
requestId,
isBYOK
)
} }
return NextResponse.json({ success: true, content: generatedContent }) return NextResponse.json({ success: true, content: generatedContent })
@@ -472,10 +557,6 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
message: error?.message || 'Unknown error', message: error?.message || 'Unknown error',
code: error?.code, code: error?.code,
status: error?.status, status: error?.status,
responseStatus: error instanceof OpenAI.APIError ? error.status : error?.response?.status,
responseData: (error as any)?.response?.data
? safeStringify((error as any).response.data)
: undefined,
stack: error?.stack, stack: error?.stack,
useWandAzure, useWandAzure,
model: useWandAzure ? wandModelName : 'gpt-4o', model: useWandAzure ? wandModelName : 'gpt-4o',
@@ -484,26 +565,19 @@ Use this context to calculate relative dates like "yesterday", "last week", "beg
}) })
let clientErrorMessage = 'Wand generation failed. Please try again later.' let clientErrorMessage = 'Wand generation failed. Please try again later.'
let status = 500 let status = typeof (error as any)?.status === 'number' ? (error as any).status : 500
if (error instanceof OpenAI.APIError) { if (useWandAzure && error?.message?.includes('DeploymentNotFound')) {
status = error.status || 500
logger.error(
`[${requestId}] ${useWandAzure ? 'Azure OpenAI' : 'OpenAI'} API Error: ${status} - ${error.message}`
)
if (status === 401) {
clientErrorMessage = 'Authentication failed. Please check your API key configuration.'
} else if (status === 429) {
clientErrorMessage = 'Rate limit exceeded. Please try again later.'
} else if (status >= 500) {
clientErrorMessage =
'The wand generation service is currently unavailable. Please try again later.'
}
} else if (useWandAzure && error.message?.includes('DeploymentNotFound')) {
clientErrorMessage = clientErrorMessage =
'Azure OpenAI deployment not found. Please check your model deployment configuration.' 'Azure OpenAI deployment not found. Please check your model deployment configuration.'
status = 404 status = 404
} else if (status === 401) {
clientErrorMessage = 'Authentication failed. Please check your API key configuration.'
} else if (status === 429) {
clientErrorMessage = 'Rate limit exceeded. Please try again later.'
} else if (status >= 500) {
clientErrorMessage =
'The wand generation service is currently unavailable. Please try again later.'
} }
return NextResponse.json( return NextResponse.json(

View File

@@ -1,190 +0,0 @@
import { db, workflowDeploymentVersion } from '@sim/db'
import { createLogger } from '@sim/logger'
import { and, eq } from 'drizzle-orm'
import type { NextRequest } from 'next/server'
import { generateRequestId } from '@/lib/core/utils/request'
import { syncMcpToolsForWorkflow } from '@/lib/mcp/workflow-mcp-sync'
import { restorePreviousVersionWebhooks, saveTriggerWebhooksForDeploy } from '@/lib/webhooks/deploy'
import { activateWorkflowVersion } from '@/lib/workflows/persistence/utils'
import {
cleanupDeploymentVersion,
createSchedulesForDeploy,
validateWorkflowSchedules,
} from '@/lib/workflows/schedules'
import { validateWorkflowPermissions } from '@/lib/workflows/utils'
import { createErrorResponse, createSuccessResponse } from '@/app/api/workflows/utils'
import type { BlockState } from '@/stores/workflows/workflow/types'
const logger = createLogger('WorkflowActivateDeploymentAPI')
export const dynamic = 'force-dynamic'
export const runtime = 'nodejs'
export async function POST(
request: NextRequest,
{ params }: { params: Promise<{ id: string; version: string }> }
) {
const requestId = generateRequestId()
const { id, version } = await params
try {
const {
error,
session,
workflow: workflowData,
} = await validateWorkflowPermissions(id, requestId, 'admin')
if (error) {
return createErrorResponse(error.message, error.status)
}
const actorUserId = session?.user?.id
if (!actorUserId) {
logger.warn(`[${requestId}] Unable to resolve actor user for deployment activation: ${id}`)
return createErrorResponse('Unable to determine activating user', 400)
}
const versionNum = Number(version)
if (!Number.isFinite(versionNum)) {
return createErrorResponse('Invalid version number', 400)
}
const [versionRow] = await db
.select({
id: workflowDeploymentVersion.id,
state: workflowDeploymentVersion.state,
})
.from(workflowDeploymentVersion)
.where(
and(
eq(workflowDeploymentVersion.workflowId, id),
eq(workflowDeploymentVersion.version, versionNum)
)
)
.limit(1)
if (!versionRow?.state) {
return createErrorResponse('Deployment version not found', 404)
}
const [currentActiveVersion] = await db
.select({ id: workflowDeploymentVersion.id })
.from(workflowDeploymentVersion)
.where(
and(
eq(workflowDeploymentVersion.workflowId, id),
eq(workflowDeploymentVersion.isActive, true)
)
)
.limit(1)
const previousVersionId = currentActiveVersion?.id
const deployedState = versionRow.state as { blocks?: Record<string, BlockState> }
const blocks = deployedState.blocks
if (!blocks || typeof blocks !== 'object') {
return createErrorResponse('Invalid deployed state structure', 500)
}
const scheduleValidation = validateWorkflowSchedules(blocks)
if (!scheduleValidation.isValid) {
return createErrorResponse(`Invalid schedule configuration: ${scheduleValidation.error}`, 400)
}
const triggerSaveResult = await saveTriggerWebhooksForDeploy({
request,
workflowId: id,
workflow: workflowData as Record<string, unknown>,
userId: actorUserId,
blocks,
requestId,
deploymentVersionId: versionRow.id,
previousVersionId,
forceRecreateSubscriptions: true,
})
if (!triggerSaveResult.success) {
return createErrorResponse(
triggerSaveResult.error?.message || 'Failed to sync trigger configuration',
triggerSaveResult.error?.status || 500
)
}
const scheduleResult = await createSchedulesForDeploy(id, blocks, db, versionRow.id)
if (!scheduleResult.success) {
await cleanupDeploymentVersion({
workflowId: id,
workflow: workflowData as Record<string, unknown>,
requestId,
deploymentVersionId: versionRow.id,
})
if (previousVersionId) {
await restorePreviousVersionWebhooks({
request,
workflow: workflowData as Record<string, unknown>,
userId: actorUserId,
previousVersionId,
requestId,
})
}
return createErrorResponse(scheduleResult.error || 'Failed to sync schedules', 500)
}
const result = await activateWorkflowVersion({ workflowId: id, version: versionNum })
if (!result.success) {
await cleanupDeploymentVersion({
workflowId: id,
workflow: workflowData as Record<string, unknown>,
requestId,
deploymentVersionId: versionRow.id,
})
if (previousVersionId) {
await restorePreviousVersionWebhooks({
request,
workflow: workflowData as Record<string, unknown>,
userId: actorUserId,
previousVersionId,
requestId,
})
}
return createErrorResponse(result.error || 'Failed to activate deployment', 400)
}
if (previousVersionId && previousVersionId !== versionRow.id) {
try {
logger.info(
`[${requestId}] Cleaning up previous version ${previousVersionId} webhooks/schedules`
)
await cleanupDeploymentVersion({
workflowId: id,
workflow: workflowData as Record<string, unknown>,
requestId,
deploymentVersionId: previousVersionId,
skipExternalCleanup: true,
})
logger.info(`[${requestId}] Previous version cleanup completed`)
} catch (cleanupError) {
logger.error(
`[${requestId}] Failed to clean up previous version ${previousVersionId}`,
cleanupError
)
}
}
await syncMcpToolsForWorkflow({
workflowId: id,
requestId,
state: versionRow.state,
context: 'activate',
})
return createSuccessResponse({
success: true,
deployedAt: result.deployedAt,
warnings: triggerSaveResult.warnings,
})
} catch (error: any) {
logger.error(`[${requestId}] Error activating deployment for workflow: ${id}`, error)
return createErrorResponse(error.message || 'Failed to activate deployment', 500)
}
}

View File

@@ -4,8 +4,17 @@ import { and, eq } from 'drizzle-orm'
import type { NextRequest } from 'next/server' import type { NextRequest } from 'next/server'
import { z } from 'zod' import { z } from 'zod'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { syncMcpToolsForWorkflow } from '@/lib/mcp/workflow-mcp-sync'
import { restorePreviousVersionWebhooks, saveTriggerWebhooksForDeploy } from '@/lib/webhooks/deploy'
import { activateWorkflowVersion } from '@/lib/workflows/persistence/utils'
import {
cleanupDeploymentVersion,
createSchedulesForDeploy,
validateWorkflowSchedules,
} from '@/lib/workflows/schedules'
import { validateWorkflowPermissions } from '@/lib/workflows/utils' import { validateWorkflowPermissions } from '@/lib/workflows/utils'
import { createErrorResponse, createSuccessResponse } from '@/app/api/workflows/utils' import { createErrorResponse, createSuccessResponse } from '@/app/api/workflows/utils'
import type { BlockState } from '@/stores/workflows/workflow/types'
const logger = createLogger('WorkflowDeploymentVersionAPI') const logger = createLogger('WorkflowDeploymentVersionAPI')
@@ -23,10 +32,14 @@ const patchBodySchema = z
.max(500, 'Description must be 500 characters or less') .max(500, 'Description must be 500 characters or less')
.nullable() .nullable()
.optional(), .optional(),
isActive: z.literal(true).optional(), // Set to true to activate this version
}) })
.refine((data) => data.name !== undefined || data.description !== undefined, { .refine(
message: 'At least one of name or description must be provided', (data) => data.name !== undefined || data.description !== undefined || data.isActive === true,
}) {
message: 'At least one of name, description, or isActive must be provided',
}
)
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
export const runtime = 'nodejs' export const runtime = 'nodejs'
@@ -82,7 +95,22 @@ export async function PATCH(
const { id, version } = await params const { id, version } = await params
try { try {
const { error } = await validateWorkflowPermissions(id, requestId, 'write') const body = await request.json()
const validation = patchBodySchema.safeParse(body)
if (!validation.success) {
return createErrorResponse(validation.error.errors[0]?.message || 'Invalid request body', 400)
}
const { name, description, isActive } = validation.data
// Activation requires admin permission, other updates require write
const requiredPermission = isActive ? 'admin' : 'write'
const {
error,
session,
workflow: workflowData,
} = await validateWorkflowPermissions(id, requestId, requiredPermission)
if (error) { if (error) {
return createErrorResponse(error.message, error.status) return createErrorResponse(error.message, error.status)
} }
@@ -92,15 +120,193 @@ export async function PATCH(
return createErrorResponse('Invalid version', 400) return createErrorResponse('Invalid version', 400)
} }
const body = await request.json() // Handle activation
const validation = patchBodySchema.safeParse(body) if (isActive) {
const actorUserId = session?.user?.id
if (!actorUserId) {
logger.warn(`[${requestId}] Unable to resolve actor user for deployment activation: ${id}`)
return createErrorResponse('Unable to determine activating user', 400)
}
if (!validation.success) { const [versionRow] = await db
return createErrorResponse(validation.error.errors[0]?.message || 'Invalid request body', 400) .select({
id: workflowDeploymentVersion.id,
state: workflowDeploymentVersion.state,
})
.from(workflowDeploymentVersion)
.where(
and(
eq(workflowDeploymentVersion.workflowId, id),
eq(workflowDeploymentVersion.version, versionNum)
)
)
.limit(1)
if (!versionRow?.state) {
return createErrorResponse('Deployment version not found', 404)
}
const [currentActiveVersion] = await db
.select({ id: workflowDeploymentVersion.id })
.from(workflowDeploymentVersion)
.where(
and(
eq(workflowDeploymentVersion.workflowId, id),
eq(workflowDeploymentVersion.isActive, true)
)
)
.limit(1)
const previousVersionId = currentActiveVersion?.id
const deployedState = versionRow.state as { blocks?: Record<string, BlockState> }
const blocks = deployedState.blocks
if (!blocks || typeof blocks !== 'object') {
return createErrorResponse('Invalid deployed state structure', 500)
}
const scheduleValidation = validateWorkflowSchedules(blocks)
if (!scheduleValidation.isValid) {
return createErrorResponse(
`Invalid schedule configuration: ${scheduleValidation.error}`,
400
)
}
const triggerSaveResult = await saveTriggerWebhooksForDeploy({
request,
workflowId: id,
workflow: workflowData as Record<string, unknown>,
userId: actorUserId,
blocks,
requestId,
deploymentVersionId: versionRow.id,
previousVersionId,
forceRecreateSubscriptions: true,
})
if (!triggerSaveResult.success) {
return createErrorResponse(
triggerSaveResult.error?.message || 'Failed to sync trigger configuration',
triggerSaveResult.error?.status || 500
)
}
const scheduleResult = await createSchedulesForDeploy(id, blocks, db, versionRow.id)
if (!scheduleResult.success) {
await cleanupDeploymentVersion({
workflowId: id,
workflow: workflowData as Record<string, unknown>,
requestId,
deploymentVersionId: versionRow.id,
})
if (previousVersionId) {
await restorePreviousVersionWebhooks({
request,
workflow: workflowData as Record<string, unknown>,
userId: actorUserId,
previousVersionId,
requestId,
})
}
return createErrorResponse(scheduleResult.error || 'Failed to sync schedules', 500)
}
const result = await activateWorkflowVersion({ workflowId: id, version: versionNum })
if (!result.success) {
await cleanupDeploymentVersion({
workflowId: id,
workflow: workflowData as Record<string, unknown>,
requestId,
deploymentVersionId: versionRow.id,
})
if (previousVersionId) {
await restorePreviousVersionWebhooks({
request,
workflow: workflowData as Record<string, unknown>,
userId: actorUserId,
previousVersionId,
requestId,
})
}
return createErrorResponse(result.error || 'Failed to activate deployment', 400)
}
if (previousVersionId && previousVersionId !== versionRow.id) {
try {
logger.info(
`[${requestId}] Cleaning up previous version ${previousVersionId} webhooks/schedules`
)
await cleanupDeploymentVersion({
workflowId: id,
workflow: workflowData as Record<string, unknown>,
requestId,
deploymentVersionId: previousVersionId,
skipExternalCleanup: true,
})
logger.info(`[${requestId}] Previous version cleanup completed`)
} catch (cleanupError) {
logger.error(
`[${requestId}] Failed to clean up previous version ${previousVersionId}`,
cleanupError
)
}
}
await syncMcpToolsForWorkflow({
workflowId: id,
requestId,
state: versionRow.state,
context: 'activate',
})
// Apply name/description updates if provided alongside activation
let updatedName: string | null | undefined
let updatedDescription: string | null | undefined
if (name !== undefined || description !== undefined) {
const activationUpdateData: { name?: string; description?: string | null } = {}
if (name !== undefined) {
activationUpdateData.name = name
}
if (description !== undefined) {
activationUpdateData.description = description
}
const [updated] = await db
.update(workflowDeploymentVersion)
.set(activationUpdateData)
.where(
and(
eq(workflowDeploymentVersion.workflowId, id),
eq(workflowDeploymentVersion.version, versionNum)
)
)
.returning({
name: workflowDeploymentVersion.name,
description: workflowDeploymentVersion.description,
})
if (updated) {
updatedName = updated.name
updatedDescription = updated.description
logger.info(
`[${requestId}] Updated deployment version ${version} metadata during activation`,
{ name: activationUpdateData.name, description: activationUpdateData.description }
)
}
}
return createSuccessResponse({
success: true,
deployedAt: result.deployedAt,
warnings: triggerSaveResult.warnings,
...(updatedName !== undefined && { name: updatedName }),
...(updatedDescription !== undefined && { description: updatedDescription }),
})
} }
const { name, description } = validation.data // Handle name/description updates
const updateData: { name?: string; description?: string | null } = {} const updateData: { name?: string; description?: string | null } = {}
if (name !== undefined) { if (name !== undefined) {
updateData.name = name updateData.name = name

View File

@@ -1,235 +0,0 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import { v4 as uuidv4 } from 'uuid'
import { z } from 'zod'
import { checkHybridAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { SSE_HEADERS } from '@/lib/core/utils/sse'
import { markExecutionCancelled } from '@/lib/execution/cancellation'
import { preprocessExecution } from '@/lib/execution/preprocessing'
import { LoggingSession } from '@/lib/logs/execution/logging-session'
import { executeWorkflowCore } from '@/lib/workflows/executor/execution-core'
import { createSSECallbacks } from '@/lib/workflows/executor/execution-events'
import { ExecutionSnapshot } from '@/executor/execution/snapshot'
import type { ExecutionMetadata, SerializableExecutionState } from '@/executor/execution/types'
import { hasExecutionResult } from '@/executor/utils/errors'
const logger = createLogger('ExecuteFromBlockAPI')
const ExecuteFromBlockSchema = z.object({
startBlockId: z.string().min(1, 'Start block ID is required'),
sourceSnapshot: z.object({
blockStates: z.record(z.any()),
executedBlocks: z.array(z.string()),
blockLogs: z.array(z.any()),
decisions: z.object({
router: z.record(z.string()),
condition: z.record(z.string()),
}),
completedLoops: z.array(z.string()),
loopExecutions: z.record(z.any()).optional(),
parallelExecutions: z.record(z.any()).optional(),
parallelBlockMapping: z.record(z.any()).optional(),
activeExecutionPath: z.array(z.string()),
}),
input: z.any().optional(),
})
export const runtime = 'nodejs'
export const dynamic = 'force-dynamic'
export async function POST(req: NextRequest, { params }: { params: Promise<{ id: string }> }) {
const requestId = generateRequestId()
const { id: workflowId } = await params
try {
const auth = await checkHybridAuth(req, { requireWorkflowId: false })
if (!auth.success || !auth.userId) {
return NextResponse.json({ error: auth.error || 'Unauthorized' }, { status: 401 })
}
const userId = auth.userId
let body: unknown
try {
body = await req.json()
} catch {
return NextResponse.json({ error: 'Invalid JSON body' }, { status: 400 })
}
const validation = ExecuteFromBlockSchema.safeParse(body)
if (!validation.success) {
logger.warn(`[${requestId}] Invalid request body:`, validation.error.errors)
return NextResponse.json(
{
error: 'Invalid request body',
details: validation.error.errors.map((e) => ({
path: e.path.join('.'),
message: e.message,
})),
},
{ status: 400 }
)
}
const { startBlockId, sourceSnapshot, input } = validation.data
const executionId = uuidv4()
// Run preprocessing checks (billing, rate limits, usage limits)
const preprocessResult = await preprocessExecution({
workflowId,
userId,
triggerType: 'manual',
executionId,
requestId,
checkRateLimit: false, // Manual executions don't rate limit
checkDeployment: false, // Run-from-block doesn't require deployment
})
if (!preprocessResult.success) {
const { error } = preprocessResult
logger.warn(`[${requestId}] Preprocessing failed for run-from-block`, {
workflowId,
error: error?.message,
statusCode: error?.statusCode,
})
return NextResponse.json(
{ error: error?.message || 'Execution blocked' },
{ status: error?.statusCode || 500 }
)
}
const workflowRecord = preprocessResult.workflowRecord
if (!workflowRecord?.workspaceId) {
return NextResponse.json({ error: 'Workflow not found or has no workspace' }, { status: 404 })
}
const workspaceId = workflowRecord.workspaceId
const workflowUserId = workflowRecord.userId
logger.info(`[${requestId}] Starting run-from-block execution`, {
workflowId,
startBlockId,
executedBlocksCount: sourceSnapshot.executedBlocks.length,
billingActorUserId: preprocessResult.actorUserId,
})
const loggingSession = new LoggingSession(workflowId, executionId, 'manual', requestId)
const abortController = new AbortController()
let isStreamClosed = false
const stream = new ReadableStream<Uint8Array>({
async start(controller) {
const { sendEvent, onBlockStart, onBlockComplete, onStream } = createSSECallbacks({
executionId,
workflowId,
controller,
isStreamClosed: () => isStreamClosed,
setStreamClosed: () => {
isStreamClosed = true
},
})
const metadata: ExecutionMetadata = {
requestId,
workflowId,
userId,
executionId,
triggerType: 'manual',
workspaceId,
workflowUserId,
useDraftState: true,
isClientSession: true,
startTime: new Date().toISOString(),
}
const snapshot = new ExecutionSnapshot(metadata, {}, input || {}, {})
try {
const startTime = new Date()
sendEvent({
type: 'execution:started',
timestamp: startTime.toISOString(),
executionId,
workflowId,
data: { startTime: startTime.toISOString() },
})
const result = await executeWorkflowCore({
snapshot,
loggingSession,
abortSignal: abortController.signal,
runFromBlock: {
startBlockId,
sourceSnapshot: sourceSnapshot as SerializableExecutionState,
},
callbacks: { onBlockStart, onBlockComplete, onStream },
})
if (result.status === 'cancelled') {
sendEvent({
type: 'execution:cancelled',
timestamp: new Date().toISOString(),
executionId,
workflowId,
data: { duration: result.metadata?.duration || 0 },
})
} else {
sendEvent({
type: 'execution:completed',
timestamp: new Date().toISOString(),
executionId,
workflowId,
data: {
success: result.success,
output: result.output,
duration: result.metadata?.duration || 0,
startTime: result.metadata?.startTime || startTime.toISOString(),
endTime: result.metadata?.endTime || new Date().toISOString(),
},
})
}
} catch (error: unknown) {
const errorMessage = error instanceof Error ? error.message : 'Unknown error'
logger.error(`[${requestId}] Run-from-block execution failed: ${errorMessage}`)
const executionResult = hasExecutionResult(error) ? error.executionResult : undefined
sendEvent({
type: 'execution:error',
timestamp: new Date().toISOString(),
executionId,
workflowId,
data: {
error: executionResult?.error || errorMessage,
duration: executionResult?.metadata?.duration || 0,
},
})
} finally {
if (!isStreamClosed) {
try {
controller.enqueue(new TextEncoder().encode('data: [DONE]\n\n'))
controller.close()
} catch {}
}
}
},
cancel() {
isStreamClosed = true
abortController.abort()
markExecutionCancelled(executionId).catch(() => {})
},
})
return new NextResponse(stream, {
headers: { ...SSE_HEADERS, 'X-Execution-Id': executionId },
})
} catch (error: unknown) {
const errorMessage = error instanceof Error ? error.message : 'Unknown error'
logger.error(`[${requestId}] Failed to start run-from-block execution:`, error)
return NextResponse.json(
{ error: errorMessage || 'Failed to start execution' },
{ status: 500 }
)
}
}

View File

@@ -1,10 +1,14 @@
import { createLogger } from '@sim/logger' import { createLogger } from '@sim/logger'
import { tasks } from '@trigger.dev/sdk'
import { type NextRequest, NextResponse } from 'next/server' import { type NextRequest, NextResponse } from 'next/server'
import { validate as uuidValidate, v4 as uuidv4 } from 'uuid' import { validate as uuidValidate, v4 as uuidv4 } from 'uuid'
import { z } from 'zod' import { z } from 'zod'
import { checkHybridAuth } from '@/lib/auth/hybrid' import { checkHybridAuth } from '@/lib/auth/hybrid'
import { isTriggerDevEnabled } from '@/lib/core/config/feature-flags' import { getJobQueue, shouldExecuteInline } from '@/lib/core/async-jobs'
import {
createTimeoutAbortController,
getTimeoutErrorMessage,
isTimeoutError,
} from '@/lib/core/execution-limits'
import { generateRequestId } from '@/lib/core/utils/request' import { generateRequestId } from '@/lib/core/utils/request'
import { SSE_HEADERS } from '@/lib/core/utils/sse' import { SSE_HEADERS } from '@/lib/core/utils/sse'
import { getBaseUrl } from '@/lib/core/utils/urls' import { getBaseUrl } from '@/lib/core/utils/urls'
@@ -12,6 +16,7 @@ import { markExecutionCancelled } from '@/lib/execution/cancellation'
import { processInputFileFields } from '@/lib/execution/files' import { processInputFileFields } from '@/lib/execution/files'
import { preprocessExecution } from '@/lib/execution/preprocessing' import { preprocessExecution } from '@/lib/execution/preprocessing'
import { LoggingSession } from '@/lib/logs/execution/logging-session' import { LoggingSession } from '@/lib/logs/execution/logging-session'
import { buildTraceSpans } from '@/lib/logs/execution/trace-spans/trace-spans'
import { import {
cleanupExecutionBase64Cache, cleanupExecutionBase64Cache,
hydrateUserFilesWithBase64, hydrateUserFilesWithBase64,
@@ -25,7 +30,7 @@ import {
} from '@/lib/workflows/persistence/utils' } from '@/lib/workflows/persistence/utils'
import { createStreamingResponse } from '@/lib/workflows/streaming/streaming' import { createStreamingResponse } from '@/lib/workflows/streaming/streaming'
import { createHttpResponseFromBlock, workflowHasResponseBlock } from '@/lib/workflows/utils' import { createHttpResponseFromBlock, workflowHasResponseBlock } from '@/lib/workflows/utils'
import type { WorkflowExecutionPayload } from '@/background/workflow-execution' import { executeWorkflowJob, type WorkflowExecutionPayload } from '@/background/workflow-execution'
import { normalizeName } from '@/executor/constants' import { normalizeName } from '@/executor/constants'
import { ExecutionSnapshot } from '@/executor/execution/snapshot' import { ExecutionSnapshot } from '@/executor/execution/snapshot'
import type { ExecutionMetadata, IterationContext } from '@/executor/execution/types' import type { ExecutionMetadata, IterationContext } from '@/executor/execution/types'
@@ -54,6 +59,25 @@ const ExecuteWorkflowSchema = z.object({
}) })
.optional(), .optional(),
stopAfterBlockId: z.string().optional(), stopAfterBlockId: z.string().optional(),
runFromBlock: z
.object({
startBlockId: z.string().min(1, 'Start block ID is required'),
sourceSnapshot: z.object({
blockStates: z.record(z.any()),
executedBlocks: z.array(z.string()),
blockLogs: z.array(z.any()),
decisions: z.object({
router: z.record(z.string()),
condition: z.record(z.string()),
}),
completedLoops: z.array(z.string()),
loopExecutions: z.record(z.any()).optional(),
parallelExecutions: z.record(z.any()).optional(),
parallelBlockMapping: z.record(z.any()).optional(),
activeExecutionPath: z.array(z.string()),
}),
})
.optional(),
}) })
export const runtime = 'nodejs' export const runtime = 'nodejs'
@@ -118,45 +142,66 @@ type AsyncExecutionParams = {
userId: string userId: string
input: any input: any
triggerType: CoreTriggerType triggerType: CoreTriggerType
executionId: string
} }
/**
* Handles async workflow execution by queueing a background job.
* Returns immediately with a 202 Accepted response containing the job ID.
*/
async function handleAsyncExecution(params: AsyncExecutionParams): Promise<NextResponse> { async function handleAsyncExecution(params: AsyncExecutionParams): Promise<NextResponse> {
const { requestId, workflowId, userId, input, triggerType } = params const { requestId, workflowId, userId, input, triggerType, executionId } = params
if (!isTriggerDevEnabled) {
logger.warn(`[${requestId}] Async mode requested but TRIGGER_DEV_ENABLED is false`)
return NextResponse.json(
{ error: 'Async execution is not enabled. Set TRIGGER_DEV_ENABLED=true to use async mode.' },
{ status: 400 }
)
}
const payload: WorkflowExecutionPayload = { const payload: WorkflowExecutionPayload = {
workflowId, workflowId,
userId, userId,
input, input,
triggerType, triggerType,
executionId,
} }
try { try {
const handle = await tasks.trigger('workflow-execution', payload) const jobQueue = await getJobQueue()
const jobId = await jobQueue.enqueue('workflow-execution', payload, {
metadata: { workflowId, userId },
})
logger.info(`[${requestId}] Queued async workflow execution`, { logger.info(`[${requestId}] Queued async workflow execution`, {
workflowId, workflowId,
jobId: handle.id, jobId,
}) })
if (shouldExecuteInline()) {
void (async () => {
try {
await jobQueue.startJob(jobId)
const output = await executeWorkflowJob(payload)
await jobQueue.completeJob(jobId, output)
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error)
logger.error(`[${requestId}] Async workflow execution failed`, {
jobId,
error: errorMessage,
})
try {
await jobQueue.markJobFailed(jobId, errorMessage)
} catch (markFailedError) {
logger.error(`[${requestId}] Failed to mark job as failed`, {
jobId,
error:
markFailedError instanceof Error
? markFailedError.message
: String(markFailedError),
})
}
}
})()
}
return NextResponse.json( return NextResponse.json(
{ {
success: true, success: true,
async: true, async: true,
jobId: handle.id, jobId,
executionId,
message: 'Workflow execution queued', message: 'Workflow execution queued',
statusUrl: `${getBaseUrl()}/api/jobs/${handle.id}`, statusUrl: `${getBaseUrl()}/api/jobs/${jobId}`,
}, },
{ status: 202 } { status: 202 }
) )
@@ -224,6 +269,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
base64MaxBytes, base64MaxBytes,
workflowStateOverride, workflowStateOverride,
stopAfterBlockId, stopAfterBlockId,
runFromBlock,
} = validation.data } = validation.data
// For API key and internal JWT auth, the entire body is the input (except for our control fields) // For API key and internal JWT auth, the entire body is the input (except for our control fields)
@@ -240,6 +286,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
base64MaxBytes, base64MaxBytes,
workflowStateOverride, workflowStateOverride,
stopAfterBlockId: _stopAfterBlockId, stopAfterBlockId: _stopAfterBlockId,
runFromBlock: _runFromBlock,
workflowId: _workflowId, // Also exclude workflowId used for internal JWT auth workflowId: _workflowId, // Also exclude workflowId used for internal JWT auth
...rest ...rest
} = body } = body
@@ -318,6 +365,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
userId: actorUserId, userId: actorUserId,
input, input,
triggerType: loggingTriggerType, triggerType: loggingTriggerType,
executionId,
}) })
} }
@@ -405,6 +453,10 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
if (!enableSSE) { if (!enableSSE) {
logger.info(`[${requestId}] Using non-SSE execution (direct JSON response)`) logger.info(`[${requestId}] Using non-SSE execution (direct JSON response)`)
const timeoutController = createTimeoutAbortController(
preprocessResult.executionTimeout?.sync
)
try { try {
const metadata: ExecutionMetadata = { const metadata: ExecutionMetadata = {
requestId, requestId,
@@ -438,8 +490,38 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
includeFileBase64, includeFileBase64,
base64MaxBytes, base64MaxBytes,
stopAfterBlockId, stopAfterBlockId,
runFromBlock,
abortSignal: timeoutController.signal,
}) })
if (
result.status === 'cancelled' &&
timeoutController.isTimedOut() &&
timeoutController.timeoutMs
) {
const timeoutErrorMessage = getTimeoutErrorMessage(null, timeoutController.timeoutMs)
logger.info(`[${requestId}] Non-SSE execution timed out`, {
timeoutMs: timeoutController.timeoutMs,
})
await loggingSession.markAsFailed(timeoutErrorMessage)
return NextResponse.json(
{
success: false,
output: result.output,
error: timeoutErrorMessage,
metadata: result.metadata
? {
duration: result.metadata.duration,
startTime: result.metadata.startTime,
endTime: result.metadata.endTime,
}
: undefined,
},
{ status: 408 }
)
}
const outputWithBase64 = includeFileBase64 const outputWithBase64 = includeFileBase64
? ((await hydrateUserFilesWithBase64(result.output, { ? ((await hydrateUserFilesWithBase64(result.output, {
requestId, requestId,
@@ -450,9 +532,6 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
const resultWithBase64 = { ...result, output: outputWithBase64 } const resultWithBase64 = { ...result, output: outputWithBase64 }
// Cleanup base64 cache for this execution
await cleanupExecutionBase64Cache(executionId)
const hasResponseBlock = workflowHasResponseBlock(resultWithBase64) const hasResponseBlock = workflowHasResponseBlock(resultWithBase64)
if (hasResponseBlock) { if (hasResponseBlock) {
return createHttpResponseFromBlock(resultWithBase64) return createHttpResponseFromBlock(resultWithBase64)
@@ -460,6 +539,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
const filteredResult = { const filteredResult = {
success: result.success, success: result.success,
executionId,
output: outputWithBase64, output: outputWithBase64,
error: result.error, error: result.error,
metadata: result.metadata metadata: result.metadata
@@ -474,10 +554,17 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
return NextResponse.json(filteredResult) return NextResponse.json(filteredResult)
} catch (error: unknown) { } catch (error: unknown) {
const errorMessage = error instanceof Error ? error.message : 'Unknown error' const errorMessage = error instanceof Error ? error.message : 'Unknown error'
logger.error(`[${requestId}] Non-SSE execution failed: ${errorMessage}`) logger.error(`[${requestId}] Non-SSE execution failed: ${errorMessage}`)
const executionResult = hasExecutionResult(error) ? error.executionResult : undefined const executionResult = hasExecutionResult(error) ? error.executionResult : undefined
await loggingSession.safeCompleteWithError({
totalDurationMs: executionResult?.metadata?.duration,
error: { message: errorMessage },
traceSpans: executionResult?.logs as any,
})
return NextResponse.json( return NextResponse.json(
{ {
success: false, success: false,
@@ -493,6 +580,15 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
}, },
{ status: 500 } { status: 500 }
) )
} finally {
timeoutController.cleanup()
if (executionId) {
try {
await cleanupExecutionBase64Cache(executionId)
} catch (error) {
logger.error(`[${requestId}] Failed to cleanup base64 cache`, { error })
}
}
} }
} }
@@ -506,7 +602,6 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
cachedWorkflowData?.blocks || {} cachedWorkflowData?.blocks || {}
) )
const streamVariables = cachedWorkflowData?.variables ?? (workflow as any).variables const streamVariables = cachedWorkflowData?.variables ?? (workflow as any).variables
const stream = await createStreamingResponse({ const stream = await createStreamingResponse({
requestId, requestId,
workflow: { workflow: {
@@ -524,6 +619,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
workflowTriggerType: triggerType === 'chat' ? 'chat' : 'api', workflowTriggerType: triggerType === 'chat' ? 'chat' : 'api',
includeFileBase64, includeFileBase64,
base64MaxBytes, base64MaxBytes,
timeoutMs: preprocessResult.executionTimeout?.sync,
}, },
executionId, executionId,
}) })
@@ -535,7 +631,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
} }
const encoder = new TextEncoder() const encoder = new TextEncoder()
const abortController = new AbortController() const timeoutController = createTimeoutAbortController(preprocessResult.executionTimeout?.sync)
let isStreamClosed = false let isStreamClosed = false
const stream = new ReadableStream<Uint8Array>({ const stream = new ReadableStream<Uint8Array>({
@@ -567,6 +663,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
blockId: string, blockId: string,
blockName: string, blockName: string,
blockType: string, blockType: string,
executionOrder: number,
iterationContext?: IterationContext iterationContext?: IterationContext
) => { ) => {
logger.info(`[${requestId}] 🔷 onBlockStart called:`, { blockId, blockName, blockType }) logger.info(`[${requestId}] 🔷 onBlockStart called:`, { blockId, blockName, blockType })
@@ -579,6 +676,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
blockId, blockId,
blockName, blockName,
blockType, blockType,
executionOrder,
...(iterationContext && { ...(iterationContext && {
iterationCurrent: iterationContext.iterationCurrent, iterationCurrent: iterationContext.iterationCurrent,
iterationTotal: iterationContext.iterationTotal, iterationTotal: iterationContext.iterationTotal,
@@ -617,6 +715,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
error: callbackData.output.error, error: callbackData.output.error,
durationMs: callbackData.executionTime || 0, durationMs: callbackData.executionTime || 0,
startedAt: callbackData.startedAt, startedAt: callbackData.startedAt,
executionOrder: callbackData.executionOrder,
endedAt: callbackData.endedAt, endedAt: callbackData.endedAt,
...(iterationContext && { ...(iterationContext && {
iterationCurrent: iterationContext.iterationCurrent, iterationCurrent: iterationContext.iterationCurrent,
@@ -644,6 +743,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
output: callbackData.output, output: callbackData.output,
durationMs: callbackData.executionTime || 0, durationMs: callbackData.executionTime || 0,
startedAt: callbackData.startedAt, startedAt: callbackData.startedAt,
executionOrder: callbackData.executionOrder,
endedAt: callbackData.endedAt, endedAt: callbackData.endedAt,
...(iterationContext && { ...(iterationContext && {
iterationCurrent: iterationContext.iterationCurrent, iterationCurrent: iterationContext.iterationCurrent,
@@ -727,10 +827,11 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
onStream, onStream,
}, },
loggingSession, loggingSession,
abortSignal: abortController.signal, abortSignal: timeoutController.signal,
includeFileBase64, includeFileBase64,
base64MaxBytes, base64MaxBytes,
stopAfterBlockId, stopAfterBlockId,
runFromBlock,
}) })
if (result.status === 'paused') { if (result.status === 'paused') {
@@ -763,16 +864,37 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
} }
if (result.status === 'cancelled') { if (result.status === 'cancelled') {
logger.info(`[${requestId}] Workflow execution was cancelled`) if (timeoutController.isTimedOut() && timeoutController.timeoutMs) {
sendEvent({ const timeoutErrorMessage = getTimeoutErrorMessage(null, timeoutController.timeoutMs)
type: 'execution:cancelled', logger.info(`[${requestId}] Workflow execution timed out`, {
timestamp: new Date().toISOString(), timeoutMs: timeoutController.timeoutMs,
executionId, })
workflowId,
data: { await loggingSession.markAsFailed(timeoutErrorMessage)
duration: result.metadata?.duration || 0,
}, sendEvent({
}) type: 'execution:error',
timestamp: new Date().toISOString(),
executionId,
workflowId,
data: {
error: timeoutErrorMessage,
duration: result.metadata?.duration || 0,
},
})
} else {
logger.info(`[${requestId}] Workflow execution was cancelled`)
sendEvent({
type: 'execution:cancelled',
timestamp: new Date().toISOString(),
executionId,
workflowId,
data: {
duration: result.metadata?.duration || 0,
},
})
}
return return
} }
@@ -795,14 +917,26 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
endTime: result.metadata?.endTime || new Date().toISOString(), endTime: result.metadata?.endTime || new Date().toISOString(),
}, },
}) })
// Cleanup base64 cache for this execution
await cleanupExecutionBase64Cache(executionId)
} catch (error: unknown) { } catch (error: unknown) {
const errorMessage = error instanceof Error ? error.message : 'Unknown error' const isTimeout = isTimeoutError(error) || timeoutController.isTimedOut()
logger.error(`[${requestId}] SSE execution failed: ${errorMessage}`) const errorMessage = isTimeout
? getTimeoutErrorMessage(error, timeoutController.timeoutMs)
: error instanceof Error
? error.message
: 'Unknown error'
logger.error(`[${requestId}] SSE execution failed: ${errorMessage}`, { isTimeout })
const executionResult = hasExecutionResult(error) ? error.executionResult : undefined const executionResult = hasExecutionResult(error) ? error.executionResult : undefined
const { traceSpans, totalDuration } = executionResult
? buildTraceSpans(executionResult)
: { traceSpans: [], totalDuration: 0 }
await loggingSession.safeCompleteWithError({
totalDurationMs: totalDuration || executionResult?.metadata?.duration,
error: { message: errorMessage },
traceSpans,
})
sendEvent({ sendEvent({
type: 'execution:error', type: 'execution:error',
@@ -815,20 +949,23 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
}, },
}) })
} finally { } finally {
timeoutController.cleanup()
if (executionId) {
await cleanupExecutionBase64Cache(executionId)
}
if (!isStreamClosed) { if (!isStreamClosed) {
try { try {
controller.enqueue(encoder.encode('data: [DONE]\n\n')) controller.enqueue(encoder.encode('data: [DONE]\n\n'))
controller.close() controller.close()
} catch { } catch {}
// Stream already closed - nothing to do
}
} }
} }
}, },
cancel() { cancel() {
isStreamClosed = true isStreamClosed = true
timeoutController.cleanup()
logger.info(`[${requestId}] Client aborted SSE stream, signalling cancellation`) logger.info(`[${requestId}] Client aborted SSE stream, signalling cancellation`)
abortController.abort() timeoutController.abort()
markExecutionCancelled(executionId).catch(() => {}) markExecutionCancelled(executionId).catch(() => {})
}, },
}) })

View File

@@ -102,7 +102,7 @@ describe('Workspace Invitations API Route', () => {
inArray: vi.fn().mockImplementation((field, values) => ({ type: 'inArray', field, values })), inArray: vi.fn().mockImplementation((field, values) => ({ type: 'inArray', field, values })),
})) }))
vi.doMock('@/executor/utils/permission-check', () => ({ vi.doMock('@/ee/access-control/utils/permission-check', () => ({
validateInvitationsAllowed: vi.fn().mockResolvedValue(undefined), validateInvitationsAllowed: vi.fn().mockResolvedValue(undefined),
InvitationsNotAllowedError: class InvitationsNotAllowedError extends Error { InvitationsNotAllowedError: class InvitationsNotAllowedError extends Error {
constructor() { constructor() {

View File

@@ -21,7 +21,7 @@ import { getFromEmailAddress } from '@/lib/messaging/email/utils'
import { import {
InvitationsNotAllowedError, InvitationsNotAllowedError,
validateInvitationsAllowed, validateInvitationsAllowed,
} from '@/executor/utils/permission-check' } from '@/ee/access-control/utils/permission-check'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -38,7 +38,6 @@ export async function GET(req: NextRequest) {
} }
try { try {
// Get all workspaces where the user has permissions
const userWorkspaces = await db const userWorkspaces = await db
.select({ id: workspace.id }) .select({ id: workspace.id })
.from(workspace) .from(workspace)
@@ -55,10 +54,8 @@ export async function GET(req: NextRequest) {
return NextResponse.json({ invitations: [] }) return NextResponse.json({ invitations: [] })
} }
// Get all workspaceIds where the user is a member
const workspaceIds = userWorkspaces.map((w) => w.id) const workspaceIds = userWorkspaces.map((w) => w.id)
// Find all invitations for those workspaces
const invitations = await db const invitations = await db
.select() .select()
.from(workspaceInvitation) .from(workspaceInvitation)

View File

@@ -14,11 +14,11 @@ import {
ChatMessageContainer, ChatMessageContainer,
EmailAuth, EmailAuth,
PasswordAuth, PasswordAuth,
SSOAuth,
VoiceInterface, VoiceInterface,
} from '@/app/chat/components' } from '@/app/chat/components'
import { CHAT_ERROR_MESSAGES, CHAT_REQUEST_TIMEOUT_MS } from '@/app/chat/constants' import { CHAT_ERROR_MESSAGES, CHAT_REQUEST_TIMEOUT_MS } from '@/app/chat/constants'
import { useAudioStreaming, useChatStreaming } from '@/app/chat/hooks' import { useAudioStreaming, useChatStreaming } from '@/app/chat/hooks'
import SSOAuth from '@/ee/sso/components/sso-auth'
const logger = createLogger('ChatClient') const logger = createLogger('ChatClient')

View File

@@ -1,6 +1,5 @@
export { default as EmailAuth } from './auth/email/email-auth' export { default as EmailAuth } from './auth/email/email-auth'
export { default as PasswordAuth } from './auth/password/password-auth' export { default as PasswordAuth } from './auth/password/password-auth'
export { default as SSOAuth } from './auth/sso/sso-auth'
export { ChatErrorState } from './error-state/error-state' export { ChatErrorState } from './error-state/error-state'
export { ChatHeader } from './header/header' export { ChatHeader } from './header/header'
export { ChatInput } from './input/input' export { ChatInput } from './input/input'

View File

@@ -508,8 +508,10 @@ export default function TemplateDetails({ isWorkspaceContext = false }: Template
setIsApproving(true) setIsApproving(true)
try { try {
const response = await fetch(`/api/templates/${template.id}/approve`, { const response = await fetch(`/api/templates/${template.id}`, {
method: 'POST', method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ status: 'approved' }),
}) })
if (response.ok) { if (response.ok) {
@@ -531,8 +533,10 @@ export default function TemplateDetails({ isWorkspaceContext = false }: Template
setIsRejecting(true) setIsRejecting(true)
try { try {
const response = await fetch(`/api/templates/${template.id}/reject`, { const response = await fetch(`/api/templates/${template.id}`, {
method: 'POST', method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ status: 'rejected' }),
}) })
if (response.ok) { if (response.ok) {
@@ -554,10 +558,11 @@ export default function TemplateDetails({ isWorkspaceContext = false }: Template
setIsVerifying(true) setIsVerifying(true)
try { try {
const endpoint = `/api/creators/${template.creator.id}/verify` const response = await fetch(`/api/creators/${template.creator.id}`, {
const method = template.creator.verified ? 'DELETE' : 'POST' method: 'PUT',
headers: { 'Content-Type': 'application/json' },
const response = await fetch(endpoint, { method }) body: JSON.stringify({ verified: !template.creator.verified }),
})
if (response.ok) { if (response.ok) {
// Refresh page to show updated verification status // Refresh page to show updated verification status

View File

@@ -1,7 +1,7 @@
import { redirect } from 'next/navigation' import { redirect } from 'next/navigation'
import { getSession } from '@/lib/auth' import { getSession } from '@/lib/auth'
import { verifyWorkspaceMembership } from '@/app/api/workflows/utils' import { verifyWorkspaceMembership } from '@/app/api/workflows/utils'
import { getUserPermissionConfig } from '@/executor/utils/permission-check' import { getUserPermissionConfig } from '@/ee/access-control/utils/permission-check'
import { Knowledge } from './knowledge' import { Knowledge } from './knowledge'
interface KnowledgePageProps { interface KnowledgePageProps {
@@ -23,7 +23,6 @@ export default async function KnowledgePage({ params }: KnowledgePageProps) {
redirect('/') redirect('/')
} }
// Check permission group restrictions
const permissionConfig = await getUserPermissionConfig(session.user.id) const permissionConfig = await getUserPermissionConfig(session.user.id)
if (permissionConfig?.hideKnowledgeBaseTab) { if (permissionConfig?.hideKnowledgeBaseTab) {
redirect(`/workspace/${workspaceId}`) redirect(`/workspace/${workspaceId}`)

View File

@@ -104,14 +104,12 @@ function FileCard({ file, isExecutionFile = false, workspaceId }: FileCardProps)
} }
return ( return (
<div className='flex flex-col gap-[8px] rounded-[6px] bg-[var(--surface-1)] px-[10px] py-[8px]'> <div className='flex flex-col gap-[4px] rounded-[6px] bg-[var(--surface-1)] px-[8px] py-[6px]'>
<div className='flex items-center justify-between'> <div className='flex min-w-0 items-center justify-between gap-[8px]'>
<div className='flex items-center gap-[8px]'> <span className='min-w-0 flex-1 truncate font-medium text-[12px] text-[var(--text-secondary)]'>
<span className='truncate font-medium text-[12px] text-[var(--text-secondary)]'> {file.name}
{file.name} </span>
</span> <span className='flex-shrink-0 font-medium text-[12px] text-[var(--text-tertiary)]'>
</div>
<span className='font-medium text-[12px] text-[var(--text-tertiary)]'>
{formatFileSize(file.size)} {formatFileSize(file.size)}
</span> </span>
</div> </div>
@@ -142,20 +140,18 @@ export function FileCards({ files, isExecutionFile = false, workspaceId }: FileC
} }
return ( return (
<div className='flex w-full flex-col gap-[6px] rounded-[6px] bg-[var(--surface-2)] px-[10px] py-[8px]'> <div className='mt-[4px] flex flex-col gap-[6px] rounded-[6px] border border-[var(--border)] bg-[var(--surface-2)] px-[10px] py-[8px] dark:bg-transparent'>
<span className='font-medium text-[12px] text-[var(--text-tertiary)]'> <span className='font-medium text-[12px] text-[var(--text-tertiary)]'>
Files ({files.length}) Files ({files.length})
</span> </span>
<div className='flex flex-col gap-[8px]'> {files.map((file, index) => (
{files.map((file, index) => ( <FileCard
<FileCard key={file.id || `file-${index}`}
key={file.id || `file-${index}`} file={file}
file={file} isExecutionFile={isExecutionFile}
isExecutionFile={isExecutionFile} workspaceId={workspaceId}
workspaceId={workspaceId} />
/> ))}
))}
</div>
</div> </div>
) )
} }

View File

@@ -18,6 +18,7 @@ import {
import { ScrollArea } from '@/components/ui/scroll-area' import { ScrollArea } from '@/components/ui/scroll-area'
import { BASE_EXECUTION_CHARGE } from '@/lib/billing/constants' import { BASE_EXECUTION_CHARGE } from '@/lib/billing/constants'
import { cn } from '@/lib/core/utils/cn' import { cn } from '@/lib/core/utils/cn'
import { formatDuration } from '@/lib/core/utils/formatting'
import { filterHiddenOutputKeys } from '@/lib/logs/execution/trace-spans/trace-spans' import { filterHiddenOutputKeys } from '@/lib/logs/execution/trace-spans/trace-spans'
import { import {
ExecutionSnapshot, ExecutionSnapshot,
@@ -453,7 +454,7 @@ export const LogDetails = memo(function LogDetails({
Duration Duration
</span> </span>
<span className='font-medium text-[13px] text-[var(--text-secondary)]'> <span className='font-medium text-[13px] text-[var(--text-secondary)]'>
{log.duration || '—'} {formatDuration(log.duration, { precision: 2 }) || '—'}
</span> </span>
</div> </div>

View File

@@ -6,11 +6,11 @@ import Link from 'next/link'
import { List, type RowComponentProps, useListRef } from 'react-window' import { List, type RowComponentProps, useListRef } from 'react-window'
import { Badge, buttonVariants } from '@/components/emcn' import { Badge, buttonVariants } from '@/components/emcn'
import { cn } from '@/lib/core/utils/cn' import { cn } from '@/lib/core/utils/cn'
import { formatDuration } from '@/lib/core/utils/formatting'
import { import {
DELETED_WORKFLOW_COLOR, DELETED_WORKFLOW_COLOR,
DELETED_WORKFLOW_LABEL, DELETED_WORKFLOW_LABEL,
formatDate, formatDate,
formatDuration,
getDisplayStatus, getDisplayStatus,
LOG_COLUMNS, LOG_COLUMNS,
StatusBadge, StatusBadge,
@@ -113,7 +113,7 @@ const LogRow = memo(
<div className={`${LOG_COLUMNS.duration.width} ${LOG_COLUMNS.duration.minWidth}`}> <div className={`${LOG_COLUMNS.duration.width} ${LOG_COLUMNS.duration.minWidth}`}>
<Badge variant='default' className='rounded-[6px] px-[9px] py-[2px] text-[12px]'> <Badge variant='default' className='rounded-[6px] px-[9px] py-[2px] text-[12px]'>
{formatDuration(log.duration) || '—'} {formatDuration(log.duration, { precision: 2 }) || '—'}
</Badge> </Badge>
</div> </div>
</div> </div>

View File

@@ -1,6 +1,7 @@
import React from 'react' import React from 'react'
import { format } from 'date-fns' import { format } from 'date-fns'
import { Badge } from '@/components/emcn' import { Badge } from '@/components/emcn'
import { formatDuration } from '@/lib/core/utils/formatting'
import { getIntegrationMetadata } from '@/lib/logs/get-trigger-options' import { getIntegrationMetadata } from '@/lib/logs/get-trigger-options'
import { getBlock } from '@/blocks/registry' import { getBlock } from '@/blocks/registry'
import { CORE_TRIGGER_TYPES } from '@/stores/logs/filters/types' import { CORE_TRIGGER_TYPES } from '@/stores/logs/filters/types'
@@ -362,47 +363,14 @@ export function mapToExecutionLogAlt(log: RawLogResponse): ExecutionLog {
} }
} }
/**
* Format duration for display in logs UI
* If duration is under 1 second, displays as milliseconds (e.g., "500ms")
* If duration is 1 second or more, displays as seconds (e.g., "1.23s")
* @param duration - Duration string (e.g., "500ms") or null
* @returns Formatted duration string or null
*/
export function formatDuration(duration: string | null): string | null {
if (!duration) return null
// Extract numeric value from duration string (e.g., "500ms" -> 500)
const ms = Number.parseInt(duration.replace(/[^0-9]/g, ''), 10)
if (!Number.isFinite(ms)) return duration
if (ms < 1000) {
return `${ms}ms`
}
// Convert to seconds with up to 2 decimal places
const seconds = ms / 1000
return `${seconds.toFixed(2).replace(/\.?0+$/, '')}s`
}
/** /**
* Format latency value for display in dashboard UI * Format latency value for display in dashboard UI
* If latency is under 1 second, displays as milliseconds (e.g., "500ms")
* If latency is 1 second or more, displays as seconds (e.g., "1.23s")
* @param ms - Latency in milliseconds (number) * @param ms - Latency in milliseconds (number)
* @returns Formatted latency string * @returns Formatted latency string
*/ */
export function formatLatency(ms: number): string { export function formatLatency(ms: number): string {
if (!Number.isFinite(ms) || ms <= 0) return '—' if (!Number.isFinite(ms) || ms <= 0) return '—'
return formatDuration(ms, { precision: 2 }) ?? '—'
if (ms < 1000) {
return `${Math.round(ms)}ms`
}
// Convert to seconds with up to 2 decimal places
const seconds = ms / 1000
return `${seconds.toFixed(2).replace(/\.?0+$/, '')}s`
} }
export const formatDate = (dateString: string) => { export const formatDate = (dateString: string) => {

View File

@@ -6,7 +6,7 @@ import { getSession } from '@/lib/auth'
import { verifyWorkspaceMembership } from '@/app/api/workflows/utils' import { verifyWorkspaceMembership } from '@/app/api/workflows/utils'
import type { Template as WorkspaceTemplate } from '@/app/workspace/[workspaceId]/templates/templates' import type { Template as WorkspaceTemplate } from '@/app/workspace/[workspaceId]/templates/templates'
import Templates from '@/app/workspace/[workspaceId]/templates/templates' import Templates from '@/app/workspace/[workspaceId]/templates/templates'
import { getUserPermissionConfig } from '@/executor/utils/permission-check' import { getUserPermissionConfig } from '@/ee/access-control/utils/permission-check'
interface TemplatesPageProps { interface TemplatesPageProps {
params: Promise<{ params: Promise<{

View File

@@ -1,5 +1,5 @@
import { memo, useCallback } from 'react' import { memo, useCallback } from 'react'
import { ArrowLeftRight, ArrowUpDown, Circle, CircleOff, LogOut } from 'lucide-react' import { ArrowLeftRight, ArrowUpDown, Circle, CircleOff, Lock, LogOut, Unlock } from 'lucide-react'
import { Button, Copy, PlayOutline, Tooltip, Trash2 } from '@/components/emcn' import { Button, Copy, PlayOutline, Tooltip, Trash2 } from '@/components/emcn'
import { cn } from '@/lib/core/utils/cn' import { cn } from '@/lib/core/utils/cn'
import { isInputDefinitionTrigger } from '@/lib/workflows/triggers/input-definition-triggers' import { isInputDefinitionTrigger } from '@/lib/workflows/triggers/input-definition-triggers'
@@ -49,6 +49,7 @@ export const ActionBar = memo(
collaborativeBatchRemoveBlocks, collaborativeBatchRemoveBlocks,
collaborativeBatchToggleBlockEnabled, collaborativeBatchToggleBlockEnabled,
collaborativeBatchToggleBlockHandles, collaborativeBatchToggleBlockHandles,
collaborativeBatchToggleLocked,
} = useCollaborativeWorkflow() } = useCollaborativeWorkflow()
const { setPendingSelection } = useWorkflowRegistry() const { setPendingSelection } = useWorkflowRegistry()
const { handleRunFromBlock } = useWorkflowExecution() const { handleRunFromBlock } = useWorkflowExecution()
@@ -84,16 +85,28 @@ export const ActionBar = memo(
) )
}, [blockId, addNotification, collaborativeBatchAddBlocks, setPendingSelection]) }, [blockId, addNotification, collaborativeBatchAddBlocks, setPendingSelection])
const { isEnabled, horizontalHandles, parentId, parentType } = useWorkflowStore( const {
isEnabled,
horizontalHandles,
parentId,
parentType,
isLocked,
isParentLocked,
isParentDisabled,
} = useWorkflowStore(
useCallback( useCallback(
(state) => { (state) => {
const block = state.blocks[blockId] const block = state.blocks[blockId]
const parentId = block?.data?.parentId const parentId = block?.data?.parentId
const parentBlock = parentId ? state.blocks[parentId] : undefined
return { return {
isEnabled: block?.enabled ?? true, isEnabled: block?.enabled ?? true,
horizontalHandles: block?.horizontalHandles ?? false, horizontalHandles: block?.horizontalHandles ?? false,
parentId, parentId,
parentType: parentId ? state.blocks[parentId]?.type : undefined, parentType: parentBlock?.type,
isLocked: block?.locked ?? false,
isParentLocked: parentBlock?.locked ?? false,
isParentDisabled: parentBlock ? !parentBlock.enabled : false,
} }
}, },
[blockId] [blockId]
@@ -161,25 +174,27 @@ export const ActionBar = memo(
{!isNoteBlock && !isInsideSubflow && ( {!isNoteBlock && !isInsideSubflow && (
<Tooltip.Root> <Tooltip.Root>
<Tooltip.Trigger asChild> <Tooltip.Trigger asChild>
<Button <span className='inline-flex'>
variant='ghost' <Button
onClick={(e) => { variant='ghost'
e.stopPropagation() onClick={(e) => {
if (canRunFromBlock && !disabled) { e.stopPropagation()
handleRunFromBlockClick() if (canRunFromBlock && !disabled) {
} handleRunFromBlockClick()
}} }
className={ACTION_BUTTON_STYLES} }}
disabled={disabled || !canRunFromBlock} className={ACTION_BUTTON_STYLES}
> disabled={disabled || !canRunFromBlock}
<PlayOutline className={ICON_SIZE} /> >
</Button> <PlayOutline className={ICON_SIZE} />
</Button>
</span>
</Tooltip.Trigger> </Tooltip.Trigger>
<Tooltip.Content side='top'> <Tooltip.Content side='top'>
{(() => { {(() => {
if (disabled) return getTooltipMessage('Run from block') if (disabled) return getTooltipMessage('Run from block')
if (isExecuting) return 'Execution in progress' if (isExecuting) return 'Execution in progress'
if (!dependenciesSatisfied) return 'Run upstream blocks first' if (!dependenciesSatisfied) return 'Run previous blocks first'
return 'Run from block' return 'Run from block'
})()} })()}
</Tooltip.Content> </Tooltip.Content>
@@ -193,18 +208,54 @@ export const ActionBar = memo(
variant='ghost' variant='ghost'
onClick={(e) => { onClick={(e) => {
e.stopPropagation() e.stopPropagation()
if (!disabled) { // Can't enable if parent is disabled (must enable parent first)
const cantEnable = !isEnabled && isParentDisabled
if (!disabled && !isLocked && !isParentLocked && !cantEnable) {
collaborativeBatchToggleBlockEnabled([blockId]) collaborativeBatchToggleBlockEnabled([blockId])
} }
}} }}
className={ACTION_BUTTON_STYLES} className={ACTION_BUTTON_STYLES}
disabled={disabled} disabled={
disabled || isLocked || isParentLocked || (!isEnabled && isParentDisabled)
}
> >
{isEnabled ? <Circle className={ICON_SIZE} /> : <CircleOff className={ICON_SIZE} />} {isEnabled ? <Circle className={ICON_SIZE} /> : <CircleOff className={ICON_SIZE} />}
</Button> </Button>
</Tooltip.Trigger> </Tooltip.Trigger>
<Tooltip.Content side='top'> <Tooltip.Content side='top'>
{getTooltipMessage(isEnabled ? 'Disable Block' : 'Enable Block')} {isLocked || isParentLocked
? 'Block is locked'
: !isEnabled && isParentDisabled
? 'Parent container is disabled'
: getTooltipMessage(isEnabled ? 'Disable Block' : 'Enable Block')}
</Tooltip.Content>
</Tooltip.Root>
)}
{userPermissions.canAdmin && (
<Tooltip.Root>
<Tooltip.Trigger asChild>
<Button
variant='ghost'
onClick={(e) => {
e.stopPropagation()
// Can't unlock a block if its parent container is locked
if (!disabled && !(isLocked && isParentLocked)) {
collaborativeBatchToggleLocked([blockId])
}
}}
className={ACTION_BUTTON_STYLES}
disabled={disabled || (isLocked && isParentLocked)}
>
{isLocked ? <Unlock className={ICON_SIZE} /> : <Lock className={ICON_SIZE} />}
</Button>
</Tooltip.Trigger>
<Tooltip.Content side='top'>
{isLocked && isParentLocked
? 'Parent container is locked'
: isLocked
? 'Unlock Block'
: 'Lock Block'}
</Tooltip.Content> </Tooltip.Content>
</Tooltip.Root> </Tooltip.Root>
)} )}
@@ -216,17 +267,21 @@ export const ActionBar = memo(
variant='ghost' variant='ghost'
onClick={(e) => { onClick={(e) => {
e.stopPropagation() e.stopPropagation()
if (!disabled) { if (!disabled && !isLocked && !isParentLocked) {
handleDuplicateBlock() handleDuplicateBlock()
} }
}} }}
className={ACTION_BUTTON_STYLES} className={ACTION_BUTTON_STYLES}
disabled={disabled} disabled={disabled || isLocked || isParentLocked}
> >
<Copy className={ICON_SIZE} /> <Copy className={ICON_SIZE} />
</Button> </Button>
</Tooltip.Trigger> </Tooltip.Trigger>
<Tooltip.Content side='top'>{getTooltipMessage('Duplicate Block')}</Tooltip.Content> <Tooltip.Content side='top'>
{isLocked || isParentLocked
? 'Block is locked'
: getTooltipMessage('Duplicate Block')}
</Tooltip.Content>
</Tooltip.Root> </Tooltip.Root>
)} )}
@@ -237,12 +292,12 @@ export const ActionBar = memo(
variant='ghost' variant='ghost'
onClick={(e) => { onClick={(e) => {
e.stopPropagation() e.stopPropagation()
if (!disabled) { if (!disabled && !isLocked && !isParentLocked) {
collaborativeBatchToggleBlockHandles([blockId]) collaborativeBatchToggleBlockHandles([blockId])
} }
}} }}
className={ACTION_BUTTON_STYLES} className={ACTION_BUTTON_STYLES}
disabled={disabled} disabled={disabled || isLocked || isParentLocked}
> >
{horizontalHandles ? ( {horizontalHandles ? (
<ArrowLeftRight className={ICON_SIZE} /> <ArrowLeftRight className={ICON_SIZE} />
@@ -252,7 +307,9 @@ export const ActionBar = memo(
</Button> </Button>
</Tooltip.Trigger> </Tooltip.Trigger>
<Tooltip.Content side='top'> <Tooltip.Content side='top'>
{getTooltipMessage(horizontalHandles ? 'Vertical Ports' : 'Horizontal Ports')} {isLocked || isParentLocked
? 'Block is locked'
: getTooltipMessage(horizontalHandles ? 'Vertical Ports' : 'Horizontal Ports')}
</Tooltip.Content> </Tooltip.Content>
</Tooltip.Root> </Tooltip.Root>
)} )}
@@ -264,19 +321,23 @@ export const ActionBar = memo(
variant='ghost' variant='ghost'
onClick={(e) => { onClick={(e) => {
e.stopPropagation() e.stopPropagation()
if (!disabled && userPermissions.canEdit) { if (!disabled && userPermissions.canEdit && !isLocked && !isParentLocked) {
window.dispatchEvent( window.dispatchEvent(
new CustomEvent('remove-from-subflow', { detail: { blockIds: [blockId] } }) new CustomEvent('remove-from-subflow', { detail: { blockIds: [blockId] } })
) )
} }
}} }}
className={ACTION_BUTTON_STYLES} className={ACTION_BUTTON_STYLES}
disabled={disabled || !userPermissions.canEdit} disabled={disabled || !userPermissions.canEdit || isLocked || isParentLocked}
> >
<LogOut className={ICON_SIZE} /> <LogOut className={ICON_SIZE} />
</Button> </Button>
</Tooltip.Trigger> </Tooltip.Trigger>
<Tooltip.Content side='top'>{getTooltipMessage('Remove from Subflow')}</Tooltip.Content> <Tooltip.Content side='top'>
{isLocked || isParentLocked
? 'Block is locked'
: getTooltipMessage('Remove from Subflow')}
</Tooltip.Content>
</Tooltip.Root> </Tooltip.Root>
)} )}
@@ -286,17 +347,19 @@ export const ActionBar = memo(
variant='ghost' variant='ghost'
onClick={(e) => { onClick={(e) => {
e.stopPropagation() e.stopPropagation()
if (!disabled) { if (!disabled && !isLocked && !isParentLocked) {
collaborativeBatchRemoveBlocks([blockId]) collaborativeBatchRemoveBlocks([blockId])
} }
}} }}
className={ACTION_BUTTON_STYLES} className={ACTION_BUTTON_STYLES}
disabled={disabled} disabled={disabled || isLocked || isParentLocked}
> >
<Trash2 className={ICON_SIZE} /> <Trash2 className={ICON_SIZE} />
</Button> </Button>
</Tooltip.Trigger> </Tooltip.Trigger>
<Tooltip.Content side='top'>{getTooltipMessage('Delete Block')}</Tooltip.Content> <Tooltip.Content side='top'>
{isLocked || isParentLocked ? 'Block is locked' : getTooltipMessage('Delete Block')}
</Tooltip.Content>
</Tooltip.Root> </Tooltip.Root>
</div> </div>
) )

View File

@@ -20,6 +20,9 @@ export interface BlockInfo {
horizontalHandles: boolean horizontalHandles: boolean
parentId?: string parentId?: string
parentType?: string parentType?: string
locked?: boolean
isParentLocked?: boolean
isParentDisabled?: boolean
} }
/** /**
@@ -46,10 +49,17 @@ export interface BlockMenuProps {
showRemoveFromSubflow?: boolean showRemoveFromSubflow?: boolean
/** Whether run from block is available (has snapshot, was executed, not inside subflow) */ /** Whether run from block is available (has snapshot, was executed, not inside subflow) */
canRunFromBlock?: boolean canRunFromBlock?: boolean
/** Whether to disable edit actions (user can't edit OR blocks are locked) */
disableEdit?: boolean disableEdit?: boolean
/** Whether the user has edit permission (ignoring locked state) */
userCanEdit?: boolean
isExecuting?: boolean isExecuting?: boolean
/** Whether the selected block is a trigger (has no incoming edges) */ /** Whether the selected block is a trigger (has no incoming edges) */
isPositionalTrigger?: boolean isPositionalTrigger?: boolean
/** Callback to toggle locked state of selected blocks */
onToggleLocked?: () => void
/** Whether the user has admin permissions */
canAdmin?: boolean
} }
/** /**
@@ -78,13 +88,22 @@ export function BlockMenu({
showRemoveFromSubflow = false, showRemoveFromSubflow = false,
canRunFromBlock = false, canRunFromBlock = false,
disableEdit = false, disableEdit = false,
userCanEdit = true,
isExecuting = false, isExecuting = false,
isPositionalTrigger = false, isPositionalTrigger = false,
onToggleLocked,
canAdmin = false,
}: BlockMenuProps) { }: BlockMenuProps) {
const isSingleBlock = selectedBlocks.length === 1 const isSingleBlock = selectedBlocks.length === 1
const allEnabled = selectedBlocks.every((b) => b.enabled) const allEnabled = selectedBlocks.every((b) => b.enabled)
const allDisabled = selectedBlocks.every((b) => !b.enabled) const allDisabled = selectedBlocks.every((b) => !b.enabled)
const allLocked = selectedBlocks.every((b) => b.locked)
const allUnlocked = selectedBlocks.every((b) => !b.locked)
// Can't unlock blocks that have locked parents
const hasBlockWithLockedParent = selectedBlocks.some((b) => b.locked && b.isParentLocked)
// Can't enable blocks that have disabled parents
const hasBlockWithDisabledParent = selectedBlocks.some((b) => !b.enabled && b.isParentDisabled)
const hasSingletonBlock = selectedBlocks.some( const hasSingletonBlock = selectedBlocks.some(
(b) => (b) =>
@@ -108,6 +127,12 @@ export function BlockMenu({
return 'Toggle Enabled' return 'Toggle Enabled'
} }
const getToggleLockedLabel = () => {
if (allLocked) return 'Unlock'
if (allUnlocked) return 'Lock'
return 'Toggle Lock'
}
return ( return (
<Popover <Popover
open={isOpen} open={isOpen}
@@ -139,7 +164,7 @@ export function BlockMenu({
</PopoverItem> </PopoverItem>
<PopoverItem <PopoverItem
className='group' className='group'
disabled={disableEdit || !hasClipboard} disabled={!userCanEdit || !hasClipboard}
onClick={() => { onClick={() => {
onPaste() onPaste()
onClose() onClose()
@@ -164,13 +189,15 @@ export function BlockMenu({
{!allNoteBlocks && <PopoverDivider />} {!allNoteBlocks && <PopoverDivider />}
{!allNoteBlocks && ( {!allNoteBlocks && (
<PopoverItem <PopoverItem
disabled={disableEdit} disabled={disableEdit || hasBlockWithDisabledParent}
onClick={() => { onClick={() => {
onToggleEnabled() if (!disableEdit && !hasBlockWithDisabledParent) {
onClose() onToggleEnabled()
onClose()
}
}} }}
> >
{getToggleEnabledLabel()} {hasBlockWithDisabledParent ? 'Parent is disabled' : getToggleEnabledLabel()}
</PopoverItem> </PopoverItem>
)} )}
{!allNoteBlocks && !isSubflow && ( {!allNoteBlocks && !isSubflow && (
@@ -195,6 +222,19 @@ export function BlockMenu({
Remove from Subflow Remove from Subflow
</PopoverItem> </PopoverItem>
)} )}
{canAdmin && onToggleLocked && (
<PopoverItem
disabled={hasBlockWithLockedParent}
onClick={() => {
if (!hasBlockWithLockedParent) {
onToggleLocked()
onClose()
}
}}
>
{hasBlockWithLockedParent ? 'Parent is locked' : getToggleLockedLabel()}
</PopoverItem>
)}
{/* Single block actions */} {/* Single block actions */}
{isSingleBlock && <PopoverDivider />} {isSingleBlock && <PopoverDivider />}

View File

@@ -34,6 +34,8 @@ export interface CanvasMenuProps {
canUndo?: boolean canUndo?: boolean
canRedo?: boolean canRedo?: boolean
isInvitationsDisabled?: boolean isInvitationsDisabled?: boolean
/** Whether the workflow has locked blocks (disables auto-layout) */
hasLockedBlocks?: boolean
} }
/** /**
@@ -60,6 +62,7 @@ export function CanvasMenu({
disableEdit = false, disableEdit = false,
canUndo = false, canUndo = false,
canRedo = false, canRedo = false,
hasLockedBlocks = false,
}: CanvasMenuProps) { }: CanvasMenuProps) {
return ( return (
<Popover <Popover
@@ -129,11 +132,12 @@ export function CanvasMenu({
</PopoverItem> </PopoverItem>
<PopoverItem <PopoverItem
className='group' className='group'
disabled={disableEdit} disabled={disableEdit || hasLockedBlocks}
onClick={() => { onClick={() => {
onAutoLayout() onAutoLayout()
onClose() onClose()
}} }}
title={hasLockedBlocks ? 'Unlock blocks to use auto-layout' : undefined}
> >
<span>Auto-layout</span> <span>Auto-layout</span>
<span className='ml-auto opacity-70 group-hover:opacity-100'>L</span> <span className='ml-auto opacity-70 group-hover:opacity-100'>L</span>

View File

@@ -807,7 +807,7 @@ export function Chat() {
const newReservedFields: StartInputFormatField[] = missingStartReservedFields.map( const newReservedFields: StartInputFormatField[] = missingStartReservedFields.map(
(fieldName) => { (fieldName) => {
const defaultType = fieldName === 'files' ? 'files' : 'string' const defaultType = fieldName === 'files' ? 'file[]' : 'string'
return { return {
id: crypto.randomUUID(), id: crypto.randomUUID(),

View File

@@ -1,6 +1,7 @@
import { memo, useCallback, useMemo } from 'react' import { memo, useCallback, useMemo } from 'react'
import ReactMarkdown from 'react-markdown' import ReactMarkdown from 'react-markdown'
import type { NodeProps } from 'reactflow' import type { NodeProps } from 'reactflow'
import remarkBreaks from 'remark-breaks'
import remarkGfm from 'remark-gfm' import remarkGfm from 'remark-gfm'
import { cn } from '@/lib/core/utils/cn' import { cn } from '@/lib/core/utils/cn'
import { BLOCK_DIMENSIONS } from '@/lib/workflows/blocks/block-dimensions' import { BLOCK_DIMENSIONS } from '@/lib/workflows/blocks/block-dimensions'
@@ -305,7 +306,7 @@ function getEmbedInfo(url: string): EmbedInfo | null {
const NoteMarkdown = memo(function NoteMarkdown({ content }: { content: string }) { const NoteMarkdown = memo(function NoteMarkdown({ content }: { content: string }) {
return ( return (
<ReactMarkdown <ReactMarkdown
remarkPlugins={[remarkGfm]} remarkPlugins={[remarkGfm, remarkBreaks]}
components={{ components={{
p: ({ children }: any) => ( p: ({ children }: any) => (
<p className='mb-1 break-words text-[var(--text-primary)] text-sm leading-[1.25rem] last:mb-0'> <p className='mb-1 break-words text-[var(--text-primary)] text-sm leading-[1.25rem] last:mb-0'>

View File

@@ -3,6 +3,7 @@
import { memo, useEffect, useMemo, useRef, useState } from 'react' import { memo, useEffect, useMemo, useRef, useState } from 'react'
import clsx from 'clsx' import clsx from 'clsx'
import { ChevronUp } from 'lucide-react' import { ChevronUp } from 'lucide-react'
import { formatDuration } from '@/lib/core/utils/formatting'
import { CopilotMarkdownRenderer } from '../markdown-renderer' import { CopilotMarkdownRenderer } from '../markdown-renderer'
/** Removes thinking tags (raw or escaped) and special tags from streamed content */ /** Removes thinking tags (raw or escaped) and special tags from streamed content */
@@ -241,15 +242,11 @@ export function ThinkingBlock({
return () => window.clearInterval(intervalId) return () => window.clearInterval(intervalId)
}, [isStreaming, isExpanded, userHasScrolledAway]) }, [isStreaming, isExpanded, userHasScrolledAway])
/** Formats duration in milliseconds to seconds (minimum 1s) */
const formatDuration = (ms: number) => {
const seconds = Math.max(1, Math.round(ms / 1000))
return `${seconds}s`
}
const hasContent = cleanContent.length > 0 const hasContent = cleanContent.length > 0
const isThinkingDone = !isStreaming || hasFollowingContent || hasSpecialTags const isThinkingDone = !isStreaming || hasFollowingContent || hasSpecialTags
const durationText = `${label} for ${formatDuration(duration)}` // Round to nearest second (minimum 1s) to match original behavior
const roundedMs = Math.max(1000, Math.round(duration / 1000) * 1000)
const durationText = `${label} for ${formatDuration(roundedMs)}`
const getStreamingLabel = (lbl: string) => { const getStreamingLabel = (lbl: string) => {
if (lbl === 'Thought') return 'Thinking' if (lbl === 'Thought') return 'Thinking'

View File

@@ -1,6 +1,7 @@
'use client' 'use client'
import { memo, useEffect, useMemo, useRef, useState } from 'react' import { memo, useEffect, useMemo, useRef, useState } from 'react'
import { createLogger } from '@sim/logger'
import clsx from 'clsx' import clsx from 'clsx'
import { ChevronUp, LayoutList } from 'lucide-react' import { ChevronUp, LayoutList } from 'lucide-react'
import Editor from 'react-simple-code-editor' import Editor from 'react-simple-code-editor'
@@ -15,6 +16,7 @@ import {
hasInterrupt as hasInterruptFromConfig, hasInterrupt as hasInterruptFromConfig,
isSpecialTool as isSpecialToolFromConfig, isSpecialTool as isSpecialToolFromConfig,
} from '@/lib/copilot/tools/client/ui-config' } from '@/lib/copilot/tools/client/ui-config'
import { formatDuration } from '@/lib/core/utils/formatting'
import { CopilotMarkdownRenderer } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/copilot-message/components/markdown-renderer' import { CopilotMarkdownRenderer } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/copilot-message/components/markdown-renderer'
import { SmoothStreamingText } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/copilot-message/components/smooth-streaming' import { SmoothStreamingText } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/copilot-message/components/smooth-streaming'
import { ThinkingBlock } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/copilot-message/components/thinking-block' import { ThinkingBlock } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/copilot/components/copilot-message/components/thinking-block'
@@ -848,13 +850,10 @@ const SubagentContentRenderer = memo(function SubagentContentRenderer({
(allParsed.options && Object.keys(allParsed.options).length > 0) (allParsed.options && Object.keys(allParsed.options).length > 0)
) )
const formatDuration = (ms: number) => {
const seconds = Math.max(1, Math.round(ms / 1000))
return `${seconds}s`
}
const outerLabel = getSubagentCompletionLabel(toolCall.name) const outerLabel = getSubagentCompletionLabel(toolCall.name)
const durationText = `${outerLabel} for ${formatDuration(duration)}` // Round to nearest second (minimum 1s) to match original behavior
const roundedMs = Math.max(1000, Math.round(duration / 1000) * 1000)
const durationText = `${outerLabel} for ${formatDuration(roundedMs)}`
const renderCollapsibleContent = () => ( const renderCollapsibleContent = () => (
<> <>
@@ -1259,99 +1258,42 @@ function shouldShowRunSkipButtons(toolCall: CopilotToolCall): boolean {
return false return false
} }
const toolCallLogger = createLogger('CopilotToolCall')
async function sendToolDecision(
toolCallId: string,
status: 'accepted' | 'rejected' | 'background'
) {
try {
await fetch('/api/copilot/confirm', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ toolCallId, status }),
})
} catch (error) {
toolCallLogger.warn('Failed to send tool decision', {
toolCallId,
status,
error: error instanceof Error ? error.message : String(error),
})
}
}
async function handleRun( async function handleRun(
toolCall: CopilotToolCall, toolCall: CopilotToolCall,
setToolCallState: any, setToolCallState: any,
onStateChange?: any, onStateChange?: any,
editedParams?: any editedParams?: any
) { ) {
const instance = getClientTool(toolCall.id) setToolCallState(toolCall, 'executing', editedParams ? { params: editedParams } : undefined)
onStateChange?.('executing')
if (!instance && isIntegrationTool(toolCall.name)) { await sendToolDecision(toolCall.id, 'accepted')
onStateChange?.('executing')
try {
await useCopilotStore.getState().executeIntegrationTool(toolCall.id)
} catch (e) {
setToolCallState(toolCall, 'error', { error: e instanceof Error ? e.message : String(e) })
onStateChange?.('error')
try {
await fetch('/api/copilot/tools/mark-complete', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
id: toolCall.id,
name: toolCall.name,
status: 500,
message: e instanceof Error ? e.message : 'Tool execution failed',
data: { error: e instanceof Error ? e.message : String(e) },
}),
})
} catch {
console.error('[handleRun] Failed to notify backend of tool error:', toolCall.id)
}
}
return
}
if (!instance) return
try {
const mergedParams =
editedParams ||
(toolCall as any).params ||
(toolCall as any).parameters ||
(toolCall as any).input ||
{}
await instance.handleAccept?.(mergedParams)
onStateChange?.('executing')
} catch (e) {
setToolCallState(toolCall, 'error', { error: e instanceof Error ? e.message : String(e) })
}
} }
async function handleSkip(toolCall: CopilotToolCall, setToolCallState: any, onStateChange?: any) { async function handleSkip(toolCall: CopilotToolCall, setToolCallState: any, onStateChange?: any) {
const instance = getClientTool(toolCall.id)
if (!instance && isIntegrationTool(toolCall.name)) {
setToolCallState(toolCall, 'rejected')
onStateChange?.('rejected')
let notified = false
for (let attempt = 0; attempt < 3 && !notified; attempt++) {
try {
const res = await fetch('/api/copilot/tools/mark-complete', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
id: toolCall.id,
name: toolCall.name,
status: 400,
message: 'Tool execution skipped by user',
data: { skipped: true, reason: 'user_skipped' },
}),
})
if (res.ok) {
notified = true
}
} catch (e) {
if (attempt < 2) {
await new Promise((resolve) => setTimeout(resolve, 500))
}
}
}
if (!notified) {
console.error('[handleSkip] Failed to notify backend after 3 attempts:', toolCall.id)
}
return
}
if (instance) {
try {
await instance.handleReject?.()
} catch {}
}
setToolCallState(toolCall, 'rejected') setToolCallState(toolCall, 'rejected')
onStateChange?.('rejected') onStateChange?.('rejected')
await sendToolDecision(toolCall.id, 'rejected')
} }
function getDisplayName(toolCall: CopilotToolCall): string { function getDisplayName(toolCall: CopilotToolCall): string {
@@ -1511,7 +1453,7 @@ export function ToolCall({
// Check if this integration tool is auto-allowed // Check if this integration tool is auto-allowed
// Subscribe to autoAllowedTools so we re-render when it changes // Subscribe to autoAllowedTools so we re-render when it changes
const autoAllowedTools = useCopilotStore((s) => s.autoAllowedTools) const autoAllowedTools = useCopilotStore((s) => s.autoAllowedTools)
const { removeAutoAllowedTool } = useCopilotStore() const { removeAutoAllowedTool, setToolCallState } = useCopilotStore()
const isAutoAllowed = isIntegrationTool(toolCall.name) && autoAllowedTools.includes(toolCall.name) const isAutoAllowed = isIntegrationTool(toolCall.name) && autoAllowedTools.includes(toolCall.name)
// Update edited params when toolCall params change (deep comparison to avoid resetting user edits on ref change) // Update edited params when toolCall params change (deep comparison to avoid resetting user edits on ref change)
@@ -2213,16 +2155,9 @@ export function ToolCall({
<div className='mt-[10px]'> <div className='mt-[10px]'>
<Button <Button
onClick={async () => { onClick={async () => {
try { setToolCallState(toolCall, ClientToolCallState.background)
const instance = getClientTool(toolCall.id) onStateChange?.('background')
instance?.setState?.((ClientToolCallState as any).background) await sendToolDecision(toolCall.id, 'background')
await instance?.markToolComplete?.(
200,
'The user has chosen to move the workflow execution to the background. Check back with them later to know when the workflow execution is complete'
)
forceUpdate({})
onStateChange?.('background')
} catch {}
}} }}
variant='tertiary' variant='tertiary'
title='Move to Background' title='Move to Background'
@@ -2234,21 +2169,9 @@ export function ToolCall({
<div className='mt-[10px]'> <div className='mt-[10px]'>
<Button <Button
onClick={async () => { onClick={async () => {
try { setToolCallState(toolCall, ClientToolCallState.background)
const instance = getClientTool(toolCall.id) onStateChange?.('background')
const elapsedSeconds = instance?.getElapsedSeconds?.() || 0 await sendToolDecision(toolCall.id, 'background')
instance?.setState?.((ClientToolCallState as any).background, {
result: { _elapsedSeconds: elapsedSeconds },
})
const { updateToolCallParams } = useCopilotStore.getState()
updateToolCallParams?.(toolCall.id, { _elapsedSeconds: Math.round(elapsedSeconds) })
await instance?.markToolComplete?.(
200,
`User woke you up after ${Math.round(elapsedSeconds)} seconds`
)
forceUpdate({})
onStateChange?.('background')
} catch {}
}} }}
variant='tertiary' variant='tertiary'
title='Wake' title='Wake'

View File

@@ -114,6 +114,7 @@ export const Copilot = forwardRef<CopilotRef, CopilotProps>(({ panelWidth }, ref
clearPlanArtifact, clearPlanArtifact,
savePlanArtifact, savePlanArtifact,
loadAutoAllowedTools, loadAutoAllowedTools,
resumeActiveStream,
} = useCopilotStore() } = useCopilotStore()
// Initialize copilot // Initialize copilot
@@ -126,6 +127,7 @@ export const Copilot = forwardRef<CopilotRef, CopilotProps>(({ panelWidth }, ref
loadAutoAllowedTools, loadAutoAllowedTools,
currentChat, currentChat,
isSendingMessage, isSendingMessage,
resumeActiveStream,
}) })
// Handle scroll management (80px stickiness for copilot) // Handle scroll management (80px stickiness for copilot)
@@ -421,8 +423,8 @@ export const Copilot = forwardRef<CopilotRef, CopilotProps>(({ panelWidth }, ref
</div> </div>
</div> </div>
{/* Show loading state until fully initialized */} {/* Show loading state until fully initialized, but skip if actively streaming (resume case) */}
{!isInitialized ? ( {!isInitialized && !isSendingMessage ? (
<div className='flex h-full w-full items-center justify-center'> <div className='flex h-full w-full items-center justify-center'>
<div className='flex flex-col items-center gap-3'> <div className='flex flex-col items-center gap-3'>
<p className='text-muted-foreground text-sm'>Loading copilot</p> <p className='text-muted-foreground text-sm'>Loading copilot</p>

View File

@@ -14,6 +14,7 @@ interface UseCopilotInitializationProps {
loadAutoAllowedTools: () => Promise<void> loadAutoAllowedTools: () => Promise<void>
currentChat: any currentChat: any
isSendingMessage: boolean isSendingMessage: boolean
resumeActiveStream: () => Promise<boolean>
} }
/** /**
@@ -32,11 +33,13 @@ export function useCopilotInitialization(props: UseCopilotInitializationProps) {
loadAutoAllowedTools, loadAutoAllowedTools,
currentChat, currentChat,
isSendingMessage, isSendingMessage,
resumeActiveStream,
} = props } = props
const [isInitialized, setIsInitialized] = useState(false) const [isInitialized, setIsInitialized] = useState(false)
const lastWorkflowIdRef = useRef<string | null>(null) const lastWorkflowIdRef = useRef<string | null>(null)
const hasMountedRef = useRef(false) const hasMountedRef = useRef(false)
const hasResumedRef = useRef(false)
/** Initialize on mount - loads chats if needed. Never loads during streaming */ /** Initialize on mount - loads chats if needed. Never loads during streaming */
useEffect(() => { useEffect(() => {
@@ -105,6 +108,16 @@ export function useCopilotInitialization(props: UseCopilotInitializationProps) {
isSendingMessage, isSendingMessage,
]) ])
/** Try to resume active stream on mount - runs early, before waiting for chats */
useEffect(() => {
if (hasResumedRef.current || isSendingMessage) return
hasResumedRef.current = true
// Resume immediately on mount - don't wait for isInitialized
resumeActiveStream().catch((err) => {
logger.warn('[Copilot] Failed to resume active stream', err)
})
}, [isSendingMessage, resumeActiveStream])
/** Load auto-allowed tools once on mount - runs immediately, independent of workflow */ /** Load auto-allowed tools once on mount - runs immediately, independent of workflow */
const hasLoadedAutoAllowedToolsRef = useRef(false) const hasLoadedAutoAllowedToolsRef = useRef(false)
useEffect(() => { useEffect(() => {

Some files were not shown because too many files have changed in this diff Show More