Files
sim/AGENTS.md
Vikhyath Mondreti 5f0f0edd63 improvement(repo): separate realtime into separate app (#4262)
* improvement(repo): restructuring to make realtime image narrower scoped

* improvements

* chore(repo): rebase fixes and quality improvements for realtime split

Addresses merge-time issues and gaps from the realtime app split:
- Retarget stale vi.mock paths to @sim/workflow-persistence/subblocks
- Restore README branding, fix AGENTS.md script reference
- Restore TSDoc on workflow-persistence subblocks helpers
- Use toError() from @sim/utils/errors in save.ts
- Add vitest config + local mocks so @sim/audit tests run standalone
- Move socket.io-client to devDependencies in apps/realtime
- Add missing package COPY steps to docker/app.Dockerfile
- Add check:boundaries/check:realtime-prune scripts and wire into CI

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(security): consolidate crypto primitives into @sim/security

Move general-purpose crypto primitives out of apps/sim into the
@sim/security package so both apps/sim and apps/realtime can share them.

@sim/security exports (all pure, dependency-free):
  ./compare    safeCompare (constant-time HMAC-wrapped equality)
  ./encryption encrypt/decrypt (AES-256-GCM, iv:cipher:tag format)
  ./hash       sha256Hex
  ./tokens     generateSecureToken (base64url)

Migrate apps/sim call sites to use these + @sim/utils helpers:
  crypto.randomUUID()            -> generateId() from @sim/utils/id
  createHash('sha256').digest    -> sha256Hex
  timingSafeEqual on hashed hex  -> safeCompare
  new Promise(setTimeout)        -> sleep from @sim/utils/helpers

No behavior change: encryption format, digest output, and token
length are preserved exactly.

* refactor(copilot): use toError in remaining otel/finalize sites

Replace the last two `error instanceof Error ? error : new Error(String(error))`
patterns with toError from @sim/utils/errors. Completes the sweep of clean
candidates — no behavior change.

* refactor(security): consolidate HMAC-SHA256 primitives into @sim/security

Adds hmacSha256Hex and hmacSha256Base64 to @sim/security/hmac and migrates
15 webhook providers plus 5 other hot paths (deployment token signing,
outbound webhook requests, workspace notification delivery, notification
test route, Shopify OAuth callback) off bare `createHmac` calls. Secret
parameter accepts `string | Buffer` to cover base64-decoded Svix-style
secrets (Resend) and MS Teams' HMAC scheme. AWS SigV4 signing in S3 and
Textract tools intentionally retains direct `createHmac` usage — its
multi-step key derivation chain doesn't fit a generic helper.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(packages): post-audit test + packaging polish

- Add safeCompare unit tests (identity, length mismatch, hex-nibble diff).
- Add Buffer-secret cases to hmac tests to lock in Svix/MS-Teams contract.
- Declare `reactflow` as a peerDependency on @sim/workflow-types — only used for type imports.
- Add a barrel export to @sim/workflow-persistence for consumers that prefer package-level imports; subpath exports retained.
- Document the data-field invariant in load.ts for loop/parallel subflow patching.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(realtime): address PR review feedback

- Remove redundant SOCKET_PORT=3002 env from Dockerfile runner stage
  (env.PORT already defaults to 3002 via zod schema).
- Reorder PORT fallback so an explicitly-set SOCKET_PORT wins over
  the schema default for PORT; keeps SOCKET_PORT functional as an
  override instead of dead code.
- Add dedicated type-check CI step for @sim/realtime so TS errors
  surface pre-deploy (the Dockerfile runs source TS via Bun and has
  no implicit build-time type check).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(realtime): remove unused SOCKET_PORT env var

SOCKET_PORT has lived in the socket server since the June 2025 refactor
but was never actually set in any deploy config — docker-compose.prod,
helm values/templates, .env.example, and docs all use PORT or the 3002
default exclusively. No self-hoster was ever pointed at SOCKET_PORT, so
removing it is safe.

Simplifies realtime port resolution to `env.PORT` (zod-validated with a
3002 default) and drops the orphaned sim-side schema entry.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-22 23:06:16 -07:00

14 KiB

Sim Development Guidelines

You are a professional software engineer. All code must follow best practices: accurate, readable, clean, and efficient.

Global Standards

  • Logging: Import createLogger from @sim/logger. Use logger.info, logger.warn, logger.error instead of console.log
  • Comments: Use TSDoc for documentation. No ==== separators. No non-TSDoc comments
  • Styling: Never update global styles. Keep all styling local to components
  • ID Generation: Never use crypto.randomUUID(), nanoid, or uuid package. Use generateId() (UUID v4) or generateShortId() (compact) from @sim/utils/id
  • Package Manager: Use bun and bunx, not npm and npx

Architecture

Core Principles

  1. Single Responsibility: Each component, hook, store has one clear purpose
  2. Composition Over Complexity: Break down complex logic into smaller pieces
  3. Type Safety First: TypeScript interfaces for all props, state, return types
  4. Predictable State: Zustand for global state, useState for UI-only concerns

Root Structure

apps/
├── sim/                    # Next.js app (UI + API routes + workflow editor)
│   ├── app/                # Next.js app router (pages, API routes)
│   ├── blocks/             # Block definitions and registry
│   ├── components/         # Shared UI (emcn/, ui/)
│   ├── executor/           # Workflow execution engine
│   ├── hooks/              # Shared hooks (queries/, selectors/)
│   ├── lib/                # App-wide utilities
│   ├── providers/          # LLM provider integrations
│   ├── stores/             # Zustand stores
│   ├── tools/              # Tool definitions
│   └── triggers/           # Trigger definitions
└── realtime/               # Bun Socket.IO server (collaborative canvas)
    └── src/                # auth, config, database, handlers, middleware,
                            # rooms, routes, internal/webhook-cleanup.ts

packages/
├── audit/                  # @sim/audit — recordAudit + AuditAction + AuditResourceType
├── auth/                   # @sim/auth — @sim/auth/verify (shared Better Auth verifier)
├── db/                     # @sim/db — drizzle schema + client
├── logger/                 # @sim/logger
├── realtime-protocol/      # @sim/realtime-protocol — socket operation constants + zod schemas
├── security/               # @sim/security — safeCompare
├── tsconfig/               # shared tsconfig presets
├── utils/                  # @sim/utils
├── workflow-authz/         # @sim/workflow-authz — authorizeWorkflowByWorkspacePermission
├── workflow-persistence/   # @sim/workflow-persistence — raw load/save + subflow helpers
└── workflow-types/         # @sim/workflow-types — pure BlockState/Loop/Parallel/... types

Package boundaries

  • apps/* → packages/* only. Packages never import from apps/*.
  • Each package has explicit subpath exports maps; no barrels that accidentally pull in heavy halves.
  • apps/realtime intentionally avoids Next.js, React, the block/tool registry, provider SDKs, and the executor. CI enforces this via scripts/check-monorepo-boundaries.ts and scripts/check-realtime-prune-graph.ts.
  • Auth is shared across services via the Better Auth "Shared Database Session" pattern: both apps read the same BETTER_AUTH_SECRET and point at the same DB via @sim/db.

Naming Conventions

  • Components: PascalCase (WorkflowList)
  • Hooks: use prefix (useWorkflowOperations)
  • Files: kebab-case (workflow-list.tsx)
  • Stores: stores/feature/store.ts
  • Constants: SCREAMING_SNAKE_CASE
  • Interfaces: PascalCase with suffix (WorkflowListProps)

Imports

Always use absolute imports. Never use relative imports.

// ✓ Good
import { useWorkflowStore } from '@/stores/workflows/store'

// ✗ Bad
import { useWorkflowStore } from '../../../stores/workflows/store'

Use barrel exports (index.ts) when a folder has 3+ exports. Do not re-export from non-barrel files; import directly from the source.

Import Order

  1. React/core libraries
  2. External libraries
  3. UI components (@/components/emcn, @/components/ui)
  4. Utilities (@/lib/...)
  5. Stores (@/stores/...)
  6. Feature imports
  7. CSS imports

Use import type { X } for type-only imports.

TypeScript

  1. No any - Use proper types or unknown with type guards
  2. Always define props interface for components
  3. as const for constant objects/arrays
  4. Explicit ref types: useRef<HTMLDivElement>(null)

Components

'use client' // Only if using hooks

const CONFIG = { SPACING: 8 } as const

interface ComponentProps {
  requiredProp: string
  optionalProp?: boolean
}

export function Component({ requiredProp, optionalProp = false }: ComponentProps) {
  // Order: refs → external hooks → store hooks → custom hooks → state → useMemo → useCallback → useEffect → return
}

Extract when: 50+ lines, used in 2+ files, or has own state/logic. Keep inline when: < 10 lines, single use, purely presentational.

Hooks

interface UseFeatureProps { id: string }

export function useFeature({ id }: UseFeatureProps) {
  const idRef = useRef(id)
  const [data, setData] = useState<Data | null>(null)
  
  useEffect(() => { idRef.current = id }, [id])
  
  const fetchData = useCallback(async () => { ... }, []) // Empty deps when using refs
  
  return { data, fetchData }
}

Zustand Stores

Stores live in stores/. Complex stores split into store.ts + types.ts.

import { create } from 'zustand'
import { devtools } from 'zustand/middleware'

const initialState = { items: [] as Item[] }

export const useFeatureStore = create<FeatureState>()(
  devtools(
    (set, get) => ({
      ...initialState,
      setItems: (items) => set({ items }),
      reset: () => set(initialState),
    }),
    { name: 'feature-store' }
  )
)

Use devtools middleware. Use persist only when data should survive reload with partialize to persist only necessary state.

React Query

All React Query hooks live in hooks/queries/. All server state must go through React Query — never use useState + fetch in components for data fetching or mutations.

Query Key Factory

Every file must have a hierarchical key factory with an all root key and intermediate plural keys for prefix invalidation:

export const entityKeys = {
  all: ['entity'] as const,
  lists: () => [...entityKeys.all, 'list'] as const,
  list: (workspaceId?: string) => [...entityKeys.lists(), workspaceId ?? ''] as const,
  details: () => [...entityKeys.all, 'detail'] as const,
  detail: (id?: string) => [...entityKeys.details(), id ?? ''] as const,
}

Query Hooks

  • Every queryFn must forward signal for request cancellation
  • Every query must have an explicit staleTime
  • Use keepPreviousData only on variable-key queries (where params change), never on static keys
export function useEntityList(workspaceId?: string) {
  return useQuery({
    queryKey: entityKeys.list(workspaceId),
    queryFn: ({ signal }) => fetchEntities(workspaceId as string, signal),
    enabled: Boolean(workspaceId),
    staleTime: 60 * 1000,
    placeholderData: keepPreviousData, // OK: workspaceId varies
  })
}

Mutation Hooks

  • Use targeted invalidation (entityKeys.lists()) not broad (entityKeys.all) when possible
  • For optimistic updates: use onSettled (not onSuccess) for cache reconciliation — onSettled fires on both success and error
  • Don't include mutation objects in useCallback deps — .mutate() is stable in TanStack Query v5
export function useUpdateEntity() {
  const queryClient = useQueryClient()
  return useMutation({
    mutationFn: async (variables) => { /* ... */ },
    onMutate: async (variables) => {
      await queryClient.cancelQueries({ queryKey: entityKeys.detail(variables.id) })
      const previous = queryClient.getQueryData(entityKeys.detail(variables.id))
      queryClient.setQueryData(entityKeys.detail(variables.id), /* optimistic */)
      return { previous }
    },
    onError: (_err, variables, context) => {
      queryClient.setQueryData(entityKeys.detail(variables.id), context?.previous)
    },
    onSettled: (_data, _error, variables) => {
      queryClient.invalidateQueries({ queryKey: entityKeys.lists() })
      queryClient.invalidateQueries({ queryKey: entityKeys.detail(variables.id) })
    },
  })
}

Styling

Use Tailwind only, no inline styles. Use cn() from @/lib/utils for conditional classes.

<div className={cn('base-classes', isActive && 'active-classes')} />

EMCN Components

Import from @/components/emcn, never from subpaths (except CSS files). Use CVA when 2+ variants exist.

Testing

Use Vitest. Test files: feature.tsfeature.test.ts. See .cursor/rules/sim-testing.mdc for full details.

Global Mocks (vitest.setup.ts)

@sim/db, drizzle-orm, @sim/logger, @/blocks/registry, @trigger.dev/sdk, and store mocks are provided globally. Do NOT re-mock them unless overriding behavior.

Standard Test Pattern

/**
 * @vitest-environment node
 */
import { createMockRequest } from '@sim/testing'
import { beforeEach, describe, expect, it, vi } from 'vitest'

const { mockGetSession } = vi.hoisted(() => ({
  mockGetSession: vi.fn(),
}))

vi.mock('@/lib/auth', () => ({
  auth: { api: { getSession: vi.fn() } },
  getSession: mockGetSession,
}))

import { GET } from '@/app/api/my-route/route'

describe('my route', () => {
  beforeEach(() => {
    vi.clearAllMocks()
    mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
  })
  it('returns data', async () => { ... })
})

Performance Rules

  • NEVER use vi.resetModules() + vi.doMock() + await import() — use vi.hoisted() + vi.mock() + static imports
  • NEVER use vi.importActual() — mock everything explicitly
  • NEVER use mockAuth(), mockConsoleLogger(), setupCommonApiMocks() from @sim/testing — they use vi.doMock() internally
  • Mock heavy deps (@/blocks, @/tools/registry, @/triggers) in tests that don't need them
  • Use @vitest-environment node unless DOM APIs are needed (window, document, FormData)
  • Avoid real timers — use 1ms delays or vi.useFakeTimers()

Use @sim/testing mocks/factories over local test data.

Utils Rules

  • Never create utils.ts for single consumer - inline it
  • Create utils.ts when 2+ files need the same helper
  • Check existing sources in lib/ before duplicating

Adding Integrations

New integrations require: ToolsBlockIcon → (optional) Trigger

Always look up the service's API docs first.

1. Tools (tools/{service}/)

tools/{service}/
├── index.ts      # Barrel export
├── types.ts      # Params/response types
└── {action}.ts   # Tool implementation

Tool structure:

export const serviceTool: ToolConfig<Params, Response> = {
  id: 'service_action',
  name: 'Service Action',
  description: '...',
  version: '1.0.0',
  oauth: { required: true, provider: 'service' },
  params: { /* ... */ },
  request: { url: '/api/tools/service/action', method: 'POST', ... },
  transformResponse: async (response) => { /* ... */ },
  outputs: { /* ... */ },
}

Register in tools/registry.ts.

2. Block (blocks/blocks/{service}.ts)

export const ServiceBlock: BlockConfig = {
  type: 'service',
  name: 'Service',
  description: '...',
  category: 'tools',
  bgColor: '#hexcolor',
  icon: ServiceIcon,
  subBlocks: [ /* see SubBlock Properties */ ],
  tools: { access: ['service_action'], config: { tool: (p) => `service_${p.operation}`, params: (p) => ({ /* type coercions here */ }) } },
  inputs: { /* ... */ },
  outputs: { /* ... */ },
}

Register in blocks/registry.ts (alphabetically).

Important: tools.config.tool runs during serialization (before variable resolution). Never do Number() or other type coercions there — dynamic references like <Block.output> will be destroyed. Use tools.config.params for type coercions (it runs during execution, after variables are resolved).

SubBlock Properties:

{
  id: 'field', title: 'Label', type: 'short-input', placeholder: '...',
  required: true,                    // or condition object
  condition: { field: 'op', value: 'send' },  // show/hide
  dependsOn: ['credential'],         // clear when dep changes
  mode: 'basic',                     // 'basic' | 'advanced' | 'both' | 'trigger'
}

condition examples:

  • { field: 'op', value: 'send' } - show when op === 'send'
  • { field: 'op', value: ['a','b'] } - show when op is 'a' OR 'b'
  • { field: 'op', value: 'x', not: true } - show when op !== 'x'
  • { field: 'op', value: 'x', not: true, and: { field: 'type', value: 'dm', not: true } } - complex

dependsOn: ['field'] or { all: ['a'], any: ['b', 'c'] }

File Input Pattern (basic/advanced mode):

// Basic: file-upload UI
{ id: 'uploadFile', type: 'file-upload', canonicalParamId: 'file', mode: 'basic' },
// Advanced: reference from other blocks
{ id: 'fileRef', type: 'short-input', canonicalParamId: 'file', mode: 'advanced' },

In tools.config.tool, normalize with:

import { normalizeFileInput } from '@/blocks/utils'
const file = normalizeFileInput(params.uploadFile || params.fileRef, { single: true })
if (file) params.file = file

For file uploads, create an internal API route (/api/tools/{service}/upload) that uses downloadFileFromStorage to get file content from UserFile objects.

3. Icon (components/icons.tsx)

export function ServiceIcon(props: SVGProps<SVGSVGElement>) {
  return <svg {...props}>/* SVG from brand assets */</svg>
}

4. Trigger (triggers/{service}/) - Optional

triggers/{service}/
├── index.ts      # Barrel export
├── webhook.ts    # Webhook handler
└── {event}.ts    # Event-specific handlers

Register in triggers/registry.ts.

Integration Checklist

  • Look up API docs
  • Create tools/{service}/ with types and tools
  • Register tools in tools/registry.ts
  • Add icon to components/icons.tsx
  • Create block in blocks/blocks/{service}.ts
  • Register block in blocks/registry.ts
  • (Optional) Create and register triggers
  • (If file uploads) Create internal API route with downloadFileFromStorage
  • (If file uploads) Use normalizeFileInput in block config