Commit Graph

122 Commits

Author SHA1 Message Date
Vikhyath Mondreti
d93a6f57bc chore(guide): update contributing guide (#4296)
* chore(guide): update contributing guide

* fix md
2026-04-24 18:47:57 -07:00
Waleed
5fba724818 chore(bun): bump bun to 1.3.13 (#4291) 2026-04-24 15:59:10 -07:00
Vikhyath Mondreti
5f0f0edd63 improvement(repo): separate realtime into separate app (#4262)
* improvement(repo): restructuring to make realtime image narrower scoped

* improvements

* chore(repo): rebase fixes and quality improvements for realtime split

Addresses merge-time issues and gaps from the realtime app split:
- Retarget stale vi.mock paths to @sim/workflow-persistence/subblocks
- Restore README branding, fix AGENTS.md script reference
- Restore TSDoc on workflow-persistence subblocks helpers
- Use toError() from @sim/utils/errors in save.ts
- Add vitest config + local mocks so @sim/audit tests run standalone
- Move socket.io-client to devDependencies in apps/realtime
- Add missing package COPY steps to docker/app.Dockerfile
- Add check:boundaries/check:realtime-prune scripts and wire into CI

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(security): consolidate crypto primitives into @sim/security

Move general-purpose crypto primitives out of apps/sim into the
@sim/security package so both apps/sim and apps/realtime can share them.

@sim/security exports (all pure, dependency-free):
  ./compare    safeCompare (constant-time HMAC-wrapped equality)
  ./encryption encrypt/decrypt (AES-256-GCM, iv:cipher:tag format)
  ./hash       sha256Hex
  ./tokens     generateSecureToken (base64url)

Migrate apps/sim call sites to use these + @sim/utils helpers:
  crypto.randomUUID()            -> generateId() from @sim/utils/id
  createHash('sha256').digest    -> sha256Hex
  timingSafeEqual on hashed hex  -> safeCompare
  new Promise(setTimeout)        -> sleep from @sim/utils/helpers

No behavior change: encryption format, digest output, and token
length are preserved exactly.

* refactor(copilot): use toError in remaining otel/finalize sites

Replace the last two `error instanceof Error ? error : new Error(String(error))`
patterns with toError from @sim/utils/errors. Completes the sweep of clean
candidates — no behavior change.

* refactor(security): consolidate HMAC-SHA256 primitives into @sim/security

Adds hmacSha256Hex and hmacSha256Base64 to @sim/security/hmac and migrates
15 webhook providers plus 5 other hot paths (deployment token signing,
outbound webhook requests, workspace notification delivery, notification
test route, Shopify OAuth callback) off bare `createHmac` calls. Secret
parameter accepts `string | Buffer` to cover base64-decoded Svix-style
secrets (Resend) and MS Teams' HMAC scheme. AWS SigV4 signing in S3 and
Textract tools intentionally retains direct `createHmac` usage — its
multi-step key derivation chain doesn't fit a generic helper.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(packages): post-audit test + packaging polish

- Add safeCompare unit tests (identity, length mismatch, hex-nibble diff).
- Add Buffer-secret cases to hmac tests to lock in Svix/MS-Teams contract.
- Declare `reactflow` as a peerDependency on @sim/workflow-types — only used for type imports.
- Add a barrel export to @sim/workflow-persistence for consumers that prefer package-level imports; subpath exports retained.
- Document the data-field invariant in load.ts for loop/parallel subflow patching.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(realtime): address PR review feedback

- Remove redundant SOCKET_PORT=3002 env from Dockerfile runner stage
  (env.PORT already defaults to 3002 via zod schema).
- Reorder PORT fallback so an explicitly-set SOCKET_PORT wins over
  the schema default for PORT; keeps SOCKET_PORT functional as an
  override instead of dead code.
- Add dedicated type-check CI step for @sim/realtime so TS errors
  surface pre-deploy (the Dockerfile runs source TS via Bun and has
  no implicit build-time type check).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(realtime): remove unused SOCKET_PORT env var

SOCKET_PORT has lived in the socket server since the June 2025 refactor
but was never actually set in any deploy config — docker-compose.prod,
helm values/templates, .env.example, and docs all use PORT or the 3002
default exclusively. No self-hoster was ever pointed at SOCKET_PORT, so
removing it is safe.

Simplifies realtime port resolution to `env.PORT` (zod-validated with a
3002 default) and drops the orphaned sim-side schema entry.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-22 23:06:16 -07:00
Waleed
48d5101151 fix(ci): replace dynamic secret access with explicit secret references (#4151)
* fix(ci): replace dynamic secret access with explicit secret references

Resolves CodeQL "Excessive Secrets Exposure" warning by replacing
secrets[matrix.ecr_repo_secret] with conditional expressions that
reference only the specific secrets needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(ci): add explicit ECR_REALTIME guard and use env block for secret injection

- Prevent silent fallthrough to ECR_REALTIME for unrecognized secret keys
- Move build-amd64 secret resolution to env: block matching build-dev pattern

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 22:10:52 -07:00
Vikhyath Mondreti
c71ae49da0 chore(copilot): streaming paths reviewer group (#4144)
* chore(copilot): streaming paths reviewer group

* narrow scope
2026-04-13 18:28:18 -07:00
Theodore Li
6587afb97e fix(ci): Increase build application memory (#4140)
Co-authored-by: Theodore Li <theo@sim.ai>
2026-04-13 20:35:47 -04:00
Siddharth Ganesan
0abcc6e813 improvement(mothership): restructured stream, tool structures, code typing, file write/patch/append tools, timing issues (#4090)
* fix build error

* improvement(mothership): new agent loop (#3920)

* feat(transport): replace shared chat transport with mothership-stream module

* improvement(contracts): regenerate contracts from go

* feat(tools): add tool catalog codegen from go tool contracts

* feat(tools): add tool-executor dispatch framework for sim side tool routing

* feat(orchestrator): rewrite tool dispatch with catalog-driven executor and simplified resume loop

* feat(orchestrator): checkpoint resume flow

* refactor(copilot): consolidate orchestrator into request/ layer

* refactor(mothership): reorganize lib/copilot into structured subdirectories

* refactor(mothership): canonical transcript layer, dead code cleanup, type consolidation

* refactor(mothership): rebase onto latest staging

* refactor(mothership): rename request continue to lifecycle

* feat(trace): add initial version of request traces

* improvement(stream): batch stream from redis

* fix(resume): fix the resume checkpoint

* fix(resume): fix resume client tool

* fix(subagents): subagent resume should join on existing subagent text block

* improvement(reconnect): harden reconnect logic

* fix(superagent): fix superagent integration tools

* improvement(stream): improve stream perf

* Rebase with origin dev

* fix(tests): fix failing test

* fix(build): fix type errors

* fix(build): fix build errors

* fix(build): fix type errors

* feat(mothership): add cli execution

* fix(mothership): fix function execute tests

* Force redeploy

* feat(motheship): add docx support

* feat(mothership): append

* Add deps

* improvement(mothership): docs

* File types

* Add client retry logic

* Fix stream reconnect

* Eager tool streaming

* Fix client side tools

* Security

* Fix shell var injection

* Remove auto injected tasks

* Fix 10mb tool response limit

* Fix trailing leak

* Remove dead tools

* file/folder tools

* Folder tools

* Hide function code inline

* Dont show internal tool result reads

* Fix spacing

* Auth vfs

* Empty folders should show in vfs

* Fix run workflow

* change to node runtime

* revert back to bun runtime

* Fix

* Appends

* Remove debug logs

* Patch

* Fix patch tool

* Temp

* Checkpoint

* File writes

* Fix

* Remove tool truncation limits

* Bad hook

* replace react markdown with streamdown

* Checkpoitn

* fix code block

* fix stream persistence

* temp

* Fix file tools

* tool joining

* cleanup subagent + streaming issues

* streamed text change

* Tool display intetns

* Fix dev

* Fix tests

* Fix dev

* Speed up dev ci

* Add req id

* Fix persistence

* Tool call names

* fix payload accesses

* Fix name

* fix snapshot crash bug

* fix

* Fix

* remove worker code

* Clickable resources

* Options ordering

* Folder vfs

* Restore and mass delete tools

* Fix

* lint

* Update request tracing and skills and handlers

* Fix editable

* fix type error

* Html code

* fix(chat): make inline code inherit parent font size in markdown headers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* improved autolayout

* durable stream for files

* one more fix

* POSSIBLE BREAKAGE: SCROLLING

* Fixes

* Fixes

* Lint fix

* fix(resource): fix resource view disappearing on ats (#4103)

Co-authored-by: Theodore Li <theo@sim.ai>

* Fixes

* feat(mothership): add execution logs as a resource type

Adds `log` as a first-class mothership resource type so copilot can open
and display workflow execution logs as tabs alongside workflows, tables,
files, and knowledge bases.

- Add `log` to MothershipResourceType, all Zod enums, and VALID_RESOURCE_TYPES
- Register log in RESOURCE_REGISTRY (Library icon) and RESOURCE_INVALIDATORS
- Add EmbeddedLog and EmbeddedLogActions components in resource-content
- Export WorkflowOutputSection from log-details for reuse in EmbeddedLog
- Add log resolution branch in open_resource handler via new getLogById service
- Include log id in get_workflow_logs response and extract resources from output
- Exclude log from manual add-resource dropdown (enters via copilot tools only)
- Regenerate copilot contracts after adding log to open_resource Go enum

* Fix perf and message queueing

* Fix abort

* fix(ui): dont delete resource on clearing from context, set resource closed on new task (#4113)

Co-authored-by: Theodore Li <theo@sim.ai>

* improvement(mothership): structure sim side typing

* address comments

* reactive text editor tweaks

* Fix file read and tool call name persistence bug

* Fix code stream + create file opening resource

* fix use chat race + headless trace issues

* Fix type issue

* Fix mothership block req lifecycle

* Fix build

* Move copy reqid

* Fix

* fix(ui): fix resource tag transition from home to task (#4132)

Co-authored-by: Theodore Li <theo@sim.ai>

* Fix persistence

---------

Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Theodore Li <theo@sim.ai>
Co-authored-by: Theodore Li <theodoreqili@gmail.com>
2026-04-13 16:46:35 -07:00
Waleed
05b5588a7b improvement(ci): parallelize Docker builds and fix test timeouts (#4083)
* improvement(ci): parallelize Docker builds with tests and remove duplicate turbo install

* fix(test): use SecureFetchResponse shape in mock instead of standard Response
2026-04-09 15:18:19 -07:00
Waleed
12deb0f5b4 chore(ci): bump actions/checkout to v6 and dorny/paths-filter to v4 (#4082)
* chore(ci): bump actions/checkout to v6 and dorny/paths-filter to v4

* fix(ci): mock secureFetchWithPinnedIP in tools tests to prevent timeouts

* lint
2026-04-09 14:33:11 -07:00
Waleed
df62502903 feat(infra): add dev environment support (#3867)
* feat(infra): add dev environment support

* fix(ci): push :dev ECR tag when building from dev branch

* fix(feature-flags): simplify isHosted subdomain check

* fix(ci,feature-flags): guard URL parse, fix dev AWS creds in images.yml
2026-03-31 15:36:57 -07:00
Waleed
8a481b612d chore(config): clean up bun, turbo, and next.js config (#3788)
* chore(config): clean up bun, turbo, and next.js config

* chore(ci): bump bun to 1.3.11 in dockerfiles and workflows
2026-03-26 13:04:23 -07:00
Waleed
ae887185a1 fix(memory): upgrade bun from 1.3.9 to 1.3.10 (#3441) 2026-03-06 11:35:46 -08:00
Vikhyath Mondreti
28f8e0fd97 fix(kbs): legacy subblock id migration + CI check (#3425)
* fix(kbs): legacy subblock id migration + CI check

* cleanup migration code

* address regex inaccuracy
2026-03-05 12:38:12 -08:00
Waleed
b3424e2047 improvement(ci): add sticky disk caches and bump runner for faster builds (#3373) 2026-02-27 00:12:36 -08:00
Waleed
2fc2e12cb2 feat(slack): added ephemeral message send tool, updated ci, updated docs (#3278)
* feat(slack): added ephemeral message send tool, updated ci, updated docs

* added block kit support

* upgrade turborepo

* added wandConfig for slack block kit

* fix generation type
2026-02-20 16:53:10 -08:00
Waleed
1b8d666c93 fix(build): fix corrupted sticky disk cache on blacksmith (#3273) 2026-02-20 13:03:23 -08:00
Waleed
40a3962c1e fix(docs): update docs and disable i18n action, upgrade turborepo (#3227)
* feat(i18n): change lockfile (#3216)

* fix: update i18n.lock

* feat(docs): enhance documentation with new sections on file handling, form deployment, quick reference, agent skills, and A2A integration

* fix(docs): update docs and disable i18n action

* fix build

---------

Co-authored-by: cherkanov_art <45258907+cherkanovart@users.noreply.github.com>
2026-02-16 00:20:25 -08:00
Waleed
f5dc180d9f fix(memory): upgrade bun from 1.3.3 to 1.3.9 (#3186) 2026-02-10 19:30:31 -08:00
Waleed
12495ef89c feat(ci): auto-create github releases and add workflow permissions (#3009) 2026-01-26 13:28:59 -08:00
Waleed
1952b196a0 fix(releases): improve commit categorization and ci security (#2992)
* fix(releases): improve commit categorization and CI security

* fix(releases): remove redundant update check
2026-01-24 22:33:04 -08:00
Waleed
d7e0d9ba43 fix(i18n): update translations action to run once per week on sunday (#2816) 2026-01-14 11:23:26 -08:00
Vikhyath Mondreti
40a066f39c improvement(FF): CI check to prevent hardcoding of FFs (#2790)
* improvement(FF): CI check to prevent hardcoding of FFs

* revert test change

* add FF lint checks
2026-01-13 10:47:32 -08:00
Waleed
7c0a3c15ac improvement(build): migrate to blacksmith sticky disks for faster builds, other build improvements (#2611) 2025-12-27 18:09:29 -08:00
Vikhyath Mondreti
1720fa8749 feat(compare-schema): ci check to make sure schema.ts never goes out of sync with migrations (#2449)
* feat(compare-schema): ci check to make sure schema.ts never goes out of sync with migrations

* test out of sync [do not merge]

* Revert "test out of sync [do not merge]"

This reverts commit 9771f66b84.
2025-12-18 11:25:19 -08:00
Waleed
323e03520d feat(releases): tag releases to main with version numbers, speed up docker builds (#2337)
* feat(releases): tag releases to main with version numbers, speed up docker builds

* resize runners
2025-12-12 15:57:10 -08:00
Waleed
c864d17991 chore(deps): upgrade to bun v1.3 (#2181)
* chore(deps): upgrade to bun v1.3

* ack PR comments
2025-12-04 00:43:01 -08:00
Emir Karabeg
3dbf0f5679 feat: keyboard navigation; improvement: SEO/GEO; refactor: file structure, unused fonts; fix: chat streaming, notification stack (#2083)
* improvement: panel tabs handler on click

* fix: output break words

* feat: keyboard navigation; improvement: SEO/GEO; refactor: file structure, unused fonts; fix: chat streaming, notification stack

* feat: unresolved value handling on error; fix: layout refresh; refactor: delete old panel

* refactor: control bar delete; improvement: workflow constants; fix: clear notifications keyboard shortcut

* update developers count

* fixed relative imports

---------

Co-authored-by: waleed <walif6@gmail.com>
2025-11-20 19:37:09 -08:00
Waleed
4a0450d1fc fix(embeddings): modified embeddings utils to only index english docs (#2078) 2025-11-20 14:03:49 -08:00
Waleed
00ae718692 improvement(runners): upgrade runners, remove trigger deploy action (#2082) 2025-11-20 13:49:17 -08:00
Waleed
570b8d61f0 improvement(runners): added blacksmith optimizations to workflows and dockerfiles to enhance performance (#2055)
* added blacksmith optimizations to workflows and dockerfiles to enhance performance. please review before pushing to production

* remove cache from and cache to directives from docker based actions, per blacksmith docs

---------

Co-authored-by: Connor Mulholland <connormul@Connors-MacBook-Pro.local>
2025-11-19 13:07:03 -08:00
Waleed
4b37f92f3a improvement(ci): migrate to blacksmith runners (#1879) 2025-11-10 18:53:21 -08:00
Emir Karabeg
eed2072723 feat/platform-v3 (#1698)
* feat(fonts): season replacing geist

* feat(emcnn): created emcn

* feat(sidebar): created new sidebar with header and workflow list

* improvement(sidebar): expanded workflow/folder item text sizing and adjusted button padding

* feat(sidebar): added search UI, updated workflows styling

* improvement: globals styling with antialiased in dark mode only

* feat(sidebar): blocks and triggers ui/ux updated

* refactor(sidebar): moved logic into hooks

* feat(sidebar): improved workflow/folder dragging UI/UX; refactored logic into hooks

* improvement(sidebar): adjusted triggers/blocks padding for header

* improvement(sidebar): dragging hover handler; closed folders by default minus active path

* improvement(sidebar): panel resize logic

* improvement(sidebar): blocks and triggers expanded indicator

* feat(tooltips): new emcn component emerged

* feat(sidebar): workflow list handling updated

* refactor: added cursorrules

* feat(panel): new panel layout

* improvement(workspaces): firname's workspace instead of fn ln's workspace

* feat(platform): panel header, new emcn icons, more button variants, refactor sidebar components

* improvement(emcn): added button variants

* feat(panel): tab system

* feat(copilot): refactor, adjusted welcome and user-input UI/UX

* feat(copilot): baseline user-input ui/ux improvement

* feat(emcn): badge outline variant

* fix: build errors

* feat(copilot): base UI copilot

* refactor(workflow-block): added hooks, components

* feat(design): created design panel and removed isWide

* refactor(subblock): edited components, styling

* feat: emcn, editor

* feat(panel): toolbar, editor

* feat(workflow-block): refactor, adjust base styling

* feat(workflow-block): new block, edge

* feat: workflow-block, connections, action-bar, copilot

* feat: panel, workflow, emcn, workflow block, subblocks; clean copilot

* sim-326: remove remote code execution toggle, hide dropdown for language if E2B is not enabled

* feat: sidebar navigation, tag coloring; refactor: rebased to staging

* fix: build errors

* improvement: subblock styles

* feat: workspaces, terminal, emcn, controls

* feat: delete workflow

* fix: rebased

* fix build errors

---------

Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
2025-11-08 10:58:31 -08:00
Waleed
3a0019bd13 fix(docker): remove prebuild lint check from docker (#1681) 2025-10-17 22:10:25 -07:00
Waleed
c6ef5785c8 fix(i18n): fix SDK and guardrails translation corruption, restore i18n gh action for docs (#1652)
* fix(i18n): fix SDK and guardrails translation corruption

* re-enable i18n gh action
2025-10-15 22:52:48 -07:00
Waleed
d325fdde6c fix(i18n): exclude SDK files from automatic translation to prevent corruption (#1634)
* fix(i18n): exclude SDK files from automatic translation to prevent corruption

* fix(i18n): temporarily disable auto-translate action while fixing translation corruption

* disable i18n action
2025-10-14 19:09:19 -07:00
Waleed
1773530325 fix(sdk): fixed ts sdk publishing job to use later node version (#1619) 2025-10-13 12:13:50 -07:00
Waleed
5a943bca32 fix(ci): pin all workflows and Dockerfiles to Bun v1.2.22 (#1598) 2025-10-10 17:36:06 -07:00
Waleed
923595f57e fix(webhooks): use next public app url instead of request origin for webhook registration (#1596)
* fix(webhooks): use next public app url instead of request origin for webhook registration

* ack PR comments

* ci: pin Bun to v1.2.22 to avoid Bun 1.3 breaking changes
2025-10-10 17:19:51 -07:00
Waleed
1a0fdb32fe chore(docs): update docs (#1578) 2025-10-08 00:55:18 -07:00
Waleed
9d45b8df1e fix(ts-sdk): fix job to publish ts sdk (#1576) 2025-10-08 00:46:30 -07:00
Waleed
86ed32ea10 feat(kb): added json/yaml parser+chunker, added dedicated csv chunker (#1539)
* feat(kb): added json/yaml parser+chunker, added dedicated csv chunker

* ack PR comments

* improved kb upload
2025-10-04 14:59:21 -07:00
Vikhyath Mondreti
35d857ef2e fix(trigger): inject project id env var in correctly (#1520) 2025-10-01 17:16:28 -07:00
Waleed
896f7bb0a0 fix(ci): update trigger.dev ci to only push to staging on merge to staging & for prod as well (#1518) 2025-10-01 13:22:04 -07:00
Siddharth Ganesan
7e6a5dc7e2 Fix/remove trigger promotion (#1513)
* Revert trigger promotion

* Move trigger

* Fix ci
2025-09-30 18:29:28 -07:00
Siddharth Ganesan
c1a3500bde fix(ci): capture correct deployment version output (#1512)
* Capture correct deployment version output

* Add trigger access token to each step

* Use correct arn
2025-09-30 16:36:19 -07:00
Siddharth Ganesan
561b6f2778 fix(ci): fix trigger version capture 2025-09-30 16:20:25 -07:00
Siddharth Ganesan
cdfee16b8a Fix trigger ci creds (#1510) 2025-09-30 14:03:38 -07:00
Siddharth Ganesan
9f6cb1becf fix(ci): trigger permissions 2025-09-30 13:53:02 -07:00
Siddharth Ganesan
dca8745c44 fix(ci): add skip promotion to trigger ci 2025-09-30 13:37:07 -07:00
Siddharth Ganesan
87c00cec6d improvement(ci): trigger.dev pushes (#1506)
* Fix trigger workflow ci

* Update trigger location
2025-09-30 13:22:24 -07:00