Commit Graph

76 Commits

Author SHA1 Message Date
Waleed
7b7586d093 fix(memory-util): fixed unbounded array of gmail/outlook pollers causing high memory util, added missing db indexes/removed unused ones, auto-disable schedules/webhooks after 10 consecutive failures (#2115)
* fix(memory-util): fixed unbounded array of gmail/outlook pollers causing high memory util, added missing db indexes/removed unused ones, auto-disable schedules/webhooks after 10 consecutive failures

* ack PR comments

* ack
2025-11-25 14:15:46 -08:00
Siddharth Ganesan
ddd3219126 improvement(copilot): v0.2 (#2086)
* Edit workflow returns workflow

* Condense workflkow conosle

* Limit console results to 2

* mark checkoff v1

* Mark and checkoff todo

* Fixes

* Plan mode

* fix agent output

* broken

* Fixes

* Fix diff mode persist

* Fix diff coloring

* Undo/redo

* Checkpoint udno redo

* fix(templates): fix templates details page (#1942)

* Fix template details

* Fix deps

* fix(templates-details): restore approval feature, and keep details UI consistent, smoothen out creation of profile (#1943)

* fix(templates): view current ui

* update UI to be less cluttered

* make state management for creating user profile smoother

* fix autoselect logic

* fix lint

* fix(landing): need to propagate landing page copilot prompt (#1944)

* fix(wand): subblocks should not be overwritten after wand gen (#1946)

* fix(settings): fix broken api keys, help modal, logs, workflow renaming (#1945)

* fix(settings): fix broken api keys, help modal, logs, workflow renaming

* fix build

* cleanup

* use emcn

* fix(files): changed file input value sample from string -> object (#1947)

* improvement: usage-indicator UI (#1948)

* fix(deploy): fix button (#1949)

* fix(executor): consolidate execution hooks (#1950)

* fix(autoconnect): should check if triggermode is set from the toolbar drag event directly (#1951)

* improvement: templates styling (#1952)

* improvement: template use button (#1954)

* feat(newgifs): added new gifs (#1953)

* new gifs

* changed wording

* changed wording

* lowercase

* changed wording

* remove blog stuff

---------

Co-authored-by: aadamgough <adam@sim.ai>
Co-authored-by: waleed <walif6@gmail.com>

* feat(drizzle): added ods for analytics from drizzle (#1956)

* feat(drizzle): added ods for analytics from drizzle

* clean

* fix(sheets): file selector had incorrect provider (#1958)

* feat(docs): added docs analytics drizzle ods (#1957)

* feat(docs): added docs analytics drizzle ods

* fix build

* fix provider for docs selector (#1959)

* fix(custom-tools): updates to legacy + copilot generated custom tools  (#1960)

* fix(custom-tools): updates to existing tools

* don't reorder custom tools in modal based on edit time

* restructure custom tools to persist copilot generated tools

* fix tests

* fix(slack): remove update message incorrect canonical param (#1964)

* fix(slack): send message tool

* add logging

* add more logs

* add more logs

* fix canonical param

* improvement(templates): add share button, serve public templates routes for unauthenticated users and workspace one for authenticated users, improve settings style and organization (#1962)

* improvement(templates): add share button, serve public templates routes for unauthenticated users and workspace one for authenticated users, improve settings style and organization

* fix lint

---------

Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>

* fix(onedrive): incorrect canonical param (#1966)

* fix(onedrive): incorrect canonical param

* fix download file

* fix

* fix

* Revert "fix"

This reverts commit f68ccd75fd.

* Revert "fix"

This reverts commit f3d8acee7d.

* fix(executor): streaming after tool calls (#1963)

* Provider changes

* Fix lint

* fix(code): readd wand to code subblock (#1969)

* fix(cmd-k): z-index + reoder tools, triggers (#1970)

* fix(cmd-k): z-index + reoder tools, triggers

* fix more z-index styling

* fix(executor): streaming response format (#1972)

* fix(workflow-block): fix redeploy header to not repeatedly show redeploy when redeploy is not necessary (#1973)

* fix(workflow-block): fix redeploy header to not repeatedly show redeploy when redeploy is not necessary

* cleanup

* improvement(platform): chat, emcn, terminal, usage-limit (#1974)

* improvement(usage-indicator): layout

* improvement: expand default terminal height

* fix: swap workflow block ports

* improvement: chat initial positioning

* improvement(chat): display; improvement(emcn): popover attributes

* fix lint

* feat(settings): added reactquery for settings, removed zustand stores, added apollo, added workflow block selector dropdown search, added add environment variable option to empty env var dropdown (#1971)

* feat(settings): added reactquery for settings, removed zustand stores, added apollo, added workflow block selector dropdown search, added add environment variable option to empty env var dropdown

* fix delete dialog for copilot keys

* simplify combobox

* fix more z indices

* consolidated duplicate hooks

---------

Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>

* fix(copilot-subflows): copilot-added subflows id mismatch (#1977)

* feat(i18n): update translations (#1978)

* feat(i18n): update translations

* fix build

---------

Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com>

* fix(logs): show block inputs (#1979)

* Fix executor lgos block inputs

* Fix Comment

* fix(onedrive): parse array values correctly (#1981)

* fix(onedrive): parse array values correctly

* fix onedrive

* fix

* fix onedrive input parsing by reusing code subblock

* fix type

* feat(files): add presigned URL generation support for execution files (#1980)

* fix(popovers): billed account + async example command (#1982)

* fix(settings): update usage data in settings > subs to use reactquery hooks (#1983)

* fix(settings): update usage data in settings > subs to use reactquery hooks

* standardize usage pills calculation

* fix(output-selector): z-index in chat deploy modal (#1984)

* improvement(logs): improved logs search (#1985)

* improvement(logs): improved logs search

* more

* ack PR comments

* feat(slack): added slack full message object in response (#1987)

* feat(slack): add better error messages, reminder to add bot to app (#1990)

* feat(i18n): update translations (#1989)

Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com>

* fix(landing): hero stripe icon (#1988)

* Temp

* Update

* Update

* Update

* Progress

* Diff store fixes

* simplify sockets ops for diff store

* Search patterns tool

* Better tool calls

* Fix sanitizationg

* Context window display

* fix(variables): fix double stringification (#1991)

* improvement(variables): support dot notation for nested objects (#1992)

* improvement(tanstack): migrate multiple stores (#1994)

* improvement(tanstack): migrate folders, knowledge to tanstack

* fix types

* fix(folders): duplicate (#1996)

* fix(variables): fix variables block json resolution (#1997)

* Improvement(ui/ux): signup, command-list, cursors, search modal, workflow runs, usage indicator (#1998)

* improvement: signup loading, command-list, cursors, search modal ordering

* improvement: workflow runs, search modal

* improvement(usage-indicator): ui/ux

* test(pr): hackathon (#1999)

* test(pr): github trigger (#2000)

* fix(usage-indicator): conditional rendering, upgrade, and ui/ux (#2001)

* fix: usage-limit indicator and render conditonally on is billing enabled

* fix: upgrade render

* fix(notes): fix notes, tighten spacing, update deprecated zustand function, update use mention data to ignore block positon (#2002)

* fix(pdfs): use unpdf instead of pdf-parse (#2004)

* fix(modals): fix z-index for various modals and output selector and variables (#2005)

* fix(condition): treat condition input the same as the code subblock (#2006)

* feat(models): added gpt-5.1 (#2007)

* improvement: runpath edges, blocks, active (#2008)

* feat(i18n): update translations (#2009)

* fix(triggers): check triggermode and consolidate block type (#2011)

* fix(triggers): disabled trigger shouldn't be added to dag (#2012)

* Fix disabled blocks

* Comments

* Fix api/chat trigger not found message

* fix(tags): only show start block upstream if is ancestor (#2013)

* fix(variables): Fix resolution on double < (#2016)

* Fix variable <>

* Ling

* Clean

* feat(billing): add notif for first failed payment, added upgrade email from free, updated providers that supported granular tool control to support them, fixed envvar popover, fixed redirect to wrong workspace after oauth connect (#2015)

* feat(billing): add notif for first failed payment, added upgrade email from free, updated providers that supported granular tool control to support them, fixed envvar popover, fixed redirect to wrong workspace after oauth connect

* fix build

* ack PR comments

* feat(performance): added reactquery hooks for workflow operations, for logs, fixed logs reloading, fix subscription UI (#2017)

* feat(performance): added reactquery hooks for workflow operations, for logs, fixed logs reloading, fix subscription UI

* use useInfiniteQuery for logs fetching

* fix(copilot): run workflow supports input format and fix run id (#2018)

* fix(router): fix error edge in router block + fix source handle problem (#2019)

* Fix router block error port handling

* Remove comment

* Fix edge execution

* improvement: code subblock, action bar, connections (#2024)

* improvement: action bar, connections

* fix: code block draggable resize

* fix(response): fix response block http format (#2027)

* Fix response block

* Lint

* fix(notes): fix notes block spacing, additional logs for billing transfer route (#2029)

* fix(usage-data): refetch on usage limit update in settings (#2032)

* fix(overage): fix pill calculation in the usage indicator to be consistent across views (#2034)

* fix(workflows): fixed workflow loading in without start block, added templates RQ hook, cleaned up unused templates code (#2035)

* fix(triggers): dedup + not surfacing deployment status log (#2033)

* fix(triggers): dedup + not surfacing deployment status log

* fix ms teams

* change to microsoftteams

* Revert "change to microsoftteams"

This reverts commit 217f808641.

* fix

* fix

* fix provider name

* fix oauth for msteams

* improvement(undo-redo): expand undo-redo store to store 100 ops instead of 15 (#2036)

* improvement(undo-redo): expand undo-redo store to store 100 ops instead of 15

* prevent undo-redo from interfering with subblock browser text undo

* improvement(docs): remove copy page from mobile view on docs (#2037)

* improvement(docs): remove copy page from mobile view on docs

* bring title and pagenav lower on mobile

* added cursor pointer to clickable components in docs

* fix(workflow-block): clearing child workflow input format field must lazy cascade parent workflow state deletion (#2038)

* Error tools

* Ui tools

* Navigate ui tool

* Combine env vars and oauth tools

* Plan mode section

* Plan mode v1

* Plan mode v1

* Plan mode improvements

* Build plan button

* Make workflow plan editable

* Tool improvements

* Fix build plan button

* Lint

* Fix

* Fix lint

* Fix plan

* Remove migrations

* fix undo/redo settling of ops

* Add migratinos back

* Smoothen out diff store

* Hide plan mode

* Fix lint

* Edit run workflow params

* Fix lint

* Fix tests

* Fix typing

* Fix build errors and lint

* Fix build

* Fix tests

---------

Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com>
Co-authored-by: Waleed <walif6@gmail.com>
Co-authored-by: Emir Karabeg <78010029+emir-karabeg@users.noreply.github.com>
Co-authored-by: Adam Gough <77861281+aadamgough@users.noreply.github.com>
Co-authored-by: aadamgough <adam@sim.ai>
Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu>
Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com>
2025-11-21 10:22:08 -08:00
Waleed
3faab2cb01 improvement(notifications): add option to disable error notifications, remove deprecated autoFillEnvVars field (#2045) 2025-11-18 16:28:59 -08:00
Emir Karabeg
02d9fedf0c feat(agent): messages array, memory (#2023)
* feat(agent): messages array, memory options

* feat(messages-input): re-order messages

* backend for new memory setup, backwards compatibility in loadWorkflowsFromNormalizedTable from old agent block to new format

* added memories all conversation sliding token window, standardized modals

* lint

* fix build

* reorder popover for output selector for chat

* add internal auth, finish memories

* fix rebase

* fix failing test

---------

Co-authored-by: waleed <walif6@gmail.com>
2025-11-18 15:58:10 -08:00
Vikhyath Mondreti
7a8d47a72e fix(schedules): locking schedules to prevent double runs (#1854)
* fix(schedules): locking schedules to prevent double runs

* add migration file

* fix
2025-11-07 19:45:25 -08:00
Vikhyath Mondreti
a73e2aaa8b improvement(templates): make it top-level route and change management/editing process (#1834)
* fix(billing): should allow restoring subscription (#1728)

* fix(already-cancelled-sub): UI should allow restoring subscription

* restore functionality fixed

* fix

* make templates root level url and make it part of deployment system

* separate updating template and deployment versions

* add tags

* add credentials extraction logic + use should import with workflow variables

* fix credential extraction

* add trigger mode indicator

* add starred tracking

* last updated field

* progress on creator profiles

* revert creator profile context type

* progress fix image uploads

* render templates details with creator details

* fix collab rules for workflow edit button

* creator profile perm check improvements

* restore accidental changes

* fix accessibility issues for non logged in users

* remove unused code

* fix type errors

---------

Co-authored-by: Waleed <walif6@gmail.com>
Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com>
2025-11-07 17:57:53 -08:00
Siddharth Ganesan
742d59f54d feat(hitl): add human in the loop block (#1832)
* fix(billing): should allow restoring subscription (#1728)

* fix(already-cancelled-sub): UI should allow restoring subscription

* restore functionality fixed

* fix

* Add pause resume block

* Add db schema

* Initial test passes

* Tests pass

* Execution pauses

* Snapshot serializer

* Ui checkpoint

* Works 1

* Pause resume simple v1

* Hitl block works in parallel branches without timing overlap

* Pending status to logs

* Pause resume ui link

* Big context consolidation

* HITL works in loops

* Fix parallels

* Reference blocks properly

* Fix tag dropdown and start block resolution

* Filter console logs for hitl block

* Fix notifs

* Fix logs page

* Fix logs page again

* Fix

* Checkpoint

* Cleanup v1

* Refactor v2

* Refactor v3

* Refactor v4

* Refactor v5

* Resume page

* Fix variables in loops

* Fix var res bugs

* Ui changes

* Approval block

* Hitl works e2e v1

* Fix tets

* Row level lock

---------

Co-authored-by: Waleed <walif6@gmail.com>
Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com>
2025-11-06 15:59:28 -08:00
Vikhyath Mondreti
a072e6d1d8 improvement(custom-tools): make them workspace scoped + ux to manage them (#1772)
* improvement(custom-tools): make them workspace scoped

* fix auth check

* remove comments

* add dup check

* fix dup error message display

* fix tests

* fix on app loading of custom tools
2025-10-30 17:40:38 -07:00
Vikhyath Mondreti
fe9ebbf81b improvement(api-keys): move to workspace level (#1765)
* fix(billing): should allow restoring subscription (#1728)

* fix(already-cancelled-sub): UI should allow restoring subscription

* restore functionality fixed

* fix

* improvement(api-keys): move to workspace level

* remove migration to prep merge

* remove two more unused cols

* prep staging  merge

* add migration back

---------

Co-authored-by: Waleed <walif6@gmail.com>
Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com>
2025-10-30 11:42:58 -07:00
Waleed
c99bb0aaa2 feat(cost): added hidden cost breakdown component to settings > subscription, start collecting current period copilot cost and last period copilot cost (#1770)
* feat(cost): added hidden cost breakdown component to settings > subscription, start collecting current period copilot cost and last period copilot cost

* don't rerender envvars when switching between workflows in the same workspace
2025-10-30 11:09:47 -07:00
Waleed
48f520b3c7 feat(files): added file manager table, enforce permissions for viewing files (#1766)
* feat(files): added file manager table, enforce permissions for viewing files

* rename types

* cleanup

* cleanup

* confirm local file system works with all contexts

* clean

* remove isAsync

* ignore expiresAt

* add relative imports instead of absolute ones

* absl imports

* remove redundant comments
2025-10-29 22:03:27 -07:00
Waleed
ef5b6999ab fix(migrations): add missing migration for processing columns in documents table (#1749) 2025-10-27 23:23:42 -07:00
Waleed
38614fad79 fix(mcp): resolve variables & block references in mcp subblocks (#1735)
* fix(mcp): resolve variables & block references in mcp subblocks

* cleanup

* ack PR comment

* added variables access to mcp tools when added in agent block

* fix sequential migrations issues
2025-10-27 13:13:11 -07:00
Waleed
98e98496e8 Revert "improvement(consts): removed redundant default consts in favor of env…" (#1739)
This reverts commit 659b46fa2f.
2025-10-26 23:06:14 -07:00
Waleed
659b46fa2f improvement(consts): removed redundant default consts in favor of envvar defaults for storage & usage limits (#1737)
* improvement(consts): removed redundant default consts in favor of envvar defaults for storage & usage limits

* remove unnecessary tests
2025-10-26 21:43:00 -07:00
Waleed
ce4893a53c feat(sso-chat-deployment): added sso auth option for chat deployment (#1729)
* feat(sso-chat-deployment): added sso auth option for chat deployment

* ack PR comments
2025-10-25 14:58:25 -07:00
Vikhyath Mondreti
35c551984f feat(files): gmail upload attachment, workspace files, file storage limits (#1666)
* feat(gmail): add attachment uploads

* add workspace files

* update landing page

* fix lint

* fix test

* fixed UI

* added additional S3 tools to upload files

* added search filters for gmail trigger

* added files to every block

* works

* fix

* register sharepoint tool

---------

Co-authored-by: waleed <waleed>
2025-10-17 14:44:22 -07:00
Waleed
8f06aec68b fix(vulns): fix various vulnerabilities and enhanced code security (#1611)
* fix(vulns): fix SSRF vulnerabilities

* cleanup

* cleanup

* regen docs

* remove unused deps

* fix failing tests

* cleanup

* update deps

* regen bun lock
2025-10-11 22:14:31 -07:00
Waleed
1de6f09069 feat(versions): added the ability to rename deployment versions (#1610) 2025-10-11 21:18:36 -07:00
Waleed
e697e50d4e improvement(performance): remove unused source/target indices, add index on snapshot id (#1603) 2025-10-11 20:37:35 -07:00
Waleed
2d49892aaa feat(deployed-chat): added file upload to workflow execute API, added to deployed chat, updated chat panel (#1588)
* feat(deployed-chat): updated chat panel UI, deployed chat and API can now accept files

* added nested tag dropdown for files

* added duplicate file validation to chat panel

* update docs & SDKs

* fixed build

* rm extraneous comments

* ack PR comments, cut multiple DB roundtrips for permissions & api key checks in api/workflows

* allow read-only users to access deployment info, but not take actions

* add downloadable file to logs for files passed in via API

* protect files/serve route that is only used client-side

---------

Co-authored-by: waleed <waleed>
2025-10-10 17:19:51 -07:00
Vikhyath Mondreti
8ce5a1b7c0 feat(billing): bill by threshold to prevent cancellation edge case (#1583)
* feat(billing): bill by threshold to prevent cancellation edge case

* fix org billing

* fix idempotency key issue

* small optimization for team checks

* remove console log

* remove unused type

* fix error handling
2025-10-10 17:19:51 -07:00
Waleed
c04eb01aed fix(db): revert to dedicated sockets db connection establishment (#1581) 2025-10-08 08:37:58 -07:00
Waleed
1a0fdb32fe chore(docs): update docs (#1578) 2025-10-08 00:55:18 -07:00
Waleed
9d45b8df1e fix(ts-sdk): fix job to publish ts sdk (#1576) 2025-10-08 00:46:30 -07:00
Waleed
ae3a7f0865 fix(db): reduce overall number of db max conncetions to incr performance (#1575) 2025-10-08 00:30:44 -07:00
Waleed
715f42c1a6 feat(highlighting): added resolved vars highlighting to code subblock, to be consistent with other subblocks (#1570) 2025-10-07 17:17:39 -07:00
Waleed
872e034312 feat(chat-streaming): added a stream option to workflow execute route, updated SDKs, updated docs (#1565)
* feat(chat-stream): updated workflow id execute route to support streaming via API

* enable streaming via api

* added only text stream option

* cleanup deployed preview componnet

* updated selectedOutputIds to selectedOutput

* updated TS and Python SDKs with async, rate limits, usage, and streaming API routes

* stream non-streaming blocks when streaming is specified

* fix(chat-panel): add onBlockComplete handler to chat panel to stream back blocks as they complete

* update docs

* cleanup

* ack PR comments

* updated next config

* removed getAssetUrl in favor of local assets

* resolve merge conflicts

* remove extra logic to create sensitive result

* simplify internal auth

* remove vercel blob from CSP + next config
2025-10-07 15:10:37 -07:00
Waleed
f03f395225 fix(db): enable database connection pooling in production (#1564)
* fix: enable database connection pooling in production

* debug: add diagnostic endpoints to test NODE_ENV and database pooling

* test: add connection testing endpoint to diagnose production delay

* redeuce num of concurrent connections
2025-10-07 10:27:33 -07:00
Vikhyath Mondreti
c42d2a32f3 feat(copilot): fix context / json parsing edge cases (#1542)
* Add get ops examples

* input format incorrectly created by copilot should not crash workflow

* fix tool edits triggering overall delta

* fix(db): add more options for SSL connection, add envvar for base64 db cert (#1533)

* fix trigger additions

* fix nested outputs for triggers

* add condition subblock sanitization

* fix custom tools json

* Model selector

* fix response format sanitization

* remove dead code

* fix export sanitization

* Update migration

* fix import race cond

* Copilot settings

* fix response format

* stop loops/parallels copilot generation from breaking diff view

* fix lint

* Apply suggestion from @greptile-apps[bot]

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix tests

* fix lint

---------

Co-authored-by: Siddharth Ganesan <siddharthganesan@gmail.com>
Co-authored-by: Waleed <walif6@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-10-03 19:08:57 -07:00
Waleed
10692b5e5a fix(db): remove overly complex db connection logic (#1538) 2025-10-02 19:54:32 -07:00
Waleed
5f1518ffd9 fix(db): added SSL config to migrations container (#1535) 2025-10-02 18:04:31 -07:00
Waleed
fa9c97816b fix(db): add more options for SSL connection, add envvar for base64 db cert (#1533) 2025-10-02 15:53:45 -07:00
Waleed
ace83ebcae feat(cmdk): added knowledgebases to the cmdk modal (#1530) 2025-10-01 21:21:42 -07:00
Waleed
980a6d8347 improvement(db): enforce SSL everywhere where a DB connection is established (#1522)
* improvement(db): enforce SSL everywhere where a DB connection is established

* remove extraneous comment
2025-10-01 19:09:08 -07:00
Waleed
6e63eafb79 improvement(db): remove vercel, remove railway, remove crons, improve DB connection config (#1519)
* improvement(db): remove vercel, remove railway, remove crons, improve DB connection config

* remove NEXT_PUBLIC_VERCEL_URL

* remove db url fallbacks

* remove railway & more vercel stuff

---------

Co-authored-by: waleed <waleed>
2025-10-01 16:37:13 -07:00
Siddharth Ganesan
79461840c3 fix(migrations): make sso migration idempotent 2025-09-30 11:04:44 -07:00
Waleed
bf1c178282 feat(sso): add support for login with SAML/SSO (#1489)
* feat(sso): added login with SAML/SSO

* restore env

* fixed login styling

* upgrade deps, update UI

* more styling improvements

* reran migrations, tested with script

* improvement(auth): created SSO page

* improvement(auth): remove email option for SSO if not enabled

* cleanup

* cleaned up, added documentation for SSO/SAML config + tested registering either one with script and UI form

* cleanup

* ack PR comments

* move sso known providers to consts

---------

Co-authored-by: waleed <waleed>
Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu>
2025-09-29 20:05:26 -07:00
Waleed
dd1985c99c improvement(chat): deployed chat no longer uses subdomains, uses sim.ai/chat/[identifier] (#1474)
* improvement(chat): deployed chat no longer uses subdomains, uses sim.ai/chat/[identifier]

* ui fix

* added back validate route, remove backwards compatibility for subdomain redirects

* cleanup

* cleanup

* add chat page to conditional theme layout, and remove custom chat css

* consolidate theme providers

* ack pr commnets

---------

Co-authored-by: waleed <waleed>
2025-09-27 17:32:10 -07:00
Vikhyath Mondreti
928581f387 feat(deployments): make deployed state source of truth for non-manual executions + versioning (#1242)
* feat(deployments): make deployed state source of truth for non-manual executions + versioning

* fix lint

* fix test

* add script to migrate to deployed vesions

* fix deployed chat

* address greptile commands

* Remove 84th migration files to prepare for regeneration

* fix script + update migration

* fix script

* cleanup typing

* use shared helper

* fix tests

* fix duplicate route

* revert migrations prep

* add migration back

* add workflow in workflow block func

* fix UI

* fix lint

* make migration idempotent

* remove migrations

* add migrations back'

* adjust script to reuse helper

* add test webhook URL functionality

* consolidate test URL + prod URL code for webhooks

* fixes

* update trigger config save with correct dependencies

* make frozen canvas respect trigger mode

* chore(db): remove latest migration 0088, snapshot, journal entry; delete migrate-deployment-versions script

* separate parent id cleanup migration

* chore(db): remove 0089 migration, snapshot, and prune journal entry

* chore(db): generate 0090 migration after removing 0089 and merging staging

* fix

* chore(db): remove 0090 migration, snapshot, and prune journal entry

* chore(db): generate 0091 migration after merging staging and restoring idempotency flow

* fix some imports

* remove dead code

* fix tests

* remove comment

* working test url functionality restored

* works

* some styling changes

* make deploy text accurate

* chore(db): remove latest migration 0091 and snapshot; update journal before merging staging

* db(migrations): generate 0093_medical_sentinel and snapshots after merging staging

* reconcile with other merge

* fix trigger test

* remove extra use effect

* fix test url"

* don't swallow serializer errors

* fix lint

* fix tests

* fix tests

* expose root for generic webhook

* root access for webhook

* add is workflow ready check correctly
2025-09-24 20:28:09 -07:00
Vikhyath Mondreti
094dae3d3f feat(copilot): add training interface (#1445)
* progress

* cleanup UI

* progress

* fix trigger mode in yaml + copilot side

* persist user settings

* wrap operations correctly

* add trigger mode to add op

* remove misplaced comment

* add sent notification

* remove unused tab:
2025-09-24 14:19:11 -07:00
Waleed
8eaa83fe21 fix(billing): reset usage on transition from free -> paid plan (#1397)
* fix(billing): reset usage on transition from free -> paid plan

* fixes: pro->team upgrade logic, single org server side check on invite routes

* ui improvements

* cleanup team-members code

* minor renaming

* progress

* fix pro->team upgrade to prevent double billing

* add subscription delete case handler

---------

Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
2025-09-22 13:58:29 -07:00
Waleed
6fada45cd8 improvement(readme): update readme.md (#1412) 2025-09-22 11:12:25 -07:00
Vikhyath Mondreti
16f5819941 fix(missing-user-stats): missing user stats rows covered via migration' (#1409) 2025-09-22 10:50:04 -07:00
Waleed
6747a497fc fix(migrations): upgrade drizzle-kit in migrations container (#1374)
* fix(migrations): upgrade drizzle-kit in migrations container

* fix comments

* rm unused file
2025-09-18 11:04:06 -07:00
Vikhyath Mondreti
9de7a00373 improvement(code-structure): move db into separate package (#1364)
* improvement(code-structure): move db into separate package

* make db separate package

* remake bun lock

* update imports to not maintain two separate ones

* fix CI for tests by adding dummy url

* vercel build fix attempt

* update bun lock

* regenerate bun lock

* fix mocks

* remove db commands from apps/sim package json
2025-09-17 15:41:13 -07:00
Waleed Latif
ae43381d84 feat(domain): drop the 'studio' (#818)
* feat(domain): drop the

* change all references for Sim Studio to Sim

* change back license and notice

* lint

---------

Co-authored-by: waleedlatif <waleedlatif@waleedlatifs-MacBook-Pro.local>
2025-07-29 12:51:43 -07:00
Aditya Tripathi
e93a5ce646 fix(cli): package type for esm imports, missing realtime (#574)
* fix: package type for esm imports, missing realtime calls and use of migrate

* chore: bump cli
2025-06-28 08:36:01 -07:00
Aditya Tripathi
00334e501f feat(ci): socket realtime image for hosting (#565)
* feat: socket server for self/local deployment

* ci: memory limit and redundant dependency install

* chore: update readme, devcontainer, cli package

* chore: add new dev scripts and update README for full development setup
2025-06-27 09:03:29 -07:00
Waleed Latif
76df2b9cd9 fix(sockets): added throttling, refactor entire socket server, added tests (#534)
* refactor(kb): use chonkie locally (#475)

* feat(parsers): text and markdown parsers (#473)

* feat: text and markdown parsers

* fix: don't readfile on buffer, convert buffer to string instead

* fix(knowledge-wh): fixed authentication error on webhook trigger

fix(knowledge-wh): fixed authentication error on webhook trigger

* feat(tools): add huggingface tools/blcok  (#472)

* add hugging face tool

* docs: add Hugging Face tool documentation

* fix: format and lint Hugging Face integration files

* docs: add manual intro section to Hugging Face documentation

* feat: replace Record<string, any> with proper HuggingFaceRequestBody interface

* accidental local files added

* restore some docs

* make layout full for model field

* change huggingface logo

* add manual content

* fix lint

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>

* fix(knowledge-ux): fixed ux for knowledge base (#478)

fix(knowledge-ux): fixed ux for knowledge base (#478)

* fix(billing): bump better-auth version & fix existing subscription issue when adding seats (#484)

* bump better-auth version & fix existing subscription issue Bwhen adding seats

* ack PR comments

* fix(env): added NEXT_PUBLIC_APP_URL to .env.example (#485)

* feat(subworkflows): workflows as a block within workflows (#480)

* feat(subworkflows) workflows in workflows

* revert sync changes

* working output vars

* fix greptile comments

* add cycle detection

* add tests

* working tests

* works

* fix formatting

* fix input var handling

* add images

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>

* fix(kb): fixed kb race condition resulting in no chunks found (#487)

* fix: added all blocks activeExecutionPath (#486)

* refactor(chunker): replace chonkie with custom TextChunker (#479)

* refactor(chunker): replace chonkie with custom TextChunker implementation and update document processing logic

* chore: cleanup unimplemented types

* fix: KB tests updated

* fix(tab-sync): sync between tabs on change (#489)

* fix(tab-sync): sync between tabs on change

* refactor: optimize JSON.stringify operations that are redundant

* fix(file-upload): upload presigned url to kb for file upload instead of the whole file, circumvents 4.5MB serverless func limit (#491)

* feat(folders): folders to manage workflows (#490)

* feat(subworkflows) workflows in workflows

* revert sync changes

* working output vars

* fix greptile comments

* add cycle detection

* add tests

* working tests

* works

* fix formatting

* fix input var handling

* fix(tab-sync): sync between tabs on change

* feat(folders): folders to organize workflows

* address comments

* change schema types

* fix lint error

* fix typing error

* fix race cond

* delete unused files

* improved UI

* updated naming conventions

* revert unrelated changes to db schema

* fixed collapsed sidebar subfolders

* add logs filters for folders

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>
Co-authored-by: Waleed Latif <walif6@gmail.com>

* revert tab sync

* improvement(folders): added multi-select for moving folders (#493)

* added multi-select for folders

* allow drag into root

* remove extraneous comments

* instantly create worfklow on plus

* styling improvements, fixed flicker

* small improvement to dragover container

* ack PR comments

* fix(deployed-chat): made the chat mobile friendly (#494)

* improvement(ui/ux): chat deploy (#496)

* improvement(ui/ux): chat deploy experience

* improvement(ui/ux): chat fontweight

* feat(gmail): added option to access raw gmail from gmail polling service (#495)

* added option to grab raw gmail from gmail polling service

* safe json parse for function block execution to prevent vars in raw email from being resolved as sim studio vars

* added tests

* remove extraneous comments

* fix(ui): fix the UI for folder deletion, huggingface icon, workflow block icon, standardized alert dialog (#498)

* fixed folder delete UI

* fixed UI for workflow block, huggingface, & added alert dialog for deleting folders

* consistently style all alert dialogs

* fix(reset-data): remove reset all data button from settings modal along with logic (#499)

* fix(airtable): fixed airtable oauth token refresh, added tests (#502)

* fixed airtable token refresh, added tests

* added helpers for refreshOAuthToken function

* feat(registration): disable registration + handle env booleans (#501)

* feat: disable registration + handle env booleans

* chore: removing pre-process because we need to use util

* chore: format

* feat(providers): added azure openai (#503)

* added azure openai

* fix request params being passed through agent block for azure

* remove o1 from azure-openai models list

* fix: add vscode settings to gitignore

* feat(file-upload): generalized storage to support azure blob, enhanced error logging in kb, added xlsx parser (#506)

* added blob storage option for azure, refactored storage client to be provider agnostic, tested kb & file upload and s3 is undisrupted, still have to test blob

* updated CORS policy for blob, added azure blob-specific headers

* remove extraneous comments

* add file size limit and timeout

* added some extra error handling in kb add documents

* grouped envvars

* ack PR comments

* added sheetjs and xlsx parser

* fix(folders): modified folder deletion to delete subfolders & workflows in it instead of moving to root (#508)

* modified folder deletion to delete subfolders & workflows in it instead of moving to root

* added additional testing utils

* ack PR comments

* feat: api response block and implementation

* improvement(local-storage): remove use of local storage except for oauth and last active workspace id (#497)

* remove local storage usage

* remove migration for last active workspace id

* Update apps/sim/app/w/[id]/components/workflow-block/components/sub-block/components/file-selector/components/jira-issue-selector.tsx

Add fallback for required scopes

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* add url builder util

* fi

* fix lint

* lint

* modify pre commit hook

* fix oauth

* get last active workspace working again

* new workspace logic works

* fetch locks

* works now

* remove empty useEffect

* fix loading issue

* skip empty workflow syncs

* use isWorkspace in transition flag

* add logging

* add data initialized flag

* fix lint

* fix: build error by create a server-side utils

* remove migration snapshots

* reverse search for workspace based on workflow id

* fix lint

* improvement: loading check and animation

* remove unused utils

* remove console  logs

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@vikhyaths-air.lan>

* feat(multi-select): simplified chat to always return readable stream, can select multiple outputs and get response streamed back in chat panel & deployed chat (#507)

* improvement: all workflow executions return ReadableStream & use sse to support multiple streamed outputs in chats

* fixed build

* remove extraneous comments

* general improvemetns

* ack PR comments

* fixed built

* improvement(workflow-state): split workflow state into separate tables  (#511)

* new tables to track workflow state

* fix lint

* refactor into separate tables

* fix typing

* fix lint

* add tests

* fix lint

* add correct foreign key constraint

* add self ref

* remove unused checks

* fix types

* fix type

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>

* feat(models): added new openai models, updated model pricing, added new groq model (#513)

* fix(autocomplete): fixed extra closing tag on tag dropdown autocomplete (#514)

* chore: enable input format again

* fix: process the input made on api calls with proper extraction

* feat: add json-object for ai generation for response block and others

* chore: add documentation for response block

* chore: rollback temp fix and uncomment original input handler

* chore: add missing mock for response handler

* chore: add missing mock

* chore: greptile recommendations

* added cost tracking for router & evaluator blocks, consolidated model information into a single file, hosted keys for evaluator & router, parallelized unit tests (#516)

* fix(deployState): deploy not persisting bug  (#518)

* fix(undeploy-bug): fix deployment persistence failing bug

* fix lint

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>

* fix decimal entry issues

* remove unused files

* fix(db): decimal position entry issues (#520)

* fix decimal entry issues

* remove unused files

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>

* fix lint

* fix test

* improvement(kb): added configurability for chunks, query across multiple knowledge bases (#512)

* refactor: consolidate create modal file

* fix: identify dead processes

* fix: mark failed in DB after processing timeout

* improvement: added overlap chunks and fixed modal UI

* feat: multiselect logic

* fix: biome changes for css ordering warn instead of error

* improvement: create chunk ui

* fix: removed unused schema columns

* fix: removed references to deleted columns

* improvement: sped up vector search time

* feat: multi-kb search

* add bulk endpoint to disable/delete multiple chunks

* add bulk endpoint to disable/delete multiple chunks

* fix: removed unused schema columns

* fix: removed references to deleted columns

* made endpoints for knowledge more RESTful, added tests

* added batch operations for delete/enable/disable docs, alr have this for chunks

* added migrations

* added migrations

---------

Co-authored-by: Waleed Latif <walif6@gmail.com>

* fix(models): remove temp from models that don't support it

* feat(sdk): added ts and python SDKs + docs (#524)

* added ts & python sdk, renamed cli from simstudio to cli

* added docs

* ack PR comments

* improvements

* fixed issue where it goes to random workspace when you click reload

fixed lint issue

* feat: better response builder + doc update

* fix(auth): added preview URLs to list of trusted origins (#525)

* trusted origins

* lint error

* removed localhost

* ran lint

---------

Co-authored-by: Waleed Latif <walif6@gmail.com>

* fix(sdk): remove dev script from SDK

* PR: changes for migration

* add changes on top of db migration changes

* fix: allow removing single input field

* improvement(permissions): workspace permissions improvements, added provider and reduced API calls by 85% (#530)

* improved permissions UI & access patterns, show outstanding invites

* added logger

* added provider for workspace permissions, 85% reduction in API calls to get user permissions and improved performance for invitations

* ack PR comments

* cleanup

* fix disabled tooltips

* improvement(tests): parallelized tests and build fixes (#531)

* added provider for workspace permissions, 85% reduction in API calls to get user permissions and improved performance for invitations

* parallelized more tests, fixed test warnings

* removed waitlist verification route, use more utils in tests

* fixed build

* ack PR comments

* fix

* fix(kb): reduced params in kb block, added advanced mode to starter block, updated docs

* feat(realtime): sockets + normalized tables + deprecate sync (#523)

* feat: implement real-time collaborative workflow editing with Socket.IO

- Add Socket.IO server with room-based architecture for workflow collaboration
- Implement socket context for client-side real-time communication
- Add collaborative workflow hook for synchronized state management
- Update CSP to allow socket connections to localhost:3002
- Add fallback authentication for testing collaborative features
- Enable real-time broadcasting of workflow operations between tabs
- Support multi-user editing of blocks, edges, and workflow state

Key components:
- socket-server/: Complete Socket.IO server with authentication and room management
- contexts/socket-context.tsx: Client-side socket connection and state management
- hooks/use-collaborative-workflow.ts: Hook for collaborative workflow operations
- Workflow store integration for real-time state synchronization

Status: Basic collaborative features working, authentication bypass enabled for testing

* feat: complete collaborative subblock editing implementation

 All collaborative features now working perfectly:
- Real-time block movement and positioning
- Real-time subblock value editing (text fields, inputs)
- Real-time edge operations and parent updates
- Multi-user workflow rooms with proper broadcasting
- Socket.IO server with room-based architecture
- Permission bypass system for testing

🔧 Technical improvements:
- Modified useSubBlockValue hook to use collaborative event system
- All subblock setValue calls now dispatch 'update-subblock-value' events
- Collaborative workflow hook handles all real-time operations
- Socket server processes and persists all operations to database
- Clean separation between local and collaborative state management

🧪 Tested and verified:
- Multiple browser tabs with different fallback users
- Block dragging and positioning updates in real-time
- Subblock text editing reflects immediately across tabs
- Workflow room management and user presence
- Database persistence of all collaborative operations

Status: Full collaborative workflow editing working with fallback authentication

* feat: implement proper authentication for collaborative Socket.IO server

 **Authentication System Complete**:
- Removed all fallback authentication code and bypasses
- Socket server now requires valid Better Auth session cookies
- Proper session validation using auth.api.getSession()
- Authentication errors properly handled and logged
- User info extracted from session: userId, userName, email, organizationId

🔧 **Technical Implementation**:
- Updated CSP to allow WebSocket connections (ws://localhost:3002)
- Socket authentication middleware validates session tokens
- Proper error handling for missing/invalid sessions
- Permission system enforces workflow access controls
- Clean separation between authenticated and unauthenticated states

🧪 **Testing Status**:
- Socket server properly rejects unauthenticated connections
- Authentication errors logged with clear messages
- CSP updated to allow both HTTP and WebSocket protocols
- Ready for testing with authenticated users

Status: Production-ready collaborative authentication system

* feat: complete authentication integration for collaborative Socket.IO system

🎉 **PRODUCTION-READY COLLABORATIVE SYSTEM**

 **Authentication Integration Complete**:
- Fixed Socket.IO client to send credentials (withCredentials: true)
- Updated server CORS to accept credentials with specific origin
- Removed all fallback authentication bypasses
- Proper Better Auth session validation working

🔧 **Technical Fixes**:
- Socket client: Enable withCredentials for cookie transmission
- Socket server: Accept credentials with origin 'http://localhost:3000'
- Better Auth cookie utility integration for session parsing
- Comprehensive authentication middleware with proper error handling

🧪 **Verified Working Features**:
-  Real user authentication (Vikhyath Mondreti authenticated)
-  Multi-user workflow rooms (2+ users in same workflow)
-  Permission system enforcing workflow access controls
-  Real-time subblock editing across browser tabs
-  Block movement and positioning updates
-  Automatic room cleanup and management
-  Database persistence of all collaborative operations

🚀 **Status**: Complete enterprise-grade collaborative workflow editing system
- No more fallback users - production authentication
- Multi-tab collaboration working perfectly
- Secure access control with Better Auth integration
- Real-time updates for all workflow operations

* remove sync system and move to server side

* fix lint

* delete unused file

* added socketio dep

* fix subblock persistence bug

* working deletion of workflows

* fix lint

* added railway

* add debug logging for railway deployment

* improve typing

* fix lint

* working subflow persistence

* fix lint

* working cascade deletion

* fix lint

* working subflow inside subflow

* works

* fix lint

* prevent subflow in subflow

* fix lint

* add additional logs, add localhost as allowedOrigin

* add additional logs, add localhost as allowedOrigin

* fix type error

* remove unused code

* fix lint

* fix tests

* fix lint

* fix build error

* workign folder updates

* fix typing issue

* fix lint

* fix typing issues

* lib/

* fix tests

* added old presence component back, updated to use one-time-token better auth plugin for socket server auth, tested

* fix errors

* fix bugs

* add migration scripts to run

* fix lint

* fix deploy tests

* fix lint

* fix minor issues

* fix lint

* fix migration script

* allow comma separateds id file input to migration script

* fix lint

* fixed

* fix lint

* fix fallback case

* fix type errors

* address greptile comments

* fix lint

* fix script to generate new block ids

* fix lint

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@vikhyaths-air.lan>
Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>

* fix(sockets): updated CSP

* remove unecessary logs

* fix lint

* added throttling, refactor entire socket server, added tests

* improvements

* remove self monitoring func, add block name event

* working isWide, isAdvanced toggles with sockets

* fix lint

* fix duplicate key issue for user avatar

* fix lint

* fix user presence

* working parallel badges / loop badges updates

* working connection output persistence

* fix lint

* fix build errors

* fix lint

* logs removed

* fix cascade var name update bug

* works

* fix lint

* fix parallel blocks

* fix placeholder

* fix test

* fixed tests

---------

Co-authored-by: Aditya Tripathi <aditya@climactic.co>
Co-authored-by: Adam Gough <77861281+aadamgough@users.noreply.github.com>
Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>
Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu>
Co-authored-by: Emir Karabeg <78010029+emir-karabeg@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@vikhyaths-air.lan>
Co-authored-by: Ajit Kadaveru <ajit.kadaveru@berkeley.edu>
2025-06-24 17:44:30 -07:00