github/sim
1
1
Fork 0
mirror of https://github.com/simstudioai/sim.git synced 2026-04-28 03:00:29 -04:00
Code Issues Packages Projects Releases Wiki Activity
3,497 Commits 110 Branches 261 Tags
6ee73fa22f71e038d4547a58d2096be7ac7a75c4
Commit Graph

36 Commits

Author SHA1 Message Date
Waleed
2f492cacc1 feat(providers): add Gemini Deep Research via Interactions API (#3192) 
* feat(providers): add Gemini Deep Research via Interactions API

* fix(providers): hide memory UI for deep research models

* feat(providers): add multi-turn support and token logging for deep research

* fix(providers): only collect user messages as deep research input

* fix(providers): forward previousInteractionId to provider request

* fix(blocks): hide memory child fields for deep research models

* remove memory params from models that don't support it in provider requests

* update blog
 2026-02-11 01:01:59 -08:00
Waleed
99ae5435e3 feat(models): updated model configs, updated anthropic provider to propagate errors back to user if any (#3159) 
* feat(models): updated model configs, updated anthropic provider to propagate errors back to user if any

* moved max tokens to advanced

* updated model configs and testesd

* removed default in max config for output tokens

* moved more stuff to advanced mode in the agent block

* stronger typing

* move api key under model, update mistral and groq

* update openrouter, fixed serializer to allow ollama/vllm models without api key

* removed ollama handling
 2026-02-06 22:35:57 -08:00
Waleed
1d4d61a10a feat(azure): added azure anthropic, added backwards compat support for chat completions API, added opus 4.6 (#3145) 
* feat(azure): added azure anthropic, added backwards compat support for chat completions API, added opus 4.6

* added max thinking level

* update tests

* ack comments

* update cql validation
 2026-02-05 10:52:18 -08:00
Vikhyath Mondreti
656beb8383 fix(anthropic): token limits for streaming with tool calls (#3084) 
* remove for bedrock since they handle on their own

* fix

* fix inference config reference

* add to docs

* make it min between max tokens
 2026-01-30 12:08:16 -08:00
Waleed
ef613ef035 fix(models): update cerebras and groq models (#3038) 2026-01-27 18:12:48 -08:00
Vikhyath Mondreti
5189473e06 fix(max-tokens): anthropic models streaming vs non-streaming (#2999) 
* fix(max-tokens): anthropic models streaming vs non-streaming

* default max inc

* update the docs
 2026-01-26 17:05:33 -08:00
Waleed
26dff7cffe feat(bedrock): added aws bedrock as a model provider (#2722) 2026-01-07 20:08:03 -08:00
Waleed
b0748c82f9 fix(search): removed full text param from built-in search, anthropic provider streaming fix (#2542) 
* fix(search): removed full text param from built-in search, anthropic provider streaming fix

* rewrite gemini provider with official sdk + add thinking capability

* vertex gemini consolidation

* never silently use different model

* pass oauth client through the googleAuthOptions param directly

* make server side provider registry

* remove comments

* take oauth selector below model selector

---------

Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
 2025-12-22 23:57:11 -08:00
Vikhyath Mondreti
8c2c49eb14 fix(models): memory fixes, provider code typing, cost calculation cleanup (#2515) 
* improvement(memory): should not be block scoped

* cleanup provider code

* update other providers

* cleanup fallback code

* remove flaky test

* fix memory

* move streaming fix to right level

* cleanup streaming server

* make memories workspace scoped

* update docs

* fix dedup logic

* fix streaming parsing issue for multiple onStream calls for same block

* fix(provieders): support parallel agent tool calls, consolidate utils

* address greptile comments

* remove all comments

* fixed openrouter response format handling, groq & cerebras response formats

* removed duplicate type

---------

Co-authored-by: waleed <walif6@gmail.com>
 2025-12-22 15:59:53 -08:00
Waleed
7b5405e968 feat(vertex): added vertex to list of supported providers (#2430) 
* feat(vertex): added vertex to list of supported providers

* added utils files for each provider, consolidated gemini utils, added dynamic verbosity and reasoning fetcher
 2025-12-17 14:57:58 -08:00
Waleed
d02499a915 fix(ollama): fixed messages array for ollama, added gpt-5.2 (#2315) 
* fix(ollama): fixed messages array for ollama, added gpt-5.2

* ack PR comments
 2025-12-11 13:51:51 -08:00
Waleed
bf54c88ae4 feat(models): added xai models and updated gemini pricing (#2161) 2025-12-02 14:31:23 -08:00
Waleed
6fda9bd72e feat(models): added latest mistral models (#2159) 2025-12-02 13:46:00 -08:00
Waleed
8000394e98 feat(models): host google gemini models (#2122) 
* feat(models): host google gemini models

* remove unused primary key
 2025-11-26 12:30:20 -08:00
Waleed
a5b7897b34 feat(models): added claude opus 4.5 (#2111) 
* feat(models): added claude opus 4.5

* add to copilot dropdown
 2025-11-24 12:12:30 -08:00
Waleed
6114c213d2 feature(models): added vllm provider (#2103) 
* Add vLLM self-hosted provider

* updated vllm to have pull parity with openai, dynamically fetch models

---------

Co-authored-by: MagellaX <alphacr792@gmail.com>
 2025-11-22 14:50:43 -08:00
Waleed
a7fe1d3aea fix(models): remove unrelease oai models, fix help modal (#2046) 2025-11-18 17:08:47 -08:00
Emir Karabeg
02d9fedf0c feat(agent): messages array, memory (#2023) 
* feat(agent): messages array, memory options

* feat(messages-input): re-order messages

* backend for new memory setup, backwards compatibility in loadWorkflowsFromNormalizedTable from old agent block to new format

* added memories all conversation sliding token window, standardized modals

* lint

* fix build

* reorder popover for output selector for chat

* add internal auth, finish memories

* fix rebase

* fix failing test

---------

Co-authored-by: waleed <walif6@gmail.com>
 2025-11-18 15:58:10 -08:00
Waleed
d99d5fe39c feat(billing): add notif for first failed payment, added upgrade email from free, updated providers that supported granular tool control to support them, fixed envvar popover, fixed redirect to wrong workspace after oauth connect (#2015) 
* feat(billing): add notif for first failed payment, added upgrade email from free, updated providers that supported granular tool control to support them, fixed envvar popover, fixed redirect to wrong workspace after oauth connect

* fix build

* ack PR comments
 2025-11-15 16:09:58 -08:00
Waleed
f8070f9029 feat(models): added gpt-5.1 (#2007) 2025-11-14 23:23:47 -08:00
Siddharth Ganesan
cc0ace7de6 improvement(copilot): version update (#1689) 
* Add exa to search online tool

* Larger font size

* Copilot UI improvements

* Fix models options

* Add haiku 4.5 to copilot

* Model ui for haiku

* Fix lint

* Revert

* Only allow one revert to message

* Clear diff on revert

* Fix welcome screen flash

* Add focus onto the user input box when clicked

* Fix grayout of new stream on old edit message

* Lint

* Make edit message submit smoother

* Allow message sent while streaming

* Revert popup improvements: gray out stuff below, show cursor on revert

* Fix lint

* Improve chat history dropdown

* Improve get block metadata tool

* Update update cost route

* Fix env

* Context usage endpoint

* Make chat history scrollable

* Fix lint

* Copilot revert popup updates

* Fix lint

* Fix tests and lint

* Add summary tool

* Fix env.ts
 2025-10-18 12:59:48 -07:00
Waleed
9efc08a832 fix(telemetry): updated telemetry, added nested sanitization, added granular trace spans for logs and updated UI (#1627) 
* feat(logs): updated telemetry, added nested sanitization, added granular trace spans for logs and updated UI

* refactor trace spans into separate components

* remove any's from tool defs

* updated UI and overlayed spans

* cleanup

* ack PR comments

* stricter type safety

* clean
 2025-10-14 16:41:12 -07:00
Waleed
b10b2461a5 feat(mistal): added mistral as a provider, updated model prices (#1607) 
* feat(mistal): added mistral as a provider, updated model prices

* remove the ability for a block to reference its own outluts

* fixed order of responses for guardrails block
 2025-10-11 20:37:35 -07:00
Vikhyath Mondreti
7d62c200fa feat(openrouter): add open router to model block (#1172) 
* feat(openrouter): add open router to model block

* improvement(openrouter): streaming fix, temperature fix

* pr comments

---------

Co-authored-by: waleedlatif1 <walif6@gmail.com>
 2025-08-28 18:47:36 -07:00
Waleed Latif
ff43528d35 fix(gpt-5): fixed verbosity and reasoning params (#1069) 
* fix(gpt-5): fixed verbosity and reasoning parsm

* fixed dropdown

* default values for verbosity and reasoning effort

* cleanup

* use default value in dropdown
 2025-08-20 20:18:02 -07:00
Waleed Latif
cea42f5135 improvement(gpt-5): added reasoning level and verbosity to gpt-5 models (#1058) 2025-08-20 17:04:39 -07:00
Waleed Latif
75963eb851 fix(gpt-5): remove temp, decr socket debounce to 25ms (#898) 
* fix(gpt-5): remove temp

* decr debounce for sockets
 2025-08-07 11:24:10 -07:00
Waleed Latif
ed7d7a7101 fix(gpt-5): updated pricing (#897) 2025-08-07 10:39:55 -07:00
Waleed Latif
d5c13b72e9 feat(gpt-5): added gpt-5 models (#896) 
* feat(gpt-5): added gpt-5

* restore grok-3-fast

* added azure gpt-5
 2025-08-07 10:25:49 -07:00
Waleed Latif
4880d34786 feat(models): add openai oss models (#880) 
* feat(models): add openai oss models

* fix tool usage control for groq
 2025-08-05 12:16:32 -07:00
Waleed Latif
ff2b1d33c8 fix(tool-input): added tool input, visibility enum for tool params, fixed google provider bugs (#674) 
* transfrom from block-centric tool input component to tool-centric tool input component for agent tools

* added additional type safety, created generic wrapper for tool input & reused across all subblock types

* stop retries if tool call fails, implemented for all providers except google

* bug fix with tool name extraction

* another bug fix

* ran script to update docs

* update contributing guide tool/block add example to reflect new param structure

* update README

* add key control to combobox, fixed google

* fixed google provider, fixed combobox

* fixed a ton of tools, ensured that the agent tool has full parity with actual tool for all tools

* update docs to reflect new structure

* updated visibility for gmail draft

* standardize dropdown values for tool definitions

* add asterisk for user-only + required fields

* updated visibility for tools

* consolidate redactApiKey util, fixed console entry bug that overwrites previous block logs

* updated docs

* update contributing guide to guide users to point their branches at staging instead of main

* nits

* move socket tests
 2025-07-13 20:16:49 -07:00
Waleed Latif
e83745fcaf feat(models): add grok-4 (#655) 2025-07-10 14:19:45 -07:00
Waleed Latif
3887733da5 feat(kb): added cost for kb blocks (#654) 
* added cost to kb upload + search

* small fix

* ack PR comments
 2025-07-10 13:53:20 -07:00
Vikhyath Mondreti
1604ce4d7c v0.2.4: feat, improvement, fix (#595) 
* feat(function): added more granular error logs for function execution for easier debugging (#593)

* added more granular error logs for function execution

* added tests

* fixed syntax error reporting

* feat(models): added temp controls for gpt-4.1 family of models (#594)

* improvement(knowledge-upload): create and upload document to KB (#579)

* improvement: added knowledge upload

* improvement: added greptile comments (#579)

* improvement: changed to text to doc (#579)

* improvement: removed comment (#579)

* added input validation, tested persistence of KB selector

* update docs

---------

Co-authored-by: Adam Gough <adamgough@Mac.attlocal.net>
Co-authored-by: Waleed Latif <walif6@gmail.com>

* fix(remove workflow.state usage): no more usage of deprecated state column in any routes (#586)

* fix(remove workflow.state usage): no more usage of deprecated state col in routes

* fix lint

* fix chat route to only use deployed state

* fix lint

* better typing

* remove useless logs

* fix lint

* restore workflow handler file

* removed all other usages of deprecated 'state' column from workflows table, updated tests

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
Co-authored-by: Waleed Latif <walif6@gmail.com>

---------

Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Adam Gough <77861281+aadamgough@users.noreply.github.com>
Co-authored-by: Adam Gough <adamgough@Mac.attlocal.net>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
 2025-06-30 19:23:37 -07:00
Vikhyath Mondreti
bf0ea10e62 merge(staging-to-main) (#577) 
* feat(agent): agent model dropdown combobox (#572)

* feat: agent model dropdown combobox

* fix(cli): package type for esm imports, missing realtime (#574)

* fix: package type for esm imports, missing realtime calls and use of migrate

* chore: bump cli

* fix sourceBlock null check

* fix(kb): fix kb navigation URLs

* fix(csp): update CSP to allow for google drive picker

* feat(dropdown): added optional icon to agent model dropdown

---------

Co-authored-by: Aditya Tripathi <aditya@climactic.co>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
Co-authored-by: Waleed Latif <walif6@gmail.com>

* fix concurrent req check (#576)

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>

---------

Co-authored-by: Emir Karabeg <78010029+emir-karabeg@users.noreply.github.com>
Co-authored-by: Aditya Tripathi <aditya@climactic.co>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>
 2025-06-28 13:38:10 -07:00
Waleed Latif
76df2b9cd9 fix(sockets): added throttling, refactor entire socket server, added tests (#534) 
* refactor(kb): use chonkie locally (#475)

* feat(parsers): text and markdown parsers (#473)

* feat: text and markdown parsers

* fix: don't readfile on buffer, convert buffer to string instead

* fix(knowledge-wh): fixed authentication error on webhook trigger

fix(knowledge-wh): fixed authentication error on webhook trigger

* feat(tools): add huggingface tools/blcok  (#472)

* add hugging face tool

* docs: add Hugging Face tool documentation

* fix: format and lint Hugging Face integration files

* docs: add manual intro section to Hugging Face documentation

* feat: replace Record<string, any> with proper HuggingFaceRequestBody interface

* accidental local files added

* restore some docs

* make layout full for model field

* change huggingface logo

* add manual content

* fix lint

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>

* fix(knowledge-ux): fixed ux for knowledge base (#478)

fix(knowledge-ux): fixed ux for knowledge base (#478)

* fix(billing): bump better-auth version & fix existing subscription issue when adding seats (#484)

* bump better-auth version & fix existing subscription issue Bwhen adding seats

* ack PR comments

* fix(env): added NEXT_PUBLIC_APP_URL to .env.example (#485)

* feat(subworkflows): workflows as a block within workflows (#480)

* feat(subworkflows) workflows in workflows

* revert sync changes

* working output vars

* fix greptile comments

* add cycle detection

* add tests

* working tests

* works

* fix formatting

* fix input var handling

* add images

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>

* fix(kb): fixed kb race condition resulting in no chunks found (#487)

* fix: added all blocks activeExecutionPath (#486)

* refactor(chunker): replace chonkie with custom TextChunker (#479)

* refactor(chunker): replace chonkie with custom TextChunker implementation and update document processing logic

* chore: cleanup unimplemented types

* fix: KB tests updated

* fix(tab-sync): sync between tabs on change (#489)

* fix(tab-sync): sync between tabs on change

* refactor: optimize JSON.stringify operations that are redundant

* fix(file-upload): upload presigned url to kb for file upload instead of the whole file, circumvents 4.5MB serverless func limit (#491)

* feat(folders): folders to manage workflows (#490)

* feat(subworkflows) workflows in workflows

* revert sync changes

* working output vars

* fix greptile comments

* add cycle detection

* add tests

* working tests

* works

* fix formatting

* fix input var handling

* fix(tab-sync): sync between tabs on change

* feat(folders): folders to organize workflows

* address comments

* change schema types

* fix lint error

* fix typing error

* fix race cond

* delete unused files

* improved UI

* updated naming conventions

* revert unrelated changes to db schema

* fixed collapsed sidebar subfolders

* add logs filters for folders

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>
Co-authored-by: Waleed Latif <walif6@gmail.com>

* revert tab sync

* improvement(folders): added multi-select for moving folders (#493)

* added multi-select for folders

* allow drag into root

* remove extraneous comments

* instantly create worfklow on plus

* styling improvements, fixed flicker

* small improvement to dragover container

* ack PR comments

* fix(deployed-chat): made the chat mobile friendly (#494)

* improvement(ui/ux): chat deploy (#496)

* improvement(ui/ux): chat deploy experience

* improvement(ui/ux): chat fontweight

* feat(gmail): added option to access raw gmail from gmail polling service (#495)

* added option to grab raw gmail from gmail polling service

* safe json parse for function block execution to prevent vars in raw email from being resolved as sim studio vars

* added tests

* remove extraneous comments

* fix(ui): fix the UI for folder deletion, huggingface icon, workflow block icon, standardized alert dialog (#498)

* fixed folder delete UI

* fixed UI for workflow block, huggingface, & added alert dialog for deleting folders

* consistently style all alert dialogs

* fix(reset-data): remove reset all data button from settings modal along with logic (#499)

* fix(airtable): fixed airtable oauth token refresh, added tests (#502)

* fixed airtable token refresh, added tests

* added helpers for refreshOAuthToken function

* feat(registration): disable registration + handle env booleans (#501)

* feat: disable registration + handle env booleans

* chore: removing pre-process because we need to use util

* chore: format

* feat(providers): added azure openai (#503)

* added azure openai

* fix request params being passed through agent block for azure

* remove o1 from azure-openai models list

* fix: add vscode settings to gitignore

* feat(file-upload): generalized storage to support azure blob, enhanced error logging in kb, added xlsx parser (#506)

* added blob storage option for azure, refactored storage client to be provider agnostic, tested kb & file upload and s3 is undisrupted, still have to test blob

* updated CORS policy for blob, added azure blob-specific headers

* remove extraneous comments

* add file size limit and timeout

* added some extra error handling in kb add documents

* grouped envvars

* ack PR comments

* added sheetjs and xlsx parser

* fix(folders): modified folder deletion to delete subfolders & workflows in it instead of moving to root (#508)

* modified folder deletion to delete subfolders & workflows in it instead of moving to root

* added additional testing utils

* ack PR comments

* feat: api response block and implementation

* improvement(local-storage): remove use of local storage except for oauth and last active workspace id (#497)

* remove local storage usage

* remove migration for last active workspace id

* Update apps/sim/app/w/[id]/components/workflow-block/components/sub-block/components/file-selector/components/jira-issue-selector.tsx

Add fallback for required scopes

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* add url builder util

* fi

* fix lint

* lint

* modify pre commit hook

* fix oauth

* get last active workspace working again

* new workspace logic works

* fetch locks

* works now

* remove empty useEffect

* fix loading issue

* skip empty workflow syncs

* use isWorkspace in transition flag

* add logging

* add data initialized flag

* fix lint

* fix: build error by create a server-side utils

* remove migration snapshots

* reverse search for workspace based on workflow id

* fix lint

* improvement: loading check and animation

* remove unused utils

* remove console  logs

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@vikhyaths-air.lan>

* feat(multi-select): simplified chat to always return readable stream, can select multiple outputs and get response streamed back in chat panel & deployed chat (#507)

* improvement: all workflow executions return ReadableStream & use sse to support multiple streamed outputs in chats

* fixed build

* remove extraneous comments

* general improvemetns

* ack PR comments

* fixed built

* improvement(workflow-state): split workflow state into separate tables  (#511)

* new tables to track workflow state

* fix lint

* refactor into separate tables

* fix typing

* fix lint

* add tests

* fix lint

* add correct foreign key constraint

* add self ref

* remove unused checks

* fix types

* fix type

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>

* feat(models): added new openai models, updated model pricing, added new groq model (#513)

* fix(autocomplete): fixed extra closing tag on tag dropdown autocomplete (#514)

* chore: enable input format again

* fix: process the input made on api calls with proper extraction

* feat: add json-object for ai generation for response block and others

* chore: add documentation for response block

* chore: rollback temp fix and uncomment original input handler

* chore: add missing mock for response handler

* chore: add missing mock

* chore: greptile recommendations

* added cost tracking for router & evaluator blocks, consolidated model information into a single file, hosted keys for evaluator & router, parallelized unit tests (#516)

* fix(deployState): deploy not persisting bug  (#518)

* fix(undeploy-bug): fix deployment persistence failing bug

* fix lint

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>

* fix decimal entry issues

* remove unused files

* fix(db): decimal position entry issues (#520)

* fix decimal entry issues

* remove unused files

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>

* fix lint

* fix test

* improvement(kb): added configurability for chunks, query across multiple knowledge bases (#512)

* refactor: consolidate create modal file

* fix: identify dead processes

* fix: mark failed in DB after processing timeout

* improvement: added overlap chunks and fixed modal UI

* feat: multiselect logic

* fix: biome changes for css ordering warn instead of error

* improvement: create chunk ui

* fix: removed unused schema columns

* fix: removed references to deleted columns

* improvement: sped up vector search time

* feat: multi-kb search

* add bulk endpoint to disable/delete multiple chunks

* add bulk endpoint to disable/delete multiple chunks

* fix: removed unused schema columns

* fix: removed references to deleted columns

* made endpoints for knowledge more RESTful, added tests

* added batch operations for delete/enable/disable docs, alr have this for chunks

* added migrations

* added migrations

---------

Co-authored-by: Waleed Latif <walif6@gmail.com>

* fix(models): remove temp from models that don't support it

* feat(sdk): added ts and python SDKs + docs (#524)

* added ts & python sdk, renamed cli from simstudio to cli

* added docs

* ack PR comments

* improvements

* fixed issue where it goes to random workspace when you click reload

fixed lint issue

* feat: better response builder + doc update

* fix(auth): added preview URLs to list of trusted origins (#525)

* trusted origins

* lint error

* removed localhost

* ran lint

---------

Co-authored-by: Waleed Latif <walif6@gmail.com>

* fix(sdk): remove dev script from SDK

* PR: changes for migration

* add changes on top of db migration changes

* fix: allow removing single input field

* improvement(permissions): workspace permissions improvements, added provider and reduced API calls by 85% (#530)

* improved permissions UI & access patterns, show outstanding invites

* added logger

* added provider for workspace permissions, 85% reduction in API calls to get user permissions and improved performance for invitations

* ack PR comments

* cleanup

* fix disabled tooltips

* improvement(tests): parallelized tests and build fixes (#531)

* added provider for workspace permissions, 85% reduction in API calls to get user permissions and improved performance for invitations

* parallelized more tests, fixed test warnings

* removed waitlist verification route, use more utils in tests

* fixed build

* ack PR comments

* fix

* fix(kb): reduced params in kb block, added advanced mode to starter block, updated docs

* feat(realtime): sockets + normalized tables + deprecate sync (#523)

* feat: implement real-time collaborative workflow editing with Socket.IO

- Add Socket.IO server with room-based architecture for workflow collaboration
- Implement socket context for client-side real-time communication
- Add collaborative workflow hook for synchronized state management
- Update CSP to allow socket connections to localhost:3002
- Add fallback authentication for testing collaborative features
- Enable real-time broadcasting of workflow operations between tabs
- Support multi-user editing of blocks, edges, and workflow state

Key components:
- socket-server/: Complete Socket.IO server with authentication and room management
- contexts/socket-context.tsx: Client-side socket connection and state management
- hooks/use-collaborative-workflow.ts: Hook for collaborative workflow operations
- Workflow store integration for real-time state synchronization

Status: Basic collaborative features working, authentication bypass enabled for testing

* feat: complete collaborative subblock editing implementation

✅ All collaborative features now working perfectly:
- Real-time block movement and positioning
- Real-time subblock value editing (text fields, inputs)
- Real-time edge operations and parent updates
- Multi-user workflow rooms with proper broadcasting
- Socket.IO server with room-based architecture
- Permission bypass system for testing

🔧 Technical improvements:
- Modified useSubBlockValue hook to use collaborative event system
- All subblock setValue calls now dispatch 'update-subblock-value' events
- Collaborative workflow hook handles all real-time operations
- Socket server processes and persists all operations to database
- Clean separation between local and collaborative state management

🧪 Tested and verified:
- Multiple browser tabs with different fallback users
- Block dragging and positioning updates in real-time
- Subblock text editing reflects immediately across tabs
- Workflow room management and user presence
- Database persistence of all collaborative operations

Status: Full collaborative workflow editing working with fallback authentication

* feat: implement proper authentication for collaborative Socket.IO server

✅ **Authentication System Complete**:
- Removed all fallback authentication code and bypasses
- Socket server now requires valid Better Auth session cookies
- Proper session validation using auth.api.getSession()
- Authentication errors properly handled and logged
- User info extracted from session: userId, userName, email, organizationId

🔧 **Technical Implementation**:
- Updated CSP to allow WebSocket connections (ws://localhost:3002)
- Socket authentication middleware validates session tokens
- Proper error handling for missing/invalid sessions
- Permission system enforces workflow access controls
- Clean separation between authenticated and unauthenticated states

🧪 **Testing Status**:
- Socket server properly rejects unauthenticated connections
- Authentication errors logged with clear messages
- CSP updated to allow both HTTP and WebSocket protocols
- Ready for testing with authenticated users

Status: Production-ready collaborative authentication system

* feat: complete authentication integration for collaborative Socket.IO system

🎉 **PRODUCTION-READY COLLABORATIVE SYSTEM**

✅ **Authentication Integration Complete**:
- Fixed Socket.IO client to send credentials (withCredentials: true)
- Updated server CORS to accept credentials with specific origin
- Removed all fallback authentication bypasses
- Proper Better Auth session validation working

🔧 **Technical Fixes**:
- Socket client: Enable withCredentials for cookie transmission
- Socket server: Accept credentials with origin 'http://localhost:3000'
- Better Auth cookie utility integration for session parsing
- Comprehensive authentication middleware with proper error handling

🧪 **Verified Working Features**:
- ✅ Real user authentication (Vikhyath Mondreti authenticated)
- ✅ Multi-user workflow rooms (2+ users in same workflow)
- ✅ Permission system enforcing workflow access controls
- ✅ Real-time subblock editing across browser tabs
- ✅ Block movement and positioning updates
- ✅ Automatic room cleanup and management
- ✅ Database persistence of all collaborative operations

🚀 **Status**: Complete enterprise-grade collaborative workflow editing system
- No more fallback users - production authentication
- Multi-tab collaboration working perfectly
- Secure access control with Better Auth integration
- Real-time updates for all workflow operations

* remove sync system and move to server side

* fix lint

* delete unused file

* added socketio dep

* fix subblock persistence bug

* working deletion of workflows

* fix lint

* added railway

* add debug logging for railway deployment

* improve typing

* fix lint

* working subflow persistence

* fix lint

* working cascade deletion

* fix lint

* working subflow inside subflow

* works

* fix lint

* prevent subflow in subflow

* fix lint

* add additional logs, add localhost as allowedOrigin

* add additional logs, add localhost as allowedOrigin

* fix type error

* remove unused code

* fix lint

* fix tests

* fix lint

* fix build error

* workign folder updates

* fix typing issue

* fix lint

* fix typing issues

* lib/

* fix tests

* added old presence component back, updated to use one-time-token better auth plugin for socket server auth, tested

* fix errors

* fix bugs

* add migration scripts to run

* fix lint

* fix deploy tests

* fix lint

* fix minor issues

* fix lint

* fix migration script

* allow comma separateds id file input to migration script

* fix lint

* fixed

* fix lint

* fix fallback case

* fix type errors

* address greptile comments

* fix lint

* fix script to generate new block ids

* fix lint

---------

Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@vikhyaths-air.lan>
Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>

* fix(sockets): updated CSP

* remove unecessary logs

* fix lint

* added throttling, refactor entire socket server, added tests

* improvements

* remove self monitoring func, add block name event

* working isWide, isAdvanced toggles with sockets

* fix lint

* fix duplicate key issue for user avatar

* fix lint

* fix user presence

* working parallel badges / loop badges updates

* working connection output persistence

* fix lint

* fix build errors

* fix lint

* logs removed

* fix cascade var name update bug

* works

* fix lint

* fix parallel blocks

* fix placeholder

* fix test

* fixed tests

---------

Co-authored-by: Aditya Tripathi <aditya@climactic.co>
Co-authored-by: Adam Gough <77861281+aadamgough@users.noreply.github.com>
Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net>
Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu>
Co-authored-by: Emir Karabeg <78010029+emir-karabeg@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@vikhyaths-air.lan>
Co-authored-by: Ajit Kadaveru <ajit.kadaveru@berkeley.edu>
 2025-06-24 17:44:30 -07:00