Priyanshu Solanki 4f69b171f2 feat(kb): Adding support for more tags to the KB (#2433)
* creating boolean, number and date tags with different equality matchings

* feat: add UI for tag field types with filter operators

- Update base-tags-modal with field type selector dropdown
- Update document-tags-modal with different input types per fieldType
- Update knowledge-tag-filters with operator dropdown and type-specific inputs
- Update search routes to support all tag slot types
- Update hook to use AllTagSlot type

* feat: add field type support to document-tag-entry component

- Add dropdown with all field types (Text, Number, Date, Boolean)
- Render different value inputs based on field type
- Update slot counting to include all field types (28 total)

* fix: resolve MAX_TAG_SLOTS error and z-index dropdown issue

- Replace MAX_TAG_SLOTS with totalSlots in document-tag-entry
- Add z-index to SelectContent in base-tags-modal for proper layering

* fix: handle non-text columns in getTagUsage query

- Only apply empty string check for text columns (tag1-tag7)
- Numeric/date/boolean columns only check IS NOT NULL
- Cast values to text for consistent output

* refactor: use EMCN components for KB UI

- Replace @/components/ui imports with @/components/emcn
- Use Combobox instead of Select for dropdowns
- Use EMCN Switch, Button, Input, Label components
- Remove unsupported 'size' prop from EMCN Button

* fix: layout for delete button next to date picker

- Change delete button from absolute to inline positioning
- Add proper column width (w-10) for delete button
- Add empty header cell for delete column
- Apply fix to both document-tag-entry and knowledge-tag-filters

* fix: clear value when switching tag field type

- Reset value to empty when changing type (e.g., boolean to text)
- Reset value when tag name changes and type differs
- Prevents 'true'/'false' from sticking in text inputs

* feat: add full support for number/date/boolean tag filtering in KB search

- Copy all tag types (number, date, boolean) from document to embedding records
- Update processDocumentTags to handle all field types with proper type conversion
- Add number/date/boolean columns to document queries in checkDocumentWriteAccess
- Update chunk creation to inherit all tag types from parent document
- Add getSearchResultFields helper for consistent query result selection
- Support structured filters with operators (eq, gt, lt, between, etc.)
- Fix search queries to include all 28 tag fields in results

* fixing tags import issue

* fix rm file

* reduced number to 3 and date to 2

* fixing lint

* fixed the prop size issue

* increased number from 3 to 5 and boolean from 7 to 2

* fixed number the sql stuff

* progress

* fix document tag and kb tag modals

* update datepicker emcn component

* fix ui

* progress on KB block tags UI

* fix issues with date filters

* fix execution parsing of types for KB tags

* remove migration before merge

* regen migrations

* fix tests and types

* address greptile comments

* fix more greptile comments

* fix filtering logic for multiple of same row

* fix tests

---------

Co-authored-by: priyanshu.solanki <priyanshu.solanki@saviynt.com>
Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
2025-12-19 21:00:35 -08:00
2025-12-09 20:50:28 -08:00
2025-04-01 12:10:58 -07:00
2025-07-29 12:51:43 -07:00

Sim Logo

Build and deploy AI agent workflows in minutes.

Sim.ai Discord Twitter Documentation

Build Workflows with Ease

Design agent workflows visually on a canvas—connect agents, tools, and blocks, then run them instantly.

Workflow Builder Demo

Supercharge with Copilot

Leverage Copilot to generate nodes, fix errors, and iterate on flows directly from natural language.

Copilot Demo

Integrate Vector Databases

Upload documents to a vector store and let agents answer questions grounded in your specific content.

Knowledge Uploads and Retrieval Demo

Quickstart

Cloud-hosted: sim.ai

Sim.ai

Self-hosted: NPM Package

npx simstudio

http://localhost:3000

Note

Docker must be installed and running on your machine.

Options

Flag Description
-p, --port <port> Port to run Sim on (default 3000)
--no-pull Skip pulling latest Docker images

Self-hosted: Docker Compose

# Clone the repository
git clone https://github.com/simstudioai/sim.git

# Navigate to the project directory
cd sim

# Start Sim
docker compose -f docker-compose.prod.yml up -d

Access the application at http://localhost:3000/

Using Local Models with Ollama

Run Sim with local AI models using Ollama - no external APIs required:

# Start with GPU support (automatically downloads gemma3:4b model)
docker compose -f docker-compose.ollama.yml --profile setup up -d

# For CPU-only systems:
docker compose -f docker-compose.ollama.yml --profile cpu --profile setup up -d

Wait for the model to download, then visit http://localhost:3000. Add more models with:

docker compose -f docker-compose.ollama.yml exec ollama ollama pull llama3.1:8b

Using an External Ollama Instance

If you already have Ollama running on your host machine (outside Docker), you need to configure the OLLAMA_URL to use host.docker.internal instead of localhost:

# Docker Desktop (macOS/Windows)
OLLAMA_URL=http://host.docker.internal:11434 docker compose -f docker-compose.prod.yml up -d

# Linux (add extra_hosts or use host IP)
docker compose -f docker-compose.prod.yml up -d  # Then set OLLAMA_URL to your host's IP

Why? When running inside Docker, localhost refers to the container itself, not your host machine. host.docker.internal is a special DNS name that resolves to the host.

For Linux users, you can either:

  • Use your host machine's actual IP address (e.g., http://192.168.1.100:11434)
  • Add extra_hosts: ["host.docker.internal:host-gateway"] to the simstudio service in your compose file

Using vLLM

Sim also supports vLLM for self-hosted models with OpenAI-compatible API:

# Set these environment variables
VLLM_BASE_URL=http://your-vllm-server:8000
VLLM_API_KEY=your_optional_api_key  # Only if your vLLM instance requires auth

When running with Docker, use host.docker.internal if vLLM is on your host machine (same as Ollama above).

Self-hosted: Dev Containers

  1. Open VS Code with the Remote - Containers extension
  2. Open the project and click "Reopen in Container" when prompted
  3. Run bun run dev:full in the terminal or use the sim-start alias
    • This starts both the main application and the realtime socket server

Self-hosted: Manual Setup

Requirements:

Note: Sim uses vector embeddings for AI features like knowledge bases and semantic search, which requires the pgvector PostgreSQL extension.

  1. Clone and install dependencies:
git clone https://github.com/simstudioai/sim.git
cd sim
bun install
  1. Set up PostgreSQL with pgvector:

You need PostgreSQL with the vector extension for embedding support. Choose one option:

Option A: Using Docker (Recommended)

# Start PostgreSQL with pgvector extension
docker run --name simstudio-db \
  -e POSTGRES_PASSWORD=your_password \
  -e POSTGRES_DB=simstudio \
  -p 5432:5432 -d \
  pgvector/pgvector:pg17

Option B: Manual Installation

  1. Set up environment:
cd apps/sim
cp .env.example .env  # Configure with required variables (DATABASE_URL, BETTER_AUTH_SECRET, BETTER_AUTH_URL)

Update your .env file with the database URL:

DATABASE_URL="postgresql://postgres:your_password@localhost:5432/simstudio"
  1. Set up the database:

First, configure the database package environment:

cd packages/db
cp .env.example .env 

Update your packages/db/.env file with the database URL:

DATABASE_URL="postgresql://postgres:your_password@localhost:5432/simstudio"

Then run the migrations:

cd packages/db # Required so drizzle picks correct .env file
bunx drizzle-kit migrate --config=./drizzle.config.ts
  1. Start the development servers:

Recommended approach - run both servers together (from project root):

bun run dev:full

This starts both the main Next.js application and the realtime socket server required for full functionality.

Alternative - run servers separately:

Next.js app (from project root):

bun run dev

Realtime socket server (from apps/sim directory in a separate terminal):

cd apps/sim
bun run dev:sockets

Copilot API Keys

Copilot is a Sim-managed service. To use Copilot on a self-hosted instance:

  • Go to https://sim.ai → Settings → Copilot and generate a Copilot API key
  • Set COPILOT_API_KEY environment variable in your self-hosted apps/sim/.env file to that value

Environment Variables

Key environment variables for self-hosted deployments (see apps/sim/.env.example for full list):

Variable Required Description
DATABASE_URL Yes PostgreSQL connection string with pgvector
BETTER_AUTH_SECRET Yes Auth secret (openssl rand -hex 32)
BETTER_AUTH_URL Yes Your app URL (e.g., http://localhost:3000)
NEXT_PUBLIC_APP_URL Yes Public app URL (same as above)
ENCRYPTION_KEY Yes Encryption key (openssl rand -hex 32)
OLLAMA_URL No Ollama server URL (default: http://localhost:11434)
VLLM_BASE_URL No vLLM server URL for self-hosted models
COPILOT_API_KEY No API key from sim.ai for Copilot features

Troubleshooting

Ollama models not showing in dropdown (Docker)

If you're running Ollama on your host machine and Sim in Docker, change OLLAMA_URL from localhost to host.docker.internal:

OLLAMA_URL=http://host.docker.internal:11434 docker compose -f docker-compose.prod.yml up -d

See Using an External Ollama Instance for details.

Database connection issues

Ensure PostgreSQL has the pgvector extension installed. When using Docker, wait for the database to be healthy before running migrations.

Port conflicts

If ports 3000, 3002, or 5432 are in use, configure alternatives:

# Custom ports
NEXT_PUBLIC_APP_URL=http://localhost:3100 POSTGRES_PORT=5433 docker compose up -d

Tech Stack

Contributing

We welcome contributions! Please see our Contributing Guide for details.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Made with ❤️ by the Sim Team

Description
No description provided
Readme Apache-2.0 478 MiB
Languages
TypeScript 69.4%
MDX 30.1%
Python 0.2%
CSS 0.1%