This commit introduces several related updates across the application, focusing on establishing core functionality for bot management, admin routing, and metadata handling.
Key changes include:
- Updating API routes for admin and proxy functionality.
- Refactoring the `BotManager` to handle the starting of active bots.
- Implementing necessary CRUD operations for model metadata and server information.
- Updating database migration and session management files.
- Adjusting relevant template files for the admin dashboard display.
This commit updates several core components related to administrative settings, data store management, and memory management within the application.
Specifically, it includes:
- Updates to API routes for managing data stores and proxy response wrapping.
- Refinements in the memory management system.
- Adjustments to database migration logic related to settings.
- Updates to relevant schema and template files.
This commit introduces several improvements related to context handling, server/model association logic, and database connection pooling safety.
### Changes Implemented:
In `app/api/v1/routes/proxy.py`:
* **Context Filtering with Exception**: Implemented conditional filtering of available metadata based on required context tokens. A specific exception is added to bypass this filter for system tasks (e.g., `sys_build_` requests) to prevent deadlocks during large context operations.
In `app/crud/server_crud.py`:
* **Flexible Model Matching**: Updated the logic for matching servers to models to allow flexible matching, supporting scenarios where `model` can match `model:latest` and vice versa, and explicitly handling Ollama naming conventions.
In `app/database/session.py`:
* **Safe Connection Pooling**: Removed specific pool configuration parameters (`pool_size`, `max_overflow`, `pool_timeout`, `pool_recycle`) from `create_async_engine` initialization to rely on safer defaults, particularly when using SQLite/aiosqlite, improving robustness in high-concurrency scenarios.
This commit addresses several areas to improve the stability, performance, and graceful shutdown procedure of the application:
### Changes:
* **Refine Background Task Handling (`app/api/v1/routes/proxy.py`):** Enhanced the background memory processing task to include specific error handling for `asyncio.CancelledError` and general exceptions, ensuring that background tasks are suppressed gracefully during shutdown or failure.
* **Optimize Database Connection Pooling (`app/database/session.py`):** Tuned the SQLAlchemy engine configuration for better performance and SQLite compatibility by setting explicit pool sizes (`pool_size`, `max_overflow`, `pool_recycle`) and adjusting connection arguments (`check_same_thread: False`).
* **Implement Graceful Shutdown (`app/main.py`):** Updated the `lifespan` hook to implement a more robust shutdown sequence. This now explicitly cancels background tasks, waits for pending tasks to settle, and explicitly disposes of the database engine, preventing potential connection warnings on exit.
Summary of Changes:
- [INFRA] Added `space_hunter.py` to the project root for new system utilities.
- [CONFIG] Updated `.gitignore` to include `*.db-wal` files and exclude `docs/article` directory.
- [UI] Refreshed admin templates (`models_manager.html`, `settings.html`) to reflect new architectural capabilities.
- [CORE] Modified `architect_manager.py` to support image generation capabilities (`supports_images`).
- [CORE] Updated `memory_manager.py` to improve context formatting and category handling.
- [API] Implemented new route logic and rate limiting dependencies in `proxy.py` and `admin.py`.
- [MODELS] Updated `models.py` to support new fields and `migrations.py` to handle schema evolution.
Technical Notes:
- The `architect_manager` now manages assets including image nodes.
- Memory formatting has been streamlined to handle multiple titles per category.
- Admin settings and model management pages have been updated to visualize these new capabilities.
BREAKING CHANGE: None.
- Add error logging for token tracking stream failures in proxy
- Add asyncio import to datastores module for async operations
- Minor formatting and cleanup across admin, openai_proxy, and bot_manager
Standardize naming conventions across API routes, database layer, and
templates to ensure UI labels match internal SafeStore identifiers.
Updates affect datastores, proxy authentication, migrations, and
knowledge node configuration.
This commit updates several core components across the application to establish foundational routing, proxy handling, robust logging setup, database session management, and server CRUD operations.
Changes include:
- Updating API route definitions for conception and proxy endpoints.
- Enhancing logging configuration with rotating file handlers.
- Adjusting server CRUD logic to incorporate model filtering.
- Refinements in database session management.
- Updating the base HTML template structure.
This commit introduces the foundational structure and routes for the administrative interface, including necessary API endpoints, core memory management services, database models, agent node logic, and the corresponding admin dashboard templates.
Key changes include:
- Updating API routes for administration, data stores, and playground features.
- Implementing core memory management and asset handling.
- Defining database models, sessions, and migrations.
- Establishing the structure for agent components and administrative templates.
- Add MCP servers configuration to VirtualAgent model (JSON field)
- Implement binary update endpoint with background task processing
- Enhance proxy routes with encryption utilities for secure data handling
- Add graceful shutdown flag to BotManager for task lifecycle control
- Update InstanceManager with pipmaster integration for dependency management
- Extend admin UI templates for instances and settings management
- Implement POST/PUT/DELETE endpoints in tools API routes
- Extend ToolsManager with save, update, and delete capabilities
- Add responsive styling fixes for admin panel layout
- Update admin navigation icons in base template
- Enhance bots management UI with improved layout and functionality
Add chunk_size and chunk_overlap fields to DataStore model for
configurable document processing. Update admin routes, tools, and
templates to support new DataStore settings management.
- Add DataStore configuration columns (chunk_size, chunk_overlap)
- Update admin API routes for settings handling
- Enhance tools and conception endpoints with DataStore integration
- Refresh admin templates (base, settings, tools)
- Refine server CRUD error handling
- Refactor conception routes with improved JSON handling
- Enhance admin API with carbon tracking and system info
- Update node builder dependencies and validation
- Streamline OpenAI proxy and reverse proxy logic
- Extend CRUD operations for model metadata and servers
- Update database migrations and core models
- Improve workflow engine and node registry architecture
- Refresh admin dashboard templates and UI components
- add internal system bypass for API key validation in dependencies
- update admin routes with improved settings form handling
- extend OpenAI proxy with datetime logging capabilities
- refactor core events and database migration infrastructure
- update admin templates (base, conception, help, model playground, settings)
- sync project dependencies in pyproject.toml and requirements.txt
- Add image description extraction for multimodal requests in proxy routing
- Update admin models manager with improved template context handling
- Reduce default retry attempts from 5 to 2 for faster failure response
- Fix deep copy issue in playground chat streaming
- Update model metadata CRUD with token pricing improvements
- Enhance admin UI templates and styling for ensembles and routers
- Add encrypted_api_key column to OllamaServer model
- Update migrations for new database schema
- Enhance server CRUD operations to handle encrypted API keys
- Modify instance manager initialization
- Update admin edit server form and help documentation
- Add vision_processor column to database schema
- Extend playground chat with improved model filtering and streaming
- Update server pool selection logic in proxy routes
- Refactor server CRUD update operations
- Enhance admin UI templates for agents, ensembles, and server management
- Extend SmartRouter with rules (JSON) and is_active fields
- Update admin API for vision enabler form handling
- Add migration for legacy model pools schema fixes
- Adjust admin routers template layout
Implement ensemble orchestrator feature that distributes requests across
multiple models in parallel and aggregates responses. Includes:
- API endpoints for ensemble CRUD operations and request handling
- Database models and migrations for ensemble configuration
- Enhanced server CRUD with model-aware filtering
- Admin UI for creating, editing, and managing ensemble configurations
Ensembles support configurable master/parallel models, routing strategies,
and response aggregation methods.
- Extend admin API with settings endpoints and metadata controls
- Update proxy routing for improved request handling
- Refactor instance manager with async networking capabilities
- Enhance server CRUD for active model queries across all servers
- Update database migrations and models schema
- Refresh admin UI templates (agents, pools, ensembles, routers, models, instances, dashboard, settings, help)
- Add TTFT, TPS, and token_count fields to ProxyEvent tracking
- Add is_code_model and is_chat_model flags to ModelMetadata
- Create database migration for total_tokens column
- Update admin API routes and templates for new metadata fields
- Track Time To First Token (TTFT) in ProxyEvent for performance monitoring
- Enhance token tracking stream handling in proxy routes
- Add instance management database model and settings schema
- Update admin templates for live status and settings management
- Add cryptographic imports to dependencies for enhanced security
- Implement content type validation in admin routes
- Update proxy router with dependency injection improvements
- Tune retry configuration defaults for better reliability
- Enhance database migrations with regex support and expanded logic
