This PR extracts backend changes from the hackathon/copilot branch,
adding enhanced chat capabilities, agent management tools, store
embeddings, and hybrid search functionality.
### Changes 🏗️
**Chat Features:**
- Added chat database layer (`db.py`) for conversation and message
persistence
- Extended chat models with new types and response structures
- New onboarding system prompt for guided user experiences
- Enhanced chat routes with additional endpoints
- Expanded chat service with more capabilities
**Chat Agent Tools:**
- `agent_output.py` - Handle agent execution outputs
- `create_agent.py` - Tool for creating new agents via chat
- `edit_agent.py` - Tool for modifying existing agents
- `find_library_agent.py` - Search and discover library agents
- Enhanced `run_agent.py` with additional functionality
- New `models.py` for shared tool types
**Store Enhancements:**
- `embeddings.py` - Vector embeddings support for semantic search
- `hybrid_search.py` - Combined keyword and semantic search
- `backfill_embeddings.py` - Utility for backfilling existing data
- Updated store database operations
**Admin:**
- Enhanced store admin routes
**Data Layer:**
- New `understanding.py` module for agent understanding/context
**Database Migrations:**
- `add_chat_tables` - Chat conversation and message tables
- `add_store_embeddings` - Embeddings storage for store items
- `enhance_search` - Search index improvements
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Chat endpoints respond correctly
- [x] Agent tools (create/edit/find/run) function properly
- [x] Store embeddings and hybrid search work
- [x] Database migrations apply cleanly
#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
---------
Co-authored-by: Torantulino <40276179@live.napier.ac.uk>
Prisma's generated `types.py` file is 57,000+ lines with complex
recursive TypedDict definitions that exhaust Pyright's type inference
budget. This causes random type errors and makes the type checker
unreliable.
### Changes 🏗️
- Add `gen_prisma_types_stub.py` script that generates a lightweight
`.pyi` stub file
- The stub preserves safe types (Literal, TypeVar) while collapsing
complex TypedDicts to `dict[str, Any]`
- Integrate stub generation into all workflows that run `prisma
generate`:
- `platform-backend-ci.yml`
- `claude.yml`
- `claude-dependabot.yml`
- `copilot-setup-steps.yml`
- `docker-compose.platform.yml`
- `Dockerfile`
- `Makefile` (migrate & reset-db targets)
- `linter.py` (lint & format commands)
- Add `gen-prisma-stub` poetry script entry
- Fix two pre-existing type errors that were previously masked:
- `store/db.py`: Replace private type
`_StoreListingVersion_version_OrderByInput` with dict literal
- `airtable/_webhook.py`: Add cast for `Serializable` type
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Run `poetry run format` - passes with 0 errors (down from 57+)
- [x] Run `poetry run lint` - passes with 0 errors
- [x] Run `poetry run gen-prisma-stub` - generates stub successfully
- [x] Verify stub file is created at correct location with proper
content
#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added a lightweight Prisma type-stub generator and integrated it into
build, lint, CI/CD, and container workflows.
* Build, migration, formatting, and lint steps now generate these stubs
to improve type-checking performance and reduce overhead during builds
and deployments.
* Exposed a project command to run stub generation manually.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds a collection of pre-built store agents that can be loaded
into test databases for development and testing purposes.
### Changes 🏗️
- Add 17 exported agent JSON files in `backend/agents/` directory
- Add `StoreAgent_rows.csv` containing store listing metadata (titles,
descriptions, categories, images)
- Add `load_store_agents.py` script to load agents into the test
database
- Add `load-store-agents` Makefile target for easy execution
**Included Agents:**
- Flux AI Image Generator
- YouTube Transcription Scraper
- Decision Maker Lead Finder
- Smart Meeting Prep
- Automated Support Agent
- Unspirational Poster Maker
- AI Video Generator
- Automated SEO Blog Writer
- Lead Finder (Local Businesses)
- LinkedIn Post Generator
- YouTube to LinkedIn Post Converter
- Personal Newsletter
- Email Scout - Contact Finder Assistant
- YouTube Video to SEO Blog Writer
- AI Webpage Copy Improver
- Domain Name Finder
- AI Function
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Run `make load-store-agents` and verify agents are loaded into the
database
- [x] Verify store listings appear correctly with metadata from CSV
- [x] Confirm no sensitive information (API keys, secrets) is included
in the exported agents
#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
No configuration changes required - this only adds test data and a
loading script.
