Commit Graph

5 Commits

Author SHA1 Message Date
Krzysztof Czerwinski
4041e1f39c feat(backend/infra): Cleanup platform and db docker compose files (#10830)
Supabase `db/docker/docker-compose.yml` overrides env vars set in
`autogpt_platform/.env` file. This PR fixes that and simplifies the
compose files further.

### Changes 🏗️

`autogpt_platform/docker-compose.platform.yml`:
- Move hardcoded `DATABASE_URL` and `DIRECT_URL` to `x-backend-env` on
top as it repeats for most services.
- Remove `RABBITMQ_DEFAULT_USER` and `RABBITMQ_DEFAULT_PASS` from
`rabbitmq` service and use env files instead

`autogpt_platform/db/docker/docker-compose.yml`:
- Remove hardcoded env vars from `x-supabase-env` - these are already
defined in `.env`
- Remove env vars from services that are already defined in `.env` files

*Changes to db compose file only affect self-hosted Supabase*

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Platform, db works when self-hosting
2025-09-04 12:01:49 +00:00
Zamil Majdy
b53c373a59 feat(docker): streamline Supabase to minimal essential services (#10639)
## Summary
Streamline Supabase stack from 13 services to 3 core services for faster
startup and lower resource usage while maintaining full API
compatibility.

## Changes Made

### Core Services (Always Running)
- **Kong**: API gateway providing standard `/auth/v1/` endpoints and API
key validation
- **Auth**: GoTrue authentication service for user management
- **Database**: PostgreSQL with pgvector support for data persistence

### Removed Services (9 services eliminated)
- `rest` (PostgREST API) - not needed for auth-only usage
- `realtime` (real-time subscriptions) - not used by platform
- `storage` (file storage) - platform uses separate file handling  
- `imgproxy` (image processing) - not required for core functionality
- `meta` (database metadata) - not needed for runtime operations
- `functions` (edge functions) - not utilized
- `analytics` (Logflare) - monitoring overhead not needed locally
- `vector` (log collection) - not required for basic operation
- `supavisor` (connection pooler) - direct DB access sufficient for
local dev

### Studio (Development Only)  
- Moved to `local` profile: `docker compose --profile local up`
- Available for database management during development
- Excluded from normal startup for cleaner production-like environment

## Benefits
- **80% faster startup**: 3 services vs 13 services  
- **Lower resource usage**: Significant reduction in memory/CPU
consumption
- **Simpler debugging**: Fewer moving parts, cleaner logs, easier
troubleshooting
- **Maintained compatibility**: All auth functionality preserved through
Kong

## Backwards Compatibility
 **No breaking changes**
- All existing auth endpoints (`/auth/v1/*`) work unchanged
- API key authentication (`anon`/`service_role`) preserved  
- CORS and security policies maintained via Kong
- No application code changes required

## Testing
- [x] Docker compose starts successfully with minimal services
- [x] Auth endpoints accessible via Kong at `/auth/v1/`
- [x] Database connectivity maintained
- [x] Studio accessible with `--profile local` flag
- [x] All existing environment variables preserved

## File Changes
- `autogpt_platform/docker-compose.yml`: Removed unnecessary Supabase
services, moved studio to local profile
- `autogpt_platform/db/docker/docker-compose.yml`: Cleaned up service
dependencies on analytics/vector

🤖 Generated with [Claude Code](https://claude.ai/code)
2025-08-14 04:55:45 +00:00
Zamil Majdy
4bfeddc03d feat(platform/docker): add frontend service to docker-compose with env config improvements (#10615)
## Summary
This PR adds the frontend service to the Docker Compose configuration,
enabling `docker compose up` to run the complete stack, including the
frontend. It also implements comprehensive environment variable
improvements, unified .env file support, and fixes Docker networking
issues.

## Key Changes

### 🐳 Docker Compose Improvements
- **Added frontend service** to `docker-compose.yml` and
`docker-compose.platform.yml`
- **Production build**: Uses `pnpm build + serve` instead of dev server
for better stability and lower memory usage
- **Service dependencies**: Frontend now waits for backend services
(`rest_server`, `websocket_server`) to be ready
- **YAML anchors**: Implemented DRY configuration to avoid duplicating
environment values

### 📁 Unified .env File Support
- **Frontend .env loading**: Automatically loads `.env` file during
Docker build and runtime
- **Backend .env loading**: Optional `.env` file support with fallback
to sensible defaults in `settings.py`
- **Single source of truth**: All `NEXT_PUBLIC_*` and API keys can be
defined in respective `.env` files
- **Docker integration**: Updated `.dockerignore` to include `.env`
files in build context
- **Git tracking**: Frontend and backend `.env` files are now trackable
(removed from gitignore)

### 🔧 Environment Variable Architecture
- **Dual environment strategy**: 
- Server-side code uses Docker service names
(`http://rest_server:8006/api`)
  - Client-side code uses localhost URLs (`http://localhost:8006/api`)
- **Comprehensive config**: Added build args and runtime environment
variables
- **Network compatibility**: Fixes connection issues between frontend
and backend containers
- **Shared backend variables**: Common environment variables (service
hosts, auth settings) centralized using YAML anchors

### 🛠️ Code Improvements
- **Centralized env-config helper** (`/frontend/src/lib/env-config.ts`)
with server-side priority
- **Updated all frontend code** to use shared environment helpers
instead of direct `process.env` access
- **Consistent API**: All environment variable access now goes through
helper functions
- **Settings.py improvements**: Better defaults for CORS origins and
optional .env file loading

### 🔗 Files Changed
- `docker-compose.yml` & `docker-compose.platform.yml` - Added frontend
service and shared backend env vars
- `frontend/Dockerfile` - Simplified build process to use .env files
directly
- `backend/settings.py` - Optional .env loading and better defaults
- `frontend/src/lib/env-config.ts` - New centralized environment
configuration
- `.dockerignore` - Allow .env files in build context
- `.gitignore` - Updated to allow frontend/backend .env files
- Multiple frontend files - Updated to use env helpers
- Updates to both auto installer scripts to work with the latest setup!

## Benefits
-  **Single command deployment**: `docker compose up` now runs
everything
-  **Better reliability**: Production build reduces memory usage and
crashes
-  **Network compatibility**: Proper container-to-container
communication
-  **Maintainable config**: Centralized environment variable management
with .env files
-  **Development friendly**: Works in both Docker and local development
-  **API key management**: Easy configuration through .env files for
all services
-  **No more manual env vars**: Frontend and backend automatically load
their respective .env files

## Testing
-  Verified Docker service communication works correctly
-  Frontend responds and serves content properly  
-  Environment variables are correctly resolved in both server and
client contexts
-  No connection errors after implementing service dependencies
-  .env file loading works correctly in both build and runtime phases
-  Backend services work with and without .env files present

### Checklist 📋

#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Lluis Agusti <hi@llu.lu>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: Claude <claude@users.noreply.github.com>
Co-authored-by: Bentlybro <Github@bentlybro.com>
2025-08-14 03:28:18 +00:00
Zamil Majdy
4e6144803b fix(platform): Fix possible db-config permission denied when running two different Supabase versions (#9652)
The change in https://github.com/Significant-Gravitas/AutoGPT/pull/9620
introduces a breaking change in the database volume content; however,
the database's volume location does not change, making switching between
two versions clash.

### Changes 🏗️

Renamed db-config named volume to supabase-config.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] CI
2025-03-19 23:39:09 +07:00
Zamil Majdy
780fddc2a0 feat(platform)!: Lock Supabase docker-compose code (#9620)
We have been submoduling Supabase for provisioning local Supabase
instances using docker-compose. Aside from the huge size of unrelated
code being pulled, there is also the risk of pulling unintentional
breaking change from the upstream to the platform.

The latest Supabase changes hide the 5432 port from the supabase-db
container and shift it to the supavisor, the instance that we are
currently not using. This causes an error in the existing setup.

## BREAKING CHANGES

This change will introduce different volume locations for the database
content, pulling this change will make the data content fresh from the
start. To keep your old data with this change, execute this command:
```
cp -r supabase/docker/volumes/db/data db/docker/volumes/db/data
```


### Changes 🏗️

The scope of this PR is snapshotting the current docker-compose code
obtained from the Supabase repository and embedding it into our
repository. This will eliminate the need for submodule / recursive
cloning and bringing the entire Supabase repository into the platform.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Existing CI
2025-03-19 23:39:09 +07:00