## Overview This PR adds comprehensive Airtable integration to the AutoGPT platform, enabling users to seamlessly connect their Airtable bases with AutoGPT workflows for powerful no-code automation capabilities. ## Why Airtable Integration? Airtable is one of the most popular no-code databases used by teams for project management, CRMs, inventory tracking, and countless other use cases. This integration brings significant value: - **Data Automation**: Automate data entry, updates, and synchronization between Airtable and other services - **Workflow Triggers**: React to changes in Airtable bases with webhook-based triggers - **Schema Management**: Programmatically create and manage Airtable table structures - **Bulk Operations**: Efficiently process large amounts of data with batch create/update/delete operations ## Key Features ### 🔌 Webhook Trigger - **AirtableWebhookTriggerBlock**: Listens for changes in Airtable bases and triggers workflows - Supports filtering by table, view, and specific fields - Includes webhook signature validation for security ### 📊 Record Operations - **AirtableCreateRecordsBlock**: Create single or multiple records (up to 10 at once) - **AirtableUpdateRecordsBlock**: Update existing records with upsert support - **AirtableDeleteRecordsBlock**: Delete single or multiple records - **AirtableGetRecordBlock**: Retrieve specific record details - **AirtableListRecordsBlock**: Query records with filtering, sorting, and pagination ### 🏗️ Schema Management - **AirtableCreateTableBlock**: Create new tables with custom field definitions - **AirtableUpdateTableBlock**: Modify table properties - **AirtableAddFieldBlock**: Add new fields to existing tables - **AirtableUpdateFieldBlock**: Update field properties ## Technical Implementation Details ### Authentication - Supports both API Key and OAuth authentication methods - OAuth implementation includes proper token refresh handling - Credentials are securely managed through the platform's credential system ### Webhook Security - Added `credentials` parameter to WebhooksManager interface for proper signature validation - HMAC-SHA256 signature verification ensures webhook authenticity - Webhook cursor tracking prevents duplicate event processing ### API Integration - Comprehensive API client (`_api.py`) with full type safety - Proper error handling and response validation - Support for all Airtable field types and operations ## Changes 🏗️ ### Added Blocks: - AirtableWebhookTriggerBlock - AirtableCreateRecordsBlock - AirtableDeleteRecordsBlock - AirtableGetRecordBlock - AirtableListRecordsBlock - AirtableUpdateRecordsBlock - AirtableAddFieldBlock - AirtableCreateTableBlock - AirtableUpdateFieldBlock - AirtableUpdateTableBlock ### Modified Files: - Updated WebhooksManager interface to support credential-based validation - Modified all webhook handlers to support the new interface ## Test Plan 📋 ### Manual Testing Performed: 1. **Authentication Testing** - ✅ Verified API key authentication works correctly - ✅ Tested OAuth flow including token refresh - ✅ Confirmed credentials are properly encrypted and stored 2. **Webhook Testing** - ✅ Created webhook subscriptions for different table events - ✅ Verified signature validation prevents unauthorized requests - ✅ Tested cursor tracking to ensure no duplicate events - ✅ Confirmed webhook cleanup on block deletion 3. **Record Operations Testing** - ✅ Created single and batch records with various field types - ✅ Updated records with and without upsert functionality - ✅ Listed records with filtering, sorting, and pagination - ✅ Deleted single and multiple records - ✅ Retrieved individual record details 4. **Schema Management Testing** - ✅ Created tables with multiple field types - ✅ Added fields to existing tables - ✅ Updated table and field properties - ✅ Verified proper error handling for invalid field types 5. **Error Handling Testing** - ✅ Tested with invalid credentials - ✅ Verified proper error messages for API limits - ✅ Confirmed graceful handling of network errors ### Security Considerations 🔒 1. **API Key Management** - API keys are stored encrypted in the credential system - Keys are never logged or exposed in error messages - Credentials are passed securely through the execution context 2. **Webhook Security** - HMAC-SHA256 signature validation on all incoming webhooks - Webhook URLs use secure ingress endpoints - Proper cleanup of webhooks when blocks are deleted 3. **OAuth Security** - OAuth tokens are securely stored and refreshed - Scopes are limited to necessary permissions - Token refresh happens automatically before expiration ## Configuration Requirements No additional environment variables or configuration changes are required. The integration uses the existing credential management system. ## Checklist 📋 #### For code changes: - [x] I have read the [contributing instructions](https://github.com/Significant-Gravitas/AutoGPT/blob/master/.github/CONTRIBUTING.md) - [x] Confirmed that `make lint` passes - [x] Confirmed that `make test` passes - [x] Updated documentation where needed - [x] Added/updated tests for new functionality - [x] Manually tested all blocks with real Airtable bases - [x] Verified backwards compatibility of webhook interface changes #### Security: - [x] No hard-coded secrets or sensitive information - [x] Proper input validation on all user inputs - [x] Secure credential handling throughout
This is the frontend for AutoGPT's next generation
🧢 Getting Started
This project uses pnpm as the package manager via corepack. Corepack is a Node.js tool that automatically manages package managers without requiring global installations.
Prerequisites
Make sure you have Node.js 16.10+ installed. Corepack is included with Node.js by default.
⚠️ Migrating from yarn
This project was previously using yarn1, make sure to clean up the old files if you set it up previously with yarn:
rm -f yarn.lock && rm -rf node_modulesThen follow the setup steps below.
Setup
-
Enable corepack (run this once on your system):
corepack enableThis enables corepack to automatically manage pnpm based on the
packageManagerfield inpackage.json. -
Install dependencies:
pnpm i -
Start the development server:
pnpm dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.
Subsequent Runs
For subsequent development sessions, you only need to run:
pnpm dev
Every time a new Front-end dependency is added by you or others, you will need to run pnpm i to install the new dependencies.
Available Scripts
pnpm dev- Start development serverpnpm build- Build for productionpnpm start- Start production serverpnpm lint- Run ESLint and Prettier checkspnpm format- Format code with Prettierpnpm type-check- Run TypeScript type checkingpnpm test- Run Playwright testspnpm test-ui- Run Playwright tests with UIpnpm fetch:openapi- Fetch OpenAPI spec from backendpnpm generate:api-client- Generate API client from OpenAPI specpnpm generate:api-all- Fetch OpenAPI spec and generate API client
This project uses next/font to automatically optimize and load Inter, a custom Google Font.
🔄 Data Fetching Strategy
Note
You don't need to run the OpenAPI commands below to run the Front-end. You will only need to run them when adding or modifying endpoints on the Backend API and wanting to use those on the Frontend.
This project uses an auto-generated API client powered by Orval, which creates type-safe API clients from OpenAPI specifications.
How It Works
- Backend Requirements: Each API endpoint needs a summary and tag in the OpenAPI spec
- Operation ID Generation: FastAPI generates operation IDs using the pattern
{method}{tag}{summary} - Spec Fetching: The OpenAPI spec is fetched from
http://localhost:8006/openapi.jsonand saved to the frontend - Spec Transformation: The OpenAPI spec is cleaned up using a custom transformer (see
autogpt_platform/frontend/src/app/api/transformers) - Client Generation: Auto-generated client includes TypeScript types, API endpoints, and Zod schemas, organized by tags
API Client Commands
# Fetch OpenAPI spec from backend and generate client
pnpm generate:api-all
# Only fetch the OpenAPI spec
pnpm fetch:openapi
# Only generate the client (after spec is fetched)
pnpm generate:api-client
Using the Generated Client
The generated client provides React Query hooks for both queries and mutations:
Queries (GET requests)
import { useGetV1GetNotificationPreferences } from "@/app/api/__generated__/endpoints/auth/auth";
const { data, isLoading, isError } = useGetV1GetNotificationPreferences({
query: {
select: (res) => res.data,
// Other React Query options
},
});
Mutations (POST, PUT, DELETE requests)
import { useDeleteV2DeleteStoreSubmission } from "@/app/api/__generated__/endpoints/store/store";
import { getGetV2ListMySubmissionsQueryKey } from "@/app/api/__generated__/endpoints/store/store";
import { useQueryClient } from "@tanstack/react-query";
const queryClient = useQueryClient();
const { mutateAsync: deleteSubmission } = useDeleteV2DeleteStoreSubmission({
mutation: {
onSuccess: () => {
// Invalidate related queries to refresh data
queryClient.invalidateQueries({
queryKey: getGetV2ListMySubmissionsQueryKey(),
});
},
},
});
// Usage
await deleteSubmission({
submissionId: submission_id,
});
Server Actions
For server-side operations, you can also use the generated client functions directly:
import { postV1UpdateNotificationPreferences } from "@/app/api/__generated__/endpoints/auth/auth";
// In a server action
const preferences = {
email: "user@example.com",
preferences: {
AGENT_RUN: true,
ZERO_BALANCE: false,
// ... other preferences
},
daily_limit: 0,
};
await postV1UpdateNotificationPreferences(preferences);
Server-Side Prefetching
For server-side components, you can prefetch data on the server and hydrate it in the client cache. This allows immediate access to cached data when queries are called:
import { getQueryClient } from "@/lib/tanstack-query/getQueryClient";
import {
prefetchGetV2ListStoreAgentsQuery,
prefetchGetV2ListStoreCreatorsQuery
} from "@/app/api/__generated__/endpoints/store/store";
import { HydrationBoundary, dehydrate } from "@tanstack/react-query";
// In your server component
const queryClient = getQueryClient();
await Promise.all([
prefetchGetV2ListStoreAgentsQuery(queryClient, {
featured: true,
}),
prefetchGetV2ListStoreAgentsQuery(queryClient, {
sorted_by: "runs",
}),
prefetchGetV2ListStoreCreatorsQuery(queryClient, {
featured: true,
sorted_by: "num_agents",
}),
]);
return (
<HydrationBoundary state={dehydrate(queryClient)}>
<MainMarkeplacePage />
</HydrationBoundary>
);
This pattern improves performance by serving pre-fetched data from the server while maintaining the benefits of client-side React Query features.
Configuration
The Orval configuration is located in autogpt_platform/frontend/orval.config.ts. It generates two separate clients:
- autogpt_api_client: React Query hooks for client-side data fetching
- autogpt_zod_schema: Zod schemas for validation
For more details, see the Orval documentation or check the configuration file.
🚩 Feature Flags
This project uses LaunchDarkly for feature flags, allowing us to control feature rollouts and A/B testing.
Using Feature Flags
Check if a feature is enabled
import { Flag, useGetFlag } from "@/services/feature-flags/use-get-flag";
function MyComponent() {
const isAgentActivityEnabled = useGetFlag(Flag.AGENT_ACTIVITY);
if (!isAgentActivityEnabled) {
return null; // Hide feature
}
return <div>Feature is enabled!</div>;
}
Protect entire components
import { withFeatureFlag } from "@/services/feature-flags/with-feature-flag";
const MyFeaturePage = withFeatureFlag(MyPageComponent, "my-feature-flag");
Testing with Feature Flags
For local development or running Playwright tests locally, use mocked feature flags by setting NEXT_PUBLIC_PW_TEST=true in your .env file. This bypasses LaunchDarkly and uses the mock values defined in the code.
Adding New Flags
- Add the flag to the
Flagenum inuse-get-flag.ts - Add the flag type to
FlagValuestype - Add mock value to
mockFlagsfor testing - Configure the flag in LaunchDarkly dashboard
🚚 Deploy
TODO
📙 Storybook
Storybook is a powerful development environment for UI components. It allows you to build UI components in isolation, making it easier to develop, test, and document your components independently from your main application.
Purpose in the Development Process
- Component Development: Develop and test UI components in isolation.
- Visual Testing: Easily spot visual regressions.
- Documentation: Automatically document components and their props.
- Collaboration: Share components with your team or stakeholders for feedback.
How to Use Storybook
-
Start Storybook: Run the following command to start the Storybook development server:
pnpm storybookThis will start Storybook on port 6006. Open http://localhost:6006 in your browser to view your component library.
-
Build Storybook: To build a static version of Storybook for deployment, use:
pnpm build-storybook -
Running Storybook Tests: Storybook tests can be run using:
pnpm test-storybook -
Writing Stories: Create
.stories.tsxfiles alongside your components to define different states and variations of your components.
By integrating Storybook into our development workflow, we can streamline UI development, improve component reusability, and maintain a consistent design system across the project.
🔭 Tech Stack
Core Framework & Language
- Next.js - React framework with App Router
- React - UI library for building user interfaces
- TypeScript - Typed JavaScript for better developer experience
Styling & UI Components
- Tailwind CSS - Utility-first CSS framework
- shadcn/ui - Re-usable components built with Radix UI and Tailwind CSS
- Radix UI - Headless UI components for accessibility
- Lucide React - Beautiful & consistent icons
- Framer Motion - Animation library for React
Development & Testing
- Storybook - Component development environment
- Playwright - End-to-end testing framework
- ESLint - JavaScript/TypeScript linting
- Prettier - Code formatting
Backend & Services
- Supabase - Backend-as-a-Service (database, auth, storage)
- Sentry - Error monitoring and performance tracking
Package Management
Additional Libraries
- React Hook Form - Forms with easy validation
- Zod - TypeScript-first schema validation
- React Table - Headless table library
- React Flow - Interactive node-based diagrams
- React Query - Data fetching and caching
- React Query DevTools - Debugging tool for React Query
Development Tools
NEXT_PUBLIC_REACT_QUERY_DEVTOOL- Enable React Query DevTools. Set totrueto enable.