mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-02-10 06:45:28 -05:00
## Overview This PR adds comprehensive Airtable integration to the AutoGPT platform, enabling users to seamlessly connect their Airtable bases with AutoGPT workflows for powerful no-code automation capabilities. ## Why Airtable Integration? Airtable is one of the most popular no-code databases used by teams for project management, CRMs, inventory tracking, and countless other use cases. This integration brings significant value: - **Data Automation**: Automate data entry, updates, and synchronization between Airtable and other services - **Workflow Triggers**: React to changes in Airtable bases with webhook-based triggers - **Schema Management**: Programmatically create and manage Airtable table structures - **Bulk Operations**: Efficiently process large amounts of data with batch create/update/delete operations ## Key Features ### 🔌 Webhook Trigger - **AirtableWebhookTriggerBlock**: Listens for changes in Airtable bases and triggers workflows - Supports filtering by table, view, and specific fields - Includes webhook signature validation for security ### 📊 Record Operations - **AirtableCreateRecordsBlock**: Create single or multiple records (up to 10 at once) - **AirtableUpdateRecordsBlock**: Update existing records with upsert support - **AirtableDeleteRecordsBlock**: Delete single or multiple records - **AirtableGetRecordBlock**: Retrieve specific record details - **AirtableListRecordsBlock**: Query records with filtering, sorting, and pagination ### 🏗️ Schema Management - **AirtableCreateTableBlock**: Create new tables with custom field definitions - **AirtableUpdateTableBlock**: Modify table properties - **AirtableAddFieldBlock**: Add new fields to existing tables - **AirtableUpdateFieldBlock**: Update field properties ## Technical Implementation Details ### Authentication - Supports both API Key and OAuth authentication methods - OAuth implementation includes proper token refresh handling - Credentials are securely managed through the platform's credential system ### Webhook Security - Added `credentials` parameter to WebhooksManager interface for proper signature validation - HMAC-SHA256 signature verification ensures webhook authenticity - Webhook cursor tracking prevents duplicate event processing ### API Integration - Comprehensive API client (`_api.py`) with full type safety - Proper error handling and response validation - Support for all Airtable field types and operations ## Changes 🏗️ ### Added Blocks: - AirtableWebhookTriggerBlock - AirtableCreateRecordsBlock - AirtableDeleteRecordsBlock - AirtableGetRecordBlock - AirtableListRecordsBlock - AirtableUpdateRecordsBlock - AirtableAddFieldBlock - AirtableCreateTableBlock - AirtableUpdateFieldBlock - AirtableUpdateTableBlock ### Modified Files: - Updated WebhooksManager interface to support credential-based validation - Modified all webhook handlers to support the new interface ## Test Plan 📋 ### Manual Testing Performed: 1. **Authentication Testing** - ✅ Verified API key authentication works correctly - ✅ Tested OAuth flow including token refresh - ✅ Confirmed credentials are properly encrypted and stored 2. **Webhook Testing** - ✅ Created webhook subscriptions for different table events - ✅ Verified signature validation prevents unauthorized requests - ✅ Tested cursor tracking to ensure no duplicate events - ✅ Confirmed webhook cleanup on block deletion 3. **Record Operations Testing** - ✅ Created single and batch records with various field types - ✅ Updated records with and without upsert functionality - ✅ Listed records with filtering, sorting, and pagination - ✅ Deleted single and multiple records - ✅ Retrieved individual record details 4. **Schema Management Testing** - ✅ Created tables with multiple field types - ✅ Added fields to existing tables - ✅ Updated table and field properties - ✅ Verified proper error handling for invalid field types 5. **Error Handling Testing** - ✅ Tested with invalid credentials - ✅ Verified proper error messages for API limits - ✅ Confirmed graceful handling of network errors ### Security Considerations 🔒 1. **API Key Management** - API keys are stored encrypted in the credential system - Keys are never logged or exposed in error messages - Credentials are passed securely through the execution context 2. **Webhook Security** - HMAC-SHA256 signature validation on all incoming webhooks - Webhook URLs use secure ingress endpoints - Proper cleanup of webhooks when blocks are deleted 3. **OAuth Security** - OAuth tokens are securely stored and refreshed - Scopes are limited to necessary permissions - Token refresh happens automatically before expiration ## Configuration Requirements No additional environment variables or configuration changes are required. The integration uses the existing credential management system. ## Checklist 📋 #### For code changes: - [x] I have read the [contributing instructions](https://github.com/Significant-Gravitas/AutoGPT/blob/master/.github/CONTRIBUTING.md) - [x] Confirmed that `make lint` passes - [x] Confirmed that `make test` passes - [x] Updated documentation where needed - [x] Added/updated tests for new functionality - [x] Manually tested all blocks with real Airtable bases - [x] Verified backwards compatibility of webhook interface changes #### Security: - [x] No hard-coded secrets or sensitive information - [x] Proper input validation on all user inputs - [x] Secure credential handling throughout
152 lines
4.5 KiB
Python
152 lines
4.5 KiB
Python
"""
|
|
Tests for the SDK's integration patching mechanism.
|
|
|
|
This test suite verifies that the AutoRegistry correctly patches
|
|
existing integration points to include SDK-registered components.
|
|
"""
|
|
|
|
from unittest.mock import MagicMock, Mock, patch
|
|
|
|
import pytest
|
|
|
|
from backend.integrations.providers import ProviderName
|
|
from backend.sdk import (
|
|
AutoRegistry,
|
|
BaseOAuthHandler,
|
|
BaseWebhooksManager,
|
|
Credentials,
|
|
ProviderBuilder,
|
|
)
|
|
|
|
|
|
class MockOAuthHandler(BaseOAuthHandler):
|
|
"""Mock OAuth handler for testing."""
|
|
|
|
PROVIDER_NAME = ProviderName.GITHUB
|
|
|
|
@classmethod
|
|
async def authorize(cls, *args, **kwargs):
|
|
return "mock_auth"
|
|
|
|
|
|
class MockWebhookManager(BaseWebhooksManager):
|
|
"""Mock webhook manager for testing."""
|
|
|
|
PROVIDER_NAME = ProviderName.GITHUB
|
|
|
|
@classmethod
|
|
async def validate_payload(cls, webhook, request, credentials: Credentials | None):
|
|
return {}, "test_event"
|
|
|
|
async def _register_webhook(self, *args, **kwargs):
|
|
return "mock_webhook_id", {}
|
|
|
|
async def _deregister_webhook(self, *args, **kwargs):
|
|
pass
|
|
|
|
|
|
class TestWebhookPatching:
|
|
"""Test webhook manager patching functionality."""
|
|
|
|
def setup_method(self):
|
|
"""Clear registry."""
|
|
AutoRegistry.clear()
|
|
|
|
def test_webhook_manager_patching(self):
|
|
"""Test that webhook managers are correctly patched."""
|
|
|
|
# Mock the original load_webhook_managers function
|
|
def mock_load_webhook_managers():
|
|
return {
|
|
"existing_webhook": Mock(spec=BaseWebhooksManager),
|
|
}
|
|
|
|
# Register a provider with webhooks
|
|
(
|
|
ProviderBuilder("webhook_provider")
|
|
.with_webhook_manager(MockWebhookManager)
|
|
.build()
|
|
)
|
|
|
|
# Mock the webhooks module
|
|
mock_webhooks_module = MagicMock()
|
|
mock_webhooks_module.load_webhook_managers = mock_load_webhook_managers
|
|
|
|
with patch.dict(
|
|
"sys.modules", {"backend.integrations.webhooks": mock_webhooks_module}
|
|
):
|
|
AutoRegistry.patch_integrations()
|
|
|
|
# Call the patched function
|
|
result = mock_webhooks_module.load_webhook_managers()
|
|
|
|
# Original webhook should still exist
|
|
assert "existing_webhook" in result
|
|
|
|
# New webhook should be added
|
|
assert "webhook_provider" in result
|
|
assert result["webhook_provider"] == MockWebhookManager
|
|
|
|
def test_webhook_patching_no_original_function(self):
|
|
"""Test webhook patching when load_webhook_managers doesn't exist."""
|
|
# Mock webhooks module without load_webhook_managers
|
|
mock_webhooks_module = MagicMock(spec=[])
|
|
|
|
# Register a provider
|
|
(
|
|
ProviderBuilder("test_provider")
|
|
.with_webhook_manager(MockWebhookManager)
|
|
.build()
|
|
)
|
|
|
|
with patch.dict(
|
|
"sys.modules", {"backend.integrations.webhooks": mock_webhooks_module}
|
|
):
|
|
# Should not raise an error
|
|
AutoRegistry.patch_integrations()
|
|
|
|
# Function should not be added if it didn't exist
|
|
assert not hasattr(mock_webhooks_module, "load_webhook_managers")
|
|
|
|
|
|
class TestPatchingIntegration:
|
|
"""Test the complete patching integration flow."""
|
|
|
|
def setup_method(self):
|
|
"""Clear registry."""
|
|
AutoRegistry.clear()
|
|
|
|
def test_complete_provider_registration_and_patching(self):
|
|
"""Test the complete flow from provider registration to patching."""
|
|
# Mock webhooks module
|
|
mock_webhooks = MagicMock()
|
|
mock_webhooks.load_webhook_managers = lambda: {"original": Mock()}
|
|
|
|
# Create a fully featured provider
|
|
(
|
|
ProviderBuilder("complete_provider")
|
|
.with_api_key("COMPLETE_KEY", "Complete API Key")
|
|
.with_oauth(MockOAuthHandler, scopes=["read", "write"])
|
|
.with_webhook_manager(MockWebhookManager)
|
|
.build()
|
|
)
|
|
|
|
# Apply patches
|
|
with patch.dict(
|
|
"sys.modules",
|
|
{
|
|
"backend.integrations.webhooks": mock_webhooks,
|
|
},
|
|
):
|
|
AutoRegistry.patch_integrations()
|
|
|
|
# Verify webhook patching
|
|
webhook_result = mock_webhooks.load_webhook_managers()
|
|
assert "complete_provider" in webhook_result
|
|
assert webhook_result["complete_provider"] == MockWebhookManager
|
|
assert "original" in webhook_result # Original preserved
|
|
|
|
|
|
if __name__ == "__main__":
|
|
pytest.main([__file__, "-v"])
|