Files
AutoGPT/autogpt_platform/backend/test/sdk/test_sdk_patching.py
Swifty c955e9a4d7 feat(blocks): Add Airtable Integration (#10338)
## Overview

This PR adds comprehensive Airtable integration to the AutoGPT platform,
enabling users to seamlessly connect their Airtable bases with AutoGPT
workflows for powerful no-code automation capabilities.

## Why Airtable Integration?

Airtable is one of the most popular no-code databases used by teams for
project management, CRMs, inventory tracking, and countless other use
cases. This integration brings significant value:

- **Data Automation**: Automate data entry, updates, and synchronization
between Airtable and other services
- **Workflow Triggers**: React to changes in Airtable bases with
webhook-based triggers
- **Schema Management**: Programmatically create and manage Airtable
table structures
- **Bulk Operations**: Efficiently process large amounts of data with
batch create/update/delete operations

## Key Features

### 🔌 Webhook Trigger
- **AirtableWebhookTriggerBlock**: Listens for changes in Airtable bases
and triggers workflows
- Supports filtering by table, view, and specific fields
- Includes webhook signature validation for security

### 📊 Record Operations  
- **AirtableCreateRecordsBlock**: Create single or multiple records (up
to 10 at once)
- **AirtableUpdateRecordsBlock**: Update existing records with upsert
support
- **AirtableDeleteRecordsBlock**: Delete single or multiple records
- **AirtableGetRecordBlock**: Retrieve specific record details
- **AirtableListRecordsBlock**: Query records with filtering, sorting,
and pagination

### 🏗️ Schema Management
- **AirtableCreateTableBlock**: Create new tables with custom field
definitions
- **AirtableUpdateTableBlock**: Modify table properties
- **AirtableAddFieldBlock**: Add new fields to existing tables
- **AirtableUpdateFieldBlock**: Update field properties

## Technical Implementation Details

### Authentication
- Supports both API Key and OAuth authentication methods
- OAuth implementation includes proper token refresh handling
- Credentials are securely managed through the platform's credential
system

### Webhook Security
- Added `credentials` parameter to WebhooksManager interface for proper
signature validation
- HMAC-SHA256 signature verification ensures webhook authenticity
- Webhook cursor tracking prevents duplicate event processing

### API Integration
- Comprehensive API client (`_api.py`) with full type safety
- Proper error handling and response validation
- Support for all Airtable field types and operations

## Changes 🏗️ 

### Added Blocks:
- AirtableWebhookTriggerBlock
- AirtableCreateRecordsBlock
- AirtableDeleteRecordsBlock
- AirtableGetRecordBlock
- AirtableListRecordsBlock
- AirtableUpdateRecordsBlock
- AirtableAddFieldBlock
- AirtableCreateTableBlock
- AirtableUpdateFieldBlock
- AirtableUpdateTableBlock

### Modified Files:
- Updated WebhooksManager interface to support credential-based
validation
- Modified all webhook handlers to support the new interface

## Test Plan 📋

### Manual Testing Performed:
1. **Authentication Testing**
   -  Verified API key authentication works correctly
   -  Tested OAuth flow including token refresh
   -  Confirmed credentials are properly encrypted and stored

2. **Webhook Testing**
   -  Created webhook subscriptions for different table events
   -  Verified signature validation prevents unauthorized requests
   -  Tested cursor tracking to ensure no duplicate events
   -  Confirmed webhook cleanup on block deletion

3. **Record Operations Testing**
   -  Created single and batch records with various field types
   -  Updated records with and without upsert functionality
   -  Listed records with filtering, sorting, and pagination
   -  Deleted single and multiple records
   -  Retrieved individual record details

4. **Schema Management Testing**
   -  Created tables with multiple field types
   -  Added fields to existing tables
   -  Updated table and field properties
   -  Verified proper error handling for invalid field types

5. **Error Handling Testing**
   -  Tested with invalid credentials
   -  Verified proper error messages for API limits
   -  Confirmed graceful handling of network errors

### Security Considerations 🔒

1. **API Key Management**
   - API keys are stored encrypted in the credential system
   - Keys are never logged or exposed in error messages
   - Credentials are passed securely through the execution context

2. **Webhook Security**
   - HMAC-SHA256 signature validation on all incoming webhooks
   - Webhook URLs use secure ingress endpoints
   - Proper cleanup of webhooks when blocks are deleted

3. **OAuth Security**
   - OAuth tokens are securely stored and refreshed
   - Scopes are limited to necessary permissions
   - Token refresh happens automatically before expiration

## Configuration Requirements

No additional environment variables or configuration changes are
required. The integration uses the existing credential management
system.

## Checklist 📋

#### For code changes:
- [x] I have read the [contributing
instructions](https://github.com/Significant-Gravitas/AutoGPT/blob/master/.github/CONTRIBUTING.md)
- [x] Confirmed that `make lint` passes
- [x] Confirmed that `make test` passes  
- [x] Updated documentation where needed
- [x] Added/updated tests for new functionality
- [x] Manually tested all blocks with real Airtable bases
- [x] Verified backwards compatibility of webhook interface changes

#### Security:
- [x] No hard-coded secrets or sensitive information
- [x] Proper input validation on all user inputs
- [x] Secure credential handling throughout
2025-07-25 13:26:23 +00:00

152 lines
4.5 KiB
Python

"""
Tests for the SDK's integration patching mechanism.
This test suite verifies that the AutoRegistry correctly patches
existing integration points to include SDK-registered components.
"""
from unittest.mock import MagicMock, Mock, patch
import pytest
from backend.integrations.providers import ProviderName
from backend.sdk import (
AutoRegistry,
BaseOAuthHandler,
BaseWebhooksManager,
Credentials,
ProviderBuilder,
)
class MockOAuthHandler(BaseOAuthHandler):
"""Mock OAuth handler for testing."""
PROVIDER_NAME = ProviderName.GITHUB
@classmethod
async def authorize(cls, *args, **kwargs):
return "mock_auth"
class MockWebhookManager(BaseWebhooksManager):
"""Mock webhook manager for testing."""
PROVIDER_NAME = ProviderName.GITHUB
@classmethod
async def validate_payload(cls, webhook, request, credentials: Credentials | None):
return {}, "test_event"
async def _register_webhook(self, *args, **kwargs):
return "mock_webhook_id", {}
async def _deregister_webhook(self, *args, **kwargs):
pass
class TestWebhookPatching:
"""Test webhook manager patching functionality."""
def setup_method(self):
"""Clear registry."""
AutoRegistry.clear()
def test_webhook_manager_patching(self):
"""Test that webhook managers are correctly patched."""
# Mock the original load_webhook_managers function
def mock_load_webhook_managers():
return {
"existing_webhook": Mock(spec=BaseWebhooksManager),
}
# Register a provider with webhooks
(
ProviderBuilder("webhook_provider")
.with_webhook_manager(MockWebhookManager)
.build()
)
# Mock the webhooks module
mock_webhooks_module = MagicMock()
mock_webhooks_module.load_webhook_managers = mock_load_webhook_managers
with patch.dict(
"sys.modules", {"backend.integrations.webhooks": mock_webhooks_module}
):
AutoRegistry.patch_integrations()
# Call the patched function
result = mock_webhooks_module.load_webhook_managers()
# Original webhook should still exist
assert "existing_webhook" in result
# New webhook should be added
assert "webhook_provider" in result
assert result["webhook_provider"] == MockWebhookManager
def test_webhook_patching_no_original_function(self):
"""Test webhook patching when load_webhook_managers doesn't exist."""
# Mock webhooks module without load_webhook_managers
mock_webhooks_module = MagicMock(spec=[])
# Register a provider
(
ProviderBuilder("test_provider")
.with_webhook_manager(MockWebhookManager)
.build()
)
with patch.dict(
"sys.modules", {"backend.integrations.webhooks": mock_webhooks_module}
):
# Should not raise an error
AutoRegistry.patch_integrations()
# Function should not be added if it didn't exist
assert not hasattr(mock_webhooks_module, "load_webhook_managers")
class TestPatchingIntegration:
"""Test the complete patching integration flow."""
def setup_method(self):
"""Clear registry."""
AutoRegistry.clear()
def test_complete_provider_registration_and_patching(self):
"""Test the complete flow from provider registration to patching."""
# Mock webhooks module
mock_webhooks = MagicMock()
mock_webhooks.load_webhook_managers = lambda: {"original": Mock()}
# Create a fully featured provider
(
ProviderBuilder("complete_provider")
.with_api_key("COMPLETE_KEY", "Complete API Key")
.with_oauth(MockOAuthHandler, scopes=["read", "write"])
.with_webhook_manager(MockWebhookManager)
.build()
)
# Apply patches
with patch.dict(
"sys.modules",
{
"backend.integrations.webhooks": mock_webhooks,
},
):
AutoRegistry.patch_integrations()
# Verify webhook patching
webhook_result = mock_webhooks.load_webhook_managers()
assert "complete_provider" in webhook_result
assert webhook_result["complete_provider"] == MockWebhookManager
assert "original" in webhook_result # Original preserved
if __name__ == "__main__":
pytest.main([__file__, "-v"])