mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-04-30 03:00:41 -04:00
## Block Development SDK - Simplifying Block Creation ### Problem Currently, creating a new block requires manual updates to **5+ files** scattered across the codebase: - `backend/data/block_cost_config.py` - Manually add block costs - `backend/integrations/credentials_store.py` - Add default credentials - `backend/integrations/providers.py` - Register new providers - `backend/integrations/oauth/__init__.py` - Register OAuth handlers - `backend/integrations/webhooks/__init__.py` - Register webhook managers This creates significant friction for developers, increases the chance of configuration errors, and makes the platform difficult to scale. ### Solution This PR introduces a **Block Development SDK** that provides: - Single import for all block development needs: `from backend.sdk import *` - Automatic registration of all block configurations - Zero external file modifications required - Provider-based configuration with inheritance ### Changes 🏗️ #### 1. **New SDK Module** (`backend/sdk/`) - **`__init__.py`**: Unified exports of 68+ block development components - **`registry.py`**: Central auto-registration system for all block configurations - **`builder.py`**: `ProviderBuilder` class for fluent provider configuration - **`provider.py`**: Provider configuration management - **`cost_integration.py`**: Automatic cost application system #### 2. **Provider Builder Pattern** ```python # Configure once, use everywhere my_provider = ( ProviderBuilder("my-service") .with_api_key("MY_SERVICE_API_KEY", "My Service API Key") .with_base_cost(5, BlockCostType.RUN) .build() ) ``` #### 3. **Automatic Cost System** - Provider base costs automatically applied to all blocks using that provider - Override with `@cost` decorator for block-specific pricing - Tiered pricing support with cost filters #### 4. **Dynamic Provider Support** - Modified `ProviderName` enum to accept any string via `_missing_` method - No more manual enum updates for new providers #### 5. **Application Integration** - Added `sync_all_provider_costs()` to `initialize_blocks()` for automatic cost registration - Maintains full backward compatibility with existing blocks #### 6. **Comprehensive Examples** (`backend/blocks/examples/`) - `simple_example_block.py` - Basic block structure - `example_sdk_block.py` - Provider with credentials - `cost_example_block.py` - Various cost patterns - `advanced_provider_example.py` - Custom API clients - `example_webhook_sdk_block.py` - Webhook configuration #### 7. **Extensive Testing** - 6 new test modules with 30+ test cases - Integration tests for all SDK features - Cost calculation verification - Provider registration tests ### Before vs After **Before SDK:** ```python # 1. Multiple complex imports from backend.data.block import Block, BlockCategory, BlockOutput from backend.data.model import SchemaField, CredentialsField # ... many more imports # 2. Update block_cost_config.py BLOCK_COSTS[MyBlock] = [BlockCost(...)] # 3. Update credentials_store.py DEFAULT_CREDENTIALS.append(...) # 4. Update providers.py enum # 5. Update oauth/__init__.py # 6. Update webhooks/__init__.py ``` **After SDK:** ```python from backend.sdk import * # Everything configured in one place my_provider = ( ProviderBuilder("my-service") .with_api_key("MY_API_KEY", "My API Key") .with_base_cost(10, BlockCostType.RUN) .build() ) class MyBlock(Block): class Input(BlockSchema): credentials: CredentialsMetaInput = my_provider.credentials_field() data: String = SchemaField(description="Input data") class Output(BlockSchema): result: String = SchemaField(description="Result") # That's it\! No external files to modify ``` ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Created new blocks using SDK pattern with provider configuration - [x] Verified automatic cost registration for provider-based blocks - [x] Tested cost override with @cost decorator - [x] Confirmed custom providers work without enum modifications - [x] Verified all example blocks execute correctly - [x] Tested backward compatibility with existing blocks - [x] Ran all SDK tests (30+ tests, all passing) - [x] Created blocks with credentials and verified authentication - [x] Tested webhook block configuration - [x] Verified application startup with auto-registration #### For configuration changes: - [x] `.env.example` is updated or already compatible with my changes (no changes needed) - [x] `docker-compose.yml` is updated or already compatible with my changes (no changes needed) - [x] I have included a list of my configuration changes in the PR description (under **Changes**) ### Impact - **Developer Experience**: Block creation time reduced from hours to minutes - **Maintainability**: All block configuration in one place - **Scalability**: Support hundreds of blocks without enum updates - **Type Safety**: Full IDE support with proper type hints - **Testing**: Easier to test blocks in isolation --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Abhimanyu Yadav <122007096+Abhi1992002@users.noreply.github.com>
151 lines
4.5 KiB
Python
151 lines
4.5 KiB
Python
"""
|
|
Tests for the SDK's integration patching mechanism.
|
|
|
|
This test suite verifies that the AutoRegistry correctly patches
|
|
existing integration points to include SDK-registered components.
|
|
"""
|
|
|
|
from unittest.mock import MagicMock, Mock, patch
|
|
|
|
import pytest
|
|
|
|
from backend.integrations.providers import ProviderName
|
|
from backend.sdk import (
|
|
AutoRegistry,
|
|
BaseOAuthHandler,
|
|
BaseWebhooksManager,
|
|
ProviderBuilder,
|
|
)
|
|
|
|
|
|
class MockOAuthHandler(BaseOAuthHandler):
|
|
"""Mock OAuth handler for testing."""
|
|
|
|
PROVIDER_NAME = ProviderName.GITHUB
|
|
|
|
@classmethod
|
|
async def authorize(cls, *args, **kwargs):
|
|
return "mock_auth"
|
|
|
|
|
|
class MockWebhookManager(BaseWebhooksManager):
|
|
"""Mock webhook manager for testing."""
|
|
|
|
PROVIDER_NAME = ProviderName.GITHUB
|
|
|
|
@classmethod
|
|
async def validate_payload(cls, webhook, request):
|
|
return {}, "test_event"
|
|
|
|
async def _register_webhook(self, *args, **kwargs):
|
|
return "mock_webhook_id", {}
|
|
|
|
async def _deregister_webhook(self, *args, **kwargs):
|
|
pass
|
|
|
|
|
|
class TestWebhookPatching:
|
|
"""Test webhook manager patching functionality."""
|
|
|
|
def setup_method(self):
|
|
"""Clear registry."""
|
|
AutoRegistry.clear()
|
|
|
|
def test_webhook_manager_patching(self):
|
|
"""Test that webhook managers are correctly patched."""
|
|
|
|
# Mock the original load_webhook_managers function
|
|
def mock_load_webhook_managers():
|
|
return {
|
|
"existing_webhook": Mock(spec=BaseWebhooksManager),
|
|
}
|
|
|
|
# Register a provider with webhooks
|
|
(
|
|
ProviderBuilder("webhook_provider")
|
|
.with_webhook_manager(MockWebhookManager)
|
|
.build()
|
|
)
|
|
|
|
# Mock the webhooks module
|
|
mock_webhooks_module = MagicMock()
|
|
mock_webhooks_module.load_webhook_managers = mock_load_webhook_managers
|
|
|
|
with patch.dict(
|
|
"sys.modules", {"backend.integrations.webhooks": mock_webhooks_module}
|
|
):
|
|
AutoRegistry.patch_integrations()
|
|
|
|
# Call the patched function
|
|
result = mock_webhooks_module.load_webhook_managers()
|
|
|
|
# Original webhook should still exist
|
|
assert "existing_webhook" in result
|
|
|
|
# New webhook should be added
|
|
assert "webhook_provider" in result
|
|
assert result["webhook_provider"] == MockWebhookManager
|
|
|
|
def test_webhook_patching_no_original_function(self):
|
|
"""Test webhook patching when load_webhook_managers doesn't exist."""
|
|
# Mock webhooks module without load_webhook_managers
|
|
mock_webhooks_module = MagicMock(spec=[])
|
|
|
|
# Register a provider
|
|
(
|
|
ProviderBuilder("test_provider")
|
|
.with_webhook_manager(MockWebhookManager)
|
|
.build()
|
|
)
|
|
|
|
with patch.dict(
|
|
"sys.modules", {"backend.integrations.webhooks": mock_webhooks_module}
|
|
):
|
|
# Should not raise an error
|
|
AutoRegistry.patch_integrations()
|
|
|
|
# Function should not be added if it didn't exist
|
|
assert not hasattr(mock_webhooks_module, "load_webhook_managers")
|
|
|
|
|
|
class TestPatchingIntegration:
|
|
"""Test the complete patching integration flow."""
|
|
|
|
def setup_method(self):
|
|
"""Clear registry."""
|
|
AutoRegistry.clear()
|
|
|
|
def test_complete_provider_registration_and_patching(self):
|
|
"""Test the complete flow from provider registration to patching."""
|
|
# Mock webhooks module
|
|
mock_webhooks = MagicMock()
|
|
mock_webhooks.load_webhook_managers = lambda: {"original": Mock()}
|
|
|
|
# Create a fully featured provider
|
|
(
|
|
ProviderBuilder("complete_provider")
|
|
.with_api_key("COMPLETE_KEY", "Complete API Key")
|
|
.with_oauth(MockOAuthHandler, scopes=["read", "write"])
|
|
.with_webhook_manager(MockWebhookManager)
|
|
.build()
|
|
)
|
|
|
|
# Apply patches
|
|
with patch.dict(
|
|
"sys.modules",
|
|
{
|
|
"backend.integrations.webhooks": mock_webhooks,
|
|
},
|
|
):
|
|
AutoRegistry.patch_integrations()
|
|
|
|
# Verify webhook patching
|
|
webhook_result = mock_webhooks.load_webhook_managers()
|
|
assert "complete_provider" in webhook_result
|
|
assert webhook_result["complete_provider"] == MockWebhookManager
|
|
assert "original" in webhook_result # Original preserved
|
|
|
|
|
|
if __name__ == "__main__":
|
|
pytest.main([__file__, "-v"])
|