feat(platform, blocks): Webhook-triggered blocks (#8358)

- feat(blocks): Add GitHub Pull Request Trigger block ## feat(platform): Add support for Webhook-triggered blocks - ⚠️ Add `PLATFORM_BASE_URL` setting - Add webhook config option and `BlockType.WEBHOOK` to `Block` - Add check to `Block.__init__` to enforce type and shape of webhook event filter - Add check to `Block.__init__` to enforce `payload` input on webhook blocks - Add check to `Block.__init__` to disable webhook blocks if `PLATFORM_BASE_URL` is not set - Add `Webhook` model + CRUD functions in `backend.data.integrations` to represent webhooks created by our system - Add `IntegrationWebhook` to DB schema + reference `AgentGraphNode.webhook_id` - Add `set_node_webhook(..)` in `backend.data.graph` - Add webhook-related endpoints: - `POST /integrations/{provider}/webhooks/{webhook_id}/ingress` endpoint, to receive webhook payloads, and for all associated nodes create graph executions - Add `Node.is_triggered_by_event_type(..)` helper method - `POST /integrations/{provider}/webhooks/{webhook_id}/ping` endpoint, to allow testing a webhook - Add `WebhookEvent` + pub/sub functions in `backend.data.integrations` - Add `backend.integrations.webhooks` module, including: - `graph_lifecycle_hooks`, e.g. `on_graph_activate(..)`, to handle corresponding webhook creation etc. - Add calls to these hooks in the graph create/update endpoints - `BaseWebhooksManager` + `GithubWebhooksManager` to handle creating + registering, removing + deregistering, and retrieving existing webhooks, and validating incoming payloads ## Other improvements - fix(blocks): Allow having an input and output pin with the same name - fix(blocks): Add tooltip with description in places where block inputs are rendered without `NodeHandle` - feat(blocks): Allow hiding inputs (e.g. `payload`) with `SchemaField(hidden=True)` - fix(frontend): Fix `MultiSelector` component styling - feat(frontend): Add `AlertDialog` UI component - feat(frontend): Add `NodeMultiSelectInput` component - feat(backend/data): Add `NodeModel` with `graph_id`, `graph_version`; `GraphModel` with `user_id` - Add `make_graph_model(..)` helper function in `backend.data.graph` - refactor(backend/data): Make `RedisEventQueue` generic and move to `backend.data.execution` - refactor(frontend): Deduplicate & clean up code for different block types in `generateInputHandles(..)` in `CustomNode` - dx(backend): Add `MissingConfigError`, `NeedConfirmation` exception --------- Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2026-01-09 15:17:59 -05:00 · 2024-11-25 18:42:36 +01:00
parent 464b5309d7
commit eef9bbe991
44 changed files with 2788 additions and 302 deletions
--- a/docs/content/platform/new_blocks.md
+++ b/docs/content/platform/new_blocks.md
@@ -83,7 +83,7 @@ Follow these steps to create and test a new block:

     In this case, we're mocking the `get_request` method to always return a dictionary with an 'extract' key, simulating a successful API response. This allows us to test the block's logic without making actual network requests, which could be slow, unreliable, or rate-limited.

-5. **Implement the `run` method with error handling:**, this should contain the main logic of the block:
+5. **Implement the `run` method with error handling.** This should contain the main logic of the block:

   ```python
   def run(self, input_data: Input, **kwargs) -> BlockOutput:
@@ -234,7 +234,7 @@ All our existing handlers and the base class can be found [here][OAuth2 handlers

 Every handler must implement the following parts of the [`BaseOAuthHandler`] interface:

-```python title="autogpt_platform/backend/backend/integrations/oauth/base.py"
+```python title="backend/integrations/oauth/base.py"
 --8<-- "autogpt_platform/backend/backend/integrations/oauth/base.py:BaseOAuthHandler1"
 --8<-- "autogpt_platform/backend/backend/integrations/oauth/base.py:BaseOAuthHandler2"
 --8<-- "autogpt_platform/backend/backend/integrations/oauth/base.py:BaseOAuthHandler3"
@@ -249,13 +249,13 @@ Aside from implementing the `OAuthHandler` itself, adding a handler into the sys

 - Adding the handler class to `HANDLERS_BY_NAME` under [`integrations/oauth/__init__.py`](https://github.com/Significant-Gravitas/AutoGPT/blob/master/autogpt_platform/backend/backend/integrations/oauth/__init__.py)

-```python title="autogpt_platform/backend/backend/integrations/oauth/__init__.py"
+```python title="backend/integrations/oauth/__init__.py"
 --8<-- "autogpt_platform/backend/backend/integrations/oauth/__init__.py:HANDLERS_BY_NAMEExample"
 ```

 - Adding `{provider}_client_id` and `{provider}_client_secret` to the application's `Secrets` under [`util/settings.py`](https://github.com/Significant-Gravitas/AutoGPT/blob/master/autogpt_platform/backend/backend/util/settings.py)

-```python title="autogpt_platform/backend/backend/util/settings.py"
+```python title="backend/util/settings.py"
 --8<-- "autogpt_platform/backend/backend/util/settings.py:OAuthServerCredentialsExample"
 ```

@@ -286,13 +286,13 @@ Finally you will need to add the provider to the `CredentialsType` enum in [`fro

 - GitHub blocks with API key + OAuth2 support: [`blocks/github`](https://github.com/Significant-Gravitas/AutoGPT/tree/master/autogpt_platform/backend/backend/blocks/github/)

-```python title="blocks/github/issues.py"
+```python title="backend/blocks/github/issues.py"
 --8<-- "autogpt_platform/backend/backend/blocks/github/issues.py:GithubCommentBlockExample"
 ```

 - GitHub OAuth2 handler: [`integrations/oauth/github.py`](https://github.com/Significant-Gravitas/AutoGPT/blob/master/autogpt_platform/backend/backend/integrations/oauth/github.py)

-```python title="blocks/github/github.py"
+```python title="backend/integrations/oauth/github.py"
 --8<-- "autogpt_platform/backend/backend/integrations/oauth/github.py:GithubOAuthHandlerExample"
 ```

@@ -300,18 +300,148 @@ Finally you will need to add the provider to the `CredentialsType` enum in [`fro

 - Google OAuth2 handler: [`integrations/oauth/google.py`](https://github.com/Significant-Gravitas/AutoGPT/blob/master/autogpt_platform/backend/backend/integrations/oauth/google.py)

-```python title="integrations/oauth/google.py"
+```python title="backend/integrations/oauth/google.py"
 --8<-- "autogpt_platform/backend/backend/integrations/oauth/google.py:GoogleOAuthHandlerExample"
 ```

 You can see that google has defined a `DEFAULT_SCOPES` variable, this is used to set the scopes that are requested no matter what the user asks for.

-```python title="blocks/google/_auth.py"
+```python title="backend/blocks/google/_auth.py"
 --8<-- "autogpt_platform/backend/backend/blocks/google/_auth.py:GoogleOAuthIsConfigured"
 ```

 You can also see that `GOOGLE_OAUTH_IS_CONFIGURED` is used to disable the blocks that require OAuth if the oauth is not configured. This is in the `__init__` method of each block. This is because there is no api key fallback for google blocks so we need to make sure that the oauth is configured before we allow the user to use the blocks.

+### Webhook-triggered Blocks
+
+Webhook-triggered blocks allow your agent to respond to external events in real-time.
+These blocks are triggered by incoming webhooks from third-party services
+rather than being executed manually.
+
+Creating and running a webhook-triggered block involves three main components:
+
+- The block itself, which specifies:
+    - Inputs for the user to select a resource and events to subscribe to
+    - A `credentials` input with the scopes needed to manage webhooks
+    - Logic to turn the webhook payload into outputs for the webhook block
+- The `WebhooksManager` for the corresponding webhook service provider, which handles:
+    - (De)registering webhooks with the provider
+    - Parsing and validating incoming webhook payloads
+- The credentials system for the corresponding service provider, which may include an `OAuthHandler`
+
+There is more going on under the hood, e.g. to store and retrieve webhooks and their
+links to nodes, but to add a webhook-triggered block you shouldn't need to make changes
+to those parts of the system.
+
+#### Creating a Webhook-triggered Block
+
+To create a webhook-triggered block, follow these additional steps on top of the basic block creation process:
+
+1. **Define `webhook_config`** in your block's `__init__` method.
+
+    <details>
+    <summary>Example: <code>GitHubPullRequestTriggerBlock</code></summary>
+
+    ```python title="backend/blocks/github/triggers.py"
+    --8<-- "autogpt_platform/backend/backend/blocks/github/triggers.py:example-webhook_config"
+    ```
+    </details>
+
+    <details>
+    <summary><code>BlockWebhookConfig</code> definition</summary>
+
+    ```python title="backend/data/block.py"
+    --8<-- "autogpt_platform/backend/backend/data/block.py:BlockWebhookConfig"
+    ```
+    </details>
+
+2. **Define event filter input** in your block's Input schema.
+    This allows the user to select which specific types of events will trigger the block in their agent.
+
+    <details>
+    <summary>Example: <code>GitHubPullRequestTriggerBlock</code></summary>
+
+    ```python title="backend/blocks/github/triggers.py"
+    --8<-- "autogpt_platform/backend/backend/blocks/github/triggers.py:example-event-filter"
+    ```
+    </details>
+
+    - The name of the input field (`events` in this case) must match `webhook_config.event_filter_input`.
+    - The event filter itself must be a Pydantic model with only boolean fields.
+
+4. **Include payload field** in your block's Input schema.
+
+    <details>
+    <summary>Example: <code>GitHubTriggerBase</code></summary>
+
+    ```python title="backend/blocks/github/triggers.py"
+    --8<-- "autogpt_platform/backend/backend/blocks/github/triggers.py:example-payload-field"
+    ```
+    </details>
+
+5. **Define `credentials` input** in your block's Input schema.
+    - Its scopes must be sufficient to manage a user's webhooks through the provider's API
+    - See [Blocks with authentication](#blocks-with-authentication) for further details
+
+6. **Process webhook payload** and output relevant parts of it in your block's `run` method.
+
+    <details>
+    <summary>Example: <code>GitHubPullRequestTriggerBlock</code></summary>
+
+    ```python
+    def run(self, input_data: Input, **kwargs) -> BlockOutput:
+        yield "payload", input_data.payload
+        yield "sender", input_data.payload["sender"]
+        yield "event", input_data.payload["action"]
+        yield "number", input_data.payload["number"]
+        yield "pull_request", input_data.payload["pull_request"]
+    ```
+
+    Note that the `credentials` parameter can be omitted if the credentials
+    aren't used at block runtime, like in the example.
+    </details>
+
+#### Adding a Webhooks Manager
+
+To add support for a new webhook provider, you'll need to create a WebhooksManager that implements the `BaseWebhooksManager` interface:
+
+```python title="backend/integrations/webhooks/base.py"
+--8<-- "autogpt_platform/backend/backend/integrations/webhooks/base.py:BaseWebhooksManager1"
+
+--8<-- "autogpt_platform/backend/backend/integrations/webhooks/base.py:BaseWebhooksManager2"
+--8<-- "autogpt_platform/backend/backend/integrations/webhooks/base.py:BaseWebhooksManager3"
+--8<-- "autogpt_platform/backend/backend/integrations/webhooks/base.py:BaseWebhooksManager4"
+--8<-- "autogpt_platform/backend/backend/integrations/webhooks/base.py:BaseWebhooksManager5"
+```
+
+And add a reference to your `WebhooksManager` class in `WEBHOOK_MANAGERS_BY_NAME`:
+
+```python title="backend/integrations/webhooks/__init__.py"
+--8<-- "autogpt_platform/backend/backend/integrations/webhooks/__init__.py:WEBHOOK_MANAGERS_BY_NAME"
+```
+
+#### Example: GitHub Webhook Integration
+
+<details>
+<summary>
+GitHub Webhook triggers: <a href="https://github.com/Significant-Gravitas/AutoGPT/blob/master/autogpt_platform/backend/backend/blocks/github/triggers.py"><code>blocks/github/triggers.py</code></a>
+</summary>
+
+```python title="backend/blocks/github/triggers.py"
+--8<-- "autogpt_platform/backend/backend/blocks/github/triggers.py:GithubTriggerExample"
+```
+</details>
+
+<details>
+<summary>
+GitHub Webhooks Manager: <a href="https://github.com/Significant-Gravitas/AutoGPT/blob/master/autogpt_platform/backend/backend/integrations/webhooks/github.py"><code>integrations/webhooks/github.py</code></a>
+</summary>
+
+```python title="backend/integrations/webhooks/github.py"
+--8<-- "autogpt_platform/backend/backend/integrations/webhooks/github.py:GithubWebhooksManager"
+```
+</details>
+
 ## Key Points to Remember

 - **Unique ID**: Give your block a unique ID in the **init** method.