- Resolves#8748
The webhooks system as is works really well for full blown enterprise
webhooks managed via a UI. It does not work for more "chill guy" webhook
tools that just send notifications sometimes.
## Changes 🏗️
- feat(blocks): Add Compass transcription trigger block
- feat(backend): Amend webhooks system to support manual-set-up webhooks
- Make event filter input optional on webhook-triggered nodes
- Make credentials optional on webhook-triggered nodes
- Add code path to re-use existing manual webhook on graph update
- Add `ManualWebhookManagerBase`
- feat(frontend): Add UI to pass webhook URL to user on manual-set-up
webhook blocks
- fix(backend): Strip webhook info from node objects for graph export
- refactor(backend): Rename `backend.integrations.webhooks.base` to
`._base`
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
Bumps [cryptography](https://github.com/pyca/cryptography) from 43.0.3
to 44.0.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst">cryptography's
changelog</a>.</em></p>
<blockquote>
<p>44.0.0 - 2024-11-27</p>
<pre><code>
* **BACKWARDS INCOMPATIBLE:** Dropped support for LibreSSL < 3.9.
* Deprecated Python 3.7 support. Python 3.7 is no longer supported by
the
Python core team. Support for Python 3.7 will be removed in a future
``cryptography`` release.
* Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL
3.4.0.
* macOS wheels are now built against the macOS 10.13 SDK. Users on older
versions of macOS should upgrade, or they will need to build
``cryptography`` themselves.
* Enforce the :rfc:`5280` requirement that extended key usage extensions
must
not be empty.
* Added support for timestamp extraction to the
:class:`~cryptography.fernet.MultiFernet` class.
* Relax the Authority Key Identifier requirements on root CA
certificates
during X.509 verification to allow fields permitted by :rfc:`5280` but
forbidden by the CA/Browser BRs.
* Added support for
:class:`~cryptography.hazmat.primitives.kdf.argon2.Argon2id`
when using OpenSSL 3.2.0+.
* Added support for the :class:`~cryptography.x509.Admissions`
certificate extension.
* Added basic support for PKCS7 decryption (including S/MIME 3.2) via
:func:`~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_der`,
:func:`~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_pem`,
and
:func:`~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_smime`.
<p>.. _v43-0-3:<br />
</code></pre></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="f299a48153"><code>f299a48</code></a>
remove deprecated call (<a
href="https://redirect.github.com/pyca/cryptography/issues/12052">#12052</a>)</li>
<li><a
href="439eb0594a"><code>439eb05</code></a>
Bump version for 44.0.0 (<a
href="https://redirect.github.com/pyca/cryptography/issues/12051">#12051</a>)</li>
<li><a
href="2c5ad4d8dc"><code>2c5ad4d</code></a>
chore(deps): bump maturin from 1.7.4 to 1.7.5 in /.github/requirements
(<a
href="https://redirect.github.com/pyca/cryptography/issues/12050">#12050</a>)</li>
<li><a
href="d23968addd"><code>d23968a</code></a>
chore(deps): bump libc from 0.2.165 to 0.2.166 (<a
href="https://redirect.github.com/pyca/cryptography/issues/12049">#12049</a>)</li>
<li><a
href="133c0e02ed"><code>133c0e0</code></a>
Bump x509-limbo and/or wycheproof in CI (<a
href="https://redirect.github.com/pyca/cryptography/issues/12047">#12047</a>)</li>
<li><a
href="f2259d7aa0"><code>f2259d7</code></a>
Bump BoringSSL and/or OpenSSL in CI (<a
href="https://redirect.github.com/pyca/cryptography/issues/12046">#12046</a>)</li>
<li><a
href="e201c870b8"><code>e201c87</code></a>
fixed metadata in changelog (<a
href="https://redirect.github.com/pyca/cryptography/issues/12044">#12044</a>)</li>
<li><a
href="c6104cc366"><code>c6104cc</code></a>
Prohibit Python 3.9.0, 3.9.1 -- they have a bug that causes errors (<a
href="https://redirect.github.com/pyca/cryptography/issues/12045">#12045</a>)</li>
<li><a
href="d6cac753c2"><code>d6cac75</code></a>
Add support for decrypting S/MIME messages (<a
href="https://redirect.github.com/pyca/cryptography/issues/11555">#11555</a>)</li>
<li><a
href="b8e5bfd4d7"><code>b8e5bfd</code></a>
chore(deps): bump libc from 0.2.164 to 0.2.165 (<a
href="https://redirect.github.com/pyca/cryptography/issues/12042">#12042</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/pyca/cryptography/compare/43.0.3...44.0.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
- Resolves part of #8731
### Changes
- Added `depends_on` parameter to SchemaField in `model.py` to specify
field dependencies.
- Updated `useAgentGraph` hook to validate input fields based on their
dependencies, ensuring required fields are set when dependent fields are
filled.
- Modified `BlockIOSubSchemaMeta` to include `depends_on` as an optional
property.
https://github.com/user-attachments/assets/64fd47b3-34dc-48fa-ad90-1c9c5cd4c4a3
---------
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
- resolves part of #8731
### Changes
- Introduced `mutually_exclusive` parameter in `SchemaField` to manage
input exclusivity.
- Implemented logic in `NodeGenericInputField` to disable inputs based
on mutual exclusivity.
- Updated related components to support the new `disabled` state for
inputs.
- Enhanced `BlockIOSubSchemaMeta` to include `mutually_exclusive`
property.
> Currently, I’m disabling the input from the same group (I haven’t
added any frontend validation to prevent users from bypassing it).
https://github.com/user-attachments/assets/71fb9fe4-943b-4724-8acb-6aed2232ed6b
---------
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
<!-- Clearly explain the need for these changes: -->
Monitor page is broken for me
### Changes 🏗️
<!-- Concisely describe all of the changes made in this pull request:
-->
- Updates monitor page to what was in dev before store pr went in
- Updates graph getting endpoint to handle invalid graphs a bit more
graceful
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Test to make sure I can start and view my monitor page
# 🌎 Overview
AutoGPT Store Version 2 expands on the Pre-Store by enhancing agent
discovery, providing richer content presentation, and introducing new
user engagement features. The focus is on creating a visually appealing
and interactive marketplace that allows users to explore and evaluate
agents through images, videos, and detailed descriptions.
### Vision
To create a visually compelling and interactive open-source marketplace
for autonomous AI agents, where users can easily discover, evaluate, and
interact with agents through media-rich listings, ratings, and version
history.
### Objectives
📊 Incorporate visuals (icons, images, videos) into agent listings.
⭐ Introduce a rating system and agent run count.
🔄 Provide version history and update logs from creators.
🔍 Improve user experience with advanced search and filtering features.
### Changes 🏗️
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
---------
Co-authored-by: Bently <tomnoon9@gmail.com>
Co-authored-by: Aarushi <aarushik93@gmail.com>
### Background
Currently, AutoGPT only supports ollama servers running locally. Often,
this is not the case as the ollama server could be running on a more
suited instance, such as a Jetson board. This PR adds "ollama host" to
the input of all LLM blocks, allowing users to select the ollama host
for the LLM blocks.
### Changes 🏗️
- Changes contained within blocks/llm.py:
- Adding ollama host input to all LLM blocks
- Fixed incorrect parsing of prompt when passing to ollama in the
StructuredResponse block
- Used ollama.Client instances to accomplish this.
### Testing 🔍
Tested all LLM blocks with Ollama remote hosts as well as with the
default localhost value.
### Related issues
https://github.com/Significant-Gravitas/AutoGPT/issues/8225
---------
Co-authored-by: Fried-Squid <Fried-Squid>
Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
Though this is technically possible with the AddToDictionary and
AddToList Blocks, that approach alone feels like a hidden work-around
rather than an intuitive feature, and I'm happy with the duplication in
the name of better experience for our users here.
Changes 🏗️
Added CreateDictionaryBlock class that creates a dictionary from the
provided key-value pairs.
Added CreateListBlock class that creates a list from the provided
values.
---------
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
- Resolves#8931
- Follow-up to #8358
### Changes 🏗️
- Avoid double specifying provider and cred types on `credentials`
inputs
- Move `credentials` sub-schema validation from `CredentialsField` to
`CredentialsMetaInput.validate_credentials_field_schema(..)`, which is
called in `BlockSchema.__pydantic_init_subclass__`
- Use `ProviderName` enum globally
First step for the PAYG System.
### Changes 🏗️
- Add `stripeCustomerId` to `User` model
- Rename model `UserBlockCredit` to `CreditTransaction`
- Rename model `UserBlockCreditType` to `CreditTransactionType`
- Update related code
- Add a migration
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
This is a follow-up of
https://github.com/Significant-Gravitas/AutoGPT/pull/8752
There are several APIs and functions related to graph execution that are
unused now.
There is also confusion about the name of `GraphExecution` that exists
in graph.py & execution.py.
### Changes 🏗️
* Renamed `GraphExecution` in `execution.py` to `GraphExecutionEntry`,
this is only used as a queue entry for execution.
* Removed unused `get_graph_execution` & `list_executions` in
`execution.py`.
* Removed `with_run` option on `get_graph` function in `graph.py`.
* Removed `GraphMetaWithRuns`
* Removed exposed functions only for testing.
* Removed `executions` fields in Graph model.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
---------
Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>
The graph version is bumped on each save. While the agent version is
changed, the past execution history is gone because the monitor page
only shows the latest version's execution history.
### Changes 🏗️
- Add `get_executions` on the backend that returns all executions of all
graphs for a user
- Display all executions (for all versions) for graphs in Monitor
- Rename ts mirror type `ExecutionMeta` to `GraphExecution` for
consistency with the backend
- Remove redundant `FlowRun` type on the frontend and use
`GraphExecution` instead
- Round execution duration text in Monitor to one decimal place
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
---------
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
ExtractTextInformationBlock is only supporting extracting one match.
### Changes 🏗️
Adding find_all option to ExtractTextInformationBlock.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
Some table foreign key sources are not properly indexed, causing the
potential full table scan on the code queries.
### Changes 🏗️
Added DB indexes on several tables.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
<!-- Clearly explain the need for these changes: -->
### Background
The Github PR blocks are not able to function properly because the
correct endpoint is not getting called.
- Resolves#8667
### Changes 🏗️
* Added logic to derive correct endpoint from the given PR url.
* This logic is implemented in multiple blocks viz.
`GithubReadPullRequestBlock`, `GithubAssignPRReviewerBlock`,
`GithubUnassignPRReviewerBlock`, `GithubListPRReviewersBlock`.
### Test
* Github List PR Reviewers
<img width="511" alt="Screenshot 2024-12-03 at 11 03 59 PM"
src="https://github.com/user-attachments/assets/9c69edcf-c2f4-42d2-954d-0fc4d903ae22">
* Github Read Pull Request (Include Pr Changes checked)
<img width="417" alt="Screenshot 2024-12-06 at 12 01 41 PM"
src="https://github.com/user-attachments/assets/986fada7-7fbb-41b6-a42a-47d1e11fa562">
---------
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
**Summary:**
This PR removes the `GEMINI_FLASH_1_5_EXP` model (due to inference on
OpenRouter not working) and introduces several new models to the
`LlmModel` enum. Corresponding updates have been made to the metadata
configurations and block cost settings to reflect the changes.
**Key Changes:**
1. **Removed Models:**
- `GEMINI_FLASH_1_5_EXP`
2. **Added New Models:**
- `QWEN_QWQ_32B_PREVIEW`
- `NOUSRESEARCH_HERMES_3_LLAMA_3_1_405B`
- `NOUSRESEARCH_HERMES_3_LLAMA_3_1_70B`
- `AMAZON_NOVA_LITE_V1`
- `AMAZON_NOVA_MICRO_V1`
- `AMAZON_NOVA_PRO_V1`
- `MICROSOFT_WIZARDLM_2_8X22B`
- `GRYPHE_MYTHOMAX_L2_13B`
3. **Metadata Updates:**
- Added metadata entries for the new models with a max output tokens of
4,000 tokens.
4. **Cost Configuration Updates:**
- Defined block costs for the newly added models:
- `QWEN_QWQ_32B_PREVIEW`: 2 credits
- All other new models: 1 credit
**Testing:**
- Verified that all models can be called without errors with the AI Text
generator block
We aren't using Launch Darkly locally and so it's not set up but it was
still attempting to shut down LaunchDarkly when the app shutdown,
causing errors on shutdown. This PR fixes that issue by entirely
disabling LD on local machines.
### Changes 🏗️
Added a contextmanager to handle LaunchDarkly start up and shutdown
Added a check for local environment in said context manager
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
Adding Exa API blocks because it does very cool search and web scrapping
### Changes 🏗️
Adding Exa API blocks:
Search
Added a new calendar and time input
Added _auth.py for Exa API too.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
---------
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
### Changes 🏗️
We've seen some symptoms where during the initial startup of the
application the obtained db connection produces an error when the
network is unreachable. This code made sure that the obtained connection
could run the query and retry it on the spot if it was unable to do so.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
We still use plain Jinja objects for text formatting in our block codes.
### Changes 🏗️
Introduced a `TextFormatter` utility class that uses jina
SandboxedEnvironment for safer text formatting.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
We need stricter URL validation for the hostname we can request in the
block code.
### Changes 🏗️
* Canonicalization: Ensures \ are converted to /, adds http:// if
missing, and normalizes the input URL.
* Scheme Check: Only http or https are allowed.
* Hostname Validation:
- Ensures a hostname exists.
- Converts it to an IDNA ASCII form to prevent Unicode spoofing.
- Verifies that the hostname matches a safe DNS pattern.
* Trusted Origins Check: Allows certain hostnames explicitly if needed.
* IP Resolution and Blocking:
- Resolves the hostname to its IP addresses.
- Checks against a list of private/reserved IP networks to prevent SSRF
to internal services.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
- Resolves#8743
- Follow-up to #8358
### Demo
https://github.com/user-attachments/assets/f983dfa2-2dc2-4ab0-8373-e768ba17e6f7
### Changes 🏗️
- feat(frontend): Add webhook status indicator on `CustomNode`
- Add `webhookId` to frontend node data model
- fix(backend): Fix webhook ping endpoint
- Remove `provider` path parameter
- Fix return values and error handling
- Fix `WebhooksManager.trigger_ping(..)`
- Add `credentials` parameter
- Fix usage of credentials
- Fix `.data.integrations.wait_for_webhook_event(..)`
- Add `AsyncRedisEventBus.wait_for_event(..)`
- feat(frontend): Add `BackendAPIProvider` + `useBackendAPI`
- feat(frontend): Improve layout of node header
Before:
After:
- refactor(backend): Clean up `.data.integrations`
- refactor(backend): Fix naming in `.data.queue` for understandability
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Add webhook block, save -> gray indicator
- [x] Add necessary info to webhook block, save -> green indicator
- [x] Remove necessary info, save -> gray indicator
---------
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
<!-- Clearly explain the need for these changes: -->
I want to be able to have agents 3d print things and deliver them to my
house!
### Changes 🏗️
<!-- Concisely describe all of the changes made in this pull request:
-->
- Adds slant3d as a provider
- Adds slant3d order webhook (disabled on the cloud by default due to
how it notifies users)
- Adds several blocks to order from slant3d
- Diables Get Orders (for the same reason as webhook)
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan
<details>
<summary>Test Plan</summary>
- [ ] Add filament block and fill API key
- [ ] Run filament block
- [ ] Add slice block and use this value:
https://files.printables.com/media/prints/1081287/stls/8176524_a9edde2d-68c1-41de-a207-b584fcf42f30_f9127d5b-39ed-4ef8-b59f-d3a0bc874373/rod-holder.stl
- [ ] Run slice block
- [ ] Add estimate blocks (print and shipping) and use your address, and
the above file
- [ ] select petg and count 1
- [ ] run the blocks
- [ ] Create an order using same information
- [ ] Run the block and note the order number
- [ ] Delete the create order block so you don't keep ordering stuff
- [ ] Run get orders block
- [ ] Check your order exists
- [ ] Run the cancel order block with the order id
- [ ] run the get orders block and check the order no longer exists
</details>
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
- resolve#8739
I don't think so that this is a frontend issue [might be wrong] ,
because if we are not classifying that a particular input is `advanced =
true/false`. Then we automatically get `advanced = True`.
<img width="1142" alt="Screenshot 2024-11-27 at 10 36 59 AM"
src="https://github.com/user-attachments/assets/e8d9c037-5b8b-45b2-b40b-8390bc63de99">
- Resolves#8859
- Follow-up to #8751
### Changes
- Add `autogpt_libs` to the backend CI path filter
- Add `ruff format` step for `autogpt_libs` to `linter.py` and
`pre-commit` config
- Run `poetry run format` with the new setup
<!-- Clearly explain the need for these changes: -->
On windows this file load kept crashing stuff on startup so I specified
the encoding
### Changes 🏗️
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] Run the app!
Replace Dict with dict
### Changes 🏗️
Replace Dict with dict
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
linter.py, only applies in the `backend` module, not `autogpt_libs`.
The scope of this PR is to clear this out.
### Changes 🏗️
* Add a linting scope to both the `backend` & `autogpt_libs` modules,
and apply the linter.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
When calculating the next month, we are not rolling the month number
causing an error on credits.
### Changes 🏗️
Add modulo while calculating next month.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
- Move `autogpt_libs.supabase_integration_credentials_store` into
`backend`
- `.store` -> `backend.integrations.credentials_store`
- `.types` -> added to `backend.data.model`
- Rename `SupabaseIntegrationCredentialsStore` to
`IntegrationCredentialsStore`
We wanted to get a few security things in quickly in #8403 and had to
make some compromises to do so. This picks those up and fixes them.
- Resolves#8540
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
This fix is triggered by an error observed on db connection failure on
SupaBase:
```
2024-11-28 07:45:24,724 INFO [DatabaseManager] Starting...
2024-11-28 07:45:24,726 INFO [PID-18|DatabaseManager|Prisma-7f32369c-6432-4edb-8e71-ef820332b9e4] Acquiring connection started...
2024-11-28 07:45:24,726 INFO [PID-18|DatabaseManager|Prisma-7f32369c-6432-4edb-8e71-ef820332b9e4] Acquiring connection completed successfully.
{"is_panic":false,"message":"Can't reach database server at `...pooler.supabase.com:5432`\n\nPlease make sure your database server is running at `....pooler.supabase.com:5432`.","meta":{"database_host":"...pooler.supabase.com","database_port":5432},"error_code":"P1001"}
2024-11-28 07:45:35,153 INFO [PID-18|DatabaseManager|Prisma-7f32369c-6432-4edb-8e71-ef820332b9e4] Acquiring connection failed: Could not connect to the query engine. Retrying now...
2024-11-28 07:45:36,155 INFO [PID-18|DatabaseManager|Redis-e14a33de-2d81-4536-b48b-a8aa4b1f4766] Acquiring connection started...
2024-11-28 07:45:36,181 INFO [PID-18|DatabaseManager|Redis-e14a33de-2d81-4536-b48b-a8aa4b1f4766] Acquiring connection completed successfully.
2024-11-28 07:45:36,183 INFO [PID-18|DatabaseManager|Pyro-2722cd29-4dbd-4cf9-882f-73842658599d] Starting Pyro Service started...
2024-11-28 07:45:36,189 INFO [DatabaseManager] Connected to Pyro; URI = PYRO:DatabaseManager@0.0.0.0:8005
2024-11-28 07:46:28,241 ERROR Error in get_user_integrations: All connection attempts failed
```
Where even
```
2024-11-28 07:45:35,153 INFO [PID-18|DatabaseManager|Prisma-7f32369c-6432-4edb-8e71-ef820332b9e4] Acquiring connection failed: Could not connect to the query engine. Retrying now...
```
is present, the Redis connection is still proceeding without waiting for
the retry to complete. This was likely caused by Tenacity not fully
awaiting the DB connection acquisition command.
### Changes 🏗️
* Add special handling for the async function to explicitly await the
function execution result on each retry.
* Explicitly raise exceptions on `db.connect()` if the db is not
connected even after `prisma.connect()` command.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
This PR adds the first few Hubspot blocks so we can create _real_ sales
and marketing agents.
### Changes 🏗️
Added Hubspot blocks;
- Aded auth for hubspot
- Added Company block
- Added Contact block
- Added Engagement block
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
We've started enabling cost based on the *partial value* of the
`credentials` field. And this logic has never been supported.
### Changes 🏗️
* Add partial object matching on the input data filter for evaluating
the block cost.
* Add missing credentials for `ExtractWebsiteContentBlock`
* Removed fallback cost on LLM blocks.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
