Commit Graph

94 Commits

Author SHA1 Message Date
Nicholas Tindle
3c0dea0017 [Snyk] Security upgrade python from 3.11-slim-buster to 3.11.10-slim-bookworm (#8619)
fix: autogpt_platform/backend/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-DEBIAN10-GNUTLS28-6159414
- https://snyk.io/vuln/SNYK-DEBIAN10-NCURSES-1655739
- https://snyk.io/vuln/SNYK-DEBIAN10-NCURSES-1655739
- https://snyk.io/vuln/SNYK-DEBIAN10-NCURSES-5421196
- https://snyk.io/vuln/SNYK-DEBIAN10-NCURSES-5421196

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-11-11 20:33:39 +00:00
Toran Bruce Richards
1f0cbc6500 feat(block): Add AI Music Generator Block with Meta Music Gen (#8532)
* feat(platform): Add AIMusicGeneratorBlock for music generation

* refactor(platform): Refactor AIMusicGeneratorBlock for improved error handling and logging

* refactor(ui): Refactor ContentRenderer to support audio rendering

* format

* Frontend format and lint
2024-11-11 09:36:39 -08:00
Aarushi
e6e47373ac feat(blocks/jina): Add default credentials for Jina (#8603)
add jina defaults
2024-11-10 17:18:53 +00:00
Aarushi
359ae8307a feat(backend): Add API key DB table (#8593)
* add api key db tables

* remove uniqueness constraint on prefix

* add postfix
2024-11-08 17:48:37 +00:00
Aarushi
f719c7e70e feat(blocks): Pinecone blocks (#8535)
* update pinecone

* update blocks

* fix linting

* update test

* update requests

* mock funciton
2024-11-08 17:13:55 +00:00
Aarushi
47759f6951 refactor(backend): Remove config.default.json (#8581)
remove config default josn

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2024-11-07 21:20:15 -06:00
jackfromeast
bcaf3241da fix (backend): Patching the SSRF vulnerability in Github/Web Search/Request related blocks (#8531) 2024-11-08 00:29:18 +00:00
Aarushi
c25d03e945 fix(searchtheweb): Fix the Jina Search Block (#8583)
* update jina search web block

* update to false
2024-11-07 17:26:17 -06:00
Zamil Majdy
91edf08540 feat(backend): Improve pyro reliability by adding connection timeout, retry, cleanup, and dynamic connection thread size (#8574) 2024-11-07 11:34:32 +07:00
Zamil Majdy
86c544177e refactor(backend): Introduced Graph Input & Output Schema, Merge GraphMeta & Graph, Remove subgraph functionality (#8526) 2024-11-07 09:30:51 +07:00
Zamil Majdy
af9ea5bc31 fix(frontend): Broken UI caused by lack of error propagation on Run (#8575) 2024-11-07 08:49:32 +07:00
dependabot[bot]
4259ad686e build(deps): bump the production-dependencies group across 1 directory with 11 updates (#8567)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-05 17:32:15 -06:00
dependabot[bot]
f17c20ed91 build(deps): manual fix!! bump replicate from 0.34.1 to 1.0.3 in /autogpt_platform/backend (#8476)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-05 14:34:37 -06:00
SwiftyOS
c19703150a fix(gitignore): Allow file extensions after .ign. or .ignore. 2024-11-05 16:09:19 +01:00
SwiftyOS
3e7d0e7f1b feat(backend): update .gitignore to include ignore / ign file extension 2024-11-05 16:07:13 +01:00
Zamil Majdy
45fe26b58d feat(platform): Toast Graph Creation Error (#8536)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-05 03:23:41 +00:00
dependabot[bot]
11a4f9fdaa build(deps-dev): bump the development-dependencies group in /autogpt_platform/backend with 2 updates (#8543)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 19:14:05 -06:00
Swifty
594aa996d7 refactor(platform/rest): Update REST API to use standard FastAPI structure (#8519) 2024-11-04 18:12:21 +07:00
Swifty
4d4cc60071 chore(platform): Tidy up repo structure (#8521)
* tidy up

* nextjs env specific .env files
2024-11-01 15:33:26 +00:00
Swifty
edaa938f99 feat(pre-commit): Detect secrets (#8507)
Add detect secrets pre commit
2024-10-31 08:49:50 +00:00
Nicholas Tindle
74d8aa6d90 Update llm.py (#8513)
* Update llm.py

* Update llm.py

* Update llm.py

* Update llm.py

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-30 22:40:38 -07:00
Nicholas Tindle
3aebed61d2 feat(backend): credential swap to api keys in cred store (#8403)
* feat(backend): credential swap

* ci: formatting

* fix: importing is hard okay

* fix: spelln' is hard

* feat: better credential provider handling

* docs: update the imports locations

* fix: test credentials + formatting

* feat: drop continuous read mode

* fix: lint

* feat: fallback credentials

* feat: charge for credential useage and have a bad backup mechnism

* fix: don't save default credentials + add d_id

* fix: formatting

* feat: basic encryption/decryption

* ref: move files around

* ref: sign all blocks out of their credentials

* ref: update target to match a new, and encrypted future

* wip: llm provider merger

* don't delete `credentials` input on nodes

* fix llm block ci issues

* updated get AICredentials

* fix fix

* insert migration to move integration credentials from `auth.user` metadata to `platform.User.metadata`

* fixed migration

* add migration for existing user integration credentials

* disabled reddit and email block

* fix credential handling in LLM blocks

* add other secret fields to credential scrubber migration

* add other secret fields to credential scrubber migration (vol. 2)

* fix: pr fixes

* fix: mock funciton

* add encrypted values

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: SwiftyOS <craigswift13@gmail.com>
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
Co-authored-by: Aarushi <aarushik93@gmail.com>
2024-10-31 02:36:38 +00:00
dependabot[bot]
67ff738fda build(deps): bump croniter from 2.0.7 to 5.0.1 in /autogpt_platform/backend (#8498)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-29 18:54:49 -05:00
Swifty
c7cd6dc026 feat(platform): Add target database model to reflect domain (#8375) 2024-10-29 08:22:35 +01:00
dependabot[bot]
fa04f802b2 build(deps-dev): bump the development-dependencies group in /autogpt_platform/backend with 4 updates (#8471)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-29 00:34:02 +00:00
dependabot[bot]
6fef4fd66e build(deps): bump tenacity from 8.5.0 to 9.0.0 in /autogpt_platform/backend (#8472)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 23:52:08 +00:00
dependabot[bot]
c028dd647e build(deps): bump psutil from 5.9.8 to 6.1.0 in /autogpt_platform/backend (#8473)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 18:45:56 -05:00
dependabot[bot]
308a9bc40e build(deps): bump sentry-sdk from 1.45.0 to 2.17.0 in /autogpt_platform/backend (#8475)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 17:04:48 -05:00
dependabot[bot]
d8317c8257 build(deps): bump websockets from 12.0 to 13.1 in /autogpt_platform/backend (#8477)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-28 20:44:24 +00:00
Zamil Majdy
8938209d0d fix(backend): Avoid long synchronous call to block FastAPI event-loop (#8429) 2024-10-27 23:54:38 +07:00
Zamil Majdy
1e620fdb13 feat(backend): RedisEventQueue into Pub/Sub (#8387) 2024-10-27 23:49:33 +07:00
Aarushi
e2df6019fb feat(blocks): Add pinecone and jina blocks (#8401)
* add pinecone and jina blocks

* udpate based on comments

* backend updates

* frontend updates

* type hint

* more type hints

* another type hint

* update run signature

* shared jina provider

* fix linting

* lockfile

* remove noqa

* remove noqa

* remove vector db folder

* line

* update pincone credentials provider

* fix imports

* formating

* update frontend

* Test (#8425)

* h

* Discard changes to autogpt_platform/backend/poetry.lock

* fix: broken dep

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-24 21:01:53 +01:00
Zamil Majdy
1321faf9db fix(backend): Fix error pin output not being propagated into the next nodes (#8408) 2024-10-24 17:13:58 +00:00
Zamil Majdy
9e411efd6e fix(backend): Reverse pyro config refactor (#8410)
* fix(backend): Fix error pin output not being propagated into the next nodes

* fix(backend): Reverse pyro config refactor

* Revert "fix(backend): Fix error pin output not being propagated into the next nodes"

This reverts commit 2ff50a94ec.

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-23 15:54:29 +00:00
Zamil Majdy
27c9ec5bcd fix(platform): Enable auth on local-backend mode by default (#8405) 2024-10-23 03:39:15 +00:00
Zamil Majdy
17e79ad88d fix(backend): Fix DatabaseManager usage by calling it on-demand (#8404) 2024-10-23 10:09:23 +07:00
Zamil Majdy
7f318685af Revert "fix(backend): Fix DatabaseManager usage by calling it on-demand"
This reverts commit e5f5005ab8.
2024-10-23 07:41:59 +07:00
Zamil Majdy
e5f5005ab8 fix(backend): Fix DatabaseManager usage by calling it on-demand 2024-10-23 07:39:15 +07:00
Zamil Majdy
f4dac22335 feat(backend): Track LLM token usage + LLM blocks cleanup (#8367) 2024-10-22 18:13:27 -05:00
Aarushi
81febb6589 feat(platform): Added latest claude version (#8397)
latest sonnet model
2024-10-22 16:51:23 +01:00
Nicholas Tindle
1622a4aaa8 refactor(backend): Move credentials storage to prisma user (#8283)
* feat(frontend,backend): testing

* feat: testing

* feat(backend): it works for reading email

* feat(backend): more docs on google

* fix(frontend,backend): formatting

* feat(backend): more logigin (i know this should be debug)

* feat(backend): make real the default scopes

* feat(backend): tests and linting

* fix: code review prep

* feat: sheets block

* feat: liniting

* Update route.ts

* Update autogpt_platform/backend/backend/integrations/oauth/google.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* Update autogpt_platform/backend/backend/server/routers/integrations.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* fix: revert opener change

* feat(frontend): add back opener

required to work on mac edge

* feat(frontend): drop typing list import from gmail

* fix: code review comments

* feat: code review changes

* feat: code review changes

* fix(backend): move from asserts to checks so they don't get optimized away in the future

* fix(backend): code review changes

* fix(backend): remove google specific check

* fix: add typing

* fix: only enable google blocks when oauth is configured for google

* fix: errors are real and valid outputs always when output

* fix(backend): add provider detail for debuging scope declines

* Update autogpt_platform/frontend/src/components/integrations/credentials-input.tsx

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* fix(frontend): enhance with comment, typeof error isn't known so this is best way to ensure the stringifyication will work

* feat: code review change requests

* fix: linting

* fix: reduce error catching

* fix: doc messages in code

* fix: check the correct scopes object 😄

* fix: remove double (and not needed) try catch

* fix: lint

* fix: scopes

* feat: handle the default scopes better

* feat: better email objectification

* feat: process attachements

turns out an email doesn't need a body

* fix: lint

* Update google.py

* Update autogpt_platform/backend/backend/data/block.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* fix: quit trying and except failure

* Update autogpt_platform/backend/backend/server/routers/integrations.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* feat: don't allow expired states

* fix: clarify function name and purpose

* feat: code links updates

* feat: additional docs on adding a block

* fix: type hint missing which means the block won't work

* fix: linting

* fix: docs formatting

* Update issues.py

* fix: improve the naming

* fix: formatting

* Update new_blocks.md

* Update new_blocks.md

* feat: better docs on what the args mean

* feat: more details on yield

* Update new_blocks.md

* fix: remove ignore from docs build

* feat: initial migration

* feat: migration tested with supabase-> prisma data location

* add custom migrations and script

* update migration command

* formatting and linting

* updated migration script

* add direct db url

* add find files

* rename

* use binary instead of source

* temp adding supabase

* remove unused functions

* adding missed merge

* fix: commit hash for lock

* ci: fix lint

* fix: minor bugs that prevented connecting and migrating to dbs and auth

* fix: linting

* fix: missed await

* fix(backend): phase one pr updates

* fix: handle error with returning user object from database_manager

* fix: linting

* Address comments

* Make the migration safe

* Update migration doc

* Move misplaced model functions

* Grammar

* Revert lock

* Remove irrelevant changes

* Remove irrelevant changes

* Avoid adding trigger on public schema

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
Co-authored-by: Aarushi <aarushik93@gmail.com>
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-22 13:33:54 +01:00
Aarushi
6f3828fc99 fix(dockercompose): Fix db manager connection (#8377)
* add db host

* remove unused variable
2024-10-18 12:49:56 +00:00
Zamil Majdy
26b1bca033 refactor(backend): Make block fields consistently use SchemaField (#8360) 2024-10-18 10:22:05 +07:00
Zamil Majdy
2c0286e411 feat(backend): Add credit for Jina/Search & LLM blocks (#8361) 2024-10-17 20:51:10 +07:00
Aarushi
ac33c1eb03 fix(platform): Include health router (#8362)
include health router
2024-10-17 11:09:33 +01:00
Aarushi
2d0e51fe28 security(platform/backend): Add health endpoint (#8341)
* add health endpoint

* fix linting

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-16 14:57:23 +00:00
Zamil Majdy
6f07d24e93 feat(backend): Introduce executors shared DB connection (#8340) 2024-10-16 21:15:23 +07:00
Krzysztof Czerwinski
bd5d2b1e86 feat(platform): List and revoke credentials in user profile (#8207)
Display existing credentials (OAuth and API keys) for all current providers: Google, Github, Notion and allow user to remove them. For providers that support it, we also revoke the tokens through the API: of the providers we currently have, Google and GitHub support it; Notion doesn't.

- Add credentials list and `Delete` button in `/profile`
- Add `revoke_tokens` abstract method to `BaseOAuthHandler` and implement it in each provider
- Revoke OAuth tokens for providers on `DELETE` `/{provider}/credentials/{cred_id}`, and return whether tokens could be revoked
   - Update `autogpt-server-api/baseClient.ts:deleteCredentials` with `CredentialsDeleteResponse` return type

Bonus:
- Update `autogpt-server-api/baseClient.ts:_request` to properly handle empty server responses
2024-10-14 17:50:55 +02:00
Reinier van der Leer
992989ee71 feat(backend): Ensure validity of OAuth credentials during graph execution (#8191)
- feat(backend/executor): Change credential injection mechanism to acquire credentials from `AgentServer` just before execution
  - Also locks the credentials for the duration of the execution

- feat(backend/server): Add thread-safe `IntegrationCredentialsManager` to handle and synchronize credentials-related operations

- feat(libs): Add mutexes to `SupabaseIntegrationCredentialsStore` to ensure thread-safety

Also:
- feat(backend): Added Pydantic model (de)serialization support to `@expose` decorator

Refactorings:
- refactor(backend, libs): Move `KeyedMutex` to `autogpt_libs.utils.synchronize`
- refactor(backend/server): Make `backend.server.integrations` module with `router`, `creds_manager`, and `utils` in it
2024-10-10 16:45:43 +00:00
Zamil Majdy
9ad5e1f808 fix(platform): Remove blind try-except for yielding error on block (#8287) 2024-10-10 23:25:29 +07:00