Commit Graph

89 Commits

Author SHA1 Message Date
Aarushi
47759f6951 refactor(backend): Remove config.default.json (#8581)
remove config default josn

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2024-11-07 21:20:15 -06:00
jackfromeast
bcaf3241da fix (backend): Patching the SSRF vulnerability in Github/Web Search/Request related blocks (#8531) 2024-11-08 00:29:18 +00:00
Aarushi
c25d03e945 fix(searchtheweb): Fix the Jina Search Block (#8583)
* update jina search web block

* update to false
2024-11-07 17:26:17 -06:00
Zamil Majdy
91edf08540 feat(backend): Improve pyro reliability by adding connection timeout, retry, cleanup, and dynamic connection thread size (#8574) 2024-11-07 11:34:32 +07:00
Zamil Majdy
86c544177e refactor(backend): Introduced Graph Input & Output Schema, Merge GraphMeta & Graph, Remove subgraph functionality (#8526) 2024-11-07 09:30:51 +07:00
Zamil Majdy
af9ea5bc31 fix(frontend): Broken UI caused by lack of error propagation on Run (#8575) 2024-11-07 08:49:32 +07:00
dependabot[bot]
4259ad686e build(deps): bump the production-dependencies group across 1 directory with 11 updates (#8567)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-05 17:32:15 -06:00
dependabot[bot]
f17c20ed91 build(deps): manual fix!! bump replicate from 0.34.1 to 1.0.3 in /autogpt_platform/backend (#8476)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-05 14:34:37 -06:00
SwiftyOS
c19703150a fix(gitignore): Allow file extensions after .ign. or .ignore. 2024-11-05 16:09:19 +01:00
SwiftyOS
3e7d0e7f1b feat(backend): update .gitignore to include ignore / ign file extension 2024-11-05 16:07:13 +01:00
Zamil Majdy
45fe26b58d feat(platform): Toast Graph Creation Error (#8536)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-05 03:23:41 +00:00
dependabot[bot]
11a4f9fdaa build(deps-dev): bump the development-dependencies group in /autogpt_platform/backend with 2 updates (#8543)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 19:14:05 -06:00
Swifty
594aa996d7 refactor(platform/rest): Update REST API to use standard FastAPI structure (#8519) 2024-11-04 18:12:21 +07:00
Swifty
4d4cc60071 chore(platform): Tidy up repo structure (#8521)
* tidy up

* nextjs env specific .env files
2024-11-01 15:33:26 +00:00
Swifty
edaa938f99 feat(pre-commit): Detect secrets (#8507)
Add detect secrets pre commit
2024-10-31 08:49:50 +00:00
Nicholas Tindle
74d8aa6d90 Update llm.py (#8513)
* Update llm.py

* Update llm.py

* Update llm.py

* Update llm.py

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-30 22:40:38 -07:00
Nicholas Tindle
3aebed61d2 feat(backend): credential swap to api keys in cred store (#8403)
* feat(backend): credential swap

* ci: formatting

* fix: importing is hard okay

* fix: spelln' is hard

* feat: better credential provider handling

* docs: update the imports locations

* fix: test credentials + formatting

* feat: drop continuous read mode

* fix: lint

* feat: fallback credentials

* feat: charge for credential useage and have a bad backup mechnism

* fix: don't save default credentials + add d_id

* fix: formatting

* feat: basic encryption/decryption

* ref: move files around

* ref: sign all blocks out of their credentials

* ref: update target to match a new, and encrypted future

* wip: llm provider merger

* don't delete `credentials` input on nodes

* fix llm block ci issues

* updated get AICredentials

* fix fix

* insert migration to move integration credentials from `auth.user` metadata to `platform.User.metadata`

* fixed migration

* add migration for existing user integration credentials

* disabled reddit and email block

* fix credential handling in LLM blocks

* add other secret fields to credential scrubber migration

* add other secret fields to credential scrubber migration (vol. 2)

* fix: pr fixes

* fix: mock funciton

* add encrypted values

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: SwiftyOS <craigswift13@gmail.com>
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
Co-authored-by: Aarushi <aarushik93@gmail.com>
2024-10-31 02:36:38 +00:00
dependabot[bot]
67ff738fda build(deps): bump croniter from 2.0.7 to 5.0.1 in /autogpt_platform/backend (#8498)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-29 18:54:49 -05:00
Swifty
c7cd6dc026 feat(platform): Add target database model to reflect domain (#8375) 2024-10-29 08:22:35 +01:00
dependabot[bot]
fa04f802b2 build(deps-dev): bump the development-dependencies group in /autogpt_platform/backend with 4 updates (#8471)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-29 00:34:02 +00:00
dependabot[bot]
6fef4fd66e build(deps): bump tenacity from 8.5.0 to 9.0.0 in /autogpt_platform/backend (#8472)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 23:52:08 +00:00
dependabot[bot]
c028dd647e build(deps): bump psutil from 5.9.8 to 6.1.0 in /autogpt_platform/backend (#8473)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 18:45:56 -05:00
dependabot[bot]
308a9bc40e build(deps): bump sentry-sdk from 1.45.0 to 2.17.0 in /autogpt_platform/backend (#8475)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 17:04:48 -05:00
dependabot[bot]
d8317c8257 build(deps): bump websockets from 12.0 to 13.1 in /autogpt_platform/backend (#8477)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-28 20:44:24 +00:00
Zamil Majdy
8938209d0d fix(backend): Avoid long synchronous call to block FastAPI event-loop (#8429) 2024-10-27 23:54:38 +07:00
Zamil Majdy
1e620fdb13 feat(backend): RedisEventQueue into Pub/Sub (#8387) 2024-10-27 23:49:33 +07:00
Aarushi
e2df6019fb feat(blocks): Add pinecone and jina blocks (#8401)
* add pinecone and jina blocks

* udpate based on comments

* backend updates

* frontend updates

* type hint

* more type hints

* another type hint

* update run signature

* shared jina provider

* fix linting

* lockfile

* remove noqa

* remove noqa

* remove vector db folder

* line

* update pincone credentials provider

* fix imports

* formating

* update frontend

* Test (#8425)

* h

* Discard changes to autogpt_platform/backend/poetry.lock

* fix: broken dep

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-24 21:01:53 +01:00
Zamil Majdy
1321faf9db fix(backend): Fix error pin output not being propagated into the next nodes (#8408) 2024-10-24 17:13:58 +00:00
Zamil Majdy
9e411efd6e fix(backend): Reverse pyro config refactor (#8410)
* fix(backend): Fix error pin output not being propagated into the next nodes

* fix(backend): Reverse pyro config refactor

* Revert "fix(backend): Fix error pin output not being propagated into the next nodes"

This reverts commit 2ff50a94ec.

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-23 15:54:29 +00:00
Zamil Majdy
27c9ec5bcd fix(platform): Enable auth on local-backend mode by default (#8405) 2024-10-23 03:39:15 +00:00
Zamil Majdy
17e79ad88d fix(backend): Fix DatabaseManager usage by calling it on-demand (#8404) 2024-10-23 10:09:23 +07:00
Zamil Majdy
7f318685af Revert "fix(backend): Fix DatabaseManager usage by calling it on-demand"
This reverts commit e5f5005ab8.
2024-10-23 07:41:59 +07:00
Zamil Majdy
e5f5005ab8 fix(backend): Fix DatabaseManager usage by calling it on-demand 2024-10-23 07:39:15 +07:00
Zamil Majdy
f4dac22335 feat(backend): Track LLM token usage + LLM blocks cleanup (#8367) 2024-10-22 18:13:27 -05:00
Aarushi
81febb6589 feat(platform): Added latest claude version (#8397)
latest sonnet model
2024-10-22 16:51:23 +01:00
Nicholas Tindle
1622a4aaa8 refactor(backend): Move credentials storage to prisma user (#8283)
* feat(frontend,backend): testing

* feat: testing

* feat(backend): it works for reading email

* feat(backend): more docs on google

* fix(frontend,backend): formatting

* feat(backend): more logigin (i know this should be debug)

* feat(backend): make real the default scopes

* feat(backend): tests and linting

* fix: code review prep

* feat: sheets block

* feat: liniting

* Update route.ts

* Update autogpt_platform/backend/backend/integrations/oauth/google.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* Update autogpt_platform/backend/backend/server/routers/integrations.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* fix: revert opener change

* feat(frontend): add back opener

required to work on mac edge

* feat(frontend): drop typing list import from gmail

* fix: code review comments

* feat: code review changes

* feat: code review changes

* fix(backend): move from asserts to checks so they don't get optimized away in the future

* fix(backend): code review changes

* fix(backend): remove google specific check

* fix: add typing

* fix: only enable google blocks when oauth is configured for google

* fix: errors are real and valid outputs always when output

* fix(backend): add provider detail for debuging scope declines

* Update autogpt_platform/frontend/src/components/integrations/credentials-input.tsx

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* fix(frontend): enhance with comment, typeof error isn't known so this is best way to ensure the stringifyication will work

* feat: code review change requests

* fix: linting

* fix: reduce error catching

* fix: doc messages in code

* fix: check the correct scopes object 😄

* fix: remove double (and not needed) try catch

* fix: lint

* fix: scopes

* feat: handle the default scopes better

* feat: better email objectification

* feat: process attachements

turns out an email doesn't need a body

* fix: lint

* Update google.py

* Update autogpt_platform/backend/backend/data/block.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* fix: quit trying and except failure

* Update autogpt_platform/backend/backend/server/routers/integrations.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* feat: don't allow expired states

* fix: clarify function name and purpose

* feat: code links updates

* feat: additional docs on adding a block

* fix: type hint missing which means the block won't work

* fix: linting

* fix: docs formatting

* Update issues.py

* fix: improve the naming

* fix: formatting

* Update new_blocks.md

* Update new_blocks.md

* feat: better docs on what the args mean

* feat: more details on yield

* Update new_blocks.md

* fix: remove ignore from docs build

* feat: initial migration

* feat: migration tested with supabase-> prisma data location

* add custom migrations and script

* update migration command

* formatting and linting

* updated migration script

* add direct db url

* add find files

* rename

* use binary instead of source

* temp adding supabase

* remove unused functions

* adding missed merge

* fix: commit hash for lock

* ci: fix lint

* fix: minor bugs that prevented connecting and migrating to dbs and auth

* fix: linting

* fix: missed await

* fix(backend): phase one pr updates

* fix: handle error with returning user object from database_manager

* fix: linting

* Address comments

* Make the migration safe

* Update migration doc

* Move misplaced model functions

* Grammar

* Revert lock

* Remove irrelevant changes

* Remove irrelevant changes

* Avoid adding trigger on public schema

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
Co-authored-by: Aarushi <aarushik93@gmail.com>
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-22 13:33:54 +01:00
Aarushi
6f3828fc99 fix(dockercompose): Fix db manager connection (#8377)
* add db host

* remove unused variable
2024-10-18 12:49:56 +00:00
Zamil Majdy
26b1bca033 refactor(backend): Make block fields consistently use SchemaField (#8360) 2024-10-18 10:22:05 +07:00
Zamil Majdy
2c0286e411 feat(backend): Add credit for Jina/Search & LLM blocks (#8361) 2024-10-17 20:51:10 +07:00
Aarushi
ac33c1eb03 fix(platform): Include health router (#8362)
include health router
2024-10-17 11:09:33 +01:00
Aarushi
2d0e51fe28 security(platform/backend): Add health endpoint (#8341)
* add health endpoint

* fix linting

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-16 14:57:23 +00:00
Zamil Majdy
6f07d24e93 feat(backend): Introduce executors shared DB connection (#8340) 2024-10-16 21:15:23 +07:00
Krzysztof Czerwinski
bd5d2b1e86 feat(platform): List and revoke credentials in user profile (#8207)
Display existing credentials (OAuth and API keys) for all current providers: Google, Github, Notion and allow user to remove them. For providers that support it, we also revoke the tokens through the API: of the providers we currently have, Google and GitHub support it; Notion doesn't.

- Add credentials list and `Delete` button in `/profile`
- Add `revoke_tokens` abstract method to `BaseOAuthHandler` and implement it in each provider
- Revoke OAuth tokens for providers on `DELETE` `/{provider}/credentials/{cred_id}`, and return whether tokens could be revoked
   - Update `autogpt-server-api/baseClient.ts:deleteCredentials` with `CredentialsDeleteResponse` return type

Bonus:
- Update `autogpt-server-api/baseClient.ts:_request` to properly handle empty server responses
2024-10-14 17:50:55 +02:00
Reinier van der Leer
992989ee71 feat(backend): Ensure validity of OAuth credentials during graph execution (#8191)
- feat(backend/executor): Change credential injection mechanism to acquire credentials from `AgentServer` just before execution
  - Also locks the credentials for the duration of the execution

- feat(backend/server): Add thread-safe `IntegrationCredentialsManager` to handle and synchronize credentials-related operations

- feat(libs): Add mutexes to `SupabaseIntegrationCredentialsStore` to ensure thread-safety

Also:
- feat(backend): Added Pydantic model (de)serialization support to `@expose` decorator

Refactorings:
- refactor(backend, libs): Move `KeyedMutex` to `autogpt_libs.utils.synchronize`
- refactor(backend/server): Make `backend.server.integrations` module with `router`, `creds_manager`, and `utils` in it
2024-10-10 16:45:43 +00:00
Zamil Majdy
9ad5e1f808 fix(platform): Remove blind try-except for yielding error on block (#8287) 2024-10-10 23:25:29 +07:00
vishesh10
7b92bae942 Fix block execution status in case of error (#8267) 2024-10-10 02:59:26 +00:00
Aarushi
c03e2fb949 tweak(platform): Remove importing templates from local dir (#8276)
* always filter on user id

* add user id to doc string

* fix linting

* fix imports function

* remove import templates from local directory
2024-10-09 23:13:46 +00:00
Nicholas Tindle
e654aa1e7a feat(backend): add the capibility to disable llm models in the cloud env (#8285)
* feat(backend): logic to disable enums based on python logic

* feat(backend): add behave as setting and clarify its purpose and APP_ENV

APP_ENV is used for not cloud vs local but the application environment such as local/dev/prod so we need BehaveAs as well

* fix(backend): various uses of AppEnvironment without the Enum or incorrectly

AppEnv in the logging library will never be cloud due to the restrictions applied when loading settings in by pydantic settings. This commit fixes this error, however the code path for logging may now be incorrect

* feat(backend): use a metaclass to disable ollama in the cloud environment

* fix: formatting

* fix(backend): typing improvements

* fix(backend): more linting 😭
2024-10-09 10:12:48 +01:00
Aarushi
e37744b9f2 fix(platform): Update deletion of secret values to not do it in place (#8284)
update deletion of secret values to not do it in place
2024-10-08 22:43:36 +01:00
Aarushi
bc1df92c29 fix(platform): Fix marketplace leaking secrets (#8281)
* add hide secrets param

* Update autogpt_platform/backend/backend/data/graph.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* Update autogpt_platform/frontend/src/lib/autogpt-server-api/baseClient.ts

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* rename hide secrets

* use builtin dict

* delete key

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-10-08 20:20:18 +01:00