Commit Graph

108 Commits

Author SHA1 Message Date
Zamil Majdy
6a1cea4c4e fix(backend): Add execution persistence for execution scheduler service (#8649)
* fix(backend): Add execution persistence for execution scheduler service

* scheduler REST API cleanup

* Fix to binary

* Adapt UI with new API

* Remove schedule.py

* Remove unused class

* Fix linting
2024-11-15 11:17:37 +01:00
Reinier van der Leer
f9633ffb71 Revert "fix(platform): Remove migrate and encrypt function" (#8654)
Reverts c707ee9 (#8646)

The problem analysis that led to #8646 contained some errors, so the migration removed in the PR doesn't seem to have been the cause of the problem we were hunting. Also, this migration is an essential part of the security improvement that we made 2 weeks ago.
2024-11-14 23:34:30 +00:00
Abhimanyu Yadav
dd0081ab35 feat(platform) : scheduling agent runner (#8634)
* add: ui for scheduling agent

* adding requests and type for schedule endpoints

* feat : monitor schdules on monitor page

* add: Complete monitor page

* fix filter on monitor page

* fix linting

* PR nits

* Added Docker Compose env var

---------

Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
Co-authored-by: Swifty <craigswift13@gmail.com>
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2024-11-14 16:18:56 +01:00
Abhimanyu Yadav
bbbdb5665b feat(platform): Add api generator functions and endpoints (#8597)
* add: api generator functions and endpoints

* Rebase onto dev, refactor API manager location, remove suspended key revoke, and update API code for Prisma compatibility

* add: key_manager

* reversing changes og poetry.lock

* add: changing hash mexhansim in API Manager

* add: changing hash mexhansim in API Manager

* fixing some simple bugs

* fix linting and adding better error handling

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-11-14 14:33:27 +00:00
Toran Bruce Richards
e628a25533 feat(blocks): Add AIImageGeneratorBlock (#8525)
* feat(block): Add AIImageGeneratorBlock

This commit adds the AIImageGeneratorBlock class to the backend. The AIImageGeneratorBlock is responsible for generating images using various AI models through a unified interface.

* Remove unsupported inputs and add more styles

* Update autogpt_platform/backend/backend/blocks/ai_image_generator_block.py

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>

* run format

* Add test mock

* mock client run

* Refactor AIImageGeneratorBlock to use a separate function for running the client

* Update Credential description

* Rename ModelProvider to ImageGenModel

* Add missing block run function

* fix mock

* .

* Refactor AIImageGeneratorBlock to move run_client function inside class

* Fix broken reference to run client and tidy code.

* Refactor AIImageGeneratorBlock to improve code structure and error handling

* Move client into run client instantiation function.

* Refactor AIImageGeneratorBlock to handle output as FileOutput and improve error handling

* run format

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-11-14 12:28:24 +00:00
vishesh10
639242ac68 Add provision for other languages in Youtube Video Block (#8630)
Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
2024-11-14 09:20:46 +00:00
Aarushi
c707ee9cb6 fix(platform): Remove migrate and encrypt function (#8646)
remove migrate and encrypt function
2024-11-13 23:09:26 +00:00
Toran Bruce Richards
aafc101224 tweak(backend): Update all block costs (#8639)
* Add support for default credentials to unreal block

* Refactor block cost configuration and add new blocks

This commit refactors the block cost configuration file and adds support for new blocks. The changes include:
- Importing the `AIMusicGeneratorBlock`, `JinaEmbeddingBlock`, and `UnrealTextToSpeechBlock` classes
- Updating the `BLOCK_COSTS` dictionary to include costs for the new blocks

These changes enable the usage of the newly introduced blocks.
2024-11-13 16:05:32 +00:00
Reinier van der Leer
aaa0b79f08 fix(frontend): Unbreak credentials input on single-provider blocks (#8636)
- Resolves #8635

- fix(frontend): Fix type mismatch of `CredentialsField` schema between frontend and backend
   - Fix usages of `credentialsSchema.credentials_provider`

- refactor(backend): Create `CredentialsFieldSchemaExtra` model in backend so it can be mirrored directly in frontend
   - Add check to enforce multi-provider `CredentialsField` always has `discriminator`

- dx: Add type checking shortcut `yarn type-check` / `npm run type-check` for frontend
2024-11-13 13:48:15 +00:00
Krzysztof Czerwinski
e907ffda6e feat(platform): Simplify Credentials UX (#8524)
- Change `provider` of default credentials to actual provider names (e.g. `anthropic`), remove `llm` provider
- Add `discriminator` and `discriminator_mapping` to `CredentialsField` that allows to filter credentials input to only allow  providers for matching models in `useCredentials` hook (thanks @ntindle for the idea!); e.g. user chooses `GPT4_TURBO` so then only OpenAI credentials are allowed
- Choose credentials automatically and hide credentials input on the node completely if there's only one possible option
- Move `getValue` and `parseKeys` to utils
- Add `ANTHROPIC`, `GROQ` and `OLLAMA` to providers in frontend `types.ts`
- Add `hidden` field to credentials that is used for default system keys to hide them in user profile
- Now `provider` field in `CredentialsField` can accept multiple providers as a list

-----------------
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-11-12 16:55:48 +01:00
Zamil Majdy
ef7e50403e refactor(backend): Centralize Block Cost into a Single File (#8623) 2024-11-12 14:09:59 +00:00
Zamil Majdy
1e872406ca feat(platform): Introduced Agent Execution Block (#8533) 2024-11-12 13:03:15 +07:00
dependabot[bot]
4a163e5b54 build(deps-dev): bump the development-dependencies group in /autogpt_platform/backend with 3 updates (#8611)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-11 17:31:13 -06:00
dependabot[bot]
603fec3467 build(deps): bump the production-dependencies group in /autogpt_platform/backend with 2 updates (#8610)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-11 22:21:44 +00:00
Nicholas Tindle
3c0dea0017 [Snyk] Security upgrade python from 3.11-slim-buster to 3.11.10-slim-bookworm (#8619)
fix: autogpt_platform/backend/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-DEBIAN10-GNUTLS28-6159414
- https://snyk.io/vuln/SNYK-DEBIAN10-NCURSES-1655739
- https://snyk.io/vuln/SNYK-DEBIAN10-NCURSES-1655739
- https://snyk.io/vuln/SNYK-DEBIAN10-NCURSES-5421196
- https://snyk.io/vuln/SNYK-DEBIAN10-NCURSES-5421196

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-11-11 20:33:39 +00:00
Toran Bruce Richards
1f0cbc6500 feat(block): Add AI Music Generator Block with Meta Music Gen (#8532)
* feat(platform): Add AIMusicGeneratorBlock for music generation

* refactor(platform): Refactor AIMusicGeneratorBlock for improved error handling and logging

* refactor(ui): Refactor ContentRenderer to support audio rendering

* format

* Frontend format and lint
2024-11-11 09:36:39 -08:00
Aarushi
e6e47373ac feat(blocks/jina): Add default credentials for Jina (#8603)
add jina defaults
2024-11-10 17:18:53 +00:00
Aarushi
359ae8307a feat(backend): Add API key DB table (#8593)
* add api key db tables

* remove uniqueness constraint on prefix

* add postfix
2024-11-08 17:48:37 +00:00
Aarushi
f719c7e70e feat(blocks): Pinecone blocks (#8535)
* update pinecone

* update blocks

* fix linting

* update test

* update requests

* mock funciton
2024-11-08 17:13:55 +00:00
Aarushi
47759f6951 refactor(backend): Remove config.default.json (#8581)
remove config default josn

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2024-11-07 21:20:15 -06:00
jackfromeast
bcaf3241da fix (backend): Patching the SSRF vulnerability in Github/Web Search/Request related blocks (#8531) 2024-11-08 00:29:18 +00:00
Aarushi
c25d03e945 fix(searchtheweb): Fix the Jina Search Block (#8583)
* update jina search web block

* update to false
2024-11-07 17:26:17 -06:00
Zamil Majdy
91edf08540 feat(backend): Improve pyro reliability by adding connection timeout, retry, cleanup, and dynamic connection thread size (#8574) 2024-11-07 11:34:32 +07:00
Zamil Majdy
86c544177e refactor(backend): Introduced Graph Input & Output Schema, Merge GraphMeta & Graph, Remove subgraph functionality (#8526) 2024-11-07 09:30:51 +07:00
Zamil Majdy
af9ea5bc31 fix(frontend): Broken UI caused by lack of error propagation on Run (#8575) 2024-11-07 08:49:32 +07:00
dependabot[bot]
4259ad686e build(deps): bump the production-dependencies group across 1 directory with 11 updates (#8567)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-05 17:32:15 -06:00
dependabot[bot]
f17c20ed91 build(deps): manual fix!! bump replicate from 0.34.1 to 1.0.3 in /autogpt_platform/backend (#8476)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-05 14:34:37 -06:00
SwiftyOS
c19703150a fix(gitignore): Allow file extensions after .ign. or .ignore. 2024-11-05 16:09:19 +01:00
SwiftyOS
3e7d0e7f1b feat(backend): update .gitignore to include ignore / ign file extension 2024-11-05 16:07:13 +01:00
Zamil Majdy
45fe26b58d feat(platform): Toast Graph Creation Error (#8536)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-11-05 03:23:41 +00:00
dependabot[bot]
11a4f9fdaa build(deps-dev): bump the development-dependencies group in /autogpt_platform/backend with 2 updates (#8543)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 19:14:05 -06:00
Swifty
594aa996d7 refactor(platform/rest): Update REST API to use standard FastAPI structure (#8519) 2024-11-04 18:12:21 +07:00
Swifty
4d4cc60071 chore(platform): Tidy up repo structure (#8521)
* tidy up

* nextjs env specific .env files
2024-11-01 15:33:26 +00:00
Swifty
edaa938f99 feat(pre-commit): Detect secrets (#8507)
Add detect secrets pre commit
2024-10-31 08:49:50 +00:00
Nicholas Tindle
74d8aa6d90 Update llm.py (#8513)
* Update llm.py

* Update llm.py

* Update llm.py

* Update llm.py

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-30 22:40:38 -07:00
Nicholas Tindle
3aebed61d2 feat(backend): credential swap to api keys in cred store (#8403)
* feat(backend): credential swap

* ci: formatting

* fix: importing is hard okay

* fix: spelln' is hard

* feat: better credential provider handling

* docs: update the imports locations

* fix: test credentials + formatting

* feat: drop continuous read mode

* fix: lint

* feat: fallback credentials

* feat: charge for credential useage and have a bad backup mechnism

* fix: don't save default credentials + add d_id

* fix: formatting

* feat: basic encryption/decryption

* ref: move files around

* ref: sign all blocks out of their credentials

* ref: update target to match a new, and encrypted future

* wip: llm provider merger

* don't delete `credentials` input on nodes

* fix llm block ci issues

* updated get AICredentials

* fix fix

* insert migration to move integration credentials from `auth.user` metadata to `platform.User.metadata`

* fixed migration

* add migration for existing user integration credentials

* disabled reddit and email block

* fix credential handling in LLM blocks

* add other secret fields to credential scrubber migration

* add other secret fields to credential scrubber migration (vol. 2)

* fix: pr fixes

* fix: mock funciton

* add encrypted values

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: SwiftyOS <craigswift13@gmail.com>
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
Co-authored-by: Aarushi <aarushik93@gmail.com>
2024-10-31 02:36:38 +00:00
dependabot[bot]
67ff738fda build(deps): bump croniter from 2.0.7 to 5.0.1 in /autogpt_platform/backend (#8498)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-29 18:54:49 -05:00
Swifty
c7cd6dc026 feat(platform): Add target database model to reflect domain (#8375) 2024-10-29 08:22:35 +01:00
dependabot[bot]
fa04f802b2 build(deps-dev): bump the development-dependencies group in /autogpt_platform/backend with 4 updates (#8471)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-29 00:34:02 +00:00
dependabot[bot]
6fef4fd66e build(deps): bump tenacity from 8.5.0 to 9.0.0 in /autogpt_platform/backend (#8472)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 23:52:08 +00:00
dependabot[bot]
c028dd647e build(deps): bump psutil from 5.9.8 to 6.1.0 in /autogpt_platform/backend (#8473)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 18:45:56 -05:00
dependabot[bot]
308a9bc40e build(deps): bump sentry-sdk from 1.45.0 to 2.17.0 in /autogpt_platform/backend (#8475)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 17:04:48 -05:00
dependabot[bot]
d8317c8257 build(deps): bump websockets from 12.0 to 13.1 in /autogpt_platform/backend (#8477)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-28 20:44:24 +00:00
Zamil Majdy
8938209d0d fix(backend): Avoid long synchronous call to block FastAPI event-loop (#8429) 2024-10-27 23:54:38 +07:00
Zamil Majdy
1e620fdb13 feat(backend): RedisEventQueue into Pub/Sub (#8387) 2024-10-27 23:49:33 +07:00
Aarushi
e2df6019fb feat(blocks): Add pinecone and jina blocks (#8401)
* add pinecone and jina blocks

* udpate based on comments

* backend updates

* frontend updates

* type hint

* more type hints

* another type hint

* update run signature

* shared jina provider

* fix linting

* lockfile

* remove noqa

* remove noqa

* remove vector db folder

* line

* update pincone credentials provider

* fix imports

* formating

* update frontend

* Test (#8425)

* h

* Discard changes to autogpt_platform/backend/poetry.lock

* fix: broken dep

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-24 21:01:53 +01:00
Zamil Majdy
1321faf9db fix(backend): Fix error pin output not being propagated into the next nodes (#8408) 2024-10-24 17:13:58 +00:00
Zamil Majdy
9e411efd6e fix(backend): Reverse pyro config refactor (#8410)
* fix(backend): Fix error pin output not being propagated into the next nodes

* fix(backend): Reverse pyro config refactor

* Revert "fix(backend): Fix error pin output not being propagated into the next nodes"

This reverts commit 2ff50a94ec.

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-23 15:54:29 +00:00
Zamil Majdy
27c9ec5bcd fix(platform): Enable auth on local-backend mode by default (#8405) 2024-10-23 03:39:15 +00:00
Zamil Majdy
17e79ad88d fix(backend): Fix DatabaseManager usage by calling it on-demand (#8404) 2024-10-23 10:09:23 +07:00