Commit Graph

7791 Commits

Author SHA1 Message Date
Toran Bruce Richards
6bdb849150 feat(Block): Add blocks for Building Lists, Dictionaries & Strings (#7678)
* feat(Block): Add block for building lists

* feat(Block): Add block for building dictionaries

* feat(Block): Add TextCombinerBlock for combining multiple input texts

* Fix linting issues where default values are None

---------

Co-authored-by: Bently <tomnoon9@gmail.com>
2024-08-02 16:49:51 -05:00
Bently
8469fafc6f Feat(Builder): Discord bots blocks (#7670)
* Super early version of the discord bots blocks

* updated to add secrets for token + other fixes

* lint & format

* rename DiscordBot to DiscordReader

* add discord-py

* fix poetry lock

* rm duplicated file

* updated name to add Block to end

* update .env.template to add DISCORD_BOT_TOKEN

* updates to add description Field

* swap channel name and message content + add field description

---------

Co-authored-by: Bently <bently@bentlybro.com>
2024-08-02 22:21:58 +01:00
Reinier van der Leer
3c2c3e57a0 feat(builder): Rewrite node input UI (#7626)
- feat(builder): Rewrite & split up `NodeInputField`
  - Create `NodeObjectInputTree`
  - Create `NodeGenericInputField`
  - Create `NodeKeyValueInput`
  - Create `NodeArrayInput`
  - Create `NodeStringInput`
  - Create `NodeNumberInput`
  - Create `NodeBooleanInput`
  - Create `NodeFallbackInput`
  - Create `ClickableInput` from `renderClickableInput(..)`
  - Amend usage in `CustomNode`
  - Remove deprecated/unused styling from `flow.css` and `customnode.css`
  - Fix alignment between `NodeHandle` and `NodeInputField`
  - Split up `BlockIOSchema` & rename to `BlockIOSubSchema`
    - Create `BlockIOObjectSubSchema`
    - Create `BlockIOKVSubSchema`
    - Create `BlockIOArraySubSchema`
    - Create `BlockIOStringSubSchema`
    - Create `BlockIONumberSubSchema`
    - Create `BlockIOBooleanSubSchema`
    - Create `BlockIONullSubSchema`
  - Install `Select` component from shad/cn

- refactor(builder): Move `NodeInputField.tsx` to `node-input.tsx`
2024-08-02 19:28:46 +01:00
lelcruz
ec6bae0467 fix(documentation): Corrected Markdown formatting for server start command (#7649)
Co-authored-by: Leslie Cruz <lelcruz@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-08-02 12:33:12 -05:00
Bently
f5fe96260e Fix(Builder): beautifyString duplicated import (#7679) 2024-08-02 15:39:42 +01:00
SwiftyOS
49a18437ac fix(builder) fix merge conflict errors 2024-08-02 16:29:55 +02:00
Reinier van der Leer
3cee893314 fix(builder): Rewrite additionalProperties input field (#7608)
Co-authored-by: Swifty <craigswift13@gmail.com>
2024-08-02 16:11:38 +02:00
Reinier van der Leer
5d1035aeb0 fix(builder): Fix BlockSchema handling & type propagation (#7604)
- Merge `Flow.tsx:CustomNodeData` type definition into `CustomNode.tsx:CustomNodeData`
- Move `lib/types:BlockSchema` type definition into `lib/autogpt-server-api/types` in place of `ObjectSchema`
- Expand and rename `BlockSchema` -> `BlockIOSchema` + `BlockIORootSchema`
- Fix all `BlockIOSchema` related type narrowing checks
- Add warning messages to fallback cases in `NodeInputField` logic

Co-authored-by: Swifty <craigswift13@gmail.com>
2024-08-02 16:09:46 +02:00
Bently
2e2c6fed52 feat(Builder): Implement undo/redo functionality (#7559)
* feat(Builder): Implement undo/redo functionality

* updates to work with latest UI

* add CTRL + Z & CTRL + Y support

* fixed undo/redo for inputs in nodes

* fix for deleting node

* fixes to make the undo/redo work better

* small fix

* Updates based on feedback

* add margin right to space out buttons

* added CTRL + SHIFT + Z for redo
2024-08-02 16:06:40 +02:00
Krzysztof Czerwinski
ca9c52f76a feat(builder): Add Supabase Auth, session and sign in UI (#7655)
This PR adds Supabase Auth (cloud) integration, login and profile UI, configures password login and three OAuth providers: Google, GitHub and Discord.

For `Account` button to show up and ability to login two env vars need to be set in `.env.local`: `NEXT_PUBLIC_SUPABASE_URL` and `NEXT_PUBLIC_SUPABASE_ANON_KEY`. OAuth providers are by the Supabase and don't require env vars.

Email confirmation (for email/password signup) is disabled because there's limit of 3 emails per hour without custom SMTP server configuration. [Link](https://supabase.com/dashboard/project/adfjtextkuilwuhzdjpf/auth/templates) to configure custom SMTP server and email template.

### Added dependencies:
- "@supabase/ssr": "^0.4.0"
- "@supabase/supabase-js": "^2.45.0"
- "react-icons": "^5.2.1"

### Added pages/routes:
- `app/auth/auth-code-error/page.tsx`: displayed when login using OAuth provider fails
- `app/auth/callback/route.ts`: route accessed when logging in using OAuth provider; it passes session code to Supabase
- `app/auth/confirm/route.ts`: accessed when confirming email, users will be directed here from email they get after signing in.
- `app/error/page.tsx`: Generic error page without explanation (any errors should be visible in the console)
- `app/login/page.tsx` and `app/login/actions.ts`: Login page and related login/signup server actions
- `app/profile/page.tsx`: Profile page, displays email address of the user and button to logout

### Changes
- Update `layout.tsx`: add `Log In` button and make icons consistent. The log in button shows up if user is logged out, avatar is shown when logged in, and if supabase is unavailable nothing shows up.
- Login form is verified using `zod` on the frontend (recommended by shadcn) and in case login fails feedback is displayed. On successful login users are redirected to `/profile`
- Add `PasswordInput` component, [source](https://gist.github.com/mjbalcueva/b21f39a8787e558d4c536bf68e267398)
- Add `SupabaseProvider` with context for Supabase accessed via hook `useSupabase(): { supabase: SupabaseClient | null, isLoading: boolean }`
- Add `useUser` hook to get `{ user, session, isLoading, error }` on the client 
- Add `getServerUser`: async function to get `{ user: User | null, error: string | null }` on the server side
- Add `src/middleware.ts` and `client.ts`, `server.ts`, `middleware.ts` in `src/lib/supabase` which are utility functions and middleware to refresh auth token
2024-08-02 12:58:28 +01:00
Zamil Majdy
973822d973 feat(rnd): Add InputBlock & OutputBlock (#7654)
### Background

We need an explicit block for providing input & output for the graph.
This will later allow us to build a subgraph with pre-declared input & output schema.
This will also allow us to set input for the node in the middle of the graph, and enable a graph to have output values.

### Changes 🏗️

* Add InputBlock & OutputBlock
* Add graph structure validation  on the graph execution step that asserts the following property:
    - All mandatory input pin, has to be connected or have a default value, except the `InputBlock` node.
    - All links have to connect valid nodes, and the sink & source name using the valid block field.
2024-08-02 13:33:34 +07:00
Bently
e773329391 Feat(Builder): Clear Status and Output upon graph edit (#7656)
* Feat(Builder): Clear Status and Output upon graph edit

* close output dropdown on graph edit

* make deleting edge clear block outputs

* make it so deleting nodes clears block output
2024-08-01 17:21:01 +01:00
Zamil Majdy
c9d41e69bd feat(rnd): Add postgres schema check on linting step + auto gen migration and schema sync command (poetry run schema) (#7667)
### Background

Migration not being synced has caused issues and CI is not catching this yet.

### Changes 🏗️

* Added schema check in both sqlite & postgres prisma.schema on linting step.
* Introduced `poetry run schema` for syncing prisma.schema + generating migration in both files.
2024-08-01 20:11:07 +07:00
Nicholas Tindle
08905d71f9 docs(server): add docs link to server readme (#7638)
Co-authored-by: Swifty <craigswift13@gmail.com>
2024-08-01 14:32:08 +02:00
Aarushi
8becde370c docs(ollama) Add ollama to mkdocs (#7666)
add ollama to mkdocs
2024-08-01 13:14:40 +01:00
Aarushi
dccc33152b feat(docs) Add how to run ollama with AutoGPT (#7665)
add ollama howto
2024-08-01 12:12:01 +01:00
Aarushi
b23bd9c479 feat(server): Add JWT validation (#7642)
* add auth middleware

* update readnme

* Update rnd/autogpt_server/pyproject.toml

Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>

* address newline feedback

* update poetry lock

---------

Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>
2024-08-01 11:11:40 +01:00
Aarushi
ac45b7cae9 feat(rnd,infra): Add Helm linting in CI (#7633)
* Set up helm and tf for backend

* update helm charts and settings

* remove example files

* use latest tag

* delay and timeouts for probes

* env based pyro host

* default backend

* linting

* add helm linting in CI

* read from settings

* fix formatting

* update to use config
2024-08-01 09:22:04 +01:00
Aarushi
3d54a9103c feat(rnd, infra): Add deployment for backend (#7612)
* Set up helm and tf for backend

* update helm charts and settings

* remove example files

* use latest tag

* delay and timeouts for probes

* env based pyro host

* default backend

* linting

* read from settings

* fix formatting

* update to use config
2024-08-01 09:21:12 +01:00
Nicholas Tindle
ca7182403b fix(server): enable other methods of setting configs outside of the config.json file (#7645) 2024-07-31 12:33:36 -05:00
Zamil Majdy
53826ab360 fix(rnd): Fix bug caused by different AJV version (#7653) 2024-07-31 16:47:31 +07:00
Andy Hooker
eac5548023 feat(builder): UI / UX upgrade and simplification of tailwind styles (#7647)
* fix(builder): Implementing a basic shadCn theme until color palette is decided upon

* feat(builder): Separating NavBar into its own component and providing a standard UI/UX Approach

* feat(builder): Removing old implementation of logo, removing excessive css implementation, updating styles to better support standard desktop views.

* feature(builder): Addition of UI component Sheet from ShadCn for update
2024-07-31 10:43:13 +02:00
Zamil Majdy
122f544966 fix(rnd): Disable unused prisma connection on pyro API Server process (#7641)
### Background

Pyro for API Server is not using Prisma, but still holding a Prisma connection.
The fast-API thread is also holding a Prisma connection, making Prisma connected in two different loop within a single process.

### Changes 🏗️

Disable a Prisma connection on Pyro thread for Server API process.
Fix test flakiness issue due to concurrency issue.
2024-07-30 22:33:28 +07:00
Zamil Majdy
29ba4c2c73 feat(rnd): Add freezed/static input link feature for retaining input data to be re-used in multiple node executions (#7631)
### Background

Input from the input pin is consumed only once. While this is required in most of the use cases, there are some cases where the input can only be produced once, and that input needs to be re-used just like an input default value, that is passively providing input data, without triggering any execution. The scope of this change is providing that functionality in the link level, this property will be called **`static link`** in this system.

### Changes 🏗️

Provides a static link feature with the following behaviours:
* A link can be marked `static` to become a static link.
* Once a node produces an output it will persist the output data and propagate the output to the other nodes through the link, for a static link, instead of making the data queued in the input pin, it will override the default value.
* Any input executions still waiting for the input will be backfilled using this output produced by the static link.
* And any upcoming executions that will use the input will always reuse the output produced by the static link.

See the added test to see the expected usage.
2024-07-30 10:37:38 +07:00
Nicholas Tindle
76feead3b1 docs(server): written setup guide for server (#7595)
Co-authored-by: Swifty <craigswift13@gmail.com>
2024-07-29 22:24:03 +01:00
Bently
081df805df Fix(Builder): Copy and Paste when editing textbox causes issue (#7636)
Fix(Builder): Copy and Paste when editing textbox causes graph elements to be erroneously pasted
2024-07-29 21:55:19 +01:00
Nicholas Tindle
acc1d79146 feat(builder): add delete handles to the edges and nodes (#7614)
Co-authored-by: Andy Hooker <58448663+andrewhooker2@users.noreply.github.com>
Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
2024-07-29 21:30:56 +01:00
Nicholas Tindle
07811b2133 fix(builder): show colors for all statuses (#7625) 2024-07-29 20:04:48 +01:00
Amr Aly
01b6c2d4bf Update README.advanced.md (#7607)
* Update README.advanced.md

The copy .env command is wrong. it should be .env.example instead of env.example

* fix(server): extra space removal

---------

Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-07-29 12:00:58 +01:00
Krzysztof Czerwinski
905b1df218 feat(builder): Add input validation on the frontend (#7606)
- Add `ajv` dependency to check values against json schema
- Add `errors` and `setErrors` to `CustomNodeData`
- Add `validateNodes` run before executing agent
- Add `*` on labels for required fields
- Add `setNestedProperty` and `removeEmptyStringsAndNulls` utility function
2024-07-29 10:17:27 +01:00
Andy Hooker
edf84fb9f8 feat(builder): Apply standard theming for shadCn and centralize providers (#7622) 2024-07-26 17:59:44 -05:00
Swifty
b62c24dc77 Add Marketplace template (#7605)
* init

* Added sentry.io integration

* updated readme
2024-07-26 17:11:26 +02:00
Zamil Majdy
dfa855f533 fix(rnd): Fix broken server execution update due to split prisma connection to different loops (#7620) 2024-07-26 15:47:41 +01:00
Aarushi
da2111bafb fix(server): Add missing PG migrations (#7619)
missing migrations
2024-07-26 14:38:05 +01:00
Reinier van der Leer
b2dba39810 fix(builder/api-client): Fix websocket client types (#7603)
- Fix type signatures of `sendWebSocketMessage(..)`, `onWebSocketMessage(..)`, `runGraph(..)` in `autogpt-server-api/client`
  - Add `WebsocketMessageTypeMap`

- Fix type signature of `updateNodesWithExecutionData` in `FlowEditor`
2024-07-26 13:26:11 +02:00
Zamil Majdy
d2a5bb286f feat(rnd): Fix concurrency issue on Agent Server & Apply max-1-execution-per-node constraint in graph execution (#7551)
### Background

When multiple executors are executing the same node within the same graph execution, two node executions can read the same queue of input and read the same value—making the data that is supposed to be consumed once, consumed by two executions. The lack of lock & concurrency support for parallel execution within a single graph causes this issue.

Node concurrency also introduces poor UX in the current frontend implementation, when two nodes are executed in parallel, the current UI will not display its parallel execution update, but instead, it shows the updates that override each other. Until the execution observability is improved on the builder UI, this capability will be limited.

### Changes 🏗️

The scope of this change is to solve this issue by:
* Decouple Graph execution & Node execution, each has its own configured process pool.
* Make sure there is only 1 execution per node (we still allow parallel executions on different nodes) in a graph.
* Fixed concurrency issue by adding distributed lock API on agent_server.
* Few cleanups:
    - Add more logging with geid & neid prefix on graph/node executions
    - Moved execution status update to agent-server for a single source of status update (required by conn-manager/web-socket)
    - Configured node parallelism to 10 & graph parallelism to 10 by default, so in the very rare worst-case, there can be 100 node executions.
    - Re-use server resource for each integration test run
2024-07-26 17:08:03 +07:00
Toran Bruce Richards
36b9a0a930 feat(Blocks): Add conditional logic block (#7536)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-07-25 14:25:07 -05:00
Aarushi
f40db85b43 feat(rnd, infra) Add static ips module (#7588)
* add terraform

* gitignore update

* linting

* formatting and linting in ci

* store state in backend bucket

* Update .github/workflows/autogpt-infra-ci.yml

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>

* add static ips module

* formatting

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: Swifty <craigswift13@gmail.com>
2024-07-25 17:50:19 +01:00
Reinier van der Leer
0767b17779 dx: Set Python type checking mode to "basic" for all subprojects (#7599) 2024-07-25 12:25:56 +01:00
Nicholas Tindle
0f0c13bae8 fix(server): gitignore was missing new file name (#7596)
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-07-25 10:07:16 +01:00
A.Daee
3b0cd9518d feat(froge): Improvement in web components (#7068)
- Add `duckduckgo_backend` field to `WebSearchComponent` configuration
- Add `selenium_proxy` to `WebSeleniumComponent` configuration
- Update docs
2024-07-25 10:02:34 +01:00
Aarushi
22b6dbbf6a feat(rnd,infra) Set up terraform (#7565)
* add terraform

* gitignore update

* linting

* formatting and linting in ci

* store state in backend bucket
2024-07-25 09:45:36 +01:00
Reinier van der Leer
d9a1a1edc8 fix(builder): Correctly load node connections (#7592)
- Set `node.data.connections` based on `graph.links` in `loadGraph(..)`
- Use reactflow's built-in `useNodesState`, `useEdgesState` to replace some boilerplate functions that do exactly the same

Resolves #7392
2024-07-25 09:30:35 +01:00
vasnt
3c0d37d5d1 docs(forge):fix #7198 dead image links (#7593)
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
2024-07-24 22:55:07 -05:00
Bently
c98061bc3b fix(Builder): fixes to copy and pasted blocks (#7591)
* fix(Builder): fixes to copy and pasted blocks

* clear the blocks state on paste
2024-07-24 17:00:59 +01:00
Toran Bruce Richards
a8c0cbef54 refactor(Blocks): Simplify MathsBlock and CounterBlock to Return Numeric Results Only (#7582)
* refactor(MathsBlock): Simplify output to return numeric result directly

- Remove MathsResult class and explanation field
- Update Output schema to use float type
- Simplify run method to yield numeric result only
- Adjust error handling to return inf or nan for errors
- Update test cases to reflect new output structure

* run format

* refactor(CounterBlock): Simplify output to return count as integer

- Remove CounterResult class
- Update Output schema to use int type directly
- Simplify run method to yield count without explanation
- Modify error handling to return -1 for any errors
- Update test case to reflect new output structure
2024-07-24 08:55:18 -05:00
Zamil Majdy
8ccd14c4bf fix(rnd): Disable BlockInstallationBlock and add disable field for block (#7583) 2024-07-24 08:52:14 -05:00
Nicholas Tindle
3e384c9771 ci(server): add sqlite processing (#7586)
* ci(server): add sqlite processing

* ci(server): try setting DATABASE_URL based on db platform

* fix(server): swap default back to sqlite

* ci(server): go back to database url

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-07-24 14:46:39 +01:00
Aarushi
22f2a05f08 feat(server) Fix schema file (#7589)
* fix schema file

* remove invalide migrations

* make sqlite url hardcode
2024-07-24 14:31:48 +01:00
Krzysztof Czerwinski
e94a7b08c9 refactor(builder): Move renderInputField to NodeInputField component (#7581) 2024-07-24 11:14:20 +01:00