Commit Graph

5697 Commits

Author SHA1 Message Date
Krzysztof Czerwinski
2618d1d87c refactor(builder): Fix linting warning and errors (#8021)
* Fix lint errors

* Fix dependency loop
2024-09-09 09:54:36 +02:00
Andy Hooker
e17ea22a0a feat(builder): Aligning error and loading for NextJS best practices (#7894)
* feat(builder): Add skeleton loading components for Monitor views

Introduce skeleton components for Agents, Flow Runs List, and Flow Runs Status sections to enhance loading state indication. These components help improve user experience by visually outlining content placeholders while data is being fetched.

* feat(builder): Leveraging NextJS's error boundary with error.tsx

Replace the basic error page with a more detailed and interactive error component. The new component includes a retry option, a link to the homepage, and logs the error details to the console. It also aligns with NextJS standards

---------
2024-09-08 12:47:36 +02:00
Nicholas Tindle
60669903a0 fix: couple block categories improved (#8017) 2024-09-06 22:47:40 -05:00
Zamil Majdy
b1b31390a4 feat(rnd): Add creation & update time for AgentGraphExecutionSchedule, AgentGraphExecution, and AgentGraph (#8015) 2024-09-06 21:29:53 +00:00
Nicholas Tindle
3c12a398ae feat: marketplace analytics (#7998) 2024-09-06 19:22:24 +00:00
Aarushi
126d070396 tweak(rnd,redis) Make redis logging more clear (#8014)
make redis logging more clear
2024-09-06 15:23:33 +00:00
Reinier van der Leer
090f22b05c fix(server): Improve logging consistency (#8012)
- Make process/service startup/shutdown messages consistent
- Configure `uvicorn` to use our logging config instead of its own
- Replace `print(..)` statements in ws_api.py with log statements
- Improve log statements in ws_api.py
2024-09-06 17:05:30 +02:00
Reinier van der Leer
1b9adf5434 fix(server): Always JSON-encode AgentNodeExecutionInputOutput data (#8010)
- Handle JSON-encoding inside `.data.execution.upsert_execution_output(..)` to ensure it is always encoded the same
- Amend `.executor.manager.execute_node(..)` to pass unencoded data into `upsert_execution_output(..)`
2024-09-06 16:58:04 +02:00
Reinier van der Leer
3bd8040d6a feat(server): Clean up resources when spinning down services/processes (#7938)
- Add SIGTERM handler and `cleanup()` hook to `AppProcess`
- Implement `cleanup()` on `AppService` to close DB and Redis connections
- Implement `cleanup()` on `ExecutionManager` to shut down worker pool
- Add `atexit` and SIGTERM handlers to node executor to close DB connection and shut down node workers
- Improve logging in `.executor.manager`
- Fix shutdown order of `.util.test:SpinTestServer`
2024-09-06 16:50:59 +02:00
Aarushi
b12dba13f4 docs(readme) Update readme to include non docker steps (#8013)
update readme
2024-09-06 15:33:17 +01:00
Swifty
2cae9ba8da feat(server): Updated Output block (#7997)
* Add Block UI Types and StickyNote Block

* Renamed StickyNote to Note

* Add comment

* Updated Input Block

* rename default_values to placeholder_values

* Update sample graph

* Update input block output to match previous change

* fixing test

* Updated the output block

* re-remove old output block
2024-09-06 15:56:41 +02:00
Swifty
3753906482 feat(builder): Block UI Types and StickyNote (#7994)
* Add Block UI Types and StickyNote Block

* Renamed StickyNote to Note

* Add comment
2024-09-06 15:52:03 +02:00
Swifty
fd54ad8666 update(server): update input nodes (#7996)
* Add Block UI Types and StickyNote Block

* Updated Input Block
2024-09-06 13:27:37 +01:00
Zamil Majdy
e645cc4b33 tweak(rnd): Remove duplicated call 2024-09-05 17:20:55 -05:00
Zamil Majdy
010a8ffaaf tweak(rnd): Stop publishing custom Sentry metrics (#8000) 2024-09-05 21:51:00 +00:00
Zamil Majdy
2df325d033 fix(rnd): Fix prisma connection acquisition intermittent error on linux (#7999) 2024-09-06 03:22:25 +07:00
Nicholas Tindle
79ebc4c13b feat(builder): promotion/demotion of featured agents (#7932) 2024-09-05 20:04:11 +00:00
Zamil Majdy
e5eb42d84a tweak(rnd): Post infra change cleanup - fix process creation lifecycle (#7981) 2024-09-06 01:41:24 +07:00
Bently
d62b940baf Feat(Monitor/Builder): Fix loading of "Import from file" modal (#7976)
* Feat(Monitor/Builder): Fix loading of "Import from file" modal

* prettier

* prettier - forgot page.tsx
2024-09-05 15:02:48 +00:00
Reinier van der Leer
8fd22bcfd7 feat(server, builder): Implement "STOP" button for graph runs (#7892)
- feat(builder): Add "Stop Run" buttons to monitor and builder
  - Implement additional state management in `useAgentGraph` hook
    - Add "stop" request mechanism
    - Implement execution status tracking using WebSockets
    - Add `isSaving`, `isRunning`, `isStopping` outputs
    - Add `requestStopRun` method
      - Rename `requestSaveRun` to `requestSaveAndRun` for clarity
  - Add needed functionality for the above to `AutoGPTServerAPI` client
    - Add `stopGraphExecution` method
    - Add support for multiple handlers per WebSocket method
    - Fix parsing of timestamps in `execution_event` WebSocket messages
  - Add `IconSquare` from Lucide to `@/components/ui/icons`

- feat(server): Add `POST /graphs/{graph_id}/executions/{graph_exec_id}/stop` route
  - Add `stop_graph_run` method to `AgentServer`

- feat(server): Add `cancel_execution` method to `ExecutionManager`
  - Replace node executor `ProcessPoolExecutor` by `multiprocessing.Pool` (which has a `terminate()` method)
    - Remove now unnecessary `Executor.wait_future(..)` method
  - Add `get_graph_execution(..)` in `.data.execution`

- fix(server): Reduce number of node executors to 5 per graph executor
  This is necessary because `multiprocessing.Pool` spawns its workers on init, instead of based on demand like `ProcessPoolExecutor` does

- dx(server): Improve debug logging in `ExecutionManager`
- ci(server): Add debug logging mode to CI Pytest step

### Other improvements
Server:
- Improve output type of `ExecutionManager.add_execution(..)`
- Renamed a few things in `.server.rest_api` for consistency

Front end:
- Improved typing in `AutoGPTServerAPI` client
2024-09-05 14:42:28 +02:00
Bently
11827835a0 Feat(Builder): Add tooltips to Blocks and Save buttons (#7975) 2024-09-05 12:04:29 +00:00
Zamil Majdy
70fab8711a fix(rnd): avoid duplicating name on input/output pin for blocks (#7979) 2024-09-05 10:54:02 +00:00
Swifty
8ec015ba72 fix(builder): update tutorial routing to work on safari (#7992) 2024-09-05 10:27:15 +00:00
SwiftyOS
bc7d2f0f37 added test password as default for redis queue 2024-09-05 11:41:53 +02:00
Zamil Majdy
54694709bb tweak(rnd): Hide non required field on Blocks (#7977)
Add prompt_values for LLM block, and make it non-advanced.
Make all-field, advanced by default.
cleanup .env.example and .env.template
2024-09-05 05:00:14 +07:00
Zamil Majdy
b4b5a09b6b fix(rnd): Dockerfile Avoid full rebuild on each file change (#7971)
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-09-04 17:30:13 +00:00
Reinier van der Leer
82239dd129 fix(server): Fix node input concurrency deadlock (#7936)
In `autogpt_server.util.lock:KeyedMutex`:
- track number of pending requests for each lock
- only remove a lock from `self.locks` when the number of pending lock requests hits 0
2024-09-04 18:13:35 +02:00
Aarushi
078ad29356 update docker compose (#7972) 2024-09-04 16:19:41 +01:00
Aarushi
5000aa7ee0 tweak(rnd,docker) Remove SQLite (#7966)
* move migrations, update networking and dockignore

* update docs

* remove sqlite from ci

* remove schema linting checks

* fix formatting

* remove schema linting

* add test script

* formatting and linting

* stop pg not down

* seperate test db

* diff port

* remove duplicate
2024-09-04 10:18:57 +01:00
Bently
dc1077f893 Feat(Builder): Replace `Math Block with Calculator Block` (#7969) 2024-09-04 08:12:01 +00:00
Swifty
80df44a978 feat(server): Add endpoint to calculate required graph inputs (#7965) 2024-09-04 09:45:45 +02:00
Zamil Majdy
c2a79d2f10 feat(rnd): Add Node & Graph level execution stats instrumentation (#7957)
### **User description**
### Background

The scope of this change is collecting the required information that will be needed for the execution analytics.

### Changes 🏗️

* Add sentry integration.
* Refactor logging_metadata on manager.py.
* Collect graph-level & node-level instrumentation.
* Introduced `stats` column for `AgentNodeExecution` & `AgentGraphExecution`.
2024-09-04 02:45:19 +07:00
Bently
7db85a8990 Feat(Builder): Update tutorial to work with latest block UI (#7962)
* Feat(Builder): Update tutorial to work with latest block UI

* prettier
2024-09-03 14:56:55 +00:00
Krzysztof Czerwinski
0454a9a7be fix(builder): Prevent zooming on input field modal
- Add `nowheel` class to Textarea parent div
2024-09-03 14:30:35 +02:00
Krzysztof Czerwinski
09951fed4b feat(builder, server): Add advanced block inputs (#7934)
- Add `advanced` to `SchemaField` and pass it to `json_extra`
- Add `advanced` to `BlockIOSubSchemaMeta` type
- Update `CustomNode`, so that:
  - non-required advanced inputs are hidden
  - non-advanced and required inputs are always shown
2024-09-03 12:01:42 +01:00
Aarushi
6204d82d84 feat(server): Integrate forge.logging (#7915)
* feat(server): Integrate `forge.logging`

- Add `configure_logging()` in `.util.logging` - a wrapper for `forge.logging.configure_logging()` with project-specific extras
- Call `configure_logging()` in `.app.main()`, and in child process initializers (e.g. `AppProcess.execute_run_command(..)`, `ExecutorManager.on_graph_executor_start()`)
- Change some `logger.warning` statements to `logger.info` where appropriate

* fix warnings to info

* fix(rnd): Fix broken test and Input/Output block field renaming

* Rename

* fix(rnd): Fix flaky CI

* feat(server): Add OAuth handlers for GitHub, Notion, Google & amend store data structure (#7868)

- Add `BaseOAuthHandler` + 3 initial implementations
  - Add `GitHubOAuthHandler`
  - Add `NotionOAuthHandler`
  - Add `GoogleOAuthHandler`
- Amend `OAuth2Credentials` type
  - Add `metadata` attribute
  - Make `access_token_expires_at`, `refresh_token`, `refresh_token_expires_at` optional

* extend GCP Logger

* update manager & add flag

* linting

* use default logger behaviour

* update messages

* update another message

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2024-09-03 11:56:21 +01:00
Krzysztof Czerwinski
8c9fe5c167 feat(builder): Resizable input modal (#7955)
- Add minimize/maximize button in the corner of modal to make it significantly larger and centered
- Add copy button to copy all text
- Add optional `title` to display as a modal header
2024-09-03 11:42:25 +01:00
Reinier van der Leer
71de1a6a5e fix(server): Fix type checking and propagation issues (#7941)
- fix type propagation by `AppService.run_and_wait(..)`
- fix type propagation by `@expose` and add note
- fix type propagation by `wait(..)` in `.executor.manager.execute_node(..)`
- fix type propagation by `wait(..)` in `.executor.manager._enqueue_next_nodes(..)`
- remove unnecessary null checks for `.data.graph.get_node(..)`
- fix type issue in `ExecutionScheduler`
- reduce use of `# type: ignore` in `.data.execution`
- reduce usage of `# type: ignore` in `.executor.manager`
- reduce usage of `# type: ignore` in `.server`
- reduce usage of `# type: ignore` in cli.py
- update `pyright` to v1.1.378
2024-09-02 14:13:56 +00:00
Bently
956165adf3 Feat(Builder): Fix block menu width (#7946) 2024-09-02 09:54:52 +00:00
Aarushi
e4dc16a867 docs(server): Update docs with new docker compose steps (#7944)
* update setup

* add docker links
2024-09-02 10:55:54 +02:00
Aarushi
cfa0b6610c feat(rnd,infra): Pull out websockets away from server api (#7899)
* standalone websocket server

* add websocket url

* wip: talk to ws directly

* rename to api server

* dockerfile and queue

* fix paths

* update poetry lock

* helm charts for websockets

* create seperate deployments for websockets and rest server with redis queue for async comms

* delete duplicate queue

* add depends in ws_api

* singleton for conn manager

* update from review

* fix CI

* address feedback

* update readme

* update docker file and add migration step in readm

* ad watch

* add step to copy example env file

* put connect back in
2024-09-02 10:32:51 +02:00
Krzysztof Czerwinski
933baa0e8d feat(builder): Store and display output data history in nodes (#7930)
- Update styling and use tailwind more
- Add `react-toast` dependency
- Fix output button not changing checked state on execution
- Make status a badge in node's corner
- Rename `output_data` to `executionResults` and store multiple results with execution UUIDs
- Add `DataTable` component that displays execution outputs
- Outputs can be copied and there's a toast displayed as a feedback
2024-09-01 11:18:57 +01:00
Bently
370b2dabe8 Feat(Builder): Add block categories to block menu (#7918) 2024-08-31 19:22:37 +00:00
Nicholas Tindle
baa00a5b03 feat(builder,server): add review of agent submissions to the admin portal (#7914)
Co-authored-by: Aarushi <aarushik93@gmail.com>
2024-08-30 18:20:59 +00:00
Zamil Majdy
60a8e00578 fix(rnd): Fix overflowing card due to long block name (#7931) 2024-08-30 18:02:55 +00:00
Bently
85e7d678ce Feat(Builder): Fix Multiple Connections Between Pins (#7924) 2024-08-30 18:15:18 +01:00
Krzysztof Czerwinski
476b307d69 feat(builder): Draw selected edges above nodes (#7893) 2024-08-30 16:39:22 +01:00
Bently
5dbfb4e3f1 feat(builder): Turn block border red on error (#7910)
* Turn block border red on error

* prettier
2024-08-30 15:09:19 +00:00
Zamil Majdy
f6d09c74f5 feat(rnd): Rename Blocks (#7925)
### Background

Standardize block names with a focus of making them intuitive to a non-technical person.

### Changes 🏗️

Replace these names:
* TextLlmCallBlock -> AITextGeneratorBlock
* ObjectLlmCallBlock -> AIStructuredResponseBlock
* AdvancedLlmCallBlock -> AIConversationBlock
* CreateTalkingAvatarClipBlock -> CreateTalkingAvatarVideoBlock
* DiscordReaderBlock -> ReadDiscordMessagesBlock
* DiscordMessageSenderBlock -> SendDiscordMessageBlock
* YouTubeTranscriberBlock -> TranscribeYouTubeVideoBlock
* CreateMediumPostBlock -> PublishToMediumBlock
* ForEachBlock -> ListIteratorBlock
* MathsBlock -> CalculatorBlock
* CounterBlock -> CountItemsBlock
* ValueBlock -> StoreValueBlock
* PrintingBlock -> PrintToConsoleBlock
* DictionaryAddEntryBlock -> AddToDictionaryBlock
* ListAddEntryBlock -> AddToListBlock
* ObjectLookupBlock -> FindInDictionaryBlock
* RedditGetPostsBlock -> GetRedditPostsBlock
* RedditPostCommentBlock -> PostRedditCommentBlock
* WikipediaSummaryBlock -> GetWikipediaSummaryBlock
* WebSearchBlock -> SearchTheWebBlock
* WebScraperBlock -> ExtractWebsiteContentBlock
* GetOpenWeatherMapBlock -> GetWeatherInformationBlock
* HttpRequestBlock -> SendWebRequestBlock
* CurrentTimeBlock -> GetCurrentTimeBlock
* CurrentDateBlock -> GetCurrentDateBlock
* CurrentDateAndTimeBlock -> GetCurrentDateAndTimeBlock
* TimerBlock -> CountdownTimerBlock
* RSSReaderBlock -> ReadRSSFeedBlock
* TextMatcherBlock -> MatchTextPatternBlock
* TextParserBlock -> ExtractTextInformationBlock
* TextFormatterBlock -> FillTextTemplateBlock
* TextCombinerBlock -> CombineTextsBlock
2024-08-30 21:36:42 +07:00
Zamil Majdy
6d17e627e8 fix(rnd): Add null checking & remove console logs on Input UI component (#7927)
fix(rnd): And null checking & remove logs
2024-08-30 12:21:09 +00:00