Commit Graph

6446 Commits

Author SHA1 Message Date
Nicholas Tindle
fb1f3d7e79 ci(test): add rabbitmq instance to ci 2025-02-13 06:56:50 -06:00
Nicholas Tindle
bfaf1e5fc2 fix(backend): manually start the notification service in testing? 2025-02-13 06:48:24 -06:00
Nicholas Tindle
78125371fc feat(backend): handle migration error 2025-02-13 06:43:48 -06:00
Nicholas Tindle
7e8207eb9b Merge branch 'dev' into ntindle/secrt-1077-add-email-service-smaller 2025-02-12 17:19:29 -06:00
Nicholas Tindle
7e04fbd25f feat(backend): schema updates, migration, queries for Email Notification Service (#9445)
<!-- Clearly explain the need for these changes: -->

The email service has requirements to
- Email users when some activity has happened on their account on some
scheduled basis -> We need a way to get active users and the executions
that happened while they were active
- Allow users to configure what emails they get -> Need a user
preference
- Get User email by Id so that we can email them -> Pretty
self-explanatory

We need to add a few new backend queries + db models for the
notification to start handling these details. This is the first set of
those changes based on experience building the app service

### Changes 🏗️

<!-- Concisely describe all of the changes made in this pull request:
-->
- Adds a new DB Model, `UserNotificationPreferences,` with related
migration
- Adds a new DB Model `NotificationEvent` with related migration to
track what notifications we've sent and how many and such (how much we
add here is open to change depending on what limits on data we want)
- Adds a new DB Model `UserNotificationBatch` with related migration to
handle batching of like models
- Adds queries to get users and executions by `datetime` ranges as `ISO`
strings
- Adds new queries to the `DatabaseManager` and exposes them to the
other `AppService`s
- Exposes all new queries plus an existing one `get_user_by_id`

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
- [x] I extracted these changes from a working implementation of the
service, and tested they don't bring down the service by being uncalled
by running the standard agent tests we do on release

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2025-02-12 22:54:16 +00:00
Nicholas Tindle
dc558966b8 Merge branch 'ntindle/secrt-1088-add-db-models-for-the-notification-service' into ntindle/secrt-1077-add-email-service-smaller 2025-02-12 16:44:22 -06:00
Nicholas Tindle
20267d62f6 Discard changes to autogpt_platform/frontend/yarn.lock 2025-02-12 16:20:01 -06:00
Nicholas Tindle
47e14d5525 fix(backend): move stuff to the user object and stop messing with 1-1 unique relationship 2025-02-12 15:59:25 -06:00
Nicholas Tindle
efe63503ac Merge branch 'ntindle/secrt-1088-add-db-models-for-the-notification-service' into ntindle/secrt-1077-add-email-service-smaller 2025-02-12 12:25:24 -06:00
Nicholas Tindle
21c3a6fdcc Merge branch 'dev' into ntindle/secrt-1088-add-db-models-for-the-notification-service 2025-02-12 12:18:00 -06:00
Bently
016ec0ff6b fix(frontend) update PublishAgentAwaitingReview router push path (#9471)
Updates the PublishAgentAwaitingReview router.push path, it was going to
``/marketplace/dashboard`` it should be ``/profile/dashboard``

### Changes 🏗️


8181ee8cd1/autogpt_platform/frontend/src/components/agptui/composite/PublishAgentPopout.tsx (L265-L267)

to be

```tsx
		onViewProgress={() => {
		  router.push("/profile/dashboard");
		  handleClose();
		}}
```
autogpt-platform-beta-v0.4.10
2025-02-12 15:32:47 +00:00
Zamil Majdy
3b8cde6d11 feat(block): Add batch matched result and its count on ExtractTextInformationBlock (#9470)
### Changes 🏗️

Introduced `matched_result` & `matched_count` as a batch matched result
list and its count for the block execution of
ExtractTextInformationBlock.

### Checklist 📋

#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [ ] ...

<details>
  <summary>Example test plan</summary>
  
  - [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
  - [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
  - [ ] Edit an agent from monitor, and confirm it executes correctly
</details>

#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)

<details>
  <summary>Examples of configuration changes</summary>

  - Changing ports
  - Adding new services that need to communicate with each other
  - Secrets or environment variable changes
  - New or infrastructure changes such as databases
</details>
2025-02-12 12:22:26 +00:00
Keith Webber
d050a3f77c docs: Provide feedback when cloning submodules (#9448)
<!-- Clearly explain the need for these changes: -->

### Changes 🏗️
Added `--progress` to the submodule update, so that cloning progress can
be tracked and does not appear to hang.
<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [x] No code change, just docs.
 

<details>
  <summary>Example test plan</summary>
  
  - [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
  - [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
  - [ ] Edit an agent from monitor, and confirm it executes correctly
</details>

#### For configuration changes:
- [x] No configuration change, just docs.
 

<details>
  <summary> Provide feedback when cloning submodules </summary>

  -  now updating submodules shows the cloning repo's progress
 
</details>

Co-authored-by: Your Name <you@example.com>
2025-02-12 03:43:52 +00:00
Nicholas Tindle
3e2f34180d Merge branch 'ntindle/secrt-1088-add-db-models-for-the-notification-service' into ntindle/secrt-1077-add-email-service-smaller 2025-02-11 21:36:38 -06:00
Nicholas Tindle
aa21bf7b2d fix(backend): format 2025-02-11 21:36:22 -06:00
Nicholas Tindle
e012e3aaf6 Merge branch 'ntindle/secrt-1088-add-db-models-for-the-notification-service' into ntindle/secrt-1077-add-email-service-smaller 2025-02-11 21:10:39 -06:00
Nicholas Tindle
ddd2b9de15 fix(backend): the tests did dumb stuff like reusing ids 2025-02-11 21:09:51 -06:00
Nicholas Tindle
85f8f4136f fix(backend): relock 2025-02-11 20:33:49 -06:00
Nicholas Tindle
1cd90ef113 Merge branch 'dev' into ntindle/secrt-1088-add-db-models-for-the-notification-service 2025-02-11 20:32:47 -06:00
Nicholas Tindle
3562090495 Merge branch 'ntindle/secrt-1088-add-db-models-for-the-notification-service' into ntindle/secrt-1077-add-email-service-smaller 2025-02-11 20:30:48 -06:00
Nicholas Tindle
c69df5cb79 ref(backend): update from pr changes 2025-02-11 20:30:33 -06:00
Reinier van der Leer
1626bf9e16 fix(backend): Support Python 3.10 (#9468)
- Resolves #9467

### Changes 🏗️

- Loosen Python version requirement to include v3.10

Also, fixed a few issues in pyproject.toml:
- Re-sort dependency list
- Update `autogpt-platform-backend` package version to match latest
release
2025-02-12 02:02:00 +00:00
Nicholas Tindle
957ebe697f refactor(backend): raise errors from db queries 2025-02-11 15:08:33 -06:00
Nicholas Tindle
82d96ee7b0 Merge branch 'ntindle/secrt-1088-add-db-models-for-the-notification-service' into ntindle/secrt-1077-add-email-service-smaller 2025-02-11 15:01:25 -06:00
Nicholas Tindle
11cda46724 ref(backend): pr changes 2025-02-11 15:01:15 -06:00
Nicholas Tindle
bf49a0a08a Update autogpt_platform/backend/backend/data/notifications.py 2025-02-11 14:26:33 -06:00
Nicholas Tindle
e2441c5220 Apply suggestions from code review
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2025-02-11 14:23:00 -06:00
Nicholas Tindle
63e3582114 Merge branch 'ntindle/secrt-1088-add-db-models-for-the-notification-service' of https://github.com/Significant-Gravitas/AutoGPT into ntindle/secrt-1088-add-db-models-for-the-notification-service 2025-02-11 14:20:11 -06:00
Nicholas Tindle
0e99bdc742 ref(backend): smash migrations and apply changes to api 2025-02-11 14:20:04 -06:00
Nicholas Tindle
91df11b44c ref(backend): use lowercase types
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2025-02-11 14:19:33 -06:00
Andy Hooker
40613fe23e fix(frontend): Update user profile from marketplace to appropriate profile route (#9465)
### Background
Resolves: #9313

The application is incorrectly nesting the user's profile settings
within the route of `/marketplace` instead of the appropriate route of
`/profile`

This pr will modify the existing code to handle the relocation of the
(user) directory from the /marketplace to /profile.

### Changes 🏗️

1. Refactored directory of `(user)`:
- Moved the directory of (user) from `/marketplace/(user)` to
`profile/(user)`


2. Update Sidebar and Navbar components:
- Updating the existing code from the routing of market to profile by
modifying the existing routes.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:

<details>
  <summary>Test Plan</summary>
  
  - [ ] Navigate to the route of `profile/` and observe the moved page.
- [ ] Navigate to the route of `profile/integrations` and observe the
moved page.
- [ ] Navigate to the route of `profile/api_keys` and observe the moved
page.
- [ ] Navigate to the route of `profile/profile` and observe the moved
page.
- [ ] Navigate to the route of `profile/settings` and observe the moved
page.
- [ ] Navigate to the route of `profile/credits` and observe the moved
page.
</details>
2025-02-11 11:25:15 +00:00
Andy Hooker
6eee9206f7 fix(market): Market featured agent card (#9463)
### Background
Resolves: #9313

The marketplace featured agent's section has a bug where if you hover
over a featured agent's card we are getting an incorrect background
color applied to the description.

### Changes 🏗️

1. Refactored `FeaturedStoreCard` to `FeaturedAgentCard`:
   - Condensed props and leverage StoreAgent type from api
- Removed onClick handler from props as this is not json serializable
and is not inline with NextJS best practices
- Used built in Card Components from ShadCN to minimize custom styling.
- Optimize images with implementation of the Image component from NextJS

2. Enhanced `FeaturedCardSection` components:
- Removing extensive prop passing and leverage the agent itself with the
StoreAgent type.
- Implemented Link from NextJS to better handler routing and remove the
`useRouter` implementation
   - Removed unnecessary handleCardClick method.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:

Test Plan
<details>
  <summary></summary>
 
  - [ ] Goto the landing page of the application or /marketplace 
  - [ ] Scroll to the featured agents section
- [ ] Move mouse over each of the cards and observe the image
disappearing and text being shown
- [ ] Observe the background color of the text that replaced the image
matches that of the card
</details>
2025-02-10 22:01:29 +00:00
Nicholas Tindle
cee7929b02 feat(backend): setup first notification service 2025-02-10 15:54:02 -06:00
Nicholas Tindle
c259e5b145 feat(backend): spawn the notifications service 2025-02-10 15:53:16 -06:00
Nicholas Tindle
c71bb17658 fix(backend): relock 2025-02-10 15:31:11 -06:00
Nicholas Tindle
4481827480 Merge branch 'dev' into ntindle/secrt-1088-add-db-models-for-the-notification-service 2025-02-10 15:28:12 -06:00
Nicholas Tindle
bada17d6d3 feat(backend): pull up changes from downstream 2025-02-10 15:25:13 -06:00
Muhammad Safi
64050faef6 feature(block): Add XML Parser Block (#9450)
-Updated pyproject.toml for new dependency gravitasml
-Updated poetry.lock

<!-- Clearly explain the need for these changes: -->
- Issue no #9317 stated that, the addition of an XMLParserBlock is
required.
- It was suggested that the use of gravitasml as external package to be
used for parsing.
- Changes incorporated and tested as per requirements.
### Changes 🏗️
- Added xml_parser.py
- updated pyproject.toml
- updated poetry.lock for dependency changes 
<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [ ] ...

<details>
  <summary>Example test plan</summary>
  
  - [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
  - [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
  - [ ] Edit an agent from monitor, and confirm it executes correctly
</details>

#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

<details>
  <summary>Examples of configuration changes</summary>

  - Changing ports
  - Adding new services that need to communicate with each other
  - Secrets or environment variable changes
  - New or infrastructure changes such as databases
</details>

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2025-02-10 21:00:09 +00:00
Nicholas Tindle
26ce2811e3 Merge branch 'dev' into ntindle/secrt-1088-add-db-models-for-the-notification-service 2025-02-10 08:21:32 -06:00
Krzysztof Czerwinski
00c312d02c feat(platform): Schedule specific agent version (#9444)
Scheduling always takes the newest version of an agent.

### Changes 🏗️

This PR allows to schedule any graph version by adding number input to
schedule popup. Number is automatically set to the newest version when
agent is chosen.

<img width="533" alt="Screenshot 2025-02-07 at 5 05 56 PM"
src="https://github.com/user-attachments/assets/357b8810-6f02-4066-b7a3-824d9bfd62af"
/>

- Update API, so it accepts graph version
- Update schedule pop up, so it lets user input version number
- Open and schedule correct agent
- Add `Version` column to the schedules table

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Can schedule version between 1 and max version
  - [x] Reject incorrect version
  - [x] Table shows proper version
  - [x] Removing schedule works
2025-02-10 13:24:25 +00:00
Nicholas Tindle
6ed6fa1033 fix(backend): broken db query oops 2025-02-10 06:35:34 -06:00
Nicholas Tindle
64e6294abf fix(backend): linting 2025-02-10 06:21:14 -06:00
Nicholas Tindle
eeec3c3424 fix(backend): add the package deps 2025-02-10 06:13:21 -06:00
Nicholas Tindle
1b28a04072 feat(backend): add more migrations 2025-02-10 06:03:49 -06:00
Nicholas Tindle
7dbb1a2d30 feat(backend): bring downstream changes up 2025-02-10 05:59:41 -06:00
Nicholas Tindle
ea07f03638 Merge branch 'dev' into ntindle/secrt-1088-add-db-models-for-the-notification-service 2025-02-10 02:51:47 -06:00
Nicholas Tindle
610be988c4 feat(backend): attach rabbitmq to the AppService (#9438)
### Changes 🏗️
For Emailing, we need to make a new App Service (NotificationManager)
that will require us to have rabbitmq as a dependency. This is the
backing data library to make that happen + registering it with the app
service base class and connecting when we spawn up a service

<!-- Concisely describe all of the changes made in this pull request:
-->
- Adds a rabbitmq library following the existing standard for redis
- Adds rabbitmq to the service
- Adds rabbitmq mgmt library (pika) so that we can connect to rabbitmq

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
- [x] I tested by adding the following to the executor `@expose
add_execution` and verifying via the UI that the messages show up in the
queue as expected + the agent executes and behaves as normal!

![image](https://github.com/user-attachments/assets/3ebfb850-d482-4b11-901c-d3bf3397a346)

```diff
      diff --git a/autogpt_platform/backend/backend/executor/manager.py b/autogpt_platform/backend/backend/executor/manager.py
index 1d965e012..4cd5b403c 100644
--- a/autogpt_platform/backend/backend/executor/manager.py
+++ b/autogpt_platform/backend/backend/executor/manager.py
@@ -18,7 +18,7 @@ if TYPE_CHECKING:
 from autogpt_libs.utils.cache import thread_cached
 
 from backend.blocks.agent import AgentExecutorBlock
-from backend.data import redis
+from backend.data import rabbitmq, redis
 from backend.data.block import (
     Block,
     BlockData,
@@ -750,6 +750,19 @@ class ExecutionManager(AppService):
     def __init__(self):
         super().__init__()
         self.use_redis = True
+        self.use_rabbitmq = rabbitmq.RabbitMQConfig(
+            exchanges=[
+                rabbitmq.Exchange(name="execution", type=rabbitmq.ExchangeType.FANOUT),
+            ],
+            queues=[
+                rabbitmq.Queue(
+                    name="execution",
+                    exchange=rabbitmq.Exchange(
+                        name="execution", type=rabbitmq.ExchangeType.FANOUT
+                    ),
+                ),
+            ],
+        )
         self.use_supabase = True
         self.pool_size = settings.config.num_graph_workers
         self.queue = ExecutionQueue[GraphExecutionEntry]()
@@ -876,6 +889,12 @@ class ExecutionManager(AppService):
         )
         self.queue.add(graph_exec)
 
+        # test rabbitmq
+        self.rabbit.publish_message(
+            exchange=self.rabbit_config.exchanges[0],
+            routing_key=self.rabbit_config.exchanges[0].name,
+            message=graph_exec_id,
+        )
         return graph_exec
 
     @expose
```
2025-02-08 21:30:30 +00:00
Nicholas Tindle
74b1ea05ec feat(backend): update models a bit more 2025-02-07 20:47:40 -06:00
Nicholas Tindle
815935f2f5 feat(backend): expose added queries 2025-02-07 16:13:22 -06:00
Nicholas Tindle
7ff6f48873 feat(backend): executions in time range query 2025-02-07 16:13:12 -06:00