passing logger context

fix docker issues with market, and DB connection

.dockerignore

@@ -36,3 +36,5 @@ rnd/autogpt_builder/.env.example
 rnd/autogpt_builder/.env.local
 rnd/autogpt_server/.env
 rnd/autogpt_server/.venv/
+
+rnd/market/.env

docs/content/forge/components/built-in-components.md

@@ -69,6 +69,8 @@ Lets the agent execute non-interactive Shell commands and Python code. Python ex
 | `shell_denylist`         | List of prohibited shell commands                    | `List[str]`                 | `[]`              |
 | `docker_container_name`  | Name of the Docker container used for code execution | `str`                       | `"agent_sandbox"` |
 
+All shell command configurations are expected to be for convience only. This component is not secure and should not be used in production environments. It is recommended to use more appropriate sandboxing.
+
 ### CommandProvider
 
 - `execute_shell` execute shell command

docs/content/server/setup.md

@@ -73,6 +73,7 @@ Once you have installed Yarn and Poetry, you can run the following command to in
 
 ```bash
 cd rnd/autogpt_server
+cp .env.example .env
 poetry install
 ```
 
@@ -90,7 +91,7 @@ Once you have installed the dependencies, you can proceed to the next step.
 In order to setup the database, you need to run the following commands, in the same terminal you ran the `poetry install` command:
 
    ```sh
-   docker compose up postgres -d
+   docker compose up postgres redis -d
    poetry run prisma migrate dev
    ```
 After deploying the migration, to ensure that the database schema is correctly mapped to your codebase, allowing the application to interact with the database properly, you need to generate the Prisma database model:
@@ -101,7 +102,15 @@ poetry run prisma generate
 
 Without running this command, the necessary Python modules (prisma.models) won't be available, leading to a `ModuleNotFoundError`.
 
-### Running the server
+### Running the server without Docker
+
+To run the server, you can run the following commands in the same terminal you ran the `poetry install` command:
+
+```bash
+poetry run app
+```
+
+### Running the server within Docker
 
 To run the server, you can run the following commands in the same terminal you ran the `poetry install` command:
 
@@ -110,7 +119,7 @@ docker compose build
 docker compose up
 ```
 
-In the other terminal, you can run the following command to start the frontend:
+In the other terminal from autogpt_builder, you can run the following command to start the frontend:
 
 ```bash
 yarn dev
@@ -119,3 +128,10 @@ yarn dev
 ### Checking if the server is running
 
 You can check if the server is running by visiting [http://localhost:3000](http://localhost:3000) in your browser.
+
+### Notes: 
+By default the daemons for different services run on the following ports: 
+
+Execution Manager Daemon: 8002
+Execution Scheduler Daemon: 8003
+Rest Server Daemon: 8004

rnd/README.md

@@ -1,36 +1,114 @@
-This is a guide to setting up and running the AutoGPT Server and Builder. This tutorial will cover downloading the necessary files, setting up the server, and testing the system.
+# AutoGPT Platform
 
-https://github.com/user-attachments/assets/fd0d0f35-3155-4263-b575-ba3efb126cb4
+Welcome to the AutoGPT Platform - a powerful system for creating and running AI agents to solve business problems. This platform enables you to harness the power of artificial intelligence to automate tasks, analyze data, and generate insights for your organization.
 
-1. Navigate to the AutoGPT GitHub repository.
-2. Click the "Code" button, then select "Download ZIP".
-3. Once downloaded, extract the ZIP file to a folder of your choice.
+## Getting Started
 
-4. Open the extracted folder and navigate to the "rnd" directory.
-5. Enter the "AutoGPT server" folder.
-6. Open a terminal window in this directory.
-7. Locate and open the README file in the AutoGPT server folder: [doc](./autogpt_server/README.md#setup).
-8. Copy and paste each command from the setup section in the README into your terminal.
-   - Important: Wait for each command to finish before running the next one.
-9. If all commands run without errors, enter the final command: `poetry run app`
-10. You should now see the server running in your terminal.
+### Prerequisites
+
+- Docker
+- Docker Compose V2 (comes with Docker Desktop, or can be installed separately)
+
+### Running the System
+
+To run the AutoGPT Platform, follow these steps:
+
+1. Clone this repository to your local machine.
+2. Navigate to the project directory.
+3. Run the following command:
+
+   ```
+   docker compose up -d
+   ```
+
+   This command will start all the necessary services defined in the `docker-compose.yml` file in detached mode.
+
+### Docker Compose Commands
+
+Here are some useful Docker Compose commands for managing your AutoGPT Platform:
+
+- `docker compose up -d`: Start the services in detached mode.
+- `docker compose stop`: Stop the running services without removing them.
+- `docker compose rm`: Remove stopped service containers.
+- `docker compose build`: Build or rebuild services.
+- `docker compose down`: Stop and remove containers, networks, and volumes.
+- `docker compose watch`: Watch for changes in your services and automatically update them.
+
+
+### Sample Scenarios
+
+Here are some common scenarios where you might use multiple Docker Compose commands:
+
+1. Updating and restarting a specific service:
+   ```
+   docker compose build api_srv
+   docker compose up -d --no-deps api_srv
+   ```
+   This rebuilds the `api_srv` service and restarts it without affecting other services.
+
+2. Viewing logs for troubleshooting:
+   ```
+   docker compose logs -f api_srv ws_srv
+   ```
+   This shows and follows the logs for both `api_srv` and `ws_srv` services.
+
+3. Scaling a service for increased load:
+   ```
+   docker compose up -d --scale executor=3
+   ```
+   This scales the `executor` service to 3 instances to handle increased load.
+
+4. Stopping the entire system for maintenance:
+   ```
+   docker compose stop
+   docker compose rm -f
+   docker compose pull
+   docker compose up -d
+   ```
+   This stops all services, removes containers, pulls the latest images, and restarts the system.
+
+5. Developing with live updates:
+   ```
+   docker compose watch
+   ```
+   This watches for changes in your code and automatically updates the relevant services.
+
+6. Checking the status of services:
+   ```
+   docker compose ps
+   ```
+   This shows the current status of all services defined in your docker-compose.yml file.
+
+These scenarios demonstrate how to use Docker Compose commands in combination to manage your AutoGPT Platform effectively.
+
+
+### Persisting Data
+
+To persist data for PostgreSQL and Redis, you can modify the `docker-compose.yml` file to add volumes. Here's how:
+
+1. Open the `docker-compose.yml` file in a text editor.
+2. Add volume configurations for PostgreSQL and Redis services:
+
+   ```yaml
+   services:
+     postgres:
+       # ... other configurations ...
+       volumes:
+         - postgres_data:/var/lib/postgresql/data
+
+     redis:
+       # ... other configurations ...
+       volumes:
+         - redis_data:/data
+
+   volumes:
+     postgres_data:
+     redis_data:
+   ```
+
+3. Save the file and run `docker compose up -d` to apply the changes.
+
+This configuration will create named volumes for PostgreSQL and Redis, ensuring that your data persists across container restarts.
 
-11. Navigate back to the "rnd" folder.
-12. Open the "AutoGPT builder" folder.
-13. Open the README file in this folder: [doc](./autogpt_builder/README.md#getting-started).
-14. In your terminal, run the following commands:
-    ```
-    npm install
-    ```
-    ```
-    npm run dev
-    ```
 
-15. Once the front-end is running, click the link to navigate to `localhost:3000`.
-16. Click on the "Build" option.
-17. Add a few blocks to test the functionality.
-18. Connect the blocks together.
-19. Click "Run".
-20. Check your terminal window - you should see that the server has received the request, is processing it, and has executed it.
 
-And there you have it! You've successfully set up and tested AutoGPT.

rnd/autogpt_builder/.storybook/main.ts

@@ -1,18 +0,0 @@
-import type { StorybookConfig } from "@storybook/nextjs";
-
-const config: StorybookConfig = {
-  stories: ["../src/**/*.mdx", "../src/**/*.stories.@(js|jsx|mjs|ts|tsx)"],
-  addons: [
-    "@storybook/addon-onboarding",
-    "@storybook/addon-links",
-    "@storybook/addon-essentials",
-    "@chromatic-com/storybook",
-    "@storybook/addon-interactions",
-  ],
-  framework: {
-    name: "@storybook/nextjs",
-    options: {},
-  },
-  staticDirs: ["../public"],
-};
-export default config;

rnd/autogpt_builder/.storybook/preview.ts

@@ -1,14 +0,0 @@
-import type { Preview } from "@storybook/react";
-
-const preview: Preview = {
-  parameters: {
-    controls: {
-      matchers: {
-        color: /(background|color)$/i,
-        date: /Date$/i,
-      },
-    },
-  },
-};
-
-export default preview;

rnd/autogpt_builder/src/app/error.tsx

@@ -26,8 +26,8 @@ export default function Error({
           Oops, something went wrong!
         </h1>
         <p className="mt-4 text-muted-foreground">
-          We're sorry, but an unexpected error has occurred. Please try again
-          later or contact support if the issue persists.
+          We&apos;re sorry, but an unexpected error has occurred. Please try
+          again later or contact support if the issue persists.
         </p>
         <div className="mt-6 flex flex-row justify-center gap-4">
           <Button onClick={reset} variant="outline">

rnd/autogpt_builder/src/app/page.tsx

@@ -1,5 +1,5 @@
 "use client";
-import React, { useEffect, useState } from "react";
+import React, { useCallback, useEffect, useMemo, useState } from "react";
 
 import AutoGPTServerAPI, {
   GraphMeta,
@@ -22,54 +22,57 @@ const Monitor = () => {
   const [selectedFlow, setSelectedFlow] = useState<GraphMeta | null>(null);
   const [selectedRun, setSelectedRun] = useState<FlowRun | null>(null);
 
-  const api = new AutoGPTServerAPI();
+  const api = useMemo(() => new AutoGPTServerAPI(), []);
 
-  useEffect(() => fetchFlowsAndRuns(), []);
+  const refreshFlowRuns = useCallback(
+    (flowID: string) => {
+      // Fetch flow run IDs
+      api.listGraphRunIDs(flowID).then((runIDs) =>
+        runIDs.map((runID) => {
+          let run;
+          if (
+            (run = flowRuns.find((fr) => fr.id == runID)) &&
+            !["waiting", "running"].includes(run.status)
+          ) {
+            return;
+          }
+
+          // Fetch flow run
+          api.getGraphExecutionInfo(flowID, runID).then((execInfo) =>
+            setFlowRuns((flowRuns) => {
+              if (execInfo.length == 0) return flowRuns;
+
+              const flowRunIndex = flowRuns.findIndex((fr) => fr.id == runID);
+              const flowRun = flowRunFromNodeExecutionResults(execInfo);
+              if (flowRunIndex > -1) {
+                flowRuns.splice(flowRunIndex, 1, flowRun);
+              } else {
+                flowRuns.push(flowRun);
+              }
+              return [...flowRuns];
+            }),
+          );
+        }),
+      );
+    },
+    [api, flowRuns],
+  );
+
+  const fetchFlowsAndRuns = useCallback(() => {
+    api.listGraphs().then((flows) => {
+      setFlows(flows);
+      flows.map((flow) => refreshFlowRuns(flow.id));
+    });
+  }, [api, refreshFlowRuns]);
+
+  useEffect(() => fetchFlowsAndRuns(), [fetchFlowsAndRuns]);
   useEffect(() => {
     const intervalId = setInterval(
       () => flows.map((f) => refreshFlowRuns(f.id)),
       5000,
     );
     return () => clearInterval(intervalId);
-  }, []);
-
-  function fetchFlowsAndRuns() {
-    api.listGraphs().then((flows) => {
-      setFlows(flows);
-      flows.map((flow) => refreshFlowRuns(flow.id));
-    });
-  }
-
-  function refreshFlowRuns(flowID: string) {
-    // Fetch flow run IDs
-    api.listGraphRunIDs(flowID).then((runIDs) =>
-      runIDs.map((runID) => {
-        let run;
-        if (
-          (run = flowRuns.find((fr) => fr.id == runID)) &&
-          !["waiting", "running"].includes(run.status)
-        ) {
-          return;
-        }
-
-        // Fetch flow run
-        api.getGraphExecutionInfo(flowID, runID).then((execInfo) =>
-          setFlowRuns((flowRuns) => {
-            if (execInfo.length == 0) return flowRuns;
-
-            const flowRunIndex = flowRuns.findIndex((fr) => fr.id == runID);
-            const flowRun = flowRunFromNodeExecutionResults(execInfo);
-            if (flowRunIndex > -1) {
-              flowRuns.splice(flowRunIndex, 1, flowRun);
-            } else {
-              flowRuns.push(flowRun);
-            }
-            return [...flowRuns];
-          }),
-        );
-      }),
-    );
-  }
+  }, [flows, refreshFlowRuns]);
 
   const column1 = "md:col-span-2 xl:col-span-3 xxl:col-span-2";
   const column2 = "md:col-span-3 lg:col-span-2 xl:col-span-3 space-y-4";

rnd/autogpt_builder/src/components/CustomEdge.tsx

@@ -1,4 +1,4 @@
-import React, { useContext, useEffect, useState } from "react";
+import React, { useCallback, useContext, useEffect, useState } from "react";
 import {
   BaseEdge,
   EdgeLabelRenderer,
@@ -65,24 +65,27 @@ export function CustomEdge({
   const beadDiameter = 12;
   const deltaTime = 16;
 
-  function setTargetPositions(beads: Bead[]) {
-    const distanceBetween = Math.min(
-      (length - beadDiameter) / (beads.length + 1),
-      beadDiameter,
-    );
+  const setTargetPositions = useCallback(
+    (beads: Bead[]) => {
+      const distanceBetween = Math.min(
+        (length - beadDiameter) / (beads.length + 1),
+        beadDiameter,
+      );
 
-    return beads.map((bead, index) => {
-      const distanceFromEnd = beadDiameter * 1.35;
-      const targetPosition = distanceBetween * index + distanceFromEnd;
-      const t = getTForDistance(-targetPosition);
+      return beads.map((bead, index) => {
+        const distanceFromEnd = beadDiameter * 1.35;
+        const targetPosition = distanceBetween * index + distanceFromEnd;
+        const t = getTForDistance(-targetPosition);
 
-      return {
-        ...bead,
-        t: visualizeBeads === "animate" ? bead.t : t,
-        targetT: t,
-      } as Bead;
-    });
-  }
+        return {
+          ...bead,
+          t: visualizeBeads === "animate" ? bead.t : t,
+          targetT: t,
+        } as Bead;
+      });
+    },
+    [getTForDistance, length, visualizeBeads],
+  );
 
   useEffect(() => {
     if (data?.beadUp === 0 && data?.beadDown === 0) {
@@ -170,7 +173,7 @@ export function CustomEdge({
     }, deltaTime);
 
     return () => clearInterval(interval);
-  }, [data]);
+  }, [data, setTargetPositions, visualizeBeads]);
 
   const middle = getPointForT(0.5);
 

rnd/autogpt_builder/src/components/CustomNode.tsx

@@ -12,8 +12,10 @@ import InputModalComponent from "./InputModalComponent";
 import OutputModalComponent from "./OutputModalComponent";
 import {
   BlockIORootSchema,
+  BlockIOStringSubSchema,
   Category,
   NodeExecutionResult,
+  BlockUIType,
 } from "@/lib/autogpt-server-api/types";
 import { beautifyString, cn, setNestedProperty } from "@/lib/utils";
 import { Button } from "@/components/ui/button";
@@ -21,7 +23,10 @@ import { Switch } from "@/components/ui/switch";
 import { Copy, Trash2 } from "lucide-react";
 import { history } from "./history";
 import NodeHandle from "./NodeHandle";
-import { NodeGenericInputField } from "./node-input-components";
+import {
+  NodeGenericInputField,
+  NodeTextBoxInput,
+} from "./node-input-components";
 import SchemaTooltip from "./SchemaTooltip";
 import { getPrimaryCategoryColor } from "@/lib/utils";
 import { FlowContext } from "./Flow";
@@ -59,6 +64,7 @@ export type CustomNodeData = {
   backend_id?: string;
   errors?: { [key: string]: string };
   isOutputStatic?: boolean;
+  uiType: BlockUIType;
 };
 
 export type CustomNode = Node<CustomNodeData, "custom">;
@@ -96,7 +102,7 @@ export function CustomNode({ data, id, width, height }: NodeProps<CustomNode>) {
 
   useEffect(() => {
     setIsAnyModalOpen?.(isModalOpen || isOutputModalOpen);
-  }, [isModalOpen, isOutputModalOpen, data]);
+  }, [isModalOpen, isOutputModalOpen, data, setIsAnyModalOpen]);
 
   useEffect(() => {
     isInitialSetup.current = false;
@@ -118,8 +124,16 @@ export function CustomNode({ data, id, width, height }: NodeProps<CustomNode>) {
     setIsAdvancedOpen(checked);
   };
 
-  const generateOutputHandles = (schema: BlockIORootSchema) => {
-    if (!schema?.properties) return null;
+  const generateOutputHandles = (
+    schema: BlockIORootSchema,
+    nodeType: BlockUIType,
+  ) => {
+    if (
+      !schema?.properties ||
+      nodeType === BlockUIType.OUTPUT ||
+      nodeType === BlockUIType.NOTE
+    )
+      return null;
     const keys = Object.keys(schema.properties);
     return keys.map((key) => (
       <div key={key}>
@@ -133,6 +147,137 @@ export function CustomNode({ data, id, width, height }: NodeProps<CustomNode>) {
     ));
   };
 
+  const generateInputHandles = (
+    schema: BlockIORootSchema,
+    nodeType: BlockUIType,
+  ) => {
+    if (!schema?.properties) return null;
+    let keys = Object.entries(schema.properties);
+    switch (nodeType) {
+      case BlockUIType.INPUT:
+        // For INPUT blocks, dont include connection handles
+        return keys.map(([propKey, propSchema]) => {
+          const isRequired = data.inputSchema.required?.includes(propKey);
+          const isConnected = isHandleConnected(propKey);
+          const isAdvanced = propSchema.advanced;
+          return (
+            (isRequired || isAdvancedOpen || !isAdvanced) && (
+              <div key={propKey}>
+                <span className="text-m green -mb-1 text-gray-900">
+                  {propSchema.title || beautifyString(propKey)}
+                </span>
+                <div key={propKey} onMouseOver={() => {}}>
+                  {!isConnected && (
+                    <NodeGenericInputField
+                      className="mb-2 mt-1"
+                      propKey={propKey}
+                      propSchema={propSchema}
+                      currentValue={getValue(propKey)}
+                      connections={data.connections}
+                      handleInputChange={handleInputChange}
+                      handleInputClick={handleInputClick}
+                      errors={data.errors ?? {}}
+                      displayName={propSchema.title || beautifyString(propKey)}
+                    />
+                  )}
+                </div>
+              </div>
+            )
+          );
+        });
+
+      case BlockUIType.NOTE:
+        // For NOTE blocks, don't render any input handles
+        const [noteKey, noteSchema] = keys[0];
+        return (
+          <div key={noteKey}>
+            <NodeTextBoxInput
+              className=""
+              selfKey={noteKey}
+              schema={noteSchema as BlockIOStringSubSchema}
+              value={getValue(noteKey)}
+              handleInputChange={handleInputChange}
+              handleInputClick={handleInputClick}
+              error={data.errors?.[noteKey] ?? ""}
+              displayName={noteSchema.title || beautifyString(noteKey)}
+            />
+          </div>
+        );
+
+      case BlockUIType.OUTPUT:
+        // For OUTPUT blocks, only show the 'value' property
+        return keys.map(([propKey, propSchema]) => {
+          const isRequired = data.inputSchema.required?.includes(propKey);
+          const isConnected = isHandleConnected(propKey);
+          const isAdvanced = propSchema.advanced;
+          return (
+            (isRequired || isAdvancedOpen || !isAdvanced) && (
+              <div key={propKey} onMouseOver={() => {}}>
+                {propKey !== "value" ? (
+                  <span className="text-m green -mb-1 text-gray-900">
+                    {propSchema.title || beautifyString(propKey)}
+                  </span>
+                ) : (
+                  <NodeHandle
+                    keyName={propKey}
+                    isConnected={isConnected}
+                    isRequired={isRequired}
+                    schema={propSchema}
+                    side="left"
+                  />
+                )}
+                {!isConnected && (
+                  <NodeGenericInputField
+                    className="mb-2 mt-1"
+                    propKey={propKey}
+                    propSchema={propSchema}
+                    currentValue={getValue(propKey)}
+                    connections={data.connections}
+                    handleInputChange={handleInputChange}
+                    handleInputClick={handleInputClick}
+                    errors={data.errors ?? {}}
+                    displayName={propSchema.title || beautifyString(propKey)}
+                  />
+                )}
+              </div>
+            )
+          );
+        });
+
+      default:
+        return keys.map(([propKey, propSchema]) => {
+          const isRequired = data.inputSchema.required?.includes(propKey);
+          const isConnected = isHandleConnected(propKey);
+          const isAdvanced = propSchema.advanced;
+          return (
+            (isRequired || isAdvancedOpen || isConnected || !isAdvanced) && (
+              <div key={propKey} onMouseOver={() => {}}>
+                <NodeHandle
+                  keyName={propKey}
+                  isConnected={isConnected}
+                  isRequired={isRequired}
+                  schema={propSchema}
+                  side="left"
+                />
+                {!isConnected && (
+                  <NodeGenericInputField
+                    className="mb-2 mt-1"
+                    propKey={propKey}
+                    propSchema={propSchema}
+                    currentValue={getValue(propKey)}
+                    connections={data.connections}
+                    handleInputChange={handleInputChange}
+                    handleInputClick={handleInputClick}
+                    errors={data.errors ?? {}}
+                    displayName={propSchema.title || beautifyString(propKey)}
+                  />
+                )}
+              </div>
+            )
+          );
+        });
+    }
+  };
   const handleInputChange = (path: string, value: any) => {
     const keys = parseKeys(path);
     const newValues = JSON.parse(JSON.stringify(data.hardcodedValues));
@@ -378,13 +523,13 @@ export function CustomNode({ data, id, width, height }: NodeProps<CustomNode>) {
 
   return (
     <div
-      className={`${blockClasses} ${errorClass} ${statusClass}`}
+      className={`${data.uiType === BlockUIType.NOTE ? "w-[300px]" : "w-[500px]"} ${blockClasses} ${errorClass} ${statusClass} ${data.uiType === BlockUIType.NOTE ? "bg-yellow-100" : "bg-white"}`}
       onMouseEnter={handleHovered}
       onMouseLeave={handleMouseLeave}
       data-id={`custom-node-${id}`}
     >
       <div
-        className={`mb-2 p-3 ${getPrimaryCategoryColor(data.categories)} rounded-t-xl`}
+        className={`mb-2 p-3 ${data.uiType === BlockUIType.NOTE ? "bg-yellow-100" : getPrimaryCategoryColor(data.categories)} rounded-t-xl`}
       >
         <div className="flex items-center justify-between">
           <div className="font-roboto p-3 text-lg font-semibold">
@@ -417,53 +562,24 @@ export function CustomNode({ data, id, width, height }: NodeProps<CustomNode>) {
           )}
         </div>
       </div>
-      <div className="flex items-start justify-between gap-2 p-3">
+      {data.uiType !== BlockUIType.NOTE ? (
+        <div className="flex items-start justify-between p-3">
+          <div>
+            {data.inputSchema &&
+              generateInputHandles(data.inputSchema, data.uiType)}
+          </div>
+          <div className="flex-none">
+            {data.outputSchema &&
+              generateOutputHandles(data.outputSchema, data.uiType)}
+          </div>
+        </div>
+      ) : (
         <div>
           {data.inputSchema &&
-            Object.entries(data.inputSchema.properties).map(
-              ([propKey, propSchema]) => {
-                const isRequired = data.inputSchema.required?.includes(propKey);
-                const isConnected = isHandleConnected(propKey);
-                const isAdvanced = propSchema.advanced;
-                return (
-                  (isRequired ||
-                    isAdvancedOpen ||
-                    isConnected ||
-                    !isAdvanced) && (
-                    <div key={propKey} onMouseOver={() => {}}>
-                      <NodeHandle
-                        keyName={propKey}
-                        isConnected={isConnected}
-                        isRequired={isRequired}
-                        schema={propSchema}
-                        side="left"
-                      />
-                      {!isConnected && (
-                        <NodeGenericInputField
-                          className="mb-2 mt-1"
-                          propKey={propKey}
-                          propSchema={propSchema}
-                          currentValue={getValue(propKey)}
-                          connections={data.connections}
-                          handleInputChange={handleInputChange}
-                          handleInputClick={handleInputClick}
-                          errors={data.errors ?? {}}
-                          displayName={
-                            propSchema.title || beautifyString(propKey)
-                          }
-                        />
-                      )}
-                    </div>
-                  )
-                );
-              },
-            )}
+            generateInputHandles(data.inputSchema, data.uiType)}
         </div>
-        <div className="flex-none">
-          {data.outputSchema && generateOutputHandles(data.outputSchema)}
-        </div>
-      </div>
-      {isOutputOpen && (
+      )}
+      {isOutputOpen && data.uiType !== BlockUIType.NOTE && (
         <div
           data-id="latest-output"
           className="nodrag m-3 break-words rounded-md border-[1.5px] p-2"
@@ -486,25 +602,27 @@ export function CustomNode({ data, id, width, height }: NodeProps<CustomNode>) {
           )}
         </div>
       )}
-      <div className="mt-2.5 flex items-center pb-4 pl-4">
-        <Switch checked={isOutputOpen} onCheckedChange={toggleOutput} />
-        <span className="m-1 mr-4">Output</span>
-        {hasAdvancedFields && (
-          <>
-            <Switch onCheckedChange={toggleAdvancedSettings} />
-            <span className="m-1">Advanced</span>
-          </>
-        )}
-        {data.status && (
-          <Badge
-            variant="outline"
-            data-id={`badge-${id}-${data.status}`}
-            className={cn(data.status.toLowerCase(), "ml-auto mr-5")}
-          >
-            {data.status}
-          </Badge>
-        )}
-      </div>
+      {data.uiType !== BlockUIType.NOTE && (
+        <div className="mt-2.5 flex items-center pb-4 pl-4">
+          <Switch checked={isOutputOpen} onCheckedChange={toggleOutput} />
+          <span className="m-1 mr-4">Output</span>
+          {hasAdvancedFields && (
+            <>
+              <Switch onCheckedChange={toggleAdvancedSettings} />
+              <span className="m-1">Advanced</span>
+            </>
+          )}
+          {data.status && (
+            <Badge
+              variant="outline"
+              data-id={`badge-${id}-${data.status}`}
+              className={cn(data.status.toLowerCase(), "ml-auto mr-5")}
+            >
+              {data.status}
+            </Badge>
+          )}
+        </div>
+      )}
       <InputModalComponent
         title={activeKey ? `Enter ${beautifyString(activeKey)}` : undefined}
         isOpen={isModalOpen}

rnd/autogpt_builder/src/components/Flow.tsx

@@ -121,7 +121,7 @@ const FlowEditor: React.FC<{
         localStorage.setItem("shepherd-tour", "yes");
       }
     }
-  }, [availableNodes, tutorialStarted]);
+  }, [availableNodes, tutorialStarted, router, pathname]);
 
   useEffect(() => {
     const handleKeyDown = (event: KeyboardEvent) => {
@@ -256,7 +256,7 @@ const FlowEditor: React.FC<{
       }
 
       const edgeColor = getTypeColor(
-        getOutputType(connection.source!, connection.sourceHandle!),
+        getOutputType(nodes, connection.source!, connection.sourceHandle!),
       );
       const sourceNode = getNode(connection.source!);
       const newEdge: CustomEdge = {
@@ -295,6 +295,7 @@ const FlowEditor: React.FC<{
       addEdges,
       deleteElements,
       clearNodesStatusAndOutput,
+      nodes,
       edges,
       formatEdgeID,
       getOutputType,
@@ -377,7 +378,7 @@ const FlowEditor: React.FC<{
         clearNodesStatusAndOutput();
       }
     },
-    [setNodes, clearNodesStatusAndOutput],
+    [setNodes, clearNodesStatusAndOutput, setEdges],
   );
 
   const getNextNodeId = useCallback(() => {
@@ -416,6 +417,7 @@ const FlowEditor: React.FC<{
           isOutputOpen: false,
           block_id: blockId,
           isOutputStatic: nodeSchema.staticOutput,
+          uiType: nodeSchema.uiType,
         },
       };
 
@@ -434,7 +436,6 @@ const FlowEditor: React.FC<{
       nodeId,
       availableNodes,
       addNodes,
-      setNodes,
       deleteElements,
       clearNodesStatusAndOutput,
       x,

rnd/autogpt_builder/src/components/NavBar.stories.tsx

@@ -1,42 +0,0 @@
-import type { Meta, StoryObj } from '@storybook/react';
-import { NavBar } from './NavBar';
-import { UserProvider } from '@/context/UserContext'; // You'll need to create this context
-
-const meta: Meta<typeof NavBar> = {
-  title: 'Components/NavBar',
-  component: NavBar,
-  parameters: {
-    layout: 'fullscreen',
-  },
-  decorators: [
-    (Story) => (
-      <UserProvider>
-        <Story />
-      </UserProvider>
-    ),
-  ],
-};
-
-export default meta;
-type Story = StoryObj<typeof NavBar>;
-
-export const Default: Story = {
-  render: () => <NavBar />,
-};
-
-// You might need to mock the server-side functionality
-// for the story to work properly in Storybook
-export const LoggedOut: Story = {
-  parameters: {
-    userContext: { user: null, isAvailable: true },
-  },
-};
-
-export const LoggedIn: Story = {
-  parameters: {
-    userContext: { 
-      user: { id: '1', name: 'John Doe', email: 'john@example.com' },
-      isAvailable: true
-    },
-  },
-};

rnd/autogpt_builder/src/components/customnode.css

@@ -1,6 +1,5 @@
 .custom-node {
   color: #000000;
-  width: 500px;
   box-sizing: border-box;
   transition: border-color 0.3s ease-in-out;
 }

rnd/autogpt_builder/src/components/monitor/AgentFlowList.tsx

@@ -1,5 +1,5 @@
 import AutoGPTServerAPI, { GraphMeta } from "@/lib/autogpt-server-api";
-import React, { useEffect, useState } from "react";
+import React, { useEffect, useMemo, useState } from "react";
 import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
 import { Button } from "@/components/ui/button";
 import Link from "next/link";
@@ -45,10 +45,10 @@ export const AgentFlowList = ({
   className?: string;
 }) => {
   const [templates, setTemplates] = useState<GraphMeta[]>([]);
-  const api = new AutoGPTServerAPI();
+  const api = useMemo(() => new AutoGPTServerAPI(), []);
   useEffect(() => {
     api.listTemplates().then((templates) => setTemplates(templates));
-  }, []);
+  }, [api]);
 
   return (
     <Card className={className}>

rnd/autogpt_builder/src/components/monitor/FlowInfo.tsx

@@ -1,4 +1,4 @@
-import React, { useEffect, useState } from "react";
+import React, { useEffect, useMemo, useState } from "react";
 import AutoGPTServerAPI, {
   Graph,
   GraphMeta,
@@ -28,7 +28,7 @@ export const FlowInfo: React.FC<
     flowVersion?: number | "all";
   }
 > = ({ flow, flowRuns, flowVersion, ...props }) => {
-  const api = new AutoGPTServerAPI();
+  const api = useMemo(() => new AutoGPTServerAPI(), []);
 
   const [flowVersions, setFlowVersions] = useState<Graph[] | null>(null);
   const [selectedVersion, setSelectedFlowVersion] = useState(
@@ -41,7 +41,7 @@ export const FlowInfo: React.FC<
 
   useEffect(() => {
     api.getGraphAllVersions(flow.id).then((result) => setFlowVersions(result));
-  }, [flow.id]);
+  }, [flow.id, api]);
 
   return (
     <Card {...props}>

rnd/autogpt_builder/src/components/node-input-components.tsx

@@ -10,7 +10,7 @@ import {
   BlockIONumberSubSchema,
   BlockIOBooleanSubSchema,
 } from "@/lib/autogpt-server-api/types";
-import { FC, useEffect, useState } from "react";
+import React, { FC, useCallback, useEffect, useState } from "react";
 import { Button } from "./ui/button";
 import { Switch } from "./ui/switch";
 import {
@@ -296,7 +296,7 @@ const NodeKeyValueInput: FC<{
   className,
   displayName,
 }) => {
-  const getPairValues = () => {
+  const getPairValues = useCallback(() => {
     let defaultEntries = new Map<string, any>();
 
     connections
@@ -311,7 +311,7 @@ const NodeKeyValueInput: FC<{
     });
 
     return Array.from(defaultEntries, ([key, value]) => ({ key, value }));
-  };
+  }, [connections, entries, schema.default, selfKey]);
 
   const [keyValuePairs, setKeyValuePairs] = useState<
     { key: string; value: string | number | null }[]
@@ -319,7 +319,7 @@ const NodeKeyValueInput: FC<{
 
   useEffect(
     () => setKeyValuePairs(getPairValues()),
-    [connections, entries, schema.default],
+    [connections, entries, schema.default, getPairValues],
   );
 
   function updateKeyValuePairs(newPairs: typeof keyValuePairs) {
@@ -587,6 +587,52 @@ const NodeStringInput: FC<{
   );
 };
 
+export const NodeTextBoxInput: FC<{
+  selfKey: string;
+  schema: BlockIOStringSubSchema;
+  value?: string;
+  error?: string;
+  handleInputChange: NodeObjectInputTreeProps["handleInputChange"];
+  handleInputClick: NodeObjectInputTreeProps["handleInputClick"];
+  className?: string;
+  displayName: string;
+}> = ({
+  selfKey,
+  schema,
+  value = "",
+  error,
+  handleInputChange,
+  handleInputClick,
+  className,
+  displayName,
+}) => {
+  return (
+    <div className={className}>
+      <div
+        className="nodrag relative m-0 h-[200px] w-full bg-yellow-100 p-4"
+        onClick={schema.secret ? () => handleInputClick(selfKey) : undefined}
+      >
+        <textarea
+          id={selfKey}
+          value={schema.secret && value ? "********" : value}
+          readOnly={schema.secret}
+          placeholder={
+            schema?.placeholder || `Enter ${beautifyString(displayName)}`
+          }
+          onChange={(e) => handleInputChange(selfKey, e.target.value)}
+          onBlur={(e) => handleInputChange(selfKey, e.target.value)}
+          className="h-full w-full resize-none overflow-hidden border-none bg-transparent text-lg text-black outline-none"
+          style={{
+            fontSize: "min(1em, 16px)",
+            lineHeight: "1.2",
+          }}
+        />
+      </div>
+      {error && <span className="error-message">{error}</span>}
+    </div>
+  );
+};
+
 const NodeNumberInput: FC<{
   selfKey: string;
   schema: BlockIONumberSubSchema;

rnd/autogpt_builder/src/components/ui/input.tsx

@@ -19,7 +19,7 @@ const Input = React.forwardRef<HTMLInputElement, InputProps>(
       ) {
         ref.current.value = value;
       }
-    }, [value, type]);
+    }, [value, type, ref]);
     return (
       <input
         type={type}

rnd/autogpt_builder/src/hooks/useAgentGraph.ts

@@ -61,8 +61,10 @@ export default function useAgentGraph(
   const [nodes, setNodes] = useState<CustomNode[]>([]);
   const [edges, setEdges] = useState<CustomEdge[]>([]);
 
-  const apiUrl = process.env.NEXT_PUBLIC_AGPT_SERVER_URL!;
-  const api = useMemo(() => new AutoGPTServerAPI(apiUrl), [apiUrl]);
+  const api = useMemo(
+    () => new AutoGPTServerAPI(process.env.NEXT_PUBLIC_AGPT_SERVER_URL!),
+    [],
+  );
 
   // Connect to WebSocket
   useEffect(() => {
@@ -89,16 +91,227 @@ export default function useAgentGraph(
       .getBlocks()
       .then((blocks) => setAvailableNodes(blocks))
       .catch();
+  }, [api]);
+
+  //TODO to utils? repeated in Flow
+  const formatEdgeID = useCallback((conn: Link | Connection): string => {
+    if ("sink_id" in conn) {
+      return `${conn.source_id}_${conn.source_name}_${conn.sink_id}_${conn.sink_name}`;
+    } else {
+      return `${conn.source}_${conn.sourceHandle}_${conn.target}_${conn.targetHandle}`;
+    }
   }, []);
 
+  const getOutputType = useCallback(
+    (nodes: CustomNode[], nodeId: string, handleId: string) => {
+      const node = nodes.find((n) => n.id === nodeId);
+      if (!node) return "unknown";
+
+      const outputSchema = node.data.outputSchema;
+      if (!outputSchema) return "unknown";
+
+      const outputHandle = outputSchema.properties[handleId];
+      if (!("type" in outputHandle)) return "unknown";
+      return outputHandle.type;
+    },
+    [],
+  );
+
   // Load existing graph
+  const loadGraph = useCallback(
+    (graph: Graph) => {
+      setSavedAgent(graph);
+      setAgentName(graph.name);
+      setAgentDescription(graph.description);
+
+      setNodes(() => {
+        const newNodes = graph.nodes.map((node) => {
+          const block = availableNodes.find(
+            (block) => block.id === node.block_id,
+          )!;
+          const newNode: CustomNode = {
+            id: node.id,
+            type: "custom",
+            position: {
+              x: node.metadata.position.x,
+              y: node.metadata.position.y,
+            },
+            data: {
+              block_id: block.id,
+              blockType: block.name,
+              categories: block.categories,
+              description: block.description,
+              title: `${block.name} ${node.id}`,
+              inputSchema: block.inputSchema,
+              outputSchema: block.outputSchema,
+              hardcodedValues: node.input_default,
+              connections: graph.links
+                .filter((l) => [l.source_id, l.sink_id].includes(node.id))
+                .map((link) => ({
+                  edge_id: formatEdgeID(link),
+                  source: link.source_id,
+                  sourceHandle: link.source_name,
+                  target: link.sink_id,
+                  targetHandle: link.sink_name,
+                })),
+              isOutputOpen: false,
+            },
+          };
+          return newNode;
+        });
+        setEdges((_) =>
+          graph.links.map((link) => ({
+            id: formatEdgeID(link),
+            type: "custom",
+            data: {
+              edgeColor: getTypeColor(
+                getOutputType(newNodes, link.source_id, link.source_name!),
+              ),
+              sourcePos: newNodes.find((node) => node.id === link.source_id)
+                ?.position,
+              isStatic: link.is_static,
+              beadUp: 0,
+              beadDown: 0,
+              beadData: [],
+            },
+            markerEnd: {
+              type: MarkerType.ArrowClosed,
+              strokeWidth: 2,
+              color: getTypeColor(
+                getOutputType(newNodes, link.source_id, link.source_name!),
+              ),
+            },
+            source: link.source_id,
+            target: link.sink_id,
+            sourceHandle: link.source_name || undefined,
+            targetHandle: link.sink_name || undefined,
+          })),
+        );
+        return newNodes;
+      });
+    },
+    [availableNodes, formatEdgeID, getOutputType],
+  );
+
+  const getFrontendId = useCallback(
+    (backendId: string, nodes: CustomNode[]) => {
+      const node = nodes.find((node) => node.data.backend_id === backendId);
+      return node?.id;
+    },
+    [],
+  );
+
+  const updateEdgeBeads = useCallback(
+    (executionData: NodeExecutionResult) => {
+      setEdges((edges) => {
+        return edges.map((e) => {
+          const edge = { ...e, data: { ...e.data } } as CustomEdge;
+
+          if (executionData.status === "COMPLETED") {
+            // Produce output beads
+            for (let key in executionData.output_data) {
+              if (
+                edge.source !== getFrontendId(executionData.node_id, nodes) ||
+                edge.sourceHandle !== key
+              ) {
+                continue;
+              }
+              edge.data!.beadUp = (edge.data!.beadUp ?? 0) + 1;
+              // For static edges beadDown is always one less than beadUp
+              // Because there's no queueing and one bead is always at the connection point
+              if (edge.data?.isStatic) {
+                edge.data!.beadDown = (edge.data!.beadUp ?? 0) - 1;
+                edge.data!.beadData = edge.data!.beadData!.slice(0, -1);
+                continue;
+              }
+              //todo kcze this assumes output at key is always array with one element
+              edge.data!.beadData = [
+                executionData.output_data[key][0],
+                ...edge.data!.beadData!,
+              ];
+            }
+          } else if (executionData.status === "RUNNING") {
+            // Consume input beads
+            for (let key in executionData.input_data) {
+              if (
+                edge.target !== getFrontendId(executionData.node_id, nodes) ||
+                edge.targetHandle !== key
+              ) {
+                continue;
+              }
+              // Skip decreasing bead count if edge doesn't match or if it's static
+              if (
+                edge.data!.beadData![edge.data!.beadData!.length - 1] !==
+                  executionData.input_data[key] ||
+                edge.data?.isStatic
+              ) {
+                continue;
+              }
+              edge.data!.beadDown = (edge.data!.beadDown ?? 0) + 1;
+              edge.data!.beadData = edge.data!.beadData!.slice(0, -1);
+            }
+          }
+          return edge;
+        });
+      });
+    },
+    [getFrontendId, nodes],
+  );
+
+  const updateNodesWithExecutionData = useCallback(
+    (executionData: NodeExecutionResult) => {
+      if (passDataToBeads) {
+        updateEdgeBeads(executionData);
+      }
+      setNodes((nodes) => {
+        const nodeId = nodes.find(
+          (node) => node.data.backend_id === executionData.node_id,
+        )?.id;
+        if (!nodeId) {
+          console.error(
+            "Node not found for execution data:",
+            executionData,
+            "This shouldn't happen and means that the frontend and backend are out of sync.",
+          );
+          return nodes;
+        }
+        return nodes.map((node) =>
+          node.id === nodeId
+            ? {
+                ...node,
+                data: {
+                  ...node.data,
+                  status: executionData.status,
+                  executionResults:
+                    Object.keys(executionData.output_data).length > 0
+                      ? [
+                          ...(node.data.executionResults || []),
+                          {
+                            execId: executionData.node_exec_id,
+                            data: executionData.output_data,
+                          },
+                        ]
+                      : node.data.executionResults,
+                  isOutputOpen: true,
+                },
+              }
+            : node,
+        );
+      });
+    },
+    [passDataToBeads, updateEdgeBeads],
+  );
+
   useEffect(() => {
     if (!flowID || availableNodes.length == 0) return;
 
-    (template ? api.getTemplate(flowID) : api.getGraph(flowID)).then((graph) =>
-      loadGraph(graph),
+    (template ? api.getTemplate(flowID) : api.getGraph(flowID)).then(
+      (graph) => {
+        console.log("Loading graph");
+        loadGraph(graph);
+      },
     );
-  }, [flowID, template, availableNodes]);
+  }, [flowID, template, availableNodes, api, loadGraph]);
 
   // Update nodes with execution data
   useEffect(() => {
@@ -119,7 +332,68 @@ export default function useAgentGraph(
       });
       return [];
     });
-  }, [updateQueue, nodesSyncedWithSavedAgent]);
+  }, [updateQueue, nodesSyncedWithSavedAgent, updateNodesWithExecutionData]);
+
+  const validateNodes = useCallback((): boolean => {
+    let isValid = true;
+
+    nodes.forEach((node) => {
+      const validate = ajv.compile(node.data.inputSchema);
+      const errors = {} as { [key: string]: string };
+
+      // Validate values against schema using AJV
+      const valid = validate(node.data.hardcodedValues);
+      if (!valid) {
+        // Populate errors if validation fails
+        validate.errors?.forEach((error) => {
+          // Skip error if there's an edge connected
+          const path =
+            "dataPath" in error
+              ? (error.dataPath as string)
+              : error.instancePath;
+          const handle = path.split(/[\/.]/)[0];
+          if (
+            node.data.connections.some(
+              (conn) => conn.target === node.id || conn.targetHandle === handle,
+            )
+          ) {
+            return;
+          }
+          console.warn("Error", error);
+          isValid = false;
+          if (path && error.message) {
+            const key = path.slice(1);
+            console.log("Error", key, error.message);
+            setNestedProperty(
+              errors,
+              key,
+              error.message[0].toUpperCase() + error.message.slice(1),
+            );
+          } else if (error.keyword === "required") {
+            const key = error.params.missingProperty;
+            setNestedProperty(errors, key, "This field is required");
+          }
+        });
+      }
+      // Set errors
+      setNodes((nodes) => {
+        return nodes.map((n) => {
+          if (n.id === node.id) {
+            return {
+              ...n,
+              data: {
+                ...n.data,
+                errors,
+              },
+            };
+          }
+          return n;
+        });
+      });
+    });
+
+    return isValid;
+  }, [nodes]);
 
   // Handle user requests
   useEffect(() => {
@@ -224,7 +498,13 @@ export default function useAgentGraph(
         .stopGraphExecution(savedAgent.id, saveRunRequest.activeExecutionID)
         .then(() => setSaveRunRequest({ request: "none", state: "none" }));
     }
-  }, [saveRunRequest, savedAgent, nodesSyncedWithSavedAgent]);
+  }, [
+    api,
+    saveRunRequest,
+    savedAgent,
+    nodesSyncedWithSavedAgent,
+    validateNodes,
+  ]);
 
   // Check if node ids are synced with saved agent
   useEffect(() => {
@@ -241,275 +521,6 @@ export default function useAgentGraph(
     setNodesSyncedWithSavedAgent(oneNodeSynced);
   }, [savedAgent, nodes]);
 
-  const validateNodes = useCallback((): boolean => {
-    let isValid = true;
-
-    nodes.forEach((node) => {
-      const validate = ajv.compile(node.data.inputSchema);
-      const errors = {} as { [key: string]: string };
-
-      // Validate values against schema using AJV
-      const valid = validate(node.data.hardcodedValues);
-      if (!valid) {
-        // Populate errors if validation fails
-        validate.errors?.forEach((error) => {
-          // Skip error if there's an edge connected
-          const path =
-            "dataPath" in error
-              ? (error.dataPath as string)
-              : error.instancePath;
-          const handle = path.split(/[\/.]/)[0];
-          if (
-            node.data.connections.some(
-              (conn) => conn.target === node.id || conn.targetHandle === handle,
-            )
-          ) {
-            return;
-          }
-          console.warn("Error", error);
-          isValid = false;
-          if (path && error.message) {
-            const key = path.slice(1);
-            console.log("Error", key, error.message);
-            setNestedProperty(
-              errors,
-              key,
-              error.message[0].toUpperCase() + error.message.slice(1),
-            );
-          } else if (error.keyword === "required") {
-            const key = error.params.missingProperty;
-            setNestedProperty(errors, key, "This field is required");
-          }
-        });
-      }
-      // Set errors
-      setNodes((nodes) => {
-        return nodes.map((n) => {
-          if (n.id === node.id) {
-            return {
-              ...n,
-              data: {
-                ...n.data,
-                errors,
-              },
-            };
-          }
-          return n;
-        });
-      });
-    });
-
-    return isValid;
-  }, [nodes]);
-
-  const getFrontendId = useCallback(
-    (backendId: string, nodes: CustomNode[]) => {
-      const node = nodes.find((node) => node.data.backend_id === backendId);
-      return node?.id;
-    },
-    [],
-  );
-
-  const updateEdgeBeads = useCallback(
-    (executionData: NodeExecutionResult) => {
-      setEdges((edges) => {
-        return edges.map((e) => {
-          const edge = { ...e, data: { ...e.data } } as CustomEdge;
-
-          if (executionData.status === "COMPLETED") {
-            // Produce output beads
-            for (let key in executionData.output_data) {
-              if (
-                edge.source !== getFrontendId(executionData.node_id, nodes) ||
-                edge.sourceHandle !== key
-              ) {
-                continue;
-              }
-              edge.data!.beadUp = (edge.data!.beadUp ?? 0) + 1;
-              // For static edges beadDown is always one less than beadUp
-              // Because there's no queueing and one bead is always at the connection point
-              if (edge.data?.isStatic) {
-                edge.data!.beadDown = (edge.data!.beadUp ?? 0) - 1;
-                edge.data!.beadData = edge.data!.beadData!.slice(0, -1);
-                continue;
-              }
-              //todo kcze this assumes output at key is always array with one element
-              edge.data!.beadData = [
-                executionData.output_data[key][0],
-                ...edge.data!.beadData!,
-              ];
-            }
-          } else if (executionData.status === "RUNNING") {
-            // Consume input beads
-            for (let key in executionData.input_data) {
-              if (
-                edge.target !== getFrontendId(executionData.node_id, nodes) ||
-                edge.targetHandle !== key
-              ) {
-                continue;
-              }
-              // Skip decreasing bead count if edge doesn't match or if it's static
-              if (
-                edge.data!.beadData![edge.data!.beadData!.length - 1] !==
-                  executionData.input_data[key] ||
-                edge.data?.isStatic
-              ) {
-                continue;
-              }
-              edge.data!.beadDown = (edge.data!.beadDown ?? 0) + 1;
-              edge.data!.beadData = edge.data!.beadData!.slice(0, -1);
-            }
-          }
-          return edge;
-        });
-      });
-    },
-    [edges],
-  );
-
-  const updateNodesWithExecutionData = useCallback(
-    (executionData: NodeExecutionResult) => {
-      if (passDataToBeads) {
-        updateEdgeBeads(executionData);
-      }
-      setNodes((nodes) => {
-        const nodeId = nodes.find(
-          (node) => node.data.backend_id === executionData.node_id,
-        )?.id;
-        if (!nodeId) {
-          console.error(
-            "Node not found for execution data:",
-            executionData,
-            "This shouldn't happen and means that the frontend and backend are out of sync.",
-          );
-          return nodes;
-        }
-        return nodes.map((node) =>
-          node.id === nodeId
-            ? {
-                ...node,
-                data: {
-                  ...node.data,
-                  status: executionData.status,
-                  executionResults:
-                    Object.keys(executionData.output_data).length > 0
-                      ? [
-                          ...(node.data.executionResults || []),
-                          {
-                            execId: executionData.node_exec_id,
-                            data: executionData.output_data,
-                          },
-                        ]
-                      : node.data.executionResults,
-                  isOutputOpen: true,
-                },
-              }
-            : node,
-        );
-      });
-    },
-    [nodes],
-  );
-
-  //TODO to utils? repeated in Flow
-  const formatEdgeID = useCallback((conn: Link | Connection): string => {
-    if ("sink_id" in conn) {
-      return `${conn.source_id}_${conn.source_name}_${conn.sink_id}_${conn.sink_name}`;
-    } else {
-      return `${conn.source}_${conn.sourceHandle}_${conn.target}_${conn.targetHandle}`;
-    }
-  }, []);
-
-  const getOutputType = useCallback(
-    (nodeId: string, handleId: string) => {
-      const node = nodes.find((n) => n.id === nodeId);
-      if (!node) return "unknown";
-
-      const outputSchema = node.data.outputSchema;
-      if (!outputSchema) return "unknown";
-
-      const outputHandle = outputSchema.properties[handleId];
-      if (!("type" in outputHandle)) return "unknown";
-      return outputHandle.type;
-    },
-    [nodes],
-  );
-
-  const loadGraph = useCallback(
-    (graph: Graph) => {
-      setSavedAgent(graph);
-      setAgentName(graph.name);
-      setAgentDescription(graph.description);
-
-      setNodes(() => {
-        const newNodes = graph.nodes.map((node) => {
-          const block = availableNodes.find(
-            (block) => block.id === node.block_id,
-          )!;
-          const newNode: CustomNode = {
-            id: node.id,
-            type: "custom",
-            position: {
-              x: node.metadata.position.x,
-              y: node.metadata.position.y,
-            },
-            data: {
-              block_id: block.id,
-              blockType: block.name,
-              categories: block.categories,
-              description: block.description,
-              title: `${block.name} ${node.id}`,
-              inputSchema: block.inputSchema,
-              outputSchema: block.outputSchema,
-              hardcodedValues: node.input_default,
-              connections: graph.links
-                .filter((l) => [l.source_id, l.sink_id].includes(node.id))
-                .map((link) => ({
-                  edge_id: formatEdgeID(link),
-                  source: link.source_id,
-                  sourceHandle: link.source_name,
-                  target: link.sink_id,
-                  targetHandle: link.sink_name,
-                })),
-              isOutputOpen: false,
-            },
-          };
-          return newNode;
-        });
-        setEdges((_) =>
-          graph.links.map((link) => ({
-            id: formatEdgeID(link),
-            type: "custom",
-            data: {
-              edgeColor: getTypeColor(
-                getOutputType(link.source_id, link.source_name!),
-              ),
-              sourcePos: nodes.find((node) => node.id === link.source_id)
-                ?.position,
-              isStatic: link.is_static,
-              beadUp: 0,
-              beadDown: 0,
-              beadData: [],
-            },
-            markerEnd: {
-              type: MarkerType.ArrowClosed,
-              strokeWidth: 2,
-              color: getTypeColor(
-                getOutputType(link.source_id, link.source_name!),
-              ),
-            },
-            source: link.source_id,
-            target: link.sink_id,
-            sourceHandle: link.source_name || undefined,
-            targetHandle: link.sink_name || undefined,
-          })),
-        );
-        return newNodes;
-      });
-    },
-    [availableNodes],
-  );
-
   const prepareNodeInputData = useCallback(
     (node: CustomNode) => {
       console.debug(
@@ -696,7 +707,15 @@ export default function useAgentGraph(
         }));
       });
     },
-    [nodes, edges, savedAgent],
+    [
+      api,
+      nodes,
+      edges,
+      savedAgent,
+      agentName,
+      agentDescription,
+      prepareNodeInputData,
+    ],
   );
 
   const requestSave = useCallback(

rnd/autogpt_builder/src/hooks/useBezierPath.ts

@@ -84,12 +84,12 @@ export function useBezierPath(
 
       return length;
     },
-    [path],
+    [getPointForT],
   );
 
   const length = useMemo(() => {
     return getArcLength(1);
-  }, [path]);
+  }, [getArcLength]);
 
   const getBezierDerivative = useCallback(
     (t: number) => {
@@ -131,7 +131,7 @@ export function useBezierPath(
 
       return t;
     },
-    [path],
+    [getArcLength, getBezierDerivative, length],
   );
 
   const getPointAtDistance = useCallback(
@@ -143,7 +143,7 @@ export function useBezierPath(
       const t = getTForDistance(distance);
       return getPointForT(t);
     },
-    [path],
+    [getTForDistance, getPointForT, length],
   );
 
   return {

rnd/autogpt_builder/src/lib/autogpt-server-api/types.ts

@@ -13,6 +13,7 @@ export type Block = {
   inputSchema: BlockIORootSchema;
   outputSchema: BlockIORootSchema;
   staticOutput: boolean;
+  uiType: BlockUIType;
 };
 
 export type BlockIORootSchema = {
@@ -182,3 +183,10 @@ export type User = {
   id: string;
   email: string;
 };
+
+export enum BlockUIType {
+  STANDARD = "Standard",
+  INPUT = "Input",
+  OUTPUT = "Output",
+  NOTE = "Note",
+}

rnd/autogpt_builder/src/lib/marketplace-api/client.ts

@@ -46,7 +46,7 @@ export default class MarketplaceAPI {
     pageSize: number = 10,
   ): Promise<AgentListResponse> {
     return this._get(
-      `/top-downloads/agents?page=${page}&page_size=${pageSize}`,
+      `agents/top-downloads?page=${page}&page_size=${pageSize}`,
     );
   }
 
@@ -54,7 +54,7 @@ export default class MarketplaceAPI {
     page: number = 1,
     pageSize: number = 10,
   ): Promise<AgentListResponse> {
-    return this._get(`/featured/agents?page=${page}&page_size=${pageSize}`);
+    return this._get(`/agents/featured?page=${page}&page_size=${pageSize}`);
   }
 
   async searchAgents(

rnd/autogpt_builder/src/stories/Button.stories.ts

@@ -1,59 +0,0 @@
-import type { Meta, StoryObj } from '@storybook/react';
-import { fn } from '@storybook/test';
-import { Button } from './Button';
-
-// More on how to set up stories at: https://storybook.js.org/docs/writing-stories#default-export
-const meta = {
-  title: 'Example/Button',
-  component: Button,
-  parameters: {
-    // Optional parameter to center the component in the Canvas. More info: https://storybook.js.org/docs/configure/story-layout
-    layout: 'centered',
-  },
-  // This component will have an automatically generated Autodocs entry: https://storybook.js.org/docs/writing-docs/autodocs
-  tags: ['autodocs'],
-  // More on argTypes: https://storybook.js.org/docs/api/argtypes
-  argTypes: {
-    backgroundColor: { control: 'color' },
-  },
-  // Use `fn` to spy on the onClick arg, which will appear in the actions panel once invoked: https://storybook.js.org/docs/essentials/actions#action-args
-  args: { onClick: fn() },
-} satisfies Meta<typeof Button>;
-
-export default meta;
-type Story = StoryObj<typeof meta>;
-
-// More on writing stories with args: https://storybook.js.org/docs/writing-stories/args
-export const Primary: Story = {
-  args: {
-    primary: true,
-    label: 'Button',
-  },
-};
-
-export const Secondary: Story = {
-  args: {
-    label: 'Button',
-  },
-};
-
-export const Large: Story = {
-  args: {
-    size: 'large',
-    label: 'Button',
-  },
-};
-
-export const Small: Story = {
-  args: {
-    size: 'small',
-    label: 'Button',
-  },
-};
-
-export const MySampleStory: Story = {
-  args: {
-    primary: false,
-    label: "Button"
-  }
-};

rnd/autogpt_builder/src/stories/Button.tsx

@@ -1,52 +0,0 @@
-import React from 'react';
-import './button.css';
-
-export interface ButtonProps {
-  /**
-   * Is this the principal call to action on the page?
-   */
-  primary?: boolean;
-  /**
-   * What background color to use
-   */
-  backgroundColor?: string;
-  /**
-   * How large should the button be?
-   */
-  size?: 'small' | 'medium' | 'large';
-  /**
-   * Button contents
-   */
-  label: string;
-  /**
-   * Optional click handler
-   */
-  onClick?: () => void;
-}
-
-/**
- * Primary UI component for user interaction
- */
-export const Button = ({
-  primary = false,
-  size = 'medium',
-  backgroundColor,
-  label,
-  ...props
-}: ButtonProps) => {
-  const mode = primary ? 'storybook-button--primary' : 'storybook-button--secondary';
-  return (
-    <button
-      type="button"
-      className={['storybook-button', `storybook-button--${size}`, mode].join(' ')}
-      {...props}
-    >
-      {label}
-      <style jsx>{`
-        button {
-          background-color: ${backgroundColor};
-        }
-      `}</style>
-    </button>
-  );
-};

rnd/autogpt_builder/src/stories/Configure.mdx

@@ -1,446 +0,0 @@
-import { Meta } from "@storybook/blocks";
-import Image from "next/image";
-
-import Github from "./assets/github.svg";
-import Discord from "./assets/discord.svg";
-import Youtube from "./assets/youtube.svg";
-import Tutorials from "./assets/tutorials.svg";
-import Styling from "./assets/styling.png";
-import Context from "./assets/context.png";
-import Assets from "./assets/assets.png";
-import Docs from "./assets/docs.png";
-import Share from "./assets/share.png";
-import FigmaPlugin from "./assets/figma-plugin.png";
-import Testing from "./assets/testing.png";
-import Accessibility from "./assets/accessibility.png";
-import Theming from "./assets/theming.png";
-import AddonLibrary from "./assets/addon-library.png";
-
-export const RightArrow = () => <svg 
-    viewBox="0 0 14 14" 
-    width="8px" 
-    height="14px" 
-    style={{ 
-      marginLeft: '4px',
-      display: 'inline-block',
-      shapeRendering: 'inherit',
-      verticalAlign: 'middle',
-      fill: 'currentColor',
-      'path fill': 'currentColor'
-    }}
->
-  <path d="m11.1 7.35-5.5 5.5a.5.5 0 0 1-.7-.7L10.04 7 4.9 1.85a.5.5 0 1 1 .7-.7l5.5 5.5c.2.2.2.5 0 .7Z" />
-</svg>
-
-<Meta title="Configure your project" />
-
-<div className="sb-container">
-  <div className='sb-section-title'>
-    # Configure your project
-
-    Because Storybook works separately from your app, you'll need to configure it for your specific stack and setup. Below, explore guides for configuring Storybook with popular frameworks and tools. If you get stuck, learn how you can ask for help from our community.
-  </div>
-  <div className="sb-section">
-    <div className="sb-section-item">
-      <Image
-        src={Styling}
-        alt="A wall of logos representing different styling technologies"
-        width={0}
-        height={0}
-        style={{ width: '100%', height: 'auto' }}
-      />
-      <h4 className="sb-section-item-heading">Add styling and CSS</h4>
-      <p className="sb-section-item-paragraph">Like with web applications, there are many ways to include CSS within Storybook. Learn more about setting up styling within Storybook.</p>
-      <a
-        href="https://storybook.js.org/docs/configure/styling-and-css/?renderer=react"
-        target="_blank"
-      >Learn more<RightArrow /></a>
-    </div>
-    <div className="sb-section-item">
-      <Image 
-        width={0}
-        height={0}
-        style={{ width: '100%', height: 'auto' }}
-        src={Context}
-        alt="An abstraction representing the composition of data for a component"
-      />
-      <h4 className="sb-section-item-heading">Provide context and mocking</h4>
-      <p className="sb-section-item-paragraph">Often when a story doesn't render, it's because your component is expecting a specific environment or context (like a theme provider) to be available.</p>
-      <a
-        href="https://storybook.js.org/docs/writing-stories/decorators/?renderer=react#context-for-mocking"
-        target="_blank"
-      >Learn more<RightArrow /></a>
-    </div>
-    <div className="sb-section-item">
-      <Image 
-        width={0}
-        height={0}
-        style={{ width: '100%', height: 'auto' }} 
-        src={Assets} 
-        alt="A representation of typography and image assets" 
-      />
-      <div>
-        <h4 className="sb-section-item-heading">Load assets and resources</h4>
-        <p className="sb-section-item-paragraph">To link static files (like fonts) to your projects and stories, use the
-        `staticDirs` configuration option to specify folders to load when
-        starting Storybook.</p>
-        <a
-          href="https://storybook.js.org/docs/configure/images-and-assets/?renderer=react"
-          target="_blank"
-        >Learn more<RightArrow /></a>
-      </div>
-    </div>
-  </div>
-</div>
-<div className="sb-container">
-  <div className='sb-section-title'>
-    # Do more with Storybook
-
-    Now that you know the basics, let's explore other parts of Storybook that will improve your experience. This list is just to get you started. You can customise Storybook in many ways to fit your needs.
-  </div>
-
-  <div className="sb-section">
-    <div className="sb-features-grid">
-      <div className="sb-grid-item">
-        <Image 
-          width={0}
-          height={0}
-          style={{ width: '100%', height: 'auto' }} 
-          src={Docs} 
-          alt="A screenshot showing the autodocs tag being set, pointing a docs page being generated" 
-        />
-        <h4 className="sb-section-item-heading">Autodocs</h4>
-        <p className="sb-section-item-paragraph">Auto-generate living,
-          interactive reference documentation from your components and stories.</p>
-        <a
-          href="https://storybook.js.org/docs/writing-docs/autodocs/?renderer=react"
-          target="_blank"
-        >Learn more<RightArrow /></a>
-      </div>
-      <div className="sb-grid-item">
-        <Image 
-          width={0}
-          height={0}
-          style={{ width: '100%', height: 'auto' }} 
-          src={Share} 
-          alt="A browser window showing a Storybook being published to a chromatic.com URL" 
-        />
-        <h4 className="sb-section-item-heading">Publish to Chromatic</h4>
-        <p className="sb-section-item-paragraph">Publish your Storybook to review and collaborate with your entire team.</p>
-        <a
-          href="https://storybook.js.org/docs/sharing/publish-storybook/?renderer=react#publish-storybook-with-chromatic"
-          target="_blank"
-        >Learn more<RightArrow /></a>
-      </div>
-      <div className="sb-grid-item">
-        <Image 
-          width={0}
-          height={0}
-          style={{ width: '100%', height: 'auto' }} 
-          src={FigmaPlugin} 
-          alt="Windows showing the Storybook plugin in Figma" 
-        />
-        <h4 className="sb-section-item-heading">Figma Plugin</h4>
-        <p className="sb-section-item-paragraph">Embed your stories into Figma to cross-reference the design and live
-          implementation in one place.</p>
-        <a
-          href="https://storybook.js.org/docs/sharing/design-integrations/?renderer=react#embed-storybook-in-figma-with-the-plugin"
-          target="_blank"
-        >Learn more<RightArrow /></a>
-      </div>
-      <div className="sb-grid-item">
-        <Image 
-          width={0}
-          height={0}
-          style={{ width: '100%', height: 'auto' }} 
-          src={Testing} 
-          alt="Screenshot of tests passing and failing" 
-        />
-        <h4 className="sb-section-item-heading">Testing</h4>
-        <p className="sb-section-item-paragraph">Use stories to test a component in all its variations, no matter how
-          complex.</p>
-        <a
-          href="https://storybook.js.org/docs/writing-tests/?renderer=react"
-          target="_blank"
-        >Learn more<RightArrow /></a>
-      </div>
-      <div className="sb-grid-item">
-        <Image 
-          width={0}
-          height={0}
-          style={{ width: '100%', height: 'auto' }} 
-          src={Accessibility} 
-          alt="Screenshot of accessibility tests passing and failing" 
-        />
-        <h4 className="sb-section-item-heading">Accessibility</h4>
-        <p className="sb-section-item-paragraph">Automatically test your components for a11y issues as you develop.</p>
-        <a
-          href="https://storybook.js.org/docs/writing-tests/accessibility-testing/?renderer=react"
-          target="_blank"
-        >Learn more<RightArrow /></a>
-      </div>
-      <div className="sb-grid-item">
-        <Image 
-          width={0}
-          height={0}
-          style={{ width: '100%', height: 'auto' }} 
-          src={Theming} 
-          alt="Screenshot of Storybook in light and dark mode" 
-        />
-        <h4 className="sb-section-item-heading">Theming</h4>
-        <p className="sb-section-item-paragraph">Theme Storybook's UI to personalize it to your project.</p>
-        <a
-          href="https://storybook.js.org/docs/configure/theming/?renderer=react"
-          target="_blank"
-        >Learn more<RightArrow /></a>
-      </div>
-    </div>
-  </div>
-</div>
-<div className='sb-addon'>
-  <div className='sb-addon-text'>
-    <h4>Addons</h4>
-    <p className="sb-section-item-paragraph">Integrate your tools with Storybook to connect workflows.</p>
-    <a
-        href="https://storybook.js.org/addons/"
-        target="_blank"
-      >Discover all addons<RightArrow /></a>
-  </div>
-  <div className='sb-addon-img'>
-    <Image 
-      width={650}
-      height={347}
-      src={AddonLibrary} 
-      alt="Integrate your tools with Storybook to connect workflows." 
-    />
-  </div>
-</div>
-
-<div className="sb-section sb-socials">
-    <div className="sb-section-item">
-      <Image 
-        width={32}
-        height={32}
-        layout="fixed"
-        src={Github} 
-        alt="Github logo" 
-        className="sb-explore-image"
-      />
-      Join our contributors building the future of UI development.
-
-      <a
-        href="https://github.com/storybookjs/storybook"
-        target="_blank"
-      >Star on GitHub<RightArrow /></a>
-    </div>
-    <div className="sb-section-item">
-      <Image 
-        width={33}
-        height={32}
-        layout="fixed"
-        src={Discord} 
-        alt="Discord logo" 
-        className="sb-explore-image"
-      />
-      <div>
-        Get support and chat with frontend developers.
-
-        <a
-          href="https://discord.gg/storybook"
-          target="_blank"
-        >Join Discord server<RightArrow /></a>
-      </div>
-    </div>
-    <div className="sb-section-item">
-      <Image 
-        width={32}
-        height={32}
-        layout="fixed"
-        src={Youtube} 
-        alt="Youtube logo" 
-        className="sb-explore-image"
-      />
-      <div>
-        Watch tutorials, feature previews and interviews.
-
-        <a
-          href="https://www.youtube.com/@chromaticui"
-          target="_blank"
-        >Watch on YouTube<RightArrow /></a>
-      </div>
-    </div>
-    <div className="sb-section-item">
-      <Image 
-        width={33}
-        height={32}
-        layout="fixed"
-        src={Tutorials} 
-        alt="A book" 
-        className="sb-explore-image"
-      />
-      <p>Follow guided walkthroughs on for key workflows.</p>
-
-      <a
-          href="https://storybook.js.org/tutorials/"
-          target="_blank"
-        >Discover tutorials<RightArrow /></a>
-    </div>
-</div>
-
-<style>
-  {`
-  .sb-container {
-    margin-bottom: 48px;
-  }
-
-  .sb-section {
-    width: 100%;
-    display: flex;
-    flex-direction: row;
-    gap: 20px;
-  }
-
-  img {
-    object-fit: cover;
-  }
-
-  .sb-section-title {
-    margin-bottom: 32px;
-  }
-
-  .sb-section a:not(h1 a, h2 a, h3 a) {
-    font-size: 14px;
-  }
-
-  .sb-section-item, .sb-grid-item {
-    flex: 1;
-    display: flex;
-    flex-direction: column;
-  }
-
-  .sb-section-item-heading {
-    padding-top: 20px !important;
-    padding-bottom: 5px !important;
-    margin: 0 !important;
-  }
-  .sb-section-item-paragraph {
-    margin: 0;
-    padding-bottom: 10px;
-  }
-
-  .sb-chevron {
-    margin-left: 5px;
-  }
-
-  .sb-features-grid {
-    display: grid;
-    grid-template-columns: repeat(2, 1fr);
-    grid-gap: 32px 20px;
-  }
-
-  .sb-socials {
-    display: grid;
-    grid-template-columns: repeat(4, 1fr);
-  }
-
-  .sb-socials p {
-    margin-bottom: 10px;
-  }
-
-  .sb-explore-image {
-    max-height: 32px;
-    align-self: flex-start;
-  }
-
-  .sb-addon {
-    width: 100%;
-    display: flex;
-    align-items: center;
-    position: relative;
-    background-color: #EEF3F8;
-    border-radius: 5px;
-    border: 1px solid rgba(0, 0, 0, 0.05);
-    background: #EEF3F8;
-    height: 180px;
-    margin-bottom: 48px;
-    overflow: hidden;
-  }
-
-  .sb-addon-text {
-    padding-left: 48px;
-    max-width: 240px;
-  }
-
-  .sb-addon-text h4 {
-    padding-top: 0px;
-  }
-
-  .sb-addon-img {
-    position: absolute;
-    left: 345px;
-    top: 0;
-    height: 100%;
-    width: 200%;
-    overflow: hidden;
-  }
-
-  .sb-addon-img img {
-    width: 650px;
-    transform: rotate(-15deg);
-    margin-left: 40px;
-    margin-top: -72px;
-    box-shadow: 0 0 1px rgba(255, 255, 255, 0);
-    backface-visibility: hidden;
-  }
-
-  @media screen and (max-width: 800px) {
-    .sb-addon-img {
-      left: 300px;
-    }
-  }
-
-  @media screen and (max-width: 600px) {
-    .sb-section {
-      flex-direction: column;
-    }
-
-    .sb-features-grid {
-      grid-template-columns: repeat(1, 1fr);
-    }
-
-    .sb-socials {
-      grid-template-columns: repeat(2, 1fr);
-    }
-
-    .sb-addon {
-      height: 280px;
-      align-items: flex-start;
-      padding-top: 32px;
-      overflow: hidden;
-    }
-
-    .sb-addon-text {
-      padding-left: 24px;
-    }
-
-    .sb-addon-img {
-      right: 0;
-      left: 0;
-      top: 130px;
-      bottom: 0;
-      overflow: hidden;
-      height: auto;
-      width: 124%;
-    }
-
-    .sb-addon-img img {
-      width: 1200px;
-      transform: rotate(-12deg);
-      margin-left: 0;
-      margin-top: 48px;
-      margin-bottom: -40px;
-      margin-left: -24px;
-    }
-  }
-  `}
-</style>

rnd/autogpt_builder/src/stories/Header.stories.ts

@@ -1,32 +0,0 @@
-import type { Meta, StoryObj } from '@storybook/react';
-import { fn } from '@storybook/test';
-import { Header } from './Header';
-
-const meta = {
-  title: 'Example/Header',
-  component: Header,
-  // This component will have an automatically generated Autodocs entry: https://storybook.js.org/docs/writing-docs/autodocs
-  tags: ['autodocs'],
-  parameters: {
-    // More on how to position stories at: https://storybook.js.org/docs/configure/story-layout
-    layout: 'fullscreen',
-  },
-  args: {
-    onLogin: fn(),
-    onLogout: fn(),
-    onCreateAccount: fn(),
-  },
-} satisfies Meta<typeof Header>;
-
-export default meta;
-type Story = StoryObj<typeof meta>;
-
-export const LoggedIn: Story = {
-  args: {
-    user: {
-      name: 'Jane Doe',
-    },
-  },
-};
-
-export const LoggedOut: Story = {};

rnd/autogpt_builder/src/stories/Header.tsx

@@ -1,56 +0,0 @@
-import React from 'react';
-
-import { Button } from './Button';
-import './header.css';
-
-type User = {
-  name: string;
-};
-
-export interface HeaderProps {
-  user?: User;
-  onLogin?: () => void;
-  onLogout?: () => void;
-  onCreateAccount?: () => void;
-}
-
-export const Header = ({ user, onLogin, onLogout, onCreateAccount }: HeaderProps) => (
-  <header>
-    <div className="storybook-header">
-      <div>
-        <svg width="32" height="32" viewBox="0 0 32 32" xmlns="http://www.w3.org/2000/svg">
-          <g fill="none" fillRule="evenodd">
-            <path
-              d="M10 0h12a10 10 0 0110 10v12a10 10 0 01-10 10H10A10 10 0 010 22V10A10 10 0 0110 0z"
-              fill="#FFF"
-            />
-            <path
-              d="M5.3 10.6l10.4 6v11.1l-10.4-6v-11zm11.4-6.2l9.7 5.5-9.7 5.6V4.4z"
-              fill="#555AB9"
-            />
-            <path
-              d="M27.2 10.6v11.2l-10.5 6V16.5l10.5-6zM15.7 4.4v11L6 10l9.7-5.5z"
-              fill="#91BAF8"
-            />
-          </g>
-        </svg>
-        <h1>Acme</h1>
-      </div>
-      <div>
-        {user ? (
-          <>
-            <span className="welcome">
-              Welcome, <b>{user.name}</b>!
-            </span>
-            <Button size="small" onClick={onLogout} label="Log out" />
-          </>
-        ) : (
-          <>
-            <Button size="small" onClick={onLogin} label="Log in" />
-            <Button primary size="small" onClick={onCreateAccount} label="Sign up" />
-          </>
-        )}
-      </div>
-    </div>
-  </header>
-);

rnd/autogpt_builder/src/stories/Page.stories.ts

@@ -1,32 +0,0 @@
-import type { Meta, StoryObj } from '@storybook/react';
-import { within, userEvent, expect } from '@storybook/test';
-
-import { Page } from './Page';
-
-const meta = {
-  title: 'Example/Page',
-  component: Page,
-  parameters: {
-    // More on how to position stories at: https://storybook.js.org/docs/configure/story-layout
-    layout: 'fullscreen',
-  },
-} satisfies Meta<typeof Page>;
-
-export default meta;
-type Story = StoryObj<typeof meta>;
-
-export const LoggedOut: Story = {};
-
-// More on interaction testing: https://storybook.js.org/docs/writing-tests/interaction-testing
-export const LoggedIn: Story = {
-  play: async ({ canvasElement }) => {
-    const canvas = within(canvasElement);
-    const loginButton = canvas.getByRole('button', { name: /Log in/i });
-    await expect(loginButton).toBeInTheDocument();
-    await userEvent.click(loginButton);
-    await expect(loginButton).not.toBeInTheDocument();
-
-    const logoutButton = canvas.getByRole('button', { name: /Log out/i });
-    await expect(logoutButton).toBeInTheDocument();
-  },
-};

rnd/autogpt_builder/src/stories/Page.tsx

@@ -1,73 +0,0 @@
-import React from 'react';
-
-import { Header } from './Header';
-import './page.css';
-
-type User = {
-  name: string;
-};
-
-export const Page: React.FC = () => {
-  const [user, setUser] = React.useState<User>();
-
-  return (
-    <article>
-      <Header
-        user={user}
-        onLogin={() => setUser({ name: 'Jane Doe' })}
-        onLogout={() => setUser(undefined)}
-        onCreateAccount={() => setUser({ name: 'Jane Doe' })}
-      />
-
-      <section className="storybook-page">
-        <h2>Pages in Storybook</h2>
-        <p>
-          We recommend building UIs with a{' '}
-          <a href="https://componentdriven.org" target="_blank" rel="noopener noreferrer">
-            <strong>component-driven</strong>
-          </a>{' '}
-          process starting with atomic components and ending with pages.
-        </p>
-        <p>
-          Render pages with mock data. This makes it easy to build and review page states without
-          needing to navigate to them in your app. Here are some handy patterns for managing page
-          data in Storybook:
-        </p>
-        <ul>
-          <li>
-            Use a higher-level connected component. Storybook helps you compose such data from the
-            "args" of child component stories
-          </li>
-          <li>
-            Assemble data in the page component from your services. You can mock these services out
-            using Storybook.
-          </li>
-        </ul>
-        <p>
-          Get a guided tutorial on component-driven development at{' '}
-          <a href="https://storybook.js.org/tutorials/" target="_blank" rel="noopener noreferrer">
-            Storybook tutorials
-          </a>
-          . Read more in the{' '}
-          <a href="https://storybook.js.org/docs" target="_blank" rel="noopener noreferrer">
-            docs
-          </a>
-          .
-        </p>
-        <div className="tip-wrapper">
-          <span className="tip">Tip</span> Adjust the width of the canvas with the{' '}
-          <svg width="10" height="10" viewBox="0 0 12 12" xmlns="http://www.w3.org/2000/svg">
-            <g fill="none" fillRule="evenodd">
-              <path
-                d="M1.5 5.2h4.8c.3 0 .5.2.5.4v5.1c-.1.2-.3.3-.4.3H1.4a.5.5 0 01-.5-.4V5.7c0-.3.2-.5.5-.5zm0-2.1h6.9c.3 0 .5.2.5.4v7a.5.5 0 01-1 0V4H1.5a.5.5 0 010-1zm0-2.1h9c.3 0 .5.2.5.4v9.1a.5.5 0 01-1 0V2H1.5a.5.5 0 010-1zm4.3 5.2H2V10h3.8V6.2z"
-                id="a"
-                fill="#999"
-              />
-            </g>
-          </svg>
-          Viewports addon in the toolbar
-        </div>
-      </section>
-    </article>
-  );
-};

rnd/autogpt_builder/src/stories/assets/accessibility.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" width="48" height="48" fill="none" viewBox="0 0 48 48"><title>Accessibility</title><circle cx="24.334" cy="24" r="24" fill="#A849FF" fill-opacity=".3"/><path fill="#A470D5" fill-rule="evenodd" d="M27.8609 11.585C27.8609 9.59506 26.2497 7.99023 24.2519 7.99023C22.254 7.99023 20.6429 9.65925 20.6429 11.585C20.6429 13.575 22.254 15.1799 24.2519 15.1799C26.2497 15.1799 27.8609 13.575 27.8609 11.585ZM21.8922 22.6473C21.8467 23.9096 21.7901 25.4788 21.5897 26.2771C20.9853 29.0462 17.7348 36.3314 17.3325 37.2275C17.1891 37.4923 17.1077 37.7955 17.1077 38.1178C17.1077 39.1519 17.946 39.9902 18.9802 39.9902C19.6587 39.9902 20.253 39.6293 20.5814 39.0889L20.6429 38.9874L24.2841 31.22C24.2841 31.22 27.5529 37.9214 27.9238 38.6591C28.2948 39.3967 28.8709 39.9902 29.7168 39.9902C30.751 39.9902 31.5893 39.1519 31.5893 38.1178C31.5893 37.7951 31.3639 37.2265 31.3639 37.2265C30.9581 36.3258 27.698 29.0452 27.0938 26.2771C26.8975 25.4948 26.847 23.9722 26.8056 22.7236C26.7927 22.333 26.7806 21.9693 26.7653 21.6634C26.7008 21.214 27.0231 20.8289 27.4097 20.7005L35.3366 18.3253C36.3033 18.0685 36.8834 16.9773 36.6256 16.0144C36.3678 15.0515 35.2722 14.4737 34.3055 14.7305C34.3055 14.7305 26.8619 17.1057 24.2841 17.1057C21.7062 17.1057 14.456 14.7947 14.456 14.7947C13.4893 14.5379 12.3937 14.9873 12.0715 15.9502C11.7493 16.9131 12.3293 18.0044 13.3604 18.3253L21.2873 20.7005C21.674 20.8289 21.9318 21.214 21.9318 21.6634C21.9174 21.9493 21.9053 22.2857 21.8922 22.6473Z" clip-rule="evenodd"/></svg>

rnd/autogpt_builder/src/stories/assets/discord.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" width="33" height="32" fill="none" viewBox="0 0 33 32"><g clip-path="url(#clip0_10031_177575)"><mask id="mask0_10031_177575" style="mask-type:luminance" width="33" height="25" x="0" y="4" maskUnits="userSpaceOnUse"><path fill="#fff" d="M32.5034 4.00195H0.503906V28.7758H32.5034V4.00195Z"/></mask><g mask="url(#mask0_10031_177575)"><path fill="#5865F2" d="M27.5928 6.20817C25.5533 5.27289 23.3662 4.58382 21.0794 4.18916C21.0378 4.18154 20.9962 4.20057 20.9747 4.23864C20.6935 4.73863 20.3819 5.3909 20.1637 5.90358C17.7042 5.53558 15.2573 5.53558 12.8481 5.90358C12.6299 5.37951 12.307 4.73863 12.0245 4.23864C12.003 4.20184 11.9614 4.18281 11.9198 4.18916C9.63431 4.58255 7.44721 5.27163 5.40641 6.20817C5.38874 6.21578 5.3736 6.22848 5.36355 6.24497C1.21508 12.439 0.078646 18.4809 0.636144 24.4478C0.638667 24.477 0.655064 24.5049 0.677768 24.5227C3.41481 26.5315 6.06609 27.7511 8.66815 28.5594C8.70979 28.5721 8.75392 28.5569 8.78042 28.5226C9.39594 27.6826 9.94461 26.7968 10.4151 25.8653C10.4428 25.8107 10.4163 25.746 10.3596 25.7244C9.48927 25.3945 8.66058 24.9922 7.86343 24.5354C7.80038 24.4986 7.79533 24.4084 7.85333 24.3653C8.02108 24.2397 8.18888 24.109 8.34906 23.977C8.37804 23.9529 8.41842 23.9478 8.45249 23.963C13.6894 26.3526 19.359 26.3526 24.5341 23.963C24.5682 23.9465 24.6086 23.9516 24.6388 23.9757C24.799 24.1077 24.9668 24.2397 25.1358 24.3653C25.1938 24.4084 25.19 24.4986 25.127 24.5354C24.3298 25.0011 23.5011 25.3945 22.6296 25.7232C22.5728 25.7447 22.5476 25.8107 22.5754 25.8653C23.0559 26.7955 23.6046 27.6812 24.2087 28.5213C24.234 28.5569 24.2794 28.5721 24.321 28.5594C26.9357 27.7511 29.5869 26.5315 32.324 24.5227C32.348 24.5049 32.3631 24.4783 32.3656 24.4491C33.0328 17.5506 31.2481 11.5584 27.6344 6.24623C27.6256 6.22848 27.6105 6.21578 27.5928 6.20817ZM11.1971 20.8146C9.62043 20.8146 8.32129 19.3679 8.32129 17.5913C8.32129 15.8146 9.59523 14.368 11.1971 14.368C12.8115 14.368 14.0981 15.8273 14.0729 17.5913C14.0729 19.3679 12.7989 20.8146 11.1971 20.8146ZM21.8299 20.8146C20.2533 20.8146 18.9541 19.3679 18.9541 17.5913C18.9541 15.8146 20.228 14.368 21.8299 14.368C23.4444 14.368 24.7309 15.8273 24.7057 17.5913C24.7057 19.3679 23.4444 20.8146 21.8299 20.8146Z"/></g></g><defs><clipPath id="clip0_10031_177575"><rect width="32" height="32" fill="#fff" transform="translate(0.5)"/></clipPath></defs></svg>

rnd/autogpt_builder/src/stories/assets/github.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" fill="none" viewBox="0 0 32 32"><path fill="#161614" d="M16.0001 0C7.16466 0 0 7.17472 0 16.0256C0 23.1061 4.58452 29.1131 10.9419 31.2322C11.7415 31.3805 12.0351 30.8845 12.0351 30.4613C12.0351 30.0791 12.0202 28.8167 12.0133 27.4776C7.56209 28.447 6.62283 25.5868 6.62283 25.5868C5.89499 23.7345 4.8463 23.2419 4.8463 23.2419C3.39461 22.2473 4.95573 22.2678 4.95573 22.2678C6.56242 22.3808 7.40842 23.9192 7.40842 23.9192C8.83547 26.3691 11.1514 25.6609 12.0645 25.2514C12.2081 24.2156 12.6227 23.5087 13.0803 23.1085C9.52648 22.7032 5.7906 21.3291 5.7906 15.1886C5.7906 13.4389 6.41563 12.0094 7.43916 10.8871C7.27303 10.4834 6.72537 8.85349 7.59415 6.64609C7.59415 6.64609 8.93774 6.21539 11.9953 8.28877C13.2716 7.9337 14.6404 7.75563 16.0001 7.74953C17.3599 7.75563 18.7297 7.9337 20.0084 8.28877C23.0623 6.21539 24.404 6.64609 24.404 6.64609C25.2749 8.85349 24.727 10.4834 24.5608 10.8871C25.5868 12.0094 26.2075 13.4389 26.2075 15.1886C26.2075 21.3437 22.4645 22.699 18.9017 23.0957C19.4756 23.593 19.9869 24.5683 19.9869 26.0634C19.9869 28.2077 19.9684 29.9334 19.9684 30.4613C19.9684 30.8877 20.2564 31.3874 21.0674 31.2301C27.4213 29.1086 32 23.1037 32 16.0256C32 7.17472 24.8364 0 16.0001 0ZM5.99257 22.8288C5.95733 22.9084 5.83227 22.9322 5.71834 22.8776C5.60229 22.8253 5.53711 22.7168 5.57474 22.6369C5.60918 22.5549 5.7345 22.5321 5.85029 22.587C5.9666 22.6393 6.03284 22.7489 5.99257 22.8288ZM6.7796 23.5321C6.70329 23.603 6.55412 23.5701 6.45291 23.4581C6.34825 23.3464 6.32864 23.197 6.40601 23.125C6.4847 23.0542 6.62937 23.0874 6.73429 23.1991C6.83895 23.3121 6.85935 23.4605 6.7796 23.5321ZM7.31953 24.4321C7.2215 24.5003 7.0612 24.4363 6.96211 24.2938C6.86407 24.1513 6.86407 23.9804 6.96422 23.9119C7.06358 23.8435 7.2215 23.905 7.32191 24.0465C7.41968 24.1914 7.41968 24.3623 7.31953 24.4321ZM8.23267 25.4743C8.14497 25.5712 7.95818 25.5452 7.82146 25.413C7.68156 25.2838 7.64261 25.1004 7.73058 25.0035C7.81934 24.9064 8.00719 24.9337 8.14497 25.0648C8.28381 25.1938 8.3262 25.3785 8.23267 25.4743ZM9.41281 25.8262C9.37413 25.9517 9.19423 26.0088 9.013 25.9554C8.83203 25.9005 8.7136 25.7535 8.75016 25.6266C8.78778 25.5003 8.96848 25.4408 9.15104 25.4979C9.33174 25.5526 9.45044 25.6985 9.41281 25.8262ZM10.7559 25.9754C10.7604 26.1076 10.6067 26.2172 10.4165 26.2196C10.2252 26.2238 10.0704 26.1169 10.0683 25.9868C10.0683 25.8534 10.2185 25.7448 10.4098 25.7416C10.6001 25.7379 10.7559 25.8441 10.7559 25.9754ZM12.0753 25.9248C12.0981 26.0537 11.9658 26.1862 11.7769 26.2215C11.5912 26.2554 11.4192 26.1758 11.3957 26.0479C11.3726 25.9157 11.5072 25.7833 11.6927 25.7491C11.8819 25.7162 12.0512 25.7937 12.0753 25.9248Z"/></svg>

rnd/autogpt_builder/src/stories/assets/tutorials.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" width="33" height="32" fill="none" viewBox="0 0 33 32"><g clip-path="url(#clip0_10031_177597)"><path fill="#B7F0EF" fill-rule="evenodd" d="M17 7.87059C17 6.48214 17.9812 5.28722 19.3431 5.01709L29.5249 2.99755C31.3238 2.64076 33 4.01717 33 5.85105V22.1344C33 23.5229 32.0188 24.7178 30.6569 24.9879L20.4751 27.0074C18.6762 27.3642 17 25.9878 17 24.1539L17 7.87059Z" clip-rule="evenodd" opacity=".7"/><path fill="#87E6E5" fill-rule="evenodd" d="M1 5.85245C1 4.01857 2.67623 2.64215 4.47507 2.99895L14.6569 5.01848C16.0188 5.28861 17 6.48354 17 7.87198V24.1553C17 25.9892 15.3238 27.3656 13.5249 27.0088L3.34311 24.9893C1.98119 24.7192 1 23.5242 1 22.1358V5.85245Z" clip-rule="evenodd"/><path fill="#61C1FD" fill-rule="evenodd" d="M15.543 5.71289C15.543 5.71289 16.8157 5.96289 17.4002 6.57653C17.9847 7.19016 18.4521 9.03107 18.4521 9.03107C18.4521 9.03107 18.4521 25.1106 18.4521 26.9629C18.4521 28.8152 19.3775 31.4174 19.3775 31.4174L17.4002 28.8947L16.2575 31.4174C16.2575 31.4174 15.543 29.0765 15.543 27.122C15.543 25.1674 15.543 5.71289 15.543 5.71289Z" clip-rule="evenodd"/></g><defs><clipPath id="clip0_10031_177597"><rect width="32" height="32" fill="#fff" transform="translate(0.5)"/></clipPath></defs></svg>

rnd/autogpt_builder/src/stories/assets/youtube.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" fill="none" viewBox="0 0 32 32"><path fill="#ED1D24" d="M31.3313 8.44657C30.9633 7.08998 29.8791 6.02172 28.5022 5.65916C26.0067 5.00026 16 5.00026 16 5.00026C16 5.00026 5.99333 5.00026 3.4978 5.65916C2.12102 6.02172 1.03665 7.08998 0.668678 8.44657C0 10.9053 0 16.0353 0 16.0353C0 16.0353 0 21.1652 0.668678 23.6242C1.03665 24.9806 2.12102 26.0489 3.4978 26.4116C5.99333 27.0703 16 27.0703 16 27.0703C16 27.0703 26.0067 27.0703 28.5022 26.4116C29.8791 26.0489 30.9633 24.9806 31.3313 23.6242C32 21.1652 32 16.0353 32 16.0353C32 16.0353 32 10.9053 31.3313 8.44657Z"/><path fill="#fff" d="M12.7266 20.6934L21.0902 16.036L12.7266 11.3781V20.6934Z"/></svg>

rnd/autogpt_builder/src/stories/button.css

@@ -1,30 +0,0 @@
-.storybook-button {
-  font-family: 'Nunito Sans', 'Helvetica Neue', Helvetica, Arial, sans-serif;
-  font-weight: 700;
-  border: 0;
-  border-radius: 3em;
-  cursor: pointer;
-  display: inline-block;
-  line-height: 1;
-}
-.storybook-button--primary {
-  color: white;
-  background-color: #1ea7fd;
-}
-.storybook-button--secondary {
-  color: #333;
-  background-color: transparent;
-  box-shadow: rgba(0, 0, 0, 0.15) 0px 0px 0px 1px inset;
-}
-.storybook-button--small {
-  font-size: 12px;
-  padding: 10px 16px;
-}
-.storybook-button--medium {
-  font-size: 14px;
-  padding: 11px 20px;
-}
-.storybook-button--large {
-  font-size: 16px;
-  padding: 12px 24px;
-}

rnd/autogpt_builder/src/stories/header.css

@@ -1,32 +0,0 @@
-.storybook-header {
-  font-family: 'Nunito Sans', 'Helvetica Neue', Helvetica, Arial, sans-serif;
-  border-bottom: 1px solid rgba(0, 0, 0, 0.1);
-  padding: 15px 20px;
-  display: flex;
-  align-items: center;
-  justify-content: space-between;
-}
-
-.storybook-header svg {
-  display: inline-block;
-  vertical-align: top;
-}
-
-.storybook-header h1 {
-  font-weight: 700;
-  font-size: 20px;
-  line-height: 1;
-  margin: 6px 0 6px 10px;
-  display: inline-block;
-  vertical-align: top;
-}
-
-.storybook-header button + button {
-  margin-left: 10px;
-}
-
-.storybook-header .welcome {
-  color: #333;
-  font-size: 14px;
-  margin-right: 10px;
-}

rnd/autogpt_builder/src/stories/page.css

@@ -1,69 +0,0 @@
-.storybook-page {
-  font-family: 'Nunito Sans', 'Helvetica Neue', Helvetica, Arial, sans-serif;
-  font-size: 14px;
-  line-height: 24px;
-  padding: 48px 20px;
-  margin: 0 auto;
-  max-width: 600px;
-  color: #333;
-}
-
-.storybook-page h2 {
-  font-weight: 700;
-  font-size: 32px;
-  line-height: 1;
-  margin: 0 0 4px;
-  display: inline-block;
-  vertical-align: top;
-}
-
-.storybook-page p {
-  margin: 1em 0;
-}
-
-.storybook-page a {
-  text-decoration: none;
-  color: #1ea7fd;
-}
-
-.storybook-page ul {
-  padding-left: 30px;
-  margin: 1em 0;
-}
-
-.storybook-page li {
-  margin-bottom: 8px;
-}
-
-.storybook-page .tip {
-  display: inline-block;
-  border-radius: 1em;
-  font-size: 11px;
-  line-height: 12px;
-  font-weight: 700;
-  background: #e7fdd8;
-  color: #66bf3c;
-  padding: 4px 12px;
-  margin-right: 10px;
-  vertical-align: top;
-}
-
-.storybook-page .tip-wrapper {
-  font-size: 13px;
-  line-height: 20px;
-  margin-top: 40px;
-  margin-bottom: 40px;
-}
-
-.storybook-page .tip-wrapper svg {
-  display: inline-block;
-  height: 12px;
-  width: 12px;
-  margin-right: 4px;
-  vertical-align: top;
-  margin-top: 3px;
-}
-
-.storybook-page .tip-wrapper svg path {
-  fill: #1ea7fd;
-}

rnd/autogpt_libs/autogpt_libs/auth/depends_tests.py

@@ -1,6 +1,6 @@
 import pytest
 
-from .depends import verify_user, requires_admin_user, requires_user
+from .depends import requires_admin_user, requires_user, verify_user
 
 
 def test_verify_user_no_payload():

rnd/autogpt_libs/autogpt_libs/logging/__init__.py

@@ -0,0 +1,9 @@
+from .config import configure_logging
+from .filters import BelowLevelFilter
+from .formatters import FancyConsoleFormatter
+
+__all__ = [
+    "configure_logging",
+    "BelowLevelFilter",
+    "FancyConsoleFormatter",
+]

rnd/autogpt_libs/autogpt_libs/logging/config.py

@@ -0,0 +1,166 @@
+"""Logging module for Auto-GPT."""
+
+import logging
+import sys
+from pathlib import Path
+
+from pydantic import Field, field_validator
+from pydantic_settings import BaseSettings, SettingsConfigDict
+from .filters import BelowLevelFilter
+from .formatters import AGPTFormatter, StructuredLoggingFormatter
+
+LOG_DIR = Path(__file__).parent.parent.parent.parent / "logs"
+LOG_FILE = "activity.log"
+DEBUG_LOG_FILE = "debug.log"
+ERROR_LOG_FILE = "error.log"
+
+SIMPLE_LOG_FORMAT = "%(asctime)s %(levelname)s  %(title)s%(message)s"
+
+DEBUG_LOG_FORMAT = (
+    "%(asctime)s %(levelname)s %(filename)s:%(lineno)d" "  %(title)s%(message)s"
+)
+
+
+class LoggingConfig(BaseSettings):
+
+    level: str = Field(
+        default="INFO",
+        description="Logging level",
+        validation_alias="LOG_LEVEL",
+    )
+
+    enable_cloud_logging: bool = Field(
+        default=False,
+        description="Enable logging to Google Cloud Logging",
+    )
+
+    enable_file_logging: bool = Field(
+        default=False,
+        description="Enable logging to file",
+    )
+    # File output
+    log_dir: Path = Field(
+        default=LOG_DIR,
+        description="Log directory",
+    )
+
+    model_config = SettingsConfigDict(
+        env_prefix="",
+        env_file=".env",
+        env_file_encoding="utf-8",
+        extra="ignore",
+    )
+
+    @field_validator("level", mode="before")
+    @classmethod
+    def parse_log_level(cls, v):
+        if isinstance(v, str):
+            v = v.upper()
+            if v not in ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]:
+                raise ValueError(f"Invalid log level: {v}")
+            return v
+        return v
+
+
+def configure_logging(force_cloud_logging: bool = False) -> None:
+    """Configure the native logging module based on the LoggingConfig settings.
+
+    This function sets up logging handlers and formatters according to the
+    configuration specified in the LoggingConfig object. It supports various
+    logging outputs including console, file, cloud, and JSON logging.
+
+    The function uses the LoggingConfig object to determine which logging
+    features to enable and how to configure them. This includes setting
+    log levels, log formats, and output destinations.
+
+    No arguments are required as the function creates its own LoggingConfig
+    instance internally.
+
+    Note: This function is typically called at the start of the application
+    to set up the logging infrastructure.
+    """
+
+    config = LoggingConfig()
+
+    log_handlers: list[logging.Handler] = []
+
+    # Cloud logging setup
+    if config.enable_cloud_logging or force_cloud_logging:
+        import google.cloud.logging
+        from google.cloud.logging.handlers import CloudLoggingHandler
+        from google.cloud.logging_v2.handlers.transports.sync import SyncTransport
+
+        client = google.cloud.logging.Client()
+        cloud_handler = CloudLoggingHandler(
+            client,
+            name="autogpt_logs",
+            transport=SyncTransport,
+        )
+        cloud_handler.setLevel(config.level)
+        cloud_handler.setFormatter(StructuredLoggingFormatter())
+        log_handlers.append(cloud_handler)
+        print("Cloud logging enabled")
+    else:
+        # Console output handlers
+        stdout = logging.StreamHandler(stream=sys.stdout)
+        stdout.setLevel(config.level)
+        stdout.addFilter(BelowLevelFilter(logging.WARNING))
+        if config.level == logging.DEBUG:
+            stdout.setFormatter(AGPTFormatter(DEBUG_LOG_FORMAT))
+        else:
+            stdout.setFormatter(AGPTFormatter(SIMPLE_LOG_FORMAT))
+
+        stderr = logging.StreamHandler()
+        stderr.setLevel(logging.WARNING)
+        if config.level == logging.DEBUG:
+            stderr.setFormatter(AGPTFormatter(DEBUG_LOG_FORMAT))
+        else:
+            stderr.setFormatter(AGPTFormatter(SIMPLE_LOG_FORMAT))
+
+        log_handlers += [stdout, stderr]
+        print("Console logging enabled")
+
+    # File logging setup
+    if config.enable_file_logging:
+        # create log directory if it doesn't exist
+        if not config.log_dir.exists():
+            config.log_dir.mkdir(parents=True, exist_ok=True)
+
+        print(f"Log directory: {config.log_dir}")
+
+        # Activity log handler (INFO and above)
+        activity_log_handler = logging.FileHandler(
+            config.log_dir / LOG_FILE, "a", "utf-8"
+        )
+        activity_log_handler.setLevel(config.level)
+        activity_log_handler.setFormatter(
+            AGPTFormatter(SIMPLE_LOG_FORMAT, no_color=True)
+        )
+        log_handlers.append(activity_log_handler)
+
+        if config.level == logging.DEBUG:
+            # Debug log handler (all levels)
+            debug_log_handler = logging.FileHandler(
+                config.log_dir / DEBUG_LOG_FILE, "a", "utf-8"
+            )
+            debug_log_handler.setLevel(logging.DEBUG)
+            debug_log_handler.setFormatter(
+                AGPTFormatter(DEBUG_LOG_FORMAT, no_color=True)
+            )
+            log_handlers.append(debug_log_handler)
+
+        # Error log handler (ERROR and above)
+        error_log_handler = logging.FileHandler(
+            config.log_dir / ERROR_LOG_FILE, "a", "utf-8"
+        )
+        error_log_handler.setLevel(logging.ERROR)
+        error_log_handler.setFormatter(AGPTFormatter(DEBUG_LOG_FORMAT, no_color=True))
+        log_handlers.append(error_log_handler)
+        print("File logging enabled")
+
+    # Configure the root logger
+    logging.basicConfig(
+        format=DEBUG_LOG_FORMAT if config.level == logging.DEBUG else SIMPLE_LOG_FORMAT,
+        level=config.level,
+        handlers=log_handlers,
+    )

rnd/autogpt_libs/autogpt_libs/logging/filters.py

@@ -0,0 +1,12 @@
+import logging
+
+
+class BelowLevelFilter(logging.Filter):
+    """Filter for logging levels below a certain threshold."""
+
+    def __init__(self, below_level: int):
+        super().__init__()
+        self.below_level = below_level
+
+    def filter(self, record: logging.LogRecord):
+        return record.levelno < self.below_level

rnd/autogpt_libs/autogpt_libs/logging/formatters.py

@@ -0,0 +1,95 @@
+import logging
+
+from colorama import Fore, Style
+from google.cloud.logging_v2.handlers import CloudLoggingFilter, StructuredLogHandler
+
+from .utils import remove_color_codes
+
+
+class FancyConsoleFormatter(logging.Formatter):
+    """
+    A custom logging formatter designed for console output.
+
+    This formatter enhances the standard logging output with color coding. The color
+    coding is based on the level of the log message, making it easier to distinguish
+    between different types of messages in the console output.
+
+    The color for each level is defined in the LEVEL_COLOR_MAP class attribute.
+    """
+
+    # level -> (level & text color, title color)
+    LEVEL_COLOR_MAP = {
+        logging.DEBUG: Fore.LIGHTBLACK_EX,
+        logging.INFO: Fore.BLUE,
+        logging.WARNING: Fore.YELLOW,
+        logging.ERROR: Fore.RED,
+        logging.CRITICAL: Fore.RED + Style.BRIGHT,
+    }
+
+    def format(self, record: logging.LogRecord) -> str:
+        # Make sure `msg` is a string
+        if not hasattr(record, "msg"):
+            record.msg = ""
+        elif type(record.msg) is not str:
+            record.msg = str(record.msg)
+
+        # Determine default color based on error level
+        level_color = ""
+        if record.levelno in self.LEVEL_COLOR_MAP:
+            level_color = self.LEVEL_COLOR_MAP[record.levelno]
+            record.levelname = f"{level_color}{record.levelname}{Style.RESET_ALL}"
+
+        # Determine color for message
+        color = getattr(record, "color", level_color)
+        color_is_specified = hasattr(record, "color")
+
+        # Don't color INFO messages unless the color is explicitly specified.
+        if color and (record.levelno != logging.INFO or color_is_specified):
+            record.msg = f"{color}{record.msg}{Style.RESET_ALL}"
+
+        return super().format(record)
+
+
+class AGPTFormatter(FancyConsoleFormatter):
+    def __init__(self, *args, no_color: bool = False, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.no_color = no_color
+
+    def format(self, record: logging.LogRecord) -> str:
+        # Make sure `msg` is a string
+        if not hasattr(record, "msg"):
+            record.msg = ""
+        elif type(record.msg) is not str:
+            record.msg = str(record.msg)
+
+        # Strip color from the message to prevent color spoofing
+        if record.msg and not getattr(record, "preserve_color", False):
+            record.msg = remove_color_codes(record.msg)
+
+        # Determine color for title
+        title = getattr(record, "title", "")
+        title_color = getattr(record, "title_color", "") or self.LEVEL_COLOR_MAP.get(
+            record.levelno, ""
+        )
+        if title and title_color:
+            title = f"{title_color + Style.BRIGHT}{title}{Style.RESET_ALL}"
+        # Make sure record.title is set, and padded with a space if not empty
+        record.title = f"{title} " if title else ""
+
+        if self.no_color:
+            return remove_color_codes(super().format(record))
+        else:
+            return super().format(record)
+
+
+class StructuredLoggingFormatter(StructuredLogHandler, logging.Formatter):
+    def __init__(self):
+        # Set up CloudLoggingFilter to add diagnostic info to the log records
+        self.cloud_logging_filter = CloudLoggingFilter()
+
+        # Init StructuredLogHandler
+        super().__init__()
+
+    def format(self, record: logging.LogRecord) -> str:
+        self.cloud_logging_filter.filter(record)
+        return super().format(record)

rnd/autogpt_libs/autogpt_libs/logging/handlers.py

@@ -0,0 +1,14 @@
+from __future__ import annotations
+
+import json
+import logging
+
+
+class JsonFileHandler(logging.FileHandler):
+    def format(self, record: logging.LogRecord) -> str:
+        record.json_data = json.loads(record.getMessage())
+        return json.dumps(getattr(record, "json_data"), ensure_ascii=False, indent=4)
+
+    def emit(self, record: logging.LogRecord) -> None:
+        with open(self.baseFilename, "w", encoding="utf-8") as f:
+            f.write(self.format(record))

rnd/autogpt_libs/autogpt_libs/logging/test_utils.py

@@ -0,0 +1,36 @@
+import pytest
+
+from .utils import remove_color_codes
+
+
+@pytest.mark.parametrize(
+    "raw_text, clean_text",
+    [
+        (
+            "COMMAND = \x1b[36mbrowse_website\x1b[0m  "
+            "ARGUMENTS = \x1b[36m{'url': 'https://www.google.com',"
+            " 'question': 'What is the capital of France?'}\x1b[0m",
+            "COMMAND = browse_website  "
+            "ARGUMENTS = {'url': 'https://www.google.com',"
+            " 'question': 'What is the capital of France?'}",
+        ),
+        (
+            "{'Schaue dir meine Projekte auf github () an, als auch meine Webseiten': "
+            "'https://github.com/Significant-Gravitas/AutoGPT,"
+            " https://discord.gg/autogpt und https://twitter.com/Auto_GPT'}",
+            "{'Schaue dir meine Projekte auf github () an, als auch meine Webseiten': "
+            "'https://github.com/Significant-Gravitas/AutoGPT,"
+            " https://discord.gg/autogpt und https://twitter.com/Auto_GPT'}",
+        ),
+        ("", ""),
+        ("hello", "hello"),
+        ("hello\x1B[31m world", "hello world"),
+        ("\x1B[36mHello,\x1B[32m World!", "Hello, World!"),
+        (
+            "\x1B[1m\x1B[31mError:\x1B[0m\x1B[31m file not found",
+            "Error: file not found",
+        ),
+    ],
+)
+def test_remove_color_codes(raw_text, clean_text):
+    assert remove_color_codes(raw_text) == clean_text

rnd/autogpt_libs/autogpt_libs/logging/utils.py

@@ -0,0 +1,27 @@
+import logging
+import re
+from typing import Any
+
+from colorama import Fore
+
+
+def remove_color_codes(s: str) -> str:
+    return re.sub(r"\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])", "", s)
+
+
+def fmt_kwargs(kwargs: dict) -> str:
+    return ", ".join(f"{n}={repr(v)}" for n, v in kwargs.items())
+
+
+def print_attribute(
+    title: str, value: Any, title_color: str = Fore.GREEN, value_color: str = ""
+) -> None:
+    logger = logging.getLogger()
+    logger.info(
+        str(value),
+        extra={
+            "title": f"{title.rstrip(':')}:",
+            "title_color": title_color,
+            "color": value_color,
+        },
+    )

rnd/autogpt_libs/autogpt_libs/supabase_integration_credentials_store/store.py

@@ -1,13 +1,21 @@
+import secrets
+from datetime import datetime, timedelta, timezone
 from typing import cast
 
-from supabase import Client, create_client
+from supabase import Client
 
-from .types import Credentials, OAuth2Credentials, UserMetadata, UserMetadataRaw
+from .types import (
+    Credentials,
+    OAuth2Credentials,
+    OAuthState,
+    UserMetadata,
+    UserMetadataRaw,
+)
 
 
 class SupabaseIntegrationCredentialsStore:
-    def __init__(self, url: str, key: str):
-        self.supabase: Client = create_client(url, key)
+    def __init__(self, supabase: Client):
+        self.supabase = supabase
 
     def add_creds(self, user_id: str, credentials: Credentials) -> None:
         if self.get_creds_by_id(user_id, credentials.id):
@@ -73,6 +81,52 @@ class SupabaseIntegrationCredentialsStore:
         ]
         self._set_user_integration_creds(user_id, filtered_credentials)
 
+    async def store_state_token(self, user_id: str, provider: str) -> str:
+        token = secrets.token_urlsafe(32)
+        expires_at = datetime.now(timezone.utc) + timedelta(minutes=10)
+
+        state = OAuthState(
+            token=token, provider=provider, expires_at=int(expires_at.timestamp())
+        )
+
+        user_metadata = self._get_user_metadata(user_id)
+        oauth_states = user_metadata.get("integration_oauth_states", [])
+        oauth_states.append(state.model_dump())
+        user_metadata["integration_oauth_states"] = oauth_states
+
+        self.supabase.auth.admin.update_user_by_id(
+            user_id, {"user_metadata": user_metadata}
+        )
+
+        return token
+
+    async def verify_state_token(self, user_id: str, token: str, provider: str) -> bool:
+        user_metadata = self._get_user_metadata(user_id)
+        oauth_states = user_metadata.get("integration_oauth_states", [])
+
+        now = datetime.now(timezone.utc)
+        valid_state = next(
+            (
+                state
+                for state in oauth_states
+                if state["token"] == token
+                and state["provider"] == provider
+                and state["expires_at"] > now.timestamp()
+            ),
+            None,
+        )
+
+        if valid_state:
+            # Remove the used state
+            oauth_states.remove(valid_state)
+            user_metadata["integration_oauth_states"] = oauth_states
+            self.supabase.auth.admin.update_user_by_id(
+                user_id, {"user_metadata": user_metadata}
+            )
+            return True
+
+        return False
+
     def _set_user_integration_creds(
         self, user_id: str, credentials: list[Credentials]
     ) -> None:

rnd/autogpt_libs/autogpt_libs/supabase_integration_credentials_store/types.py

@@ -19,9 +19,11 @@ class _BaseCredentials(BaseModel):
 class OAuth2Credentials(_BaseCredentials):
     type: Literal["oauth2"] = "oauth2"
     access_token: SecretStr
-    access_token_expires_at: Optional[int]  # seconds
+    access_token_expires_at: Optional[int]
+    """Unix timestamp (seconds) indicating when the access token expires (if at all)"""
     refresh_token: Optional[SecretStr]
-    refresh_token_expires_at: Optional[int]  # seconds
+    refresh_token_expires_at: Optional[int]
+    """Unix timestamp (seconds) indicating when the refresh token expires (if at all)"""
     scopes: list[str]
     metadata: dict[str, Any] = Field(default_factory=dict)
 
@@ -29,7 +31,8 @@ class OAuth2Credentials(_BaseCredentials):
 class APIKeyCredentials(_BaseCredentials):
     type: Literal["api_key"] = "api_key"
     api_key: SecretStr
-    expires_at: Optional[int]  # seconds
+    expires_at: Optional[int]
+    """Unix timestamp (seconds) indicating when the API key expires (if at all)"""
 
 
 Credentials = Annotated[
@@ -38,9 +41,18 @@ Credentials = Annotated[
 ]
 
 
+class OAuthState(BaseModel):
+    token: str
+    provider: str
+    expires_at: int
+    """Unix timestamp (seconds) indicating when this OAuth state expires"""
+
+
 class UserMetadata(BaseModel):
     integration_credentials: list[Credentials] = Field(default_factory=list)
+    integration_oauth_states: list[OAuthState] = Field(default_factory=list)
 
 
 class UserMetadataRaw(TypedDict, total=False):
     integration_credentials: list[dict]
+    integration_oauth_states: list[dict]

rnd/autogpt_libs/pyproject.toml

@@ -1,19 +1,21 @@
 [tool.poetry]
 name = "autogpt-libs"
-version = "0.1.0"
+version = "0.2.0"
 description = "Shared libraries across NextGen AutoGPT"
 authors = ["Aarushi <aarushik93@gmail.com>"]
 readme = "README.md"
 packages = [{ include = "autogpt_libs" }]
 
 [tool.poetry.dependencies]
-python = ">=3.10,<4.0"
+colorama = "^0.4.6"
+google-cloud-logging = "^3.8.0"
 pydantic = "^2.8.2"
+pydantic-settings = "^2.5.2"
 pyjwt = "^2.8.0"
+python = ">=3.10,<4.0"
 python-dotenv = "^1.0.1"
 supabase = "^2.7.2"
 
-
 [build-system]
 requires = ["poetry-core"]
 build-backend = "poetry.core.masonry.api"

rnd/autogpt_server/.env.example

@@ -51,3 +51,11 @@ SMTP_PASSWORD=
 # Medium
 MEDIUM_API_KEY=
 MEDIUM_AUTHOR_ID=
+
+
+# Logging Configuration
+LOG_LEVEL=INFO
+ENABLE_CLOUD_LOGGING=false
+ENABLE_FILE_LOGGING=false
+# Use to manually set the log directory
+# LOG_DIR=./logs

rnd/autogpt_server/Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.11-slim-buster AS server_base
+FROM python:3.11-slim-buster AS builder
 
 # Set environment variables
 ENV PYTHONDONTWRITEBYTECODE 1
@@ -6,15 +6,11 @@ ENV PYTHONUNBUFFERED 1
 
 WORKDIR /app
 
+# Install build dependencies
 RUN apt-get update \
-    && apt-get install -y build-essential curl ffmpeg wget libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev \
+    && apt-get install -y build-essential curl ffmpeg wget libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev postgresql-client git \
     && apt-get clean \
-    && rm -rf /var/lib/apt/lists/* \
-    && wget https://github.com/git/git/archive/v2.28.0.tar.gz -O git.tar.gz \
-    && tar -zxf git.tar.gz \
-    && cd git-* \
-    && make prefix=/usr all \
-    && make prefix=/usr install
+    && rm -rf /var/lib/apt/lists/*
 
 ENV POETRY_VERSION=1.8.3 \
     POETRY_HOME="/opt/poetry" \
@@ -23,32 +19,53 @@ ENV POETRY_VERSION=1.8.3 \
     PATH="$POETRY_HOME/bin:$PATH"
 RUN pip3 install poetry
 
-FROM server_base AS server_dependencies
+# Copy and install dependencies
+COPY rnd/autogpt_libs /app/rnd/autogpt_libs
+COPY rnd/autogpt_server/poetry.lock rnd/autogpt_server/pyproject.toml /app/rnd/autogpt_server/
+WORKDIR /app/rnd/autogpt_server
+RUN poetry config virtualenvs.create false \
+    && poetry install --no-interaction --no-ansi
+
+# Generate Prisma client
+COPY rnd/autogpt_server/schema.prisma ./
+RUN poetry config virtualenvs.create false \
+    && poetry run prisma generate
+
+FROM python:3.11-slim-buster AS server_dependencies
+
+WORKDIR /app
+
+ENV POETRY_VERSION=1.8.3 \
+    POETRY_HOME="/opt/poetry" \
+    POETRY_NO_INTERACTION=1 \
+    POETRY_VIRTUALENVS_CREATE=false \
+    PATH="$POETRY_HOME/bin:$PATH"
+
+# Copy only necessary files from builder
+COPY --from=builder /app /app
+COPY --from=builder /usr/local/lib/python3.11 /usr/local/lib/python3.11
+COPY --from=builder /usr/local/bin /usr/local/bin
+# Copy Prisma binaries
+COPY --from=builder /root/.cache/prisma-python/binaries /root/.cache/prisma-python/binaries
+
+
+ENV PATH="/app/.venv/bin:$PATH"
 
-RUN mkdir -p /app/autogpt
-RUN mkdir -p /app/forge
 RUN mkdir -p /app/rnd/autogpt_libs
 RUN mkdir -p /app/rnd/autogpt_server
 
-COPY autogpt /app/autogpt
-COPY forge /app/forge
 COPY rnd/autogpt_libs /app/rnd/autogpt_libs
 
 COPY rnd/autogpt_server/poetry.lock rnd/autogpt_server/pyproject.toml /app/rnd/autogpt_server/
 
 WORKDIR /app/rnd/autogpt_server
 
-RUN poetry install --no-interaction --no-ansi
-
-FROM server_dependencies AS server_prisma
-
-COPY rnd/autogpt_server/schema.prisma ./
-RUN poetry run prisma generate
-
-FROM server_prisma AS server
+FROM server_dependencies AS server
 
 COPY rnd/autogpt_server /app/rnd/autogpt_server
 
 ENV DATABASE_URL=""
 ENV PORT=8000
 
+CMD ["poetry", "run", "rest"]
+

rnd/autogpt_server/README.md

@@ -101,7 +101,7 @@ docker compose down
 If you run into issues with dangling orphans, try:
 
 ```sh
-docker-compose down --volumes --remove-orphans && docker-compose up --force-recreate --renew-anon-volumes --remove-orphans  
+docker compose down --volumes --remove-orphans && docker-compose up --force-recreate --renew-anon-volumes --remove-orphans  
 ```
 
 ## Testing

rnd/autogpt_server/autogpt_server/app.py

@@ -36,12 +36,5 @@ def main(**kwargs):
     )
 
 
-def execution_manager(**kwargs):
-
-    from autogpt_server.executor import ExecutionManager
-
-    run_processes(ExecutionManager(), **kwargs)
-
-
 if __name__ == "__main__":
     main()

rnd/autogpt_server/autogpt_server/blocks/__init__.py

@@ -55,15 +55,15 @@ for cls in all_subclasses(Block):
         raise ValueError(f"Block ID {block.name} error: {block.id} is already in use")
 
     # Prevent duplicate field name in input_schema and output_schema
-    duplicate_field_names = set(block.input_schema.__fields__.keys()) & set(
-        block.output_schema.__fields__.keys()
+    duplicate_field_names = set(block.input_schema.model_fields.keys()) & set(
+        block.output_schema.model_fields.keys()
     )
     if duplicate_field_names:
         raise ValueError(
             f"{block.name} has duplicate field names in input_schema and output_schema: {duplicate_field_names}"
         )
 
-    for field in block.input_schema.__fields__.values():
+    for field in block.input_schema.model_fields.values():
         if field.annotation is bool and field.default not in (True, False):
             raise ValueError(f"{block.name} has a boolean field with no default value")
 

rnd/autogpt_server/autogpt_server/blocks/agent.py

@@ -1,190 +0,0 @@
-from __future__ import annotations
-
-import asyncio
-import logging
-from pathlib import Path
-from typing import TYPE_CHECKING, Iterator
-
-from autogpt.agents.agent import Agent, AgentSettings
-from autogpt.app.config import ConfigBuilder
-from forge.agent.components import AgentComponent
-from forge.agent.protocols import CommandProvider
-from forge.command import command
-from forge.command.command import Command
-from forge.file_storage import FileStorageBackendName, get_storage
-from forge.file_storage.base import FileStorage
-from forge.llm.providers import MultiProvider
-from forge.llm.providers.openai import OpenAICredentials, OpenAIProvider
-from forge.llm.providers.schema import ModelProviderName
-from forge.models.json_schema import JSONSchema
-from pydantic import Field, SecretStr
-
-from autogpt_server.data.block import Block, BlockCategory, BlockOutput, BlockSchema
-from autogpt_server.data.model import BlockSecret, SchemaField, SecretField
-
-if TYPE_CHECKING:
-    from autogpt.app.config import AppConfig
-
-logger = logging.getLogger(__name__)
-
-
-class BlockAgentSettings(AgentSettings):
-    enabled_components: list[str] = Field(default_factory=list)
-
-
-class OutputComponent(CommandProvider):
-    def get_commands(self) -> Iterator[Command]:
-        yield self.output
-
-    @command(
-        parameters={
-            "output": JSONSchema(
-                type=JSONSchema.Type.STRING,
-                description="Output data to be returned.",
-                required=True,
-            ),
-        },
-    )
-    def output(self, output: str) -> str:
-        """Use this to output the result."""
-        return output
-
-
-class BlockAgent(Agent):
-    def __init__(
-        self,
-        settings: BlockAgentSettings,
-        llm_provider: MultiProvider,
-        file_storage: FileStorage,
-        app_config: AppConfig,
-    ):
-        super().__init__(settings, llm_provider, file_storage, app_config)
-
-        self.output = OutputComponent()
-
-        # Disable components
-        for attr_name in list(self.__dict__.keys()):
-            attr_value = getattr(self, attr_name)
-            if not isinstance(attr_value, AgentComponent):
-                continue
-            component_name = type(attr_value).__name__
-            if (
-                component_name != "SystemComponent"
-                and component_name not in settings.enabled_components
-            ):
-                delattr(self, attr_name)
-
-
-class AutoGPTAgentBlock(Block):
-    class Input(BlockSchema):
-        task: str = SchemaField(
-            description="Task description for the agent.",
-            placeholder="Calculate and use Output command",
-        )
-        input: str = SchemaField(
-            description="Input data for the task",
-            placeholder="8 + 5",
-        )
-        openai_api_key: BlockSecret = SecretField(
-            key="openai_api_key", description="OpenAI API key"
-        )
-        enabled_components: list[str] = Field(
-            default_factory=lambda: [OutputComponent.__name__],
-            description="List of [AgentComponents](https://docs.agpt.co/forge/components/built-in-components/) enabled for the agent.",
-        )
-        disabled_commands: list[str] = Field(
-            default_factory=list,
-            description="List of commands from enabled components to disable.",
-        )
-        fast_mode: bool = Field(
-            False,
-            description="If true uses fast llm, otherwise uses smart and slow llm.",
-        )
-
-    class Output(BlockSchema):
-        result: str
-
-    def __init__(self):
-        super().__init__(
-            id="d2e2ecd2-9ae6-422d-8dfe-ceca500ce6a6",
-            description="AutoGPT agent, it utilizes a Large Language Model and enabled components/tools to perform a task.",
-            categories={BlockCategory.AI},
-            input_schema=AutoGPTAgentBlock.Input,
-            output_schema=AutoGPTAgentBlock.Output,
-            test_input={
-                "task": "Make calculations and use output command to output the result",
-                "input": "5 + 3",
-                "openai_api_key": "openai_api_key",
-                "enabled_components": [OutputComponent.__name__],
-                "disabled_commands": ["finish"],
-                "fast_mode": True,
-            },
-            test_output=[
-                ("result", "8"),
-            ],
-            test_mock={
-                "get_provider": lambda _: MultiProvider(),
-                "get_result": lambda _: "8",
-            },
-        )
-
-    @staticmethod
-    def get_provider(openai_api_key: str) -> MultiProvider:
-        # LLM provider
-        settings = OpenAIProvider.default_settings.model_copy()
-        settings.credentials = OpenAICredentials(api_key=SecretStr(openai_api_key))
-        openai_provider = OpenAIProvider(settings=settings)
-
-        multi_provider = MultiProvider()
-        # HACK: Add OpenAI provider to the multi provider with api key
-        multi_provider._provider_instances[ModelProviderName.OPENAI] = openai_provider
-
-        return multi_provider
-
-    @staticmethod
-    def get_result(agent: BlockAgent) -> str:
-        error: Exception | None = None
-
-        for tries in range(3):
-            try:
-                proposal = asyncio.run(agent.propose_action())
-                result = asyncio.run(agent.execute(proposal))
-                return str(result)
-            except Exception as e:
-                error = e
-
-        raise error or Exception("Failed to get result")
-
-    def run(self, input_data: Input) -> BlockOutput:
-        # Set up configuration
-        config = ConfigBuilder.build_config_from_env()
-        # Disable commands
-        config.disabled_commands.extend(input_data.disabled_commands)
-
-        # Storage
-        local = config.file_storage_backend == FileStorageBackendName.LOCAL
-        restrict_to_root = not local or config.restrict_to_workspace
-        file_storage = get_storage(
-            config.file_storage_backend,
-            root_path=Path("data"),
-            restrict_to_root=restrict_to_root,
-        )
-        file_storage.initialize()
-
-        # State
-        state = BlockAgentSettings(
-            agent_id="TemporaryAgentID",
-            name="WrappedAgent",
-            description="Wrapped agent for the Agent Server.",
-            task=f"Your task: {input_data.task}\n" f"Input data: {input_data.input}",
-            enabled_components=input_data.enabled_components,
-        )
-        # Switch big brain mode
-        state.config.big_brain = not input_data.fast_mode
-        provider = self.get_provider(input_data.openai_api_key.get_secret_value())
-
-        agent = BlockAgent(state, provider, file_storage, config)
-
-        result = self.get_result(agent)
-
-        yield "result", result

rnd/autogpt_server/autogpt_server/blocks/basic.py

@@ -1,5 +1,7 @@
+import re
 from typing import Any, List
 
+from jinja2 import BaseLoader, Environment
 from pydantic import Field
 
 from autogpt_server.data.block import (
@@ -12,6 +14,8 @@ from autogpt_server.data.block import (
 from autogpt_server.data.model import SchemaField
 from autogpt_server.util.mock import MockObject
 
+jinja = Environment(loader=BaseLoader())
+
 
 class StoreValueBlock(Block):
     """
@@ -136,7 +140,7 @@ class FindInDictionaryBlock(Block):
             yield "missing", input_data.input
 
 
-class InputBlock(Block):
+class AgentInputBlock(Block):
     """
     This block is used to provide input to the graph.
 
@@ -148,13 +152,20 @@ class InputBlock(Block):
     class Input(BlockSchema):
         value: Any = SchemaField(description="The value to be passed as input.")
         name: str = SchemaField(description="The name of the input.")
-        description: str = SchemaField(description="The description of the input.")
+        description: str = SchemaField(
+            description="The description of the input.",
+            default="",
+            advanced=True,
+        )
         placeholder_values: List[Any] = SchemaField(
-            description="The placeholder values to be passed as input."
+            description="The placeholder values to be passed as input.",
+            default=[],
+            advanced=True,
         )
         limit_to_placeholder_values: bool = SchemaField(
             description="Whether to limit the selection to placeholder values.",
             default=False,
+            advanced=True,
         )
 
     class Output(BlockSchema):
@@ -164,8 +175,8 @@ class InputBlock(Block):
         super().__init__(
             id="c0a8e994-ebf1-4a9c-a4d8-89d09c86741b",
             description="This block is used to provide input to the graph.",
-            input_schema=InputBlock.Input,
-            output_schema=InputBlock.Output,
+            input_schema=AgentInputBlock.Input,
+            output_schema=AgentInputBlock.Output,
             test_input=[
                 {
                     "value": "Hello, World!",
@@ -194,7 +205,7 @@ class InputBlock(Block):
         yield "result", input_data.value
 
 
-class OutputBlock(Block):
+class AgentOutputBlock(Block):
     """
     Records the output of the graph for users to see.
 
@@ -215,13 +226,17 @@ class OutputBlock(Block):
     """
 
     class Input(BlockSchema):
-        recorded_value: Any = SchemaField(
-            description="The value to be recorded as output."
-        )
+        value: Any = SchemaField(description="The value to be recorded as output.")
         name: str = SchemaField(description="The name of the output.")
-        description: str = SchemaField(description="The description of the output.")
-        fmt_string: str = SchemaField(
-            description="The format string to be used to format the recorded_value."
+        description: str = SchemaField(
+            description="The description of the output.",
+            default="",
+            advanced=True,
+        )
+        format: str = SchemaField(
+            description="The format string to be used to format the recorded_value.",
+            default="",
+            advanced=True,
         )
 
     class Output(BlockSchema):
@@ -238,31 +253,31 @@ class OutputBlock(Block):
                 "This block is key for capturing and presenting final results or "
                 "important intermediate outputs of the graph execution."
             ),
-            input_schema=OutputBlock.Input,
-            output_schema=OutputBlock.Output,
+            input_schema=AgentOutputBlock.Input,
+            output_schema=AgentOutputBlock.Output,
             test_input=[
                 {
-                    "recorded_value": "Hello, World!",
+                    "value": "Hello, World!",
                     "name": "output_1",
                     "description": "This is a test output.",
-                    "fmt_string": "{value}",
+                    "format": "{{ output_1 }}!!",
                 },
                 {
-                    "recorded_value": 42,
+                    "value": "42",
                     "name": "output_2",
                     "description": "This is another test output.",
-                    "fmt_string": "{value}",
+                    "format": "{{ output_2 }}",
                 },
                 {
-                    "recorded_value": MockObject(value="!!", key="key"),
+                    "value": MockObject(value="!!", key="key"),
                     "name": "output_3",
                     "description": "This is a test output with a mock object.",
-                    "fmt_string": "{value}",
+                    "format": "{{ output_3 }}",
                 },
             ],
             test_output=[
-                ("output", "Hello, World!"),
-                ("output", 42),
+                ("output", "Hello, World!!!"),
+                ("output", "42"),
                 ("output", MockObject(value="!!", key="key")),
             ],
             categories={BlockCategory.OUTPUT, BlockCategory.BASIC},
@@ -274,13 +289,15 @@ class OutputBlock(Block):
         Attempts to format the recorded_value using the fmt_string if provided.
         If formatting fails or no fmt_string is given, returns the original recorded_value.
         """
-        if input_data.fmt_string:
+        if input_data.format:
             try:
-                yield "output", input_data.fmt_string.format(input_data.recorded_value)
-            except Exception:
-                yield "output", input_data.recorded_value
+                fmt = re.sub(r"(?<!{){[ a-zA-Z0-9_]+}", r"{\g<0>}", input_data.format)
+                template = jinja.from_string(fmt)
+                yield "output", template.render({input_data.name: input_data.value})
+            except Exception as e:
+                yield "output", f"Error: {e}, {input_data.value}"
         else:
-            yield "output", input_data.recorded_value
+            yield "output", input_data.value
 
 
 class AddToDictionaryBlock(Block):
@@ -422,7 +439,8 @@ class NoteBlock(Block):
     class Input(BlockSchema):
         text: str = SchemaField(description="The text to display in the sticky note.")
 
-    class Output(BlockSchema): ...
+    class Output(BlockSchema):
+        output: str = SchemaField(description="The text to display in the sticky note.")
 
     def __init__(self):
         super().__init__(
@@ -432,8 +450,11 @@ class NoteBlock(Block):
             input_schema=NoteBlock.Input,
             output_schema=NoteBlock.Output,
             test_input={"text": "Hello, World!"},
-            test_output=None,
+            test_output=[
+                ("output", "Hello, World!"),
+            ],
             ui_type=BlockUIType.NOTE,
         )
 
-    def run(self, input_data: Input) -> BlockOutput: ...
+    def run(self, input_data: Input) -> BlockOutput:
+        yield "output", input_data.text

rnd/autogpt_server/autogpt_server/blocks/llm.py

@@ -438,7 +438,7 @@ class Message(BlockSchema):
 class AIConversationBlock(Block):
     class Input(BlockSchema):
         messages: List[Message] = SchemaField(
-            description="List of messages in the conversation.", min_items=1
+            description="List of messages in the conversation.", min_length=1
         )
         model: LlmModel = SchemaField(
             default=LlmModel.GPT4_TURBO,

rnd/autogpt_server/autogpt_server/data/db.py

@@ -31,7 +31,7 @@ async def connect(call_count=0):
     except Exception as e:
         if call_count <= 5:
             logger.info(f"[Prisma-{conn_id}] Connection failed: {e}. Retrying now..")
-            await asyncio.sleep(call_count)
+            await asyncio.sleep(2**call_count)
             await connect(call_count + 1)
         else:
             raise e

rnd/autogpt_server/autogpt_server/data/graph.py

@@ -9,7 +9,7 @@ from prisma.models import AgentGraph, AgentNode, AgentNodeLink
 from pydantic import BaseModel, PrivateAttr
 from pydantic_core import PydanticUndefinedType
 
-from autogpt_server.blocks.basic import InputBlock, OutputBlock
+from autogpt_server.blocks.basic import AgentInputBlock, AgentOutputBlock
 from autogpt_server.data.block import BlockInput, get_block, get_blocks
 from autogpt_server.data.db import BaseDbModel, transaction
 from autogpt_server.data.user import DEFAULT_USER_ID
@@ -106,7 +106,9 @@ class Graph(GraphMeta):
     def starting_nodes(self) -> list[Node]:
         outbound_nodes = {link.sink_id for link in self.links}
         input_nodes = {
-            v.id for v in self.nodes if isinstance(get_block(v.block_id), InputBlock)
+            v.id
+            for v in self.nodes
+            if isinstance(get_block(v.block_id), AgentInputBlock)
         }
         return [
             node
@@ -116,7 +118,9 @@ class Graph(GraphMeta):
 
     @property
     def ending_nodes(self) -> list[Node]:
-        return [v for v in self.nodes if isinstance(get_block(v.block_id), OutputBlock)]
+        return [
+            v for v in self.nodes if isinstance(get_block(v.block_id), AgentOutputBlock)
+        ]
 
     @property
     def subgraph_map(self) -> dict[str, str]:
@@ -179,7 +183,9 @@ class Graph(GraphMeta):
                 + [sanitize(link.sink_name) for link in node.input_links]
             )
             for name in block.input_schema.get_required_fields():
-                if name not in provided_inputs and not isinstance(block, InputBlock):
+                if name not in provided_inputs and not isinstance(
+                    block, AgentInputBlock
+                ):
                     raise ValueError(
                         f"Node {block.name} #{node.id} required input missing: `{name}`"
                     )
@@ -193,7 +199,7 @@ class Graph(GraphMeta):
         def is_input_output_block(nid: str) -> bool:
             bid = node_map[nid].block_id
             b = get_block(bid)
-            return isinstance(b, InputBlock) or isinstance(b, OutputBlock)
+            return isinstance(b, AgentInputBlock) or isinstance(b, AgentOutputBlock)
 
         # subgraphs: all nodes in subgraph must be present in the graph.
         for subgraph_id, node_ids in self.subgraphs.items():

rnd/autogpt_server/autogpt_server/exec.py

@@ -0,0 +1,15 @@
+from autogpt_server.app import run_processes
+from autogpt_server.executor import ExecutionManager
+
+
+def main():
+    """
+    Run all the processes required for the AutoGPT-server REST API.
+    """
+    run_processes(
+        ExecutionManager(),
+    )
+
+
+if __name__ == "__main__":
+    main()

rnd/autogpt_server/autogpt_server/executor/manager.py

@@ -14,7 +14,7 @@ from typing import TYPE_CHECKING, Any, Coroutine, Generator, TypeVar
 if TYPE_CHECKING:
     from autogpt_server.server.rest_api import AgentServer
 
-from autogpt_server.blocks.basic import InputBlock
+from autogpt_server.blocks.basic import AgentInputBlock
 from autogpt_server.data import db
 from autogpt_server.data.block import Block, BlockData, BlockInput, get_block
 from autogpt_server.data.execution import (
@@ -62,6 +62,10 @@ def get_log_metadata(
     }
 
 
+def get_log_prefix(graph_eid: str, node_eid: str, block_name: str = "-"):
+    return f"[ExecutionManager][graph-eid-{graph_eid}|node-eid-{node_eid}|{block_name}]"
+
+
 T = TypeVar("T")
 ExecutionStream = Generator[NodeExecution, None, None]
 
@@ -114,10 +118,15 @@ def execute_node(
         node_id=node_id,
         block_name=node_block.name,
     )
+    prefix = get_log_prefix(
+        graph_eid=graph_exec_id,
+        node_eid=node_exec_id,
+        block_name=node_block.name,
+    )
     input_data, error = validate_exec(node, data.data, resolve_input=False)
     if input_data is None:
         logger.error(
-            "Skip execution, input validation error",
+            "{prefix} Skip execution, input validation error",
             extra={"json_fields": {**log_metadata, "error": error}},
         )
         return
@@ -126,7 +135,7 @@ def execute_node(
     input_data_str = json.dumps(input_data)
     input_size = len(input_data_str)
     logger.info(
-        "Executed node with input",
+        f"{prefix} Executed node with input",
         extra={"json_fields": {**log_metadata, "input": input_data_str}},
     )
     update_execution(ExecutionStatus.RUNNING)
@@ -136,7 +145,7 @@ def execute_node(
         for output_name, output_data in node_block.execute(input_data):
             output_size += len(json.dumps(output_data))
             logger.info(
-                "Node produced output",
+                f"{prefix} Node produced output",
                 extra={"json_fields": {**log_metadata, output_name: output_data}},
             )
             wait(upsert_execution_output(node_exec_id, output_name, output_data))
@@ -157,7 +166,7 @@ def execute_node(
     except Exception as e:
         error_msg = f"{e.__class__.__name__}: {e}"
         logger.exception(
-            "Node execution failed with error",
+            f"{prefix} Node execution failed with error",
             extra={"json_fields": {**log_metadata, error: error_msg}},
         )
         wait(upsert_execution_output(node_exec_id, "error", error_msg))
@@ -441,10 +450,15 @@ class Executor:
             node_id=node_exec.node_id,
             block_name="-",
         )
+        prefix = get_log_prefix(
+            graph_eid=node_exec.graph_exec_id,
+            node_eid=node_exec.node_exec_id,
+            block_name="-",
+        )
 
         execution_stats = {}
         timing_info, _ = cls._on_node_execution(
-            q, node_exec, log_metadata, execution_stats
+            q, node_exec, log_metadata, prefix, execution_stats
         )
         execution_stats["walltime"] = timing_info.wall_time
         execution_stats["cputime"] = timing_info.cpu_time
@@ -460,11 +474,12 @@ class Executor:
         q: ExecutionQueue[NodeExecution],
         node_exec: NodeExecution,
         log_metadata: dict,
+        prefix: str,
         stats: dict[str, Any] | None = None,
     ):
         try:
             logger.info(
-                f"Start node execution {node_exec.node_exec_id}",
+                f"{prefix} Start node execution {node_exec.node_exec_id}",
                 extra={"json_fields": {**log_metadata}},
             )
             for execution in execute_node(
@@ -472,7 +487,7 @@ class Executor:
             ):
                 q.add(execution)
             logger.info(
-                f"Finished node execution {node_exec.node_exec_id}",
+                f"{prefix} Finished node execution {node_exec.node_exec_id}",
                 extra={"json_fields": {**log_metadata}},
             )
         except Exception as e:
@@ -524,8 +539,13 @@ class Executor:
             node_eid="*",
             block_name="-",
         )
+        prefix = get_log_prefix(
+            graph_eid=graph_exec.graph_exec_id,
+            node_eid="*",
+            block_name="-",
+        )
         timing_info, node_count = cls._on_graph_execution(
-            graph_exec, cancel, log_metadata
+            graph_exec, cancel, log_metadata, prefix
         )
 
         cls.loop.run_until_complete(
@@ -542,10 +562,14 @@ class Executor:
     @classmethod
     @time_measured
     def _on_graph_execution(
-        cls, graph_exec: GraphExecution, cancel: threading.Event, log_metadata: dict
+        cls,
+        graph_exec: GraphExecution,
+        cancel: threading.Event,
+        log_metadata: dict,
+        prefix: str,
     ) -> int:
         logger.info(
-            f"Start graph execution {graph_exec.graph_exec_id}",
+            f"{prefix} Start graph execution {graph_exec.graph_exec_id}",
             extra={"json_fields": {**log_metadata}},
         )
         n_node_executions = 0
@@ -558,7 +582,7 @@ class Executor:
                 return
             cls.executor.terminate()
             logger.info(
-                f"Terminated graph execution {graph_exec.graph_exec_id}",
+                f"{prefix} Terminated graph execution {graph_exec.graph_exec_id}",
                 extra={"json_fields": {**log_metadata}},
             )
             cls._init_node_executor_pool()
@@ -599,7 +623,7 @@ class Executor:
                     execution.wait()
 
                 logger.debug(
-                    f"Dispatching node execution {exec_data.node_exec_id} "
+                    f"{prefix} Dispatching node execution {exec_data.node_exec_id} "
                     f"for node {exec_data.node_id}",
                     extra={**log_metadata},
                 )
@@ -630,12 +654,12 @@ class Executor:
                         execution.wait(3)
 
             logger.info(
-                f"Finished graph execution {graph_exec.graph_exec_id}",
+                f"{prefix} Finished graph execution {graph_exec.graph_exec_id}",
                 extra={"json_fields": {**log_metadata}},
             )
         except Exception as e:
             logger.exception(
-                f"Failed graph execution {graph_exec.graph_exec_id}: {e}",
+                f"{prefix} Failed graph execution {graph_exec.graph_exec_id}: {e}",
                 extra={"json_fields": {**log_metadata}},
             )
         finally:
@@ -699,7 +723,7 @@ class ExecutionManager(AppService):
         nodes_input = []
         for node in graph.starting_nodes:
             input_data = {}
-            if isinstance(get_block(node.block_id), InputBlock):
+            if isinstance(get_block(node.block_id), AgentInputBlock):
                 name = node.input_default.get("name")
                 if name and name in data:
                     input_data = {"value": data[name]}

rnd/autogpt_server/autogpt_server/integrations/oauth/__init__.py

@@ -0,0 +1,15 @@
+from .base import BaseOAuthHandler
+from .github import GitHubOAuthHandler
+from .google import GoogleOAuthHandler
+from .notion import NotionOAuthHandler
+
+HANDLERS_BY_NAME: dict[str, type[BaseOAuthHandler]] = {
+    handler.PROVIDER_NAME: handler
+    for handler in [
+        GitHubOAuthHandler,
+        GoogleOAuthHandler,
+        NotionOAuthHandler,
+    ]
+}
+
+__all__ = ["HANDLERS_BY_NAME"]

rnd/autogpt_server/autogpt_server/integrations/oauth/base.py

@@ -0,0 +1,48 @@
+import time
+from abc import ABC, abstractmethod
+from typing import ClassVar
+
+from autogpt_libs.supabase_integration_credentials_store import OAuth2Credentials
+
+
+class BaseOAuthHandler(ABC):
+    PROVIDER_NAME: ClassVar[str]
+
+    @abstractmethod
+    def __init__(self, client_id: str, client_secret: str, redirect_uri: str): ...
+
+    @abstractmethod
+    def get_login_url(self, scopes: list[str], state: str) -> str:
+        """Constructs a login URL that the user can be redirected to"""
+        ...
+
+    @abstractmethod
+    def exchange_code_for_tokens(self, code: str) -> OAuth2Credentials:
+        """Exchanges the acquired authorization code from login for a set of tokens"""
+        ...
+
+    @abstractmethod
+    def _refresh_tokens(self, credentials: OAuth2Credentials) -> OAuth2Credentials:
+        """Implements the token refresh mechanism"""
+        ...
+
+    def refresh_tokens(self, credentials: OAuth2Credentials) -> OAuth2Credentials:
+        if credentials.provider != self.PROVIDER_NAME:
+            raise ValueError(
+                f"{self.__class__.__name__} can not refresh tokens "
+                f"for other provider '{credentials.provider}'"
+            )
+        return self._refresh_tokens(credentials)
+
+    def get_access_token(self, credentials: OAuth2Credentials) -> str:
+        """Returns a valid access token, refreshing it first if needed"""
+        if self.needs_refresh(credentials):
+            credentials = self.refresh_tokens(credentials)
+        return credentials.access_token.get_secret_value()
+
+    def needs_refresh(self, credentials: OAuth2Credentials) -> bool:
+        """Indicates whether the given tokens need to be refreshed"""
+        return (
+            credentials.access_token_expires_at is not None
+            and credentials.access_token_expires_at < int(time.time()) + 300
+        )

rnd/autogpt_server/autogpt_server/integrations/oauth/github.py

@@ -0,0 +1,99 @@
+import time
+from typing import Optional
+from urllib.parse import urlencode
+
+import requests
+from autogpt_libs.supabase_integration_credentials_store import OAuth2Credentials
+
+from .base import BaseOAuthHandler
+
+
+class GitHubOAuthHandler(BaseOAuthHandler):
+    """
+    Based on the documentation at:
+    - [Authorizing OAuth apps - GitHub Docs](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/authorizing-oauth-apps)
+    - [Refreshing user access tokens - GitHub Docs](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/refreshing-user-access-tokens)
+
+    Notes:
+    - By default, token expiration is disabled on GitHub Apps. This means the access
+      token doesn't expire and no refresh token is returned by the authorization flow.
+    - When token expiration gets enabled, any existing tokens will remain non-expiring.
+    - When token expiration gets disabled, token refreshes will return a non-expiring
+      access token *with no refresh token*.
+    """  # noqa
+
+    PROVIDER_NAME = "github"
+
+    def __init__(self, client_id: str, client_secret: str, redirect_uri: str):
+        self.client_id = client_id
+        self.client_secret = client_secret
+        self.redirect_uri = redirect_uri
+        self.auth_base_url = "https://github.com/login/oauth/authorize"
+        self.token_url = "https://github.com/login/oauth/access_token"
+
+    def get_login_url(self, scopes: list[str], state: str) -> str:
+        params = {
+            "client_id": self.client_id,
+            "redirect_uri": self.redirect_uri,
+            "scope": " ".join(scopes),
+            "state": state,
+        }
+        return f"{self.auth_base_url}?{urlencode(params)}"
+
+    def exchange_code_for_tokens(self, code: str) -> OAuth2Credentials:
+        return self._request_tokens({"code": code, "redirect_uri": self.redirect_uri})
+
+    def _refresh_tokens(self, credentials: OAuth2Credentials) -> OAuth2Credentials:
+        if not credentials.refresh_token:
+            return credentials
+
+        return self._request_tokens(
+            {
+                "refresh_token": credentials.refresh_token.get_secret_value(),
+                "grant_type": "refresh_token",
+            }
+        )
+
+    def _request_tokens(
+        self,
+        params: dict[str, str],
+        current_credentials: Optional[OAuth2Credentials] = None,
+    ) -> OAuth2Credentials:
+        request_body = {
+            "client_id": self.client_id,
+            "client_secret": self.client_secret,
+            **params,
+        }
+        headers = {"Accept": "application/json"}
+        response = requests.post(self.token_url, data=request_body, headers=headers)
+        response.raise_for_status()
+        token_data: dict = response.json()
+
+        now = int(time.time())
+        new_credentials = OAuth2Credentials(
+            provider=self.PROVIDER_NAME,
+            title=current_credentials.title if current_credentials else "GitHub",
+            access_token=token_data["access_token"],
+            # Token refresh responses have an empty `scope` property (see docs),
+            # so we have to get the scope from the existing credentials object.
+            scopes=(
+                token_data.get("scope", "").split(",")
+                or (current_credentials.scopes if current_credentials else [])
+            ),
+            # Refresh token and expiration intervals are only given if token expiration
+            # is enabled in the GitHub App's settings.
+            refresh_token=token_data.get("refresh_token"),
+            access_token_expires_at=(
+                now + expires_in
+                if (expires_in := token_data.get("expires_in", None))
+                else None
+            ),
+            refresh_token_expires_at=(
+                now + expires_in
+                if (expires_in := token_data.get("refresh_token_expires_in", None))
+                else None
+            ),
+        )
+        if current_credentials:
+            new_credentials.id = current_credentials.id
+        return new_credentials

rnd/autogpt_server/autogpt_server/integrations/oauth/google.py

@@ -0,0 +1,96 @@
+from autogpt_libs.supabase_integration_credentials_store import OAuth2Credentials
+from google.auth.transport.requests import Request
+from google.oauth2.credentials import Credentials
+from google_auth_oauthlib.flow import Flow
+from pydantic import SecretStr
+
+from .base import BaseOAuthHandler
+
+
+class GoogleOAuthHandler(BaseOAuthHandler):
+    """
+    Based on the documentation at https://developers.google.com/identity/protocols/oauth2/web-server
+    """  # noqa
+
+    PROVIDER_NAME = "google"
+
+    def __init__(self, client_id: str, client_secret: str, redirect_uri: str):
+        self.client_id = client_id
+        self.client_secret = client_secret
+        self.redirect_uri = redirect_uri
+        self.token_uri = "https://oauth2.googleapis.com/token"
+
+    def get_login_url(self, scopes: list[str], state: str) -> str:
+        flow = self._setup_oauth_flow(scopes)
+        flow.redirect_uri = self.redirect_uri
+        authorization_url, _ = flow.authorization_url(
+            access_type="offline",
+            include_granted_scopes="true",
+            state=state,
+            prompt="consent",
+        )
+        return authorization_url
+
+    def exchange_code_for_tokens(self, code: str) -> OAuth2Credentials:
+        flow = self._setup_oauth_flow(None)
+        flow.redirect_uri = self.redirect_uri
+        flow.fetch_token(code=code)
+
+        google_creds = flow.credentials
+        # Google's OAuth library is poorly typed so we need some of these:
+        assert google_creds.token
+        assert google_creds.refresh_token
+        assert google_creds.expiry
+        assert google_creds.scopes
+        return OAuth2Credentials(
+            provider=self.PROVIDER_NAME,
+            title="Google",
+            access_token=SecretStr(google_creds.token),
+            refresh_token=SecretStr(google_creds.refresh_token),
+            access_token_expires_at=int(google_creds.expiry.timestamp()),
+            refresh_token_expires_at=None,
+            scopes=google_creds.scopes,
+        )
+
+    def _refresh_tokens(self, credentials: OAuth2Credentials) -> OAuth2Credentials:
+        # Google credentials should ALWAYS have a refresh token
+        assert credentials.refresh_token
+
+        google_creds = Credentials(
+            token=credentials.access_token.get_secret_value(),
+            refresh_token=credentials.refresh_token.get_secret_value(),
+            token_uri=self.token_uri,
+            client_id=self.client_id,
+            client_secret=self.client_secret,
+            scopes=credentials.scopes,
+        )
+        # Google's OAuth library is poorly typed so we need some of these:
+        assert google_creds.refresh_token
+        assert google_creds.scopes
+
+        google_creds.refresh(Request())
+        assert google_creds.expiry
+
+        return OAuth2Credentials(
+            id=credentials.id,
+            provider=self.PROVIDER_NAME,
+            title=credentials.title,
+            access_token=SecretStr(google_creds.token),
+            refresh_token=SecretStr(google_creds.refresh_token),
+            access_token_expires_at=int(google_creds.expiry.timestamp()),
+            refresh_token_expires_at=None,
+            scopes=google_creds.scopes,
+        )
+
+    def _setup_oauth_flow(self, scopes: list[str] | None) -> Flow:
+        return Flow.from_client_config(
+            {
+                "web": {
+                    "client_id": self.client_id,
+                    "client_secret": self.client_secret,
+                    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
+                    "token_uri": self.token_uri,
+                }
+            },
+            scopes=scopes,
+        )

rnd/autogpt_server/autogpt_server/integrations/oauth/notion.py

@@ -0,0 +1,76 @@
+from base64 import b64encode
+from urllib.parse import urlencode
+
+import requests
+from autogpt_libs.supabase_integration_credentials_store import OAuth2Credentials
+
+from .base import BaseOAuthHandler
+
+
+class NotionOAuthHandler(BaseOAuthHandler):
+    """
+    Based on the documentation at https://developers.notion.com/docs/authorization
+
+    Notes:
+    - Notion uses non-expiring access tokens and therefore doesn't have a refresh flow
+    - Notion doesn't use scopes
+    """
+
+    PROVIDER_NAME = "notion"
+
+    def __init__(self, client_id: str, client_secret: str, redirect_uri: str):
+        self.client_id = client_id
+        self.client_secret = client_secret
+        self.redirect_uri = redirect_uri
+        self.auth_base_url = "https://api.notion.com/v1/oauth/authorize"
+        self.token_url = "https://api.notion.com/v1/oauth/token"
+
+    def get_login_url(self, scopes: list[str], state: str) -> str:
+        params = {
+            "client_id": self.client_id,
+            "redirect_uri": self.redirect_uri,
+            "response_type": "code",
+            "owner": "user",
+            "state": state,
+        }
+        return f"{self.auth_base_url}?{urlencode(params)}"
+
+    def exchange_code_for_tokens(self, code: str) -> OAuth2Credentials:
+        request_body = {
+            "grant_type": "authorization_code",
+            "code": code,
+            "redirect_uri": self.redirect_uri,
+        }
+        auth_str = b64encode(f"{self.client_id}:{self.client_secret}".encode()).decode()
+        headers = {
+            "Authorization": f"Basic {auth_str}",
+            "Accept": "application/json",
+        }
+        response = requests.post(self.token_url, json=request_body, headers=headers)
+        response.raise_for_status()
+        token_data = response.json()
+
+        return OAuth2Credentials(
+            provider=self.PROVIDER_NAME,
+            title=token_data.get("workspace_name", "Notion"),
+            access_token=token_data["access_token"],
+            refresh_token=None,
+            access_token_expires_at=None,  # Notion tokens don't expire
+            refresh_token_expires_at=None,
+            scopes=[],
+            metadata={
+                "owner": token_data["owner"],
+                "bot_id": token_data["bot_id"],
+                "workspace_id": token_data["workspace_id"],
+                "workspace_name": token_data.get("workspace_name"),
+                "workspace_icon": token_data.get("workspace_icon"),
+            },
+        )
+
+    def _refresh_tokens(self, credentials: OAuth2Credentials) -> OAuth2Credentials:
+        # Notion doesn't support token refresh
+        return credentials
+
+    def needs_refresh(self, credentials: OAuth2Credentials) -> bool:
+        # Notion access tokens don't expire
+        return False

rnd/autogpt_server/autogpt_server/rest.py

@@ -1,5 +1,5 @@
 from autogpt_server.app import run_processes
-from autogpt_server.executor import ExecutionManager, ExecutionScheduler
+from autogpt_server.executor import ExecutionScheduler
 from autogpt_server.server import AgentServer
 
 
@@ -8,7 +8,6 @@ def main():
     Run all the processes required for the AutoGPT-server REST API.
     """
     run_processes(
-        ExecutionManager(),
         ExecutionScheduler(),
         AgentServer(),
     )

rnd/autogpt_server/autogpt_server/server/integrations.py

@@ -0,0 +1,105 @@
+import logging
+from typing import Annotated, Literal
+
+from autogpt_libs.supabase_integration_credentials_store import (
+    SupabaseIntegrationCredentialsStore,
+)
+from fastapi import APIRouter, Body, Depends, HTTPException, Path, Query, Request
+from pydantic import BaseModel
+from supabase import Client
+
+from autogpt_server.integrations.oauth import HANDLERS_BY_NAME, BaseOAuthHandler
+from autogpt_server.util.settings import Settings
+
+from .utils import get_supabase, get_user_id
+
+logger = logging.getLogger(__name__)
+settings = Settings()
+integrations_api_router = APIRouter()
+
+
+def get_store(supabase: Client = Depends(get_supabase)):
+    return SupabaseIntegrationCredentialsStore(supabase)
+
+
+class LoginResponse(BaseModel):
+    login_url: str
+
+
+@integrations_api_router.get("/{provider}/login")
+async def login(
+    provider: Annotated[str, Path(title="The provider to initiate an OAuth flow for")],
+    user_id: Annotated[str, Depends(get_user_id)],
+    request: Request,
+    store: Annotated[SupabaseIntegrationCredentialsStore, Depends(get_store)],
+    scopes: Annotated[
+        str, Query(title="Comma-separated list of authorization scopes")
+    ] = "",
+) -> LoginResponse:
+    handler = _get_provider_oauth_handler(request, provider)
+
+    # Generate and store a secure random state token
+    state = await store.store_state_token(user_id, provider)
+
+    requested_scopes = scopes.split(",") if scopes else []
+    login_url = handler.get_login_url(requested_scopes, state)
+
+    return LoginResponse(login_url=login_url)
+
+
+class CredentialsMetaResponse(BaseModel):
+    credentials_id: str
+    credentials_type: Literal["oauth2", "api_key"]
+
+
+@integrations_api_router.post("/{provider}/callback")
+async def callback(
+    provider: Annotated[str, Path(title="The target provider for this OAuth exchange")],
+    code: Annotated[str, Body(title="Authorization code acquired by user login")],
+    state_token: Annotated[str, Body(title="Anti-CSRF nonce")],
+    store: Annotated[SupabaseIntegrationCredentialsStore, Depends(get_store)],
+    user_id: Annotated[str, Depends(get_user_id)],
+    request: Request,
+) -> CredentialsMetaResponse:
+    handler = _get_provider_oauth_handler(request, provider)
+
+    # Verify the state token
+    if not await store.verify_state_token(user_id, state_token, provider):
+        raise HTTPException(status_code=400, detail="Invalid or expired state token")
+
+    try:
+        credentials = handler.exchange_code_for_tokens(code)
+    except Exception as e:
+        logger.warning(f"Code->Token exchange failed for provider {provider}: {e}")
+        raise HTTPException(status_code=400, detail=str(e))
+
+    store.add_creds(user_id, credentials)
+    return CredentialsMetaResponse(
+        credentials_id=credentials.id,
+        credentials_type=credentials.type,
+    )
+
+
+# -------- UTILITIES --------- #
+
+
+def _get_provider_oauth_handler(req: Request, provider_name: str) -> BaseOAuthHandler:
+    if provider_name not in HANDLERS_BY_NAME:
+        raise HTTPException(
+            status_code=404, detail=f"Unknown provider '{provider_name}'"
+        )
+
+    client_id = getattr(settings.secrets, f"{provider_name}_client_id")
+    client_secret = getattr(settings.secrets, f"{provider_name}_client_secret")
+    if not (client_id and client_secret):
+        raise HTTPException(
+            status_code=501,
+            detail=f"Integration with provider '{provider_name}' is not configured",
+        )
+
+    handler_class = HANDLERS_BY_NAME[provider_name]
+    return handler_class(
+        client_id=client_id,
+        client_secret=client_secret,
+        redirect_uri=str(req.url_for("callback", provider=provider_name)),
+    )

rnd/autogpt_server/autogpt_server/server/rest_api.py

@@ -19,11 +19,12 @@ from autogpt_server.data.queue import AsyncEventQueue, AsyncRedisEventQueue
 from autogpt_server.data.user import get_or_create_user
 from autogpt_server.executor import ExecutionManager, ExecutionScheduler
 from autogpt_server.server.model import CreateGraph, SetGraphActiveVersion
-from autogpt_server.util.auth import get_user_id
 from autogpt_server.util.lock import KeyedMutex
 from autogpt_server.util.service import AppService, expose, get_service_client
 from autogpt_server.util.settings import Config, Settings
 
+from .utils import get_user_id
+
 settings = Settings()
 
 
@@ -71,127 +72,132 @@ class AgentServer(AppService):
         )
 
         # Define the API routes
-        router = APIRouter(prefix="/api")
-        router.dependencies.append(Depends(auth_middleware))
+        api_router = APIRouter(prefix="/api")
+        api_router.dependencies.append(Depends(auth_middleware))
 
-        router.add_api_route(
+        # Import & Attach sub-routers
+        from .integrations import integrations_api_router
+
+        api_router.include_router(integrations_api_router, prefix="/integrations")
+
+        api_router.add_api_route(
             path="/auth/user",
             endpoint=self.get_or_create_user_route,
             methods=["POST"],
         )
 
-        router.add_api_route(
+        api_router.add_api_route(
             path="/blocks",
             endpoint=self.get_graph_blocks,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/blocks/{block_id}/execute",
             endpoint=self.execute_graph_block,
             methods=["POST"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs",
             endpoint=self.get_graphs,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/templates",
             endpoint=self.get_templates,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs",
             endpoint=self.create_new_graph,
             methods=["POST"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/templates",
             endpoint=self.create_new_template,
             methods=["POST"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}",
             endpoint=self.get_graph,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/templates/{graph_id}",
             endpoint=self.get_template,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}",
             endpoint=self.update_graph,
             methods=["PUT"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/templates/{graph_id}",
             endpoint=self.update_graph,
             methods=["PUT"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/versions",
             endpoint=self.get_graph_all_versions,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/templates/{graph_id}/versions",
             endpoint=self.get_graph_all_versions,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/versions/{version}",
             endpoint=self.get_graph,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/versions/active",
             endpoint=self.set_graph_active_version,
             methods=["PUT"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/input_schema",
             endpoint=self.get_graph_input_schema,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/execute",
             endpoint=self.execute_graph,
             methods=["POST"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/executions",
             endpoint=self.list_graph_runs,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/executions/{graph_exec_id}",
             endpoint=self.get_graph_run_node_execution_results,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/executions/{graph_exec_id}/stop",
             endpoint=self.stop_graph_run,
             methods=["POST"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/schedules",
             endpoint=self.create_schedule,
             methods=["POST"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/{graph_id}/schedules",
             endpoint=self.get_execution_schedules,
             methods=["GET"],
         )
-        router.add_api_route(
+        api_router.add_api_route(
             path="/graphs/schedules/{schedule_id}",
             endpoint=self.update_schedule,
             methods=["PUT"],
         )
 
-        router.add_api_route(
+        api_router.add_api_route(
             path="/settings",
             endpoint=self.update_configuration,
             methods=["POST"],
@@ -199,7 +205,7 @@ class AgentServer(AppService):
 
         app.add_exception_handler(500, self.handle_internal_http_error)
 
-        app.include_router(router)
+        app.include_router(api_router)
 
         uvicorn.run(app, host="0.0.0.0", port=8000, log_config=None)
 

rnd/autogpt_server/autogpt_server/server/utils.py

@@ -1,7 +1,11 @@
-from autogpt_libs.auth import auth_middleware
+from autogpt_libs.auth.middleware import auth_middleware
 from fastapi import Depends, HTTPException
+from supabase import Client, create_client
 
 from autogpt_server.data.user import DEFAULT_USER_ID
+from autogpt_server.util.settings import Settings
+
+settings = Settings()
 
 
 def get_user_id(payload: dict = Depends(auth_middleware)) -> str:
@@ -13,3 +17,7 @@ def get_user_id(payload: dict = Depends(auth_middleware)) -> str:
     if not user_id:
         raise HTTPException(status_code=401, detail="User ID not found in token")
     return user_id
+
+
+def get_supabase() -> Client:
+    return create_client(settings.secrets.supabase_url, settings.secrets.supabase_key)

rnd/autogpt_server/autogpt_server/usecases/sample.py

@@ -1,6 +1,6 @@
 from prisma.models import User
 
-from autogpt_server.blocks.basic import InputBlock, PrintToConsoleBlock
+from autogpt_server.blocks.basic import AgentInputBlock, PrintToConsoleBlock
 from autogpt_server.blocks.text import FillTextTemplateBlock
 from autogpt_server.data import graph
 from autogpt_server.data.graph import create_graph
@@ -28,22 +28,12 @@ def create_test_graph() -> graph.Graph:
     """
     nodes = [
         graph.Node(
-            block_id=InputBlock().id,
-            input_default={
-                "name": "input_1",
-                "description": "First input value",
-                "placeholder_values": [],
-                "limit_to_placeholder_values": False,
-            },
+            block_id=AgentInputBlock().id,
+            input_default={"name": "input_1"},
         ),
         graph.Node(
-            block_id=InputBlock().id,
-            input_default={
-                "name": "input_2",
-                "description": "Second input value",
-                "placeholder_values": [],
-                "limit_to_placeholder_values": False,
-            },
+            block_id=AgentInputBlock().id,
+            input_default={"name": "input_2"},
         ),
         graph.Node(
             block_id=FillTextTemplateBlock().id,

rnd/autogpt_server/autogpt_server/util/decorator.py

@@ -2,7 +2,7 @@ import functools
 import logging
 import os
 import time
-from typing import Callable, Tuple, TypeVar
+from typing import Callable, ParamSpec, Tuple, TypeVar
 
 from pydantic import BaseModel
 
@@ -24,18 +24,19 @@ def _end_measurement(
     return end_wall_time - start_wall_time, end_cpu_time - start_cpu_time
 
 
+P = ParamSpec("P")
 T = TypeVar("T")
 
 logger = logging.getLogger(__name__)
 
 
-def time_measured(func: Callable[..., T]) -> Callable[..., Tuple[TimingInfo, T]]:
+def time_measured(func: Callable[P, T]) -> Callable[P, Tuple[TimingInfo, T]]:
     """
     Decorator to measure the time taken by a function to execute.
     """
 
     @functools.wraps(func)
-    def wrapper(*args, **kwargs) -> Tuple[TimingInfo, T]:
+    def wrapper(*args: P.args, **kwargs: P.kwargs) -> Tuple[TimingInfo, T]:
         start_wall_time, start_cpu_time = _start_measurement()
         try:
             result = func(*args, **kwargs)
@@ -49,13 +50,13 @@ def time_measured(func: Callable[..., T]) -> Callable[..., Tuple[TimingInfo, T]]
     return wrapper
 
 
-def error_logged(func: Callable[..., T]) -> Callable[..., T | None]:
+def error_logged(func: Callable[P, T]) -> Callable[P, T | None]:
     """
     Decorator to suppress and log any exceptions raised by a function.
     """
 
     @functools.wraps(func)
-    def wrapper(*args, **kwargs) -> T | None:
+    def wrapper(*args: P.args, **kwargs: P.kwargs) -> T | None:
         try:
             return func(*args, **kwargs)
         except Exception as e:

rnd/autogpt_server/autogpt_server/util/logging.py

@@ -1,17 +1,15 @@
 import os
 
-from forge.logging.config import LogFormatName
-
 
 def configure_logging():
     import logging
 
-    from forge.logging import configure_logging
+    import autogpt_libs.logging.config
 
     if os.getenv("APP_ENV") != "cloud":
-        configure_logging()
+        autogpt_libs.logging.config.configure_logging(force_cloud_logging=False)
     else:
-        configure_logging(log_format=LogFormatName.STRUCTURED)
+        autogpt_libs.logging.config.configure_logging(force_cloud_logging=True)
 
     # Silence httpx logger
     logging.getLogger("httpx").setLevel(logging.WARNING)

rnd/autogpt_server/autogpt_server/util/service.py

@@ -103,10 +103,7 @@ class AppService(AppProcess):
         host = Config().pyro_host
         daemon = Pyro5.api.Daemon(host=host, port=self.port)
         self.uri = daemon.register(self, objectId=self.service_name)
-
-        logger.debug(
-            f"Service in start pyro [{self.service_name}] Ready. Object URI = {self.uri}"
-        )
+        logger.info(f"[{self.service_name}] Connected to Pyro; URI = {self.uri}")
         daemon.requestLoop()
 
     def __start_async_loop(self):
@@ -124,7 +121,7 @@ def get_service_client(service_type: Type[AS], port: int) -> AS:
         def __init__(self):
             host = os.environ.get(f"{service_name.upper()}_HOST", "localhost")
             uri = f"PYRO:{service_type.service_name}@{host}:{port}"
-            logger.info(f"Connecting to service [{service_name}]. URI = {uri}")
+            logger.debug(f"Connecting to service [{service_name}]. URI = {uri}")
             self.proxy = Pyro5.api.Proxy(uri)
             # Attempt to bind to ensure the connection is established
             self.proxy._pyroBind()

rnd/autogpt_server/autogpt_server/util/settings.py

@@ -108,6 +108,23 @@ class Config(UpdateTrackingModel["Config"], BaseSettings):
 class Secrets(UpdateTrackingModel["Secrets"], BaseSettings):
     """Secrets for the server."""
 
+    supabase_url: str = Field(default="", description="Supabase URL")
+    supabase_key: str = Field(default="", description="Supabase key")
+
+    # OAuth server credentials for integrations
+    github_client_id: str = Field(default="", description="GitHub OAuth client ID")
+    github_client_secret: str = Field(
+        default="", description="GitHub OAuth client secret"
+    )
+    google_client_id: str = Field(default="", description="Google OAuth client ID")
+    google_client_secret: str = Field(
+        default="", description="Google OAuth client secret"
+    )
+    notion_client_id: str = Field(default="", description="Notion OAuth client ID")
+    notion_client_secret: str = Field(
+        default="", description="Notion OAuth client secret"
+    )
+
     openai_api_key: str = Field(default="", description="OpenAI API key")
     anthropic_api_key: str = Field(default="", description="Anthropic API key")
     groq_api_key: str = Field(default="", description="Groq API key")

rnd/autogpt_server/pyproject.toml

@@ -10,10 +10,9 @@ readme = "README.md"
 
 [tool.poetry.dependencies]
 python = "^3.10"
-agpt = { path = "../../autogpt", develop = true }
+aio-pika = "^9.4.3"
 anthropic = "^0.25.1"
 apscheduler = "^3.10.4"
-autogpt-forge = { path = "../../forge", develop = true }
 autogpt-libs = { path = "../autogpt_libs" }
 click = "^8.1.7"
 croniter = "^2.0.5"
@@ -39,15 +38,14 @@ pyro5 = "^5.15"
 pytest = "^8.2.1"
 pytest-asyncio = "^0.23.7"
 python-dotenv = "^1.0.1"
+redis = "^5.0.8"
+sentry-sdk = "1.45.0"
+supabase = "^2.7.2"
 tenacity = "^8.3.0"
 uvicorn = { extras = ["standard"], version = "^0.30.1" }
 websockets = "^12.0"
 youtube-transcript-api = "^0.6.2"
 
-aio-pika = "^9.4.3"
-redis = "^5.0.8"
-sentry-sdk = "1.45.0"
-
 [tool.poetry.group.dev.dependencies]
 poethepoet = "^0.26.1"
 httpx = "^0.27.0"
@@ -66,12 +64,13 @@ build-backend = "poetry.core.masonry.api"
 app = "autogpt_server.app:main"
 rest = "autogpt_server.rest:main"
 ws = "autogpt_server.ws:main"
-executor = "autogpt_server.app:execution_manager"
+executor = "autogpt_server.exec:main"
 cli = "autogpt_server.cli:main"
 format = "linter:format"
 lint = "linter:lint"
 test = "run_tests:test"
 # https://poethepoet.natn.io/index.html
+
 [tool.poe]
 poetry_command = ""
 

rnd/autogpt_server/run_tests.py

@@ -8,7 +8,8 @@ def wait_for_postgres(max_retries=5, delay=5):
         try:
             result = subprocess.run(
                 [
-                    "docker-compose",
+                    "docker",
+                    "compose",
                     "-f",
                     "docker-compose.test.yaml",
                     "exec",
@@ -45,7 +46,8 @@ def test():
     # Start PostgreSQL with Docker Compose
     run_command(
         [
-            "docker-compose",
+            "docker",
+            "compose",
             "-f",
             "docker-compose.test.yaml",
             "up",
@@ -55,7 +57,7 @@ def test():
     )
 
     if not wait_for_postgres():
-        run_command(["docker-compose", "-f", "docker-compose.test.yaml", "down"])
+        run_command(["docker", "compose", "-f", "docker-compose.test.yaml", "down"])
         sys.exit(1)
 
     # Run Prisma migrations
@@ -64,6 +66,6 @@ def test():
     # Run the tests
     result = subprocess.run(["pytest"] + sys.argv[1:], check=False)
 
-    run_command(["docker-compose", "-f", "docker-compose.test.yaml", "down"])
+    run_command(["docker", "compose", "-f", "docker-compose.test.yaml", "down"])
 
     sys.exit(result.returncode)

rnd/autogpt_server/test/data/test_graph.py

@@ -2,7 +2,7 @@ from uuid import UUID
 
 import pytest
 
-from autogpt_server.blocks.basic import InputBlock, StoreValueBlock
+from autogpt_server.blocks.basic import AgentInputBlock, StoreValueBlock
 from autogpt_server.data.graph import Graph, Link, Node
 from autogpt_server.data.user import DEFAULT_USER_ID, create_default_user
 from autogpt_server.server.model import CreateGraph
@@ -25,7 +25,7 @@ async def test_graph_creation(server: SpinTestServer):
     await create_default_user("false")
 
     value_block = StoreValueBlock().id
-    input_block = InputBlock().id
+    input_block = AgentInputBlock().id
 
     graph = Graph(
         id="test_graph",

rnd/autogpt_server/test/executor/test_manager.py

@@ -36,23 +36,19 @@ async def assert_sample_graph_executions(
     graph_exec_id: str,
 ):
     executions = await agent_server.get_graph_run_node_execution_results(
-        test_graph.id, graph_exec_id, test_user.id
+        test_graph.id,
+        graph_exec_id,
+        test_user.id,
     )
 
     output_list = [{"result": ["Hello"]}, {"result": ["World"]}]
     input_list = [
         {
             "name": "input_1",
-            "description": "First input value",
-            "placeholder_values": [],
-            "limit_to_placeholder_values": False,
             "value": "Hello",
         },
         {
             "name": "input_2",
-            "description": "Second input value",
-            "placeholder_values": [],
-            "limit_to_placeholder_values": False,
             "value": "World",
         },
     ]
@@ -61,16 +57,24 @@ async def assert_sample_graph_executions(
     exec = executions[0]
     assert exec.status == execution.ExecutionStatus.COMPLETED
     assert exec.graph_exec_id == graph_exec_id
-    assert exec.output_data in output_list
-    assert exec.input_data in input_list
+    assert (
+        exec.output_data in output_list
+    ), f"Output data: {exec.output_data} and {output_list}"
+    assert (
+        exec.input_data in input_list
+    ), f"Input data: {exec.input_data} and {input_list}"
     assert exec.node_id in [test_graph.nodes[0].id, test_graph.nodes[1].id]
 
     # Executing StoreValueBlock
     exec = executions[1]
     assert exec.status == execution.ExecutionStatus.COMPLETED
     assert exec.graph_exec_id == graph_exec_id
-    assert exec.output_data in output_list
-    assert exec.input_data in input_list
+    assert (
+        exec.output_data in output_list
+    ), f"Output data: {exec.output_data} and {output_list}"
+    assert (
+        exec.input_data in input_list
+    ), f"Input data: {exec.input_data} and {input_list}"
     assert exec.node_id in [test_graph.nodes[0].id, test_graph.nodes[1].id]
 
     # Executing FillTextTemplateBlock

rnd/autogpt_server/test/executor/test_scheduler.py

@@ -4,6 +4,7 @@ from autogpt_server.data import db, graph
 from autogpt_server.executor import ExecutionScheduler
 from autogpt_server.usecases.sample import create_test_graph, create_test_user
 from autogpt_server.util.service import get_service_client
+from autogpt_server.util.settings import Config
 from autogpt_server.util.test import SpinTestServer
 
 
@@ -13,7 +14,9 @@ async def test_agent_schedule(server: SpinTestServer):
     test_user = await create_test_user()
     test_graph = await graph.create_graph(create_test_graph(), user_id=test_user.id)
 
-    scheduler = get_service_client(ExecutionScheduler, 8003)
+    scheduler = get_service_client(
+        ExecutionScheduler, Config().execution_scheduler_port
+    )
 
     schedules = scheduler.get_execution_schedules(test_graph.id, test_user.id)
     assert len(schedules) == 0

rnd/docker-compose.yml

@@ -14,14 +14,31 @@ services:
       - "5432:5432"
     networks:
       - app-network
-
-  server_base:
+    
+  migrate:
     build:
       context: ../
       dockerfile: rnd/autogpt_server/Dockerfile
       target: server
-    image: autogpt_server:latest
-    command: ["echo", "This is a base image and should not be run directly"]
+    develop:
+      watch:
+        - path: ./
+          target: rnd/autogpt_server/migrate
+          action: rebuild
+    command: ["poetry", "run", "prisma", "migrate", "deploy"]
+    depends_on:
+      postgres:
+        condition: service_healthy
+    environment:
+      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local?connect_timeout=60
+    networks:
+      - app-network
+    restart: on-failure
+    healthcheck:
+      test: ["CMD", "poetry", "run", "prisma", "migrate", "status"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
 
   redis:
     image: redis:latest
@@ -30,10 +47,18 @@ services:
       - "6379:6379"
     networks:
       - app-network
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
 
-  api_srv:
-    image: autogpt_server:latest
-    command: ["poetry", "run", "rest"]
+  rest_server:
+    build:
+      context: ../
+      dockerfile: rnd/autogpt_server/Dockerfile
+      target: server
+    command: ["python", "-m", "autogpt_server.rest"]
     develop:
       watch:
         - path: ./
@@ -41,27 +66,31 @@ services:
           action: rebuild
     depends_on:
       redis:
-        condition: service_started
+        condition: service_healthy
       postgres:
         condition: service_healthy
+      migrate:
+        condition: service_completed_successfully
     environment:
-      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local
+      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local?connect_timeout=60
       - REDIS_HOST=redis
       - REDIS_PORT=6379
       - REDIS_PASSWORD=password
       - AUTH_ENABLED=false
       - PYRO_HOST=0.0.0.0
       - EXECUTIONMANAGER_HOST=executor
-      - EXECUTIONSCHEDULER_HOST=execution_scheduler
-      - PORT=8000
     ports:
       - "8000:8000"
+      - "8003:8003" # execution scheduler
     networks:
       - app-network
 
   executor:
-    image: autogpt_server:latest
-    command: ["poetry", "run", "executor"]
+    build:
+      context: ../
+      dockerfile: rnd/autogpt_server/Dockerfile
+      target: server
+    command: ["python", "-m", "autogpt_server.exec"]
     develop:
       watch:
         - path: ./
@@ -69,42 +98,49 @@ services:
           action: rebuild
     depends_on:
       redis:
-        condition: service_started
+        condition: service_healthy
       postgres:
         condition: service_healthy
+      migrate:
+        condition: service_completed_successfully
     environment:
-      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local
+      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local?connect_timeout=60
       - REDIS_HOST=redis
       - REDIS_PORT=6379
       - REDIS_PASSWORD=password
       - AUTH_ENABLED=false
       - PYRO_HOST=0.0.0.0
-      - AGENTSERVER_HOST=agent_server
-      - PORT=8002
+      - AGENTSERVER_HOST=rest_server
     ports:
-      - "8002:8002"
+      - "8002:8000"
     networks:
       - app-network
 
-  ws_srv:
-    image: autogpt_server:latest
-    command: ["poetry", "run", "ws"]
+  websocket_server:
+    build:
+      context: ../
+      dockerfile: rnd/autogpt_server/Dockerfile
+      target: server
+    command: ["python", "-m", "autogpt_server.ws"]
     develop:
       watch:
         - path: ./
           target: rnd/autogpt_server/
           action: rebuild
     depends_on:
-      - postgres
-      - redis
+      postgres:
+        condition: service_healthy
+      redis:
+        condition: service_healthy
+      migrate:
+        condition: service_completed_successfully
     environment:
-      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local
+      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local?connect_timeout=60
       - REDIS_HOST=redis
       - REDIS_PORT=6379
       - REDIS_PASSWORD=password
       - AUTH_ENABLED=false
       - PYRO_HOST=0.0.0.0
-      - PORT=8001
     ports:
       - "8001:8001"
     networks:
@@ -114,12 +150,20 @@ services:
     build:
       context: ../
       dockerfile: rnd/market/Dockerfile
+    develop:
+      watch:
+        - path: ./
+          target: rnd/market/
+          action: rebuild
     depends_on:
-      - postgres
+      postgres:
+        condition: service_healthy
+      migrate:
+        condition: service_completed_successfully
     environment:
-      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local
+      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local?connect_timeout=60&schema=market
     ports:
-      - "8015:8015"
+      - "8015:8000"
     networks:
       - app-network
       
@@ -128,19 +172,30 @@ services:
       context: ../
       dockerfile: rnd/autogpt_builder/Dockerfile
       target: dev
+    develop:
+      watch:
+        - path: ./
+          target: rnd/autogpt_builder/
+          action: rebuild
     depends_on:
-      - postgres
-      - api_srv
-      - ws_srv
+      postgres:
+        condition: service_healthy
+      rest_server:
+        condition: service_started
+      websocket_server:
+        condition: service_started
+      migrate:
+        condition: service_completed_successfully
     environment:
-      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local
+      - DATABASE_URL=postgresql://agpt_user:pass123@postgres:5432/agpt_local?connect_timeout=60
       - NEXT_PUBLIC_AGPT_SERVER_URL=http://localhost:8000/api
       - NEXT_PUBLIC_AGPT_WS_SERVER_URL=ws://localhost:8001/ws
-      - NEXT_PUBLIC_AGPT_MARKETPLACE_URL=http://localhost:8005/api/v1/market
+      - NEXT_PUBLIC_AGPT_MARKETPLACE_URL=http://localhost:8015/api/v1/market
     ports:
       - "3000:3000"
     networks:
       - app-network
+
 networks:
-     app-network:
-       driver: bridge
+  app-network:
+    driver: bridge

rnd/infra/helm/autogpt-market/values.dev.yaml

@@ -13,7 +13,7 @@ serviceAccount:
 service:
   type: ClusterIP
   port: 8000
-  targetPort: 8000
+  targetPort: 8005
   annotations:
     cloud.google.com/neg: '{"ingress": true}'
     beta.cloud.google.com/backend-config: '{"default": "autogpt-market"}'

rnd/infra/helm/autogpt-server/templates/deployment.yaml

@@ -39,6 +39,7 @@ spec:
             {{- toYaml .Values.securityContext | nindent 12 }}
           image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
           imagePullPolicy: {{ .Values.image.pullPolicy }}
+          command: ["poetry", "run", "rest"]
           ports:
             - name: http
               containerPort: {{ .Values.service.port }}

rnd/infra/helm/autogpt-websocket-server/templates/deployment.yaml

@@ -39,6 +39,7 @@ spec:
             {{- toYaml .Values.securityContext | nindent 12 }}
           image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
           imagePullPolicy: {{ .Values.image.pullPolicy }}
+          command: ["poetry", "run", "ws"]
           ports:
             - name: ws
               containerPort: {{ .Values.service.port }}

rnd/infra/helm/autogpt-websocket-server/values.dev.yaml

@@ -1,7 +1,7 @@
 replicaCount: 1 # not scaling websocket server for now
 
 image:
-  repository: us-east1-docker.pkg.dev/agpt-dev/agpt-ws-server-dev/agpt-ws-server-dev
+  repository: us-east1-docker.pkg.dev/agpt-dev/agpt-server-dev/agpt-server-dev
   tag: latest
   pullPolicy: Always
 

rnd/market/Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.11-slim-buster AS server_base
+FROM python:3.11-slim-buster AS builder
 
 # Set environment variables
 ENV PYTHONDONTWRITEBYTECODE 1
@@ -6,16 +6,11 @@ ENV PYTHONUNBUFFERED 1
 
 WORKDIR /app
 
+# Install build dependencies
 RUN apt-get update \
-    && apt-get install -y build-essential curl ffmpeg wget libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev \
+    && apt-get install -y build-essential curl ffmpeg wget libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev postgresql-client git \
     && apt-get clean \
-    && rm -rf /var/lib/apt/lists/* \
-    && wget https://github.com/git/git/archive/v2.28.0.tar.gz -O git.tar.gz \
-    && tar -zxf git.tar.gz \
-    && cd git-* \
-    && make prefix=/usr all \
-    && make prefix=/usr install
-
+    && rm -rf /var/lib/apt/lists/*
 
 ENV POETRY_VERSION=1.8.3 \
     POETRY_HOME="/opt/poetry" \
@@ -24,31 +19,43 @@ ENV POETRY_VERSION=1.8.3 \
     PATH="$POETRY_HOME/bin:$PATH"
 RUN pip3 install poetry
 
-FROM server_base AS server_dependencies
+# Copy and install dependencies
+COPY rnd/autogpt_libs /app/rnd/autogpt_libs
+COPY rnd/market/poetry.lock rnd/market/pyproject.toml /app/rnd/market/
+WORKDIR /app/rnd/market
+RUN poetry config virtualenvs.create false \
+    && poetry install --no-interaction --no-ansi
+
+# Generate Prisma client
+COPY rnd/market /app/rnd/market
+RUN poetry config virtualenvs.create false \
+    && poetry run prisma generate
+
+FROM python:3.11-slim-buster AS server_dependencies
+
+WORKDIR /app
+
+# Copy only necessary files from builder
+COPY --from=builder /app /app
+COPY --from=builder /usr/local/lib/python3.11 /usr/local/lib/python3.11
+COPY --from=builder /usr/local/bin /usr/local/bin
+# Copy Prisma binaries
+COPY --from=builder /root/.cache/prisma-python/binaries /root/.cache/prisma-python/binaries
+
+ENV PATH="/app/.venv/bin:$PATH"
 
-RUN mkdir -p /app/autogpt
-RUN mkdir -p /app/forge
 RUN mkdir -p /app/rnd/autogpt_libs
 RUN mkdir -p /app/rnd/market
+
 COPY rnd/autogpt_libs /app/rnd/autogpt_libs
 
-COPY rnd/market/poetry.lock rnd/market/pyproject.toml /app/rnd/market/
+COPY rnd/market /app/rnd/market
 
 WORKDIR /app/rnd/market
 
-RUN poetry install --no-interaction --no-ansi
+FROM server_dependencies AS server
 
-FROM server_dependencies AS server_prisma
-
-# Need the market/utils/partial_types.py
-COPY rnd/market /app/rnd/market
-
-COPY rnd/market/schema.prisma ./
-RUN poetry run prisma generate
-
-FROM server_prisma AS server
-
-ENV PORT=8005
 ENV DATABASE_URL=""
+ENV PORT=8015
 
-CMD ["poetry", "run", "app"]
+CMD ["uvicorn", "market.app:app", "--reload"]

rnd/market/market/app.py

@@ -87,5 +87,11 @@ def health():
         content="<h1>Marketplace API</h1>", status_code=200
     )
 
+@app.get("/")
+def default():
+    return fastapi.responses.HTMLResponse(
+        content="<h1>Marketplace API</h1>", status_code=200
+    )
+
 
 prometheus_fastapi_instrumentator.Instrumentator().instrument(app).expose(app)

rnd/market/scripts.py

@@ -59,7 +59,7 @@ def format():
 
 def app():
     port = os.getenv("PORT", "8015")
-    run("uvicorn", "market.app:app", "--reload", "--port", port)
+    run("uvicorn", "market.app:app", "--reload", "--port", port, "--host", "0.0.0.0")
 
 
 def setup():

rnd/rest-api-go/.gitignore

@@ -0,0 +1,48 @@
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+
+# Test binary, built with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+# Dependency directories (remove the comment below to include it)
+# vendor/
+
+# Go workspace file
+go.work
+
+# IDE-specific files
+.idea/
+.vscode/
+
+# OS-specific files
+.DS_Store
+Thumbs.db
+
+# Binary output directory
+/bin/
+
+# Log files
+*.log
+
+# Environment variables file
+.env
+
+# Air temporary files (if using Air for live reloading)
+tmp/
+
+# Compiled Object files, Static and Dynamic libs (Shared Objects)
+*.o
+*.a
+
+# Debug files
+debug
+
+# Project-specific build outputs
+/gosrv