set up helm charts

* feat(blocks): Add MathsBlock for performing mathematical operations

The commit adds a new block called MathsBlock to perform various mathematical operations such as addition, subtraction, multiplication, division, and exponentiation. The block takes input parameters for the operation type, two numbers, and an option to round the result. It returns the result of the calculation along with an explanation of the performed operation.

---------

Co-authored-by: Swifty <craigswift13@gmail.com>

.gitattributes

@@ -3,3 +3,6 @@ frontend/build/** linguist-generated
 **/poetry.lock linguist-generated
 
 docs/_javascript/** linguist-vendored
+
+# Exclude VCR cassettes from stats
+forge/tests/vcr_cassettes/**/**.y*ml linguist-generated

.github/ISSUE_TEMPLATE/1.bug.yml

@@ -16,7 +16,7 @@ body:
         [discussions]: https://github.com/Significant-Gravitas/AutoGPT/discussions
         [#tech-support]: https://discord.com/channels/1092243196446249134/1092275629602394184
         [existing issues]: https://github.com/Significant-Gravitas/AutoGPT/issues?q=is%3Aissue
-        [wiki page on Contributing]: https://github.com/Significant-Gravitas/Nexus/wiki/Contributing
+        [wiki page on Contributing]: https://github.com/Significant-Gravitas/AutoGPT/wiki/Contributing
 
   - type: checkboxes
     attributes:
@@ -88,14 +88,16 @@ body:
 
   - type: dropdown
     attributes:
-      label: Do you use OpenAI GPT-3 or GPT-4?
+      label: What LLM Provider do you use?
       description: >
         If you are using AutoGPT with `SMART_LLM=gpt-3.5-turbo`, your problems may be caused by
         the [limitations](https://github.com/Significant-Gravitas/AutoGPT/issues?q=is%3Aissue+label%3A%22AI+model+limitation%22) of GPT-3.5.
       options:
-        - GPT-3.5
-        - GPT-4
-        - GPT-4(32k)
+        - Azure
+        - Groq
+        - Anthropic
+        - Llamafile
+        - Other (detail in issue)
     validations:
       required: true
 
@@ -126,6 +128,13 @@ body:
             label: Specify the area
             description: Please specify the area you think is best related to the issue.
 
+  - type: input
+    attributes:
+      label: What commit or version are you using?
+      description: It is helpful for us to reproduce to know what version of the software you were using when this happened. Please run `git log -n 1 --pretty=format:"%H"` to output the full commit hash.
+    validations:
+      required: true
+
   - type: textarea
     attributes:
       label: Describe your issue.

.github/ISSUE_TEMPLATE/2.feature.yml

@@ -5,7 +5,7 @@ body:
   - type: markdown
     attributes:
       value: |
-        First, check out our [wiki page on Contributing](https://github.com/Significant-Gravitas/Nexus/wiki/Contributing)
+        First, check out our [wiki page on Contributing](https://github.com/Significant-Gravitas/AutoGPT/wiki/Contributing)
         Please provide a searchable summary of the issue in the title above ⬆️.
   - type: checkboxes
     attributes:

.github/PULL_REQUEST_TEMPLATE.md

@@ -10,7 +10,7 @@
 
 <!--
 Check out our contribution guide:
-https://github.com/Significant-Gravitas/Nexus/wiki/Contributing
+https://github.com/Significant-Gravitas/AutoGPT/wiki/Contributing
 
 1. Avoid duplicate work, issues, PRs etc.
 2. Also consider contributing something other than code; see the [contribution guide]

.github/labeler.yml

@@ -17,3 +17,11 @@ Frontend:
 documentation:
 - changed-files:
   - any-glob-to-any-file: docs/**
+
+Builder:
+- changed-files:
+  - any-glob-to-any-file: rnd/autogpt_builder/**
+
+Server:
+- changed-files:
+  - any-glob-to-any-file: rnd/autogpt_server/**

.github/workflows/autogpt-builder-ci.yml

@@ -0,0 +1,36 @@
+name: AutoGPT Builder CI
+
+on:
+  push:
+    branches: [ master ]
+    paths:
+      - '.github/workflows/autogpt-builder-ci.yml'
+      - 'rnd/autogpt_builder/**'
+  pull_request:
+    paths:
+      - '.github/workflows/autogpt-builder-ci.yml'
+      - 'rnd/autogpt_builder/**'
+
+defaults:
+  run:
+    shell: bash
+    working-directory: rnd/autogpt_builder
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Node.js
+      uses: actions/setup-node@v4
+      with:
+        node-version: '21'
+
+    - name: Install dependencies
+      run: |
+        npm install
+
+    - name: Run lint
+      run: |
+        npm run lint

.github/workflows/autogpt-ci.yml

@@ -6,13 +6,11 @@ on:
     paths:
       - '.github/workflows/autogpt-ci.yml'
       - 'autogpt/**'
-      - '!autogpt/tests/vcr_cassettes'
   pull_request:
     branches: [ master, development, release-* ]
     paths:
       - '.github/workflows/autogpt-ci.yml'
       - 'autogpt/**'
-      - '!autogpt/tests/vcr_cassettes'
 
 concurrency:
   group: ${{ format('autogpt-ci-{0}', github.head_ref && format('{0}-{1}', github.event_name, github.event.pull_request.number) || github.sha) }}
@@ -73,37 +71,6 @@ jobs:
           git config --global user.name "Auto-GPT-Bot"
           git config --global user.email "github-bot@agpt.co"
 
-      - name: Checkout cassettes
-        if: ${{ startsWith(github.event_name, 'pull_request') }}
-        env:
-          PR_BASE: ${{ github.event.pull_request.base.ref }}
-          PR_BRANCH: ${{ github.event.pull_request.head.ref }}
-          PR_AUTHOR: ${{ github.event.pull_request.user.login }}
-        run: |
-          cassette_branch="${PR_AUTHOR}-${PR_BRANCH}"
-          cassette_base_branch="${PR_BASE}"
-          cd tests/vcr_cassettes
-
-          if ! git ls-remote --exit-code --heads origin $cassette_base_branch ; then
-            cassette_base_branch="master"
-          fi
-
-          if git ls-remote --exit-code --heads origin $cassette_branch ; then
-            git fetch origin $cassette_branch
-            git fetch origin $cassette_base_branch
-
-            git checkout $cassette_branch
-
-            # Pick non-conflicting cassette updates from the base branch
-            git merge --no-commit --strategy-option=ours origin/$cassette_base_branch
-            echo "Using cassettes from mirror branch '$cassette_branch'," \
-              "synced to upstream branch '$cassette_base_branch'."
-          else
-            git checkout -b $cassette_branch
-            echo "Branch '$cassette_branch' does not exist in cassette submodule." \
-              "Using cassettes from '$cassette_base_branch'."
-          fi
-
       - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v5
         with:
@@ -163,80 +130,6 @@ jobs:
           token: ${{ secrets.CODECOV_TOKEN }}
           flags: autogpt-agent,${{ runner.os }}
 
-      - id: setup_git_auth
-        name: Set up git token authentication
-        # Cassettes may be pushed even when tests fail
-        if: success() || failure()
-        run: |
-          config_key="http.${{ github.server_url }}/.extraheader"
-          if [ "${{ runner.os }}" = 'macOS' ]; then
-            base64_pat=$(echo -n "pat:${{ secrets.PAT_REVIEW }}" | base64)
-          else
-            base64_pat=$(echo -n "pat:${{ secrets.PAT_REVIEW }}" | base64 -w0)
-          fi
-
-          git config "$config_key" \
-            "Authorization: Basic $base64_pat"
-
-          cd tests/vcr_cassettes
-          git config "$config_key" \
-            "Authorization: Basic $base64_pat"
-
-          echo "config_key=$config_key" >> $GITHUB_OUTPUT
-
-      - id: push_cassettes
-        name: Push updated cassettes
-        # For pull requests, push updated cassettes even when tests fail
-        if: github.event_name == 'push' || (! github.event.pull_request.head.repo.fork && (success() || failure()))
-        env:
-          PR_BRANCH: ${{ github.event.pull_request.head.ref }}
-          PR_AUTHOR: ${{ github.event.pull_request.user.login }}
-        run: |
-          if [ "${{ startsWith(github.event_name, 'pull_request') }}" = "true" ]; then
-            is_pull_request=true
-            cassette_branch="${PR_AUTHOR}-${PR_BRANCH}"
-          else
-            cassette_branch="${{ github.ref_name }}"
-          fi
-
-          cd tests/vcr_cassettes
-          # Commit & push changes to cassettes if any
-          if ! git diff --quiet; then
-            git add .
-            git commit -m "Auto-update cassettes"
-            git push origin HEAD:$cassette_branch
-            if [ ! $is_pull_request ]; then
-              cd ../..
-              git add tests/vcr_cassettes
-              git commit -m "Update cassette submodule"
-              git push origin HEAD:$cassette_branch
-            fi
-            echo "updated=true" >> $GITHUB_OUTPUT
-          else
-            echo "updated=false" >> $GITHUB_OUTPUT
-            echo "No cassette changes to commit"
-          fi
-
-      - name: Post Set up git token auth
-        if: steps.setup_git_auth.outcome == 'success'
-        run: |
-          git config --unset-all '${{ steps.setup_git_auth.outputs.config_key }}'
-          git submodule foreach git config --unset-all '${{ steps.setup_git_auth.outputs.config_key }}'
-
-      - name: Apply "behaviour change" label and comment on PR
-        if: ${{ startsWith(github.event_name, 'pull_request') }}
-        run: |
-          PR_NUMBER="${{ github.event.pull_request.number }}"
-          TOKEN="${{ secrets.PAT_REVIEW }}"
-          REPO="${{ github.repository }}"
-
-          if [[ "${{ steps.push_cassettes.outputs.updated }}" == "true" ]]; then
-            echo "Adding label and comment..."
-            echo $TOKEN | gh auth login --with-token
-            gh issue edit $PR_NUMBER --add-label "behaviour change"
-            gh issue comment $PR_NUMBER --body "You changed AutoGPT's behaviour on ${{ runner.os }}. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged."
-          fi
-
       - name: Upload logs to artifact
         if: always()
         uses: actions/upload-artifact@v4

.github/workflows/autogpt-docker-ci.yml

@@ -6,13 +6,11 @@ on:
     paths:
       - '.github/workflows/autogpt-docker-ci.yml'
       - 'autogpt/**'
-      - '!autogpt/tests/vcr_cassettes'
   pull_request:
     branches: [ master, development, release-* ]
     paths:
       - '.github/workflows/autogpt-docker-ci.yml'
       - 'autogpt/**'
-      - '!autogpt/tests/vcr_cassettes'
 
 concurrency:
   group: ${{ format('autogpt-docker-ci-{0}', github.head_ref && format('pr-{0}', github.event.pull_request.number) || github.sha) }}

.github/workflows/autogpt-infra-ci.yml

@@ -0,0 +1,33 @@
+name: AutoGPT Builder CI
+
+on:
+  push:
+    branches: [ master ]
+    paths:
+      - '.github/workflows/autogpt-infra-ci.yml'
+      - 'rnd/infra/**'
+  pull_request:
+    paths:
+      - '.github/workflows/autogpt-infra-ci.yml'
+      - 'rnd/infra/**'
+
+defaults:
+  run:
+    shell: bash
+    working-directory: rnd/infra
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: TFLint
+        uses: pauloconnor/tflint-action@v0.0.2
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+            tflint_path: terraform/
+            tflint_recurse: true
+            tflint_changed_only: false

.github/workflows/autogpt-server-ci.yml

@@ -0,0 +1,148 @@
+name: AutoGPT Server CI
+
+on:
+  push:
+    branches: [master, development, ci-test*]
+    paths:
+      - ".github/workflows/autogpt-server-ci.yml"
+      - "rnd/autogpt_server/**"
+  pull_request:
+    branches: [master, development, release-*]
+    paths:
+      - ".github/workflows/autogpt-server-ci.yml"
+      - "rnd/autogpt_server/**"
+
+concurrency:
+  group: ${{ format('autogpt-server-ci-{0}', github.head_ref && format('{0}-{1}', github.event_name, github.event.pull_request.number) || github.sha) }}
+  cancel-in-progress: ${{ startsWith(github.event_name, 'pull_request') }}
+
+defaults:
+  run:
+    shell: bash
+    working-directory: rnd/autogpt_server
+
+jobs:
+  test:
+    permissions:
+      contents: read
+    timeout-minutes: 30
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10"]
+        platform-os: [ubuntu, macos, macos-arm64, windows]
+    runs-on: ${{ matrix.platform-os != 'macos-arm64' && format('{0}-latest', matrix.platform-os) || 'macos-14' }}
+
+    steps:
+      - name: Setup PostgreSQL
+        uses: ikalnytskyi/action-setup-postgres@v6
+        with:
+          username: ${{ secrets.DB_USER }}
+          password: ${{ secrets.DB_PASS }}
+          database: postgres
+          port: 5432
+        id: postgres
+
+      # Quite slow on macOS (2~4 minutes to set up Docker)
+      # - name: Set up Docker (macOS)
+      #   if: runner.os == 'macOS'
+      #   uses: crazy-max/ghaction-setup-docker@v3
+
+      - name: Start MinIO service (Linux)
+        if: runner.os == 'Linux'
+        working-directory: "."
+        run: |
+          docker pull minio/minio:edge-cicd
+          docker run -d -p 9000:9000 minio/minio:edge-cicd
+
+      - name: Start MinIO service (macOS)
+        if: runner.os == 'macOS'
+        working-directory: ${{ runner.temp }}
+        run: |
+          brew install minio/stable/minio
+          mkdir data
+          minio server ./data &
+
+      # No MinIO on Windows:
+      # - Windows doesn't support running Linux Docker containers
+      # - It doesn't seem possible to start background processes on Windows. They are
+      #   killed after the step returns.
+      #   See: https://github.com/actions/runner/issues/598#issuecomment-2011890429
+
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          submodules: true
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - id: get_date
+        name: Get date
+        run: echo "date=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT
+
+      - name: Set up Python dependency cache
+        # On Windows, unpacking cached dependencies takes longer than just installing them
+        if: runner.os != 'Windows'
+        uses: actions/cache@v4
+        with:
+          path: ${{ runner.os == 'macOS' && '~/Library/Caches/pypoetry' || '~/.cache/pypoetry' }}
+          key: poetry-${{ runner.os }}-${{ hashFiles('rnd/autogpt_server/poetry.lock') }}
+
+      - name: Install Poetry (Unix)
+        if: runner.os != 'Windows'
+        run: |
+          curl -sSL https://install.python-poetry.org | python3 -
+
+          if [ "${{ runner.os }}" = "macOS" ]; then
+            PATH="$HOME/.local/bin:$PATH"
+            echo "$HOME/.local/bin" >> $GITHUB_PATH
+          fi
+
+      - name: Install Poetry (Windows)
+        if: runner.os == 'Windows'
+        shell: pwsh
+        run: |
+          (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python -
+
+          $env:PATH += ";$env:APPDATA\Python\Scripts"
+          echo "$env:APPDATA\Python\Scripts" >> $env:GITHUB_PATH
+
+      - name: Install Python dependencies
+        run: poetry install
+
+      - name: Generate Prisma Client
+        run: poetry run prisma generate --schema postgres/schema.prisma
+
+      - name: Run Database Migrations
+        run: poetry run prisma migrate dev --schema postgres/schema.prisma --name updates
+        env:
+          CONNECTION_STR: ${{ steps.postgres.outputs.connection-uri }}
+
+      - name: Run Linter
+        run: poetry run lint
+
+      - name: Run pytest with coverage
+        run: |
+          poetry run pytest -vv \
+            test
+    env:
+      CI: true
+      PLAIN_OUTPUT: True
+      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+      DB_USER: ${{ secrets.DB_USER }}
+      DB_PASS: ${{ secrets.DB_PASS }}
+      DB_NAME: postgres
+      DB_PORT: 5432
+      RUN_ENV: local
+      PORT: 8080
+      DATABASE_URL: postgresql://${{ secrets.DB_USER }}:${{ secrets.DB_PASS }}@localhost:5432/${{ secrets.DB_NAME }}
+
+      # - name: Upload coverage reports to Codecov
+      #   uses: codecov/codecov-action@v4
+      #   with:
+      #     token: ${{ secrets.CODECOV_TOKEN }}
+      #     flags: autogpt-server,${{ runner.os }}

.github/workflows/forge-ci.yml

@@ -6,11 +6,13 @@ on:
     paths:
       - '.github/workflows/forge-ci.yml'
       - 'forge/**'
+      - '!forge/tests/vcr_cassettes'
   pull_request:
     branches: [ master, development, release-* ]
     paths:
       - '.github/workflows/forge-ci.yml'
       - 'forge/**'
+      - '!forge/tests/vcr_cassettes'
 
 concurrency:
   group: ${{ format('forge-ci-{0}', github.head_ref && format('{0}-{1}', github.event_name, github.event.pull_request.number) || github.sha) }}
@@ -66,6 +68,37 @@ jobs:
           fetch-depth: 0
           submodules: true
 
+      - name: Checkout cassettes
+        if: ${{ startsWith(github.event_name, 'pull_request') }}
+        env:
+          PR_BASE: ${{ github.event.pull_request.base.ref }}
+          PR_BRANCH: ${{ github.event.pull_request.head.ref }}
+          PR_AUTHOR: ${{ github.event.pull_request.user.login }}
+        run: |
+          cassette_branch="${PR_AUTHOR}-${PR_BRANCH}"
+          cassette_base_branch="${PR_BASE}"
+          cd tests/vcr_cassettes
+
+          if ! git ls-remote --exit-code --heads origin $cassette_base_branch ; then
+            cassette_base_branch="master"
+          fi
+
+          if git ls-remote --exit-code --heads origin $cassette_branch ; then
+            git fetch origin $cassette_branch
+            git fetch origin $cassette_base_branch
+
+            git checkout $cassette_branch
+
+            # Pick non-conflicting cassette updates from the base branch
+            git merge --no-commit --strategy-option=ours origin/$cassette_base_branch
+            echo "Using cassettes from mirror branch '$cassette_branch'," \
+              "synced to upstream branch '$cassette_base_branch'."
+          else
+            git checkout -b $cassette_branch
+            echo "Branch '$cassette_branch' does not exist in cassette submodule." \
+              "Using cassettes from '$cassette_base_branch'."
+          fi
+
       - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v5
         with:
@@ -121,6 +154,80 @@ jobs:
           token: ${{ secrets.CODECOV_TOKEN }}
           flags: forge,${{ runner.os }}
 
+      - id: setup_git_auth
+        name: Set up git token authentication
+        # Cassettes may be pushed even when tests fail
+        if: success() || failure()
+        run: |
+          config_key="http.${{ github.server_url }}/.extraheader"
+          if [ "${{ runner.os }}" = 'macOS' ]; then
+            base64_pat=$(echo -n "pat:${{ secrets.PAT_REVIEW }}" | base64)
+          else
+            base64_pat=$(echo -n "pat:${{ secrets.PAT_REVIEW }}" | base64 -w0)
+          fi
+
+          git config "$config_key" \
+            "Authorization: Basic $base64_pat"
+
+          cd tests/vcr_cassettes
+          git config "$config_key" \
+            "Authorization: Basic $base64_pat"
+
+          echo "config_key=$config_key" >> $GITHUB_OUTPUT
+
+      - id: push_cassettes
+        name: Push updated cassettes
+        # For pull requests, push updated cassettes even when tests fail
+        if: github.event_name == 'push' || (! github.event.pull_request.head.repo.fork && (success() || failure()))
+        env:
+          PR_BRANCH: ${{ github.event.pull_request.head.ref }}
+          PR_AUTHOR: ${{ github.event.pull_request.user.login }}
+        run: |
+          if [ "${{ startsWith(github.event_name, 'pull_request') }}" = "true" ]; then
+            is_pull_request=true
+            cassette_branch="${PR_AUTHOR}-${PR_BRANCH}"
+          else
+            cassette_branch="${{ github.ref_name }}"
+          fi
+
+          cd tests/vcr_cassettes
+          # Commit & push changes to cassettes if any
+          if ! git diff --quiet; then
+            git add .
+            git commit -m "Auto-update cassettes"
+            git push origin HEAD:$cassette_branch
+            if [ ! $is_pull_request ]; then
+              cd ../..
+              git add tests/vcr_cassettes
+              git commit -m "Update cassette submodule"
+              git push origin HEAD:$cassette_branch
+            fi
+            echo "updated=true" >> $GITHUB_OUTPUT
+          else
+            echo "updated=false" >> $GITHUB_OUTPUT
+            echo "No cassette changes to commit"
+          fi
+
+      - name: Post Set up git token auth
+        if: steps.setup_git_auth.outcome == 'success'
+        run: |
+          git config --unset-all '${{ steps.setup_git_auth.outputs.config_key }}'
+          git submodule foreach git config --unset-all '${{ steps.setup_git_auth.outputs.config_key }}'
+
+      - name: Apply "behaviour change" label and comment on PR
+        if: ${{ startsWith(github.event_name, 'pull_request') }}
+        run: |
+          PR_NUMBER="${{ github.event.pull_request.number }}"
+          TOKEN="${{ secrets.PAT_REVIEW }}"
+          REPO="${{ github.repository }}"
+
+          if [[ "${{ steps.push_cassettes.outputs.updated }}" == "true" ]]; then
+            echo "Adding label and comment..."
+            echo $TOKEN | gh auth login --with-token
+            gh issue edit $PR_NUMBER --add-label "behaviour change"
+            gh issue comment $PR_NUMBER --body "You changed AutoGPT's behaviour on ${{ runner.os }}. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged."
+          fi
+
       - name: Upload logs to artifact
         if: always()
         uses: actions/upload-artifact@v4

.github/workflows/pr-label.yml

@@ -5,7 +5,7 @@ on:
   push:
     branches: [ master, development, release-* ]
     paths-ignore:
-      - 'autogpt/tests/vcr_cassettes'
+      - 'forge/tests/vcr_cassettes'
       - 'benchmark/reports/**'
   # So that the `dirtyLabel` is removed if conflicts are resolve
   # We recommend `pull_request_target` so that github secrets are available.

.github/workflows/python-checks.yml

@@ -9,7 +9,7 @@ on:
       - 'forge/**'
       - 'benchmark/**'
       - '**.py'
-      - '!autogpt/tests/vcr_cassettes'
+      - '!forge/tests/vcr_cassettes'
   pull_request:
     branches: [ master, development, release-* ]
     paths:
@@ -18,7 +18,7 @@ on:
       - 'forge/**'
       - 'benchmark/**'
       - '**.py'
-      - '!autogpt/tests/vcr_cassettes'
+      - '!forge/tests/vcr_cassettes'
 
 concurrency:
   group: ${{ format('lint-ci-{0}', github.head_ref && format('{0}-{1}', github.event_name, github.event.pull_request.number) || github.sha) }}

.github/workflows/scripts/check_actions_status.py

@@ -0,0 +1,55 @@
+import os
+import requests
+import sys
+
+# GitHub API endpoint
+api_url = os.environ["GITHUB_API_URL"]
+repo = os.environ["GITHUB_REPOSITORY"]
+sha = os.environ["GITHUB_SHA"]
+
+# GitHub token for authentication
+github_token = os.environ["GITHUB_TOKEN"]
+
+# API endpoint for check runs for the specific SHA
+endpoint = f"{api_url}/repos/{repo}/commits/{sha}/check-runs"
+
+# Set up headers for authentication
+headers = {
+    "Authorization": f"token {github_token}",
+    "Accept": "application/vnd.github.v3+json"
+}
+
+# Make the API request
+response = requests.get(endpoint, headers=headers)
+
+if response.status_code != 200:
+    print(f"Error: Unable to fetch check runs data. Status code: {response.status_code}")
+    sys.exit(1)
+
+check_runs = response.json()["check_runs"]
+
+# Flag to track if all other check runs have passed
+all_others_passed = True
+
+# Current run id
+current_run_id = os.environ["GITHUB_RUN_ID"]
+
+for run in check_runs:
+    if str(run["id"]) != current_run_id:
+        status = run["status"]
+        conclusion = run["conclusion"]
+        
+        if status == "completed":
+            if conclusion not in ["success", "skipped", "neutral"]:
+                all_others_passed = False
+                print(f"Check run {run['name']} (ID: {run['id']}) has conclusion: {conclusion}")
+        else:
+            print(f"Check run {run['name']} (ID: {run['id']}) is still {status}.")
+            all_others_passed = False
+
+if all_others_passed:
+    print("All other completed check runs have passed. This check passes.")
+    sys.exit(0)
+else:
+    print("Some check runs have failed or have not completed. This check fails.")
+    sys.exit(1)

.github/workflows/workflow-checker.yml

@@ -0,0 +1,51 @@
+name: PR Status Checker
+on:
+  workflow_run:
+    workflows: ["*"]
+    types:
+      - completed
+
+jobs:
+  status-check:
+    name: Check Actions Status
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: "3.10"
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install requests
+    - name: Debug Information
+      run: |
+        echo "Event name: ${{ github.event_name }}"
+        echo "Workflow: ${{ github.workflow }}"
+        echo "Action: ${{ github.action }}"
+        echo "Actor: ${{ github.actor }}"
+        echo "Repository: ${{ github.repository }}"
+        echo "Ref: ${{ github.ref }}"
+        echo "Head ref: ${{ github.head_ref }}"
+        echo "Base ref: ${{ github.base_ref }}"
+        echo "Event payload:"
+        cat $GITHUB_EVENT_PATH
+    - name: Debug File Structure
+      run: |
+        echo "Current directory:"
+        pwd
+        echo "Directory contents:"
+        ls -R
+        echo "GitHub workspace:"
+        echo $GITHUB_WORKSPACE
+        echo "GitHub workspace contents:"
+        ls -R $GITHUB_WORKSPACE
+    - name: Check Actions Status
+      run: |
+        echo "Current directory before running Python script:"
+        pwd
+        echo "Attempting to run Python script:"
+        python .github/scripts/check_actions_status.py
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.gitignore

@@ -32,7 +32,6 @@ dist/
 downloads/
 eggs/
 .eggs/
-lib/
 lib64/
 parts/
 sdist/
@@ -162,7 +161,7 @@ agbenchmark/reports/
 
 # Nodejs
 package-lock.json
-package.json
+
 
 # Allow for locally private items
 # private
@@ -170,3 +169,5 @@ pri*
 # ignore
 ig*
 .github_access_token
+LICENSE.rtf
+rnd/autogpt_server/settings.py

.gitmodules

@@ -1,3 +1,3 @@
-[submodule "autogpt/tests/vcr_cassettes"]
-	path = autogpt/tests/vcr_cassettes
+[submodule "forge/tests/vcr_cassettes"]
+	path = forge/tests/vcr_cassettes
 	url = https://github.com/Significant-Gravitas/Auto-GPT-test-cassettes

.pre-commit-config.yaml

@@ -97,7 +97,7 @@ repos:
         alias: pyright-benchmark
         entry: poetry -C benchmark run pyright
         args: [-p, benchmark, benchmark]
-        files: ^benchmark/(agbenchmark|tests)/
+        files: ^benchmark/(agbenchmark/|tests/|poetry\.lock$)
         types: [file]
         language: system
         pass_filenames: false

.vscode/all-projects.code-workspace

@@ -0,0 +1,47 @@
+{
+  "folders": [
+    {
+      "name": "autogpt",
+      "path": "../autogpt"
+    },
+    {
+      "name": "benchmark",
+      "path": "../benchmark"
+    },
+    {
+      "name": "docs",
+      "path": "../docs"
+    },
+    {
+      "name": "forge",
+      "path": "../forge"
+    },
+    {
+      "name": "frontend",
+      "path": "../frontend"
+    },
+    {
+      "name": "autogpt_server",
+      "path": "../rnd/autogpt_server"
+    },
+    {
+      "name": "autogpt_builder",
+      "path": "../rnd/autogpt_builder"
+    },
+    {
+      "name": "[root]",
+      "path": ".."
+    }
+  ],
+  "settings": {},
+  "extensions": {
+    "recommendations": [
+      "charliermarsh.ruff",
+      "dart-code.flutter",
+      "ms-python.black-formatter",
+      "ms-python.vscode-pylance",
+      "prisma.prisma",
+      "qwtel.sqlite-viewer"
+    ]
+  }
+}

CONTRIBUTING.md

@@ -1,34 +1,38 @@
 # AutoGPT Contribution Guide
-If you are reading this, you are probably looking for our **[contribution guide]**,
-which is part of our [knowledge base].
+If you are reading this, you are probably looking for the full **[contribution guide]**,
+which is part of our [wiki].
 
-You can find our immediate priorities and their progress on our public [kanban board].
+Also check out our [🚀 Roadmap][roadmap] for information about our priorities and associated tasks.
+<!-- You can find our immediate priorities and their progress on our public [kanban board]. -->
 
-[contribution guide]: https://github.com/Significant-Gravitas/Nexus/wiki/Contributing
-[knowledge base]: https://github.com/Significant-Gravitas/Nexus/wiki
+[contribution guide]: https://github.com/Significant-Gravitas/AutoGPT/wiki/Contributing
+[wiki]: https://github.com/Significant-Gravitas/AutoGPT/wiki
+[roadmap]: https://github.com/Significant-Gravitas/AutoGPT/discussions/6971
 [kanban board]: https://github.com/orgs/Significant-Gravitas/projects/1
 
 ## In short
 1. Avoid duplicate work, issues, PRs etc.
 2. We encourage you to collaborate with fellow community members on some of our bigger
-   [todo's][kanban board]!
+   [todo's][roadmap]!
    * We highly recommend to post your idea and discuss it in the [dev channel].
-4. Create a draft PR when starting work on bigger changes.
-3. Please also consider contributing something other than code; see the
-   [contribution guide] for options.
+3. Create a draft PR when starting work on bigger changes.
+4. Adhere to the [Code Guidelines]
 5. Clearly explain your changes when submitting a PR.
-6. Don't submit stuff that's broken.
+6. Don't submit broken code: test/validate your changes.
 7. Avoid making unnecessary changes, especially if they're purely based on your personal
    preferences. Doing so is the maintainers' job. ;-)
+8. Please also consider contributing something other than code; see the
+   [contribution guide] for options.
 
 [dev channel]: https://discord.com/channels/1092243196446249134/1095817829405704305
+[code guidelines]: https://github.com/Significant-Gravitas/AutoGPT/wiki/Contributing#code-guidelines
 
 If you wish to involve with the project (beyond just contributing PRs), please read the
-wiki [catalyzing](https://github.com/Significant-Gravitas/Nexus/wiki/Catalyzing) page.
+wiki page about [Catalyzing](https://github.com/Significant-Gravitas/AutoGPT/wiki/Catalyzing).
 
 In fact, why not just look through the whole wiki (it's only a few pages) and
 hop on our Discord. See you there! :-)
 
-❤️ & 🔆  
-The team @ AutoGPT  
+❤️ & 🔆
+The team @ AutoGPT
 https://discord.gg/autogpt

FORGE-QUICKSTART.md

@@ -2,11 +2,11 @@
 
 > For the complete getting started [tutorial series](https://aiedge.medium.com/autogpt-forge-e3de53cc58ec) <- click here
 
-Welcome to the Quickstart Guide! This guide will walk you through the process of setting up and running your own AutoGPT agent. Whether you're a seasoned AI developer or just starting out, this guide will provide you with the necessary steps to jumpstart your journey in the world of AI development with AutoGPT.
+Welcome to the Quickstart Guide! This guide will walk you through setting up, building, and running your own AutoGPT agent. Whether you're a seasoned AI developer or just starting out, this guide will provide you with the steps to jumpstart your journey in AI development with AutoGPT.
 
 ## System Requirements
 
-This project supports Linux (Debian based), Mac, and Windows Subsystem for Linux (WSL). If you are using a Windows system, you will need to install WSL. You can find the installation instructions for WSL [here](https://learn.microsoft.com/en-us/windows/wsl/).
+This project supports Linux (Debian-based), Mac, and Windows Subsystem for Linux (WSL). If you use a Windows system, you must install WSL. You can find the installation instructions for WSL [here](https://learn.microsoft.com/en-us/windows/wsl/).
 
 
 ## Getting Setup
@@ -18,11 +18,11 @@ This project supports Linux (Debian based), Mac, and Windows Subsystem for Linux
    - In the top-right corner of the page, click Fork.
 
    ![Create Fork UI](docs/content/imgs/quickstart/002_fork.png)
-   - On the next page, select your GitHub account to create the fork under.
+   - On the next page, select your GitHub account to create the fork.
    - Wait for the forking process to complete. You now have a copy of the repository in your GitHub account.
 
 2. **Clone the Repository**
-   To clone the repository, you need to have Git installed on your system. If you don't have Git installed, you can download it from [here](https://git-scm.com/downloads). Once you have Git installed, follow these steps:
+   To clone the repository, you need to have Git installed on your system. If you don't have Git installed, download it from [here](https://git-scm.com/downloads). Once you have Git installed, follow these steps:
    - Open your terminal.
    - Navigate to the directory where you want to clone the repository.
    - Run the git clone command for the fork you just created
@@ -34,11 +34,11 @@ This project supports Linux (Debian based), Mac, and Windows Subsystem for Linux
    ![Open the Project in your IDE](docs/content/imgs/quickstart/004_ide.png)
 
 4. **Setup the Project**
-    Next we need to setup the required dependencies. We have a tool for helping you do all the tasks you need to on the repo.
+    Next, we need to set up the required dependencies. We have a tool to help you perform all the tasks on the repo.
     It can be accessed by running the `run` command by typing `./run` in the terminal.
 
-    The first command you need to use is `./run setup` This will guide you through the process of setting up your system.
-    Initially you will get instructions for installing flutter, chrome and setting up your github access token like the following image:
+    The first command you need to use is `./run setup.` This will guide you through setting up your system.
+    Initially, you will get instructions for installing Flutter and Chrome and setting up your GitHub access token like the following image:
     
     ![Setup the Project](docs/content/imgs/quickstart/005_setup.png)
 
@@ -47,7 +47,7 @@ This project supports Linux (Debian based), Mac, and Windows Subsystem for Linux
 If you're a Windows user and experience issues after installing WSL, follow the steps below to resolve them. 
 
 #### Update WSL 
-Run the following command in Powershell or Command Prompt to:
+Run the following command in Powershell or Command Prompt:
 1. Enable the optional WSL and Virtual Machine Platform components.
 2. Download and install the latest Linux kernel.
 3. Set WSL 2 as the default.
@@ -73,7 +73,7 @@ dos2unix ./run
 After executing the above commands, running `./run setup` should work successfully. 
 
 #### Store Project Files within the WSL File System
-If you continue to experience issues, consider storing your project files within the WSL file system instead of the Windows file system. This method avoids issues related to path translations and permissions and provides a more consistent development environment.
+If you continue to experience issues, consider storing your project files within the WSL file system instead of the Windows file system. This method avoids path translations and permissions issues and provides a more consistent development environment.
 
 You can keep running the command to get feedback on where you are up to with your setup. 
 When setup has been completed, the command will return an output like this:
@@ -83,7 +83,7 @@ When setup has been completed, the command will return an output like this:
 ## Creating Your Agent
 
 After completing the setup, the next step is to create your agent template.
-Execute the command `./run agent create YOUR_AGENT_NAME`, where `YOUR_AGENT_NAME` should be replaced with a name of your choosing.
+Execute the command `./run agent create YOUR_AGENT_NAME`, where `YOUR_AGENT_NAME` should be replaced with your chosen name.
 
 Tips for naming your agent:
 * Give it its own unique name, or name it after yourself
@@ -101,21 +101,21 @@ This starts the agent on the URL: `http://localhost:8000/`
 
 ![Start the Agent](docs/content/imgs/quickstart/009_start_agent.png)
 
-The frontend can be accessed from `http://localhost:8000/`, you will first need to login using either a google account or your github account.
+The front end can be accessed from `http://localhost:8000/`; first, you must log in using either a Google account or your GitHub account.
 
 ![Login](docs/content/imgs/quickstart/010_login.png)
 
-Upon logging in you will get a page that looks something like this. With your task history down the left hand side of the page and the 'chat' window to send tasks to your agent.
+Upon logging in, you will get a page that looks something like this: your task history down the left-hand side of the page, and the 'chat' window to send tasks to your agent.
 
 ![Login](docs/content/imgs/quickstart/011_home.png)
 
-When you have finished with your agent, or if you just need to restart it, use Ctl-C to end the session then you can re-run the start command.
+When you have finished with your agent or just need to restart it, use Ctl-C to end the session. Then, you can re-run the start command.
 
-If you are having issues and want to ensure the agent has been stopped there is a `./run agent stop` command which will kill the process using port 8000, which should be the agent. 
+If you are having issues and want to ensure the agent has been stopped, there is a `./run agent stop` command, which will kill the process using port 8000, which should be the agent. 
 
 ## Benchmarking your Agent
 
-The benchmarking system can also be accessed using the cli too:
+The benchmarking system can also be accessed using the CLI too:
 
 ```bash
 agpt % ./run benchmark
@@ -163,7 +163,7 @@ The benchmark has been split into different categories of skills you can test yo
 ![Login](docs/content/imgs/quickstart/012_tests.png)
 
 
-Finally you can run the benchmark with
+Finally, you can run the benchmark with
 
 ```bash
 ./run benchmark start YOUR_AGENT_NAME

README.md

@@ -1,10 +1,44 @@
-# AutoGPT: build & use AI agents
+# AutoGPT: Build & Use AI Agents
 
 [![Discord Follow](https://dcbadge.vercel.app/api/server/autogpt?style=flat)](https://discord.gg/autogpt)  
 [![Twitter Follow](https://img.shields.io/twitter/follow/Auto_GPT?style=social)](https://twitter.com/Auto_GPT)  
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-**AutoGPT** is the vision of the power of AI accessible to everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters:
+**AutoGPT** is a powerful tool that lets you create and run intelligent agents. These agents can perform various tasks automatically, making your life easier.
+
+## How to Get Started
+
+https://github.com/user-attachments/assets/8508f4dc-b362-4cab-900f-644964a96cdf
+
+### 🧱 AutoGPT Builder 
+
+The AutoGPT Builder is the frontend. It allows you to design agents using an easy flowchart style. You build your agent by connecting blocks, where each block performs a single action. It's simple and intuitive!
+
+[Read this guide](https://docs.agpt.co/server/new_blocks/) to learn how to build your own custom blocks.
+
+### 💽 AutoGPT Server
+
+The AutoGPT Server is the backend. This is where your agents run. Once deployed, agents can be triggered by external sources and can operate continuously.
+
+### 🐙 Example Agents
+
+Here are two examples of what you can do with AutoGPT:
+
+1. **Reddit Marketing Agent**
+   - This agent reads comments on Reddit.
+   - It looks for people asking about your product.
+   - It then automatically responds to them.
+
+2. **YouTube Content Repurposing Agent**
+   - This agent subscribes to your YouTube channel.
+   - When you post a new video, it transcribes it.
+   - It uses AI to write a search engine optimized blog post.
+   - Then, it publishes this blog post to your Medium account.
+
+These examples show just a glimpse of what you can achieve with AutoGPT!
+
+---
+Our mission is to provide the tools, so that you can focus on what matters:
 
 - 🏗️ **Building** - Lay the foundation for something amazing.
 - 🧪 **Testing** - Fine-tune your agent to perfection.
@@ -15,11 +49,13 @@ Be part of the revolution! **AutoGPT** is here to stay, at the forefront of AI i
 **📖 [Documentation](https://docs.agpt.co)**
  | 
 **🚀 [Contributing](CONTRIBUTING.md)**
- | 
-**🛠️ [Build your own Agent - Quickstart](QUICKSTART.md)**
 
-## 🧱 Building blocks
 
+---
+## 🤖 AutoGPT Classic
+> Below is information about the classic version of AutoGPT.
+
+**🛠️ [Build your own Agent - Quickstart](FORGE-QUICKSTART.md)**
 ### 🏗️ Forge
 
 **Forge your own agent!** – Forge is a ready-to-go template for your agent application. All the boilerplate code is already handled, letting you channel all your creativity into the things that set *your* agent apart. All tutorials are located [here](https://medium.com/@aiedge/autogpt-forge-e3de53cc58ec). Components from the [`forge.sdk`](/forge/forge/sdk) can also be used individually to speed up development and reduce boilerplate in your agent project.

autogpt/.env.template

@@ -11,12 +11,15 @@
 ## GROQ_API_KEY - Groq API Key (Example: gsk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx)
 # GROQ_API_KEY=
 
+## LLAMAFILE_API_BASE - Llamafile API base URL
+# LLAMAFILE_API_BASE=http://localhost:8080/v1
+
 ## TELEMETRY_OPT_IN - Share telemetry on errors and other issues with the AutoGPT team, e.g. through Sentry.
 ##   This helps us to spot and solve problems earlier & faster. (Default: DISABLED)
 # TELEMETRY_OPT_IN=true
 
-## EXECUTE_LOCAL_COMMANDS - Allow local command execution (Default: False)
-# EXECUTE_LOCAL_COMMANDS=False
+## COMPONENT_CONFIG_FILE - Path to the json config file (Default: None)
+# COMPONENT_CONFIG_FILE=
 
 ### Workspace ###
 
@@ -44,9 +47,6 @@
 
 ### Miscellaneous ###
 
-## USER_AGENT - Define the user-agent used by the requests library to browse website (string)
-# USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36"
-
 ## AUTHORISE COMMAND KEY - Key to authorise commands
 # AUTHORISE_COMMAND_KEY=y
 
@@ -96,38 +96,12 @@
 ## EMBEDDING_MODEL - Model to use for creating embeddings
 # EMBEDDING_MODEL=text-embedding-3-small
 
-################################################################################
-### SHELL EXECUTION
-################################################################################
-
-## SHELL_COMMAND_CONTROL - Whether to use "allowlist" or "denylist" to determine what shell commands can be executed (Default: denylist)
-# SHELL_COMMAND_CONTROL=denylist
-
-## ONLY if SHELL_COMMAND_CONTROL is set to denylist:
-## SHELL_DENYLIST - List of shell commands that ARE NOT allowed to be executed by AutoGPT (Default: sudo,su)
-# SHELL_DENYLIST=sudo,su
-
-## ONLY if SHELL_COMMAND_CONTROL is set to allowlist:
-## SHELL_ALLOWLIST - List of shell commands that ARE allowed to be executed by AutoGPT (Default: None)
-# SHELL_ALLOWLIST=
-
 ################################################################################
 ### IMAGE GENERATION PROVIDER
 ################################################################################
 
-### Common
-
-## IMAGE_PROVIDER - Image provider (Default: dalle)
-# IMAGE_PROVIDER=dalle
-
-## IMAGE_SIZE - Image size (Default: 256)
-# IMAGE_SIZE=256
-
 ### Huggingface (IMAGE_PROVIDER=huggingface)
 
-## HUGGINGFACE_IMAGE_MODEL - Text-to-image model from Huggingface (Default: CompVis/stable-diffusion-v1-4)
-# HUGGINGFACE_IMAGE_MODEL=CompVis/stable-diffusion-v1-4
-
 ## HUGGINGFACE_API_TOKEN - HuggingFace API token (Default: None)
 # HUGGINGFACE_API_TOKEN=
 
@@ -136,19 +110,6 @@
 ## SD_WEBUI_AUTH - Stable Diffusion Web UI username:password pair (Default: None)
 # SD_WEBUI_AUTH=
 
-## SD_WEBUI_URL - Stable Diffusion Web UI API URL (Default: http://localhost:7860)
-# SD_WEBUI_URL=http://localhost:7860
-
-################################################################################
-### AUDIO TO TEXT PROVIDER
-################################################################################
-
-## AUDIO_TO_TEXT_PROVIDER - Audio-to-text provider (Default: huggingface)
-# AUDIO_TO_TEXT_PROVIDER=huggingface
-
-## HUGGINGFACE_AUDIO_TO_TEXT_MODEL - The model for HuggingFace to use (Default: CompVis/stable-diffusion-v1-4)
-# HUGGINGFACE_AUDIO_TO_TEXT_MODEL=CompVis/stable-diffusion-v1-4
-
 ################################################################################
 ### GITHUB
 ################################################################################
@@ -163,18 +124,6 @@
 ### WEB BROWSING
 ################################################################################
 
-## HEADLESS_BROWSER - Whether to run the browser in headless mode (default: True)
-# HEADLESS_BROWSER=True
-
-## USE_WEB_BROWSER - Sets the web-browser driver to use with selenium (default: chrome)
-# USE_WEB_BROWSER=chrome
-
-## BROWSE_CHUNK_MAX_LENGTH - When browsing website, define the length of chunks to summarize (Default: 3000)
-# BROWSE_CHUNK_MAX_LENGTH=3000
-
-## BROWSE_SPACY_LANGUAGE_MODEL - spaCy language model](https://spacy.io/usage/models) to use when creating chunks. (Default: en_core_web_sm)
-# BROWSE_SPACY_LANGUAGE_MODEL=en_core_web_sm
-
 ## GOOGLE_API_KEY - Google API key (Default: None)
 # GOOGLE_API_KEY=
 
@@ -198,13 +147,6 @@
 ## ELEVENLABS_VOICE_ID - Eleven Labs voice ID (Example: None)
 # ELEVENLABS_VOICE_ID=
 
-################################################################################
-### CHAT MESSAGES
-################################################################################
-
-## CHAT_MESSAGES_ENABLED - Enable chat messages (Default: False)
-# CHAT_MESSAGES_ENABLED=False
-
 ################################################################################
 ### LOGGING
 ################################################################################

autogpt/.gitattributes

@@ -1,5 +0,0 @@
-# Exclude VCR cassettes from stats
-tests/vcr_cassettes/**/**.y*ml linguist-generated
-
-# Mark documentation as such
-docs/**.md linguist-documentation

autogpt/BULLETIN.md

@@ -2,7 +2,7 @@
 # --------------
 🌎 *Official Website*: https://agpt.co.
 📖 *User Guide*: https://docs.agpt.co/autogpt.
-👩 *Contributors Wiki*: https://github.com/Significant-Gravitas/Nexus/wiki/Contributing.
+👩 *Contributors Wiki*: https://github.com/Significant-Gravitas/AutoGPT/wiki/Contributing.
 
 # v0.5.0 RELEASE HIGHLIGHTS! 🚀🚀
 # -------------------------------

autogpt/README.md

@@ -68,10 +68,6 @@ Options:
                                   continuous mode
   --speak                         Enable Speak Mode
   --debug                         Enable Debug Mode
-  -b, --browser-name TEXT         Specifies which web-browser to use when
-                                  using selenium to scrape the web.
-  --allow-downloads               Dangerous: Allows AutoGPT to download files
-                                  natively.
   --skip-news                     Specifies whether to suppress the output of
                                   latest news on startup.
   --install-plugin-deps           Installs external dependencies for 3rd party
@@ -90,6 +86,7 @@ Options:
   --override-directives           If specified, --constraint, --resource and
                                   --best-practice will override the AI's
                                   directives instead of being appended to them
+  --component-config-file TEXT    Path to the json configuration file.
   --help                          Show this message and exit.
 ```
 </details>
@@ -111,10 +108,6 @@ Usage: python -m autogpt serve [OPTIONS]
 
 Options:
   --debug                     Enable Debug Mode
-  -b, --browser-name TEXT     Specifies which web-browser to use when using
-                              selenium to scrape the web.
-  --allow-downloads           Dangerous: Allows AutoGPT to download files
-                              natively.
   --install-plugin-deps       Installs external dependencies for 3rd party
                               plugins.
   --help                      Show this message and exit.
@@ -132,7 +125,7 @@ For more comprehensive instructions, see the [user guide][docs/usage].
 [docs/plugins]: https://docs.agpt.co/autogpt/plugins
 
 ## 📚 Resources
-* 📔 AutoGPT [team wiki](https://github.com/Significant-Gravitas/Nexus/wiki)
+* 📔 AutoGPT [project wiki](https://github.com/Significant-Gravitas/AutoGPT/wiki)
 * 🧮 AutoGPT [project kanban](https://github.com/orgs/Significant-Gravitas/projects/1)
 * 🌃 AutoGPT [roadmap](https://github.com/orgs/Significant-Gravitas/projects/2)
 

autogpt/autogpt/agent_factory/configurators.py

@@ -2,17 +2,17 @@ from typing import Optional
 
 from forge.config.ai_directives import AIDirectives
 from forge.config.ai_profile import AIProfile
-from forge.config.config import Config
 from forge.file_storage.base import FileStorage
 from forge.llm.providers import MultiProvider
 
 from autogpt.agents.agent import Agent, AgentConfiguration, AgentSettings
+from autogpt.app.config import AppConfig
 
 
 def create_agent(
     agent_id: str,
     task: str,
-    app_config: Config,
+    app_config: AppConfig,
     file_storage: FileStorage,
     llm_provider: MultiProvider,
     ai_profile: Optional[AIProfile] = None,
@@ -38,7 +38,7 @@ def create_agent(
 
 def configure_agent_with_state(
     state: AgentSettings,
-    app_config: Config,
+    app_config: AppConfig,
     file_storage: FileStorage,
     llm_provider: MultiProvider,
 ) -> Agent:
@@ -51,7 +51,7 @@ def configure_agent_with_state(
 
 
 def _configure_agent(
-    app_config: Config,
+    app_config: AppConfig,
     llm_provider: MultiProvider,
     file_storage: FileStorage,
     agent_id: str = "",
@@ -80,7 +80,7 @@ def _configure_agent(
         settings=agent_state,
         llm_provider=llm_provider,
         file_storage=file_storage,
-        legacy_config=app_config,
+        app_config=app_config,
     )
 
 
@@ -89,7 +89,7 @@ def create_agent_state(
     task: str,
     ai_profile: AIProfile,
     directives: AIDirectives,
-    app_config: Config,
+    app_config: AppConfig,
 ) -> AgentSettings:
     return AgentSettings(
         agent_id=agent_id,
@@ -104,5 +104,5 @@ def create_agent_state(
             allow_fs_access=not app_config.restrict_to_workspace,
             use_functions_api=app_config.openai_functions,
         ),
-        history=Agent.default_settings.history.copy(deep=True),
+        history=Agent.default_settings.history.model_copy(deep=True),
     )

autogpt/autogpt/agent_factory/generators.py

@@ -6,7 +6,7 @@ from forge.file_storage.base import FileStorage
 
 if TYPE_CHECKING:
     from autogpt.agents.agent import Agent
-    from forge.config.config import Config
+    from autogpt.app.config import AppConfig
     from forge.llm.providers import MultiProvider
 
 from .configurators import _configure_agent
@@ -16,7 +16,7 @@ from .profile_generator import generate_agent_profile_for_task
 async def generate_agent_for_task(
     agent_id: str,
     task: str,
-    app_config: Config,
+    app_config: AppConfig,
     file_storage: FileStorage,
     llm_provider: MultiProvider,
 ) -> Agent:

autogpt/autogpt/agent_factory/profile_generator.py

@@ -3,7 +3,6 @@ import logging
 
 from forge.config.ai_directives import AIDirectives
 from forge.config.ai_profile import AIProfile
-from forge.config.config import Config
 from forge.llm.prompting import ChatPrompt, LanguageModelClassification, PromptStrategy
 from forge.llm.providers import MultiProvider
 from forge.llm.providers.schema import (
@@ -14,11 +13,13 @@ from forge.llm.providers.schema import (
 from forge.models.config import SystemConfiguration, UserConfigurable
 from forge.models.json_schema import JSONSchema
 
+from autogpt.app.config import AppConfig
+
 logger = logging.getLogger(__name__)
 
 
 class AgentProfileGeneratorConfiguration(SystemConfiguration):
-    model_classification: LanguageModelClassification = UserConfigurable(
+    llm_classification: LanguageModelClassification = UserConfigurable(
         default=LanguageModelClassification.SMART_MODEL
     )
     _example_call: object = {
@@ -136,7 +137,7 @@ class AgentProfileGeneratorConfiguration(SystemConfiguration):
                     required=True,
                 ),
             },
-        ).dict()
+        ).model_dump()
     )
 
 
@@ -147,21 +148,21 @@ class AgentProfileGenerator(PromptStrategy):
 
     def __init__(
         self,
-        model_classification: LanguageModelClassification,
+        llm_classification: LanguageModelClassification,
         system_prompt: str,
         user_prompt_template: str,
         create_agent_function: dict,
     ):
-        self._model_classification = model_classification
+        self._llm_classification = llm_classification
         self._system_prompt_message = system_prompt
         self._user_prompt_template = user_prompt_template
-        self._create_agent_function = CompletionModelFunction.parse_obj(
+        self._create_agent_function = CompletionModelFunction.model_validate(
             create_agent_function
         )
 
     @property
-    def model_classification(self) -> LanguageModelClassification:
-        return self._model_classification
+    def llm_classification(self) -> LanguageModelClassification:
+        return self._llm_classification
 
     def build_prompt(self, user_objective: str = "", **kwargs) -> ChatPrompt:
         system_message = ChatMessage.system(self._system_prompt_message)
@@ -212,7 +213,7 @@ class AgentProfileGenerator(PromptStrategy):
 
 async def generate_agent_profile_for_task(
     task: str,
-    app_config: Config,
+    app_config: AppConfig,
     llm_provider: MultiProvider,
 ) -> tuple[AIProfile, AIDirectives]:
     """Generates an AIConfig object from the given string.
@@ -221,7 +222,7 @@ async def generate_agent_profile_for_task(
     AIConfig: The AIConfig object tailored to the user's input
     """
     agent_profile_generator = AgentProfileGenerator(
-        **AgentProfileGenerator.default_configuration.dict()  # HACK
+        **AgentProfileGenerator.default_configuration.model_dump()  # HACK
     )
 
     prompt = agent_profile_generator.build_prompt(task)

autogpt/autogpt/agents/README.md

@@ -26,10 +26,10 @@ class MyAgent(Agent):
         settings: AgentSettings,
         llm_provider: MultiProvider
         file_storage: FileStorage,
-        legacy_config: Config,
+        app_config: AppConfig,
     ):
         # Call the parent constructor to bring in the default components
-        super().__init__(settings, llm_provider, file_storage, legacy_config)
+        super().__init__(settings, llm_provider, file_storage, app_config)
         # Add your custom component
         self.my_component = MyComponent()
 ```

autogpt/autogpt/agents/agent.py

@@ -18,7 +18,11 @@ from forge.components.action_history import (
     ActionHistoryComponent,
     EpisodicActionHistory,
 )
-from forge.components.code_executor.code_executor import CodeExecutorComponent
+from forge.components.action_history.action_history import ActionHistoryConfiguration
+from forge.components.code_executor.code_executor import (
+    CodeExecutorComponent,
+    CodeExecutorConfiguration,
+)
 from forge.components.context.context import AgentContext, ContextComponent
 from forge.components.file_manager import FileManagerComponent
 from forge.components.git_operations import GitOperationsComponent
@@ -58,7 +62,7 @@ from .prompt_strategies.one_shot import (
 )
 
 if TYPE_CHECKING:
-    from forge.config.config import Config
+    from autogpt.app.config import AppConfig
 
 logger = logging.getLogger(__name__)
 
@@ -91,12 +95,14 @@ class Agent(BaseAgent[OneShotAgentActionProposal], Configurable[AgentSettings]):
         settings: AgentSettings,
         llm_provider: MultiProvider,
         file_storage: FileStorage,
-        legacy_config: Config,
+        app_config: AppConfig,
     ):
         super().__init__(settings)
 
         self.llm_provider = llm_provider
-        prompt_config = OneShotAgentPromptStrategy.default_configuration.copy(deep=True)
+        prompt_config = OneShotAgentPromptStrategy.default_configuration.model_copy(
+            deep=True
+        )
         prompt_config.use_functions_api = (
             settings.config.use_functions_api
             # Anthropic currently doesn't support tools + prefilling :(
@@ -107,33 +113,41 @@ class Agent(BaseAgent[OneShotAgentActionProposal], Configurable[AgentSettings]):
 
         # Components
         self.system = SystemComponent()
-        self.history = ActionHistoryComponent(
-            settings.history,
-            self.send_token_limit,
-            lambda x: self.llm_provider.count_tokens(x, self.llm.name),
-            legacy_config,
-            llm_provider,
-        ).run_after(WatchdogComponent)
-        self.user_interaction = UserInteractionComponent(legacy_config)
-        self.file_manager = FileManagerComponent(settings, file_storage)
+        self.history = (
+            ActionHistoryComponent(
+                settings.history,
+                lambda x: self.llm_provider.count_tokens(x, self.llm.name),
+                llm_provider,
+                ActionHistoryConfiguration(
+                    llm_name=app_config.fast_llm, max_tokens=self.send_token_limit
+                ),
+            )
+            .run_after(WatchdogComponent)
+            .run_after(SystemComponent)
+        )
+        if not app_config.noninteractive_mode:
+            self.user_interaction = UserInteractionComponent()
+        self.file_manager = FileManagerComponent(file_storage, settings)
         self.code_executor = CodeExecutorComponent(
             self.file_manager.workspace,
-            settings,
-            legacy_config,
+            CodeExecutorConfiguration(
+                docker_container_name=f"{settings.agent_id}_sandbox"
+            ),
         )
-        self.git_ops = GitOperationsComponent(legacy_config)
-        self.image_gen = ImageGeneratorComponent(
-            self.file_manager.workspace, legacy_config
+        self.git_ops = GitOperationsComponent()
+        self.image_gen = ImageGeneratorComponent(self.file_manager.workspace)
+        self.web_search = WebSearchComponent()
+        self.web_selenium = WebSeleniumComponent(
+            llm_provider,
+            app_config.app_data_dir,
         )
-        self.web_search = WebSearchComponent(legacy_config)
-        self.web_selenium = WebSeleniumComponent(legacy_config, llm_provider, self.llm)
         self.context = ContextComponent(self.file_manager.workspace, settings.context)
         self.watchdog = WatchdogComponent(settings.config, settings.history).run_after(
             ContextComponent
         )
 
         self.event_history = settings.history
-        self.legacy_config = legacy_config
+        self.app_config = app_config
 
     async def propose_action(self) -> OneShotAgentActionProposal:
         """Proposes the next action to execute, based on the task and current state.
@@ -148,7 +162,7 @@ class Agent(BaseAgent[OneShotAgentActionProposal], Configurable[AgentSettings]):
         constraints = await self.run_pipeline(DirectiveProvider.get_constraints)
         best_practices = await self.run_pipeline(DirectiveProvider.get_best_practices)
 
-        directives = self.state.directives.copy(deep=True)
+        directives = self.state.directives.model_copy(deep=True)
         directives.resources += resources
         directives.constraints += constraints
         directives.best_practices += best_practices
@@ -160,13 +174,19 @@ class Agent(BaseAgent[OneShotAgentActionProposal], Configurable[AgentSettings]):
         # Get messages
         messages = await self.run_pipeline(MessageProvider.get_messages)
 
+        include_os_info = (
+            self.code_executor.config.execute_local_commands
+            if hasattr(self, "code_executor")
+            else False
+        )
+
         prompt: ChatPrompt = self.prompt_strategy.build_prompt(
             messages=messages,
             task=self.state.task,
             ai_profile=self.state.ai_profile,
             ai_directives=directives,
             commands=function_specs_from_commands(self.commands),
-            include_os_info=self.legacy_config.execute_local_commands,
+            include_os_info=include_os_info,
         )
 
         logger.debug(f"Executing prompt:\n{dump_prompt(prompt)}")
@@ -277,7 +297,7 @@ class Agent(BaseAgent[OneShotAgentActionProposal], Configurable[AgentSettings]):
             command
             for command in self.commands
             if not any(
-                name in self.legacy_config.disabled_commands for name in command.names
+                name in self.app_config.disabled_commands for name in command.names
             )
         ]
 

autogpt/autogpt/agents/prompt_strategies/one_shot.py

@@ -28,15 +28,13 @@ _RESPONSE_INTERFACE_NAME = "AssistantResponse"
 
 class AssistantThoughts(ModelWithSummary):
     observations: str = Field(
-        ..., description="Relevant observations from your last action (if any)"
+        description="Relevant observations from your last action (if any)"
     )
-    text: str = Field(..., description="Thoughts")
-    reasoning: str = Field(..., description="Reasoning behind the thoughts")
-    self_criticism: str = Field(..., description="Constructive self-criticism")
-    plan: list[str] = Field(
-        ..., description="Short list that conveys the long-term plan"
-    )
-    speak: str = Field(..., description="Summary of thoughts, to say to user")
+    text: str = Field(description="Thoughts")
+    reasoning: str = Field(description="Reasoning behind the thoughts")
+    self_criticism: str = Field(description="Constructive self-criticism")
+    plan: list[str] = Field(description="Short list that conveys the long-term plan")
+    speak: str = Field(description="Summary of thoughts, to say to user")
 
     def summary(self) -> str:
         return self.text
@@ -96,11 +94,13 @@ class OneShotAgentPromptStrategy(PromptStrategy):
         logger: Logger,
     ):
         self.config = configuration
-        self.response_schema = JSONSchema.from_dict(OneShotAgentActionProposal.schema())
+        self.response_schema = JSONSchema.from_dict(
+            OneShotAgentActionProposal.model_json_schema()
+        )
         self.logger = logger
 
     @property
-    def model_classification(self) -> LanguageModelClassification:
+    def llm_classification(self) -> LanguageModelClassification:
         return LanguageModelClassification.FAST_MODEL  # FIXME: dynamic switching
 
     def build_prompt(
@@ -182,7 +182,7 @@ class OneShotAgentPromptStrategy(PromptStrategy):
         )
 
     def response_format_instruction(self, use_functions_api: bool) -> tuple[str, str]:
-        response_schema = self.response_schema.copy(deep=True)
+        response_schema = self.response_schema.model_copy(deep=True)
         assert response_schema.properties
         if use_functions_api and "use_tool" in response_schema.properties:
             del response_schema.properties["use_tool"]
@@ -274,5 +274,8 @@ class OneShotAgentPromptStrategy(PromptStrategy):
                 raise InvalidAgentResponseError("Assistant did not use a tool")
             assistant_reply_dict["use_tool"] = response.tool_calls[0].function
 
-        parsed_response = OneShotAgentActionProposal.parse_obj(assistant_reply_dict)
+        parsed_response = OneShotAgentActionProposal.model_validate(
+            assistant_reply_dict
+        )
+        parsed_response.raw_message = response.copy()
         return parsed_response

autogpt/autogpt/app/agent_protocol_server.py

@@ -23,7 +23,6 @@ from forge.agent_protocol.models import (
     TaskRequestBody,
     TaskStepsListResponse,
 )
-from forge.config.config import Config
 from forge.file_storage import FileStorage
 from forge.llm.providers import ModelProviderBudget, MultiProvider
 from forge.models.action import ActionErrorResult, ActionSuccessResult
@@ -35,6 +34,7 @@ from sentry_sdk import set_user
 
 from autogpt.agent_factory.configurators import configure_agent_with_state, create_agent
 from autogpt.agents.agent_manager import AgentManager
+from autogpt.app.config import AppConfig
 from autogpt.app.utils import is_port_free
 
 logger = logging.getLogger(__name__)
@@ -45,7 +45,7 @@ class AgentProtocolServer:
 
     def __init__(
         self,
-        app_config: Config,
+        app_config: AppConfig,
         database: AgentDB,
         file_storage: FileStorage,
         llm_provider: MultiProvider,
@@ -314,7 +314,7 @@ class AgentProtocolServer:
                             ""
                             if tool_result is None
                             else (
-                                orjson.loads(tool_result.json())
+                                orjson.loads(tool_result.model_dump_json())
                                 if not isinstance(tool_result, ActionErrorResult)
                                 else {
                                     "error": str(tool_result.error),
@@ -327,7 +327,7 @@ class AgentProtocolServer:
                 if last_proposal and tool_result
                 else {}
             ),
-            **assistant_response.dict(),
+            **assistant_response.model_dump(),
         }
 
         task_cumulative_cost = agent.llm_provider.get_incurred_cost()
@@ -451,7 +451,9 @@ class AgentProtocolServer:
         """
         task_llm_budget = self._task_budgets[task.task_id]
 
-        task_llm_provider_config = self.llm_provider._configuration.copy(deep=True)
+        task_llm_provider_config = self.llm_provider._configuration.model_copy(
+            deep=True
+        )
         _extra_request_headers = task_llm_provider_config.extra_request_headers
         _extra_request_headers["AP-TaskID"] = task.task_id
         if step_id:
@@ -459,7 +461,7 @@ class AgentProtocolServer:
         if task.additional_input and (user_id := task.additional_input.get("user_id")):
             _extra_request_headers["AutoGPT-UserID"] = user_id
 
-        settings = self.llm_provider._settings.copy()
+        settings = self.llm_provider._settings.model_copy()
         settings.budget = task_llm_budget
         settings.configuration = task_llm_provider_config
         task_llm_provider = self.llm_provider.__class__(

autogpt/autogpt/app/cli.py

@@ -28,24 +28,6 @@ def cli(ctx: click.Context):
     help="Defines the number of times to run in continuous mode",
 )
 @click.option("--speak", is_flag=True, help="Enable Speak Mode")
-@click.option(
-    "-b",
-    "--browser-name",
-    help="Specifies which web-browser to use when using selenium to scrape the web.",
-)
-@click.option(
-    "--allow-downloads",
-    is_flag=True,
-    help="Dangerous: Allows AutoGPT to download files natively.",
-)
-@click.option(
-    # TODO: this is a hidden option for now, necessary for integration testing.
-    # We should make this public once we're ready to roll out agent specific workspaces.
-    "--workspace-directory",
-    "-w",
-    type=click.Path(file_okay=False),
-    hidden=True,
-)
 @click.option(
     "--install-plugin-deps",
     is_flag=True,
@@ -128,13 +110,15 @@ def cli(ctx: click.Context):
     ),
     type=click.Choice([i.value for i in LogFormatName]),
 )
+@click.option(
+    "--component-config-file",
+    help="Path to a json configuration file",
+    type=click.Path(exists=True, dir_okay=False, resolve_path=True, path_type=Path),
+)
 def run(
     continuous: bool,
     continuous_limit: Optional[int],
     speak: bool,
-    browser_name: Optional[str],
-    allow_downloads: bool,
-    workspace_directory: Optional[Path],
     install_plugin_deps: bool,
     skip_news: bool,
     skip_reprompt: bool,
@@ -148,6 +132,7 @@ def run(
     log_level: Optional[str],
     log_format: Optional[str],
     log_file_format: Optional[str],
+    component_config_file: Optional[Path],
 ) -> None:
     """
     Sets up and runs an agent, based on the task specified by the user, or resumes an
@@ -165,10 +150,7 @@ def run(
         log_level=log_level,
         log_format=log_format,
         log_file_format=log_file_format,
-        browser_name=browser_name,
-        allow_downloads=allow_downloads,
         skip_news=skip_news,
-        workspace_directory=workspace_directory,
         install_plugin_deps=install_plugin_deps,
         override_ai_name=ai_name,
         override_ai_role=ai_role,
@@ -176,20 +158,11 @@ def run(
         constraints=list(constraint),
         best_practices=list(best_practice),
         override_directives=override_directives,
+        component_config_file=component_config_file,
     )
 
 
 @cli.command()
-@click.option(
-    "-b",
-    "--browser-name",
-    help="Specifies which web-browser to use when using selenium to scrape the web.",
-)
-@click.option(
-    "--allow-downloads",
-    is_flag=True,
-    help="Dangerous: Allows AutoGPT to download files natively.",
-)
 @click.option(
     "--install-plugin-deps",
     is_flag=True,
@@ -217,8 +190,6 @@ def run(
     type=click.Choice([i.value for i in LogFormatName]),
 )
 def serve(
-    browser_name: Optional[str],
-    allow_downloads: bool,
     install_plugin_deps: bool,
     debug: bool,
     log_level: Optional[str],
@@ -237,8 +208,6 @@ def serve(
         log_level=log_level,
         log_format=log_format,
         log_file_format=log_file_format,
-        browser_name=browser_name,
-        allow_downloads=allow_downloads,
         install_plugin_deps=install_plugin_deps,
     )
 

autogpt/autogpt/app/config.py

@@ -0,0 +1,221 @@
+"""Configuration class to store the state of bools for different scripts access."""
+from __future__ import annotations
+
+import logging
+import os
+import re
+from pathlib import Path
+from typing import Optional, Union
+
+import forge
+from forge.config.base import BaseConfig
+from forge.llm.providers import CHAT_MODELS, ModelName
+from forge.llm.providers.openai import OpenAICredentials, OpenAIModelName
+from forge.logging.config import LoggingConfig
+from forge.models.config import Configurable, UserConfigurable
+from pydantic import SecretStr, ValidationInfo, field_validator
+
+logger = logging.getLogger(__name__)
+
+PROJECT_ROOT = Path(forge.__file__).parent.parent
+AZURE_CONFIG_FILE = Path("azure.yaml")
+
+GPT_4_MODEL = OpenAIModelName.GPT4
+GPT_3_MODEL = OpenAIModelName.GPT3
+
+
+class AppConfig(BaseConfig):
+    name: str = "Auto-GPT configuration"
+    description: str = "Default configuration for the Auto-GPT application."
+
+    ########################
+    # Application Settings #
+    ########################
+    project_root: Path = PROJECT_ROOT
+    app_data_dir: Path = project_root / "data"
+    skip_news: bool = False
+    skip_reprompt: bool = False
+    authorise_key: str = UserConfigurable(default="y", from_env="AUTHORISE_COMMAND_KEY")
+    exit_key: str = UserConfigurable(default="n", from_env="EXIT_KEY")
+    noninteractive_mode: bool = False
+    logging: LoggingConfig = LoggingConfig()
+    component_config_file: Optional[Path] = UserConfigurable(
+        default=None, from_env="COMPONENT_CONFIG_FILE"
+    )
+
+    ##########################
+    # Agent Control Settings #
+    ##########################
+    # Model configuration
+    fast_llm: ModelName = UserConfigurable(
+        default=OpenAIModelName.GPT3,
+        from_env="FAST_LLM",
+    )
+    smart_llm: ModelName = UserConfigurable(
+        default=OpenAIModelName.GPT4_TURBO,
+        from_env="SMART_LLM",
+    )
+    temperature: float = UserConfigurable(default=0, from_env="TEMPERATURE")
+    openai_functions: bool = UserConfigurable(
+        default=False, from_env=lambda: os.getenv("OPENAI_FUNCTIONS", "False") == "True"
+    )
+    embedding_model: str = UserConfigurable(
+        default="text-embedding-3-small", from_env="EMBEDDING_MODEL"
+    )
+
+    # Run loop configuration
+    continuous_mode: bool = False
+    continuous_limit: int = 0
+
+    ############
+    # Commands #
+    ############
+    # General
+    disabled_commands: list[str] = UserConfigurable(
+        default_factory=list,
+        from_env=lambda: _safe_split(os.getenv("DISABLED_COMMANDS")),
+    )
+
+    # File ops
+    restrict_to_workspace: bool = UserConfigurable(
+        default=True,
+        from_env=lambda: os.getenv("RESTRICT_TO_WORKSPACE", "True") == "True",
+    )
+
+    ###############
+    # Credentials #
+    ###############
+    # OpenAI
+    openai_credentials: Optional[OpenAICredentials] = None
+    azure_config_file: Optional[Path] = UserConfigurable(
+        default=AZURE_CONFIG_FILE, from_env="AZURE_CONFIG_FILE"
+    )
+
+    @field_validator("openai_functions")
+    def validate_openai_functions(cls, value: bool, info: ValidationInfo):
+        if value:
+            smart_llm = info.data["smart_llm"]
+            assert CHAT_MODELS[smart_llm].has_function_call_api, (
+                f"Model {smart_llm} does not support tool calling. "
+                "Please disable OPENAI_FUNCTIONS or choose a suitable model."
+            )
+        return value
+
+
+class ConfigBuilder(Configurable[AppConfig]):
+    default_settings = AppConfig()
+
+    @classmethod
+    def build_config_from_env(cls, project_root: Path = PROJECT_ROOT) -> AppConfig:
+        """Initialize the Config class"""
+
+        config = cls.build_agent_configuration()
+        config.project_root = project_root
+
+        # Make relative paths absolute
+        for k in {
+            "azure_config_file",  # TODO: move from project root
+        }:
+            setattr(config, k, project_root / getattr(config, k))
+
+        if (
+            config.openai_credentials
+            and config.openai_credentials.api_type == SecretStr("azure")
+            and (config_file := config.azure_config_file)
+        ):
+            config.openai_credentials.load_azure_config(config_file)
+
+        return config
+
+
+async def assert_config_has_required_llm_api_keys(config: AppConfig) -> None:
+    """
+    Check if API keys (if required) are set for the configured SMART_LLM and FAST_LLM.
+    """
+    from forge.llm.providers.anthropic import AnthropicModelName
+    from forge.llm.providers.groq import GroqModelName
+    from pydantic import ValidationError
+
+    if set((config.smart_llm, config.fast_llm)).intersection(AnthropicModelName):
+        from forge.llm.providers.anthropic import AnthropicCredentials
+
+        try:
+            credentials = AnthropicCredentials.from_env()
+        except ValidationError as e:
+            if "api_key" in str(e):
+                logger.error(
+                    "Set your Anthropic API key in .env or as an environment variable"
+                )
+                logger.info(
+                    "For further instructions: "
+                    "https://docs.agpt.co/autogpt/setup/#anthropic"
+                )
+
+            raise ValueError("Anthropic is unavailable: can't load credentials") from e
+
+        key_pattern = r"^sk-ant-api03-[\w\-]{95}"
+
+        # If key is set, but it looks invalid
+        if not re.search(key_pattern, credentials.api_key.get_secret_value()):
+            logger.warning(
+                "Possibly invalid Anthropic API key! "
+                f"Configured Anthropic API key does not match pattern '{key_pattern}'. "
+                "If this is a valid key, please report this warning to the maintainers."
+            )
+
+    if set((config.smart_llm, config.fast_llm)).intersection(GroqModelName):
+        from forge.llm.providers.groq import GroqProvider
+        from groq import AuthenticationError
+
+        try:
+            groq = GroqProvider()
+            await groq.get_available_models()
+        except ValidationError as e:
+            if "api_key" not in str(e):
+                raise
+
+            logger.error("Set your Groq API key in .env or as an environment variable")
+            logger.info(
+                "For further instructions: https://docs.agpt.co/autogpt/setup/#groq"
+            )
+            raise ValueError("Groq is unavailable: can't load credentials")
+        except AuthenticationError as e:
+            logger.error("The Groq API key is invalid!")
+            logger.info(
+                "For instructions to get and set a new API key: "
+                "https://docs.agpt.co/autogpt/setup/#groq"
+            )
+            raise ValueError("Groq is unavailable: invalid API key") from e
+
+    if set((config.smart_llm, config.fast_llm)).intersection(OpenAIModelName):
+        from forge.llm.providers.openai import OpenAIProvider
+        from openai import AuthenticationError
+
+        try:
+            openai = OpenAIProvider()
+            await openai.get_available_models()
+        except ValidationError as e:
+            if "api_key" not in str(e):
+                raise
+
+            logger.error(
+                "Set your OpenAI API key in .env or as an environment variable"
+            )
+            logger.info(
+                "For further instructions: https://docs.agpt.co/autogpt/setup/#openai"
+            )
+            raise ValueError("OpenAI is unavailable: can't load credentials")
+        except AuthenticationError as e:
+            logger.error("The OpenAI API key is invalid!")
+            logger.info(
+                "For instructions to get and set a new API key: "
+                "https://docs.agpt.co/autogpt/setup/#openai"
+            )
+            raise ValueError("OpenAI is unavailable: invalid API key") from e
+
+
+def _safe_split(s: Union[str, None], sep: str = ",") -> list[str]:
+    """Split a string by a separator. Return an empty list if the string is None."""
+    if s is None:
+        return []
+    return s.split(sep)

autogpt/autogpt/app/configurator.py

@@ -5,20 +5,18 @@ import logging
 from typing import Literal, Optional
 
 import click
-from colorama import Back, Style
-from forge.config.config import GPT_3_MODEL, Config
 from forge.llm.providers import ModelName, MultiProvider
 
+from autogpt.app.config import GPT_3_MODEL, AppConfig
+
 logger = logging.getLogger(__name__)
 
 
 async def apply_overrides_to_config(
-    config: Config,
+    config: AppConfig,
     continuous: bool = False,
     continuous_limit: Optional[int] = None,
     skip_reprompt: bool = False,
-    browser_name: Optional[str] = None,
-    allow_downloads: bool = False,
     skip_news: bool = False,
 ) -> None:
     """Updates the config object with the given arguments.
@@ -33,8 +31,6 @@ async def apply_overrides_to_config(
         log_level (int): The global log level for the application.
         log_format (str): The format for the log(s).
         log_file_format (str): Override the format for the log file.
-        browser_name (str): The name of the browser to use for scraping the web.
-        allow_downloads (bool): Whether to allow AutoGPT to download files natively.
         skips_news (bool): Whether to suppress the output of latest news on startup.
     """
     config.continuous_mode = False
@@ -61,23 +57,6 @@ async def apply_overrides_to_config(
     if skip_reprompt:
         config.skip_reprompt = True
 
-    if browser_name:
-        config.selenium_web_browser = browser_name
-
-    if allow_downloads:
-        logger.warning(
-            msg=f"{Back.LIGHTYELLOW_EX}"
-            "AutoGPT will now be able to download and save files to your machine."
-            f"{Back.RESET}"
-            " It is recommended that you monitor any files it downloads carefully.",
-        )
-        logger.warning(
-            msg=f"{Back.RED + Style.BRIGHT}"
-            "NEVER OPEN FILES YOU AREN'T SURE OF!"
-            f"{Style.RESET_ALL}",
-        )
-        config.allow_downloads = True
-
     if skip_news:
         config.skip_news = True
 

autogpt/autogpt/app/main.py

@@ -21,7 +21,6 @@ from forge.components.code_executor.code_executor import (
 )
 from forge.config.ai_directives import AIDirectives
 from forge.config.ai_profile import AIProfile
-from forge.config.config import Config, ConfigBuilder, assert_config_has_openai_api_key
 from forge.file_storage import FileStorageBackendName, get_storage
 from forge.llm.providers import MultiProvider
 from forge.logging.config import configure_logging
@@ -34,6 +33,11 @@ from forge.utils.exceptions import AgentTerminated, InvalidAgentResponseError
 from autogpt.agent_factory.configurators import configure_agent_with_state, create_agent
 from autogpt.agents.agent_manager import AgentManager
 from autogpt.agents.prompt_strategies.one_shot import AssistantThoughts
+from autogpt.app.config import (
+    AppConfig,
+    ConfigBuilder,
+    assert_config_has_required_llm_api_keys,
+)
 
 if TYPE_CHECKING:
     from autogpt.agents.agent import Agent
@@ -62,10 +66,7 @@ async def run_auto_gpt(
     log_level: Optional[str] = None,
     log_format: Optional[str] = None,
     log_file_format: Optional[str] = None,
-    browser_name: Optional[str] = None,
-    allow_downloads: bool = False,
     skip_news: bool = False,
-    workspace_directory: Optional[Path] = None,
     install_plugin_deps: bool = False,
     override_ai_name: Optional[str] = None,
     override_ai_role: Optional[str] = None,
@@ -73,6 +74,7 @@ async def run_auto_gpt(
     constraints: Optional[list[str]] = None,
     best_practices: Optional[list[str]] = None,
     override_directives: bool = False,
+    component_config_file: Optional[Path] = None,
 ):
     # Set up configuration
     config = ConfigBuilder.build_config_from_env()
@@ -98,16 +100,13 @@ async def run_auto_gpt(
         tts_config=config.tts_config,
     )
 
-    # TODO: fill in llm values here
-    assert_config_has_openai_api_key(config)
+    await assert_config_has_required_llm_api_keys(config)
 
     await apply_overrides_to_config(
         config=config,
         continuous=continuous,
         continuous_limit=continuous_limit,
         skip_reprompt=skip_reprompt,
-        browser_name=browser_name,
-        allow_downloads=allow_downloads,
         skip_news=skip_news,
     )
 
@@ -132,15 +131,12 @@ async def run_auto_gpt(
         print_python_version_info(logger)
         print_attribute("Smart LLM", config.smart_llm)
         print_attribute("Fast LLM", config.fast_llm)
-        print_attribute("Browser", config.selenium_web_browser)
         if config.continuous_mode:
             print_attribute("Continuous Mode", "ENABLED", title_color=Fore.YELLOW)
             if continuous_limit:
                 print_attribute("Continuous Limit", config.continuous_limit)
         if config.tts_config.speak_mode:
             print_attribute("Speak Mode", "ENABLED")
-        if config.allow_downloads:
-            print_attribute("Native Downloading", "ENABLED")
         if we_are_running_in_a_docker_container() or is_docker_available():
             print_attribute("Code Execution", "ENABLED")
         else:
@@ -327,6 +323,14 @@ async def run_auto_gpt(
         #     )
         # ).add_done_callback(update_agent_directives)
 
+    # Load component configuration from file
+    if _config_file := component_config_file or config.component_config_file:
+        try:
+            logger.info(f"Loading component configuration from {_config_file}")
+            agent.load_component_configs(_config_file.read_text())
+        except Exception as e:
+            logger.error(f"Could not load component configuration: {e}")
+
     #################
     # Run the Agent #
     #################
@@ -353,8 +357,6 @@ async def run_auto_gpt_server(
     log_level: Optional[str] = None,
     log_format: Optional[str] = None,
     log_file_format: Optional[str] = None,
-    browser_name: Optional[str] = None,
-    allow_downloads: bool = False,
     install_plugin_deps: bool = False,
 ):
     from .agent_protocol_server import AgentProtocolServer
@@ -380,13 +382,10 @@ async def run_auto_gpt_server(
         tts_config=config.tts_config,
     )
 
-    # TODO: fill in llm values here
-    assert_config_has_openai_api_key(config)
+    await assert_config_has_required_llm_api_keys(config)
 
     await apply_overrides_to_config(
         config=config,
-        browser_name=browser_name,
-        allow_downloads=allow_downloads,
     )
 
     llm_provider = _configure_llm_provider(config)
@@ -411,7 +410,7 @@ async def run_auto_gpt_server(
     )
 
 
-def _configure_llm_provider(config: Config) -> MultiProvider:
+def _configure_llm_provider(config: AppConfig) -> MultiProvider:
     multi_provider = MultiProvider()
     for model in [config.smart_llm, config.fast_llm]:
         # Ensure model providers for configured LLMs are available
@@ -451,15 +450,15 @@ async def run_interaction_loop(
         None
     """
     # These contain both application config and agent config, so grab them here.
-    legacy_config = agent.legacy_config
+    app_config = agent.app_config
     ai_profile = agent.state.ai_profile
     logger = logging.getLogger(__name__)
 
     cycle_budget = cycles_remaining = _get_cycle_budget(
-        legacy_config.continuous_mode, legacy_config.continuous_limit
+        app_config.continuous_mode, app_config.continuous_limit
     )
     spinner = Spinner(
-        "Thinking...", plain_output=legacy_config.logging.plain_console_output
+        "Thinking...", plain_output=app_config.logging.plain_console_output
     )
     stop_reason = None
 
@@ -508,22 +507,25 @@ async def run_interaction_loop(
         ########
         handle_stop_signal()
         # Have the agent determine the next action to take.
-        with spinner:
-            try:
-                action_proposal = await agent.propose_action()
-            except InvalidAgentResponseError as e:
-                logger.warning(f"The agent's thoughts could not be parsed: {e}")
-                consecutive_failures += 1
-                if consecutive_failures >= 3:
-                    logger.error(
-                        "The agent failed to output valid thoughts"
-                        f" {consecutive_failures} times in a row. Terminating..."
-                    )
-                    raise AgentTerminated(
-                        "The agent failed to output valid thoughts"
-                        f" {consecutive_failures} times in a row."
-                    )
-                continue
+        if not (_ep := agent.event_history.current_episode) or _ep.result:
+            with spinner:
+                try:
+                    action_proposal = await agent.propose_action()
+                except InvalidAgentResponseError as e:
+                    logger.warning(f"The agent's thoughts could not be parsed: {e}")
+                    consecutive_failures += 1
+                    if consecutive_failures >= 3:
+                        logger.error(
+                            "The agent failed to output valid thoughts"
+                            f" {consecutive_failures} times in a row. Terminating..."
+                        )
+                        raise AgentTerminated(
+                            "The agent failed to output valid thoughts"
+                            f" {consecutive_failures} times in a row."
+                        )
+                    continue
+        else:
+            action_proposal = _ep.action
 
         consecutive_failures = 0
 
@@ -534,7 +536,7 @@ async def run_interaction_loop(
         update_user(
             ai_profile,
             action_proposal,
-            speak_mode=legacy_config.tts_config.speak_mode,
+            speak_mode=app_config.tts_config.speak_mode,
         )
 
         ##################
@@ -543,7 +545,7 @@ async def run_interaction_loop(
         handle_stop_signal()
         if cycles_remaining == 1:  # Last cycle
             feedback_type, feedback, new_cycles_remaining = await get_user_feedback(
-                legacy_config,
+                app_config,
                 ai_profile,
             )
 
@@ -654,7 +656,7 @@ def update_user(
 
 
 async def get_user_feedback(
-    config: Config,
+    config: AppConfig,
     ai_profile: AIProfile,
 ) -> tuple[UserFeedback, str, int | None]:
     """Gets the user's feedback on the assistant's reply.

autogpt/autogpt/app/setup.py

@@ -4,9 +4,10 @@ from typing import Optional
 
 from forge.config.ai_directives import AIDirectives
 from forge.config.ai_profile import AIProfile
-from forge.config.config import Config
 from forge.logging.utils import print_attribute
 
+from autogpt.app.config import AppConfig
+
 from .input import clean_input
 
 logger = logging.getLogger(__name__)
@@ -46,7 +47,7 @@ def apply_overrides_to_ai_settings(
 async def interactively_revise_ai_settings(
     ai_profile: AIProfile,
     directives: AIDirectives,
-    app_config: Config,
+    app_config: AppConfig,
 ):
     """Interactively revise the AI settings.
 

autogpt/pyproject.toml

@@ -20,44 +20,24 @@ serve = "autogpt.app.cli:serve"
 
 [tool.poetry.dependencies]
 python = "^3.10"
-anthropic = "^0.25.1"
 autogpt-forge = { path = "../forge", develop = true }
 # autogpt-forge = {git = "https://github.com/Significant-Gravitas/AutoGPT.git", subdirectory = "forge"}
-beautifulsoup4 = "^4.12.2"
-charset-normalizer = "^3.1.0"
 click = "*"
 colorama = "^0.4.6"
 distro = "^1.8.0"
-en-core-web-sm = { url = "https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl" }
 fastapi = "^0.109.1"
-ftfy = "^6.1.1"
-google-api-python-client = "*"
+gitpython = "^3.1.32"
 hypercorn = "^0.14.4"
-inflection = "*"
-jsonschema = "*"
-numpy = "*"
 openai = "^1.7.2"
 orjson = "^3.8.10"
-Pillow = "*"
-pydantic = "*"
-python-docx = "*"
+pydantic = "^2.7.2"
 python-dotenv = "^1.0.0"
-pyyaml = "^6.0"
-readability-lxml = "^0.8.1"
 requests = "*"
 sentry-sdk = "^1.40.4"
-spacy = "^3.7.4"
-tenacity = "^8.2.2"
-
-# OpenAI and Generic plugins import
-openapi-python-client = "^0.14.0"
 
 # Benchmarking
 agbenchmark = { path = "../benchmark", optional = true }
 # agbenchmark = {git = "https://github.com/Significant-Gravitas/AutoGPT.git", subdirectory = "benchmark", optional = true}
-psycopg2-binary = "^2.9.9"
-multidict = "6.0.5"
-cx-freeze = "7.0.0"
 
 [tool.poetry.extras]
 benchmark = ["agbenchmark"]
@@ -65,27 +45,28 @@ benchmark = ["agbenchmark"]
 [tool.poetry.group.dev.dependencies]
 black = "^23.12.1"
 flake8 = "^7.0.0"
-gitpython = "^3.1.32"
 isort = "^5.13.1"
 pre-commit = "*"
 pyright = "^1.1.364"
-types-beautifulsoup4 = "*"
+
+# Type stubs
 types-colorama = "*"
-types-Markdown = "*"
-types-Pillow = "*"
 
 # Testing
-asynctest = "*"
-coverage = "*"
 pytest = "*"
 pytest-asyncio = "*"
-pytest-benchmark = "*"
 pytest-cov = "*"
-pytest-integration = "*"
 pytest-mock = "*"
 pytest-recording = "*"
 pytest-xdist = "*"
-vcrpy = { git = "https://github.com/Significant-Gravitas/vcrpy.git", rev = "master" }
+
+[tool.poetry.group.build]
+optional = true
+
+[tool.poetry.group.build.dependencies]
+cx-freeze = { git = "https://github.com/ntindle/cx_Freeze.git", rev = "main" }
+# HACK: switch to cx-freeze release package after #2442 and #2472 are merged: https://github.com/marcelotduarte/cx_Freeze/pulls?q=is:pr+%232442+OR+%232472+
+# cx-freeze = { version = "^7.2.0", optional = true }
 
 
 [build-system]
@@ -108,7 +89,3 @@ skip_glob = ["data"]
 pythonVersion = "3.10"
 exclude = ["data/**", "**/node_modules", "**/__pycache__", "**/.*"]
 ignore = ["../forge/**"]
-
-
-[tool.pytest.ini_options]
-markers = ["slow", "requires_openai_api_key", "requires_huggingface_api_key"]

autogpt/scripts/git_log_to_release_notes.py

@@ -19,7 +19,7 @@ from autogpt.app.utils import coroutine
     help="Path to the git repository",
 )
 @coroutine
-async def generate_release_notes(repo_path: Optional[Path] = None):
+async def generate_release_notes(repo_path: Optional[str | Path] = None):
     logger = logging.getLogger(generate_release_notes.name)  # pyright: ignore
 
     repo = Repo(repo_path, search_parent_directories=True)

autogpt/scripts/llamafile/.gitignore

@@ -0,0 +1,3 @@
+*.llamafile
+*.llamafile.exe
+llamafile.exe

autogpt/scripts/llamafile/serve.py

@@ -0,0 +1,165 @@
+#!/usr/bin/env python3
+"""
+Use llamafile to serve a (quantized) mistral-7b-instruct-v0.2 model
+
+Usage:
+  cd <repo-root>/autogpt
+  ./scripts/llamafile/serve.py
+"""
+
+import os
+import platform
+import subprocess
+from pathlib import Path
+from typing import Optional
+
+import click
+
+LLAMAFILE = Path("mistral-7b-instruct-v0.2.Q5_K_M.llamafile")
+LLAMAFILE_URL = f"https://huggingface.co/jartine/Mistral-7B-Instruct-v0.2-llamafile/resolve/main/{LLAMAFILE.name}"  # noqa
+LLAMAFILE_EXE = Path("llamafile.exe")
+LLAMAFILE_EXE_URL = "https://github.com/Mozilla-Ocho/llamafile/releases/download/0.8.6/llamafile-0.8.6"  # noqa
+
+
+@click.command()
+@click.option(
+    "--llamafile",
+    type=click.Path(dir_okay=False, path_type=Path),
+    help=f"Name of the llamafile to serve. Default: {LLAMAFILE.name}",
+)
+@click.option("--llamafile_url", help="Download URL for the llamafile you want to use")
+@click.option(
+    "--host", help="Specify the address for the llamafile server to listen on"
+)
+@click.option(
+    "--port", type=int, help="Specify the port for the llamafile server to listen on"
+)
+@click.option(
+    "--force-gpu",
+    is_flag=True,
+    hidden=platform.system() != "Darwin",
+    help="Run the model using only the GPU (AMD or Nvidia). "
+    "Otherwise, both CPU and GPU may be (partially) used.",
+)
+def main(
+    llamafile: Optional[Path] = None,
+    llamafile_url: Optional[str] = None,
+    host: Optional[str] = None,
+    port: Optional[int] = None,
+    force_gpu: bool = False,
+):
+    print(f"type(llamafile) = {type(llamafile)}")
+    if not llamafile:
+        if not llamafile_url:
+            llamafile = LLAMAFILE
+        else:
+            llamafile = Path(llamafile_url.rsplit("/", 1)[1])
+            if llamafile.suffix != ".llamafile":
+                click.echo(
+                    click.style(
+                        "The given URL does not end with '.llamafile' -> "
+                        "can't get filename from URL. "
+                        "Specify the filename using --llamafile.",
+                        fg="red",
+                    ),
+                    err=True,
+                )
+                return
+
+    if llamafile == LLAMAFILE and not llamafile_url:
+        llamafile_url = LLAMAFILE_URL
+    elif llamafile_url != LLAMAFILE_URL:
+        if not click.prompt(
+            click.style(
+                "You seem to have specified a different URL for the default model "
+                f"({llamafile.name}). Are you sure this is correct? "
+                "If you want to use a different model, also specify --llamafile.",
+                fg="yellow",
+            ),
+            type=bool,
+        ):
+            return
+
+    # Go to autogpt/scripts/llamafile/
+    os.chdir(Path(__file__).resolve().parent)
+
+    on_windows = platform.system() == "Windows"
+
+    if not llamafile.is_file():
+        if not llamafile_url:
+            click.echo(
+                click.style(
+                    "Please use --lamafile_url to specify a download URL for "
+                    f"'{llamafile.name}'. "
+                    "This will only be necessary once, so we can download the model.",
+                    fg="red",
+                ),
+                err=True,
+            )
+            return
+
+        download_file(llamafile_url, llamafile)
+
+        if not on_windows:
+            llamafile.chmod(0o755)
+            subprocess.run([llamafile, "--version"], check=True)
+
+    if not on_windows:
+        base_command = [f"./{llamafile}"]
+    else:
+        # Windows does not allow executables over 4GB, so we have to download a
+        # model-less llamafile.exe and run that instead.
+        if not LLAMAFILE_EXE.is_file():
+            download_file(LLAMAFILE_EXE_URL, LLAMAFILE_EXE)
+            LLAMAFILE_EXE.chmod(0o755)
+            subprocess.run([f".\\{LLAMAFILE_EXE}", "--version"], check=True)
+
+        base_command = [f".\\{LLAMAFILE_EXE}", "-m", llamafile]
+
+    if host:
+        base_command.extend(["--host", host])
+    if port:
+        base_command.extend(["--port", str(port)])
+    if force_gpu:
+        base_command.extend(["-ngl", "9999"])
+
+    subprocess.run(
+        [
+            *base_command,
+            "--server",
+            "--nobrowser",
+            "--ctx-size",
+            "0",
+            "--n-predict",
+            "1024",
+        ],
+        check=True,
+    )
+
+    # note: --ctx-size 0 means the prompt context size will be set directly from the
+    # underlying model configuration. This may cause slow response times or consume
+    # a lot of memory.
+
+
+def download_file(url: str, to_file: Path) -> None:
+    print(f"Downloading {to_file.name}...")
+    import urllib.request
+
+    urllib.request.urlretrieve(url, to_file, reporthook=report_download_progress)
+    print()
+
+
+def report_download_progress(chunk_number: int, chunk_size: int, total_size: int):
+    if total_size != -1:
+        downloaded_size = chunk_number * chunk_size
+        percent = min(1, downloaded_size / total_size)
+        bar = "#" * int(40 * percent)
+        print(
+            f"\rDownloading: [{bar:<40}] {percent:.0%}"
+            f" - {downloaded_size/1e6:.1f}/{total_size/1e6:.1f} MB",
+            end="",
+        )
+
+
+if __name__ == "__main__":
+    main()

autogpt/setup.py

@@ -1,7 +1,9 @@
+import platform
+from pathlib import Path
 from pkgutil import iter_modules
-from shutil import which
+from typing import Union
 
-from cx_Freeze import Executable, setup
+from cx_Freeze import Executable, setup  # type: ignore
 
 packages = [
     m.name
@@ -11,11 +13,47 @@ packages = [
     and ("poetry" in m.module_finder.path)  # type: ignore
 ]
 
-icon = (
-    "../../assets/gpt_dark_RGB.icns"
-    if which("sips")
-    else "../../assets/gpt_dark_RGB.ico"
-)
+# set the icon based on the platform
+icon = "../../assets/gpt_dark_RGB.ico"
+if platform.system() == "Darwin":
+    icon = "../../assets/gpt_dark_RGB.icns"
+elif platform.system() == "Linux":
+    icon = "../../assets/gpt_dark_RGB.png"
+
+
+def txt_to_rtf(input_file: Union[str, Path], output_file: Union[str, Path]) -> None:
+    """
+    Convert a text file to RTF format.
+
+    Args:
+    input_file (Union[str, Path]): Path to the input text file.
+    output_file (Union[str, Path]): Path to the output RTF file.
+
+    Returns:
+    None
+    """
+    input_path = Path(input_file)
+    output_path = Path(output_file)
+
+    with input_path.open("r", encoding="utf-8") as txt_file:
+        content = txt_file.read()
+
+    # RTF header
+    rtf = r"{\rtf1\ansi\deff0 {\fonttbl {\f0 Times New Roman;}}\f0\fs24 "
+
+    # Replace newlines with RTF newline
+    rtf += content.replace("\n", "\\par ")
+
+    # Close RTF document
+    rtf += "}"
+
+    with output_path.open("w", encoding="utf-8") as rtf_file:
+        rtf_file.write(rtf)
+
+
+# Convert LICENSE to LICENSE.rtf
+license_file = "LICENSE.rtf"
+txt_to_rtf("../LICENSE", license_file)
 
 
 setup(
@@ -55,6 +93,7 @@ setup(
             "target_name": "AutoGPT",
             "add_to_path": True,
             "install_icon": "../assets/gpt_dark_RGB.ico",
+            "license_file": license_file,
         },
     },
 )

autogpt/tests/conftest.py

@@ -6,7 +6,6 @@ from pathlib import Path
 
 import pytest
 from forge.config.ai_profile import AIProfile
-from forge.config.config import Config, ConfigBuilder
 from forge.file_storage.local import (
     FileStorage,
     FileStorageConfiguration,
@@ -16,11 +15,11 @@ from forge.llm.providers import MultiProvider
 from forge.logging.config import configure_logging
 
 from autogpt.agents.agent import Agent, AgentConfiguration, AgentSettings
+from autogpt.app.config import AppConfig, ConfigBuilder
 from autogpt.app.main import _configure_llm_provider
 
 pytest_plugins = [
     "tests.integration.agent_factory",
-    "tests.vcr",
 ]
 
 
@@ -62,7 +61,7 @@ def config(
 
 
 @pytest.fixture(scope="session")
-def setup_logger(config: Config):
+def setup_logger():
     configure_logging(
         debug=True,
         log_dir=Path(__file__).parent / "logs",
@@ -71,12 +70,14 @@ def setup_logger(config: Config):
 
 
 @pytest.fixture
-def llm_provider(config: Config) -> MultiProvider:
+def llm_provider(config: AppConfig) -> MultiProvider:
     return _configure_llm_provider(config)
 
 
 @pytest.fixture
-def agent(config: Config, llm_provider: MultiProvider, storage: FileStorage) -> Agent:
+def agent(
+    config: AppConfig, llm_provider: MultiProvider, storage: FileStorage
+) -> Agent:
     ai_profile = AIProfile(
         ai_name="Base",
         ai_role="A base AI",
@@ -94,13 +95,13 @@ def agent(config: Config, llm_provider: MultiProvider, storage: FileStorage) ->
             allow_fs_access=not config.restrict_to_workspace,
             use_functions_api=config.openai_functions,
         ),
-        history=Agent.default_settings.history.copy(deep=True),
+        history=Agent.default_settings.history.model_copy(deep=True),
     )
 
     agent = Agent(
         settings=agent_settings,
         llm_provider=llm_provider,
         file_storage=storage,
-        legacy_config=config,
+        app_config=config,
     )
     return agent

autogpt/tests/integration/agent_factory.py

@@ -2,15 +2,15 @@ from pathlib import Path
 
 import pytest
 from forge.config.ai_profile import AIProfile
-from forge.config.config import Config
 from forge.file_storage import FileStorageBackendName, get_storage
 from forge.llm.providers import MultiProvider
 
 from autogpt.agents.agent import Agent, AgentConfiguration, AgentSettings
+from autogpt.app.config import AppConfig
 
 
 @pytest.fixture
-def dummy_agent(config: Config, llm_provider: MultiProvider):
+def dummy_agent(config: AppConfig, llm_provider: MultiProvider):
     ai_profile = AIProfile(
         ai_name="Dummy Agent",
         ai_role="Dummy Role",
@@ -28,7 +28,7 @@ def dummy_agent(config: Config, llm_provider: MultiProvider):
             smart_llm=config.smart_llm,
             use_functions_api=config.openai_functions,
         ),
-        history=Agent.default_settings.history.copy(deep=True),
+        history=Agent.default_settings.history.model_copy(deep=True),
     )
 
     local = config.file_storage_backend == FileStorageBackendName.LOCAL
@@ -44,7 +44,7 @@ def dummy_agent(config: Config, llm_provider: MultiProvider):
         settings=agent_settings,
         llm_provider=llm_provider,
         file_storage=file_storage,
-        legacy_config=config,
+        app_config=config,
     )
 
     return agent

autogpt/tests/integration/test_setup.py

@@ -3,8 +3,8 @@ from unittest.mock import patch
 import pytest
 from forge.config.ai_directives import AIDirectives
 from forge.config.ai_profile import AIProfile
-from forge.config.config import Config
 
+from autogpt.app.config import AppConfig
 from autogpt.app.setup import (
     apply_overrides_to_ai_settings,
     interactively_revise_ai_settings,
@@ -39,7 +39,7 @@ async def test_apply_overrides_to_ai_settings():
 
 
 @pytest.mark.asyncio
-async def test_interactively_revise_ai_settings(config: Config):
+async def test_interactively_revise_ai_settings(config: AppConfig):
     ai_profile = AIProfile(ai_name="Test AI", ai_role="Test Role")
     directives = AIDirectives(
         resources=["Resource1"],

autogpt/tests/unit/test_config.py

@@ -8,15 +8,15 @@ from typing import Any
 from unittest import mock
 
 import pytest
-from forge.config.config import GPT_3_MODEL, GPT_4_MODEL, Config, ConfigBuilder
 from openai.pagination import AsyncPage
 from openai.types import Model
 from pydantic import SecretStr
 
+from autogpt.app.config import GPT_3_MODEL, GPT_4_MODEL, AppConfig, ConfigBuilder
 from autogpt.app.configurator import apply_overrides_to_config
 
 
-def test_initial_values(config: Config) -> None:
+def test_initial_values(config: AppConfig) -> None:
     """
     Test if the initial values of the config class attributes are set correctly.
     """
@@ -29,7 +29,7 @@ def test_initial_values(config: Config) -> None:
 @pytest.mark.asyncio
 @mock.patch("openai.resources.models.AsyncModels.list")
 async def test_fallback_to_gpt3_if_gpt4_not_available(
-    mock_list_models: Any, config: Config
+    mock_list_models: Any, config: AppConfig
 ) -> None:
     """
     Test if models update to gpt-3.5-turbo if gpt-4 is not available.
@@ -51,7 +51,7 @@ async def test_fallback_to_gpt3_if_gpt4_not_available(
     assert config.smart_llm == GPT_3_MODEL
 
 
-def test_missing_azure_config(config: Config) -> None:
+def test_missing_azure_config(config: AppConfig) -> None:
     assert config.openai_credentials is not None
 
     config_file = config.app_data_dir / "azure_config.yaml"
@@ -68,7 +68,7 @@ def test_missing_azure_config(config: Config) -> None:
 
 
 @pytest.fixture
-def config_with_azure(config: Config):
+def config_with_azure(config: AppConfig):
     config_file = config.app_data_dir / "azure_config.yaml"
     config_file.write_text(
         f"""
@@ -91,7 +91,7 @@ azure_model_map:
     del os.environ["AZURE_CONFIG_FILE"]
 
 
-def test_azure_config(config_with_azure: Config) -> None:
+def test_azure_config(config_with_azure: AppConfig) -> None:
     assert (credentials := config_with_azure.openai_credentials) is not None
     assert credentials.api_type == SecretStr("azure")
     assert credentials.api_version == SecretStr("2023-06-01-preview")

benchmark/agbenchmark/__main__.py

@@ -97,7 +97,9 @@ def start():
     help="Write log output to a file instead of the terminal.",
 )
 # @click.argument(
-#     "agent_path", type=click.Path(exists=True, file_okay=False), required=False
+#     "agent_path",
+#     type=click.Path(exists=True, file_okay=False, path_type=Path),
+#     required=False,
 # )
 def run(
     maintain: bool,
@@ -276,7 +278,9 @@ def list_challenges(include_unavailable: bool, only_names: bool, output_json: bo
         return
 
     if output_json:
-        click.echo(json.dumps([json.loads(c.info.json()) for c in challenges]))
+        click.echo(
+            json.dumps([json.loads(c.info.model_dump_json()) for c in challenges])
+        )
         return
 
     headers = tuple(
@@ -324,7 +328,7 @@ def info(name: str, json: bool):
             continue
 
         if json:
-            click.echo(challenge.info.json())
+            click.echo(challenge.info.model_dump_json())
             break
 
         pretty_print_model(challenge.info)

benchmark/agbenchmark/app.py

@@ -16,7 +16,7 @@ from agent_protocol_client import AgentApi, ApiClient, ApiException, Configurati
 from agent_protocol_client.models import Task, TaskRequestBody
 from fastapi import APIRouter, FastAPI, HTTPException, Request, Response
 from fastapi.middleware.cors import CORSMiddleware
-from pydantic import BaseModel, Extra, ValidationError
+from pydantic import BaseModel, ConfigDict, ValidationError
 
 from agbenchmark.challenges import ChallengeInfo
 from agbenchmark.config import AgentBenchmarkConfig
@@ -52,7 +52,9 @@ while challenge_spec_files:
 
     logger.debug(f"Loading {challenge_relpath}...")
     try:
-        challenge_info = ChallengeInfo.parse_file(challenge_spec_file)
+        challenge_info = ChallengeInfo.model_validate_json(
+            challenge_spec_file.read_text()
+        )
     except ValidationError as e:
         if logging.getLogger().level == logging.DEBUG:
             logger.warning(f"Spec file {challenge_relpath} failed to load:\n{e}")
@@ -64,7 +66,7 @@ while challenge_spec_files:
         challenge_info.eval_id = str(uuid.uuid4())
         # this will sort all the keys of the JSON systematically
         # so that the order is always the same
-        write_pretty_json(challenge_info.dict(), challenge_spec_file)
+        write_pretty_json(challenge_info.model_dump(), challenge_spec_file)
 
     CHALLENGES[challenge_info.eval_id] = challenge_info
 
@@ -111,8 +113,7 @@ class CreateReportRequest(BaseModel):
     # category: Optional[str] = []
     mock: Optional[bool] = False
 
-    class Config:
-        extra = Extra.forbid  # this will forbid any extra fields
+    model_config = ConfigDict(extra="forbid")
 
 
 updates_list = []
@@ -153,7 +154,7 @@ def setup_fastapi_app(agbenchmark_config: AgentBenchmarkConfig) -> FastAPI:
         pids = find_agbenchmark_without_uvicorn()
         logger.info(f"pids already running with agbenchmark: {pids}")
 
-        logger.debug(f"Request to /reports: {body.dict()}")
+        logger.debug(f"Request to /reports: {body.model_dump()}")
 
         # Start the benchmark in a separate thread
         benchmark_process = Process(
@@ -326,7 +327,9 @@ def setup_fastapi_app(agbenchmark_config: AgentBenchmarkConfig) -> FastAPI:
                 config={},
             )
 
-            logger.debug(f"Returning evaluation data:\n{eval_info.json(indent=4)}")
+            logger.debug(
+                f"Returning evaluation data:\n{eval_info.model_dump_json(indent=4)}"
+            )
             return eval_info
         except ApiException as e:
             logger.error(f"Error {e} whilst trying to evaluate task: {task_id}")

benchmark/agbenchmark/challenges/builtin.py

@@ -15,7 +15,13 @@ from agent_protocol_client import Configuration as ClientConfig
 from agent_protocol_client import Step
 from colorama import Fore, Style
 from openai import _load_client as get_openai_client
-from pydantic import BaseModel, Field, constr, validator
+from pydantic import (
+    BaseModel,
+    Field,
+    StringConstraints,
+    ValidationInfo,
+    field_validator,
+)
 
 from agbenchmark.agent_api_interface import download_agent_artifacts_into_folder
 from agbenchmark.agent_interface import copy_challenge_artifacts_into_workspace
@@ -46,7 +52,9 @@ class BuiltinChallengeSpec(BaseModel):
 
     class Info(BaseModel):
         difficulty: DifficultyLevel
-        description: Annotated[str, constr(regex=r"^Tests if the agent can.*")]
+        description: Annotated[
+            str, StringConstraints(pattern=r"^Tests if the agent can.*")
+        ]
         side_effects: list[str] = Field(default_factory=list)
 
     info: Info
@@ -60,23 +68,26 @@ class BuiltinChallengeSpec(BaseModel):
 
         class Eval(BaseModel):
             type: str
-            scoring: Optional[Literal["percentage", "scale", "binary"]]
-            template: Optional[Literal["rubric", "reference", "question", "custom"]]
-            examples: Optional[str]
+            scoring: Optional[Literal["percentage", "scale", "binary"]] = None
+            template: Optional[
+                Literal["rubric", "reference", "question", "custom"]
+            ] = None
+            examples: Optional[str] = None
 
-            @validator("scoring", "template", always=True)
-            def validate_eval_fields(cls, v, values, field):
-                if "type" in values and values["type"] == "llm":
-                    if v is None:
+            @field_validator("scoring", "template")
+            def validate_eval_fields(cls, value, info: ValidationInfo):
+                field_name = info.field_name
+                if "type" in info.data and info.data["type"] == "llm":
+                    if value is None:
                         raise ValueError(
-                            f"{field.name} must be provided when eval type is 'llm'"
+                            f"{field_name} must be provided when eval type is 'llm'"
                         )
                 else:
-                    if v is not None:
+                    if value is not None:
                         raise ValueError(
-                            f"{field.name} should only exist when eval type is 'llm'"
+                            f"{field_name} should only exist when eval type is 'llm'"
                         )
-                return v
+                return value
 
         eval: Eval
 
@@ -142,7 +153,7 @@ class BuiltinChallenge(BaseChallenge):
 
     @classmethod
     def from_challenge_spec_file(cls, spec_file: Path) -> type["BuiltinChallenge"]:
-        challenge_spec = BuiltinChallengeSpec.parse_file(spec_file)
+        challenge_spec = BuiltinChallengeSpec.model_validate_json(spec_file.read_text())
         challenge_spec.spec_file = spec_file
         return cls.from_challenge_spec(challenge_spec)
 
@@ -187,7 +198,7 @@ class BuiltinChallenge(BaseChallenge):
                     task_id = step.task_id
 
                 n_steps += 1
-                steps.append(step.copy())
+                steps.append(step.model_copy())
                 if step.additional_output:
                     agent_task_cost = step.additional_output.get(
                         "task_total_cost",

benchmark/agbenchmark/challenges/verticals/code/6_battleship/artifacts_in/abstract_class.py

@@ -1,7 +1,7 @@
 from abc import ABC, abstractmethod
 from typing import Optional
 
-from pydantic import BaseModel, validator
+from pydantic import BaseModel, field_validator
 
 
 # Models for the request and response payloads
@@ -10,7 +10,7 @@ class ShipPlacement(BaseModel):
     start: dict  # {"row": int, "column": str}
     direction: str
 
-    @validator("start")
+    @field_validator("start")
     def validate_start(cls, start):
         row, column = start.get("row"), start.get("column")
 

benchmark/agbenchmark/challenges/verticals/code/6_battleship/artifacts_out/abstract_class.py

@@ -1,7 +1,7 @@
 from abc import ABC, abstractmethod
 from typing import Optional
 
-from pydantic import BaseModel, validator
+from pydantic import BaseModel, field_validator
 
 
 # Models for the request and response payloads
@@ -10,7 +10,7 @@ class ShipPlacement(BaseModel):
     start: dict  # {"row": int, "column": str}
     direction: str
 
-    @validator("start")
+    @field_validator("start")
     def validate_start(cls, start):
         row, column = start.get("row"), start.get("column")
 

benchmark/agbenchmark/challenges/webarena.py

@@ -6,7 +6,7 @@ from typing import ClassVar, Iterator, Literal
 import pytest
 import requests
 from agent_protocol_client import AgentApi, Step
-from pydantic import BaseModel, ValidationError, validator
+from pydantic import BaseModel, ValidationError, ValidationInfo, field_validator
 
 from agbenchmark.config import AgentBenchmarkConfig
 from agbenchmark.utils.data_types import Category, EvalResult
@@ -183,7 +183,7 @@ class WebArenaChallengeSpec(BaseModel):
     """The JungleGym site (base URL) at which to start"""
     require_login: bool
     require_reset: bool
-    storage_state: str | None
+    storage_state: str | None = None
 
     intent: str
     intent_template: str
@@ -195,36 +195,36 @@ class WebArenaChallengeSpec(BaseModel):
 
     class EvalSet(BaseModel):
         class StringMatchEvalSet(BaseModel):
-            exact_match: str | None
-            fuzzy_match: list[str] | None
-            must_include: list[str] | None
+            exact_match: str | None = None
+            fuzzy_match: list[str] | None = None
+            must_include: list[str] | None = None
 
-        reference_answers: StringMatchEvalSet | None
+        reference_answers: StringMatchEvalSet | None = None
         """For string_match eval, a set of criteria to judge the final answer"""
-        reference_answer_raw_annotation: str | None
-        string_note: str | None
-        annotation_note: str | None
+        reference_answer_raw_annotation: str | None = None
+        string_note: str | None = None
+        annotation_note: str | None = None
 
-        reference_url: str | None
+        reference_url: str | None = None
         """For url_match eval, the last URL that should be visited"""
-        url_note: str | None
+        url_note: str | None = None
 
         program_html: list[ProgramHtmlEval]
         """For program_html eval, a list of criteria to judge the site state by"""
 
         eval_types: list[EvalType]
 
-        @validator("eval_types")
-        def check_eval_parameters(cls, v: list[EvalType], values):
-            if "string_match" in v and not values.get("reference_answers"):
+        @field_validator("eval_types")
+        def check_eval_parameters(cls, value: list[EvalType], info: ValidationInfo):
+            if "string_match" in value and not info.data["reference_answers"]:
                 raise ValueError("'string_match' eval_type requires reference_answers")
-            if "url_match" in v and not values.get("reference_url"):
+            if "url_match" in value and not info.data["reference_url"]:
                 raise ValueError("'url_match' eval_type requires reference_url")
-            if "program_html" in v and not values.get("program_html"):
+            if "program_html" in value and not info.data["program_html"]:
                 raise ValueError(
                     "'program_html' eval_type requires at least one program_html eval"
                 )
-            return v
+            return value
 
         @property
         def evaluators(self) -> list[_Eval]:
@@ -292,7 +292,7 @@ class WebArenaChallenge(BaseChallenge):
         results = requests.get(source_url).json()["data"]
         if not results:
             raise ValueError(f"Could not fetch challenge {source_uri}")
-        return cls.from_challenge_spec(WebArenaChallengeSpec.parse_obj(results[0]))
+        return cls.from_challenge_spec(WebArenaChallengeSpec.model_validate(results[0]))
 
     @classmethod
     def from_challenge_spec(
@@ -500,7 +500,7 @@ def load_webarena_challenges(
     skipped = 0
     for entry in challenge_dicts:
         try:
-            challenge_spec = WebArenaChallengeSpec.parse_obj(entry)
+            challenge_spec = WebArenaChallengeSpec.model_validate(entry)
         except ValidationError as e:
             failed += 1
             logger.warning(f"Error validating WebArena challenge entry: {entry}")

benchmark/agbenchmark/config.py

@@ -4,7 +4,8 @@ from datetime import datetime
 from pathlib import Path
 from typing import Optional
 
-from pydantic import BaseSettings, Field, validator
+from pydantic import Field, ValidationInfo, field_validator
+from pydantic_settings import BaseSettings
 
 
 def _calculate_info_test_path(base_path: Path, benchmark_start_time: datetime) -> Path:
@@ -57,7 +58,7 @@ class AgentBenchmarkConfig(BaseSettings, extra="allow"):
     subject application exposes an Agent Protocol compliant API.
     """
 
-    agbenchmark_config_dir: Path = Field(..., exclude=True)
+    agbenchmark_config_dir: Path = Field(exclude=True)
     """Path to the agbenchmark_config folder of the subject agent application."""
 
     categories: list[str] | None = None
@@ -101,11 +102,11 @@ class AgentBenchmarkConfig(BaseSettings, extra="allow"):
     def config_file(self) -> Path:
         return self.agbenchmark_config_dir / "config.json"
 
-    @validator("reports_folder", pre=True, always=True)
-    def set_reports_folder(cls, v, values):
-        if not v:
-            return values["agbenchmark_config_dir"] / "reports"
-        return v
+    @field_validator("reports_folder", mode="before")
+    def set_reports_folder(cls, value: Path, info: ValidationInfo):
+        if not value:
+            return info.data["agbenchmark_config_dir"] / "reports"
+        return value
 
     def get_report_dir(self, benchmark_start_time: datetime) -> Path:
         return _calculate_info_test_path(self.reports_folder, benchmark_start_time)

benchmark/agbenchmark/reports/ReportManager.py

@@ -111,17 +111,19 @@ class SessionReportManager(BaseReportManager):
     def save(self) -> None:
         with self.report_file.open("w") as f:
             if self.report:
-                f.write(self.report.json(indent=4))
+                f.write(self.report.model_dump_json(indent=4))
             else:
-                json.dump({k: v.dict() for k, v in self.tests.items()}, f, indent=4)
+                json.dump(
+                    {k: v.model_dump() for k, v in self.tests.items()}, f, indent=4
+                )
 
     def load(self) -> None:
         super().load()
 
         if "tests" in self.tests:
-            self.report = Report.parse_obj(self.tests)
+            self.report = Report.model_validate(self.tests)
         else:
-            self.tests = {n: Test.parse_obj(d) for n, d in self.tests.items()}
+            self.tests = {n: Test.model_validate(d) for n, d in self.tests.items()}
 
     def add_test_report(self, test_name: str, test_report: Test) -> None:
         if self.report:
@@ -155,7 +157,7 @@ class SessionReportManager(BaseReportManager):
                 total_cost=self.get_total_costs(),
             ),
             tests=copy.copy(self.tests),
-            config=config.dict(exclude={"reports_folder"}, exclude_none=True),
+            config=config.model_dump(exclude={"reports_folder"}, exclude_none=True),
         )
 
         agent_categories = get_highest_achieved_difficulty_per_category(self.report)

benchmark/agbenchmark/reports/processing/process_report.py

@@ -27,7 +27,7 @@ def get_reports_data(report_path: str) -> dict[str, Any]:
         with open(Path(subdir) / file, "r") as f:
             # Load the JSON data from the file
             json_data = json.load(f)
-            converted_data = Report.parse_obj(json_data)
+            converted_data = Report.model_validate(json_data)
             # get the last directory name in the path as key
             reports_data[subdir_name] = converted_data
 

benchmark/agbenchmark/reports/processing/report_types.py

@@ -6,7 +6,13 @@ import logging
 from typing import Annotated, Any, Dict, List
 
 from agent_protocol_client import Step
-from pydantic import BaseModel, Field, constr, validator
+from pydantic import (
+    BaseModel,
+    Field,
+    StringConstraints,
+    ValidationInfo,
+    field_validator,
+)
 
 datetime_format = r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\+00:00$"
 logger = logging.getLogger(__name__)
@@ -30,20 +36,20 @@ class TestResult(BaseModel):
     cost: float | None = None
     """The (known) cost incurred by the run, e.g. from using paid LLM APIs"""
 
-    @validator("fail_reason")
-    def success_xor_fail_reason(cls, v: str | None, values: dict[str, Any]):
-        if bool(v) == bool(values["success"]):
+    @field_validator("fail_reason")
+    def success_xor_fail_reason(cls, value, info: ValidationInfo):
+        if bool(value) == bool(info.data["success"]):
             logger.error(
                 "Error validating `success ^ fail_reason` on TestResult: "
-                f"success = {repr(values['success'])}; "
-                f"fail_reason = {repr(v)}"
+                f"success = {repr(info.data['success'])}; "
+                f"fail_reason = {repr(value)}"
             )
-        if v:
-            success = values["success"]
+        if value:
+            success = info.data["success"]
             assert not success, "fail_reason must only be specified if success=False"
         else:
-            assert values["success"], "fail_reason is required if success=False"
-        return v
+            assert info.data["success"], "fail_reason is required if success=False"
+        return value
 
 
 class TestMetrics(BaseModel):
@@ -88,7 +94,7 @@ class Test(BaseModel):
 class ReportBase(BaseModel):
     command: str
     completion_time: str | None = None
-    benchmark_start_time: Annotated[str, constr(regex=datetime_format)]
+    benchmark_start_time: Annotated[str, StringConstraints(pattern=datetime_format)]
     metrics: MetricsOverall
     config: Dict[str, str | dict[str, str]]
     agent_git_commit_sha: str | None = None

benchmark/agbenchmark/reports/processing/report_types_v2.py

@@ -1,7 +1,7 @@
 """Model definitions for use in the API"""
 from typing import Annotated
 
-from pydantic import BaseModel, constr
+from pydantic import BaseModel, StringConstraints
 
 datetime_format = r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\+00:00$"
 
@@ -37,7 +37,7 @@ class RunDetails(BaseModel):
     run_id: str | None = None
     command: str
     completion_time: str | None = None
-    benchmark_start_time: Annotated[str, constr(regex=datetime_format)]
+    benchmark_start_time: Annotated[str, StringConstraints(pattern=datetime_format)]
 
 
 class BenchmarkRun(BaseModel):

benchmark/agbenchmark/reports/reports.py

@@ -45,7 +45,7 @@ def update_regression_tests(
         # if the last 3 tests were successful, add to the regression tests
         test_report.metrics.is_regression = True
         SingletonReportManager().REGRESSION_MANAGER.add_test(
-            test_name, test_report.dict(include={"difficulty", "data_path"})
+            test_name, test_report.model_dump(include={"difficulty", "data_path"})
         )
 
 

benchmark/agbenchmark/schema.py

@@ -7,10 +7,9 @@ from pydantic import BaseModel, Field
 
 class TaskRequestBody(BaseModel):
     input: str = Field(
-        ...,
         min_length=1,
         description="Input prompt for the task.",
-        example="Write the words you receive to the file 'output.txt'.",
+        examples=["Write the words you receive to the file 'output.txt'."],
     )
     additional_input: Optional[dict[str, Any]] = Field(default_factory=dict)
 

benchmark/agbenchmark/utils/dependencies/main.py

@@ -220,7 +220,7 @@ class DependencyManager(object):
         labels = {}
         for item in self.items:
             assert item.cls and issubclass(item.cls, BaseChallenge)
-            data = item.cls.info.dict()
+            data = item.cls.info.model_dump()
 
             node_name = get_name(item)
             data["name"] = node_name

benchmark/agbenchmark/utils/utils.py

@@ -135,7 +135,7 @@ def pretty_print_model(model: BaseModel, include_header: bool = True) -> None:
     if include_header:
         # Try to find the ID and/or name attribute of the model
         id, name = None, None
-        for attr, value in model.dict().items():
+        for attr, value in model.model_dump().items():
             if attr == "id" or attr.endswith("_id"):
                 id = value
             if attr.endswith("name"):
@@ -148,8 +148,8 @@ def pretty_print_model(model: BaseModel, include_header: bool = True) -> None:
         )
         indent = " " * 2
 
-    k_col_width = max(len(k) for k in model.dict().keys())
-    for k, v in model.dict().items():
+    k_col_width = max(len(k) for k in model.model_dump().keys())
+    for k, v in model.model_dump().items():
         v_fmt = repr(v)
         if v is None or v == "":
             v_fmt = click.style(v_fmt, fg="black")

benchmark/notebooks/LLM Score Experimentation.ipynb

@@ -1,367 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "code",
-   "execution_count": 28,
-   "id": "15386da8",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "from dotenv import load_dotenv\n",
-    "from openai import OpenAI\n",
-    "\n",
-    "load_dotenv()\n",
-    "\n",
-    "def llm_eval(evaluation: str) -> float:\n",
-    "    openai_client = OpenAI()\n",
-    "    answer = openai_client.chat.completions.create(\n",
-    "        model=\"gpt-4\",\n",
-    "        messages=[\n",
-    "            {\"role\": \"system\", \"content\": evaluation},\n",
-    "        ],\n",
-    "    )\n",
-    "    return answer.choices[0].message.content"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 18,
-   "id": "e9d1c3a6",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "REFERENCE_PROMPT = \"\"\"Ignore previous directions. You are now an expert at evaluating how close machine generated responses are to human answers. You essentially act as a hyper advanced BLEU score.\n",
-    "In order to score the machine generated response you will {scoring}. Make sure to factor in the distance to the ideal response into your thinking, deliberation, and final result regarding scoring. Return nothing but a float score.\n",
-    "\n",
-    "Here is the given task for you to evaluate:\n",
-    "{task}\n",
-    "\n",
-    "Here is the ideal response you're comparing to based on the task:\n",
-    "{answer}\n",
-    "\n",
-    "Here are some examples of how to score a machine generated response compared to the above ideal response:\n",
-    "{examples}\n",
-    "\n",
-    "Here is the current machine generated response to the task that you need to evaluate:\n",
-    "{response}\n",
-    "\n",
-    "Remember to always end your response with nothing but a float score.\n",
-    "Float score:\n",
-    "\"\"\"\n",
-    "\n",
-    "RUBRIC_PROMPT = \"\"\"Ignore previous directions. You are now an expert at evaluating machine generated responses to given tasks.\n",
-    "In order to score the generated texts you will {scoring}. Make sure to factor in rubric into your thinking, deliberation, and final result regarding scoring. Return nothing but a float score.\n",
-    "\n",
-    "Here is the given task for you to evaluate:\n",
-    "{task}\n",
-    "\n",
-    "Use the below rubric to guide your thinking about scoring:\n",
-    "{answer}\n",
-    "\n",
-    "Here are some examples of how to score a machine generated response based on the above rubric.\n",
-    "{examples}\n",
-    "\n",
-    "Here is the current machine generated response to the task that you need to evaluate:\n",
-    "{response}\n",
-    "\n",
-    "Remember to always end your response with nothing but a float score.\n",
-    "Float score:\n",
-    "\"\"\"\n",
-    "\n",
-    "QUESTION_PROMPT = \"\"\"Ignore previous directions. You are now an expert at evaluating machine generated responses to given tasks.\n",
-    "In order to score the generated texts you will {scoring}. Make sure to think about whether the generated response answers the question well in order to score accurately. Return nothing but a float score.\n",
-    "\n",
-    "Here is the given task:\n",
-    "{task}\n",
-    "\n",
-    "Here is a question that checks if the task was completed correctly:\n",
-    "{answer}\n",
-    "\n",
-    "Here are some examples of how to score a machine generated response based on task-response pairs.\n",
-    "{examples}\n",
-    "\n",
-    "Here is the current machine generated response to the task that you need to evaluate:\n",
-    "{response}\n",
-    "\n",
-    "Remember to always end your response with nothing but a float score.\n",
-    "Float score:\n",
-    "\"\"\"\n",
-    "\n",
-    "CUSTOM_PROMPT = \"\"\"{custom}\n",
-    "{scoring}\"\"\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 19,
-   "id": "8ff9cd33",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "task = \"As an agent powered by a large language model, you can execute any digital action. Your task is to find the cheapest flight to San Francisco from Toronto. Create a step by step plan for yourself to execute this task and output this plan into a .txt file.\"\n",
-    "\n",
-    "percentage = \"assign a float score that will represent a percentage out of 100. Use decimal points to be even more accurate. 0 represents the worst possible generation, while 100 represents the ideal generation\"\n",
-    "scale = \"assign an integer score from a scale of 1-10. 1 represents a really bad generation, while 10 represents an ideal generation\"\n",
-    "binary = \"assign a binary score of either 0 or 1. 0 represents a failure, while 1 represents a success\"\n",
-    "\n",
-    "reference_answer = \"\"\"1. Search 'Toronto to San Francisco flights' on the internet\n",
-    "2. Click on the first link that is a flight aggregator such as SkyScanner or Google Flights\n",
-    "3. Select the dates that you want to travel\n",
-    "3. Click on the 'one way' option and click search\n",
-    "4. Search through all of the given options and select the cheapest flight\"\"\"\n",
-    "\n",
-    "rubric_answer = \"\"\"- Reasonable: The plan's steps should be arranged in a logical sequence. Each step should naturally follow from the one before it, leading coherently towards the desired outcome.\n",
-    "- Atomic: Each step of the plan should be broken down into the smallest possible units of action. This level of detail allows for better tracking and control, as well as easier troubleshooting and adjustment of the plan if necessary.\n",
-    "- Relevant: Each step should directly contribute to achieving the end goal of the plan. Irrelevant steps can waste resources and detract from the main objectives.\n",
-    "- Timely: The plan should include a realistic timeline for each step, and not include any redundant steps.\"\"\"\n",
-    "\n",
-    "question_answer = \"Will this plan find the cheapest flight to San Francisco from Toronto consistently?\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 20,
-   "id": "d0bfdce5",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "FEW_SHOT_REFERENCE_EXAMPLES = \"\"\"EXAMPLE 1\n",
-    "Plan: 'Define the Parameters:\n",
-    "\n",
-    "Identify the potential range of dates for the journey. Ensure flexibility to capitalize on the best deals.\n",
-    "Research Airlines:\n",
-    "\n",
-    "Compile a list of airlines that operate flights between Toronto and San Francisco. This can be accomplished by checking each airline's website or using an airline directory.\n",
-    "Use Comparison Tools:\n",
-    "\n",
-    "Use flight comparison websites (such as Skyscanner, Expedia, Google Flights, etc.) to compare flight prices across different airlines. These tools aggregate flight data and can present the cheapest options available.\n",
-    "Set Price Alerts:\n",
-    "\n",
-    "If the trip is not immediate, set price alerts on these comparison websites. They will notify you when there is a significant drop in flight prices on the selected route.\n",
-    "Check Airlines' Official Websites:\n",
-    "\n",
-    "After identifying potential flights via comparison tools, visit the airlines' official websites. Sometimes airlines offer deals or discounts on their own websites that are not reflected on comparison sites.\n",
-    "Consider Nearby Airports:\n",
-    "\n",
-    "Check flight prices to and from airports in the vicinity of both Toronto and San Francisco. Sometimes less busy airports can offer cheaper flights.\n",
-    "Evaluate Cost Effectiveness:\n",
-    "\n",
-    "Consider factors such as baggage fees, meal costs, and transportation to and from the airport when evaluating the total cost of the flight. The cheapest ticket price does not necessarily mean the least expensive journey overall.\n",
-    "Book the Flight:\n",
-    "\n",
-    "Once the cheapest and most convenient flight has been identified, proceed to booking. Double-check the flight details before finalizing the booking.\n",
-    "Monitor Flight Details:\n",
-    "\n",
-    "After booking, keep an eye on flight status, gate information, and any potential changes to the flight schedule.'\n",
-    "Returned score: 82.7\n",
-    "Internal rationale: The plan is solid, however the plan goes to extreme lengths to make things cheap, sacrificing time and simplicity. The task just asks for a vague definition of booking a flight. There are some redundant steps.\n",
-    "\n",
-    "EXAMPLE 2\n",
-    "Plan: 'Determine the Travel Dates and Flexibility: Decide on the dates you want to travel to San Francisco and check if you have any flexibility in your travel schedule. Being flexible with your travel dates can often lead to finding cheaper flights.\n",
-    "\n",
-    "Use Flight Search Engines: Start by using popular flight search engines like Google Flights, Skyscanner, Kayak, or Expedia. These platforms allow you to compare prices from various airlines and find the most affordable options.\n",
-    "\n",
-    "Set Up Fare Alerts: If your travel dates are flexible, consider setting up fare alerts on the flight search engines. These alerts will notify you when the prices drop for the specified route.\n",
-    "\n",
-    "Check Nearby Airports: In both Toronto and San Francisco, there might be multiple airports. Check flights departing from nearby airports as they may offer better deals.\n",
-    "\n",
-    "Consider Layovers: Non-stop flights are usually more convenient but can be more expensive. Look for flights with one or more layovers as they may offer cost savings.\n",
-    "\n",
-    "Check Airlines' Official Websites: Once you find a potentially cheap flight on a search engine, verify the price directly on the airline's official website. Sometimes, booking directly with the airline can be cheaper due to exclusive deals and promotions.\n",
-    "\n",
-    "Use Incognito/Private Browsing Mode: Flight prices can sometimes increase if the website detects repeated searches for the same route. To avoid this, use the incognito or private browsing mode in your web browser.\n",
-    "\n",
-    "Consider Budget Airlines: Check if there are any budget airlines flying between Toronto and San Francisco. They often offer lower fares, but be mindful of additional fees for baggage and other services.\n",
-    "\n",
-    "Check for Deals and Promo Codes: Look for any ongoing deals or promo codes that can help you save on your flight booking. Airlines and travel websites occasionally offer special discounts.\n",
-    "\n",
-    "Be Flexible with Departure and Arrival Times: If possible, consider flying during off-peak hours or mid-week, as flights during these times can be less expensive.\n",
-    "\n",
-    "Factor in Total Costs: While searching for cheap flights, don't forget to consider other expenses like baggage fees, seat selection, and additional amenities. Some budget airlines might have hidden costs that could make the overall trip more expensive.\n",
-    "\n",
-    "Book Early: Flight prices tend to rise as the departure date approaches. Once you find a good deal that suits your preferences, don't wait too long to book your flight.'\n",
-    "Returned score: 74.9\n",
-    "Internal rationale: The individual components of this plan are better than the one previous. But this plan doesn't follow logical steps to completion, and is just more general advice.\n",
-    "\n",
-    "EXAMPLE 3\n",
-    "Plan: 'Search online for cheap flights.\n",
-    "Check different dates.\n",
-    "Look at nearby airports.\n",
-    "Consider layovers.\n",
-    "Try budget airlines.\n",
-    "Book early if you find a good deal.'\n",
-    "Returned score: 42.0\n",
-    "Internal rationale: This plan is too vague and does not provide enough detail to be useful.\n",
-    "\n",
-    "\"\"\"\n",
-    "FEW_SHOT_RUBRIC_EXAMPLES = \"\"\n",
-    "FEW_SHOT_QUESTION_EXAMPLES = \"\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 75,
-   "id": "3de1d6d4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "response1=\"\"\"Define the Parameters:\n",
-    "\n",
-    "Identify the potential range of dates for the journey. Ensure flexibility to capitalize on the best deals.\n",
-    "Research Airlines:\n",
-    "\n",
-    "Compile a list of airlines that operate flights between Toronto and San Francisco. This can be accomplished by checking each airline's website or using an airline directory.\n",
-    "Use Comparison Tools:\n",
-    "\n",
-    "Use flight comparison websites (such as Skyscanner, Expedia, Google Flights, etc.) to compare flight prices across different airlines. These tools aggregate flight data and can present the cheapest options available.\n",
-    "Set Price Alerts:\n",
-    "\n",
-    "If the trip is not immediate, set price alerts on these comparison websites. They will notify you when there is a significant drop in flight prices on the selected route.\n",
-    "Check Airlines' Official Websites:\n",
-    "\n",
-    "After identifying potential flights via comparison tools, visit the airlines' official websites. Sometimes airlines offer deals or discounts on their own websites that are not reflected on comparison sites.\n",
-    "Consider Nearby Airports:\n",
-    "\n",
-    "Check flight prices to and from airports in the vicinity of both Toronto and San Francisco. Sometimes less busy airports can offer cheaper flights.\n",
-    "Evaluate Cost Effectiveness:\n",
-    "\n",
-    "Consider factors such as baggage fees, meal costs, and transportation to and from the airport when evaluating the total cost of the flight. The cheapest ticket price does not necessarily mean the least expensive journey overall.\n",
-    "Book the Flight:\n",
-    "\n",
-    "Once the cheapest and most convenient flight has been identified, proceed to booking. Double-check the flight details before finalizing the booking.\n",
-    "Monitor Flight Details:\n",
-    "\n",
-    "After booking, keep an eye on flight status, gate information, and any potential changes to the flight schedule.\"\"\"\n",
-    "\n",
-    "response2=\"\"\"Determine the Travel Dates and Flexibility: Decide on the dates you want to travel to San Francisco and check if you have any flexibility in your travel schedule. Being flexible with your travel dates can often lead to finding cheaper flights.\n",
-    "\n",
-    "Use Flight Search Engines: Start by using popular flight search engines like Google Flights, Skyscanner, Kayak, or Expedia. These platforms allow you to compare prices from various airlines and find the most affordable options.\n",
-    "\n",
-    "Set Up Fare Alerts: If your travel dates are flexible, consider setting up fare alerts on the flight search engines. These alerts will notify you when the prices drop for the specified route.\n",
-    "\n",
-    "Check Nearby Airports: In both Toronto and San Francisco, there might be multiple airports. Check flights departing from nearby airports as they may offer better deals.\n",
-    "\n",
-    "Consider Layovers: Non-stop flights are usually more convenient but can be more expensive. Look for flights with one or more layovers as they may offer cost savings.\n",
-    "\n",
-    "Check Airlines' Official Websites: Once you find a potentially cheap flight on a search engine, verify the price directly on the airline's official website. Sometimes, booking directly with the airline can be cheaper due to exclusive deals and promotions.\n",
-    "\n",
-    "Use Incognito/Private Browsing Mode: Flight prices can sometimes increase if the website detects repeated searches for the same route. To avoid this, use the incognito or private browsing mode in your web browser.\n",
-    "\n",
-    "Consider Budget Airlines: Check if there are any budget airlines flying between Toronto and San Francisco. They often offer lower fares, but be mindful of additional fees for baggage and other services.\n",
-    "\n",
-    "Check for Deals and Promo Codes: Look for any ongoing deals or promo codes that can help you save on your flight booking. Airlines and travel websites occasionally offer special discounts.\n",
-    "\n",
-    "Be Flexible with Departure and Arrival Times: If possible, consider flying during off-peak hours or mid-week, as flights during these times can be less expensive.\n",
-    "\n",
-    "Factor in Total Costs: While searching for cheap flights, don't forget to consider other expenses like baggage fees, seat selection, and additional amenities. Some budget airlines might have hidden costs that could make the overall trip more expensive.\n",
-    "\n",
-    "Book Early: Flight prices tend to rise as the departure date approaches. Once you find a good deal that suits your preferences, don't wait too long to book your flight.\"\"\"\n",
-    "\n",
-    "response3 = \"\"\"Search online for cheap flights.\n",
-    "Check different dates.\n",
-    "Look at nearby airports.\n",
-    "Consider layovers.\n",
-    "Try budget airlines.\n",
-    "Book early if you find a good deal.\"\"\"\n",
-    "\n",
-    "ideal_response = \"\"\"1. Search 'Toronto to San Francisco flights' on the internet\n",
-    "2. Click on the first link that is a flight aggregator such as SkyScanner or Google Flights\n",
-    "3. Select the dates that you want to travel\n",
-    "3. Click on the 'one way' option and click search\n",
-    "4. Search through all of the given options and select the cheapest flight\"\"\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "a5bf2f5c",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "reference_evaluation = REFERENCE_PROMPT.format(task=task, scoring=percentage, answer=reference_answer, response=ideal_response, examples=FEW_SHOT_REFERENCE_EXAMPLES)\n",
-    "rubric_evaluation = RUBRIC_PROMPT.format(task=task, scoring=percentage, answer=rubric_answer, response=ideal_response, examples=FEW_SHOT_REFERENCE_EXAMPLES)\n",
-    "question_evaluation = QUESTION_PROMPT.format(task=task, scoring=percentage, answer=question_answer, response=ideal_response, examples=FEW_SHOT_REFERENCE_EXAMPLES)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 82,
-   "id": "1cd01a4c",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "reference_response = llm_eval(reference_evaluation)\n",
-    "rubric_response = llm_eval(rubric_evaluation)\n",
-    "question_response = llm_eval(question_evaluation)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 72,
-   "id": "72e6ac91",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "REFERENCE: 66.5 6.0 0.0\n",
-      "RUBRIC: 65.0 4.0 0.0\n",
-      "QUESTION: 87.0 6.5 0.0\n"
-     ]
-    }
-   ],
-   "source": [
-    "# response3 - the worst one\n",
-    "print('REFERENCE:', reference_response_percentage, reference_response_scale, reference_response_binary)\n",
-    "print('RUBRIC:', rubric_response_percentage, rubric_response_scale, rubric_response_binary)\n",
-    "print('QUESTION:', question_response_percentage, question_response_scale, question_response_binary)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 83,
-   "id": "a66014ee",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "REFERENCE: 100.0 10.0 1.0\n",
-      "RUBRIC: 88.0 7.5 1.0\n",
-      "QUESTION: 87.0 7.5 1.0\n"
-     ]
-    }
-   ],
-   "source": [
-    "# reference_response - the best one\n",
-    "print('REFERENCE:', reference_response_ideal_percentage, reference_response_ideal_scale, reference_response_ideal_binary)\n",
-    "print('RUBRIC:', rubric_response_ideal_percentage, rubric_response_ideal_scale, rubric_response_ideal_binary)\n",
-    "print('QUESTION:', question_response_ideal_percentage, question_response_ideal_scale, question_response_ideal_binary)"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.4"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

benchmark/notebooks/selected_logs.json

@@ -1,30 +0,0 @@
-{
-    "1": {
-        "role": "assistant",
-        "content": "{\n  \"thoughts\": {\n    \"text\": \"I need to find Tesla's revenue in 2022.\",\n    \"reasoning\": \"I will perform a web search to find the information.\",\n    \"plan\": \"- Perform a web search\\n- Extract the revenue\\n- Write the revenue to a file\",\n    \"criticism\": \"I should be cautious about the reliability of the source I find.\",\n    \"speak\": \"I will search the web for Tesla's revenue in 2022.\"\n  },\n  \"command\": {\n    \"name\": \"web_search\",\n    \"args\": {\n      \"query\": \"Tesla revenue in 2022\"\n    }\n  }\n}"
-    },
-    "2": {
-        "role": "assistant",
-        "content": "{\n  \"thoughts\": {\n    \"text\": \"I found a reliable source for Tesla's revenue in 2022.\",\n    \"reasoning\": \"I will use the Statista source to extract the revenue information.\",\n    \"plan\": \"- Browse the Statista website\\n- Extract the revenue\\n- Write the revenue to a file\",\n    \"criticism\": \"I should verify the information on the Statista website.\",\n    \"speak\": \"I will browse the Statista website to find Tesla's revenue in 2022.\"\n  },\n  \"command\": {\n    \"name\": \"browse_website\",\n    \"args\": {\n      \"url\": \"https://www.statista.com/statistics/272120/revenue-of-tesla/\",\n      \"question\": \"What was Tesla's revenue in 2022?\"\n    }\n  }\n}"
-    },
-    "3": {
-        "role": "assistant",
-        "content": "Tesla's revenue in 2022 was nearly $81.5 billion, which represents a 51 percent increase from the previous year. The majority of the revenue, over $71 billion, was generated from Tesla's automotive segment. The Model Y emerged as Tesla's best-selling vehicle, and the company has been investing in the production of the Model 3 and Model Y."
-    },
-    "4": {
-        "role": "assistant",
-        "content": "The text provides various statistics and financial information about Tesla, including its revenue from 2008 to 2022. However, the specific revenue for 2022 is not mentioned in the text."
-    },
-    "5": {
-        "role": "assistant",
-        "content": "a description of different types of cookies used on a website and the options for accepting or rejecting them. There is no information provided about Tesla's revenue in 2022."
-    },
-    "6": {
-        "role": "assistant",
-        "content": "Tesla's revenue in 2022 increased by 51% to nearly $81.5 billion, with the majority coming from the automotive segment. The Model Y was the best-selling vehicle, and the company has been focusing on producing the Model 3 and Model Y."
-    },
-    "7": {
-        "role": "assistant",
-        "content": "{\n  \"thoughts\": {\n    \"text\": \"I found Tesla's revenue in 2022.\",\n    \"reasoning\": \"I will write the revenue to a file.\",\n    \"plan\": \"- Write the revenue to a file\",\n    \"criticism\": \"I should ensure the revenue is formatted correctly.\",\n    \"speak\": \"I will write Tesla's revenue in 2022 to a file.\"\n  },\n  \"command\": {\n    \"name\": \"write_to_file\",\n    \"args\": {\n      \"filename\": \"tesla_revenue_2022.txt\",\n      \"text\": \"Tesla's revenue in 2022 was $81,500,000,000.\"\n    }\n  }\n}"
-    }
-}

benchmark/notebooks/selected_logs_nested.json

@@ -1,74 +0,0 @@
-{
-    "1": {
-        "role": "assistant",
-        "content": {
-            "thoughts": {
-                "text": "I need to find Tesla's revenue in 2022.",
-                "reasoning": "I will perform a web search to find the information.",
-                "plan": "- Perform a web search\n- Extract the revenue\n- Write the revenue to a file",
-                "criticism": "I should be cautious about the reliability of the source I find.",
-                "speak": "I will search the web for Tesla's revenue in 2022."
-            },
-            "command": {
-                "name": "web_search",
-                "args": {
-                    "query": "Tesla revenue in 2022"
-                }
-            }
-        }
-    },
-    "2": {
-        "role": "assistant",
-        "content": {
-            "thoughts": {
-                "text": "I found a reliable source for Tesla's revenue in 2022.",
-                "reasoning": "I will use the Statista source to extract the revenue information.",
-                "plan": "- Browse the Statista website\n- Extract the revenue\n- Write the revenue to a file",
-                "criticism": "I should verify the information on the Statista website.",
-                "speak": "I will browse the Statista website to find Tesla's revenue in 2022."
-            },
-            "command": {
-                "name": "browse_website",
-                "args": {
-                    "url": "https://www.statista.com/statistics/272120/revenue-of-tesla/",
-                    "question": "What was Tesla's revenue in 2022?"
-                }
-            }
-        }
-    },
-    "3": {
-        "role": "assistant",
-        "content": "Tesla's revenue in 2022 was nearly $81.5 billion, which represents a 51 percent increase from the previous year. The majority of the revenue, over $71 billion, was generated from Tesla's automotive segment. The Model Y emerged as Tesla's best-selling vehicle, and the company has been investing in the production of the Model 3 and Model Y."
-    },
-    "4": {
-        "role": "assistant",
-        "content": "The text provides various statistics and financial information about Tesla, including its revenue from 2008 to 2022. However, the specific revenue for 2022 is not mentioned in the text."
-    },
-    "5": {
-        "role": "assistant",
-        "content": "a description of different types of cookies used on a website and the options for accepting or rejecting them. There is no information provided about Tesla's revenue in 2022."
-    },
-    "6": {
-        "role": "assistant",
-        "content": "Tesla's revenue in 2022 increased by 51% to nearly $81.5 billion, with the majority coming from the automotive segment. The Model Y was the best-selling vehicle, and the company has been focusing on producing the Model 3 and Model Y."
-    },
-    "7": {
-        "role": "assistant",
-        "content": {
-            "thoughts": {
-                "text": "I found Tesla's revenue in 2022.",
-                "reasoning": "I will write the revenue to a file.",
-                "plan": "- Write the revenue to a file",
-                "criticism": "I should ensure the revenue is formatted correctly.",
-                "speak": "I will write Tesla's revenue in 2022 to a file."
-            },
-            "command": {
-                "name": "write_to_file",
-                "args": {
-                    "filename": "tesla_revenue_2022.txt",
-                    "text": "Tesla's revenue in 2022 was $81,500,000,000."
-                }
-            }
-        }
-    }
-}

benchmark/poetry.lock

@@ -5,17 +5,22 @@ name = "agent-protocol-client"
 version = "1.1.0"
 description = "Agent Communication Protocol Client"
 optional = false
-python-versions = ">=3.7,<4.0"
-files = [
-    {file = "agent_protocol_client-1.1.0-py3-none-any.whl", hash = "sha256:0e8c6c97244189666ed18e320410abddce8c9dfb75437da1e590bbef3b6268be"},
-    {file = "agent_protocol_client-1.1.0.tar.gz", hash = "sha256:aa7e1042de1249477fdc29c2df08a44f2233dade9c02c1279e37c98e9d3a0d72"},
-]
+python-versions = "^3.7"
+files = []
+develop = false
 
 [package.dependencies]
-aiohttp = ">=3.8.4,<4.0.0"
-pydantic = ">=1.10.5,<2.0.0"
-python-dateutil = ">=2.8.2,<3.0.0"
-urllib3 = ">=1.25.3,<2.0.0"
+aiohttp = "^3.8.4"
+pydantic = ">=1.10.5, <3.0.0"
+python-dateutil = "^2.8.2"
+urllib3 = "^1.25.3"
+
+[package.source]
+type = "git"
+url = "https://github.com/Significant-Gravitas/agent-protocol.git"
+reference = "HEAD"
+resolved_reference = "beb098517b0b9e255024d1b57df236f0329f4b1c"
+subdirectory = "packages/client/python"
 
 [[package]]
 name = "aiohttp"
@@ -127,6 +132,17 @@ files = [
 [package.dependencies]
 frozenlist = ">=1.1.0"
 
+[[package]]
+name = "annotated-types"
+version = "0.7.0"
+description = "Reusable constraint types to use with typing.Annotated"
+optional = false
+python-versions = ">=3.8"
+files = [
+    {file = "annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53"},
+    {file = "annotated_types-0.7.0.tar.gz", hash = "sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89"},
+]
+
 [[package]]
 name = "anyio"
 version = "4.2.0"
@@ -1440,85 +1456,101 @@ files = [
 
 [[package]]
 name = "multidict"
-version = "6.0.4"
+version = "6.0.5"
 description = "multidict implementation"
 optional = false
 python-versions = ">=3.7"
 files = [
-    {file = "multidict-6.0.4-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:0b1a97283e0c85772d613878028fec909f003993e1007eafa715b24b377cb9b8"},
-    {file = "multidict-6.0.4-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:eeb6dcc05e911516ae3d1f207d4b0520d07f54484c49dfc294d6e7d63b734171"},
-    {file = "multidict-6.0.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:d6d635d5209b82a3492508cf5b365f3446afb65ae7ebd755e70e18f287b0adf7"},
-    {file = "multidict-6.0.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c048099e4c9e9d615545e2001d3d8a4380bd403e1a0578734e0d31703d1b0c0b"},
-    {file = "multidict-6.0.4-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ea20853c6dbbb53ed34cb4d080382169b6f4554d394015f1bef35e881bf83547"},
-    {file = "multidict-6.0.4-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:16d232d4e5396c2efbbf4f6d4df89bfa905eb0d4dc5b3549d872ab898451f569"},
-    {file = "multidict-6.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:36c63aaa167f6c6b04ef2c85704e93af16c11d20de1d133e39de6a0e84582a93"},
-    {file = "multidict-6.0.4-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:64bdf1086b6043bf519869678f5f2757f473dee970d7abf6da91ec00acb9cb98"},
-    {file = "multidict-6.0.4-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:43644e38f42e3af682690876cff722d301ac585c5b9e1eacc013b7a3f7b696a0"},
-    {file = "multidict-6.0.4-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:7582a1d1030e15422262de9f58711774e02fa80df0d1578995c76214f6954988"},
-    {file = "multidict-6.0.4-cp310-cp310-musllinux_1_1_ppc64le.whl", hash = "sha256:ddff9c4e225a63a5afab9dd15590432c22e8057e1a9a13d28ed128ecf047bbdc"},
-    {file = "multidict-6.0.4-cp310-cp310-musllinux_1_1_s390x.whl", hash = "sha256:ee2a1ece51b9b9e7752e742cfb661d2a29e7bcdba2d27e66e28a99f1890e4fa0"},
-    {file = "multidict-6.0.4-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:a2e4369eb3d47d2034032a26c7a80fcb21a2cb22e1173d761a162f11e562caa5"},
-    {file = "multidict-6.0.4-cp310-cp310-win32.whl", hash = "sha256:574b7eae1ab267e5f8285f0fe881f17efe4b98c39a40858247720935b893bba8"},
-    {file = "multidict-6.0.4-cp310-cp310-win_amd64.whl", hash = "sha256:4dcbb0906e38440fa3e325df2359ac6cb043df8e58c965bb45f4e406ecb162cc"},
-    {file = "multidict-6.0.4-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:0dfad7a5a1e39c53ed00d2dd0c2e36aed4650936dc18fd9a1826a5ae1cad6f03"},
-    {file = "multidict-6.0.4-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:64da238a09d6039e3bd39bb3aee9c21a5e34f28bfa5aa22518581f910ff94af3"},
-    {file = "multidict-6.0.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:ff959bee35038c4624250473988b24f846cbeb2c6639de3602c073f10410ceba"},
-    {file = "multidict-6.0.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:01a3a55bd90018c9c080fbb0b9f4891db37d148a0a18722b42f94694f8b6d4c9"},
-    {file = "multidict-6.0.4-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c5cb09abb18c1ea940fb99360ea0396f34d46566f157122c92dfa069d3e0e982"},
-    {file = "multidict-6.0.4-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:666daae833559deb2d609afa4490b85830ab0dfca811a98b70a205621a6109fe"},
-    {file = "multidict-6.0.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:11bdf3f5e1518b24530b8241529d2050014c884cf18b6fc69c0c2b30ca248710"},
-    {file = "multidict-6.0.4-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:7d18748f2d30f94f498e852c67d61261c643b349b9d2a581131725595c45ec6c"},
-    {file = "multidict-6.0.4-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:458f37be2d9e4c95e2d8866a851663cbc76e865b78395090786f6cd9b3bbf4f4"},
-    {file = "multidict-6.0.4-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:b1a2eeedcead3a41694130495593a559a668f382eee0727352b9a41e1c45759a"},
-    {file = "multidict-6.0.4-cp311-cp311-musllinux_1_1_ppc64le.whl", hash = "sha256:7d6ae9d593ef8641544d6263c7fa6408cc90370c8cb2bbb65f8d43e5b0351d9c"},
-    {file = "multidict-6.0.4-cp311-cp311-musllinux_1_1_s390x.whl", hash = "sha256:5979b5632c3e3534e42ca6ff856bb24b2e3071b37861c2c727ce220d80eee9ed"},
-    {file = "multidict-6.0.4-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:dcfe792765fab89c365123c81046ad4103fcabbc4f56d1c1997e6715e8015461"},
-    {file = "multidict-6.0.4-cp311-cp311-win32.whl", hash = "sha256:3601a3cece3819534b11d4efc1eb76047488fddd0c85a3948099d5da4d504636"},
-    {file = "multidict-6.0.4-cp311-cp311-win_amd64.whl", hash = "sha256:81a4f0b34bd92df3da93315c6a59034df95866014ac08535fc819f043bfd51f0"},
-    {file = "multidict-6.0.4-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:67040058f37a2a51ed8ea8f6b0e6ee5bd78ca67f169ce6122f3e2ec80dfe9b78"},
-    {file = "multidict-6.0.4-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:853888594621e6604c978ce2a0444a1e6e70c8d253ab65ba11657659dcc9100f"},
-    {file = "multidict-6.0.4-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:39ff62e7d0f26c248b15e364517a72932a611a9b75f35b45be078d81bdb86603"},
-    {file = "multidict-6.0.4-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:af048912e045a2dc732847d33821a9d84ba553f5c5f028adbd364dd4765092ac"},
-    {file = "multidict-6.0.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b1e8b901e607795ec06c9e42530788c45ac21ef3aaa11dbd0c69de543bfb79a9"},
-    {file = "multidict-6.0.4-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:62501642008a8b9871ddfccbf83e4222cf8ac0d5aeedf73da36153ef2ec222d2"},
-    {file = "multidict-6.0.4-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:99b76c052e9f1bc0721f7541e5e8c05db3941eb9ebe7b8553c625ef88d6eefde"},
-    {file = "multidict-6.0.4-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:509eac6cf09c794aa27bcacfd4d62c885cce62bef7b2c3e8b2e49d365b5003fe"},
-    {file = "multidict-6.0.4-cp37-cp37m-musllinux_1_1_ppc64le.whl", hash = "sha256:21a12c4eb6ddc9952c415f24eef97e3e55ba3af61f67c7bc388dcdec1404a067"},
-    {file = "multidict-6.0.4-cp37-cp37m-musllinux_1_1_s390x.whl", hash = "sha256:5cad9430ab3e2e4fa4a2ef4450f548768400a2ac635841bc2a56a2052cdbeb87"},
-    {file = "multidict-6.0.4-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:ab55edc2e84460694295f401215f4a58597f8f7c9466faec545093045476327d"},
-    {file = "multidict-6.0.4-cp37-cp37m-win32.whl", hash = "sha256:5a4dcf02b908c3b8b17a45fb0f15b695bf117a67b76b7ad18b73cf8e92608775"},
-    {file = "multidict-6.0.4-cp37-cp37m-win_amd64.whl", hash = "sha256:6ed5f161328b7df384d71b07317f4d8656434e34591f20552c7bcef27b0ab88e"},
-    {file = "multidict-6.0.4-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:5fc1b16f586f049820c5c5b17bb4ee7583092fa0d1c4e28b5239181ff9532e0c"},
-    {file = "multidict-6.0.4-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:1502e24330eb681bdaa3eb70d6358e818e8e8f908a22a1851dfd4e15bc2f8161"},
-    {file = "multidict-6.0.4-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:b692f419760c0e65d060959df05f2a531945af31fda0c8a3b3195d4efd06de11"},
-    {file = "multidict-6.0.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:45e1ecb0379bfaab5eef059f50115b54571acfbe422a14f668fc8c27ba410e7e"},
-    {file = "multidict-6.0.4-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ddd3915998d93fbcd2566ddf9cf62cdb35c9e093075f862935573d265cf8f65d"},
-    {file = "multidict-6.0.4-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:59d43b61c59d82f2effb39a93c48b845efe23a3852d201ed2d24ba830d0b4cf2"},
-    {file = "multidict-6.0.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cc8e1d0c705233c5dd0c5e6460fbad7827d5d36f310a0fadfd45cc3029762258"},
-    {file = "multidict-6.0.4-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d6aa0418fcc838522256761b3415822626f866758ee0bc6632c9486b179d0b52"},
-    {file = "multidict-6.0.4-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:6748717bb10339c4760c1e63da040f5f29f5ed6e59d76daee30305894069a660"},
-    {file = "multidict-6.0.4-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:4d1a3d7ef5e96b1c9e92f973e43aa5e5b96c659c9bc3124acbbd81b0b9c8a951"},
-    {file = "multidict-6.0.4-cp38-cp38-musllinux_1_1_ppc64le.whl", hash = "sha256:4372381634485bec7e46718edc71528024fcdc6f835baefe517b34a33c731d60"},
-    {file = "multidict-6.0.4-cp38-cp38-musllinux_1_1_s390x.whl", hash = "sha256:fc35cb4676846ef752816d5be2193a1e8367b4c1397b74a565a9d0389c433a1d"},
-    {file = "multidict-6.0.4-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:4b9d9e4e2b37daddb5c23ea33a3417901fa7c7b3dee2d855f63ee67a0b21e5b1"},
-    {file = "multidict-6.0.4-cp38-cp38-win32.whl", hash = "sha256:e41b7e2b59679edfa309e8db64fdf22399eec4b0b24694e1b2104fb789207779"},
-    {file = "multidict-6.0.4-cp38-cp38-win_amd64.whl", hash = "sha256:d6c254ba6e45d8e72739281ebc46ea5eb5f101234f3ce171f0e9f5cc86991480"},
-    {file = "multidict-6.0.4-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:16ab77bbeb596e14212e7bab8429f24c1579234a3a462105cda4a66904998664"},
-    {file = "multidict-6.0.4-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:bc779e9e6f7fda81b3f9aa58e3a6091d49ad528b11ed19f6621408806204ad35"},
-    {file = "multidict-6.0.4-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:4ceef517eca3e03c1cceb22030a3e39cb399ac86bff4e426d4fc6ae49052cc60"},
-    {file = "multidict-6.0.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:281af09f488903fde97923c7744bb001a9b23b039a909460d0f14edc7bf59706"},
-    {file = "multidict-6.0.4-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:52f2dffc8acaba9a2f27174c41c9e57f60b907bb9f096b36b1a1f3be71c6284d"},
-    {file = "multidict-6.0.4-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b41156839806aecb3641f3208c0dafd3ac7775b9c4c422d82ee2a45c34ba81ca"},
-    {file = "multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d5e3fc56f88cc98ef8139255cf8cd63eb2c586531e43310ff859d6bb3a6b51f1"},
-    {file = "multidict-6.0.4-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8316a77808c501004802f9beebde51c9f857054a0c871bd6da8280e718444449"},
-    {file = "multidict-6.0.4-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:f70b98cd94886b49d91170ef23ec5c0e8ebb6f242d734ed7ed677b24d50c82cf"},
-    {file = "multidict-6.0.4-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:bf6774e60d67a9efe02b3616fee22441d86fab4c6d335f9d2051d19d90a40063"},
-    {file = "multidict-6.0.4-cp39-cp39-musllinux_1_1_ppc64le.whl", hash = "sha256:e69924bfcdda39b722ef4d9aa762b2dd38e4632b3641b1d9a57ca9cd18f2f83a"},
-    {file = "multidict-6.0.4-cp39-cp39-musllinux_1_1_s390x.whl", hash = "sha256:6b181d8c23da913d4ff585afd1155a0e1194c0b50c54fcfe286f70cdaf2b7176"},
-    {file = "multidict-6.0.4-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:52509b5be062d9eafc8170e53026fbc54cf3b32759a23d07fd935fb04fc22d95"},
-    {file = "multidict-6.0.4-cp39-cp39-win32.whl", hash = "sha256:27c523fbfbdfd19c6867af7346332b62b586eed663887392cff78d614f9ec313"},
-    {file = "multidict-6.0.4-cp39-cp39-win_amd64.whl", hash = "sha256:33029f5734336aa0d4c0384525da0387ef89148dc7191aae00ca5fb23d7aafc2"},
-    {file = "multidict-6.0.4.tar.gz", hash = "sha256:3666906492efb76453c0e7b97f2cf459b0682e7402c0489a95484965dbc1da49"},
+    {file = "multidict-6.0.5-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:228b644ae063c10e7f324ab1ab6b548bdf6f8b47f3ec234fef1093bc2735e5f9"},
+    {file = "multidict-6.0.5-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:896ebdcf62683551312c30e20614305f53125750803b614e9e6ce74a96232604"},
+    {file = "multidict-6.0.5-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:411bf8515f3be9813d06004cac41ccf7d1cd46dfe233705933dd163b60e37600"},
+    {file = "multidict-6.0.5-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1d147090048129ce3c453f0292e7697d333db95e52616b3793922945804a433c"},
+    {file = "multidict-6.0.5-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:215ed703caf15f578dca76ee6f6b21b7603791ae090fbf1ef9d865571039ade5"},
+    {file = "multidict-6.0.5-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7c6390cf87ff6234643428991b7359b5f59cc15155695deb4eda5c777d2b880f"},
+    {file = "multidict-6.0.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:21fd81c4ebdb4f214161be351eb5bcf385426bf023041da2fd9e60681f3cebae"},
+    {file = "multidict-6.0.5-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:3cc2ad10255f903656017363cd59436f2111443a76f996584d1077e43ee51182"},
+    {file = "multidict-6.0.5-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:6939c95381e003f54cd4c5516740faba40cf5ad3eeff460c3ad1d3e0ea2549bf"},
+    {file = "multidict-6.0.5-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:220dd781e3f7af2c2c1053da9fa96d9cf3072ca58f057f4c5adaaa1cab8fc442"},
+    {file = "multidict-6.0.5-cp310-cp310-musllinux_1_1_ppc64le.whl", hash = "sha256:766c8f7511df26d9f11cd3a8be623e59cca73d44643abab3f8c8c07620524e4a"},
+    {file = "multidict-6.0.5-cp310-cp310-musllinux_1_1_s390x.whl", hash = "sha256:fe5d7785250541f7f5019ab9cba2c71169dc7d74d0f45253f8313f436458a4ef"},
+    {file = "multidict-6.0.5-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:c1c1496e73051918fcd4f58ff2e0f2f3066d1c76a0c6aeffd9b45d53243702cc"},
+    {file = "multidict-6.0.5-cp310-cp310-win32.whl", hash = "sha256:7afcdd1fc07befad18ec4523a782cde4e93e0a2bf71239894b8d61ee578c1319"},
+    {file = "multidict-6.0.5-cp310-cp310-win_amd64.whl", hash = "sha256:99f60d34c048c5c2fabc766108c103612344c46e35d4ed9ae0673d33c8fb26e8"},
+    {file = "multidict-6.0.5-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:f285e862d2f153a70586579c15c44656f888806ed0e5b56b64489afe4a2dbfba"},
+    {file = "multidict-6.0.5-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:53689bb4e102200a4fafa9de9c7c3c212ab40a7ab2c8e474491914d2305f187e"},
+    {file = "multidict-6.0.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:612d1156111ae11d14afaf3a0669ebf6c170dbb735e510a7438ffe2369a847fd"},
+    {file = "multidict-6.0.5-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7be7047bd08accdb7487737631d25735c9a04327911de89ff1b26b81745bd4e3"},
+    {file = "multidict-6.0.5-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:de170c7b4fe6859beb8926e84f7d7d6c693dfe8e27372ce3b76f01c46e489fcf"},
+    {file = "multidict-6.0.5-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:04bde7a7b3de05732a4eb39c94574db1ec99abb56162d6c520ad26f83267de29"},
+    {file = "multidict-6.0.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:85f67aed7bb647f93e7520633d8f51d3cbc6ab96957c71272b286b2f30dc70ed"},
+    {file = "multidict-6.0.5-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:425bf820055005bfc8aa9a0b99ccb52cc2f4070153e34b701acc98d201693733"},
+    {file = "multidict-6.0.5-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:d3eb1ceec286eba8220c26f3b0096cf189aea7057b6e7b7a2e60ed36b373b77f"},
+    {file = "multidict-6.0.5-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:7901c05ead4b3fb75113fb1dd33eb1253c6d3ee37ce93305acd9d38e0b5f21a4"},
+    {file = "multidict-6.0.5-cp311-cp311-musllinux_1_1_ppc64le.whl", hash = "sha256:e0e79d91e71b9867c73323a3444724d496c037e578a0e1755ae159ba14f4f3d1"},
+    {file = "multidict-6.0.5-cp311-cp311-musllinux_1_1_s390x.whl", hash = "sha256:29bfeb0dff5cb5fdab2023a7a9947b3b4af63e9c47cae2a10ad58394b517fddc"},
+    {file = "multidict-6.0.5-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:e030047e85cbcedbfc073f71836d62dd5dadfbe7531cae27789ff66bc551bd5e"},
+    {file = "multidict-6.0.5-cp311-cp311-win32.whl", hash = "sha256:2f4848aa3baa109e6ab81fe2006c77ed4d3cd1e0ac2c1fbddb7b1277c168788c"},
+    {file = "multidict-6.0.5-cp311-cp311-win_amd64.whl", hash = "sha256:2faa5ae9376faba05f630d7e5e6be05be22913782b927b19d12b8145968a85ea"},
+    {file = "multidict-6.0.5-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:51d035609b86722963404f711db441cf7134f1889107fb171a970c9701f92e1e"},
+    {file = "multidict-6.0.5-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:cbebcd5bcaf1eaf302617c114aa67569dd3f090dd0ce8ba9e35e9985b41ac35b"},
+    {file = "multidict-6.0.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:2ffc42c922dbfddb4a4c3b438eb056828719f07608af27d163191cb3e3aa6cc5"},
+    {file = "multidict-6.0.5-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ceb3b7e6a0135e092de86110c5a74e46bda4bd4fbfeeb3a3bcec79c0f861e450"},
+    {file = "multidict-6.0.5-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:79660376075cfd4b2c80f295528aa6beb2058fd289f4c9252f986751a4cd0496"},
+    {file = "multidict-6.0.5-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e4428b29611e989719874670fd152b6625500ad6c686d464e99f5aaeeaca175a"},
+    {file = "multidict-6.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d84a5c3a5f7ce6db1f999fb9438f686bc2e09d38143f2d93d8406ed2dd6b9226"},
+    {file = "multidict-6.0.5-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:76c0de87358b192de7ea9649beb392f107dcad9ad27276324c24c91774ca5271"},
+    {file = "multidict-6.0.5-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:79a6d2ba910adb2cbafc95dad936f8b9386e77c84c35bc0add315b856d7c3abb"},
+    {file = "multidict-6.0.5-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:92d16a3e275e38293623ebf639c471d3e03bb20b8ebb845237e0d3664914caef"},
+    {file = "multidict-6.0.5-cp312-cp312-musllinux_1_1_ppc64le.whl", hash = "sha256:fb616be3538599e797a2017cccca78e354c767165e8858ab5116813146041a24"},
+    {file = "multidict-6.0.5-cp312-cp312-musllinux_1_1_s390x.whl", hash = "sha256:14c2976aa9038c2629efa2c148022ed5eb4cb939e15ec7aace7ca932f48f9ba6"},
+    {file = "multidict-6.0.5-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:435a0984199d81ca178b9ae2c26ec3d49692d20ee29bc4c11a2a8d4514c67eda"},
+    {file = "multidict-6.0.5-cp312-cp312-win32.whl", hash = "sha256:9fe7b0653ba3d9d65cbe7698cca585bf0f8c83dbbcc710db9c90f478e175f2d5"},
+    {file = "multidict-6.0.5-cp312-cp312-win_amd64.whl", hash = "sha256:01265f5e40f5a17f8241d52656ed27192be03bfa8764d88e8220141d1e4b3556"},
+    {file = "multidict-6.0.5-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:19fe01cea168585ba0f678cad6f58133db2aa14eccaf22f88e4a6dccadfad8b3"},
+    {file = "multidict-6.0.5-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6bf7a982604375a8d49b6cc1b781c1747f243d91b81035a9b43a2126c04766f5"},
+    {file = "multidict-6.0.5-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:107c0cdefe028703fb5dafe640a409cb146d44a6ae201e55b35a4af8e95457dd"},
+    {file = "multidict-6.0.5-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:403c0911cd5d5791605808b942c88a8155c2592e05332d2bf78f18697a5fa15e"},
+    {file = "multidict-6.0.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aeaf541ddbad8311a87dd695ed9642401131ea39ad7bc8cf3ef3967fd093b626"},
+    {file = "multidict-6.0.5-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e4972624066095e52b569e02b5ca97dbd7a7ddd4294bf4e7247d52635630dd83"},
+    {file = "multidict-6.0.5-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:d946b0a9eb8aaa590df1fe082cee553ceab173e6cb5b03239716338629c50c7a"},
+    {file = "multidict-6.0.5-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:b55358304d7a73d7bdf5de62494aaf70bd33015831ffd98bc498b433dfe5b10c"},
+    {file = "multidict-6.0.5-cp37-cp37m-musllinux_1_1_ppc64le.whl", hash = "sha256:a3145cb08d8625b2d3fee1b2d596a8766352979c9bffe5d7833e0503d0f0b5e5"},
+    {file = "multidict-6.0.5-cp37-cp37m-musllinux_1_1_s390x.whl", hash = "sha256:d65f25da8e248202bd47445cec78e0025c0fe7582b23ec69c3b27a640dd7a8e3"},
+    {file = "multidict-6.0.5-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:c9bf56195c6bbd293340ea82eafd0071cb3d450c703d2c93afb89f93b8386ccc"},
+    {file = "multidict-6.0.5-cp37-cp37m-win32.whl", hash = "sha256:69db76c09796b313331bb7048229e3bee7928eb62bab5e071e9f7fcc4879caee"},
+    {file = "multidict-6.0.5-cp37-cp37m-win_amd64.whl", hash = "sha256:fce28b3c8a81b6b36dfac9feb1de115bab619b3c13905b419ec71d03a3fc1423"},
+    {file = "multidict-6.0.5-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:76f067f5121dcecf0d63a67f29080b26c43c71a98b10c701b0677e4a065fbd54"},
+    {file = "multidict-6.0.5-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:b82cc8ace10ab5bd93235dfaab2021c70637005e1ac787031f4d1da63d493c1d"},
+    {file = "multidict-6.0.5-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:5cb241881eefd96b46f89b1a056187ea8e9ba14ab88ba632e68d7a2ecb7aadf7"},
+    {file = "multidict-6.0.5-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e8e94e6912639a02ce173341ff62cc1201232ab86b8a8fcc05572741a5dc7d93"},
+    {file = "multidict-6.0.5-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:09a892e4a9fb47331da06948690ae38eaa2426de97b4ccbfafbdcbe5c8f37ff8"},
+    {file = "multidict-6.0.5-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:55205d03e8a598cfc688c71ca8ea5f66447164efff8869517f175ea632c7cb7b"},
+    {file = "multidict-6.0.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:37b15024f864916b4951adb95d3a80c9431299080341ab9544ed148091b53f50"},
+    {file = "multidict-6.0.5-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f2a1dee728b52b33eebff5072817176c172050d44d67befd681609b4746e1c2e"},
+    {file = "multidict-6.0.5-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:edd08e6f2f1a390bf137080507e44ccc086353c8e98c657e666c017718561b89"},
+    {file = "multidict-6.0.5-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:60d698e8179a42ec85172d12f50b1668254628425a6bd611aba022257cac1386"},
+    {file = "multidict-6.0.5-cp38-cp38-musllinux_1_1_ppc64le.whl", hash = "sha256:3d25f19500588cbc47dc19081d78131c32637c25804df8414463ec908631e453"},
+    {file = "multidict-6.0.5-cp38-cp38-musllinux_1_1_s390x.whl", hash = "sha256:4cc0ef8b962ac7a5e62b9e826bd0cd5040e7d401bc45a6835910ed699037a461"},
+    {file = "multidict-6.0.5-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:eca2e9d0cc5a889850e9bbd68e98314ada174ff6ccd1129500103df7a94a7a44"},
+    {file = "multidict-6.0.5-cp38-cp38-win32.whl", hash = "sha256:4a6a4f196f08c58c59e0b8ef8ec441d12aee4125a7d4f4fef000ccb22f8d7241"},
+    {file = "multidict-6.0.5-cp38-cp38-win_amd64.whl", hash = "sha256:0275e35209c27a3f7951e1ce7aaf93ce0d163b28948444bec61dd7badc6d3f8c"},
+    {file = "multidict-6.0.5-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:e7be68734bd8c9a513f2b0cfd508802d6609da068f40dc57d4e3494cefc92929"},
+    {file = "multidict-6.0.5-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:1d9ea7a7e779d7a3561aade7d596649fbecfa5c08a7674b11b423783217933f9"},
+    {file = "multidict-6.0.5-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:ea1456df2a27c73ce51120fa2f519f1bea2f4a03a917f4a43c8707cf4cbbae1a"},
+    {file = "multidict-6.0.5-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cf590b134eb70629e350691ecca88eac3e3b8b3c86992042fb82e3cb1830d5e1"},
+    {file = "multidict-6.0.5-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5c0631926c4f58e9a5ccce555ad7747d9a9f8b10619621f22f9635f069f6233e"},
+    {file = "multidict-6.0.5-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:dce1c6912ab9ff5f179eaf6efe7365c1f425ed690b03341911bf4939ef2f3046"},
+    {file = "multidict-6.0.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c0868d64af83169e4d4152ec612637a543f7a336e4a307b119e98042e852ad9c"},
+    {file = "multidict-6.0.5-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:141b43360bfd3bdd75f15ed811850763555a251e38b2405967f8e25fb43f7d40"},
+    {file = "multidict-6.0.5-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:7df704ca8cf4a073334e0427ae2345323613e4df18cc224f647f251e5e75a527"},
+    {file = "multidict-6.0.5-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:6214c5a5571802c33f80e6c84713b2c79e024995b9c5897f794b43e714daeec9"},
+    {file = "multidict-6.0.5-cp39-cp39-musllinux_1_1_ppc64le.whl", hash = "sha256:cd6c8fca38178e12c00418de737aef1261576bd1b6e8c6134d3e729a4e858b38"},
+    {file = "multidict-6.0.5-cp39-cp39-musllinux_1_1_s390x.whl", hash = "sha256:e02021f87a5b6932fa6ce916ca004c4d441509d33bbdbeca70d05dff5e9d2479"},
+    {file = "multidict-6.0.5-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:ebd8d160f91a764652d3e51ce0d2956b38efe37c9231cd82cfc0bed2e40b581c"},
+    {file = "multidict-6.0.5-cp39-cp39-win32.whl", hash = "sha256:04da1bb8c8dbadf2a18a452639771951c662c5ad03aefe4884775454be322c9b"},
+    {file = "multidict-6.0.5-cp39-cp39-win_amd64.whl", hash = "sha256:d6f6d4f185481c9669b9447bf9d9cf3b95a0e9df9d169bbc17e363b7d5487755"},
+    {file = "multidict-6.0.5-py3-none-any.whl", hash = "sha256:0d63c74e3d7ab26de115c49bffc92cc77ed23395303d496eae515d4204a625e7"},
+    {file = "multidict-6.0.5.tar.gz", hash = "sha256:f7e301075edaf50500f0b341543c41194d8df3ae5caf4702f2095f3ca73dd8da"},
 ]
 
 [[package]]
@@ -2048,55 +2080,132 @@ files = [
 
 [[package]]
 name = "pydantic"
-version = "1.10.13"
-description = "Data validation and settings management using python type hints"
+version = "2.7.4"
+description = "Data validation using Python type hints"
 optional = false
-python-versions = ">=3.7"
+python-versions = ">=3.8"
 files = [
-    {file = "pydantic-1.10.13-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:efff03cc7a4f29d9009d1c96ceb1e7a70a65cfe86e89d34e4a5f2ab1e5693737"},
-    {file = "pydantic-1.10.13-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:3ecea2b9d80e5333303eeb77e180b90e95eea8f765d08c3d278cd56b00345d01"},
-    {file = "pydantic-1.10.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1740068fd8e2ef6eb27a20e5651df000978edce6da6803c2bef0bc74540f9548"},
-    {file = "pydantic-1.10.13-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:84bafe2e60b5e78bc64a2941b4c071a4b7404c5c907f5f5a99b0139781e69ed8"},
-    {file = "pydantic-1.10.13-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:bc0898c12f8e9c97f6cd44c0ed70d55749eaf783716896960b4ecce2edfd2d69"},
-    {file = "pydantic-1.10.13-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:654db58ae399fe6434e55325a2c3e959836bd17a6f6a0b6ca8107ea0571d2e17"},
-    {file = "pydantic-1.10.13-cp310-cp310-win_amd64.whl", hash = "sha256:75ac15385a3534d887a99c713aa3da88a30fbd6204a5cd0dc4dab3d770b9bd2f"},
-    {file = "pydantic-1.10.13-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:c553f6a156deb868ba38a23cf0df886c63492e9257f60a79c0fd8e7173537653"},
-    {file = "pydantic-1.10.13-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:5e08865bc6464df8c7d61439ef4439829e3ab62ab1669cddea8dd00cd74b9ffe"},
-    {file = "pydantic-1.10.13-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e31647d85a2013d926ce60b84f9dd5300d44535a9941fe825dc349ae1f760df9"},
-    {file = "pydantic-1.10.13-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:210ce042e8f6f7c01168b2d84d4c9eb2b009fe7bf572c2266e235edf14bacd80"},
-    {file = "pydantic-1.10.13-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:8ae5dd6b721459bfa30805f4c25880e0dd78fc5b5879f9f7a692196ddcb5a580"},
-    {file = "pydantic-1.10.13-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:f8e81fc5fb17dae698f52bdd1c4f18b6ca674d7068242b2aff075f588301bbb0"},
-    {file = "pydantic-1.10.13-cp311-cp311-win_amd64.whl", hash = "sha256:61d9dce220447fb74f45e73d7ff3b530e25db30192ad8d425166d43c5deb6df0"},
-    {file = "pydantic-1.10.13-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:4b03e42ec20286f052490423682016fd80fda830d8e4119f8ab13ec7464c0132"},
-    {file = "pydantic-1.10.13-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f59ef915cac80275245824e9d771ee939133be38215555e9dc90c6cb148aaeb5"},
-    {file = "pydantic-1.10.13-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:5a1f9f747851338933942db7af7b6ee8268568ef2ed86c4185c6ef4402e80ba8"},
-    {file = "pydantic-1.10.13-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:97cce3ae7341f7620a0ba5ef6cf043975cd9d2b81f3aa5f4ea37928269bc1b87"},
-    {file = "pydantic-1.10.13-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:854223752ba81e3abf663d685f105c64150873cc6f5d0c01d3e3220bcff7d36f"},
-    {file = "pydantic-1.10.13-cp37-cp37m-win_amd64.whl", hash = "sha256:b97c1fac8c49be29486df85968682b0afa77e1b809aff74b83081cc115e52f33"},
-    {file = "pydantic-1.10.13-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:c958d053453a1c4b1c2062b05cd42d9d5c8eb67537b8d5a7e3c3032943ecd261"},
-    {file = "pydantic-1.10.13-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:4c5370a7edaac06daee3af1c8b1192e305bc102abcbf2a92374b5bc793818599"},
-    {file = "pydantic-1.10.13-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7d6f6e7305244bddb4414ba7094ce910560c907bdfa3501e9db1a7fd7eaea127"},
-    {file = "pydantic-1.10.13-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d3a3c792a58e1622667a2837512099eac62490cdfd63bd407993aaf200a4cf1f"},
-    {file = "pydantic-1.10.13-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:c636925f38b8db208e09d344c7aa4f29a86bb9947495dd6b6d376ad10334fb78"},
-    {file = "pydantic-1.10.13-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:678bcf5591b63cc917100dc50ab6caebe597ac67e8c9ccb75e698f66038ea953"},
-    {file = "pydantic-1.10.13-cp38-cp38-win_amd64.whl", hash = "sha256:6cf25c1a65c27923a17b3da28a0bdb99f62ee04230c931d83e888012851f4e7f"},
-    {file = "pydantic-1.10.13-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:8ef467901d7a41fa0ca6db9ae3ec0021e3f657ce2c208e98cd511f3161c762c6"},
-    {file = "pydantic-1.10.13-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:968ac42970f57b8344ee08837b62f6ee6f53c33f603547a55571c954a4225691"},
-    {file = "pydantic-1.10.13-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9849f031cf8a2f0a928fe885e5a04b08006d6d41876b8bbd2fc68a18f9f2e3fd"},
-    {file = "pydantic-1.10.13-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:56e3ff861c3b9c6857579de282ce8baabf443f42ffba355bf070770ed63e11e1"},
-    {file = "pydantic-1.10.13-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:9f00790179497767aae6bcdc36355792c79e7bbb20b145ff449700eb076c5f96"},
-    {file = "pydantic-1.10.13-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:75b297827b59bc229cac1a23a2f7a4ac0031068e5be0ce385be1462e7e17a35d"},
-    {file = "pydantic-1.10.13-cp39-cp39-win_amd64.whl", hash = "sha256:e70ca129d2053fb8b728ee7d1af8e553a928d7e301a311094b8a0501adc8763d"},
-    {file = "pydantic-1.10.13-py3-none-any.whl", hash = "sha256:b87326822e71bd5f313e7d3bfdc77ac3247035ac10b0c0618bd99dcf95b1e687"},
-    {file = "pydantic-1.10.13.tar.gz", hash = "sha256:32c8b48dcd3b2ac4e78b0ba4af3a2c2eb6048cb75202f0ea7b34feb740efc340"},
+    {file = "pydantic-2.7.4-py3-none-any.whl", hash = "sha256:ee8538d41ccb9c0a9ad3e0e5f07bf15ed8015b481ced539a1759d8cc89ae90d0"},
+    {file = "pydantic-2.7.4.tar.gz", hash = "sha256:0c84efd9548d545f63ac0060c1e4d39bb9b14db8b3c0652338aecc07b5adec52"},
 ]
 
 [package.dependencies]
-typing-extensions = ">=4.2.0"
+annotated-types = ">=0.4.0"
+pydantic-core = "2.18.4"
+typing-extensions = ">=4.6.1"
 
 [package.extras]
-dotenv = ["python-dotenv (>=0.10.4)"]
-email = ["email-validator (>=1.0.3)"]
+email = ["email-validator (>=2.0.0)"]
+
+[[package]]
+name = "pydantic-core"
+version = "2.18.4"
+description = "Core functionality for Pydantic validation and serialization"
+optional = false
+python-versions = ">=3.8"
+files = [
+    {file = "pydantic_core-2.18.4-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:f76d0ad001edd426b92233d45c746fd08f467d56100fd8f30e9ace4b005266e4"},
+    {file = "pydantic_core-2.18.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:59ff3e89f4eaf14050c8022011862df275b552caef8082e37b542b066ce1ff26"},
+    {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a55b5b16c839df1070bc113c1f7f94a0af4433fcfa1b41799ce7606e5c79ce0a"},
+    {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:4d0dcc59664fcb8974b356fe0a18a672d6d7cf9f54746c05f43275fc48636851"},
+    {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8951eee36c57cd128f779e641e21eb40bc5073eb28b2d23f33eb0ef14ffb3f5d"},
+    {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4701b19f7e3a06ea655513f7938de6f108123bf7c86bbebb1196eb9bd35cf724"},
+    {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e00a3f196329e08e43d99b79b286d60ce46bed10f2280d25a1718399457e06be"},
+    {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:97736815b9cc893b2b7f663628e63f436018b75f44854c8027040e05230eeddb"},
+    {file = "pydantic_core-2.18.4-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:6891a2ae0e8692679c07728819b6e2b822fb30ca7445f67bbf6509b25a96332c"},
+    {file = "pydantic_core-2.18.4-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:bc4ff9805858bd54d1a20efff925ccd89c9d2e7cf4986144b30802bf78091c3e"},
+    {file = "pydantic_core-2.18.4-cp310-none-win32.whl", hash = "sha256:1b4de2e51bbcb61fdebd0ab86ef28062704f62c82bbf4addc4e37fa4b00b7cbc"},
+    {file = "pydantic_core-2.18.4-cp310-none-win_amd64.whl", hash = "sha256:6a750aec7bf431517a9fd78cb93c97b9b0c496090fee84a47a0d23668976b4b0"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:942ba11e7dfb66dc70f9ae66b33452f51ac7bb90676da39a7345e99ffb55402d"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:b2ebef0e0b4454320274f5e83a41844c63438fdc874ea40a8b5b4ecb7693f1c4"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a642295cd0c8df1b86fc3dced1d067874c353a188dc8e0f744626d49e9aa51c4"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5f09baa656c904807e832cf9cce799c6460c450c4ad80803517032da0cd062e2"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:98906207f29bc2c459ff64fa007afd10a8c8ac080f7e4d5beff4c97086a3dabd"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:19894b95aacfa98e7cb093cd7881a0c76f55731efad31073db4521e2b6ff5b7d"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0fbbdc827fe5e42e4d196c746b890b3d72876bdbf160b0eafe9f0334525119c8"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:f85d05aa0918283cf29a30b547b4df2fbb56b45b135f9e35b6807cb28bc47951"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:e85637bc8fe81ddb73fda9e56bab24560bdddfa98aa64f87aaa4e4b6730c23d2"},
+    {file = "pydantic_core-2.18.4-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:2f5966897e5461f818e136b8451d0551a2e77259eb0f73a837027b47dc95dab9"},
+    {file = "pydantic_core-2.18.4-cp311-none-win32.whl", hash = "sha256:44c7486a4228413c317952e9d89598bcdfb06399735e49e0f8df643e1ccd0558"},
+    {file = "pydantic_core-2.18.4-cp311-none-win_amd64.whl", hash = "sha256:8a7164fe2005d03c64fd3b85649891cd4953a8de53107940bf272500ba8a788b"},
+    {file = "pydantic_core-2.18.4-cp311-none-win_arm64.whl", hash = "sha256:4e99bc050fe65c450344421017f98298a97cefc18c53bb2f7b3531eb39bc7805"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:6f5c4d41b2771c730ea1c34e458e781b18cc668d194958e0112455fff4e402b2"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:2fdf2156aa3d017fddf8aea5adfba9f777db1d6022d392b682d2a8329e087cef"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4748321b5078216070b151d5271ef3e7cc905ab170bbfd27d5c83ee3ec436695"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:847a35c4d58721c5dc3dba599878ebbdfd96784f3fb8bb2c356e123bdcd73f34"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3c40d4eaad41f78e3bbda31b89edc46a3f3dc6e171bf0ecf097ff7a0ffff7cb1"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:21a5e440dbe315ab9825fcd459b8814bb92b27c974cbc23c3e8baa2b76890077"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:01dd777215e2aa86dfd664daed5957704b769e726626393438f9c87690ce78c3"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:4b06beb3b3f1479d32befd1f3079cc47b34fa2da62457cdf6c963393340b56e9"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:564d7922e4b13a16b98772441879fcdcbe82ff50daa622d681dd682175ea918c"},
+    {file = "pydantic_core-2.18.4-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:0eb2a4f660fcd8e2b1c90ad566db2b98d7f3f4717c64fe0a83e0adb39766d5b8"},
+    {file = "pydantic_core-2.18.4-cp312-none-win32.whl", hash = "sha256:8b8bab4c97248095ae0c4455b5a1cd1cdd96e4e4769306ab19dda135ea4cdb07"},
+    {file = "pydantic_core-2.18.4-cp312-none-win_amd64.whl", hash = "sha256:14601cdb733d741b8958224030e2bfe21a4a881fb3dd6fbb21f071cabd48fa0a"},
+    {file = "pydantic_core-2.18.4-cp312-none-win_arm64.whl", hash = "sha256:c1322d7dd74713dcc157a2b7898a564ab091ca6c58302d5c7b4c07296e3fd00f"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-macosx_10_12_x86_64.whl", hash = "sha256:823be1deb01793da05ecb0484d6c9e20baebb39bd42b5d72636ae9cf8350dbd2"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:ebef0dd9bf9b812bf75bda96743f2a6c5734a02092ae7f721c048d156d5fabae"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ae1d6df168efb88d7d522664693607b80b4080be6750c913eefb77e34c12c71a"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f9899c94762343f2cc2fc64c13e7cae4c3cc65cdfc87dd810a31654c9b7358cc"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:99457f184ad90235cfe8461c4d70ab7dd2680e28821c29eca00252ba90308c78"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:18f469a3d2a2fdafe99296a87e8a4c37748b5080a26b806a707f25a902c040a8"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b7cdf28938ac6b8b49ae5e92f2735056a7ba99c9b110a474473fd71185c1af5d"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:938cb21650855054dc54dfd9120a851c974f95450f00683399006aa6e8abb057"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:44cd83ab6a51da80fb5adbd9560e26018e2ac7826f9626bc06ca3dc074cd198b"},
+    {file = "pydantic_core-2.18.4-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:972658f4a72d02b8abfa2581d92d59f59897d2e9f7e708fdabe922f9087773af"},
+    {file = "pydantic_core-2.18.4-cp38-none-win32.whl", hash = "sha256:1d886dc848e60cb7666f771e406acae54ab279b9f1e4143babc9c2258213daa2"},
+    {file = "pydantic_core-2.18.4-cp38-none-win_amd64.whl", hash = "sha256:bb4462bd43c2460774914b8525f79b00f8f407c945d50881568f294c1d9b4443"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-macosx_10_12_x86_64.whl", hash = "sha256:44a688331d4a4e2129140a8118479443bd6f1905231138971372fcde37e43528"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:a2fdd81edd64342c85ac7cf2753ccae0b79bf2dfa063785503cb85a7d3593223"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:86110d7e1907ab36691f80b33eb2da87d780f4739ae773e5fc83fb272f88825f"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:46387e38bd641b3ee5ce247563b60c5ca098da9c56c75c157a05eaa0933ed154"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:123c3cec203e3f5ac7b000bd82235f1a3eced8665b63d18be751f115588fea30"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:dc1803ac5c32ec324c5261c7209e8f8ce88e83254c4e1aebdc8b0a39f9ddb443"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:53db086f9f6ab2b4061958d9c276d1dbe3690e8dd727d6abf2321d6cce37fa94"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:abc267fa9837245cc28ea6929f19fa335f3dc330a35d2e45509b6566dc18be23"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:a0d829524aaefdebccb869eed855e2d04c21d2d7479b6cada7ace5448416597b"},
+    {file = "pydantic_core-2.18.4-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:509daade3b8649f80d4e5ff21aa5673e4ebe58590b25fe42fac5f0f52c6f034a"},
+    {file = "pydantic_core-2.18.4-cp39-none-win32.whl", hash = "sha256:ca26a1e73c48cfc54c4a76ff78df3727b9d9f4ccc8dbee4ae3f73306a591676d"},
+    {file = "pydantic_core-2.18.4-cp39-none-win_amd64.whl", hash = "sha256:c67598100338d5d985db1b3d21f3619ef392e185e71b8d52bceacc4a7771ea7e"},
+    {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-macosx_10_12_x86_64.whl", hash = "sha256:574d92eac874f7f4db0ca653514d823a0d22e2354359d0759e3f6a406db5d55d"},
+    {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:1f4d26ceb5eb9eed4af91bebeae4b06c3fb28966ca3a8fb765208cf6b51102ab"},
+    {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:77450e6d20016ec41f43ca4a6c63e9fdde03f0ae3fe90e7c27bdbeaece8b1ed4"},
+    {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d323a01da91851a4f17bf592faf46149c9169d68430b3146dcba2bb5e5719abc"},
+    {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:43d447dd2ae072a0065389092a231283f62d960030ecd27565672bd40746c507"},
+    {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:578e24f761f3b425834f297b9935e1ce2e30f51400964ce4801002435a1b41ef"},
+    {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:81b5efb2f126454586d0f40c4d834010979cb80785173d1586df845a632e4e6d"},
+    {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:ab86ce7c8f9bea87b9d12c7f0af71102acbf5ecbc66c17796cff45dae54ef9a5"},
+    {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-macosx_10_12_x86_64.whl", hash = "sha256:90afc12421df2b1b4dcc975f814e21bc1754640d502a2fbcc6d41e77af5ec312"},
+    {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:51991a89639a912c17bef4b45c87bd83593aee0437d8102556af4885811d59f5"},
+    {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:293afe532740370aba8c060882f7d26cfd00c94cae32fd2e212a3a6e3b7bc15e"},
+    {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b48ece5bde2e768197a2d0f6e925f9d7e3e826f0ad2271120f8144a9db18d5c8"},
+    {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:eae237477a873ab46e8dd748e515c72c0c804fb380fbe6c85533c7de51f23a8f"},
+    {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:834b5230b5dfc0c1ec37b2fda433b271cbbc0e507560b5d1588e2cc1148cf1ce"},
+    {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:e858ac0a25074ba4bce653f9b5d0a85b7456eaddadc0ce82d3878c22489fa4ee"},
+    {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:2fd41f6eff4c20778d717af1cc50eca52f5afe7805ee530a4fbd0bae284f16e9"},
+    {file = "pydantic_core-2.18.4.tar.gz", hash = "sha256:ec3beeada09ff865c344ff3bc2f427f5e6c26401cc6113d77e372c3fdac73864"},
+]
+
+[package.dependencies]
+typing-extensions = ">=4.6.0,<4.7.0 || >4.7.0"
+
+[[package]]
+name = "pydantic-settings"
+version = "2.3.4"
+description = "Settings management using Pydantic"
+optional = false
+python-versions = ">=3.8"
+files = [
+    {file = "pydantic_settings-2.3.4-py3-none-any.whl", hash = "sha256:11ad8bacb68a045f00e4f862c7a718c8a9ec766aa8fd4c32e39a0594b207b53a"},
+    {file = "pydantic_settings-2.3.4.tar.gz", hash = "sha256:c5802e3d62b78e82522319bbc9b8f8ffb28ad1c988a99311d04f2a6051fca0a7"},
+]
+
+[package.dependencies]
+pydantic = ">=2.7.0"
+python-dotenv = ">=0.21.0"
+
+[package.extras]
+toml = ["tomli (>=2.0.1)"]
+yaml = ["pyyaml (>=6.0.1)"]
 
 [[package]]
 name = "pyflakes"
@@ -2192,21 +2301,21 @@ testing = ["argcomplete", "attrs (>=19.2.0)", "hypothesis (>=3.56)", "mock", "no
 
 [[package]]
 name = "pytest-asyncio"
-version = "0.21.1"
+version = "0.23.7"
 description = "Pytest support for asyncio"
 optional = false
-python-versions = ">=3.7"
+python-versions = ">=3.8"
 files = [
-    {file = "pytest-asyncio-0.21.1.tar.gz", hash = "sha256:40a7eae6dded22c7b604986855ea48400ab15b069ae38116e8c01238e9eeb64d"},
-    {file = "pytest_asyncio-0.21.1-py3-none-any.whl", hash = "sha256:8666c1c8ac02631d7c51ba282e0c69a8a452b211ffedf2599099845da5c5c37b"},
+    {file = "pytest_asyncio-0.23.7-py3-none-any.whl", hash = "sha256:009b48127fbe44518a547bddd25611551b0e43ccdbf1e67d12479f569832c20b"},
+    {file = "pytest_asyncio-0.23.7.tar.gz", hash = "sha256:5f5c72948f4c49e7db4f29f2521d4031f1c27f86e57b046126654083d4770268"},
 ]
 
 [package.dependencies]
-pytest = ">=7.0.0"
+pytest = ">=7.0.0,<9"
 
 [package.extras]
 docs = ["sphinx (>=5.3)", "sphinx-rtd-theme (>=1.0)"]
-testing = ["coverage (>=6.2)", "flaky (>=3.5.0)", "hypothesis (>=5.7.1)", "mypy (>=0.931)", "pytest-trio (>=0.7.0)"]
+testing = ["coverage (>=6.2)", "hypothesis (>=5.7.1)"]
 
 [[package]]
 name = "pytest-cov"
@@ -2628,31 +2737,6 @@ exceptiongroup = {version = "*", markers = "python_version < \"3.11\""}
 trio = ">=0.11"
 wsproto = ">=0.14"
 
-[[package]]
-name = "types-requests"
-version = "2.31.0.6"
-description = "Typing stubs for requests"
-optional = false
-python-versions = ">=3.7"
-files = [
-    {file = "types-requests-2.31.0.6.tar.gz", hash = "sha256:cd74ce3b53c461f1228a9b783929ac73a666658f223e28ed29753771477b3bd0"},
-    {file = "types_requests-2.31.0.6-py3-none-any.whl", hash = "sha256:a2db9cb228a81da8348b49ad6db3f5519452dd20a9c1e1a868c83c5fe88fd1a9"},
-]
-
-[package.dependencies]
-types-urllib3 = "*"
-
-[[package]]
-name = "types-urllib3"
-version = "1.26.25.14"
-description = "Typing stubs for urllib3"
-optional = false
-python-versions = "*"
-files = [
-    {file = "types-urllib3-1.26.25.14.tar.gz", hash = "sha256:229b7f577c951b8c1b92c1bc2b2fdb0b49847bd2af6d1cc2a2e3dd340f3bda8f"},
-    {file = "types_urllib3-1.26.25.14-py3-none-any.whl", hash = "sha256:9683bbb7fb72e32bfe9d2be6e04875fbe1b3eeec3cbb4ea231435aa7fd6b4f0e"},
-]
-
 [[package]]
 name = "typing-extensions"
 version = "4.9.0"
@@ -2864,4 +2948,4 @@ multidict = ">=4.0"
 [metadata]
 lock-version = "2.0"
 python-versions = "^3.10"
-content-hash = "4a980e6d8f54a2f7f6a3c55d4f40ac3a4b27b5ac6573dd2a39e11213a4b126dd"
+content-hash = "26bd75befe5223095b65be293086edf52f34f9043e49107c80a105dc0387dd6a"

benchmark/pyproject.toml

@@ -9,43 +9,47 @@ packages = [{ include = "agbenchmark" }]
 
 [tool.poetry.dependencies]
 python = "^3.10"
-pytest = "^7.3.2"
-requests = "^2.31.0"
-openai = "^1.7.2"
-pydantic = "^1.10.9"
-python-dotenv = "^1.0.0"
+agent-protocol-client = {git = "https://github.com/Significant-Gravitas/agent-protocol.git", subdirectory = "packages/client/python"}
 click = "^8.1.3"
-types-requests = "^2.31.0.1"
+click-default-group = "^1.2.4"
+colorama = "^0.4.6"
+fastapi = "^0.109.1"
+gitpython = "^3.1.32"
+httpx = "^0.24.0"
+matplotlib = "^3.7.2"
+# Multidict 6.0.4 fails to install and is a dependency of aiohttp which is a depenedency of agent-protocol-client
+multidict = "^6.0.5"
+networkx = "^3.1"
+openai = "^1.7.2"
+pandas = "^2.0.3"
 pexpect = "^4.8.0"
 psutil = "^5.9.5"
-matplotlib = "^3.7.2"
-pandas = "^2.0.3"
-gitpython = "^3.1.32"
-networkx = "^3.1"
-colorama = "^0.4.6"
-pyvis = "^0.3.2"
-selenium = "^4.11.2"
-pytest-asyncio = "^0.21.1"
-uvicorn = "^0.23.2"
-fastapi = "^0.109.1"
+pydantic = "^2.7.2"
+pydantic-settings = "^2.3.4"
+pytest = "^7.3.2"
+pytest-asyncio = "^0.23.3"
+python-dotenv = "^1.0.0"
 python-multipart = "^0.0.7"
-toml = "^0.10.2"
-# helicone = "^1.0.9"  # incompatible with openai@^1.0.0
-httpx = "^0.24.0"
-agent-protocol-client = "^1.1.0"
-click-default-group = "^1.2.4"
+pyvis = "^0.3.2"
+requests = "^2.31.0"
+selenium = "^4.11.2"
 tabulate = "^0.9.0"
+toml = "^0.10.2"
+uvicorn = ">=0.23.2,<1"
 
 [tool.poetry.group.dev.dependencies]
 black = "^23.12.1"
 flake8 = "^7.0.0"
 isort = "^5.13.1"
 pyright = "^1.1.364"
-pandas = "^2.0.3"
+pre-commit = "^3.3.3"
+
+# Testing
+pytest-cov = "^5.0.0"
+
+# Dependencies for stuff in reports/
 gspread = "^5.10.0"
 oauth2client = "^4.1.3"
-pre-commit = "^3.3.3"
-pytest-cov = "^5.0.0"
 
 [tool.poetry.scripts]
 agbenchmark = "agbenchmark.__main__:cli"

benchmark/reports/format.py

@@ -1,20 +1,24 @@
 #!/usr/bin/env python3
 
+from pathlib import Path
+
 import click
 
 from agbenchmark.reports.processing.report_types import Report
 
 
 @click.command()
-@click.argument("report_json_file", type=click.Path(exists=True, dir_okay=False))
-def print_markdown_report(report_json_file: str):
+@click.argument(
+    "report_json_file", type=click.Path(exists=True, dir_okay=False, path_type=Path)
+)
+def print_markdown_report(report_json_file: Path):
     """
     Generates a Markdown report from a given report.json file.
 
     :param report_json_file: Path to the report.json file.
     :return: A string containing the Markdown formatted report.
     """
-    report = Report.parse_file(report_json_file)
+    report = Report.model_validate_json(report_json_file.read_text())
 
     # Header and metadata
     click.echo("# Benchmark Report")

benchmark/reports/match_records.py

@@ -14,7 +14,7 @@ from pydantic import BaseModel, Field
 class Metrics(BaseModel):
     difficulty: str
     success: bool
-    success_percent: float = Field(..., alias="success_%")
+    success_percent: float = Field(alias="success_%")
     run_time: Optional[str] = None
     fail_reason: Optional[str] = None
     attempted: Optional[bool] = None
@@ -100,7 +100,7 @@ def get_reports():
                         # Load the JSON data from the file
                         json_data = json.load(f)
                         print(f"Processing {report_file}")
-                        report = Report.parse_obj(json_data)
+                        report = Report.model_validate(json_data)
 
                         for test_name, test_data in report.tests.items():
                             test_json = {

cli.py

@@ -149,10 +149,11 @@ def start(agent_name: str, no_setup: bool):
             setup_process.wait()
             click.echo()
 
-        subprocess.Popen(["./run_benchmark", "serve"], cwd=agent_dir)
-        click.echo("⌛ (Re)starting benchmark server...")
-        wait_until_conn_ready(8080)
-        click.echo()
+        # FIXME: Doesn't work: Command not found: agbenchmark
+        # subprocess.Popen(["./run_benchmark", "serve"], cwd=agent_dir)
+        # click.echo("⌛ (Re)starting benchmark server...")
+        # wait_until_conn_ready(8080)
+        # click.echo()
 
         subprocess.Popen(["./run"], cwd=agent_dir)
         click.echo(f"⌛ (Re)starting agent '{agent_name}'...")

docs/content/AutoGPT/configuration/imagegen.md

@@ -1,63 +0,0 @@
-# 🖼 Image Generation configuration
-
-| Config variable  | Values                          |                      |
-| ---------------- | ------------------------------- | -------------------- |
-| `IMAGE_PROVIDER` | `dalle` `huggingface` `sdwebui` | **default: `dalle`** |
-
-## DALL-e
-
-In `.env`, make sure `IMAGE_PROVIDER` is commented (or set to `dalle`):
-
-```ini
-# IMAGE_PROVIDER=dalle    # this is the default
-```
-
-Further optional configuration:
-
-| Config variable  | Values             |                |
-| ---------------- | ------------------ | -------------- |
-| `IMAGE_SIZE`     | `256` `512` `1024` | default: `256` |
-
-## Hugging Face
-
-To use text-to-image models from Hugging Face, you need a Hugging Face API token.
-Link to the appropriate settings page: [Hugging Face > Settings > Tokens](https://huggingface.co/settings/tokens)
-
-Once you have an API token, uncomment and adjust these variables in your `.env`:
-
-```ini
-IMAGE_PROVIDER=huggingface
-HUGGINGFACE_API_TOKEN=your-huggingface-api-token
-```
-
-Further optional configuration:
-
-| Config variable           | Values                 |                                          |
-| ------------------------- | ---------------------- | ---------------------------------------- |
-| `HUGGINGFACE_IMAGE_MODEL` | see [available models] | default: `CompVis/stable-diffusion-v1-4` |
-
-[available models]: https://huggingface.co/models?pipeline_tag=text-to-image
-
-## Stable Diffusion WebUI
-
-It is possible to use your own self-hosted Stable Diffusion WebUI with AutoGPT:
-
-```ini
-IMAGE_PROVIDER=sdwebui
-```
-
-!!! note
-    Make sure you are running WebUI with `--api` enabled.
-
-Further optional configuration:
-
-| Config variable | Values                  |                                  |
-| --------------- | ----------------------- | -------------------------------- |
-| `SD_WEBUI_URL`  | URL to your WebUI       | default: `http://127.0.0.1:7860` |
-| `SD_WEBUI_AUTH` | `{username}:{password}` | *Note: do not copy the braces!*  |
-
-## Selenium
-
-```shell
-sudo Xvfb :10 -ac -screen 0 1024x768x24 & DISPLAY=:10 <YOUR_CLIENT>
-```

docs/content/AutoGPT/configuration/options.md

@@ -1,21 +1,18 @@
 # Configuration
 
-Configuration is controlled through the `Config` object. You can set configuration variables via the `.env` file. If you don't have a `.env` file, create a copy of `.env.template` in your `AutoGPT` folder and name it `.env`.
+Configuration of sensitive settings such as API credentials is done through environment variables.
+You can set configuration variables via the `.env` file. If you don't have a `.env` file, create a copy of `.env.template` in your `AutoGPT` folder and name it `.env`.
 
 ## Environment Variables
 
-- `AUDIO_TO_TEXT_PROVIDER`: Audio To Text Provider. Only option currently is `huggingface`. Default: huggingface
 - `AUTHORISE_COMMAND_KEY`: Key response accepted when authorising commands. Default: y
 - `ANTHROPIC_API_KEY`: Set this if you want to use Anthropic models with AutoGPT
 - `AZURE_CONFIG_FILE`: Location of the Azure Config file relative to the AutoGPT root directory. Default: azure.yaml
-- `BROWSE_CHUNK_MAX_LENGTH`: When browsing website, define the length of chunks to summarize. Default: 3000
-- `BROWSE_SPACY_LANGUAGE_MODEL`: [spaCy language model](https://spacy.io/usage/models) to use when creating chunks. Default: en_core_web_sm
-- `CHAT_MESSAGES_ENABLED`: Enable chat messages. Optional
-- `DISABLED_COMMANDS`: Commands to disable. Use comma separated names of commands. See the list of commands from built-in components [here](../components/components.md). Default: None
+- `COMPONENT_CONFIG_FILE`: Path to the component configuration file (json) for an agent. Optional
+- `DISABLED_COMMANDS`: Commands to disable. Use comma separated names of commands. See the list of commands from built-in components [here](../../forge/components/components.md). Default: None
 - `ELEVENLABS_API_KEY`: ElevenLabs API Key. Optional.
 - `ELEVENLABS_VOICE_ID`: ElevenLabs Voice ID. Optional.
 - `EMBEDDING_MODEL`: LLM Model to use for embedding tasks. Default: `text-embedding-3-small`
-- `EXECUTE_LOCAL_COMMANDS`: If shell commands should be executed locally. Default: False
 - `EXIT_KEY`: Exit key accepted to exit. Default: n
 - `FAST_LLM`: LLM Model to use for most tasks. Default: `gpt-3.5-turbo-0125`
 - `GITHUB_API_KEY`: [Github API Key](https://github.com/settings/tokens). Optional.
@@ -23,26 +20,16 @@ Configuration is controlled through the `Config` object. You can set configurati
 - `GOOGLE_API_KEY`: Google API key. Optional.
 - `GOOGLE_CUSTOM_SEARCH_ENGINE_ID`: [Google custom search engine ID](https://programmablesearchengine.google.com/controlpanel/all). Optional.
 - `GROQ_API_KEY`: Set this if you want to use Groq models with AutoGPT
-- `HEADLESS_BROWSER`: Use a headless browser while AutoGPT uses a web browser. Setting to `False` will allow you to see AutoGPT operate the browser. Default: True
 - `HUGGINGFACE_API_TOKEN`: HuggingFace API, to be used for both image generation and audio to text. Optional.
-- `HUGGINGFACE_AUDIO_TO_TEXT_MODEL`: HuggingFace audio to text model. Default: CompVis/stable-diffusion-v1-4
 - `HUGGINGFACE_IMAGE_MODEL`: HuggingFace model to use for image generation. Default: CompVis/stable-diffusion-v1-4
-- `IMAGE_PROVIDER`: Image provider. Options are `dalle`, `huggingface`, and `sdwebui`. Default: dalle
-- `IMAGE_SIZE`: Default size of image to generate. Default: 256
-- `OPENAI_API_KEY`: *REQUIRED*- Your [OpenAI API Key](https://platform.openai.com/account/api-keys).
+- `LLAMAFILE_API_BASE`: Llamafile API base URL. Default: `http://localhost:8080/v1`
+- `OPENAI_API_KEY`: Set this if you want to use OpenAI models; [OpenAI API Key](https://platform.openai.com/account/api-keys).
 - `OPENAI_ORGANIZATION`: Organization ID in OpenAI. Optional.
 - `PLAIN_OUTPUT`: Plain output, which disables the spinner. Default: False
 - `RESTRICT_TO_WORKSPACE`: The restrict file reading and writing to the workspace directory. Default: True
 - `SD_WEBUI_AUTH`: Stable Diffusion Web UI username:password pair. Optional.
-- `SD_WEBUI_URL`: Stable Diffusion Web UI URL. Default: http://localhost:7860
-- `SHELL_ALLOWLIST`: List of shell commands that ARE allowed to be executed by AutoGPT. Only applies if `SHELL_COMMAND_CONTROL` is set to `allowlist`. Default: None
-- `SHELL_COMMAND_CONTROL`: Whether to use `allowlist` or `denylist` to determine what shell commands can be executed (Default: denylist)
-- `SHELL_DENYLIST`: List of shell commands that ARE NOT allowed to be executed by AutoGPT. Only applies if `SHELL_COMMAND_CONTROL` is set to `denylist`. Default: sudo,su
 - `SMART_LLM`: LLM Model to use for "smart" tasks. Default: `gpt-4-turbo-preview`
 - `STREAMELEMENTS_VOICE`: StreamElements voice to use. Default: Brian
 - `TEMPERATURE`: Value of temperature given to OpenAI. Value from 0 to 2. Lower is more deterministic, higher is more random. See https://platform.openai.com/docs/api-reference/completions/create#completions/create-temperature
 - `TEXT_TO_SPEECH_PROVIDER`: Text to Speech Provider. Options are `gtts`, `macos`, `elevenlabs`, and `streamelements`. Default: gtts
-- `USER_AGENT`: User-Agent given when browsing websites. Default: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36"
 - `USE_AZURE`: Use Azure's LLM Default: False
-- `USE_WEB_BROWSER`: Which web browser to use. Options are `chrome`, `firefox`, `safari` or `edge` Default: chrome
-- `WIPE_REDIS_ON_START`: Wipes data / index on start. Default: True

docs/content/AutoGPT/configuration/search.md

@@ -2,36 +2,36 @@
 
 !!! note
     This section is optional. Use the official Google API if search attempts return
-    error 429. To use the `google_official_search` command, you need to set up your
-    Google API key in your environment variables.
+    error 429. To use the `google` command, you need to set up your
+    Google API key in your environment variables or pass it with configuration to the [`WebSearchComponent`](../../forge/components/built-in-components.md).
 
 Create your project:
 
 1. Go to the [Google Cloud Console](https://console.cloud.google.com/).
-2. If you don't already have an account, create one and log in
-3. Create a new project by clicking on the *Select a Project* dropdown at the top of the
+1. If you don't already have an account, create one and log in
+1. Create a new project by clicking on the *Select a Project* dropdown at the top of the
     page and clicking *New Project*
-4. Give it a name and click *Create*
-5. Set up a custom search API and add to your .env file:
-    5. Go to the [APIs & Services Dashboard](https://console.cloud.google.com/apis/dashboard)
-    6. Click *Enable APIs and Services*
-    7. Search for *Custom Search API* and click on it
-    8. Click *Enable*
-    9. Go to the [Credentials](https://console.cloud.google.com/apis/credentials) page
-    10. Click *Create Credentials*
-    11. Choose *API Key*
-    12. Copy the API key
-    13. Set it as the `GOOGLE_API_KEY` in your `.env` file
-14. [Enable](https://console.developers.google.com/apis/api/customsearch.googleapis.com)
+1. Give it a name and click *Create*
+1. Set up a custom search API and add to your .env file:
+    1. Go to the [APIs & Services Dashboard](https://console.cloud.google.com/apis/dashboard)
+    1. Click *Enable APIs and Services*
+    1. Search for *Custom Search API* and click on it
+    1. Click *Enable*
+    1. Go to the [Credentials](https://console.cloud.google.com/apis/credentials) page
+    1. Click *Create Credentials*
+    1. Choose *API Key*
+    1. Copy the API key
+    1. Set it as the `GOOGLE_API_KEY` in your `.env` file
+1. [Enable](https://console.developers.google.com/apis/api/customsearch.googleapis.com)
     the Custom Search API on your project. (Might need to wait few minutes to propagate.)
     Set up a custom search engine and add to your .env file:
-    15. Go to the [Custom Search Engine](https://cse.google.com/cse/all) page
-    16. Click *Add*
-    17. Set up your search engine by following the prompts.
+    1. Go to the [Custom Search Engine](https://cse.google.com/cse/all) page
+    1. Click *Add*
+    1. Set up your search engine by following the prompts.
         You can choose to search the entire web or specific sites
-    18. Once you've created your search engine, click on *Control Panel*
-    19. Click *Basics*
-    20. Copy the *Search engine ID*
-    21. Set it as the `CUSTOM_SEARCH_ENGINE_ID` in your `.env` file
+    1. Once you've created your search engine, click on *Control Panel*
+    1. Click *Basics*
+    1. Copy the *Search engine ID*
+    1. Set it as the `CUSTOM_SEARCH_ENGINE_ID` in your `.env` file
 
 _Remember that your free daily custom search quota allows only up to 100 searches. To increase this limit, you need to assign a billing account to the project to profit from up to 10K daily searches._

docs/content/AutoGPT/setup/docker.md

@@ -71,18 +71,25 @@
           - ./logs:/app/logs
           ## uncomment following lines if you want to make use of these files
           ## you must have them existing in the same folder as this docker-compose.yml
+          ## component configuration file
+          #- type: bind
+          #  source: ./config.json
+          #  target: /app/config.json
     ```
     </details>
 
 
-4. Download [`.env.template`][.env.template] and save it as `.env` in the AutoGPT folder.
-5. Follow the standard [configuration instructions](../index.md#completing-the-setup),
+1. Download [`.env.template`][.env.template] and save it as `.env` in the AutoGPT folder.
+2. Follow the standard [configuration instructions](../index.md#completing-the-setup),
    from step 3 onwards and excluding `poetry install` steps.
-6. Pull the latest image from [Docker Hub]
+3. Pull the latest image from [Docker Hub]
 
     ```shell
     docker pull significantgravitas/auto-gpt
     ```
+4. _Optional: mount configuration file._
+      If you have component configuration file, for example `config.json`, place it in `autogpt/data/` directory. Or place it in `autogpt/` and uncomment the line in `docker-compose.yml` that mounts it.
+      To learn more about configuring, see [Component configuration](../../forge/components/components.md#json-configuration)
 
 !!! note "Docker only supports headless browsing"
     AutoGPT uses a browser in headless mode by default: `HEADLESS_BROWSER=True`.

docs/content/AutoGPT/setup/index.md

@@ -50,7 +50,13 @@ Since we don't ship AutoGPT as a desktop application, you'll need to download th
 ### Completing the Setup
 
 Once you have cloned or downloaded the project, you can find the AutoGPT Agent in the
-`autogpt/` folder. In this folder:
+`autogpt/` folder.
+Inside this folder you can configure the AutoGPT application with an `.env` file and (optionally) a JSON configuration file:
+
+- `.env` for environment variables, which are mostly used for sensitive data like API keys
+- a JSON configuration file to customize certain features of AutoGPT's [Components](../../forge/components/introduction.md)
+
+See the [Configuration](../configuration/options.md) reference for a list of available environment variables.
 
 1. Find the file named `.env.template`. This file may
     be hidden by default in some operating systems due to the dot prefix. To reveal
@@ -71,6 +77,9 @@ Once you have cloned or downloaded the project, you can find the AutoGPT Agent i
 6. Save and close the `.env` file.
 7. _Optional: run `poetry install` to install all required dependencies._ The
     application also checks for and installs any required dependencies when it starts.
+8. _Optional: configure the JSON file (e.g. `config.json`) with your desired settings._
+    The application will use default settings if you don't provide a JSON configuration file.
+    Learn how to [set up the JSON configuration file](../../forge/components/components.md#json-configuration)
 
 You should now be able to explore the CLI (`./autogpt.sh --help`) and run the application.
 
@@ -79,7 +88,6 @@ See the [user guide](../usage.md) for further instructions.
 [show hidden files/Windows]: https://support.microsoft.com/en-us/windows/view-hidden-files-and-folders-in-windows-97fbc472-c603-9d90-91d0-1166d1d9f4b5
 [show hidden files/macOS]: https://www.pcmag.com/how-to/how-to-access-your-macs-hidden-files
 
-
 ## Setting up LLM providers
 
 You can use AutoGPT with any of the following LLM providers.
@@ -190,3 +198,66 @@ If you don't know which to choose, you can safely go with OpenAI*.
 
 [groq/api-keys]: https://console.groq.com/keys
 [groq/models]: https://console.groq.com/docs/models
+
+
+### Llamafile
+
+With llamafile you can run models locally, which means no need to set up billing,
+and guaranteed data privacy.
+
+For more information and in-depth documentation, check out the [llamafile documentation].
+
+!!! warning
+    At the moment, llamafile only serves one model at a time. This means you can not
+    set `SMART_LLM` and `FAST_LLM` to two different llamafile models.
+
+!!! warning
+    Due to the issues linked below, llamafiles don't work on WSL. To use a llamafile
+    with AutoGPT in WSL, you will have to run the llamafile in Windows (outside WSL).
+
+    <details>
+    <summary>Instructions</summary>
+
+    1. Get the `llamafile/serve.py` script through one of these two ways:
+        1. Clone the AutoGPT repo somewhere in your Windows environment,
+           with the script located at `autogpt/scripts/llamafile/serve.py`
+        2. Download just the [serve.py] script somewhere in your Windows environment
+    2. Make sure you have `click` installed: `pip install click`
+    3. Run `ip route | grep default | awk '{print $3}'` *inside WSL* to get the address
+       of the WSL host machine
+    4. Run `python3 serve.py --host {WSL_HOST_ADDR}`, where `{WSL_HOST_ADDR}`
+       is the address you found at step 3.
+       If port 8080 is taken, also specify a different port using `--port {PORT}`.
+    5. In WSL, set `LLAMAFILE_API_BASE=http://{WSL_HOST_ADDR}:8080/v1` in your `.env`.
+    6. Follow the rest of the regular instructions below.
+
+    [serve.py]: https://github.com/Significant-Gravitas/AutoGPT/blob/master/autogpt/scripts/llamafile/serve.py
+    </details>
+
+    * [Mozilla-Ocho/llamafile#356](https://github.com/Mozilla-Ocho/llamafile/issues/356)
+    * [Mozilla-Ocho/llamafile#100](https://github.com/Mozilla-Ocho/llamafile/issues/100)
+
+!!! note
+    These instructions will download and use `mistral-7b-instruct-v0.2.Q5_K_M.llamafile`.
+    `mistral-7b-instruct-v0.2` is currently the only tested and supported model.
+    If you want to try other models, you'll have to add them to `LlamafileModelName` in
+    [`llamafile.py`][forge/llamafile.py].
+    For optimal results, you may also have to add some logic to adapt the message format,
+    like `LlamafileProvider._adapt_chat_messages_for_mistral_instruct(..)` does.
+
+1. Run the llamafile serve script:
+   ```shell
+   python3 ./scripts/llamafile/serve.py
+   ```
+   The first time this is run, it will download a file containing the model + runtime,
+   which may take a while and a few gigabytes of disk space.
+
+   To force GPU acceleration, add `--use-gpu` to the command.
+
+3. In `.env`, set `SMART_LLM`/`FAST_LLM` or both to `mistral-7b-instruct-v0.2`
+
+4. If the server is running on different address than `http://localhost:8080/v1`,
+   set `LLAMAFILE_API_BASE` in `.env` to the right base URL
+
+[llamafile documentation]: https://github.com/Mozilla-Ocho/llamafile#readme
+[forge/llamafile.py]: https://github.com/Significant-Gravitas/AutoGPT/blob/master/forge/forge/llm/providers/llamafile/llamafile.py

docs/content/AutoGPT/usage.md

@@ -60,10 +60,6 @@ Options:
   --debug                         Enable Debug Mode
   --gpt3only                      Enable GPT3.5 Only Mode
   --gpt4only                      Enable GPT4 Only Mode
-  -b, --browser-name TEXT         Specifies which web-browser to use when
-                                  using selenium to scrape the web.
-  --allow-downloads               Dangerous: Allows AutoGPT to download files
-                                  natively.
   --skip-news                     Specifies whether to suppress the output of
                                   latest news on startup.
   --install-plugin-deps           Installs external dependencies for 3rd party
@@ -82,6 +78,7 @@ Options:
   --override-directives           If specified, --constraint, --resource and
                                   --best-practice will override the AI's
                                   directives instead of being appended to them
+  --component-config-file TEXT    Path to the json configuration file.
   --help                          Show this message and exit.
 ```
 </details>
@@ -128,10 +125,6 @@ Options:
   --debug                     Enable Debug Mode
   --gpt3only                  Enable GPT3.5 Only Mode
   --gpt4only                  Enable GPT4 Only Mode
-  -b, --browser-name TEXT     Specifies which web-browser to use when using
-                              selenium to scrape the web.
-  --allow-downloads           Dangerous: Allows AutoGPT to download files
-                              natively.
   --install-plugin-deps       Installs external dependencies for 3rd party
                               plugins.
   --help                      Show this message and exit.
@@ -220,5 +213,5 @@ For example, to disable python coding features, set it to the value below:
 DISABLED_COMMANDS=execute_python_code,execute_python_file
 ```
 
-[components]: ./components/components.md
-[commands]: ./components/built-in-components.md
+[components]: ../forge/components/components.md
+[commands]: ../forge/components/built-in-components.md

docs/content/forge/components/built-in-components.md

@@ -1,26 +1,34 @@
 # Built-in Components
 
 This page lists all [🧩 Components](./components.md) and [⚙️ Protocols](./protocols.md) they implement that are natively provided. They are used by the AutoGPT agent.
+Some components have additional configuration options listed in the table, see [Component configuration](./components.md/#component-configuration) to learn more.
+
+!!! note
+    If a configuration field uses environment variable, it still can be passed using configuration model. ### Value from the configuration takes precedence over env var! Env var will be only applied if value in the configuration is not set.
 
 ## `SystemComponent`
 
 Essential component to allow an agent to finish.
 
-**DirectiveProvider**
+### DirectiveProvider
+
 - Constraints about API budget
-  
-**MessageProvider**
+
+### MessageProvider
+
 - Current time and date
 - Remaining API budget and warnings if budget is low
 
-**CommandProvider**
+### CommandProvider
+
 - `finish` used when task is completed
 
 ## `UserInteractionComponent`
 
 Adds ability to interact with user in CLI.
 
-**CommandProvider**
+### CommandProvider
+
 - `ask_user` used to ask user for input
 
 ## `FileManagerComponent`
@@ -28,10 +36,21 @@ Adds ability to interact with user in CLI.
 Adds ability to read and write persistent files to local storage, Google Cloud Storage or Amazon's S3.
 Necessary for saving and loading agent's state (preserving session).
 
-**DirectiveProvider**
+### `FileManagerConfiguration`
+
+| Config variable  | Details                                | Type  | Default                            |
+| ---------------- | -------------------------------------- | ----- | ---------------------------------- |
+| `storage_path`   | Path to agent files, e.g. state        | `str` | `agents/{agent_id}/`[^1]           |
+| `workspace_path` | Path to files that agent has access to | `str` | `agents/{agent_id}/workspace/`[^1] |
+
+[^1] This option is set dynamically during component construction as opposed to by default inside the configuration model, `{agent_id}` is replaced with the agent's unique identifier.
+
+### DirectiveProvider
+
 - Resource information that it's possible to read and write files
 
-**CommandProvider**
+### CommandProvider
+
 - `read_file` used to read file
 - `write_file` used to write file
 - `list_folder` lists all files in a folder 
@@ -40,7 +59,18 @@ Necessary for saving and loading agent's state (preserving session).
 
 Lets the agent execute non-interactive Shell commands and Python code. Python execution works only if Docker is available.
 
-**CommandProvider**
+### `CodeExecutorConfiguration`
+
+| Config variable          | Details                                              | Type                        | Default           |
+| ------------------------ | ---------------------------------------------------- | --------------------------- | ----------------- |
+| `execute_local_commands` | Enable shell command execution                       | `bool`                      | `False`           |
+| `shell_command_control`  | Controls which list is used                          | `"allowlist" \| "denylist"` | `"allowlist"`     |
+| `shell_allowlist`        | List of allowed shell commands                       | `List[str]`                 | `[]`              |
+| `shell_denylist`         | List of prohibited shell commands                    | `List[str]`                 | `[]`              |
+| `docker_container_name`  | Name of the Docker container used for code execution | `str`                       | `"agent_sandbox"` |
+
+### CommandProvider
+
 - `execute_shell` execute shell command
 - `execute_shell_popen` execute shell command with popen
 - `execute_python_code` execute Python code
@@ -50,38 +80,93 @@ Lets the agent execute non-interactive Shell commands and Python code. Python ex
 
 Keeps track of agent's actions and their outcomes. Provides their summary to the prompt.
 
-**MessageProvider**
+### `ActionHistoryConfiguration`
+
+| Config variable        | Details                                                 | Type        | Default            |
+| ---------------------- | ------------------------------------------------------- | ----------- | ------------------ |
+| `llm_name`             | Name of the llm model used to compress the history      | `ModelName` | `"gpt-3.5-turbo"`  |
+| `max_tokens`           | Maximum number of tokens to use for the history summary | `int`       | `1024`             |
+| `spacy_language_model` | Language model used for summary chunking using spacy    | `str`       | `"en_core_web_sm"` |
+| `full_message_count`   | Number of cycles to include unsummarized in the prompt  | `int`       | `4`                |
+
+### MessageProvider
+
 - Agent's progress summary
 
-**AfterParse**
+### AfterParse
+
 - Register agent's action
 
-**ExecutionFailuer**
+### ExecutionFailure
+
 - Rewinds the agent's action, so it isn't saved
 
-**AfterExecute**
+### AfterExecute
+
 - Saves the agent's action result in the history
 
 ## `GitOperationsComponent`
 
-**CommandProvider**
+Adds ability to iteract with git repositories and GitHub.
+
+### `GitOperationsConfiguration`
+
+| Config variable   | Details                                   | Type  | Default |
+| ----------------- | ----------------------------------------- | ----- | ------- |
+| `github_username` | GitHub username, *ENV:* `GITHUB_USERNAME` | `str` | `None`  |
+| `github_api_key`  | GitHub API key, *ENV:* `GITHUB_API_KEY`   | `str` | `None`  |
+
+### CommandProvider
+
 - `clone_repository` used to clone a git repository
 
 ## `ImageGeneratorComponent`
 
-Adds ability to generate images using various providers, see [Image Generation configuration](./../configuration/imagegen.md) to learn more.
+Adds ability to generate images using various providers.
+
+### Hugging Face
+
+To use text-to-image models from Hugging Face, you need a Hugging Face API token.
+Link to the appropriate settings page: [Hugging Face > Settings > Tokens](https://huggingface.co/settings/tokens)
+
+### Stable Diffusion WebUI
+
+It is possible to use your own self-hosted Stable Diffusion WebUI with AutoGPT. ### Make sure you are running WebUI with `--api` enabled.
+
+### `ImageGeneratorConfiguration`
+
+| Config variable           | Details                                                       | Type                                    | Default                           |
+| ------------------------- | ------------------------------------------------------------- | --------------------------------------- | --------------------------------- |
+| `image_provider`          | Image generation provider                                     | `"dalle" \| "huggingface" \| "sdwebui"` | `"dalle"`                         |
+| `huggingface_image_model` | Hugging Face image model, see [available models]              | `str`                                   | `"CompVis/stable-diffusion-v1-4"` |
+| `huggingface_api_token`   | Hugging Face API token, *ENV:* `HUGGINGFACE_API_TOKEN`        | `str`                                   | `None`                            |
+| `sd_webui_url`            | URL to self-hosted Stable Diffusion WebUI                     | `str`                                   | `"http://localhost:7860"`         |
+| `sd_webui_auth`           | Basic auth for Stable Diffusion WebUI, *ENV:* `SD_WEBUI_AUTH` | `str` of format `{username}:{password}` | `None`                            |
+
+[available models]: https://huggingface.co/models?pipeline_tag=text-to-image
+
+### CommandProvider
 
-**CommandProvider**
 - `generate_image` used to generate an image given a prompt
 
 ## `WebSearchComponent`
 
-Allows agent to search the web.
+Allows agent to search the web. Google credentials aren't required for DuckDuckGo. [Instructions how to set up Google API key](../../AutoGPT/configuration/search.md)
+
+### `WebSearchConfiguration`
+
+| Config variable                  | Details                                                                 | Type  | Default |
+| -------------------------------- | ----------------------------------------------------------------------- | ----- | ------- |
+| `google_api_key`                 | Google API key, *ENV:* `GOOGLE_API_KEY`                                 | `str` | `None`  |
+| `google_custom_search_engine_id` | Google Custom Search Engine ID, *ENV:* `GOOGLE_CUSTOM_SEARCH_ENGINE_ID` | `str` | `None`  |
+| `duckduckgo_max_attempts`        | Maximum number of attempts to search using DuckDuckGo                   | `int` | `3`     |
+
+### DirectiveProvider
 
-**DirectiveProvider**
 - Resource information that it's possible to search the web
 
-**CommandProvider**
+### CommandProvider
+
 - `search_web` used to search the web using DuckDuckGo
 - `google` used to search the web using Google, requires API key
 
@@ -89,20 +174,34 @@ Allows agent to search the web.
 
 Allows agent to read websites using Selenium.
 
-**DirectiveProvider**
+### `WebSeleniumConfiguration`
+
+| Config variable               | Details                                     | Type                                          | Default                                                                                                                      |
+| ----------------------------- | ------------------------------------------- | --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
+| `llm_name`                    | Name of the llm model used to read websites | `ModelName`                                   | `"gpt-3.5-turbo"`                                                                                                            |
+| `web_browser`                 | Web browser used by Selenium                | `"chrome" \| "firefox" \| "safari" \| "edge"` | `"chrome"`                                                                                                                   |
+| `headless`                    | Run browser in headless mode                | `bool`                                        | `True`                                                                                                                       |
+| `user_agent`                  | User agent used by the browser              | `str`                                         | `"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36"` |
+| `browse_spacy_language_model` | Spacy language model used for chunking text | `str`                                         | `"en_core_web_sm"`                                                                                                           |
+
+### DirectiveProvider
+
 - Resource information that it's possible to read websites
 
-**CommandProvider**
+### CommandProvider
+
 - `read_website` used to read a specific url and look for specific topics or answer a question
 
 ## `ContextComponent`
 
 Adds ability to keep up-to-date file and folder content in the prompt.
 
-**MessageProvider**
+### MessageProvider
+
 - Content of elements in the context
 
-**CommandProvider**
+### CommandProvider
+
 - `open_file` used to open a file into context
 - `open_folder` used to open a folder into context
 - `close_context_item` remove an item from the context
@@ -111,5 +210,6 @@ Adds ability to keep up-to-date file and folder content in the prompt.
 
 Watches if agent is looping and switches to smart mode if necessary.
 
-**AfterParse**
+### AfterParse
+
 - Investigates what happened and switches to smart mode if necessary

docs/content/forge/components/creating-components.md

@@ -37,7 +37,7 @@ Since components are regular classes you can pass data (including other componen
 For example we can pass a config object and then retrieve an API key from it when needed:
 
 ```py
-class ConfigurableComponent(MessageProvider):
+class DataComponent(MessageProvider):
     def __init__(self, config: Config):
         self.config = config
 
@@ -51,6 +51,35 @@ class ConfigurableComponent(MessageProvider):
 !!! note
     Component-specific configuration handling isn't implemented yet.
 
+## Configuring components
+
+Components can be configured using a pydantic model.
+To make component configurable, it must inherit from `ConfigurableComponent[BM]` where `BM` is the configuration class inheriting from pydantic's `BaseModel`.
+You should pass the configuration instance to the `ConfigurableComponent`'s `__init__` or set its `config` property directly.
+Using configuration allows you to load confugration from a file, and also serialize and deserialize it easily for any agent.
+To learn more about configuration, including storing sensitive information and serialization see [Component Configuration](./components.md#component-configuration).
+
+```py
+# Example component configuration
+class UserGreeterConfiguration(BaseModel):
+    user_name: str
+
+class UserGreeterComponent(MessageProvider, ConfigurableComponent[UserGreeterConfiguration]):
+    def __init__(self):
+        # Creating configuration instance
+        # You could also pass it to the component constructor
+        # e.g. `def __init__(self, config: UserGreeterConfiguration):`
+        config = UserGreeterConfiguration(user_name="World")
+        # Passing the configuration instance to the parent class
+        UserGreeterComponent.__init__(self, config)
+        # This has the same effect as the line above:
+        # self.config = UserGreeterConfiguration(user_name="World")
+
+    def get_messages(self) -> Iterator[ChatMessage]:
+        # You can use the configuration like a regular model
+        yield ChatMessage.system(f"Hello, {self.config.user_name}!")
+```
+
 ## Providing commands
 
 To extend what an agent can do, you need to provide commands using `CommandProvider` protocol. For example to allow agent to multiply two numbers, you can create a component like this:
@@ -148,12 +177,12 @@ It gives an ability for the agent to ask user for input in the terminal.
         yield self.ask_user
     ```
 
-5. Since agent isn't always running in the terminal or interactive mode, we need to disable this component by setting `self._enabled` when it's not possible to ask for user input.
+5. Since agent isn't always running in the terminal or interactive mode, we need to disable this component by setting `self._enabled=False` when it's not possible to ask for user input.
 
     ```py
-    def __init__(self, config: Config):
+    def __init__(self, interactive_mode: bool):
         self.config = config
-        self._enabled = not config.noninteractive_mode
+        self._enabled = interactive_mode
     ```
 
 The final component should look like this:
@@ -164,10 +193,10 @@ class MyUserInteractionComponent(CommandProvider):
     """Provides commands to interact with the user."""
 
     # We pass config to check if we're in noninteractive mode
-    def __init__(self, config: Config):
+    def __init__(self, interactive_mode: bool):
         self.config = config
         # 5.
-        self._enabled = not config.noninteractive_mode
+        self._enabled = interactive_mode
 
     # 4.
     def get_commands(self) -> Iterator[Command]:
@@ -205,10 +234,10 @@ class MyAgent(Agent):
         settings: AgentSettings,
         llm_provider: MultiProvider,
         file_storage: FileStorage,
-        legacy_config: Config,
+        app_config: Config,
     ):
         # Call the parent constructor to bring in the default components
-        super().__init__(settings, llm_provider, file_storage, legacy_config)
+        super().__init__(settings, llm_provider, file_storage, app_config)
         # Disable the default user interaction component by overriding it
         self.user_interaction = MyUserInteractionComponent()
 ```
@@ -222,14 +251,14 @@ class MyAgent(Agent):
         settings: AgentSettings,
         llm_provider: MultiProvider,
         file_storage: FileStorage,
-        legacy_config: Config,
+        app_config: Config,
     ):
         # Call the parent constructor to bring in the default components
-        super().__init__(settings, llm_provider, file_storage, legacy_config)
+        super().__init__(settings, llm_provider, file_storage, app_config)
         # Disable the default user interaction component
         self.user_interaction = None
         # Add our own component
-        self.my_user_interaction = MyUserInteractionComponent(legacy_config)
+        self.my_user_interaction = MyUserInteractionComponent(app_config)
 ```
 
 ## Learn more

docs/content/forge/components/introduction.md

@@ -1,10 +1,16 @@
 # Component Agents
 
+!!! important
+    [Legacy plugins] no longer work with AutoGPT. They have been replaced by components,
+    although we're still working on a new system to load plug-in components.
+
+[Legacy plugins]: https://github.com/Significant-Gravitas/Auto-GPT-Plugins
+
 This guide explains the component-based architecture of AutoGPT agents. It's a new way of building agents that is more flexible and easier to extend. Components replace some agent's logic and plugins with a more modular and composable system.
 
 Agent is composed of *components*, and each *component* implements a range of *protocols* (interfaces), each one providing a specific functionality, e.g. additional commands or messages. Each *protocol* is handled in a specific order, defined by the agent. This allows for a clear separation of concerns and a more modular design.
 
-This system is simple, flexible, requires basically no configuration, and doesn't hide any data - anything can still be passed or accessed directly from or between components.
+This system is simple, flexible, and doesn't hide any data - anything can still be passed or accessed directly from or between components.
 
 ### Definitions & Guides
 

docs/content/forge/get-started.md

@@ -19,16 +19,16 @@ Forge is a ready-to-go template for *your* agent application. All the boilerplat
 
 ### 🚀 **Get Started!**
 
+The best way to get started is to fork or download the AutoGPT repository and look at the example agent in `forge/forge/agent/forge_agent.py`.
+This can work as a starting point for your own agent.
+Agents are built using *components* which provide different functionality, see the [Component Introduction](./components/introduction.md). You can find built-in components in `forge/forge/components/`.
+
+!!! warning
+    The tutorial series below is out of date.
+
 The getting started [tutorial series](https://aiedge.medium.com/autogpt-forge-e3de53cc58ec) will guide you through the process of setting up your project all the way through to building a generalist agent.  
 
 1. [AutoGPT Forge: A Comprehensive Guide to Your First Steps](https://aiedge.medium.com/autogpt-forge-a-comprehensive-guide-to-your-first-steps-a1dfdf46e3b4)
 2. [AutoGPT Forge: The Blueprint of an AI Agent](https://aiedge.medium.com/autogpt-forge-the-blueprint-of-an-ai-agent-75cd72ffde6)
 3. [AutoGPT Forge: Interacting with your Agent](https://aiedge.medium.com/autogpt-forge-interacting-with-your-agent-1214561b06b)
 4. [AutoGPT Forge: Crafting Intelligent Agent Logic](https://medium.com/@aiedge/autogpt-forge-crafting-intelligent-agent-logic-bc5197b14cb4)
-
-Coming soon:
-
-5. Interacting with and Benchmarking your Agent
-6. Abilities
-7. The Planning Loop
-8. Memories

docs/content/server/new_blocks.md

@@ -0,0 +1,149 @@
+# Contributing to AutoGPT Agent Server: Creating and Testing Blocks
+
+This guide will walk you through the process of creating and testing a new block for the AutoGPT Agent Server, using the WikipediaSummaryBlock as an example.
+
+## Understanding Blocks and Testing
+
+Blocks are reusable components that can be connected to form a graph representing an agent's behavior. Each block has inputs, outputs, and a specific function. Proper testing is crucial to ensure blocks work correctly and consistently.
+
+## Creating and Testing a New Block
+
+Follow these steps to create and test a new block:
+
+1. **Create a new Python file** in the `autogpt_server/blocks` directory. Name it descriptively and use snake_case. For example: `get_wikipedia_summary.py`.
+
+2. **Import necessary modules and create a class that inherits from `Block`**. Make sure to include all necessary imports for your block.
+
+ Every block should contain the following:
+
+   ```python
+   from autogpt_server.data.block import Block, BlockSchema, BlockOutput
+   ```
+
+   Example for the Wikipedia summary block:
+
+   ```python
+   from autogpt_server.data.block import Block, BlockSchema, BlockOutput
+   from autogpt_server.utils.get_request import GetRequest
+   import requests
+
+   class WikipediaSummaryBlock(Block, GetRequest):
+       # Block implementation will go here
+   ```
+
+3. **Define the input and output schemas** using `BlockSchema`. These schemas specify the data structure that the block expects to receive (input) and produce (output).
+
+   - The input schema defines the structure of the data the block will process. Each field in the schema represents a required piece of input data.
+   - The output schema defines the structure of the data the block will return after processing. Each field in the schema represents a piece of output data.
+
+   Example:
+
+   ```python
+   class Input(BlockSchema):
+       topic: str  # The topic to get the Wikipedia summary for
+
+   class Output(BlockSchema):
+       summary: str  # The summary of the topic from Wikipedia
+       error: str  # Any error message if the request fails
+   ```
+
+4. **Implement the `__init__` method, including test data and mocks:**
+
+   ```python
+   def __init__(self):
+       super().__init__(
+           # Unique ID for the block
+           # you can generate this with this python one liner
+           # print(__import__('uuid').uuid4())
+           id="h5e7f8g9-1b2c-3d4e-5f6g-7h8i9j0k1l2m",
+           input_schema=WikipediaSummaryBlock.Input,  # Assign input schema
+           output_schema=WikipediaSummaryBlock.Output,  # Assign output schema
+
+            # Provide sample input, output and test mock for testing the block
+
+           test_input={"topic": "Artificial Intelligence"},
+           test_output=("summary", "summary content"),
+           test_mock={"get_request": lambda url, json: {"extract": "summary content"}},
+       )
+   ```
+
+   - `id`: A unique identifier for the block.
+
+   - `input_schema` and `output_schema`: Define the structure of the input and output data.
+
+   Let's break down the testing components:
+
+   - `test_input`: This is a sample input that will be used to test the block. It should be a valid input according to your Input schema.
+
+   - `test_output`: This is the expected output when running the block with the `test_input`. It should match your Output schema. For non-deterministic outputs or when you only want to assert the type, you can use Python types instead of specific values. In this example, `("summary", str)` asserts that the output key is "summary" and its value is a string.
+
+   - `test_mock`: This is crucial for blocks that make network calls. It provides a mock function that replaces the actual network call during testing. 
+
+     In this case, we're mocking the `get_request` method to always return a dictionary with an 'extract' key, simulating a successful API response. This allows us to test the block's logic without making actual network requests, which could be slow, unreliable, or rate-limited.
+
+5. **Implement the `run` method with error handling:**, this should contain the main logic of the block:
+
+   ```python
+   def run(self, input_data: Input) -> BlockOutput:
+       try:
+           topic = input_data.topic
+           url = f"https://en.wikipedia.org/api/rest_v1/page/summary/{topic}"
+
+           response = self.get_request(url, json=True)
+           yield "summary", response['extract']
+
+       except requests.exceptions.HTTPError as http_err:
+           yield "error", f"HTTP error occurred: {http_err}"
+       except requests.RequestException as e:
+           yield "error", f"Request to Wikipedia failed: {e}"
+       except KeyError as e:
+           yield "error", f"Error parsing Wikipedia response: {e}"
+   ```
+
+   - **Try block**: Contains the main logic to fetch and process the Wikipedia summary.
+   - **API request**: Send a GET request to the Wikipedia API.
+   - **Error handling**: Handle various exceptions that might occur during the API request and data processing.
+   - **Yield**: Use `yield` to output the results.
+
+## Key Points to Remember
+
+- **Unique ID**: Give your block a unique ID in the **init** method.
+- **Input and Output Schemas**: Define clear input and output schemas.
+- **Error Handling**: Implement error handling in the `run` method.
+- **Output Results**: Use `yield` to output results in the `run` method.
+- **Testing**: Provide test input and output in the **init** method for automatic testing.
+
+## Understanding the Testing Process
+
+The testing of blocks is handled by `test_block.py`, which does the following:
+
+1. It calls the block with the provided `test_input`.
+2. If a `test_mock` is provided, it temporarily replaces the specified methods with the mock functions.
+3. It then asserts that the output matches the `test_output`.
+
+For the WikipediaSummaryBlock:
+
+- The test will call the block with the topic "Artificial Intelligence".
+- Instead of making a real API call, it will use the mock function, which returns `{"extract": "summary content"}`.
+- It will then check if the output key is "summary" and its value is a string.
+
+This approach allows us to test the block's logic comprehensively without relying on external services, while also accommodating non-deterministic outputs.
+
+## Tips for Effective Block Testing
+
+1. **Provide realistic test_input**: Ensure your test input covers typical use cases.
+
+2. **Define appropriate test_output**: 
+   - For deterministic outputs, use specific expected values.
+   - For non-deterministic outputs or when only the type matters, use Python types (e.g., `str`, `int`, `dict`).
+   - You can mix specific values and types, e.g., `("key1", str), ("key2", 42)`.
+
+3. **Use test_mock for network calls**: This prevents tests from failing due to network issues or API changes.
+
+4. **Consider omitting test_mock for blocks without external dependencies**: If your block doesn't make network calls or use external resources, you might not need a mock.
+
+5. **Consider edge cases**: Include tests for potential error conditions in your `run` method.
+
+6. **Update tests when changing block behavior**: If you modify your block, ensure the tests are updated accordingly.
+
+By following these steps, you can create new blocks that extend the functionality of the AutoGPT Agent Server.

docs/mkdocs.yml

@@ -5,6 +5,9 @@ docs_dir: content
 nav:
   - Home: index.md
 
+  - The AutoGPT Server 🆕: 
+    - Build your own Blocks: server/new_blocks.md
+
   - AutoGPT Agent:
       - Introduction: AutoGPT/index.md
       - Setup:
@@ -15,7 +18,6 @@ nav:
               - Options: AutoGPT/configuration/options.md
               - Search: AutoGPT/configuration/search.md
               - Voice: AutoGPT/configuration/voice.md
-              - Image Generation: AutoGPT/configuration/imagegen.md
       - Usage: AutoGPT/usage.md
       - Help us improve AutoGPT:
           - Share your debug logs with us: AutoGPT/share-your-logs.md
@@ -41,7 +43,7 @@ nav:
       - Readme: https://github.com/Significant-Gravitas/AutoGPT/blob/master/frontend/README.md
 
   - Docs: docs/index.md
-
+    
   # - Challenges:
   #     - Introduction: challenges/introduction.md
   #     - List of Challenges:

forge/conftest.py

@@ -0,0 +1,41 @@
+import uuid
+from pathlib import Path
+
+import pytest
+
+from forge.file_storage.base import FileStorage, FileStorageConfiguration
+from forge.file_storage.local import LocalFileStorage
+
+pytest_plugins = [
+    "tests.vcr",
+]
+
+
+@pytest.fixture(scope="session", autouse=True)
+def load_env_vars():
+    from dotenv import load_dotenv
+
+    load_dotenv()
+
+
+@pytest.fixture()
+def tmp_project_root(tmp_path: Path) -> Path:
+    return tmp_path
+
+
+@pytest.fixture()
+def app_data_dir(tmp_project_root: Path) -> Path:
+    dir = tmp_project_root / "data"
+    dir.mkdir(parents=True, exist_ok=True)
+    return dir
+
+
+@pytest.fixture()
+def storage(app_data_dir: Path) -> FileStorage:
+    storage = LocalFileStorage(
+        FileStorageConfiguration(
+            root=Path(f"{app_data_dir}/{str(uuid.uuid4())}"), restrict_to_root=False
+        )
+    )
+    storage.initialize()
+    return storage

forge/forge/__main__.py

@@ -27,7 +27,7 @@ d88P     888  "Y88888  "Y888 "Y88P"   "Y8888P88 888           888
                 888      "Y88P"  888     "Y88888  "Y8888
                                              888
                                         Y8b d88P
-                                         "Y88P"                v0.1.0
+                                         "Y88P"                v0.2.0
 \n"""
 
 if __name__ == "__main__":

forge/forge/agent/base.py

@@ -18,12 +18,14 @@ from typing import (
 )
 
 from colorama import Fore
-from pydantic import BaseModel, Field, validator
+from pydantic import BaseModel, Field, ValidationInfo, field_validator
+from pydantic_core import from_json, to_json
 
 from forge.agent import protocols
 from forge.agent.components import (
     AgentComponent,
     ComponentEndpointError,
+    ConfigurableComponent,
     EndpointPipelineError,
 )
 from forge.config.ai_directives import AIDirectives
@@ -70,10 +72,10 @@ class BaseAgentConfiguration(SystemConfiguration):
     `0` to stop the agent.
     """
 
-    cycles_remaining = cycle_budget
+    cycles_remaining: int = cycle_budget
     """The number of cycles remaining within the `cycle_budget`."""
 
-    cycle_count = 0
+    cycle_count: int = 0
     """The number of cycles that the agent has run since its initialization."""
 
     send_token_limit: Optional[int] = None
@@ -82,14 +84,11 @@ class BaseAgentConfiguration(SystemConfiguration):
     defaults to 75% of `llm.max_tokens`.
     """
 
-    summary_max_tlength: Optional[int] = None
-    # TODO: move to ActionHistoryConfiguration
-
-    @validator("use_functions_api")
-    def validate_openai_functions(cls, v: bool, values: dict[str, Any]):
-        if v:
-            smart_llm = values["smart_llm"]
-            fast_llm = values["fast_llm"]
+    @field_validator("use_functions_api")
+    def validate_openai_functions(cls, value: bool, info: ValidationInfo):
+        if value:
+            smart_llm = info.data["smart_llm"]
+            fast_llm = info.data["fast_llm"]
             assert all(
                 [
                     not any(s in name for s in {"-0301", "-0314"})
@@ -99,7 +98,7 @@ class BaseAgentConfiguration(SystemConfiguration):
                 f"Model {smart_llm} does not support OpenAI Functions. "
                 "Please disable OPENAI_FUNCTIONS or choose a suitable model."
             )
-        return v
+        return value
 
 
 class BaseAgentSettings(SystemSettings):
@@ -272,6 +271,28 @@ class BaseAgent(Generic[AnyProposal], metaclass=AgentMeta):
                 raise e
         return method_result
 
+    def dump_component_configs(self) -> str:
+        configs: dict[str, Any] = {}
+        for component in self.components:
+            if isinstance(component, ConfigurableComponent):
+                config_type_name = component.config.__class__.__name__
+                configs[config_type_name] = component.config
+        return to_json(configs).decode()
+
+    def load_component_configs(self, serialized_configs: str):
+        configs_dict: dict[str, dict[str, Any]] = from_json(serialized_configs)
+
+        for component in self.components:
+            if not isinstance(component, ConfigurableComponent):
+                continue
+            config_type = type(component.config)
+            config_type_name = config_type.__name__
+            if config_type_name in configs_dict:
+                # Parse the serialized data and update the existing config
+                updated_data = configs_dict[config_type_name]
+                data = {**component.config.model_dump(), **updated_data}
+                component.config = component.config.__class__(**data)
+
     def _collect_components(self):
         components = [
             getattr(self, attr)
@@ -325,7 +346,7 @@ class BaseAgent(Generic[AnyProposal], metaclass=AgentMeta):
                 copied_item = item.copy()
             elif isinstance(item, BaseModel):
                 # Deep copy for Pydantic models (deep=True to also copy nested models)
-                copied_item = item.copy(deep=True)
+                copied_item = item.model_copy(deep=True)
             else:
                 # Deep copy for other objects
                 copied_item = copy.deepcopy(item)

forge/forge/agent/components.py

@@ -1,9 +1,14 @@
 from __future__ import annotations
 
 from abc import ABC
-from typing import Callable, TypeVar
+from typing import Callable, ClassVar, Generic, Optional, TypeVar
 
-T = TypeVar("T", bound="AgentComponent")
+from pydantic import BaseModel
+
+from forge.models.config import _update_user_config_from_env, deep_update
+
+AC = TypeVar("AC", bound="AgentComponent")
+BM = TypeVar("BM", bound=BaseModel)
 
 
 class AgentComponent(ABC):
@@ -24,7 +29,7 @@ class AgentComponent(ABC):
         """Return the reason this component is disabled."""
         return self._disabled_reason
 
-    def run_after(self: T, *components: type[AgentComponent] | AgentComponent) -> T:
+    def run_after(self: AC, *components: type[AgentComponent] | AgentComponent) -> AC:
         """Set the components that this component should run after."""
         for component in components:
             t = component if isinstance(component, type) else type(component)
@@ -33,6 +38,39 @@ class AgentComponent(ABC):
         return self
 
 
+class ConfigurableComponent(ABC, Generic[BM]):
+    """A component that can be configured with a Pydantic model."""
+
+    config_class: ClassVar[type[BM]]  # type: ignore
+
+    def __init__(self, configuration: Optional[BM]):
+        self._config: Optional[BM] = None
+        if configuration is not None:
+            self.config = configuration
+
+    def __init_subclass__(cls, **kwargs):
+        super().__init_subclass__(**kwargs)
+        if getattr(cls, "config_class", None) is None:
+            raise NotImplementedError(
+                f"ConfigurableComponent subclass {cls.__name__} "
+                "must define config_class class attribute."
+            )
+
+    @property
+    def config(self) -> BM:
+        if not hasattr(self, "_config") or self._config is None:
+            self.config = self.config_class()
+        return self._config  # type: ignore
+
+    @config.setter
+    def config(self, config: BM):
+        if not hasattr(self, "_config") or self._config is None:
+            # Load configuration from environment variables
+            updated = _update_user_config_from_env(config)
+            config = self.config_class(**deep_update(config.model_dump(), updated))
+        self._config = config
+
+
 class ComponentEndpointError(Exception):
     """Error of a single protocol method on a component."""
 

forge/forge/agent/forge_agent.py

@@ -0,0 +1,232 @@
+import inspect
+import logging
+from typing import Any, Optional
+from uuid import uuid4
+
+from forge.agent.base import BaseAgent, BaseAgentSettings
+from forge.agent.protocols import (
+    AfterExecute,
+    CommandProvider,
+    DirectiveProvider,
+    MessageProvider,
+)
+from forge.agent_protocol.agent import ProtocolAgent
+from forge.agent_protocol.database.db import AgentDB
+from forge.agent_protocol.models.task import (
+    Step,
+    StepRequestBody,
+    Task,
+    TaskRequestBody,
+)
+from forge.command.command import Command
+from forge.components.system.system import SystemComponent
+from forge.config.ai_profile import AIProfile
+from forge.file_storage.base import FileStorage
+from forge.llm.prompting.schema import ChatPrompt
+from forge.llm.prompting.utils import dump_prompt
+from forge.llm.providers.schema import AssistantChatMessage, AssistantFunctionCall
+from forge.llm.providers.utils import function_specs_from_commands
+from forge.models.action import (
+    ActionErrorResult,
+    ActionProposal,
+    ActionResult,
+    ActionSuccessResult,
+)
+from forge.utils.exceptions import AgentException, AgentTerminated
+
+logger = logging.getLogger(__name__)
+
+
+class ForgeAgent(ProtocolAgent, BaseAgent):
+    """
+    The goal of the Forge is to take care of the boilerplate code,
+    so you can focus on agent design.
+
+    There is a great paper surveying the agent landscape: https://arxiv.org/abs/2308.11432
+    Which I would highly recommend reading as it will help you understand the possibilities.
+
+    ForgeAgent provides component support; https://docs.agpt.co/forge/components/introduction/
+    Using Components is a new way of building agents that is more flexible and easier to extend.
+    Components replace some agent's logic and plugins with a more modular and composable system.
+    """  # noqa: E501
+
+    def __init__(self, database: AgentDB, workspace: FileStorage):
+        """
+        The database is used to store tasks, steps and artifact metadata.
+        The workspace is used to store artifacts (files).
+        """
+
+        # An example agent information; you can modify this to suit your needs
+        state = BaseAgentSettings(
+            name="Forge Agent",
+            description="The Forge Agent is a generic agent that can solve tasks.",
+            agent_id=str(uuid4()),
+            ai_profile=AIProfile(
+                ai_name="ForgeAgent", ai_role="Generic Agent", ai_goals=["Solve tasks"]
+            ),
+            task="Solve tasks",
+        )
+
+        # ProtocolAgent adds the Agent Protocol (API) functionality
+        ProtocolAgent.__init__(self, database, workspace)
+        # BaseAgent provides the component handling functionality
+        BaseAgent.__init__(self, state)
+
+        # AGENT COMPONENTS
+        # Components provide additional functionality to the agent
+        # There are NO components added by default in the BaseAgent
+        # You can create your own components or add existing ones
+        # Built-in components:
+        #   https://docs.agpt.co/forge/components/built-in-components/
+
+        # System component provides "finish" command and adds some prompt information
+        self.system = SystemComponent()
+
+    async def create_task(self, task_request: TaskRequestBody) -> Task:
+        """
+        The agent protocol, which is the core of the Forge,
+        works by creating a task and then executing steps for that task.
+        This method is called when the agent is asked to create a task.
+
+        We are hooking into function to add a custom log message.
+        Though you can do anything you want here.
+        """
+        task = await super().create_task(task_request)
+        logger.info(
+            f"📦 Task created with ID: {task.task_id} and "
+            f"input: {task.input[:40]}{'...' if len(task.input) > 40 else ''}"
+        )
+        return task
+
+    async def execute_step(self, task_id: str, step_request: StepRequestBody) -> Step:
+        """
+        Preffered method to add agent logic is to add custom components:
+        https://docs.agpt.co/forge/components/creating-components/
+
+        Outdated tutorial on how to add custom logic:
+        https://aiedge.medium.com/autogpt-forge-e3de53cc58ec
+
+        The agent protocol, which is the core of the Forge, works by creating a task and then
+        executing steps for that task. This method is called when the agent is asked to execute
+        a step.
+
+        The task that is created contains an input string, for the benchmarks this is the task
+        the agent has been asked to solve and additional input, which is a dictionary and
+        could contain anything.
+
+        If you want to get the task use:
+
+        ```
+        task = await self.db.get_task(task_id)
+        ```
+
+        The step request body is essentially the same as the task request and contains an input
+        string, for the benchmarks this is the task the agent has been asked to solve and
+        additional input, which is a dictionary and could contain anything.
+
+        You need to implement logic that will take in this step input and output the completed step
+        as a step object. You can do everything in a single step or you can break it down into
+        multiple steps. Returning a request to continue in the step output, the user can then decide
+        if they want the agent to continue or not.
+        """  # noqa: E501
+
+        step = await self.db.create_step(
+            task_id=task_id, input=step_request, is_last=False
+        )
+
+        proposal = await self.propose_action()
+
+        output = await self.execute(proposal)
+
+        if isinstance(output, ActionSuccessResult):
+            step.output = str(output.outputs)
+        elif isinstance(output, ActionErrorResult):
+            step.output = output.reason
+
+        return step
+
+    async def propose_action(self) -> ActionProposal:
+        self.reset_trace()
+
+        # Get directives
+        directives = self.state.directives.model_copy(deep=True)
+        directives.resources += await self.run_pipeline(DirectiveProvider.get_resources)
+        directives.constraints += await self.run_pipeline(
+            DirectiveProvider.get_constraints
+        )
+        directives.best_practices += await self.run_pipeline(
+            DirectiveProvider.get_best_practices
+        )
+
+        # Get commands
+        self.commands = await self.run_pipeline(CommandProvider.get_commands)
+
+        # Get messages
+        messages = await self.run_pipeline(MessageProvider.get_messages)
+
+        prompt: ChatPrompt = ChatPrompt(
+            messages=messages, functions=function_specs_from_commands(self.commands)
+        )
+
+        logger.debug(f"Executing prompt:\n{dump_prompt(prompt)}")
+
+        # Call the LLM and parse result
+        # THIS NEEDS TO BE REPLACED WITH YOUR LLM CALL/LOGIC
+        # Have a look at autogpt/agents/agent.py for an example (complete_and_parse)
+        proposal = ActionProposal(
+            thoughts="I cannot solve the task!",
+            use_tool=AssistantFunctionCall(
+                name="finish", arguments={"reason": "Unimplemented logic"}
+            ),
+            raw_message=AssistantChatMessage(
+                content="finish(reason='Unimplemented logic')"
+            ),
+        )
+
+        self.config.cycle_count += 1
+
+        return proposal
+
+    async def execute(self, proposal: Any, user_feedback: str = "") -> ActionResult:
+        tool = proposal.use_tool
+
+        # Get commands
+        self.commands = await self.run_pipeline(CommandProvider.get_commands)
+
+        # Execute the command
+        try:
+            command: Optional[Command] = None
+            for c in reversed(self.commands):
+                if tool.name in c.names:
+                    command = c
+
+            if command is None:
+                raise AgentException(f"Command {tool.name} not found")
+
+            command_result = command(**tool.arguments)
+            if inspect.isawaitable(command_result):
+                command_result = await command_result
+
+            result = ActionSuccessResult(outputs=command_result)
+        except AgentTerminated:
+            result = ActionSuccessResult(outputs="Agent terminated or finished")
+        except AgentException as e:
+            result = ActionErrorResult.from_exception(e)
+            logger.warning(f"{tool} raised an error: {e}")
+
+        await self.run_pipeline(AfterExecute.after_execute, result)
+
+        logger.debug("\n".join(self.trace))
+
+        return result
+
+    async def do_not_execute(
+        self, denied_proposal: Any, user_feedback: str
+    ) -> ActionResult:
+        result = ActionErrorResult(reason="Action denied")
+
+        await self.run_pipeline(AfterExecute.after_execute, result)
+
+        logger.debug("\n".join(self.trace))
+
+        return result

forge/forge/agent_protocol/agent.py

@@ -28,7 +28,7 @@ from forge.file_storage.base import FileStorage
 logger = logging.getLogger(__name__)
 
 
-class Agent:
+class ProtocolAgent:
     def __init__(self, database: AgentDB, workspace: FileStorage):
         self.db = database
         self.workspace = workspace

forge/forge/agent_protocol/agent_test.py

@@ -3,25 +3,20 @@ from pathlib import Path
 import pytest
 from fastapi import UploadFile
 
-from forge.agent_protocol.database.db import AgentDB
-from forge.agent_protocol.models.task import (
-    StepRequestBody,
-    Task,
-    TaskListResponse,
-    TaskRequestBody,
-)
 from forge.file_storage.base import FileStorageConfiguration
 from forge.file_storage.local import LocalFileStorage
 
-from .agent import Agent
+from .agent import ProtocolAgent
+from .database.db import AgentDB
+from .models.task import StepRequestBody, Task, TaskListResponse, TaskRequestBody
 
 
 @pytest.fixture
-def agent(test_workspace: Path):
+def agent(tmp_project_root: Path):
     db = AgentDB("sqlite:///test.db")
-    config = FileStorageConfiguration(root=test_workspace)
+    config = FileStorageConfiguration(root=tmp_project_root)
     workspace = LocalFileStorage(config)
-    return Agent(db, workspace)
+    return ProtocolAgent(db, workspace)
 
 
 @pytest.fixture
@@ -33,7 +28,7 @@ def file_upload():
 
 
 @pytest.mark.asyncio
-async def test_create_task(agent: Agent):
+async def test_create_task(agent: ProtocolAgent):
     task_request = TaskRequestBody(
         input="test_input", additional_input={"input": "additional_test_input"}
     )
@@ -42,7 +37,7 @@ async def test_create_task(agent: Agent):
 
 
 @pytest.mark.asyncio
-async def test_list_tasks(agent: Agent):
+async def test_list_tasks(agent: ProtocolAgent):
     task_request = TaskRequestBody(
         input="test_input", additional_input={"input": "additional_test_input"}
     )
@@ -52,7 +47,7 @@ async def test_list_tasks(agent: Agent):
 
 
 @pytest.mark.asyncio
-async def test_get_task(agent: Agent):
+async def test_get_task(agent: ProtocolAgent):
     task_request = TaskRequestBody(
         input="test_input", additional_input={"input": "additional_test_input"}
     )
@@ -63,7 +58,7 @@ async def test_get_task(agent: Agent):
 
 @pytest.mark.xfail(reason="execute_step is not implemented")
 @pytest.mark.asyncio
-async def test_execute_step(agent: Agent):
+async def test_execute_step(agent: ProtocolAgent):
     task_request = TaskRequestBody(
         input="test_input", additional_input={"input": "additional_test_input"}
     )
@@ -78,7 +73,7 @@ async def test_execute_step(agent: Agent):
 
 @pytest.mark.xfail(reason="execute_step is not implemented")
 @pytest.mark.asyncio
-async def test_get_step(agent: Agent):
+async def test_get_step(agent: ProtocolAgent):
     task_request = TaskRequestBody(
         input="test_input", additional_input={"input": "additional_test_input"}
     )
@@ -92,7 +87,7 @@ async def test_get_step(agent: Agent):
 
 
 @pytest.mark.asyncio
-async def test_list_artifacts(agent: Agent):
+async def test_list_artifacts(agent: ProtocolAgent):
     tasks = await agent.list_tasks()
     assert tasks.tasks, "No tasks in test.db"
 
@@ -101,7 +96,7 @@ async def test_list_artifacts(agent: Agent):
 
 
 @pytest.mark.asyncio
-async def test_create_artifact(agent: Agent, file_upload: UploadFile):
+async def test_create_artifact(agent: ProtocolAgent, file_upload: UploadFile):
     task_request = TaskRequestBody(
         input="test_input", additional_input={"input": "additional_test_input"}
     )
@@ -116,7 +111,7 @@ async def test_create_artifact(agent: Agent, file_upload: UploadFile):
 
 
 @pytest.mark.asyncio
-async def test_create_and_get_artifact(agent: Agent, file_upload: UploadFile):
+async def test_create_and_get_artifact(agent: ProtocolAgent, file_upload: UploadFile):
     task_request = TaskRequestBody(
         input="test_input", additional_input={"input": "additional_test_input"}
     )

forge/forge/agent_protocol/api_router.py

@@ -24,7 +24,7 @@ from .models import (
 )
 
 if TYPE_CHECKING:
-    from forge.agent.agent import Agent
+    from .agent import ProtocolAgent
 
 base_router = APIRouter()
 logger = logging.getLogger(__name__)
@@ -73,7 +73,7 @@ async def create_agent_task(request: Request, task_request: TaskRequestBody) ->
                 "artifacts": [],
             }
     """
-    agent: "Agent" = request["agent"]
+    agent: "ProtocolAgent" = request["agent"]
 
     try:
         task = await agent.create_task(task_request)
@@ -124,7 +124,7 @@ async def list_agent_tasks(
                 }
             }
     """
-    agent: "Agent" = request["agent"]
+    agent: "ProtocolAgent" = request["agent"]
     try:
         tasks = await agent.list_tasks(page, page_size)
         return tasks
@@ -185,7 +185,7 @@ async def get_agent_task(request: Request, task_id: str) -> Task:
                 ]
             }
     """  # noqa: E501
-    agent: "Agent" = request["agent"]
+    agent: "ProtocolAgent" = request["agent"]
     try:
         task = await agent.get_task(task_id)
         return task
@@ -239,7 +239,7 @@ async def list_agent_task_steps(
                 }
             }
     """  # noqa: E501
-    agent: "Agent" = request["agent"]
+    agent: "ProtocolAgent" = request["agent"]
     try:
         steps = await agent.list_steps(task_id, page, page_size)
         return steps
@@ -298,7 +298,7 @@ async def execute_agent_task_step(
                 ...
             }
     """
-    agent: "Agent" = request["agent"]
+    agent: "ProtocolAgent" = request["agent"]
     try:
         # An empty step request represents a yes to continue command
         if not step_request:
@@ -337,7 +337,7 @@ async def get_agent_task_step(request: Request, task_id: str, step_id: str) -> S
                 ...
             }
     """
-    agent: "Agent" = request["agent"]
+    agent: "ProtocolAgent" = request["agent"]
     try:
         step = await agent.get_step(task_id, step_id)
         return step
@@ -388,7 +388,7 @@ async def list_agent_task_artifacts(
                 }
             }
     """  # noqa: E501
-    agent: "Agent" = request["agent"]
+    agent: "ProtocolAgent" = request["agent"]
     try:
         artifacts = await agent.list_artifacts(task_id, page, page_size)
         return artifacts
@@ -430,7 +430,7 @@ async def upload_agent_task_artifacts(
                 "file_name": "main.py"
             }
     """  # noqa: E501
-    agent: "Agent" = request["agent"]
+    agent: "ProtocolAgent" = request["agent"]
 
     if file is None:
         raise HTTPException(status_code=400, detail="File must be specified")
@@ -468,7 +468,7 @@ async def download_agent_task_artifact(
         Response:
             <file_content_of_artifact>
     """
-    agent: "Agent" = request["agent"]
+    agent: "ProtocolAgent" = request["agent"]
     try:
         return await agent.get_artifact(task_id, artifact_id)
     except Exception:

forge/forge/agent_protocol/models/artifact.py

@@ -1,38 +1,34 @@
 from datetime import datetime
 
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, ConfigDict, Field
 
 
 class Artifact(BaseModel):
     created_at: datetime = Field(
-        ...,
         description="The creation datetime of the task.",
-        example="2023-01-01T00:00:00Z",
-        json_encoders={datetime: lambda v: v.isoformat()},
+        examples=["2023-01-01T00:00:00Z"],
     )
     modified_at: datetime = Field(
-        ...,
         description="The modification datetime of the task.",
-        example="2023-01-01T00:00:00Z",
-        json_encoders={datetime: lambda v: v.isoformat()},
+        examples=["2023-01-01T00:00:00Z"],
     )
     artifact_id: str = Field(
-        ...,
         description="ID of the artifact.",
-        example="b225e278-8b4c-4f99-a696-8facf19f0e56",
+        examples=["b225e278-8b4c-4f99-a696-8facf19f0e56"],
     )
     agent_created: bool = Field(
-        ...,
         description="Whether the artifact has been created by the agent.",
-        example=False,
+        examples=[False],
     )
     relative_path: str = Field(
-        ...,
         description="Relative path of the artifact in the agents workspace.",
-        example="/my_folder/my_other_folder/",
+        examples=["/my_folder/my_other_folder/"],
     )
     file_name: str = Field(
-        ...,
         description="Filename of the artifact.",
-        example="main.py",
+        examples=["main.py"],
+    )
+
+    model_config = ConfigDict(
+        json_encoders={datetime: lambda v: v.isoformat()},
     )

forge/forge/agent_protocol/models/pagination.py

@@ -2,7 +2,7 @@ from pydantic import BaseModel, Field
 
 
 class Pagination(BaseModel):
-    total_items: int = Field(..., description="Total number of items.", example=42)
-    total_pages: int = Field(..., description="Total number of pages.", example=97)
-    current_page: int = Field(..., description="Current_page page number.", example=1)
-    page_size: int = Field(..., description="Number of items per page.", example=25)
+    total_items: int = Field(description="Total number of items.", examples=[42])
+    total_pages: int = Field(description="Total number of pages.", examples=[97])
+    current_page: int = Field(description="Current_page page number.", examples=[1])
+    page_size: int = Field(description="Number of items per page.", examples=[25])

forge/forge/agent_protocol/models/task.py

@@ -4,7 +4,7 @@ from datetime import datetime
 from enum import Enum
 from typing import Any, List, Optional
 
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, ConfigDict, Field
 
 from .artifact import Artifact
 from .pagination import Pagination
@@ -12,48 +12,48 @@ from .pagination import Pagination
 
 class TaskRequestBody(BaseModel):
     input: str = Field(
-        ...,
         min_length=1,
         description="Input prompt for the task.",
-        example="Write the words you receive to the file 'output.txt'.",
+        examples=["Write the words you receive to the file 'output.txt'."],
     )
     additional_input: dict[str, Any] = Field(default_factory=dict)
 
 
 class Task(TaskRequestBody):
     created_at: datetime = Field(
-        ...,
         description="The creation datetime of the task.",
-        example="2023-01-01T00:00:00Z",
-        json_encoders={datetime: lambda v: v.isoformat()},
+        examples=["2023-01-01T00:00:00Z"],
     )
     modified_at: datetime = Field(
-        ...,
         description="The modification datetime of the task.",
-        example="2023-01-01T00:00:00Z",
-        json_encoders={datetime: lambda v: v.isoformat()},
+        examples=["2023-01-01T00:00:00Z"],
     )
     task_id: str = Field(
-        ...,
         description="The ID of the task.",
-        example="50da533e-3904-4401-8a07-c49adf88b5eb",
+        examples=["50da533e-3904-4401-8a07-c49adf88b5eb"],
     )
     artifacts: list[Artifact] = Field(
         default_factory=list,
         description="A list of artifacts that the task has produced.",
-        example=[
+        examples=[
             "7a49f31c-f9c6-4346-a22c-e32bc5af4d8e",
             "ab7b4091-2560-4692-a4fe-d831ea3ca7d6",
         ],
     )
 
+    model_config = ConfigDict(
+        json_encoders={datetime: lambda v: v.isoformat()},
+    )
+
 
 class StepRequestBody(BaseModel):
     name: Optional[str] = Field(
-        default=None, description="The name of the task step.", example="Write to file"
+        default=None,
+        description="The name of the task step.",
+        examples=["Write to file"],
     )
     input: str = Field(
-        ..., description="Input prompt for the step.", example="Washington"
+        description="Input prompt for the step.", examples=["Washington"]
     )
     additional_input: dict[str, Any] = Field(default_factory=dict)
 
@@ -66,40 +66,44 @@ class StepStatus(Enum):
 
 class Step(StepRequestBody):
     created_at: datetime = Field(
-        ...,
         description="The creation datetime of the task.",
-        example="2023-01-01T00:00:00Z",
-        json_encoders={datetime: lambda v: v.isoformat()},
+        examples=[
+            "2023-01-01T00:00:00Z",
+        ],
     )
     modified_at: datetime = Field(
-        ...,
         description="The modification datetime of the task.",
-        example="2023-01-01T00:00:00Z",
-        json_encoders={datetime: lambda v: v.isoformat()},
+        examples=[
+            "2023-01-01T00:00:00Z",
+        ],
     )
     task_id: str = Field(
-        ...,
         description="The ID of the task this step belongs to.",
-        example="50da533e-3904-4401-8a07-c49adf88b5eb",
+        examples=[
+            "50da533e-3904-4401-8a07-c49adf88b5eb",
+        ],
     )
     step_id: str = Field(
-        ...,
         description="The ID of the task step.",
-        example="6bb1801a-fd80-45e8-899a-4dd723cc602e",
+        examples=[
+            "6bb1801a-fd80-45e8-899a-4dd723cc602e",
+        ],
     )
     name: Optional[str] = Field(
-        default=None, description="The name of the task step.", example="Write to file"
+        default=None,
+        description="The name of the task step.",
+        examples=["Write to file"],
     )
     status: StepStatus = Field(
-        ..., description="The status of the task step.", example="created"
+        description="The status of the task step.", examples=["created"]
     )
     output: Optional[str] = Field(
         default=None,
         description="Output of the task step.",
-        example=(
+        examples=[
             "I am going to use the write_to_file command and write Washington "
             "to a file called output.txt <write_to_file('output.txt', 'Washington')"
-        ),
+        ],
     )
     additional_output: Optional[dict[str, Any]] = None
     artifacts: list[Artifact] = Field(
@@ -107,7 +111,11 @@ class Step(StepRequestBody):
         description="A list of artifacts that the step has produced.",
     )
     is_last: bool = Field(
-        ..., description="Whether this is the last step in the task.", example=True
+        description="Whether this is the last step in the task.", examples=[True]
+    )
+
+    model_config = ConfigDict(
+        json_encoders={datetime: lambda v: v.isoformat()},
     )
 
 

forge/forge/app.py

@@ -0,0 +1,13 @@
+import os
+from pathlib import Path
+
+from forge.agent.forge_agent import ForgeAgent
+from forge.agent_protocol.database.db import AgentDB
+from forge.file_storage import FileStorageBackendName, get_storage
+
+database_name = os.getenv("DATABASE_STRING")
+workspace = get_storage(FileStorageBackendName.LOCAL, root_path=Path("workspace"))
+database = AgentDB(database_name, debug_enabled=False)
+agent = ForgeAgent(database=database, workspace=workspace)
+
+app = agent.get_agent_app()

forge/forge/components/README.md

@@ -30,6 +30,115 @@ class MyAgent(BaseAgent):
         self.some_component = SomeComponent(self.hello_component)
 ```
 
+## Component configuration
+
+Each component can have its own configuration defined using a regular pydantic `BaseModel`.
+To ensure the configuration is loaded from the file correctly, the component must inherit from `ConfigurableComponent[BM]` where `BM` is the configuration model it uses.
+`ConfigurableComponent` provides a `config` attribute that holds the configuration instance.
+It's possible to either set the `config` attribute directly or pass the configuration instance to the component's constructor.
+Extra configuration (i.e. for components that are not part of the agent) can be passed and will be silently ignored. Extra config won't be applied even if the component is added later.
+To see the configuration of built-in components visit [Built-in Components](./built-in-components.md).
+
+```py
+from pydantic import BaseModel
+from forge.agent.components import ConfigurableComponent
+
+class MyConfig(BaseModel):
+    some_value: str
+
+class MyComponent(AgentComponent, ConfigurableComponent[MyConfig]):
+    def __init__(self, config: MyConfig):
+        super().__init__(config)
+        # This has the same effect as above:
+        # self.config = config
+
+    def get_some_value(self) -> str:
+        # Access the configuration like a regular model
+        return self.config.some_value
+```
+
+### Sensitive information
+
+While it's possible to pass sensitive data directly in code to the configuration it's recommended to use `UserConfigurable(from_env="ENV_VAR_NAME", exclude=True)` field for sensitive data like API keys.
+The data will be loaded from the environment variable but keep in mind that value passed in code takes precedence.
+All fields, even excluded ones (`exclude=True`) will be loaded when the configuration is loaded from the file.
+Exclusion allows you to skip them during *serialization*, non excluded `SecretStr` will be serialized literally as a `"**********"` string.
+
+```py
+from pydantic import BaseModel, SecretStr
+from forge.models.config import UserConfigurable
+
+class SensitiveConfig(BaseModel):
+    api_key: SecretStr = UserConfigurable(from_env="API_KEY", exclude=True)
+```
+
+### Configuration serialization
+
+`BaseAgent` provides two methods:
+1. `dump_component_configs`: Serializes all components' configurations as json string.
+1. `load_component_configs`: Deserializes json string to configuration and applies it.
+
+### JSON configuration
+
+You can specify a JSON file (e.g. `config.json`) to use for the configuration when launching an agent.
+This file contains settings for individual [Components](../components/introduction.md) that AutoGPT uses.
+To specify the file use `--component-config-file` CLI option, for example to use `config.json`:
+
+```shell
+./autogpt.sh run --component-config-file config.json
+```
+
+!!! note
+    If you're using Docker to run AutoGPT, you need to mount or copy the configuration file to the container.
+    See [Docker Guide](../../AutoGPT/setup/docker.md) for more information.
+
+### Example JSON configuration
+
+You can copy configuration you want to change, for example to `autogpt/config.json` and modify it to your needs.
+*Most configuration has default values, it's better to set only values you want to modify.*
+You can see the available configuration fields and default values in [Build-in Components](./built-in-components.md).
+You can set sensitive variables in the `.json` file as well but it's recommended to use environment variables instead.
+
+```json
+{
+    "CodeExecutorConfiguration": {
+        "execute_local_commands": false,
+        "shell_command_control": "allowlist",
+        "shell_allowlist": ["cat", "echo"],
+        "shell_denylist": [],
+        "docker_container_name": "agent_sandbox"
+    },
+    "FileManagerConfiguration": {
+        "storage_path": "agents/AutoGPT/",
+        "workspace_path": "agents/AutoGPT/workspace"
+    },
+    "GitOperationsConfiguration": {
+        "github_username": null
+    },
+    "ActionHistoryConfiguration": {
+        "llm_name": "gpt-3.5-turbo",
+        "max_tokens": 1024,
+        "spacy_language_model": "en_core_web_sm"
+    },
+    "ImageGeneratorConfiguration": {
+        "image_provider": "dalle",
+        "huggingface_image_model": "CompVis/stable-diffusion-v1-4",
+        "sd_webui_url": "http://localhost:7860"
+    },
+    "WebSearchConfiguration": {
+        "duckduckgo_max_attempts": 3
+    },
+    "WebSeleniumConfiguration": {
+        "llm_name": "gpt-3.5-turbo",
+        "web_browser": "chrome",
+        "headless": true,
+        "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36",
+        "browse_spacy_language_model": "en_core_web_sm"
+    }
+}
+
+```
+
 ## Ordering components
 
 The execution order of components is important because some may depend on the results of the previous ones.
@@ -72,6 +181,7 @@ class MyAgent(Agent):
 ## Disabling components
 
 You can control which components are enabled by setting their `_enabled` attribute.
+Components are *enabled* by default.
 Either provide a `bool` value or a `Callable[[], bool]`, will be checked each time
 the component is about to be executed. This way you can dynamically enable or disable
 components based on some conditions.