Fix issue #4196 : '[Feature request]: Allow LiteLLM to track cost when accessing models from LiteLLM Proxy'

2026-04-29 03:00:45 -04:00 · 2024-10-03 16:58:23 +00:00
484 changed files with 9066 additions and 26488 deletions
@@ -2,5 +2,7 @@
 sudo apt update
 sudo apt install -y netcat
 sudo add-apt-repository -y ppa:deadsnakes/ppa
-sudo apt install -y python3.12
-curl -sSL https://install.python-poetry.org | python3.12 -
+sudo apt install -y python3.11
+curl -sSL https://install.python-poetry.org | python3.11 -
+# chromadb requires SQLite > 3.35 but SQLite in Python3.11.9 comes with 3.31.1
+sudo cp /opt/conda/lib/libsqlite3.so.0 /lib/x86_64-linux-gnu/libsqlite3.so.0
@@ -5,55 +5,71 @@ labels: ['bug']
 body:
  - type: markdown
    attributes:
-      value: Thank you for taking the time to fill out this bug report. Please provide as much information as possible to help us understand and address the issue effectively.
+      value: Thank you for taking the time to fill out this bug report. We greatly appreciate your effort to complete this template fully. Please provide as much information as possible to help us understand and address the issue effectively.

  - type: checkboxes
    attributes:
      label: Is there an existing issue for the same bug?
      description: Please check if an issue already exists for the bug you encountered.
      options:
+      - label: I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
+        required: true
      - label: I have checked the existing issues.
        required: true

  - type: textarea
    id: bug-description
    attributes:
-      label: Describe the bug and reproduction steps
-      description: Provide a description of the issue along with any reproduction steps.
+      label: Describe the bug
+      description: Provide a short description of the problem.
    validations:
      required: true

-  - type: dropdown
-    id: installation
+  - type: textarea
+    id: current-version
    attributes:
-      label: OpenHands Installation
-      description: How are you running OpenHands?
-      options:
-        - Docker command in README
-        - Development workflow
-      default: 0
+      label: Current OpenHands version
+      description: What version of OpenHands are you using? If you're running in docker, tell us the tag you're using (e.g. ghcr.io/all-hands-ai/openhands:0.3.1).
+      render: bash
+    validations:
+      required: true

-  - type: input
-    id: openhands-version
+  - type: textarea
+    id: config
    attributes:
-      label: OpenHands Version
-      description: What version of OpenHands are you using?
-      placeholder: ex. 0.9.8, main, etc.
+      label: Installation and Configuration
+      description: Please provide any commands you ran and any configuration (redacting API keys)
+      render: bash
+    validations:
+      required: true

-  - type: dropdown
-    id: os
+  - type: textarea
+    id: model-agent
+    attributes:
+      label: Model and Agent
+      description: What model and agent are you using? You can see these settings in the UI by clicking the settings wheel.
+      placeholder: |
+        - Model:
+        - Agent:
+
+  - type: textarea
+    id: os-version
    attributes:
      label: Operating System
-      options:
-        - MacOS
-        - Linux
-        - WSL on Windows
+      description: What Operating System are you using? Linux, Mac OS, WSL on Windows
+
+  - type: textarea
+    id: repro-steps
+    attributes:
+      label: Reproduction Steps
+      description: Please list the steps to reproduce the issue.
+      placeholder: |
+        1.
+        2.
+        3.

  - type: textarea
    id: additional-context
    attributes:
      label: Logs, Errors, Screenshots, and Additional Context
-      description: Please provide any additional information you think might help. If you want to share the chat history
-        you can click the thumbs-down (👎) button above the input field and you will get a shareable link
-        (you can also click thumbs up when things are going well of course!). LLM logs will be stored in the
-        `logs/llm/default` folder. Please add any additional context about the problem here.
+      description: If you want to share the chat history you can click the thumbs-down (👎) button above the input field and you will get a shareable link (you can also click thumbs up when things are going well of course!). LLM logs will be stored in the `logs/llm/default` folder. Please add any additional context about the problem here.
@@ -14,11 +14,6 @@ on:
    branches:
      - main

-# If triggered by a PR, it will be in the same group. However, each commit on main will be in its own unique group
-concurrency:
-  group: ${{ github.workflow }}-${{ (github.head_ref && github.ref) || github.run_id }}
-  cancel-in-progress: true
-
 jobs:
  # Build the documentation website
  build:
@@ -37,7 +32,7 @@ jobs:
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
-          python-version: '3.12'
+          python-version: '3.11'
      - name: Generate Python Docs
        run: rm -rf docs/modules/python && pip install pydoc-markdown && pydoc-markdown
      - name: Install dependencies
@@ -9,11 +9,6 @@ on:
    - main
  pull_request:

-# If triggered by a PR, it will be in the same group. However, each commit on main will be in its own unique group
-concurrency:
-  group: ${{ github.workflow }}-${{ (github.head_ref && github.ref) || github.run_id }}
-  cancel-in-progress: true
-
 jobs:
  test:
    runs-on: ubuntu-latest
@@ -41,7 +36,7 @@ jobs:
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
-          python-version: '3.12'
+          python-version: '3.11'
          cache: 'poetry'
      - name: Install Python dependencies using Poetry
        run: poetry install --without evaluation,llama-index
@@ -12,11 +12,6 @@ on:
      - 'frontend/**'
      -  '.github/workflows/fe-unit-tests.yml'

-# If triggered by a PR, it will be in the same group. However, each commit on main will be in its own unique group
-concurrency:
-  group: ${{ github.workflow }}-${{ (github.head_ref && github.ref) || github.run_id }}
-  cancel-in-progress: true
-
 jobs:
  # Run frontend unit tests
  fe-test:
@@ -1,6 +1,12 @@
-# Workflow that builds, tests and then pushes the OpenHands and runtime docker images to the ghcr.io repository
+# Workflow that builds, tests and then pushes the runtime docker images to the ghcr.io repository
 name: Build, Test and Publish RT Image

+# Only run one workflow of the same group at a time.
+# There can be at most one running and one pending job in a concurrency group at any time.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
+
 # Always run on "main"
 # Always run on tags
 # Always run on PRs
@@ -19,14 +25,8 @@ on:
        required: true
        default: ''

-# If triggered by a PR, it will be in the same group. However, each commit on main will be in its own unique group
-concurrency:
-  group: ${{ github.workflow }}-${{ (github.head_ref && github.ref) || github.run_id }}
-  cancel-in-progress: true
-
 env:
-  BASE_IMAGE_FOR_HASH_EQUIVALENCE_TEST: nikolaik/python-nodejs:python3.12-nodejs22
-  RELEVANT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
+  BASE_IMAGE_FOR_HASH_EQUIVALENCE_TEST: nikolaik/python-nodejs:python3.11-nodejs22

 jobs:
  # Builds the OpenHands Docker images
@@ -83,19 +83,12 @@ jobs:
          export REPO_OWNER=${{ github.repository_owner }}
          REPO_OWNER=$(echo $REPO_OWNER | tr '[:upper:]' '[:lower:]')
          # Run the build script in the app image
-          docker run -e SANDBOX_USER_ID=0 -v /var/run/docker.sock:/var/run/docker.sock ghcr.io/${REPO_OWNER}/openhands:${{ env.RELEVANT_SHA }} /bin/bash -c "mkdir -p containers/runtime; python3 openhands/runtime/utils/runtime_build.py --base_image ${{ env.BASE_IMAGE_FOR_HASH_EQUIVALENCE_TEST }} --build_folder containers/runtime --force_rebuild" 2>&1 | tee docker-outputs.txt
+          docker run -e SANDBOX_USER_ID=0 -v /var/run/docker.sock:/var/run/docker.sock ghcr.io/${REPO_OWNER}/openhands:${{ github.sha }} /bin/bash -c "mkdir -p containers/runtime; python3 openhands/runtime/utils/runtime_build.py --base_image ${{ env.BASE_IMAGE_FOR_HASH_EQUIVALENCE_TEST }} --build_folder containers/runtime --force_rebuild" 2>&1 | tee docker-outputs.txt
          # Get the hash from the build script
          hash_from_app_image=$(cat docker-outputs.txt | grep "Hash for docker build directory" | awk -F "): " '{print $2}' | uniq | head -n1)
          echo "hash_from_app_image=$hash_from_app_image" >> $GITHUB_OUTPUT
          echo "Hash from app image: $hash_from_app_image"
-      # This test should move when we have a test suite for the app image
-      - name: Test docker in App Image
-        run: |
-          # Lowercase the repository owner
-          export REPO_OWNER=${{ github.repository_owner }}
-          REPO_OWNER=$(echo $REPO_OWNER | tr '[:upper:]' '[:lower:]')

-          docker run -e SANDBOX_USER_ID=0 -v /var/run/docker.sock:/var/run/docker.sock ghcr.io/${REPO_OWNER}/openhands:${{ env.RELEVANT_SHA }} /bin/bash -c "docker run hello-world"

  # Builds the runtime Docker images
  ghcr_build_runtime:
@@ -107,7 +100,7 @@ jobs:
    strategy:
      matrix:
        base_image:
-          - image: 'nikolaik/python-nodejs:python3.12-nodejs22'
+          - image: 'nikolaik/python-nodejs:python3.11-nodejs22'
            tag: nikolaik
    steps:
      - name: Checkout
@@ -142,7 +135,7 @@ jobs:
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
-          python-version: '3.12'
+          python-version: '3.11'
      - name: Cache Poetry dependencies
        uses: actions/cache@v4
        with:
@@ -167,7 +160,7 @@ jobs:
        if: github.event.pull_request.head.repo.fork
        uses: docker/build-push-action@v6
        with:
-          tags: ghcr.io/all-hands-ai/runtime:${{ env.RELEVANT_SHA }}-${{ matrix.base_image.tag }}
+          tags: ghcr.io/all-hands-ai/runtime:${{ github.sha }}-${{ matrix.base_image.tag }}
          outputs: type=docker,dest=/tmp/runtime-${{ matrix.base_image.tag }}.tar
          context: containers/runtime
      - name: Upload runtime image for fork
@@ -199,7 +192,7 @@ jobs:
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
-          python-version: '3.12'
+          python-version: '3.11'
      - name: Install poetry via pipx
        run: pipx install poetry
      - name: Install Python dependencies using Poetry
@@ -278,7 +271,7 @@ jobs:
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
-          python-version: '3.12'
+          python-version: '3.11'
      - name: Install poetry via pipx
        run: pipx install poetry
      - name: Install Python dependencies using Poetry
@@ -291,7 +284,7 @@ jobs:
          # Install to be able to retry on failures for flaky tests
          poetry run pip install pytest-rerunfailures

-          image_name=ghcr.io/${{ github.repository_owner }}/runtime:${{ env.RELEVANT_SHA }}-${{ matrix.base_image }}
+          image_name=ghcr.io/${{ github.repository_owner }}/runtime:${{ github.sha }}-${{ matrix.base_image }}
          image_name=$(echo $image_name | tr '[:upper:]' '[:lower:]')

          SKIP_CONTAINER_LOGS=true \
@@ -300,7 +293,7 @@ jobs:
          SANDBOX_RUNTIME_CONTAINER_IMAGE=$image_name \
          TEST_IN_CI=true \
          RUN_AS_OPENHANDS=false \
-          poetry run pytest -n 3 -raRs --reruns 2 --reruns-delay 5 --cov=openhands --cov-report=xml -s ./tests/runtime
+          poetry run pytest -n 3 -raRs --reruns 2 --reruns-delay 5 --cov=agenthub --cov=openhands --cov-report=xml -s ./tests/runtime
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        env:
@@ -356,7 +349,7 @@ jobs:
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
-          python-version: '3.12'
+          python-version: '3.11'
      - name: Install poetry via pipx
        run: pipx install poetry
      - name: Install Python dependencies using Poetry
@@ -369,7 +362,7 @@ jobs:
          # Install to be able to retry on failures for flaky tests
          poetry run pip install pytest-rerunfailures

-          image_name=ghcr.io/${{ github.repository_owner }}/runtime:${{ env.RELEVANT_SHA }}-${{ matrix.base_image }}
+          image_name=ghcr.io/${{ github.repository_owner }}/runtime:${{ github.sha }}-${{ matrix.base_image }}
          image_name=$(echo $image_name | tr '[:upper:]' '[:lower:]')

          SKIP_CONTAINER_LOGS=true \
@@ -378,7 +371,7 @@ jobs:
          SANDBOX_RUNTIME_CONTAINER_IMAGE=$image_name \
          TEST_IN_CI=true \
          RUN_AS_OPENHANDS=true \
-          poetry run pytest -n 3 -raRs --reruns 2 --reruns-delay 5 --cov=openhands --cov-report=xml -s ./tests/runtime
+          poetry run pytest -n 3 -raRs --reruns 2 --reruns-delay 5 --cov=agenthub --cov=openhands --cov-report=xml -s ./tests/runtime
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        env:
@@ -435,14 +428,14 @@ jobs:
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
-          python-version: '3.12'
+          python-version: '3.11'
      - name: Install poetry via pipx
        run: pipx install poetry
      - name: Install Python dependencies using Poetry
        run: make install-python-dependencies
      - name: Run integration tests
        run: |
-          image_name=ghcr.io/${{ github.repository_owner }}/runtime:${{ env.RELEVANT_SHA }}-${{ matrix.base_image }}
+          image_name=ghcr.io/${{ github.repository_owner }}/runtime:${{ github.sha }}-${{ matrix.base_image }}
          image_name=$(echo $image_name | tr '[:upper:]' '[:lower:]')

          TEST_RUNTIME=eventstream \
@@ -10,11 +10,6 @@ on:
    - main
  pull_request:

-# If triggered by a PR, it will be in the same group. However, each commit on main will be in its own unique group
-concurrency:
-  group: ${{ github.workflow }}-${{ (github.head_ref && github.ref) || github.run_id }}
-  cancel-in-progress: true
-
 jobs:
  # Run lint on the frontend code
  lint-frontend:
@@ -46,9 +41,9 @@ jobs:
      - name: Set up python
        uses: actions/setup-python@v5
        with:
-          python-version: 3.12
+          python-version: 3.11
          cache: 'pip'
      - name: Install pre-commit
        run: pip install pre-commit==3.7.0
      - name: Run pre-commit hooks
-        run: pre-commit run --files openhands/**/* evaluation/**/* tests/**/* --show-diff-on-failure --config ./dev_config/python/.pre-commit-config.yaml
+        run: pre-commit run --files openhands/**/* agenthub/**/* evaluation/**/* tests/**/* --show-diff-on-failure --config ./dev_config/python/.pre-commit-config.yaml
@@ -10,11 +10,6 @@ on:
      - main
  pull_request:

-# If triggered by a PR, it will be in the same group. However, each commit on main will be in its own unique group
-concurrency:
-  group: ${{ github.workflow }}-${{ (github.head_ref && github.ref) || github.run_id }}
-  cancel-in-progress: true
-
 jobs:
  # Run python unit tests on macOS
  test-on-macos:
@@ -24,7 +19,7 @@ jobs:
      INSTALL_DOCKER: '1' # Set to '0' to skip Docker installation
    strategy:
      matrix:
-        python-version: ['3.12']
+        python-version: ['3.11']
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python ${{ matrix.python-version }}
@@ -98,7 +93,7 @@ jobs:
        id: buildx
        uses: docker/setup-buildx-action@v3
      - name: Run Tests
-        run: poetry run pytest --forked --cov=openhands --cov-report=xml ./tests/unit --ignore=tests/unit/test_memory.py
+        run: poetry run pytest --forked --cov=agenthub --cov=openhands --cov-report=xml ./tests/unit
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        env:
@@ -112,7 +107,7 @@ jobs:
      INSTALL_DOCKER: '0' # Set to '0' to skip Docker installation
    strategy:
      matrix:
-        python-version: ['3.12']
+        python-version: ['3.11']
    steps:
      - uses: actions/checkout@v4
      - name: Set up Docker Buildx
@@ -130,7 +125,7 @@ jobs:
      - name: Build Environment
        run: make build
      - name: Run Tests
-        run: poetry run pytest --forked --cov=openhands --cov-report=xml -svv ./tests/unit --ignore=tests/unit/test_memory.py
+        run: poetry run pytest --forked --cov=agenthub --cov=openhands --cov-report=xml -svv ./tests/unit
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        env:
@@ -17,7 +17,7 @@ jobs:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
-          python-version: 3.12
+          python-version: 3.11
      - name: Install Poetry
        uses: snok/install-poetry@v1.4.1
        with:
@@ -26,6 +26,6 @@ jobs:
      - name: Install Poetry Dependencies
        run: poetry install --no-interaction --no-root
      - name: Build poetry project
-        run: ./build.sh
+        run: poetry build -v
      - name: publish
        run: poetry publish -u __token__ -p ${{ secrets.PYPI_TOKEN }}
@@ -35,7 +35,7 @@ jobs:
    - name: Set up Python
      uses: actions/setup-python@v5
      with:
-        python-version: "3.12"
+        python-version: "3.11"
    - name: Cache Poetry dependencies
      uses: actions/cache@v4
      with:
@@ -55,7 +55,7 @@ jobs:
      run: |
        DEBUG=${{ inputs.debug }} \
        LOG_TO_FILE=${{ inputs.log_to_file }} \
-        FORCE_REGENERATE=${{ inputs.force_regenerate_tests }} \
+        FORCE_REGENERATE_TESTS=${{ inputs.force_regenerate_tests }} \
        FORCE_USE_LLM=${{ inputs.force_use_llm }} \
        ./tests/integration/regenerate.sh
    - name: Commit changes
@@ -21,7 +21,7 @@ jobs:
    - name: Set up Python
      uses: actions/setup-python@v5
      with:
-        python-version: '3.12'
+        python-version: '3.11'
    - name: install git, github cli
      run: |
        sudo apt-get install -y git gh
@@ -121,7 +121,6 @@ celerybeat.pid

 # Environments
 .env
-frontend/.env
 .venv
 env/
 venv/
@@ -8,17 +8,16 @@ There are many ways that you can contribute:

 1. **Download and use** OpenHands, and send [issues](https://github.com/All-Hands-AI/OpenHands/issues) when you encounter something that isn't working or a feature that you'd like to see.
 2. **Send feedback** after each session by [clicking the thumbs-up thumbs-down buttons](https://docs.all-hands.dev/modules/usage/feedback), so we can see where things are working and failing, and also build an open dataset for training code agents.
-3. **Improve the Codebase** by sending PRs (see details below). In particular, we have some [good first issues](https://github.com/All-Hands-AI/OpenHands/labels/good%20first%20issue) that may be ones to start on.
+3. **Improve the Codebase** by sending PRs (see details below). In particular, we have some [good first issue](https://github.com/All-Hands-AI/OpenHands/labels/good%20first%20issue) issues that may be ones to start on.

 ## Understanding OpenHands's CodeBase

 To understand the codebase, please refer to the README in each module:
 - [frontend](./frontend/README.md)
+- [agenthub](./agenthub/README.md)
 - [evaluation](./evaluation/README.md)
 - [openhands](./openhands/README.md)
-   - [agenthub](./openhands/agenthub/README.md)
-   - [server](./openhands/server/README.md)
-
+    - [server](./openhands/server/README.md)

 When you write code, it is also good to write tests. Please navigate to the `tests` folder to see existing test suites.
 At the moment, we have two kinds of tests: `unit` and `integration`. Please refer to the README for each test suite. These tests also run on GitHub's continuous integration to ensure quality of the project.
@@ -2,7 +2,7 @@

 ## Contributors

-We would like to thank all the [contributors](https://github.com/All-Hands-AI/OpenHands/graphs/contributors) who have helped make OpenHands possible. We greatly appreciate your dedication and hard work.
+We would like to thank all the [contributors](https://github.com/All-Hands-AI/OpenHands/graphs/contributors) who have helped make OpenHands possible. Your dedication and hard work are greatly appreciated.

 ## Open Source Projects

@@ -10,7 +10,7 @@ OpenHands includes and adapts the following open source projects. We are gratefu

 #### [SWE Agent](https://github.com/princeton-nlp/swe-agent)
   - License: MIT License
-   - Description: Adapted for use in OpenHands's agent hub
+   - Description: Adapted for use in OpenHands's agenthub

 #### [Aider](https://github.com/paul-gauthier/aider)
   - License: Apache License 2.0
@@ -7,7 +7,7 @@ Otherwise, you can clone the OpenHands project directly.
 ### 1. Requirements
 * Linux, Mac OS, or [WSL on Windows](https://learn.microsoft.com/en-us/windows/wsl/install)  [ Ubuntu <= 22.04]
 * [Docker](https://docs.docker.com/engine/install/) (For those on MacOS, make sure to allow the default Docker socket to be used from advanced settings!)
-* [Python](https://www.python.org/downloads/) = 3.12
+* [Python](https://www.python.org/downloads/) = 3.11
 * [NodeJS](https://nodejs.org/en/download/package-manager) >= 18.17.1
 * [Poetry](https://python-poetry.org/docs/#installing-with-the-official-installer) >= 1.8
 * netcat => sudo apt-get install netcat
@@ -22,8 +22,8 @@ If you want to develop without system admin/sudo access to upgrade/install `Pyth
 curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
 bash Miniforge3-$(uname)-$(uname -m).sh

-# Install Python 3.12, nodejs, and poetry
-mamba install python=3.12
+# Install Python 3.11, nodejs, and poetry
+mamba install python=3.11
 mamba install conda-forge::nodejs
 mamba install conda-forge::poetry
 ```
@@ -98,11 +98,6 @@ Please refer to [this README](./tests/integration/README.md) for details.
 1. Add your dependency in `pyproject.toml` or use `poetry add xxx`
 2. Update the poetry.lock file via `poetry lock --no-update`

-### 9. Use existing Docker image
-To reduce build time (e.g., if no changes were made to the client-runtime component), you can use an existing Docker container image. Follow these steps:
-1. Set the SANDBOX_RUNTIME_CONTAINER_IMAGE environment variable to the desired Docker image.
-2. Example: export SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/all-hands-ai/runtime:0.9-nikolaik
-
 ## Develop inside Docker container

 TL;DR
@@ -10,7 +10,7 @@ DEFAULT_WORKSPACE_DIR = "./workspace"
 DEFAULT_MODEL = "gpt-4o"
 CONFIG_FILE = config.toml
 PRE_COMMIT_CONFIG_PATH = "./dev_config/python/.pre-commit-config.yaml"
-PYTHON_VERSION = 3.12
+PYTHON_VERSION = 3.11

 # ANSI color codes
 GREEN=$(shell tput -Txterm setaf 2)
@@ -195,7 +195,7 @@ start-backend:
 # Start frontend
 start-frontend:
 	@echo "$(YELLOW)Starting frontend...$(RESET)"
-	@cd frontend && VITE_BACKEND_HOST=$(BACKEND_HOST_PORT) VITE_FRONTEND_PORT=$(FRONTEND_PORT) npm run start -- --port $(FRONTEND_PORT)
+	@cd frontend && VITE_BACKEND_HOST=$(BACKEND_HOST_PORT) VITE_FRONTEND_PORT=$(FRONTEND_PORT) npm run start

 # Common setup for running the app (non-callable)
 _run_setup:
@@ -36,7 +36,7 @@ Learn more at [docs.all-hands.dev](https://docs.all-hands.dev), or jump to the [
 The easiest way to run OpenHands is in Docker. You can change `WORKSPACE_BASE` below to
 point OpenHands to existing code that you'd like to modify.

-See the [Installation](https://docs.all-hands.dev/modules/usage/installation) guide for
+See the [Getting Started](https://docs.all-hands.dev/modules/usage/getting-started) guide for
 system requirements and more information.

 ```bash
@@ -65,7 +65,7 @@ You'll need a model provider and API key. One option that works well: [Claude 3.
 You can also run OpenHands in a scriptable [headless mode](https://docs.all-hands.dev/modules/usage/how-to/headless-mode),
 or as an [interactive CLI](https://docs.all-hands.dev/modules/usage/how-to/cli-mode).

-Visit [Installation](https://docs.all-hands.dev/modules/usage/installation) for more information and setup instructions.
+Visit [Getting Started](https://docs.all-hands.dev/modules/usage/getting-started) for more information and setup instructions.

 If you want to modify the OpenHands source code, check out [Development.md](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md).

@@ -120,8 +120,8 @@ For a list of open source projects and licenses used in OpenHands, please see ou
 ## 📚 Cite

 ```
-@misc{openhands,
-      title={{OpenHands: An Open Platform for AI Software Developers as Generalist Agents}},
+@misc{opendevin,
+      title={{OpenDevin: An Open Platform for AI Software Developers as Generalist Agents}},
      author={Xingyao Wang and Boxuan Li and Yufan Song and Frank F. Xu and Xiangru Tang and Mingchen Zhuge and Jiayi Pan and Yueqi Song and Bowen Li and Jaskirat Singh and Hoang H. Tran and Fuqiang Li and Ren Ma and Mingzhang Zheng and Bill Qian and Yanjun Shao and Niklas Muennighoff and Yizhe Zhang and Binyuan Hui and Junyang Lin and Robert Brennan and Hao Peng and Heng Ji and Graham Neubig},
      year={2024},
      eprint={2407.16741},
@@ -2,7 +2,7 @@

 In this folder, there may exist multiple implementations of `Agent` that will be used by the framework.

-For example, `openhands/agenthub/codeact_agent`, etc.
+For example, `agenthub/codeact_agent`, etc.
 Contributors from different backgrounds and interests can choose to contribute to any (or all!) of these directions.

 ## Constructing an Agent
@@ -1,13 +1,13 @@
 from dotenv import load_dotenv

-from openhands.agenthub.micro.agent import MicroAgent
-from openhands.agenthub.micro.registry import all_microagents
+from agenthub.micro.agent import MicroAgent
+from agenthub.micro.registry import all_microagents
 from openhands.controller.agent import Agent

 load_dotenv()


-from openhands.agenthub import (  # noqa: E402
+from agenthub import (  # noqa: E402
    browsing_agent,
    codeact_agent,
    codeact_swe_agent,
@@ -1,4 +1,4 @@
-from openhands.agenthub.browsing_agent.browsing_agent import BrowsingAgent
+from agenthub.browsing_agent.browsing_agent import BrowsingAgent
 from openhands.controller.agent import Agent

 Agent.register('BrowsingAgent', BrowsingAgent)
@@ -3,7 +3,7 @@ import os
 from browsergym.core.action.highlevel import HighLevelActionSet
 from browsergym.utils.obs import flatten_axtree_to_str

-from openhands.agenthub.browsing_agent.response_parser import BrowsingResponseParser
+from agenthub.browsing_agent.response_parser import BrowsingResponseParser
 from openhands.controller.agent import Agent
 from openhands.controller.state.state import State
 from openhands.core.config import AgentConfig
@@ -12,7 +12,7 @@ from browsergym.core.action.base import AbstractActionSet
 from browsergym.core.action.highlevel import HighLevelActionSet
 from browsergym.core.action.python import PythonActionSet

-from openhands.agenthub.browsing_agent.utils import (
+from agenthub.browsing_agent.utils import (
    ParseError,
    parse_html_tags_raise,
 )
@@ -0,0 +1,88 @@
+import ast
+
+from openhands.controller.action_parser import ActionParser, ResponseParser
+from openhands.core.logger import openhands_logger as logger
+from openhands.events.action import (
+    Action,
+    BrowseInteractiveAction,
+)
+
+
+class BrowsingResponseParser(ResponseParser):
+    def __init__(self):
+        # Need to pay attention to the item order in self.action_parsers
+        super().__init__()
+        self.action_parsers = [BrowsingActionParserMessage()]
+        self.default_parser = BrowsingActionParserBrowseInteractive()
+
+    def parse(self, response: str) -> Action:
+        action_str = self.parse_response(response)
+        return self.parse_action(action_str)
+
+    def parse_response(self, response) -> str:
+        action_str = response['choices'][0]['message']['content']
+        if action_str is None:
+            return ''
+        action_str = action_str.strip()
+        if action_str and not action_str.endswith('```'):
+            action_str = action_str + ')```'
+        logger.debug(action_str)
+        return action_str
+
+    def parse_action(self, action_str: str) -> Action:
+        for action_parser in self.action_parsers:
+            if action_parser.check_condition(action_str):
+                return action_parser.parse(action_str)
+        return self.default_parser.parse(action_str)
+
+
+class BrowsingActionParserMessage(ActionParser):
+    """Parser action:
+    - BrowseInteractiveAction(browser_actions) - unexpected response format, message back to user
+    """
+
+    def __init__(
+        self,
+    ):
+        pass
+
+    def check_condition(self, action_str: str) -> bool:
+        return '```' not in action_str
+
+    def parse(self, action_str: str) -> Action:
+        msg = f'send_msg_to_user("""{action_str}""")'
+        return BrowseInteractiveAction(
+            browser_actions=msg,
+            thought=action_str,
+            browsergym_send_msg_to_user=action_str,
+        )
+
+
+class BrowsingActionParserBrowseInteractive(ActionParser):
+    """Parser action:
+    - BrowseInteractiveAction(browser_actions) - handle send message to user function call in BrowserGym
+    """
+
+    def __init__(
+        self,
+    ):
+        pass
+
+    def check_condition(self, action_str: str) -> bool:
+        return True
+
+    def parse(self, action_str: str) -> Action:
+        thought = action_str.split('```')[0].strip()
+        action_str = action_str.split('```')[1].strip()
+        msg_content = ''
+        for sub_action in action_str.split('\n'):
+            if 'send_msg_to_user(' in sub_action:
+                tree = ast.parse(sub_action)
+                args = tree.body[0].value.args  # type: ignore
+                msg_content = args[0].value
+
+        return BrowseInteractiveAction(
+            browser_actions=action_str,
+            thought=thought,
+            browsergym_send_msg_to_user=msg_content,
+        )
@@ -1,4 +1,4 @@
-from openhands.agenthub.codeact_agent.codeact_agent import CodeActAgent
+from agenthub.codeact_agent.codeact_agent import CodeActAgent
 from openhands.controller.agent import Agent

 Agent.register('CodeActAgent', CodeActAgent)
@@ -158,15 +158,8 @@ class CodeActActionParserAgentDelegate(ActionParser):
        ), 'self.agent_delegate should not be None when parse is called'
        thought = action_str.replace(self.agent_delegate.group(0), '').strip()
        browse_actions = self.agent_delegate.group(1).strip()
-        thought = (
-            f'{thought}\nI should start with: {browse_actions}'
-            if thought
-            else f'I should start with: {browse_actions}'
-        )
-
-        return AgentDelegateAction(
-            agent='BrowsingAgent', thought=thought, inputs={'task': browse_actions}
-        )
+        task = f'{thought}. I should start with: {browse_actions}'
+        return AgentDelegateAction(agent='BrowsingAgent', inputs={'task': task})


 class CodeActActionParserMessage(ActionParser):
@@ -1,7 +1,7 @@
 import os
 from itertools import islice

-from openhands.agenthub.codeact_agent.action_parser import CodeActResponseParser
+from agenthub.codeact_agent.action_parser import CodeActResponseParser
 from openhands.controller.agent import Agent
 from openhands.controller.state.state import State
 from openhands.core.config import AgentConfig
@@ -43,8 +43,7 @@ Include ONLY ONE <execute_ipython>, <execute_bash>, or <execute_browse> per resp
 If the assistant is finished with the task you MUST include <finish></finish> in your response.
 IMPORTANT: Execute code using <execute_ipython>, <execute_bash>, or <execute_browse> whenever possible.
 The assistant should utilize full file paths and the `pwd` command to prevent path-related errors.
-The assistant MUST NOT apologize to the user or thank the user after running commands or editing files. It should only address the user in response to an explicit message from the user, or to ask for more information.
-The assistant MUST NOT push any changes to GitHub unless explicitly requested to do so.
+The assistant must avoid apologies and thanks in its responses.

 {% endset %}
 {# Combine all parts without newlines between them #}
@@ -1,4 +1,4 @@
-from openhands.agenthub.codeact_swe_agent.codeact_swe_agent import CodeActSWEAgent
+from agenthub.codeact_swe_agent.codeact_swe_agent import CodeActSWEAgent
 from openhands.controller.agent import Agent

 Agent.register('CodeActSWEAgent', CodeActSWEAgent)
@@ -1,12 +1,10 @@
-from openhands.agenthub.codeact_swe_agent.prompt import (
+from agenthub.codeact_swe_agent.prompt import (
    COMMAND_DOCS,
    SWE_EXAMPLE,
    SYSTEM_PREFIX,
    SYSTEM_SUFFIX,
 )
-from openhands.agenthub.codeact_swe_agent.response_parser import (
-    CodeActSWEResponseParser,
-)
+from agenthub.codeact_swe_agent.response_parser import CodeActSWEResponseParser
 from openhands.controller.agent import Agent
 from openhands.controller.state.state import State
 from openhands.core.config import AgentConfig
@@ -1,4 +1,4 @@
-from openhands.agenthub.codeact_swe_agent.action_parser import (
+from agenthub.codeact_swe_agent.action_parser import (
    CodeActSWEActionParserCmdRun,
    CodeActSWEActionParserFinish,
    CodeActSWEActionParserIPythonRunCell,
@@ -1,4 +1,4 @@
-from openhands.agenthub.delegator_agent.agent import DelegatorAgent
+from agenthub.delegator_agent.agent import DelegatorAgent
 from openhands.controller.agent import Agent

 Agent.register('DelegatorAgent', DelegatorAgent)
@@ -1,4 +1,4 @@
-from openhands.agenthub.dummy_agent.agent import DummyAgent
+from agenthub.dummy_agent.agent import DummyAgent
 from openhands.controller.agent import Agent

 Agent.register('DummyAgent', DummyAgent)
@@ -1,7 +1,7 @@
 from jinja2 import BaseLoader, Environment

-from openhands.agenthub.micro.instructions import instructions
-from openhands.agenthub.micro.registry import all_microagents
+from agenthub.micro.instructions import instructions
+from agenthub.micro.registry import all_microagents
 from openhands.controller.agent import Agent
 from openhands.controller.state.state import State
 from openhands.core.config import AgentConfig
@@ -1,4 +1,4 @@
-from openhands.agenthub.planner_agent.agent import PlannerAgent
+from agenthub.planner_agent.agent import PlannerAgent
 from openhands.controller.agent import Agent

 Agent.register('PlannerAgent', PlannerAgent)
@@ -1,5 +1,5 @@
-from openhands.agenthub.planner_agent.prompt import get_prompt_and_images
-from openhands.agenthub.planner_agent.response_parser import PlannerResponseParser
+from agenthub.planner_agent.prompt import get_prompt_and_images
+from agenthub.planner_agent.response_parser import PlannerResponseParser
 from openhands.controller.agent import Agent
 from openhands.controller.state.state import State
 from openhands.core.config import AgentConfig
@@ -1,5 +0,0 @@
-#!/bin/bash
-set -e
-
-cp pyproject.toml poetry.lock openhands
-poetry build -v
@@ -13,10 +13,6 @@
 # API key for E2B
 #e2b_api_key = ""

-# API key for Modal
-#modal_api_token_id = ""
-#modal_api_token_secret = ""
-
 # Base path for the workspace
 workspace_base = "./workspace"

@@ -32,9 +28,6 @@ workspace_base = "./workspace"
 # Enable saving and restoring the session when run from CLI
 #enable_cli_session = false

-# Path to store trajectories
-#trajectories_path="./trajectories"
-
 # File store path
 #file_store_path = "/tmp/file_store"

@@ -192,7 +185,7 @@ model = "gpt-4o-mini"
 #memory_enabled = false

 # Memory maximum threads
-#memory_max_threads = 3
+#memory_max_threads = 2

 # LLM config group to use
 #llm_config = 'your-llm-config-group'
@@ -213,7 +206,7 @@ llm_config = 'gpt3'
 #user_id = 1000

 # Container image to use for the sandbox
-#base_container_image = "nikolaik/python-nodejs:python3.12-nodejs22"
+#base_container_image = "nikolaik/python-nodejs:python3.11-nodejs22"

 # Use host network
 #use_host_network = false
@@ -28,7 +28,7 @@ COPY ./pyproject.toml ./poetry.lock ./
 RUN touch README.md
 RUN export POETRY_CACHE_DIR && poetry install --without evaluation,llama-index --no-root && rm -rf $POETRY_CACHE_DIR

-FROM python:3.12.3-slim AS openhands-app
+FROM python:3.12.3-slim AS runtime

 WORKDIR /app

@@ -46,14 +46,6 @@ RUN mkdir -p $WORKSPACE_BASE
 RUN apt-get update -y \
    && apt-get install -y curl ssh sudo

-# Install Docker - https://docs.docker.com/engine/install/debian/
-RUN apt-get install ca-certificates curl \
-    && curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
-    && chmod a+r /etc/apt/keyrings/docker.asc \
-    && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian bookworm stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null \
-    && apt-get update \
-    && apt install -y docker-ce
-
 # Default is 1000, but OSX is often 501
 RUN sed -i 's/^UID_MIN.*/UID_MIN 499/' /etc/login.defs
 # Default is 60000, but we've seen up to 200000
@@ -77,7 +69,7 @@ RUN playwright install --with-deps chromium

 COPY --chown=openhands:app --chmod=770 ./openhands ./openhands
 COPY --chown=openhands:app --chmod=777 ./openhands/runtime/plugins ./openhands/runtime/plugins
-COPY --chown=openhands:app --chmod=770 ./openhands/agenthub ./openhands/agenthub
+COPY --chown=openhands:app --chmod=770 ./agenthub ./agenthub
 COPY --chown=openhands:app ./pyproject.toml ./pyproject.toml
 COPY --chown=openhands:app ./poetry.lock ./poetry.lock
 COPY --chown=openhands:app ./README.md ./README.md
@@ -90,7 +82,7 @@ RUN python openhands/core/download.py # No-op to download assets
 # openhands:openhands -> openhands:app
 RUN find /app \! -group app -exec chgrp app {} +

-COPY --chown=openhands:app --chmod=770 --from=frontend-builder /app/build/client ./frontend/build
+COPY --chown=openhands:app --chmod=770 --from=frontend-builder /app/dist ./frontend/dist
 COPY --chown=openhands:app --chmod=770 ./containers/app/entrypoint.sh /app/entrypoint.sh

 USER root
@@ -44,10 +44,10 @@ OPENHANDS_BUILD_VERSION="dev"
 cache_tag_base="buildcache"
 cache_tag="$cache_tag_base"

-if [[ -n $RELEVANT_SHA ]]; then
-  git_hash=$(git rev-parse --short "$RELEVANT_SHA")
+if [[ -n $GITHUB_SHA ]]; then
+  git_hash=$(git rev-parse --short "$GITHUB_SHA")
  tags+=("$git_hash")
-  tags+=("$RELEVANT_SHA")
+  tags+=("$GITHUB_SHA")
 fi

 if [[ -n $GITHUB_REF_NAME ]]; then
@@ -55,18 +55,18 @@ RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | d
  && apt-get clean \
  && apt-get autoremove -y

-# Python 3.12
+# Python 3.11
 RUN add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update \
-    && apt-get install -y python3.12 python3.12-venv python3.12-dev python3-pip \
-    && ln -s /usr/bin/python3.12 /usr/bin/python
+    && apt-get install -y python3.11 python3.11-venv python3.11-dev python3-pip \
+    && ln -s /usr/bin/python3.11 /usr/bin/python

 # NodeJS >= 18.17.1
 RUN curl -fsSL https://deb.nodesource.com/setup_18.x | bash - \
    && apt-get install -y nodejs

 # Poetry >= 1.8
-RUN curl -fsSL https://install.python-poetry.org | python3.12 - \
+RUN curl -fsSL https://install.python-poetry.org | python3.11 - \
    && ln -s ~/.local/bin/poetry /usr/local/bin/poetry

 #
@@ -3,10 +3,10 @@
 This folder builds a runtime image (sandbox), which will use a dynamically generated `Dockerfile`
 that depends on the `base_image` **AND** a [Python source distribution](https://docs.python.org/3.10/distutils/sourcedist.html) that is based on the current commit of `openhands`.

-The following command will generate a `Dockerfile` file for `nikolaik/python-nodejs:python3.12-nodejs22` (the default base image), an updated `config.sh` and the runtime source distribution files/folders into `containers/runtime`:
+The following command will generate a `Dockerfile` file for `nikolaik/python-nodejs:python3.11-nodejs22` (the default base image), an updated `config.sh` and the runtime source distribution files/folders into `containers/runtime`:

 ```bash
 poetry run python3 openhands/runtime/utils/runtime_build.py \
-    --base_image nikolaik/python-nodejs:python3.12-nodejs22 \
+    --base_image nikolaik/python-nodejs:python3.11-nodejs22 \
    --build_folder containers/runtime
 ```
@@ -38,6 +38,6 @@ repos:
      - id: mypy
        additional_dependencies:
          [types-requests, types-setuptools, types-pyyaml, types-toml]
-        entry: mypy --config-file dev_config/python/mypy.ini openhands/
+        entry: mypy --config-file dev_config/python/mypy.ini openhands/ agenthub/
        always_run: true
        pass_filenames: false
@@ -1,3 +1,7 @@
+---
+sidebar_position: 8
+---
+
 # 📚 Misc

 ## ⭐️ Research Strategy
@@ -1,3 +1,7 @@
+---
+sidebar_position: 3
+---
+
 # 🧠 Main Agent and Capabilities

 ## CodeActAgent
@@ -1,3 +1,7 @@
+---
+sidebar_position: 7
+---
+
 # 🏛️ System Architecture

 <div style={{ textAlign: 'center' }}>
@@ -1,3 +1,7 @@
+---
+sidebar_position: 5
+---
+
 # ✅ Providing Feedback

 When using OpenHands, you will encounter cases where things work well, and others where they don't. We encourage you to provide feedback when you use OpenHands to help give feedback to the development team, and perhaps more importantly, create an open corpus of coding agent training examples -- Share-OpenHands!
@@ -1,111 +1,67 @@
-# Getting Started with OpenHands
+---
+sidebar_position: 2
+---

-So you've [installed OpenHands](./installation) and have
-[set up your LLM](./installation#setup). Now what?
+# Getting Started

-OpenHands can help you tackle a wide variety of engineering tasks. But the technology
-is still new, and we're a long way off from having agents that can take on large, complicated
-engineering tasks without any guidance. So it's important to get a feel for what the agent
-does well, and where it might need some help.
+## System Requirements

-## Hello World
+* Docker version 26.0.0+ or Docker Desktop 4.31.0+.
+* You must be using Linux or Mac OS.
+  * If you are on Windows, you must use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).

-The first thing you might want to try is a simple "hello world" example.
-This can be more complicated than it sounds!
+## Installation

-Try prompting the agent with:
-> Please write a bash script hello.sh that prints "hello world!"
+The easiest way to run OpenHands is in Docker. You can change `WORKSPACE_BASE` below to point OpenHands to
+existing code that you'd like to modify.

-You should see that the agent not only writes the script, it sets the correct
-permissions and runs the script to check the output.
+```bash
+export WORKSPACE_BASE=$(pwd)/workspace

-You can continue prompting the agent to refine your code. This is a great way to
-work with agents. Start simple, and iterate.
+docker pull ghcr.io/all-hands-ai/runtime:0.9-nikolaik

-> Please modify hello.sh so that it accepts a name as the first argument, but defaults to "world"
+docker run -it --pull=always \
+    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/all-hands-ai/runtime:0.9-nikolaik \
+    -e SANDBOX_USER_ID=$(id -u) \
+    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
+    -v $WORKSPACE_BASE:/opt/workspace_base \
+    -v /var/run/docker.sock:/var/run/docker.sock \
+    -p 3000:3000 \
+    --add-host host.docker.internal:host-gateway \
+    --name openhands-app-$(date +%Y%m%d%H%M%S) \
+    ghcr.io/all-hands-ai/openhands:0.9
+```

-You can also work in any language you need, though the agent might need to spend some
-time setting up its environment!
+You can also run OpenHands in a scriptable [headless mode](https://docs.all-hands.dev/modules/usage/how-to/headless-mode), as an [interactive CLI](https://docs.all-hands.dev/modules/usage/how-to/cli-mode), or using the [OpenHands GitHub Action](https://docs.all-hands.dev/modules/usage/how-to/github-action).

-> Please convert hello.sh to a Ruby script, and run it
+## Setup

-## Building From Scratch
+After running the command above, you'll find OpenHands running at [http://localhost:3000](http://localhost:3000).

-Agents do exceptionally well at "greenfield" tasks (tasks where they don't need
-any context about an existing codebase) and they can just start from scratch.
+The agent will have access to the `./workspace` folder to do its work. You can copy existing code here, or change `WORKSPACE_BASE` in the
+command to point to an existing folder.

-It's best to start with a simple task, and then iterate on it. It's also best to be
-as specific as possible about what you want, what the tech stack should be, etc.
+Upon launching OpenHands, you'll see a settings modal. You **must** select an `LLM Provider` and `LLM Model` and enter a corresponding `API Key`.
+These can be changed at any time by selecting the `Settings` button (gear icon) in the UI.

-For example, we might build a TODO app:
+If the required `LLM Model` does not exist in the list, you can toggle `Advanced Options` and manually enter it with the correct prefix
+in the `Custom Model` text box.
+The `Advanced Options` also allow you to specify a `Base URL` if required.

-> Please build a basic TODO list app in React. It should be frontend-only, and all state
-> should be kept in localStorage.
+<div style={{ display: 'flex', justifyContent: 'center', gap: '20px' }}>
+  <img src="/img/settings-screenshot.png" alt="settings-modal" width="340" />
+  <img src="/img/settings-advanced.png" alt="settings-modal" width="335" />
+</div>

-We can keep iterating on the app once the skeleton is there:
+## Versions

-> Please allow adding an optional due date to every task
+The command above pulls the most recent stable release of OpenHands. You have other options as well:
+- For a specific release, use `ghcr.io/all-hands-ai/openhands:$VERSION`, replacing $VERSION with the version number.
+- We use semver, and release major, minor, and patch tags. So `0.9` will automatically point to the latest `0.9.x` release, and `0` will point to the latest `0.x.x` release.
+- For the most up-to-date development version, you can use `ghcr.io/all-hands-ai/openhands:main`. This version is unstable and is recommended for testing or development purposes only.

-Just like with normal development, it's good to commit and push your code frequently.
-This way you can always revert back to an old state if the agent goes off track.
-You can ask the agent to commit and push for you:
+You can choose the tag that best suits your needs based on stability requirements and desired features.

-> Please commit the changes and push them to a new branch called "feature/due-dates"
+For the development workflow, see [Development.md](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md).

-
-## Adding New Code
-
-OpenHands can also do a great job adding new code to an existing code base.
-
-For example, you can ask OpenHands to add a new GitHub action to your project
-which lints your code. OpenHands may take a peek at your codebase to see what language
-it should use, but then it can just drop a new file into `./github/workflows/lint.yml`
-
-> Please add a GitHub action that lints the code in this repository
-
-Some tasks might require a bit more context. While OpenHands can use `ls` and `grep`
-to search through your codebase, providing context up front allows it to move faster,
-and more accurately. And it'll cost you fewer tokens!
-
-> Please modify ./backend/api/routes.js to add a new route that returns a list of all tasks
-
-> Please add a new React component that displays a list of Widgets to the ./frontend/components
-> directory. It should use the existing Widget component.
-
-## Refactoring
-
-OpenHands does great at refactoring existing code, especially in small chunks.
-You probably don't want to try rearchitecting your whole codebase, but breaking up
-long files and functions, renaming variables, etc. tend to work very well.
-
-> Please rename all the single-letter variables in ./app.go
-
-> Please break the function `build_and_deploy_widgets` into two functions, `build_widgets` and `deploy_widgets` in widget.php
-
-> Please break ./api/routes.js into separate files for each route
-
-## Bug Fixes
-
-OpenHands can also help you track down and fix bugs in your code. But, as any
-developer knows, bug fixing can be extremely tricky, and often OpenHands will need more context.
-It helps if you've diagnosed the bug, but want OpenHands to figure out the logic.
-
-> Currently the email field in the `/subscribe` endpoint is rejecting .io domains. Please fix this.
-
-> The `search_widgets` function in ./app.py is doing a case-sensitive search. Please make it case-insensitive.
-
-It often helps to do test-driven development when bugfixing with an agent.
-You can ask the agent to write a new test, and then iterate until it fixes the bug:
-
-> The `hello` function crashes on the empty string. Please write a test that reproduces this bug, then fix the code so it passes.
-
-## More
-
-OpenHands is capable of helping out on just about any coding task. But it takes some practice
-to get the most out of it. Remember to:
-* Keep your tasks small
-* Be as specific as possible
-* Provide as much context as possible
-* Commit and push frequently
-
-See [Prompting Best Practices](./prompting-best-practices) for more tips on how to get the most out of OpenHands.
+Are you having trouble? Check out our [Troubleshooting Guide](https://docs.all-hands.dev/modules/usage/troubleshooting).
@@ -3,7 +3,7 @@
 The sandbox is where the agent does its work. Instead of running commands directly on your computer
 (which could be dangerous), the agent runs them inside of a Docker container.

-The default OpenHands sandbox (`python-nodejs:python3.12-nodejs22`
+The default OpenHands sandbox (`python-nodejs:python3.11-nodejs22`
 from [nikolaik/python-nodejs](https://hub.docker.com/r/nikolaik/python-nodejs)) comes with some packages installed such
 as python and Node.js but your use case may need additional software installed by default.

@@ -84,7 +84,7 @@ To create an evaluation workflow for your benchmark, follow these steps:

 1. Import relevant OpenHands utilities:
   ```python
-    import openhands.agenthub
+    import agenthub
    from evaluation.utils.shared import (
        EvalMetadata,
        EvalOutput,
@@ -136,7 +136,7 @@ To create an evaluation workflow for your benchmark, follow these steps:
   ```python
   def process_instance(instance: pd.Series, metadata: EvalMetadata) -> EvalOutput:
       config = get_config(instance, metadata)
-       runtime = create_runtime(config)
+       runtime = create_runtime(config, sid=instance.instance_id)
       initialize_runtime(runtime, instance)

       instruction = get_instruction(instance, metadata)
@@ -1,51 +0,0 @@
-# GUI Mode
-
-## Introduction
-
-OpenHands provides a user-friendly Graphical User Interface (GUI) mode for interacting with the AI assistant. This mode offers an intuitive way to set up the environment, manage settings, and communicate with the AI.
-
-## Installation and Setup
-
-1. Follow the instructions in the [Installation](../installation) guide to install OpenHands.
-
-2. After running the command, access OpenHands at [http://localhost:3000](http://localhost:3000).
-
-## Interacting with the GUI
-
-### Initial Setup
-
-1. Upon first launch, you'll see a settings modal.
-2. Select an `LLM Provider` and `LLM Model` from the dropdown menus.
-3. Enter the corresponding `API Key` for your chosen provider.
-4. Click "Save" to apply the settings.
-
-### Advanced Settings
-
-1. Toggle `Advanced Options` to access additional settings.
-2. Use the `Custom Model` text box to manually enter a model if it's not in the list.
-3. Specify a `Base URL` if required by your LLM provider.
-
-### Main Interface
-
-The main interface consists of several key components:
-
-1. **Chat Window**: The central area where you can view the conversation history with the AI assistant.
-2. **Input Box**: Located at the bottom of the screen, use this to type your messages or commands to the AI.
-3. **Send Button**: Click this to send your message to the AI.
-4. **Settings Button**: A gear icon that opens the settings modal, allowing you to adjust your configuration at any time.
-5. **Workspace Panel**: Displays the files and folders in your workspace, allowing you to navigate and view files, or the agent's past commands or web browsing history.
-
-### Interacting with the AI
-
-1. Type your question, request, or task description in the input box.
-2. Click the send button or press Enter to submit your message.
-3. The AI will process your input and provide a response in the chat window.
-4. You can continue the conversation by asking follow-up questions or providing additional information.
-
-## Tips for Effective Use
-
-1. Be specific in your requests to get the most accurate and helpful responses, as described in the [prompting best practices](../prompting-best-practices).
-2. Use the workspace panel to explore your project structure.
-3. Use one of the recommended models, as described in the [LLMs section](usage/llms/llms.md).
-
-Remember, the GUI mode of OpenHands is designed to make your interaction with the AI assistant as smooth and intuitive as possible. Don't hesitate to explore its features to maximize your productivity.
@@ -1,63 +0,0 @@
-# Installation
-
-## System Requirements
-
-* Docker version 26.0.0+ or Docker Desktop 4.31.0+.
-* You must be using Linux or Mac OS.
-  * If you are on Windows, you must use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).
-
-## Start the app
-
-The easiest way to run OpenHands is in Docker. You can change `WORKSPACE_BASE` below to point OpenHands to
-existing code that you'd like to modify.
-
-```bash
-export WORKSPACE_BASE=$(pwd)/workspace
-
-docker pull ghcr.io/all-hands-ai/runtime:0.9-nikolaik
-
-docker run -it --pull=always \
-    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/all-hands-ai/runtime:0.9-nikolaik \
-    -e SANDBOX_USER_ID=$(id -u) \
-    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
-    -v $WORKSPACE_BASE:/opt/workspace_base \
-    -v /var/run/docker.sock:/var/run/docker.sock \
-    -p 3000:3000 \
-    --add-host host.docker.internal:host-gateway \
-    --name openhands-app-$(date +%Y%m%d%H%M%S) \
-    ghcr.io/all-hands-ai/openhands:0.9
-```
-
-You can also run OpenHands in a scriptable [headless mode](https://docs.all-hands.dev/modules/usage/how-to/headless-mode), as an [interactive CLI](https://docs.all-hands.dev/modules/usage/how-to/cli-mode), or using the [OpenHands GitHub Action](https://docs.all-hands.dev/modules/usage/how-to/github-action).
-
-## Setup
-
-After running the command above, you'll find OpenHands running at [http://localhost:3000](http://localhost:3000).
-
-The agent will have access to the `./workspace` folder to do its work. You can copy existing code here, or change `WORKSPACE_BASE` in the
-command to point to an existing folder.
-
-Upon launching OpenHands, you'll see a settings modal. You **must** select an `LLM Provider` and `LLM Model` and enter a corresponding `API Key`.
-These can be changed at any time by selecting the `Settings` button (gear icon) in the UI.
-
-If the required `LLM Model` does not exist in the list, you can toggle `Advanced Options` and manually enter it with the correct prefix
-in the `Custom Model` text box.
-The `Advanced Options` also allow you to specify a `Base URL` if required.
-
-<div style={{ display: 'flex', justifyContent: 'center', gap: '20px' }}>
-  <img src="/img/settings-screenshot.png" alt="settings-modal" width="340" />
-  <img src="/img/settings-advanced.png" alt="settings-modal" width="335" />
-</div>
-
-## Versions
-
-The command above pulls the most recent stable release of OpenHands. You have other options as well:
- For a specific release, use `ghcr.io/all-hands-ai/openhands:$VERSION`, replacing $VERSION with the version number.
- We use semver, and release major, minor, and patch tags. So `0.9` will automatically point to the latest `0.9.x` release, and `0` will point to the latest `0.x.x` release.
- For the most up-to-date development version, you can use `ghcr.io/all-hands-ai/openhands:main`. This version is unstable and is recommended for testing or development purposes only.
-
-You can choose the tag that best suits your needs based on stability requirements and desired features.
-
-For the development workflow, see [Development.md](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md).
-
-Are you having trouble? Check out our [Troubleshooting Guide](https://docs.all-hands.dev/modules/usage/troubleshooting).
@@ -5,7 +5,7 @@ OpenHands uses LiteLLM to make calls to Azure's chat models. You can find their
 ## Azure OpenAI Configuration

 When running OpenHands, you'll need to set the following environment variable using `-e` in the
-[docker run command](/modules/usage/installation):
+[docker run command](/modules/usage/getting-started#installation):

 ```
 LLM_API_VERSION="<api-version>"              # e.g. "2023-05-15"
@@ -37,7 +37,7 @@ OpenHands uses llama-index for embeddings. You can find their documentation on A
 ### Azure OpenAI Configuration

 When running OpenHands, set the following environment variables using `-e` in the
-[docker run command](/modules/usage/installation):
+[docker run command](/modules/usage/getting-started#installation):

 ```
 LLM_EMBEDDING_MODEL="azureopenai"
@@ -16,7 +16,7 @@ If the model is not in the list, toggle `Advanced Options`, and enter it in `Cus
 ## VertexAI - Google Cloud Platform Configs

 To use Vertex AI through Google Cloud Platform when running OpenHands, you'll need to set the following environment
-variables using `-e` in the [docker run command](/modules/usage/installation):
+variables using `-e` in the [docker run command](/modules/usage/getting-started#installation):

 ```
 GOOGLE_APPLICATION_CREDENTIALS="<json-dump-of-gcp-service-account-json>"
@@ -1,25 +1,16 @@
+---
+sidebar_position: 3
+---
+
 # 🤖 LLM Backends

 OpenHands can connect to any LLM supported by LiteLLM. However, it requires a powerful model to work.
+The following are verified by the community to work with OpenHands:

-## Model Recommendations
-
-Based on a recent evaluation of language models for coding tasks (using the SWE-bench dataset), we can provide some recommendations for model selection. The full analysis can be found in [this blog article](https://www.all-hands.dev/blog/evaluation-of-llms-as-coding-agents-on-swe-bench-at-30x-speed).
-
-When choosing a model, consider both the quality of outputs and the associated costs. Here's a summary of the findings:
-
- Claude 3.5 Sonnet is the best by a fair amount, achieving a 27% resolve rate with the default agent in OpenHands.
- GPT-4o lags behind, and o1-mini actually performed somewhat worse than GPT-4o. We went in and analyzed the results a little, and briefly it seemed like o1 was sometimes "overthinking" things, performing extra environment configuration tasks when it could just go ahead and finish the task.
- Finally, the strongest open models were Llama 3.1 405 B and deepseek-v2.5, and they performed reasonably, even besting some of the closed models.
-
-Please refer to the [full article](https://www.all-hands.dev/blog/evaluation-of-llms-as-coding-agents-on-swe-bench-at-30x-speed) for more details.
-
-Based on these findings and community feedback, the following models have been verified to work reasonably well with OpenHands:
-
- claude-3-5-sonnet (recommended)
- gpt-4 / gpt-4o
- llama-3.1-405b
- deepseek-v2.5
+* claude-3-5-sonnet (recommended)
+* gemini-1.5-pro / gemini-1.5-flash
+* gpt-4 / gpt-4o
+* llama-3.1-405b / hermes-3-llama-3.1-405b

 :::warning
 OpenHands will issue many prompts to the LLM you configure. Most of these LLMs cost money, so be sure to set spending
@@ -41,30 +32,29 @@ models driving it. However, if you do find ones that work, please add them to th
 ## LLM Configuration

 The following can be set in the OpenHands UI through the Settings:
-
- `LLM Provider`
- `LLM Model`
- `API Key`
- `Base URL` (through `Advanced Settings`)
+* `LLM Provider`
+* `LLM Model`
+* `API Key`
+* `Base URL` (through `Advanced Settings`)

 There are some settings that may be necessary for some LLMs/providers that cannot be set through the UI. Instead, these
-can be set through environment variables passed to the [docker run command](/modules/usage/installation)
+can be set through environment variables passed to the [docker run command](/modules/usage/getting-started#installation)
 using `-e`:

- `LLM_API_VERSION`
- `LLM_EMBEDDING_MODEL`
- `LLM_EMBEDDING_DEPLOYMENT_NAME`
- `LLM_DROP_PARAMS`
- `LLM_DISABLE_VISION`
- `LLM_CACHING_PROMPT`
+* `LLM_API_VERSION`
+* `LLM_EMBEDDING_MODEL`
+* `LLM_EMBEDDING_DEPLOYMENT_NAME`
+* `LLM_DROP_PARAMS`
+* `LLM_DISABLE_VISION`
+* `LLM_CACHING_PROMPT`

 We have a few guides for running OpenHands with specific model providers:

- [Azure](llms/azure-llms)
- [Google](llms/google-llms)
- [Groq](llms/groq)
- [OpenAI](llms/openai-llms)
- [OpenRouter](llms/openrouter)
+* [Azure](llms/azure-llms)
+* [Google](llms/google-llms)
+* [Groq](llms/groq)
+* [OpenAI](llms/openai-llms)
+* [OpenRouter](llms/openrouter)

 ### API retries and rate limits

@@ -72,10 +62,10 @@ LLM providers typically have rate limits, sometimes very low, and may require re

 You can customize these options as you need for the provider you're using. Check their documentation, and set the following environment variables to control the number of retries and the time between retries:

- `LLM_NUM_RETRIES` (Default of 8)
- `LLM_RETRY_MIN_WAIT` (Default of 15 seconds)
- `LLM_RETRY_MAX_WAIT` (Default of 120 seconds)
- `LLM_RETRY_MULTIPLIER` (Default of 2)
+* `LLM_NUM_RETRIES` (Default of 8)
+* `LLM_RETRY_MIN_WAIT` (Default of 15 seconds)
+* `LLM_RETRY_MAX_WAIT` (Default of 120 seconds)
+* `LLM_RETRY_MULTIPLIER` (Default of 2)

 If you are running OpenHands in development mode, you can also set these options in the `config.toml` file:

@@ -37,5 +37,3 @@ Good prompts are:
 5. Specify the programming language or framework if it's not obvious from the context.

 Remember, the more precise and informative your prompt is, the better the AI can assist you in developing or modifying the OpenHands software.
-
-See [Getting Started with OpenHands](./getting-started) for more examples of helpful prompts.
--- a/Show More
+++ b/Show More