Compare commits

...

25 Commits

Author SHA1 Message Date
Engel Nyst c8d856cfa5 tests(LLM): add concurrency and provider-mapping reinit tests; fix style
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-14 16:26:02 +00:00
Engel Nyst db5c0c687c style(tests): apply pre-commit formatting fixes
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-14 05:27:36 +00:00
Engel Nyst 82d5b0388b tests(LLM): add reinit tests (basic, capability recompute, cost flag reset)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-14 05:24:38 +00:00
Engel Nyst bf192688a5 LLM: add lock + reinit helper; recompute capabilities in centralized initializer; keep update_config alias
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-14 05:18:38 +00:00
Engel Nyst 477d9b17c0 Merge branch 'main' into llm-init to resolve conflicts
Co-authored-by: OpenHands-GPT-4.1 <openhands@all-hands.dev>
2025-08-14 03:31:27 +00:00
787627858 2f32064778 fix file_ handler to TimedRotatingFileHandler type to prevent log fil… (#10089)
Co-authored-by: liwei136 <liwei136@baidu.com>
2025-08-14 03:16:44 +00:00
Xingyao Wang 5e85986f32 docs: Update documentation to promote uv as recommended installation method (#10291)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-13 23:11:02 +00:00
Xingyao Wang 4f436922ca fix: browser title not updating when conversation title changes (#10275)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-14 05:07:59 +08:00
Xingyao Wang d256348a46 refactor(git): principled way to set git configuration for agents & re-enable git settings in UI (#10293)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-13 20:45:15 +00:00
aeft 6bdc5563cf feat: allow partial modification of CLI settings (#10240) 2025-08-13 19:26:35 +00:00
Xingyao Wang c2f46200c0 chore(lint): Apply comprehensive linting and formatting fixes (#10287)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-13 21:13:19 +02:00
Xingyao Wang e39bf80239 fix(prompt): Add explicit GitHub/GitLab/Slack push instructions to templates (#10290)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-14 02:44:06 +08:00
Rohit Malhotra 368a0248e3 Modify experiment manager defaults for nested runtimes (#10269)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-13 14:41:28 -04:00
mamoodi db9ceb380a Patch release 0.52.1 (#10284)
Co-authored-by: Hiep Le <69354317+hieptl@users.noreply.github.com>
2025-08-13 14:16:34 -04:00
Copilot c64971d0c4 Reorganize unit tests by source module into structured directory hierarchy (#10092)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: enyst <6080905+enyst@users.noreply.github.com>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2025-08-13 15:22:56 +00:00
llamantino 69fa580899 fix(misc): MCP settings and other UI improvements/fixes (#10141) 2025-08-13 10:30:38 -04:00
mamoodi e3411f743d Release 0.52.0 (#10144) 2025-08-13 09:53:20 -04:00
Hiep Le 2b65b8aff2 fix(frontend): UI breaks when user message contains codeblock that's too wide (#10276) 2025-08-13 15:14:28 +04:00
OpenHands Bot 6756427217 🤖 Auto-fix Python linting issues 2025-07-30 23:33:14 +00:00
Engel Nyst 1b74920509 Merge branch 'main' into llm-init 2025-07-31 01:31:31 +02:00
Engel Nyst c933b88f36 Merge branch 'main' into llm-init 2025-07-30 02:01:52 +02:00
Engel Nyst e0e9d3d07c Merge branch 'main' into llm-init 2025-07-26 22:50:17 +02:00
Engel Nyst 00f9ff08c7 Merge branch 'main' into llm-init 2025-07-22 00:53:46 +02:00
Engel Nyst f37f8fb723 Merge branch 'main' into llm-init 2025-07-21 01:57:07 +02:00
Engel Nyst 8628da0037 Refactor LLM class to support runtime configuration updates
- Decouple partial function creation from LLM initialization
- Extract _build_completion_function() method for reusable completion setup
- Add update_config() method for hot-swapping LLM configuration at runtime
- Add _rebuild_completion_wrapper() method to recreate retry decorator wrapper
- Preserve metrics and retry listener instances during config updates
- Handle model changes by resetting model info for re-initialization
- Support custom tokenizer updates and log completion folder creation
- Add comprehensive unit tests covering all update scenarios

This prepares the LLM class for the upcoming unified configuration system
that will enable runtime config reloading without restart.

Co-authored-by: OpenHands <openhands@all-hands.dev>
2025-07-19 15:07:52 +02:00
214 changed files with 2399 additions and 1294 deletions
+1 -1
View File
@@ -159,7 +159,7 @@ poetry run pytest ./tests/unit/test_*.py
To reduce build time (e.g., if no changes were made to the client-runtime component), you can use an existing Docker
container image by setting the SANDBOX_RUNTIME_CONTAINER_IMAGE environment variable to the desired Docker image.
Example: `export SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/all-hands-ai/runtime:0.51-nikolaik`
Example: `export SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/all-hands-ai/runtime:0.52-nikolaik`
## Develop inside Docker container
+37 -11
View File
@@ -52,37 +52,63 @@ which comes with $20 in free credits for new users.
## 💻 Running OpenHands Locally
OpenHands can also run on your local system using Docker.
See the [Running OpenHands](https://docs.all-hands.dev/usage/installation) guide for
system requirements and more information.
### Option 1: CLI Launcher (Recommended)
> [!WARNING]
> On a public network? See our [Hardened Docker Installation Guide](https://docs.all-hands.dev/usage/runtimes/docker#hardened-docker-installation)
> to secure your deployment by restricting network binding and implementing additional security measures.
The easiest way to run OpenHands locally is using the CLI launcher with [uv](https://docs.astral.sh/uv/). This provides better isolation from your current project's virtual environment and is required for OpenHands' default MCP servers.
**Install uv** (if you haven't already):
See the [uv installation guide](https://docs.astral.sh/uv/getting-started/installation/) for the latest installation instructions for your platform.
**Launch OpenHands**:
```bash
# Launch the GUI server
uvx --python 3.12 --from openhands-ai openhands serve
# Or launch the CLI
uvx --python 3.12 --from openhands-ai openhands
```
You'll find OpenHands running at [http://localhost:3000](http://localhost:3000) (for GUI mode)!
### Option 2: Docker
<details>
<summary>Click to expand Docker command</summary>
You can also run OpenHands directly with Docker:
```bash
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik
docker run -it --rm --pull=always \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik \
-e LOG_ALL_EVENTS=true \
-v /var/run/docker.sock:/var/run/docker.sock \
-v ~/.openhands:/.openhands \
-p 3000:3000 \
--add-host host.docker.internal:host-gateway \
--name openhands-app \
docker.all-hands.dev/all-hands-ai/openhands:0.51
docker.all-hands.dev/all-hands-ai/openhands:0.52
```
</details>
> **Note**: If you used OpenHands before version 0.44, you may want to run `mv ~/.openhands-state ~/.openhands` to migrate your conversation history to the new location.
You'll find OpenHands running at [http://localhost:3000](http://localhost:3000)!
> [!WARNING]
> On a public network? See our [Hardened Docker Installation Guide](https://docs.all-hands.dev/usage/runtimes/docker#hardened-docker-installation)
> to secure your deployment by restricting network binding and implementing additional security measures.
### Getting Started
When you open the application, you'll be asked to choose an LLM provider and add an API key.
[Anthropic's Claude Sonnet 4](https://www.anthropic.com/api) (`anthropic/claude-sonnet-4-20250514`)
works best, but you have [many options](https://docs.all-hands.dev/usage/llms).
See the [Running OpenHands](https://docs.all-hands.dev/usage/installation) guide for
system requirements and more information.
## 💡 Other ways to run OpenHands
> [!WARNING]
@@ -93,8 +119,8 @@ works best, but you have [many options](https://docs.all-hands.dev/usage/llms).
> [OpenHands Cloud Helm Chart](https://github.com/all-Hands-AI/OpenHands-cloud)
You can [connect OpenHands to your local filesystem](https://docs.all-hands.dev/usage/runtimes/docker#connecting-to-your-filesystem),
run OpenHands in a scriptable [headless mode](https://docs.all-hands.dev/usage/how-to/headless-mode),
interact with it via a [friendly CLI](https://docs.all-hands.dev/usage/how-to/cli-mode),
run OpenHands in a scriptable [headless mode](https://docs.all-hands.dev/usage/how-to/headless-mode),
or run it on tagged issues with [a github action](https://docs.all-hands.dev/usage/how-to/github-action).
Visit [Running OpenHands](https://docs.all-hands.dev/usage/installation) for more information and setup instructions.
+3 -3
View File
@@ -51,17 +51,17 @@ OpenHands也可以使用Docker在本地系统上运行。
```bash
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik
docker run -it --rm --pull=always \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik \
-e LOG_ALL_EVENTS=true \
-v /var/run/docker.sock:/var/run/docker.sock \
-v ~/.openhands:/.openhands \
-p 3000:3000 \
--add-host host.docker.internal:host-gateway \
--name openhands-app \
docker.all-hands.dev/all-hands-ai/openhands:0.51
docker.all-hands.dev/all-hands-ai/openhands:0.52
```
> **注意**: 如果您在0.44版本之前使用过OpenHands,您可能需要运行 `mv ~/.openhands-state ~/.openhands` 来将对话历史迁移到新位置。
+3 -3
View File
@@ -42,17 +42,17 @@ OpenHandsはDockerを利用してローカル環境でも実行できます。
> 公共ネットワークで実行していますか?[Hardened Docker Installation Guide](https://docs.all-hands.dev/usage/runtimes/docker#hardened-docker-installation)を参照して、ネットワークバインディングの制限や追加のセキュリティ対策を実施してください。
```bash
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik
docker run -it --rm --pull=always \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik \
-e LOG_ALL_EVENTS=true \
-v /var/run/docker.sock:/var/run/docker.sock \
-v ~/.openhands:/.openhands \
-p 3000:3000 \
--add-host host.docker.internal:host-gateway \
--name openhands-app \
docker.all-hands.dev/all-hands-ai/openhands:0.51
docker.all-hands.dev/all-hands-ai/openhands:0.52
```
**注**: バージョン0.44以前のOpenHandsを使用していた場合は、会話履歴を移行するために `mv ~/.openhands-state ~/.openhands` を実行してください。
+1 -2
View File
@@ -93,8 +93,7 @@ def build_vscode_extension():
def build(setup_kwargs):
"""
This function is called by Poetry during the build process.
"""This function is called by Poetry during the build process.
`setup_kwargs` is a dictionary that will be passed to `setuptools.setup()`.
"""
print('--- Running custom Poetry build script (build_vscode.py) ---')
+1 -1
View File
@@ -12,7 +12,7 @@ services:
- SANDBOX_API_HOSTNAME=host.docker.internal
- DOCKER_HOST_ADDR=host.docker.internal
#
- SANDBOX_RUNTIME_CONTAINER_IMAGE=${SANDBOX_RUNTIME_CONTAINER_IMAGE:-ghcr.io/all-hands-ai/runtime:0.51-nikolaik}
- SANDBOX_RUNTIME_CONTAINER_IMAGE=${SANDBOX_RUNTIME_CONTAINER_IMAGE:-ghcr.io/all-hands-ai/runtime:0.52-nikolaik}
- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234}
- WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace}
ports:
+1 -1
View File
@@ -7,7 +7,7 @@ services:
image: openhands:latest
container_name: openhands-app-${DATE:-}
environment:
- SANDBOX_RUNTIME_CONTAINER_IMAGE=${SANDBOX_RUNTIME_CONTAINER_IMAGE:-docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik}
- SANDBOX_RUNTIME_CONTAINER_IMAGE=${SANDBOX_RUNTIME_CONTAINER_IMAGE:-docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik}
#- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234} # enable this only if you want a specific non-root sandbox user but you will have to manually adjust permissions of ~/.openhands for this user
- WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace}
ports:
+29 -13
View File
@@ -20,27 +20,42 @@ for scripting.
### Running with Python
**Note** - OpenHands requires Python version 3.12 or higher (Python 3.14 is not currently supported) and `uvx` for the default `fetch` MCP server (more details below).
**Note** - OpenHands requires Python version 3.12 or higher (Python 3.14 is not currently supported) and `uv` for the default `fetch` MCP server (more details below).
1. Install OpenHands using pip:
```bash
pip install openhands-ai
```
#### Recommended: Using uv
Or if you prefer not to manage your own Python environment, you can use `uvx`:
We recommend using [uv](https://docs.astral.sh/uv/) for the best OpenHands experience. uv provides better isolation from your current project's virtual environment and is required for OpenHands' default MCP servers.
1. **Install uv** (if you haven't already):
See the [uv installation guide](https://docs.astral.sh/uv/getting-started/installation/) for the latest installation instructions for your platform.
2. **Launch OpenHands CLI**:
```bash
uvx --python 3.12 --from openhands-ai openhands
```
<AccordionGroup>
<Accordion title="Alternative: Traditional pip installation">
If you prefer to use pip:
```bash
# Install OpenHands
pip install openhands-ai
```
Note that you'll still need `uv` installed for the default MCP servers to work properly.
</Accordion>
<Accordion title="Create shell aliases for easy access across environments">
Add the following to your shell configuration file (`.bashrc`, `.zshrc`, etc.):
```bash
# Add OpenHands aliases
# Add OpenHands aliases (recommended)
alias openhands="uvx --python 3.12 --from openhands-ai openhands"
alias oh="uvx --python 3.12 --from openhands-ai openhands"
```
@@ -72,18 +87,19 @@ source ~/.bashrc # or source ~/.zshrc
</AccordionGroup>
2. Launch an interactive OpenHands conversation from the command line:
3. Launch an interactive OpenHands conversation from the command line:
```bash
openhands
# If using uvx (recommended)
uvx --python 3.12 --from openhands-ai openhands
```
<Note>
If you have cloned the repository, you can also run the CLI directly using Poetry:
poetry run python -m openhands.cli.main
poetry run openhands
</Note>
3. Set your model, API key, and other preferences using the UI (or alternatively environment variables, below).
4. Set your model, API key, and other preferences using the UI (or alternatively environment variables, below).
This command opens an interactive prompt where you can type tasks or commands and get responses from OpenHands.
The first time you run the CLI, it will take you through configuring the required LLM
@@ -103,7 +119,7 @@ The conversation history will be saved in `~/.openhands/sessions`.
```bash
docker run -it \
--pull=always \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik \
-e SANDBOX_USER_ID=$(id -u) \
-e SANDBOX_VOLUMES=$SANDBOX_VOLUMES \
-e LLM_API_KEY=$LLM_API_KEY \
@@ -112,7 +128,7 @@ docker run -it \
-v ~/.openhands:/.openhands \
--add-host host.docker.internal:host-gateway \
--name openhands-app-$(date +%Y%m%d%H%M%S) \
docker.all-hands.dev/all-hands-ai/openhands:0.51 \
docker.all-hands.dev/all-hands-ai/openhands:0.52 \
python -m openhands.cli.main --override-cli-mode true
```
+2 -2
View File
@@ -61,7 +61,7 @@ export GITHUB_TOKEN="your-token" # Required for repository operations
# Run OpenHands
docker run -it \
--pull=always \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik \
-e SANDBOX_USER_ID=$(id -u) \
-e SANDBOX_VOLUMES=$SANDBOX_VOLUMES \
-e LLM_API_KEY=$LLM_API_KEY \
@@ -73,7 +73,7 @@ docker run -it \
-v ~/.openhands:/.openhands \
--add-host host.docker.internal:host-gateway \
--name openhands-app-$(date +%Y%m%d%H%M%S) \
docker.all-hands.dev/all-hands-ai/openhands:0.51 \
docker.all-hands.dev/all-hands-ai/openhands:0.52 \
python -m openhands.core.main -t "write a bash script that prints hi"
```
+4 -4
View File
@@ -68,23 +68,23 @@ Download and install the LM Studio desktop app from [lmstudio.ai](https://lmstud
1. Check [the installation guide](/usage/local-setup) and ensure all prerequisites are met before running OpenHands, then run:
```bash
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik
docker run -it --rm --pull=always \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik \
-e LOG_ALL_EVENTS=true \
-v /var/run/docker.sock:/var/run/docker.sock \
-v ~/.openhands:/.openhands \
-p 3000:3000 \
--add-host host.docker.internal:host-gateway \
--name openhands-app \
docker.all-hands.dev/all-hands-ai/openhands:0.51
docker.all-hands.dev/all-hands-ai/openhands:0.52
```
2. Wait until the server is running (see log below):
```
Digest: sha256:e72f9baecb458aedb9afc2cd5bc935118d1868719e55d50da73190d3a85c674f
Status: Image is up to date for docker.all-hands.dev/all-hands-ai/openhands:0.51
Status: Image is up to date for docker.all-hands.dev/all-hands-ai/openhands:0.52
Starting OpenHands...
Running OpenHands as root
14:22:13 - openhands:INFO: server_config.py:50 - Using config class None
+33 -13
View File
@@ -66,9 +66,31 @@ A system with a modern processor and a minimum of **4GB RAM** is recommended to
### Start the App
#### Option 1: Using the CLI Launcher (Recommended)
#### Option 1: Using the CLI Launcher with uv (Recommended)
If you have Python 3.12+ installed, you can use the CLI launcher for a simpler experience:
We recommend using [uv](https://docs.astral.sh/uv/) for the best OpenHands experience. uv provides better isolation from your current project's virtual environment and is required for OpenHands' default MCP servers (like the [fetch MCP server](https://github.com/modelcontextprotocol/servers/tree/main/src/fetch)).
**Install uv** (if you haven't already):
See the [uv installation guide](https://docs.astral.sh/uv/getting-started/installation/) for the latest installation instructions for your platform.
**Launch OpenHands**:
```bash
# Launch the GUI server
uvx --python 3.12 --from openhands-ai openhands serve
# Or with GPU support (requires nvidia-docker)
uvx --python 3.12 --from openhands-ai openhands serve --gpu
# Or with current directory mounted
uvx --python 3.12 --from openhands-ai openhands serve --mount-cwd
```
This will automatically handle Docker requirements checking, image pulling, and launching the GUI server. The `--gpu` flag enables GPU support via nvidia-docker, and `--mount-cwd` mounts your current directory into the container.
<Accordion title="Alternative: Traditional pip installation">
If you prefer to use pip and have Python 3.12+ installed:
```bash
# Install OpenHands
@@ -76,34 +98,32 @@ pip install openhands-ai
# Launch the GUI server
openhands serve
# Or with GPU support (requires nvidia-docker)
openhands serve --gpu
# Or with current directory mounted
openhands serve --mount-cwd
```
Or using `uvx --python 3.12 --from openhands-ai openhands serve` if you have [uv](https://docs.astral.sh/uv/) installed.
Note that you'll still need `uv` installed for the default MCP servers to work properly.
This will automatically handle Docker requirements checking, image pulling, and launching the GUI server. The `--gpu` flag enables GPU support via nvidia-docker, and `--mount-cwd` mounts your current directory into the container.
</Accordion>
#### Option 2: Using Docker Directly
<Accordion title="Docker Command (Click to expand)">
```bash
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik
docker pull docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik
docker run -it --rm --pull=always \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.51-nikolaik \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.52-nikolaik \
-e LOG_ALL_EVENTS=true \
-v /var/run/docker.sock:/var/run/docker.sock \
-v ~/.openhands:/.openhands \
-p 3000:3000 \
--add-host host.docker.internal:host-gateway \
--name openhands-app \
docker.all-hands.dev/all-hands-ai/openhands:0.51
docker.all-hands.dev/all-hands-ai/openhands:0.52
```
</Accordion>
> **Note**: If you used OpenHands before version 0.44, you may want to run `mv ~/.openhands-state ~/.openhands` to migrate your conversation history to the new location.
You'll find OpenHands running at http://localhost:3000!
@@ -506,7 +506,6 @@ def commit0_setup(dataset: pd.DataFrame, repo_split: str) -> pd.DataFrame:
Returns:
Filtered dataset based on split type
"""
filtered_dataset = pd.concat(
[
dataset[dataset['repo'].str.split('/').str[1] == repo]
@@ -89,8 +89,7 @@ def get_config(
def get_dv_query_for_real(
datasets, question, domain_knowledge=None, workflow_tags=None
):
"""
Prepare a structured query for the agent to execute on the specified datasets.
"""Prepare a structured query for the agent to execute on the specified datasets.
This function constructs a query by compiling metadata from the provided datasets, along with any relevant domain knowledge and workflow tags.
@@ -104,7 +103,6 @@ def get_dv_query_for_real(
query_to_dv: Query to be run on the dataset
dataset_meta: Metadata of the dataset
"""
dataset_meta = ''
for dataset_metadata in datasets:
dataset_meta += 'Dataset name: ' + dataset_metadata['name']
@@ -140,8 +138,7 @@ def get_dv_query_for_real(
def initialize_runtime(runtime: Runtime, data_files: list[str]):
"""
Initialize the runtime for the agent.
"""Initialize the runtime for the agent.
This function is called before the runtime is used to run the agent.
"""
@@ -231,8 +228,7 @@ def process_instance(
metadata: EvalMetadata,
reset_logger: bool = True,
):
"""
Process and evaluate a single instance of the dataset.
"""Process and evaluate a single instance of the dataset.
This function executes the OpenHands agent
for a specific instance of the dataset. It retrieves
@@ -247,7 +243,6 @@ def process_instance(
Returns:
output: EvalOutput object
"""
config = get_config(metadata)
# Setup the logger properly, so you can run
@@ -356,8 +351,7 @@ def list_csv_files(list_of_datasets):
def create_dataset(repo_location: str, split: str = 'test'):
"""
Create a dataset from the discoverybench repository
"""Create a dataset from the discoverybench repository
by walking through the repository and extracting metadata
from the metadata_{}.json files
@@ -368,7 +362,6 @@ def create_dataset(repo_location: str, split: str = 'test'):
Returns:
df: DataFrame containing the dataset instances
"""
data_dict = {}
data_location = os.path.join(repo_location, 'discoverybench', 'real', split)
@@ -105,8 +105,7 @@ def process_instance(
log_dir: str | None = None,
runtime_failure_count: int = 0,
) -> EvalOutput:
"""
Evaluate agent performance on a SWE-bench problem instance.
"""Evaluate agent performance on a SWE-bench problem instance.
Note that this signature differs from the expected input to `run_evaluation`. Use
`functools.partial` to provide optional arguments before passing to the evaluation harness.
@@ -1,11 +1,8 @@
"""
Utilities for handling binary files and patch generation in SWE-bench evaluation.
"""
"""Utilities for handling binary files and patch generation in SWE-bench evaluation."""
def remove_binary_diffs(patch_text):
"""
Remove binary file diffs from a git patch.
"""Remove binary file diffs from a git patch.
Args:
patch_text (str): The git patch text
@@ -36,8 +33,7 @@ def remove_binary_diffs(patch_text):
def remove_binary_files_from_git():
"""
Generate a bash command to remove binary files from git staging.
"""Generate a bash command to remove binary files from git staging.
Returns:
str: A bash command that removes binary files from git staging
@@ -111,8 +111,7 @@ def process_instance(
runtime_failure_count: int = 0,
conditional_imports: ConditionalImports | None = None,
) -> EvalOutput:
"""
Evaluate agent performance on a SWE-bench problem instance.
"""Evaluate agent performance on a SWE-bench problem instance.
Note that this signature differs from the expected input to `run_evaluation`. Use
`functools.partial` to provide optional arguments before passing to the evaluation harness.
@@ -16,8 +16,7 @@ from openhands.core.logger import openhands_logger as logger
class LocEvaluator:
def __init__(self, args):
"""
Localization evaluation.
"""Localization evaluation.
Args:
args: all main arguments
@@ -76,8 +75,7 @@ class LocEvaluator:
self.task_resolved = False
def _init_dir(self, directory_path):
"""
Check if a directory exists and create it if it doesn't.
"""Check if a directory exists and create it if it doesn't.
Args:
directory_path (str): Path to the directory to check/create
@@ -207,8 +205,7 @@ class LocEvaluator:
self._compute_avg_over_all()
def _write_to_json(self, data, file_name):
"""
Writes the current object data to a JSON file.
"""Writes the current object data to a JSON file.
Returns:
bool: True if writing was successful, False otherwise.
@@ -225,8 +222,7 @@ class LocEvaluator:
return False
def read_from_json(self, file_path):
"""
Reads data from a JSON file and loads it into the current object.
"""Reads data from a JSON file and loads it into the current object.
Returns:
dict: The loaded JSON data, or an empty dict if the file doesn't exist
@@ -253,8 +249,7 @@ class LocEvaluator:
return {}
def read_from_jsonl(self, file_path):
"""
Reads data from a JSON file and loads it into the current object.
"""Reads data from a JSON file and loads it into the current object.
Returns:
dict: The loaded JSON data, or an empty dict if the file doesn't exist
@@ -294,8 +289,7 @@ class LocEvaluator:
history_idx += 1
def _parse_string_to_dict(self, dict_string) -> dict:
"""
Convert a string representation of a dictionary to an actual dictionary.
"""Convert a string representation of a dictionary to an actual dictionary.
Args:
dict_string (str): String representation of a dictionary
@@ -328,8 +322,7 @@ class LocEvaluator:
return None
def _parse_value_from_args(self, argument_str: str, key: str) -> str:
"""
Parse a specific key's value from argument string.
"""Parse a specific key's value from argument string.
Args:
argument_str (str): The argument string containing key-value pairs
@@ -407,8 +400,7 @@ class LocEvaluator:
return ''
def _parse_path_from_args(self, argument_str: str) -> str:
"""
Parse path from argument string.
"""Parse path from argument string.
Args:
argument_str (str): The argument string containing path information
@@ -419,8 +411,7 @@ class LocEvaluator:
return self._parse_value_from_args(argument_str, 'path')
def _parse_func_names_from_str(self, code_patch) -> list:
"""
Parse function names from the new_str code patch.
"""Parse function names from the new_str code patch.
Args:
code_patch: Either a string (argument string) or already extracted new_str code
@@ -801,8 +792,7 @@ class LocEvaluator:
def swe_data_loader(args):
"""
Loading SWE-Bench data.
"""Loading SWE-Bench data.
Args:
args: Main arguments.
@@ -834,8 +824,7 @@ def swe_data_loader(args):
def infer_data_loader(args):
"""
Load instance IDs.
"""Load instance IDs.
Args:
args: Main arguments.
@@ -868,8 +857,7 @@ def infer_data_loader(args):
def infer_cost_calculator(args):
"""
Calculate total and average costs from metric JSON files with detailed output.
"""Calculate total and average costs from metric JSON files with detailed output.
Args:
args: Main arguments.
@@ -28,8 +28,7 @@ class LocalizationInfo:
hunks_per_file: dict[str, int] # File -> number of hunks
def to_dict(self) -> dict[str, Any]:
"""
Convert LocalizationInfo to a dictionary for JSON serialization.
"""Convert LocalizationInfo to a dictionary for JSON serialization.
Returns:
Dictionary representation of the localization information
@@ -58,8 +57,7 @@ class LocalizationInfo:
@classmethod
def from_dict(cls, data: dict[str, Any]) -> 'LocalizationInfo':
"""
Create LocalizationInfo from a dictionary (for loading from JSON).
"""Create LocalizationInfo from a dictionary (for loading from JSON).
Args:
data: Dictionary containing localization information
@@ -91,8 +89,7 @@ class LocalizationInfo:
class LocMeta:
"""
SWE-Bench dataset loader and ground-truth localization parser.
"""SWE-Bench dataset loader and ground-truth localization parser.
This class handles loading SWE-Bench datasets and extracting ground-truth
localization information from patches for code localization evaluation.
@@ -104,8 +101,7 @@ class LocMeta:
dataset_name: str = 'princeton-nlp/SWE-bench_Verified',
split: str = 'test',
):
"""
Initialize LocMeta with a SWE-Bench dataset.
"""Initialize LocMeta with a SWE-Bench dataset.
Args:
dataset_name: HuggingFace dataset name (e.g., "princeton-nlp/SWE-bench_Verified")
@@ -124,8 +120,7 @@ class LocMeta:
self._init_swe_dataset()
def _init_swe_dataset(self) -> None:
"""
Load and initialize the SWE-Bench dataset from HuggingFace.
"""Load and initialize the SWE-Bench dataset from HuggingFace.
Converts to pandas DataFrame for easy manipulation.
"""
try:
@@ -150,8 +145,7 @@ class LocMeta:
raise
def get_instance_by_id(self, instance_id: str) -> pd.Series:
"""
Retrieve a specific instance by its ID.
"""Retrieve a specific instance by its ID.
Args:
instance_id: The instance identifier
@@ -169,8 +163,7 @@ class LocMeta:
return self.df.iloc[idx]
def parse_instance_loc(self, instance: Union[pd.Series, str]) -> LocalizationInfo:
"""
Parse ground-truth localization information from a SWE-Bench instance.
"""Parse ground-truth localization information from a SWE-Bench instance.
Args:
instance: Either a pandas Series with instance data or an instance_id string
@@ -218,8 +211,7 @@ class LocMeta:
def _parse_file_patch_lines(
self, file_patch: str
) -> tuple[list[tuple[int, int]], int, int]:
"""
Parse line ranges and count changes from a single file patch.
"""Parse line ranges and count changes from a single file patch.
Args:
file_patch: Patch content for a single file
@@ -253,8 +245,7 @@ class LocMeta:
def _parse_code_structures_from_patch(
self, file_patch: str, file_path: str
) -> tuple[list[str], list[str]]:
"""
Extract function and class names from patch context (fallback method).
"""Extract function and class names from patch context (fallback method).
Args:
file_patch: Patch content for a single file
@@ -311,8 +302,7 @@ class LocMeta:
def _parse_patch_localization(
self, patch_content: str, instance_id: str
) -> LocalizationInfo:
"""
Parse localization information from a git patch (improved method).
"""Parse localization information from a git patch (improved method).
Args:
patch_content: The git patch content
@@ -390,8 +380,7 @@ class LocMeta:
def _extract_code_structures_from_patch(
self, file_patch: str, file_path: str
) -> tuple[list[str], list[str]]:
"""
Extract function and class names from patch context and content.
"""Extract function and class names from patch context and content.
Args:
file_patch: Patch content for a single file
@@ -519,8 +508,7 @@ class LocMeta:
def _parse_patch_localization_with_runtime(
self, patch_content: str, instance_id: str, runtime: Runtime
) -> LocalizationInfo:
"""
Parse localization information from a git patch using OpenHands runtime.
"""Parse localization information from a git patch using OpenHands runtime.
This is the superior method when runtime is available.
Args:
@@ -596,8 +584,7 @@ class LocMeta:
def parse_instance_loc_with_runtime(
self, instance: Union[pd.Series, str], runtime: Runtime = None
) -> LocalizationInfo:
"""
Parse ground-truth localization information using OpenHands runtime.
"""Parse ground-truth localization information using OpenHands runtime.
Args:
instance: Either a pandas Series with instance data or an instance_id string
@@ -634,8 +621,7 @@ class LocMeta:
def _analyze_source_code_with_runtime(
self, runtime: Runtime, file_path: str, affected_lines: list[int]
) -> tuple[list[str], list[str], dict[int, str], dict[int, str]]:
"""
Analyze source code using OpenHands runtime to find functions and classes.
"""Analyze source code using OpenHands runtime to find functions and classes.
Args:
runtime: OpenHands runtime object
@@ -695,8 +681,7 @@ class LocMeta:
def _parse_cython_content_with_line_mapping(
self, content: str, affected_lines: list[int]
) -> tuple[list[str], list[str], dict[int, str], dict[int, str]]:
"""
Parse Cython content to extract functions and classes with line mapping.
"""Parse Cython content to extract functions and classes with line mapping.
Since Cython files can't be parsed with Python's AST, we use regex-based parsing.
Args:
@@ -828,8 +813,7 @@ class LocMeta:
def _parse_python_content_with_line_mapping(
self, content: str, affected_lines: list[int]
) -> tuple[list[str], list[str], dict[int, str], dict[int, str]]:
"""
Parse Python content to extract functions and classes with accurate line mapping.
"""Parse Python content to extract functions and classes with accurate line mapping.
Args:
content: Python source code content
@@ -914,8 +898,7 @@ class LocMeta:
def _parse_python_content(
self, content: str, affected_lines: list[int]
) -> tuple[list[str], list[str], dict[int, str], dict[int, str]]:
"""
Parse Python content to extract functions and classes.
"""Parse Python content to extract functions and classes.
Args:
content: Python source code content
@@ -989,8 +972,7 @@ class LocMeta:
return [], [], {}, {}
def _split_patch_by_files(self, patch_content: str) -> dict[str, str]:
"""
Split a multi-file patch into individual file patches.
"""Split a multi-file patch into individual file patches.
Args:
patch_content: Complete patch content
@@ -1049,8 +1031,7 @@ class LocMeta:
def _empty_localization_info(
self, instance_id: str = 'unknown'
) -> LocalizationInfo:
"""
Return an empty LocalizationInfo object.
"""Return an empty LocalizationInfo object.
Args:
instance_id: Instance identifier
@@ -1072,8 +1053,7 @@ class LocMeta:
)
def get_dataset_statistics(self) -> dict[str, Any]:
"""
Get statistics about the loaded dataset.
"""Get statistics about the loaded dataset.
Returns:
Dictionary containing dataset statistics
@@ -1095,8 +1075,7 @@ class LocMeta:
return stats
def get_instances_by_repo(self, repo_name: str) -> pd.DataFrame:
"""
Get all instances for a specific repository.
"""Get all instances for a specific repository.
Args:
repo_name: Repository name (e.g., "django/django")
@@ -6,8 +6,7 @@ from openhands.core.logger import openhands_logger as logger
def verify_instance_costs(row: pd.Series) -> float:
"""
Verifies that the accumulated_cost matches the sum of individual costs in metrics.
"""Verifies that the accumulated_cost matches the sum of individual costs in metrics.
Also checks for duplicate consecutive costs which might indicate buggy counting.
If the consecutive costs are identical, the file is affected by this bug:
https://github.com/All-Hands-AI/OpenHands/issues/5383
@@ -181,9 +181,7 @@ def distinct_methods_stats(tree, num_lines):
def loops_stats(tree, num_lines):
"""
Calculate the average number of loops.
"""
"""Calculate the average number of loops."""
total_loops = 0
def traverse(node):
@@ -199,9 +197,7 @@ def loops_stats(tree, num_lines):
def branches_stats(tree, num_lines):
"""
Calculate the average number of branches (conditional statements).
"""
"""Calculate the average number of branches (conditional statements)."""
total_branches = 0
def traverse(node):
@@ -192,8 +192,7 @@ def run_mutation_testing(
def grade_test_output(
test_suite: str, instance: pd.Series, test_output: str, test_spec: TestSpec, runtime
):
"""
Two-pass test grading with short-circuiting:
"""Two-pass test grading with short-circuiting:
1. Run all tests to identify passing/failing tests
2. If no failing tests, evaluate coverage immediately
3. Otherwise, run only passing tests for coverage analysis
@@ -280,8 +279,7 @@ def process_instance(
reset_logger: bool = True,
log_dir: str | None = None,
) -> EvalOutput:
"""
Evaluate agent performance on a TestGenEval problem instance.
"""Evaluate agent performance on a TestGenEval problem instance.
Note that this signature differs from the expected input to `run_evaluation`. Use
`functools.partial` to provide optional arguments before passing to the evaluation harness.
@@ -453,8 +451,7 @@ def process_instance(
def count_and_log_fields(evaluated_predictions, fields, key):
"""
Count and log the sum of specified fields in the evaluated predictions,
"""Count and log the sum of specified fields in the evaluated predictions,
ignoring fields with a value of -1. If all values for a field are -1,
return -1.
@@ -4,8 +4,7 @@ from evaluation.benchmarks.testgeneval.constants import TestStatus
def parse_log_pytest(log: str) -> dict[str, str]:
"""
Parser for test logs generated with PyTest framework
"""Parser for test logs generated with PyTest framework
Args:
log (str): log content
@@ -26,8 +25,7 @@ def parse_log_pytest(log: str) -> dict[str, str]:
def parse_log_pytest_options(log: str) -> dict[str, str]:
"""
Parser for test logs generated with PyTest framework with options
"""Parser for test logs generated with PyTest framework with options
Args:
log (str): log content
@@ -61,8 +59,7 @@ def parse_log_pytest_options(log: str) -> dict[str, str]:
def parse_log_django(log: str) -> dict[str, str]:
"""
Parser for test logs generated with Django tester framework
"""Parser for test logs generated with Django tester framework
Args:
log (str): log content
@@ -141,8 +138,7 @@ def parse_log_django(log: str) -> dict[str, str]:
def parse_log_pytest_v2(log: str) -> dict[str, str]:
"""
Parser for test logs generated with PyTest framework (Later Version)
"""Parser for test logs generated with PyTest framework (Later Version)
Args:
log (str): log content
@@ -170,8 +166,7 @@ def parse_log_pytest_v2(log: str) -> dict[str, str]:
def parse_log_seaborn(log: str) -> dict[str, str]:
"""
Parser for test logs generated with seaborn testing framework
"""Parser for test logs generated with seaborn testing framework
Args:
log (str): log content
@@ -196,8 +191,7 @@ def parse_log_seaborn(log: str) -> dict[str, str]:
def parse_log_sympy(log: str) -> dict[str, str]:
"""
Parser for test logs generated with Sympy framework
"""Parser for test logs generated with Sympy framework
Args:
log (str): log content
@@ -229,8 +223,7 @@ def parse_log_sympy(log: str) -> dict[str, str]:
def parse_log_matplotlib(log: str) -> dict[str, str]:
"""
Parser for test logs generated with PyTest framework
"""Parser for test logs generated with PyTest framework
Args:
log (str): log content
+15 -30
View File
@@ -12,8 +12,7 @@ if sys.getrecursionlimit() < 10_000:
def bleu(gold: list[str], pred: list[str]) -> float:
"""
Calculate BLEU score, using smoothing method 2 with auto reweighting, in the range of 0~100.
"""Calculate BLEU score, using smoothing method 2 with auto reweighting, in the range of 0~100.
:param gold: list of gold tokens
:param pred: list of predicted tokens
@@ -30,8 +29,7 @@ def bleu(gold: list[str], pred: list[str]) -> float:
def batch_bleu(golds: list[list[str]], preds: list[list[str]]) -> list[float]:
"""
Calculate BLEU score for a batch of sentences.
"""Calculate BLEU score for a batch of sentences.
:param golds: list of gold sentences
:param preds: list of predicted sentences
@@ -43,8 +41,7 @@ def batch_bleu(golds: list[list[str]], preds: list[list[str]]) -> list[float]:
def corpus_bleu(golds: list[list[str]], preds: list[list[str]]) -> float:
"""
Calculate corpus-level BLEU score for a batch of sentences.
"""Calculate corpus-level BLEU score for a batch of sentences.
:param golds: list of gold sentences
:param preds: list of predicted sentences
@@ -63,8 +60,7 @@ def corpus_bleu(golds: list[list[str]], preds: list[list[str]]) -> float:
def edit_sim(
gold: Union[str, list[str]], pred: Union[str, list[str]], sep: str = ' '
) -> float:
"""
Calculate char-level edit similarity, in the range of 0~100.
"""Calculate char-level edit similarity, in the range of 0~100.
:param gold: gold sentence or list of gold tokens
:param pred: predicted sentence or list of predicted tokens
@@ -85,8 +81,7 @@ def batch_edit_sim(
preds: list[Union[str, list[str]]],
sep: str = ' ',
) -> list[float]:
"""
Calculate char-level edit similarity for a batch of sentences.
"""Calculate char-level edit similarity for a batch of sentences.
:param golds: list of gold sentences
:param preds: list of predicted sentences
@@ -102,8 +97,7 @@ T = TypeVar('T')
def exact_match(gold: T, pred: T) -> float:
"""
Calculate exact match accuracy, in the range of {0, 100}.
"""Calculate exact match accuracy, in the range of {0, 100}.
:param gold: gold sentence or list of gold tokens
:param pred: predicted sentence or list of predicted tokens
@@ -115,8 +109,7 @@ def exact_match(gold: T, pred: T) -> float:
def batch_exact_match(golds: list[T], preds: list[T]) -> list[float]:
"""
Calculate exact match accuracy for a batch of sentences.
"""Calculate exact match accuracy for a batch of sentences.
:param golds: list of gold sentences
:param preds: list of predicted sentences
@@ -130,8 +123,7 @@ def batch_exact_match(golds: list[T], preds: list[T]) -> list[float]:
def rouge_l(
gold: Union[str, list[str]], pred: Union[str, list[str]], sep: str = ' '
) -> dict[str, float]:
"""
Calculate ROUGE-L F1, precision, and recall scores, in the range of 0~100.
"""Calculate ROUGE-L F1, precision, and recall scores, in the range of 0~100.
:param gold: gold sentence or list of gold tokens
:param pred: predicted sentence or list of predicted tokens
@@ -156,8 +148,7 @@ def batch_rouge_l(
preds: list[Union[str, list[str]]],
sep: str = ' ',
) -> dict[str, list[float]]:
"""
Calculate ROUGE-L F1, precision, and recall scores for a batch of sentences.
"""Calculate ROUGE-L F1, precision, and recall scores for a batch of sentences.
:param golds: list of gold sentences
:param preds: list of predicted sentences
@@ -175,8 +166,7 @@ def accuracy(
pred: list[str],
ignore: Optional[Sequence[str]] = None,
) -> float:
"""
Calculate token-level accuracy, in the range of 0~100.
"""Calculate token-level accuracy, in the range of 0~100.
If gold and pred are not the same length, the longer one would be truncated.
:param gold: list of gold tokens
@@ -210,8 +200,7 @@ def batch_accuracy(
preds: list[list[str]],
ignore: Optional[Sequence[str]] = None,
) -> list[float]:
"""
Calculate token-level accuracy for a batch of sentences.
"""Calculate token-level accuracy for a batch of sentences.
:param golds: list of gold sentences
:param preds: list of predicted sentences
@@ -226,8 +215,7 @@ def batch_accuracy(
def first_match_to_topk(
first_match_list: list[int], k_values: list[int]
) -> dict[int, list[float]]:
"""
Calculate top-k accuracy with the first match ranks (1-indexed).
"""Calculate top-k accuracy with the first match ranks (1-indexed).
:param first_match: first match ranks (1-indexed)
:param k_values: k values to consider
@@ -237,8 +225,7 @@ def first_match_to_topk(
def pass_at_k(n: int, c: int, k: int) -> float:
"""
Sample pass@k metric according to the Codex paper, but in the scale of 0~100.
"""Sample pass@k metric according to the Codex paper, but in the scale of 0~100.
:param n: total number of samples
:param c: number of correct samples
:param k: k in pass@$k$
@@ -251,8 +238,7 @@ def pass_at_k(n: int, c: int, k: int) -> float:
def self_bleu(samples: list[list[str]]) -> float:
"""
Calculate self-BLEU among the samples.
"""Calculate self-BLEU among the samples.
:param samples: the chosen m samples
:return: self-BLEU
"""
@@ -274,8 +260,7 @@ def self_bleu(samples: list[list[str]]) -> float:
def self_edit_distance(samples: list[Union[str, list[str]]], sep=' ') -> float:
"""
Calculate self-edit-distance among the samples.
"""Calculate self-edit-distance among the samples.
:param samples: the chosen m samples
:param sep: the separator between tokens
:return: self-edit-distance
@@ -30,8 +30,7 @@ def check_mutation(mutation_output):
def count_methods(code_str):
"""
Counts the number of methods/functions in a given string of code.
"""Counts the number of methods/functions in a given string of code.
Args:
code_str (str): A string containing code.
@@ -46,8 +45,7 @@ def count_methods(code_str):
def get_lines_of_code(code_str):
"""
Extracts lines of code from a given string.
"""Extracts lines of code from a given string.
Args:
code_str (str): A string containing code.
@@ -7,8 +7,7 @@ import traceback
def insert_line_in_string(input_string, new_str, insert_line):
"""
Inserts a new line into a string at the specified line number.
"""Inserts a new line into a string at the specified line number.
:param input_string: The original string.
:param new_str: The string to insert.
@@ -29,8 +28,7 @@ def insert_line_in_string(input_string, new_str, insert_line):
def print_string_diff(original, modified):
"""
Prints the differences between two strings line by line.
"""Prints the differences between two strings line by line.
:param original: The original string.
:param modified: The modified string.
@@ -37,8 +37,7 @@ def extract_preamble_classes_and_functions(code):
current_position = 0
def extract_class_body(code: str, start_index: int) -> tuple[str, int]:
"""
Extracts the body of a class from the given code starting from the specified index.
"""Extracts the body of a class from the given code starting from the specified index.
Returns the class body and the end index of the class body.
"""
if not code or start_index < 0 or start_index >= len(code):
@@ -168,8 +167,8 @@ def extract_preamble_classes_and_functions(code):
def filter_passing_tests(
test_content: str, test_output: str, repo: str
) -> tuple[str, list[str], list[str]]:
"""
Filter tests based on their execution results.
"""Filter tests based on their execution results.
Returns:
Tuple containing:
- Modified test content with only passing tests
@@ -246,8 +245,7 @@ def filter_passing_tests(
def filter_tests(
test_content: str, test_output: str, repo: str
) -> tuple[str, list[str], list[str]]:
"""
Filter tests using AST parsing to remove failing test functions from the test file.
"""Filter tests using AST parsing to remove failing test functions from the test file.
Non-test functions (e.g. setup or helper methods) and classes (even if all test methods are failing)
are preserved.
+3 -11
View File
@@ -20,9 +20,7 @@ DIFF_MODIFIED_FILE_REGEX = r'--- a/(.*)'
@dataclass
class TestSpec:
"""
A dataclass that represents a test specification for a single instance of SWE-bench.
"""
"""A dataclass that represents a test specification for a single instance of SWE-bench."""
instance_id: str
id: str
@@ -86,10 +84,7 @@ def make_test_setup(specs, env_name, repo_directory, includes_tox=False):
def make_test_script_list(test_cmd, specs, env_name, repo_directory):
"""
Runs the tests.
"""
"""Runs the tests."""
includes_tox = 'tox' in test_cmd
eval_commands = make_test_setup(specs, env_name, repo_directory, includes_tox)
eval_commands += [
@@ -104,10 +99,7 @@ def make_test_script_list(test_cmd, specs, env_name, repo_directory):
def make_mutation_script_list(specs, env_name, repo_directory, mutation_timeout):
"""
Runs the tests.
"""
"""Runs the tests."""
eval_commands = make_test_setup(specs, env_name, repo_directory)
eval_commands += [
'cosmic-ray init mutation.toml mutation.sqlite',
+2 -5
View File
@@ -11,8 +11,7 @@ from evaluation.benchmarks.testgeneval.constants import (
def get_test_directives(instance: TestGenEvalInstance) -> list:
"""
Get test directives from the test_patch of a task instance
"""Get test directives from the test_patch of a task instance
Args:
instance (dict): task instance
@@ -43,9 +42,7 @@ def get_test_directives(instance: TestGenEvalInstance) -> list:
def load_testgeneval_dataset(
name='kjain14/testgeneval', split='test', ids=None
) -> list[TestGenEvalInstance]:
"""
Load SWE-bench dataset from Hugging Face Datasets or local .json/.jsonl file
"""
"""Load SWE-bench dataset from Hugging Face Datasets or local .json/.jsonl file"""
# check that all instance IDs are in the dataset
if ids:
ids = set(ids)
@@ -24,9 +24,7 @@ class ActionType(Enum):
@dataclass
class Selector:
"""
Represents either a direct anchor ID or a descriptive selector
"""
"""Represents either a direct anchor ID or a descriptive selector"""
value: str
is_anchor: bool = False
@@ -149,8 +147,7 @@ def find_matching_anchor(content: str, selector: str) -> str | None:
def resolve_action(action: BrowserAction, content: str) -> BrowserAction:
"""
Resolve any descriptive selectors in the action to anchor IDs based on the content.
"""Resolve any descriptive selectors in the action to anchor IDs based on the content.
Returns a new action with resolved selectors.
"""
if isinstance(action, (InputAction, ClickAction)):
@@ -174,8 +171,7 @@ def pre_login(
save_screenshots=True,
screenshots_dir='screenshots',
):
"""
Logs in to all the websites that are needed for the evaluation.
"""Logs in to all the websites that are needed for the evaluation.
Once logged in, the sessions would be cached in the browser, so OpenHands
agent doesn't need to log in to these websites again.
"""
@@ -68,8 +68,7 @@ def get_config(
def load_dependencies(runtime: Runtime) -> list[str]:
"""
Every task has a dependencies.yml file, which lists all the services that the
"""Every task has a dependencies.yml file, which lists all the services that the
task depends on. This function loads the file and returns all dependent service names.
"""
command = 'cat /utils/dependencies.yml'
@@ -11,9 +11,7 @@ import sys
def calculate_cost(model: str, prompt_tokens: int, completion_tokens: int) -> float:
"""
Calculate the cost of the model call.
"""
"""Calculate the cost of the model call."""
if 'claude-3-5-sonnet' in model.lower():
# https://www.anthropic.com/pricing#anthropic-api, accessed 12/11/2024
return 0.000003 * prompt_tokens + 0.000015 * completion_tokens
@@ -60,8 +58,7 @@ def calculate_cost(model: str, prompt_tokens: int, completion_tokens: int) -> fl
def analyze_eval_json_file(filepath: str) -> tuple[int, int]:
"""
Analyze a single eval JSON file and extract the total and result from final_score.
"""Analyze a single eval JSON file and extract the total and result from final_score.
Args:
filepath: Path to the JSON file
@@ -84,8 +81,7 @@ def analyze_eval_json_file(filepath: str) -> tuple[int, int]:
def analyze_traj_json_file(filepath: str) -> tuple[int, float]:
"""
Analyze a single trajectory JSON file and extract the steps and tokens
"""Analyze a single trajectory JSON file and extract the steps and tokens
for each step. Then estimate the cost based on the tokens and the model type.
Note: this is assuming there's no prompt caching at all.
"""
@@ -115,8 +111,7 @@ def analyze_traj_json_file(filepath: str) -> tuple[int, float]:
def analyze_folder(
folder_path: str,
) -> tuple[dict[str, tuple[int, int]], dict[str, tuple[int, float]]]:
"""
Analyze all eval_*.json & traj_*.json files in the specified folder.
"""Analyze all eval_*.json & traj_*.json files in the specified folder.
Args:
folder_path: Path to the folder containing JSON files
@@ -148,9 +143,7 @@ def analyze_folder(
def get_task_nature_category(task_name: str) -> str:
"""
Get the nature category of the task.
"""
"""Get the nature category of the task."""
task_nature = task_name.split('-')[0]
if task_nature.lower() in ['sde', 'pm', 'ds', 'admin', 'hr', 'finance']:
return task_nature
@@ -159,8 +152,7 @@ def get_task_nature_category(task_name: str) -> str:
def calculate_score(total: int, result: int) -> float:
"""
Calculate the score as a number between 0 and 1.
"""Calculate the score as a number between 0 and 1.
Formula: score = (result / total) * 0.5 + (result // total) * 0.5
Explanation:
@@ -178,8 +170,7 @@ def calculate_score(total: int, result: int) -> float:
def is_perfect_completion(total: int, result: int) -> bool:
"""
Check if the task achieved perfect completion.
"""Check if the task achieved perfect completion.
Args:
total: Total possible points
@@ -1,6 +1,4 @@
"""
GPT performs line level generation prediction and truncates overly long tokens
"""
"""GPT performs line level generation prediction and truncates overly long tokens"""
import json
import os
@@ -56,8 +54,7 @@ def predict(content, model_name):
def bulid_prompt(description, old_version, old_code, new_version) -> str:
"""
build prompt
"""Build prompt
:param version:
:param description:
:param masked_code:
@@ -1,6 +1,4 @@
"""
GPT performs line level generation prediction and truncates overly long tokens
"""
"""GPT performs line level generation prediction and truncates overly long tokens"""
import json
import os
@@ -56,8 +54,7 @@ def predict(content, model_name):
def bulid_prompt(version, description) -> str:
"""
build prompt
"""Build prompt
:param version:
:param description:
:param masked_code:
@@ -1,6 +1,4 @@
"""
block completion
"""
"""block completion"""
import copy
import gc
@@ -79,8 +77,7 @@ def run_inference(model_name, origin_data_list):
def bulid_prompt(version, description) -> str:
"""
build prompt
"""Build prompt
:param version:
:param description:
:param masked_code:
@@ -1,6 +1,4 @@
"""
code migration
"""
"""code migration"""
import copy
import gc
@@ -81,8 +79,7 @@ def run_inference(model_name, origin_data_list):
def bulid_prompt(description, old_version, old_code, new_version) -> str:
"""
build prompt
"""Build prompt
:param version:
:param description:
:param masked_code:
@@ -1,5 +1,4 @@
"""
评测block的预测能力
"""评测block的预测能力
1、判断是否包含正确的函数名
2、判断是否合法
3、计算ISM,和PM
@@ -22,8 +21,7 @@ def is_code_valid(code):
def longest_common_prefix_between_lists_with_elements(list1, list2):
"""
计算两个字符串列表中元素的最长前缀匹配长度
"""计算两个字符串列表中元素的最长前缀匹配长度
:param list1:
:param list2:
:return:
@@ -46,8 +44,7 @@ def longest_common_prefix_between_lists_with_elements(list1, list2):
def get_token(ans_code: str, output_code: str):
"""
对代码进行词法分析,分解成标识符,返回两个标识符列表
"""对代码进行词法分析,分解成标识符,返回两个标识符列表
:param ans_code:
:param output_code:
:return:
@@ -94,8 +91,7 @@ def get_token(ans_code: str, output_code: str):
def get_token_per_line(code: str):
"""
对每一行代码进行词法分析,记录每一行的标识符
"""对每一行代码进行词法分析,记录每一行的标识符
:param code: 代码字符串
:return: 每一行的标识符列表组成的列表
"""
@@ -117,8 +113,7 @@ def get_token_per_line(code: str):
def get_ISM(answer_code: str, model_output_list: list, answer_name: str) -> list:
"""
计算ISM,返回一个有序的得分列表
"""计算ISM,返回一个有序的得分列表
:return:
"""
score_list = []
@@ -157,8 +152,7 @@ def get_ISM(answer_code: str, model_output_list: list, answer_name: str) -> list
def get_ISM_without_verification(
answer_code: str, model_output_list: list, answer_name: str
) -> list:
"""
计算ISM,返回一个有序的得分列表
"""计算ISM,返回一个有序的得分列表
:return:
"""
score_list = []
@@ -190,8 +184,7 @@ def get_ISM_without_verification(
def longest_common_prefix_with_lengths(list1, list2):
"""
计算两个二维列表中每个子列表的最长前缀匹配长度,并记录拥有最长前缀匹配长度的两个子列表的长度
"""计算两个二维列表中每个子列表的最长前缀匹配长度,并记录拥有最长前缀匹配长度的两个子列表的长度
:param list1: 第一个二维列表
:param list2: 第二个二维列表
:return: 最长前缀匹配长度以及拥有最长前缀匹配长度的两个子列表的长度
@@ -216,8 +209,7 @@ def longest_common_prefix_with_lengths(list1, list2):
def get_PM(answer_code: str, model_output_list: list, answer_name: str) -> list:
"""
计算PM,返回一个有序的得分列表
"""计算PM,返回一个有序的得分列表
:return:
"""
score_list = []
@@ -254,8 +246,7 @@ def get_PM(answer_code: str, model_output_list: list, answer_name: str) -> list:
def get_score(score_list: list, k):
"""
计算score@n,k
"""计算score@n,k
:param score_list:
:param k:
:return:
@@ -1,6 +1,4 @@
"""
Calculate the cdc score for migration
"""
"""Calculate the cdc score for migration"""
import json
import math
@@ -11,8 +9,7 @@ import re
def is_correct_parameter_count(function_name, correct_code, test_code):
"""
判断参数数量是否一致
"""判断参数数量是否一致
:param function_name:
:param correct_code:
:param test_code:
@@ -43,8 +40,7 @@ def is_correct_parameter_count(function_name, correct_code, test_code):
def check_keyword_parameters(function_name, correct_code, test_code):
"""
判断关键词参数赋值是否正确使用
"""判断关键词参数赋值是否正确使用
:param function_name:
:param correct_code:
:param test_code:
@@ -82,8 +78,7 @@ def check_keyword_parameters(function_name, correct_code, test_code):
def with_correct(answer_code: str, model_output: str) -> bool:
"""
当answer是with结构时,判断模型生成的是不是with结构
"""当answer是with结构时,判断模型生成的是不是with结构
:param answer_code:
:param model_output:
:return:
@@ -105,9 +100,7 @@ def compute_block_score_k(
core_line_in_core_block,
core_line_in_output_clear,
):
"""
cdc需要满足五个条件,em只需要满足第一个条件
"""
"""cdc需要满足五个条件,em只需要满足第一个条件"""
c = 0
n = len(model_output)
for index, code in enumerate(model_output):
@@ -1,6 +1,4 @@
"""
Calculate the cdc score for line and block
"""
"""Calculate the cdc score for line and block"""
import json
import math
@@ -19,8 +17,7 @@ def is_code_valid(code):
def is_correct_parameter_count(function_name, correct_code, test_code):
"""
判断参数数量是否一致
"""判断参数数量是否一致
:param function_name:
:param correct_code:
:param test_code:
@@ -51,8 +48,7 @@ def is_correct_parameter_count(function_name, correct_code, test_code):
def check_keyword_parameters(function_name, correct_code, test_code):
"""
判断关键词参数赋值是否正确使用
"""判断关键词参数赋值是否正确使用
:param function_name:
:param correct_code:
:param test_code:
@@ -90,8 +86,7 @@ def check_keyword_parameters(function_name, correct_code, test_code):
def with_correct(answer_code: str, model_output: str) -> bool:
"""
当answer是with结构时,判断模型生成的是不是with结构
"""当answer是with结构时,判断模型生成的是不是with结构
:param answer_code:
:param model_output:
:return:
@@ -1,6 +1,4 @@
"""
Calculate the cdc score for line and block
"""
"""Calculate the cdc score for line and block"""
import json
import math
@@ -19,8 +17,7 @@ def is_code_valid(code):
def is_correct_parameter_count(function_name, correct_code, test_code):
"""
判断参数数量是否一致
"""判断参数数量是否一致
:param function_name:
:param correct_code:
:param test_code:
@@ -51,8 +48,7 @@ def is_correct_parameter_count(function_name, correct_code, test_code):
def check_keyword_parameters(function_name, correct_code, test_code):
"""
判断关键词参数赋值是否正确使用
"""判断关键词参数赋值是否正确使用
:param function_name:
:param correct_code:
:param test_code:
@@ -90,8 +86,7 @@ def check_keyword_parameters(function_name, correct_code, test_code):
def with_correct(answer_code: str, model_output: str) -> bool:
"""
当answer是with结构时,判断模型生成的是不是with结构
"""当answer是with结构时,判断模型生成的是不是with结构
:param answer_code:
:param model_output:
:return:
@@ -1,6 +1,4 @@
"""
Find the line of code generated by the model using the block in the version code
"""
"""Find the line of code generated by the model using the block in the version code"""
import json
import os
@@ -1,6 +1,4 @@
"""
Find the line of code generated by the model using the block in the version code
"""
"""Find the line of code generated by the model using the block in the version code"""
import json
import os
@@ -1,6 +1,4 @@
"""
Clear the<start>and<end>generated by the model in inference
"""
"""Clear the<start>and<end>generated by the model in inference"""
import json
+1 -2
View File
@@ -622,8 +622,7 @@ def compatibility_for_eval_history_pairs(
def is_fatal_evaluation_error(error: str | None) -> bool:
"""
The AgentController class overrides last error for certain exceptions
"""The AgentController class overrides last error for certain exceptions
We want to ensure those exeption do not overlap with fatal exceptions defined here
This is because we do a comparisino against the stringified error
"""
@@ -0,0 +1,135 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { renderHook } from "@testing-library/react";
import { useDocumentTitleFromState } from "#/hooks/use-document-title-from-state";
import { useActiveConversation } from "#/hooks/query/use-active-conversation";
// Mock the useActiveConversation hook
vi.mock("#/hooks/query/use-active-conversation");
const mockUseActiveConversation = vi.mocked(useActiveConversation);
describe("useDocumentTitleFromState", () => {
const originalTitle = document.title;
beforeEach(() => {
vi.clearAllMocks();
document.title = "Test";
});
afterEach(() => {
document.title = originalTitle;
vi.resetAllMocks();
});
it("should set document title to default suffix when no conversation", () => {
mockUseActiveConversation.mockReturnValue({
data: null,
} as any);
renderHook(() => useDocumentTitleFromState());
expect(document.title).toBe("OpenHands");
});
it("should set document title to custom suffix when no conversation", () => {
mockUseActiveConversation.mockReturnValue({
data: null,
} as any);
renderHook(() => useDocumentTitleFromState("Custom App"));
expect(document.title).toBe("Custom App");
});
it("should set document title with conversation title", () => {
mockUseActiveConversation.mockReturnValue({
data: {
conversation_id: "123",
title: "My Conversation",
status: "RUNNING",
},
} as any);
renderHook(() => useDocumentTitleFromState());
expect(document.title).toBe("My Conversation | OpenHands");
});
it("should update document title when conversation title changes", () => {
// Initial state - no conversation
mockUseActiveConversation.mockReturnValue({
data: null,
} as any);
const { rerender } = renderHook(() => useDocumentTitleFromState());
expect(document.title).toBe("OpenHands");
// Conversation with initial title
mockUseActiveConversation.mockReturnValue({
data: {
conversation_id: "123",
title: "Conversation 65e29",
status: "RUNNING",
},
} as any);
rerender();
expect(document.title).toBe("Conversation 65e29 | OpenHands");
// Conversation title updated to human-readable title
mockUseActiveConversation.mockReturnValue({
data: {
conversation_id: "123",
title: "Help me build a React app",
status: "RUNNING",
},
} as any);
rerender();
expect(document.title).toBe("Help me build a React app | OpenHands");
});
it("should handle conversation without title", () => {
mockUseActiveConversation.mockReturnValue({
data: {
conversation_id: "123",
title: undefined,
status: "RUNNING",
},
} as any);
renderHook(() => useDocumentTitleFromState());
expect(document.title).toBe("OpenHands");
});
it("should handle empty conversation title", () => {
mockUseActiveConversation.mockReturnValue({
data: {
conversation_id: "123",
title: "",
status: "RUNNING",
},
} as any);
renderHook(() => useDocumentTitleFromState());
expect(document.title).toBe("OpenHands");
});
it("should reset document title on cleanup", () => {
mockUseActiveConversation.mockReturnValue({
data: {
conversation_id: "123",
title: "My Conversation",
status: "RUNNING",
},
} as any);
const { unmount } = renderHook(() => useDocumentTitleFromState());
expect(document.title).toBe("My Conversation | OpenHands");
unmount();
expect(document.title).toBe("OpenHands");
});
});
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "openhands-frontend",
"version": "0.51.1",
"version": "0.52.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "openhands-frontend",
"version": "0.51.1",
"version": "0.52.1",
"dependencies": {
"@heroui/react": "^2.8.2",
"@heroui/use-infinite-scroll": "^2.2.10",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "openhands-frontend",
"version": "0.51.1",
"version": "0.52.1",
"private": true,
"type": "module",
"engines": {
@@ -53,7 +53,7 @@ export function ChatMessage({
onMouseEnter={() => setIsHovering(true)}
onMouseLeave={() => setIsHovering(false)}
className={cn(
"rounded-xl relative w-fit",
"rounded-xl relative w-fit max-w-full",
"flex flex-col gap-2",
type === "user" && " p-4 bg-tertiary self-end",
type === "agent" && "mt-6 max-w-full bg-transparent",
@@ -1,8 +1,10 @@
import { Lock } from "lucide-react";
import { useTranslation } from "react-i18next";
import { ContextMenu } from "./context-menu";
import { ContextMenuListItem } from "./context-menu-list-item";
import { useClickOutsideElement } from "#/hooks/use-click-outside-element";
import { I18nKey } from "#/i18n/declaration";
import { ContextMenuIconText } from "./context-menu-icon-text";
interface AccountSettingsContextMenuProps {
onLogout: () => void;
@@ -23,7 +25,10 @@ export function AccountSettingsContextMenu({
className="absolute right-full md:left-full -top-1 z-10 w-fit"
>
<ContextMenuListItem onClick={onLogout} data-testid="logout-button">
{t(I18nKey.ACCOUNT_SETTINGS$LOGOUT)}
<ContextMenuIconText
icon={Lock}
text={t(I18nKey.ACCOUNT_SETTINGS$LOGOUT)}
/>
</ContextMenuListItem>
</ContextMenu>
);
@@ -13,6 +13,7 @@ export function ConnectToProviderMessage() {
<Link
data-testid="navigate-to-settings-button"
to="/settings/integrations"
className="self-start"
>
<BrandButton type="button" variant="primary" isDisabled={isLoading}>
{!isLoading && t("SETTINGS$TITLE")}
@@ -1,4 +1,4 @@
import React, { useState } from "react";
import React, { useState, useRef, useEffect } from "react";
import { useTranslation, Trans } from "react-i18next";
import { MCPConfig } from "#/types/settings";
import { I18nKey } from "#/i18n/declaration";
@@ -11,6 +11,11 @@ interface MCPJsonEditorProps {
onCancel: () => void;
}
const MCP_DEFAULT_CONFIG: MCPConfig = {
sse_servers: [],
stdio_servers: [],
};
export function MCPJsonEditor({
mcpConfig,
onChange,
@@ -20,10 +25,17 @@ export function MCPJsonEditor({
const [configText, setConfigText] = useState(() =>
mcpConfig
? JSON.stringify(mcpConfig, null, 2)
: t(I18nKey.SETTINGS$MCP_DEFAULT_CONFIG),
: JSON.stringify(MCP_DEFAULT_CONFIG, null, 2),
);
const [error, setError] = useState<string | null>(null);
const textareaRef = useRef<HTMLTextAreaElement>(null);
useEffect(() => {
textareaRef.current?.focus();
}, []);
const handleTextChange = (e: React.ChangeEvent<HTMLTextAreaElement>) => {
setConfigText(e.target.value);
};
@@ -89,6 +101,7 @@ export function MCPJsonEditor({
/>
</p>
<textarea
ref={textareaRef}
className={cn(
"w-full h-64 resize-y p-2 rounded-sm text-sm font-mono",
"bg-tertiary border border-[#717888]",
@@ -118,7 +131,7 @@ export function MCPJsonEditor({
{t(I18nKey.BUTTON$CANCEL)}
</BrandButton>
<BrandButton type="button" variant="primary" onClick={handleSave}>
{t(I18nKey.SETTINGS$MCP_CONFIRM_CHANGES)}
{t(I18nKey.SETTINGS$MCP_PREVIEW_CHANGES)}
</BrandButton>
</div>
</div>
@@ -22,5 +22,5 @@ export function useDocumentTitleFromState(suffix = "OpenHands") {
return () => {
document.title = suffix;
};
}, [conversation, suffix]);
}, [conversation?.title, suffix]);
}
+1 -2
View File
@@ -51,7 +51,7 @@ export enum I18nKey {
SETTINGS$NAV_MCP = "SETTINGS$NAV_MCP",
SETTINGS$MCP_CONFIGURATION = "SETTINGS$MCP_CONFIGURATION",
SETTINGS$MCP_EDIT_CONFIGURATION = "SETTINGS$MCP_EDIT_CONFIGURATION",
SETTINGS$MCP_CONFIRM_CHANGES = "SETTINGS$MCP_CONFIRM_CHANGES",
SETTINGS$MCP_PREVIEW_CHANGES = "SETTINGS$MCP_PREVIEW_CHANGES",
SETTINGS$MCP_CONFIG_DESCRIPTION = "SETTINGS$MCP_CONFIG_DESCRIPTION",
SETTINGS$MCP_CONFIG_ERROR = "SETTINGS$MCP_CONFIG_ERROR",
SETTINGS$MCP_CONFIG_EXAMPLE = "SETTINGS$MCP_CONFIG_EXAMPLE",
@@ -71,7 +71,6 @@ export enum I18nKey {
SETTINGS$MCP_ERROR_SSE_URL = "SETTINGS$MCP_ERROR_SSE_URL",
SETTINGS$MCP_ERROR_STDIO_PROPS = "SETTINGS$MCP_ERROR_STDIO_PROPS",
SETTINGS$MCP_ERROR_INVALID_JSON = "SETTINGS$MCP_ERROR_INVALID_JSON",
SETTINGS$MCP_DEFAULT_CONFIG = "SETTINGS$MCP_DEFAULT_CONFIG",
HOME$CONNECT_PROVIDER_MESSAGE = "HOME$CONNECT_PROVIDER_MESSAGE",
HOME$LETS_START_BUILDING = "HOME$LETS_START_BUILDING",
HOME$OPENHANDS_DESCRIPTION = "HOME$OPENHANDS_DESCRIPTION",
+15 -31
View File
@@ -815,21 +815,21 @@
"de": "Konfiguration bearbeiten",
"uk": "Редагувати налаштування"
},
"SETTINGS$MCP_CONFIRM_CHANGES": {
"en": "Confirm Changes",
"ja": "変更を確定",
"zh-CN": "确认更改",
"zh-TW": "確認變更",
"ko-KR": "변경 사항 확인",
"no": "Bekreft endringer",
"it": "Conferma modifiche",
"pt": "Confirmar alterações",
"es": "Confirmar cambios",
"ar": "تأكيد التغييرات",
"fr": "Confirmer les modifications",
"tr": "Değişiklikleri Onayla",
"de": "Änderungen bestätigen",
"uk": ідтвердити зміни"
"SETTINGS$MCP_PREVIEW_CHANGES": {
"en": "Preview Changes",
"ja": "変更内容をプレビュー",
"zh-CN": "预览更改",
"zh-TW": "預覽變更",
"ko-KR": "변경 사항 미리보기",
"no": "Forhåndsvis endringer",
"it": "Anteprima modifiche",
"pt": "Visualizar alterações",
"es": "Vista previa de los cambios",
"ar": "معاينة التغييرات",
"fr": "Aperçu des modifications",
"tr": "Değişiklikleri Önizle",
"de": "Änderungen anzeigen",
"uk": ереглянути зміни"
},
"SETTINGS$MCP_CONFIG_DESCRIPTION": {
"en": "Edit the JSON configuration for MCP servers below. The configuration must include both sse_servers and stdio_servers arrays. For full configuration details and integration examples, see the <a>documentation</a>.",
@@ -1135,22 +1135,6 @@
"de": "Ungültiges JSON",
"uk": "Невірний JSON"
},
"SETTINGS$MCP_DEFAULT_CONFIG": {
"en": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"ja": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"zh-CN": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"zh-TW": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"ko-KR": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"no": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"it": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"pt": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"es": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"ar": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"fr": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"tr": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"de": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}",
"uk": "{\n \"sse_servers\": [],\n \"stdio_servers\": []\n}"
},
"HOME$CONNECT_PROVIDER_MESSAGE": {
"en": "To get started with suggested tasks, please connect your GitHub, GitLab, or Bitbucket account.",
"ja": "提案されたタスクを始めるには、GitHub、GitLab、またはBitbucketアカウントを接続してください。",
+2 -2
View File
@@ -249,11 +249,11 @@ function AppSettingsScreen() {
className="w-full max-w-[680px]" // Match the width of the language field
/>
<div className="border-t border-t-tertiary pt-6 mt-2 hidden">
<div className="border-t border-t-tertiary pt-6 mt-2">
<h3 className="text-lg font-medium mb-2">
{t(I18nKey.SETTINGS$GIT_SETTINGS)}
</h3>
<p className="text-sm text-secondary mb-4">
<p className="text-xs mb-4">
{t(I18nKey.SETTINGS$GIT_SETTINGS_DESCRIPTION)}
</p>
<div className="flex flex-col gap-6">
+8 -4
View File
@@ -1,4 +1,4 @@
import React, { useState } from "react";
import React, { useState, useEffect } from "react";
import { useTranslation } from "react-i18next";
import posthog from "posthog-js";
import { useSettings } from "#/hooks/query/use-settings";
@@ -18,11 +18,15 @@ function MCPSettingsScreen() {
const { data: settings, isLoading } = useSettings();
const { mutate: saveSettings, isPending } = useSaveSettings();
const [mcpConfig, setMcpConfig] = useState<MCPConfig | undefined>(
settings?.MCP_CONFIG,
);
const [mcpConfig, setMcpConfig] = useState<MCPConfig | undefined>(undefined);
const [isDirty, setIsDirty] = useState(false);
useEffect(() => {
if (!mcpConfig && settings?.MCP_CONFIG) {
setMcpConfig(settings.MCP_CONFIG);
}
}, [settings, mcpConfig]);
const handleConfigChange = (config: MCPConfig) => {
setMcpConfig(config);
setIsDirty(true);
+1
View File
@@ -89,6 +89,7 @@ function SecretsSettingsScreen() {
to="/settings/integrations"
data-testid="connect-git-button"
type="button"
className="self-start"
>
<BrandButton type="button" variant="secondary">
{t(I18nKey.SECRETS$CONNECT_GIT_PROVIDER)}
+2 -1
View File
@@ -47,7 +47,8 @@ export function getIndicatorColor(
webSocketStatus === "DISCONNECTED" ||
conversationStatus === "STOPPED" ||
runtimeStatus === "STATUS$STOPPED" ||
agentState === AgentState.STOPPED
agentState === AgentState.STOPPED ||
agentState === AgentState.ERROR
) {
return IndicatorColor.RED;
}
@@ -3,8 +3,7 @@ import sys
def refine_prompt(prompt: str):
"""
Refines the prompt based on the platform.
"""Refines the prompt based on the platform.
On Windows systems, replaces 'bash' with 'powershell' and 'execute_bash' with 'execute_powershell'
to ensure commands work correctly on the Windows platform.
@@ -1,6 +1,4 @@
"""
ReadOnlyAgent - A specialized version of CodeActAgent that only uses read-only tools.
"""
"""ReadOnlyAgent - A specialized version of CodeActAgent that only uses read-only tools."""
import os
from typing import TYPE_CHECKING
-3
View File
@@ -723,9 +723,6 @@ def run_cli_command(args):
except ConnectionRefusedError as e:
print_formatted_text(f'Connection refused: {e}')
sys.exit(1)
except Exception as e:
print_formatted_text(f'An error occurred: {e}')
sys.exit(1)
finally:
try:
# Cancel all running tasks
+126 -6
View File
@@ -1,4 +1,5 @@
from pathlib import Path
from typing import Optional
from prompt_toolkit import PromptSession, print_formatted_text
from prompt_toolkit.completion import FuzzyWordCompleter
@@ -19,6 +20,7 @@ from openhands.cli.utils import (
VERIFIED_OPENAI_MODELS,
VERIFIED_OPENHANDS_MODELS,
VERIFIED_PROVIDERS,
extract_model_and_provider,
organize_models_and_providers,
)
from openhands.controller.agent import Agent
@@ -124,12 +126,36 @@ async def get_validated_input(
completer=None,
validator=None,
error_message: str = 'Input cannot be empty',
*,
default_value: str = '',
enter_keeps_value: Optional[str] = None,
) -> str:
"""
Get validated input from user.
Args:
session: PromptSession instance
prompt_text: The text to display before the input
completer: Completer instance
validator: Function to validate input
error_message: Error message to display if input is invalid
default_value: Value to show prefilled in the prompt (prompt placeholder)
enter_keeps_value: If provided, pressing Enter on an empty input will
return this value (useful for keeping existing sensitive values)
Returns:
str: The validated input
"""
session.completer = completer
value = None
while True:
value = await session.prompt_async(prompt_text)
value = await session.prompt_async(prompt_text, default=default_value)
# If user submits empty input and a keep-value is provided, use it.
if not value.strip() and enter_keeps_value is not None:
value = enter_keeps_value
if validator:
is_valid = validator(value)
@@ -160,6 +186,50 @@ def save_settings_confirmation(config: OpenHandsConfig) -> bool:
)
def _get_current_values_for_modification_basic(
config: OpenHandsConfig,
) -> tuple[str, str, str]:
llm_config = config.get_llm_config()
current_provider = ''
current_model = ''
current_api_key = (
llm_config.api_key.get_secret_value() if llm_config.api_key else ''
)
if llm_config.model:
model_info = extract_model_and_provider(llm_config.model)
current_provider = model_info.provider or ''
current_model = model_info.model or ''
return current_provider, current_model, current_api_key
def _get_default_provider(provider_list: list[str]) -> str:
if 'anthropic' in provider_list:
return 'anthropic'
else:
return provider_list[0] if provider_list else ''
def _get_initial_provider_index(
verified_providers: list[str],
current_provider: str,
default_provider: str,
provider_choices: list[str],
) -> int:
if (current_provider or default_provider) in verified_providers:
return verified_providers.index(current_provider or default_provider)
elif current_provider or default_provider:
return len(provider_choices) - 1
return 0
def _get_initial_model_index(
verified_models: list[str], current_model: str, default_model: str
) -> int:
if (current_model or default_model) in verified_models:
return verified_models.index(current_model or default_model)
return 0
async def modify_llm_settings_basic(
config: OpenHandsConfig, settings_store: FileSettingsStore
) -> None:
@@ -174,23 +244,32 @@ async def modify_llm_settings_basic(
provider_completer = FuzzyWordCompleter(provider_list)
session = PromptSession(key_bindings=kb_cancel())
# Set default provider - prefer 'anthropic' if available, otherwise use first
provider = 'anthropic' if 'anthropic' in provider_list else provider_list[0]
current_provider, current_model, current_api_key = (
_get_current_values_for_modification_basic(config)
)
default_provider = _get_default_provider(provider_list)
provider = None
model = None
api_key = None
try:
# Show the default provider but allow changing it
print_formatted_text(
HTML(f'\n<grey>Default provider: </grey><green>{provider}</green>')
HTML(f'\n<grey>Default provider: </grey><green>{default_provider}</green>')
)
# Show verified providers plus "Select another provider" option
provider_choices = verified_providers + ['Select another provider']
provider_choice = cli_confirm(
config,
'(Step 1/3) Select LLM Provider:',
provider_choices,
initial_selection=_get_initial_provider_index(
verified_providers, current_provider, default_provider, provider_choices
),
)
# Ensure provider_choice is an integer (for test compatibility)
@@ -211,8 +290,19 @@ async def modify_llm_settings_basic(
completer=provider_completer,
validator=lambda x: x in organized_models,
error_message='Invalid provider selected',
default_value=(
# Prefill only for unverified providers.
current_provider
if current_provider not in verified_providers
else ''
),
)
# Reset current model and api key if provider changes
if provider != current_provider:
current_model = ''
current_api_key = ''
# Make sure the provider exists in organized_models
if provider not in organized_models:
# If the provider doesn't exist, prefer 'anthropic' if available,
@@ -276,6 +366,9 @@ async def modify_llm_settings_basic(
+ 'LLM usage is billed at the providers rates with no markup. Details: https://docs.all-hands.dev/usage/llms/openhands-llms'
),
model_choices,
initial_selection=_get_initial_model_index(
VERIFIED_OPENHANDS_MODELS, current_model, default_model
),
)
# Get the selected model from the list
@@ -291,6 +384,9 @@ async def modify_llm_settings_basic(
config,
'Do you want to use a different model?',
[f'Use {default_model}', 'Select another model'],
initial_selection=0
if (current_model or default_model) == default_model
else 1,
)
== 1
)
@@ -320,6 +416,10 @@ async def modify_llm_settings_basic(
completer=model_completer,
validator=model_validator,
error_message='Model name cannot be empty',
default_value=(
# Prefill only for models that are not the default model.
current_model if current_model != default_model else ''
),
)
else:
# Use the default model
@@ -332,10 +432,15 @@ async def modify_llm_settings_basic(
)
)
prompt_text = '(Step 3/3) Enter API Key (CTRL-c to cancel): '
if current_api_key:
prompt_text = f'(Step 3/3) Enter API Key [{current_api_key[:4]}***{current_api_key[-4:]}] (CTRL-c to cancel, ENTER to keep current, type new to change): '
api_key = await get_validated_input(
session,
'(Step 3/3) Enter API Key (CTRL-c to cancel): ',
prompt_text,
error_message='API Key cannot be empty',
default_value='',
enter_keeps_value=current_api_key,
)
except (
@@ -386,6 +491,7 @@ async def modify_llm_settings_advanced(
config: OpenHandsConfig, settings_store: FileSettingsStore
) -> None:
session = PromptSession(key_bindings=kb_cancel())
llm_config = config.get_llm_config()
custom_model = None
base_url = None
@@ -397,18 +503,28 @@ async def modify_llm_settings_advanced(
session,
'(Step 1/6) Custom Model (CTRL-c to cancel): ',
error_message='Custom Model cannot be empty',
default_value=llm_config.model or '',
)
base_url = await get_validated_input(
session,
'(Step 2/6) Base URL (CTRL-c to cancel): ',
error_message='Base URL cannot be empty',
default_value=llm_config.base_url or '',
)
prompt_text = '(Step 3/6) API Key (CTRL-c to cancel): '
current_api_key = (
llm_config.api_key.get_secret_value() if llm_config.api_key else ''
)
if current_api_key:
prompt_text = f'(Step 3/6) API Key [{current_api_key[:4]}***{current_api_key[-4:]}] (CTRL-c to cancel, ENTER to keep current, type new to change): '
api_key = await get_validated_input(
session,
'(Step 3/6) API Key (CTRL-c to cancel): ',
prompt_text,
error_message='API Key cannot be empty',
default_value='',
enter_keeps_value=current_api_key,
)
agent_list = Agent.list_agents()
@@ -419,6 +535,7 @@ async def modify_llm_settings_advanced(
completer=agent_completer,
validator=lambda x: x in agent_list,
error_message='Invalid agent selected',
default_value=config.default_agent or '',
)
enable_confirmation_mode = (
@@ -426,6 +543,7 @@ async def modify_llm_settings_advanced(
config,
question='(Step 5/6) Confirmation Mode (CTRL-c to cancel):',
choices=['Enable', 'Disable'],
initial_selection=0 if config.security.confirmation_mode else 1,
)
== 0
)
@@ -435,6 +553,7 @@ async def modify_llm_settings_advanced(
config,
question='(Step 6/6) Memory Condensation (CTRL-c to cancel):',
choices=['Enable', 'Disable'],
initial_selection=0 if config.enable_default_condenser else 1,
)
== 0
)
@@ -463,6 +582,7 @@ async def modify_llm_settings_advanced(
config.default_agent = agent
config.security.confirmation_mode = enable_confirmation_mode
config.enable_default_condenser = enable_memory_condensation
agent_config = config.get_agent_config(config.default_agent)
if enable_memory_condensation:
-1
View File
@@ -5,7 +5,6 @@ import warnings
def suppress_cli_warnings():
"""Suppress common warnings that appear during CLI usage."""
# Suppress pydub warning about ffmpeg/avconv
warnings.filterwarnings(
'ignore',
+5 -7
View File
@@ -239,8 +239,7 @@ def display_mcp_errors() -> None:
# Prompt output display functions
def display_thought_if_new(thought: str, is_agent_message: bool = False) -> None:
"""
Display a thought only if it hasn't been displayed recently.
"""Display a thought only if it hasn't been displayed recently.
Args:
thought: The thought to display
@@ -301,8 +300,7 @@ def display_event(event: Event, config: OpenHandsConfig) -> None:
def display_message(message: str, is_agent_message: bool = False) -> None:
"""
Display a message in the terminal with markdown rendering.
"""Display a message in the terminal with markdown rendering.
Args:
message: The message to display
@@ -338,8 +336,7 @@ def display_message(message: str, is_agent_message: bool = False) -> None:
def convert_markdown_to_html(text: str) -> str:
"""
Convert markdown to HTML for prompt_toolkit's HTML renderer using the markdown library.
"""Convert markdown to HTML for prompt_toolkit's HTML renderer using the markdown library.
Args:
text: Markdown text to convert
@@ -845,6 +842,7 @@ def cli_confirm(
config: OpenHandsConfig,
question: str = 'Are you sure?',
choices: list[str] | None = None,
initial_selection: int = 0,
) -> int:
"""Display a confirmation prompt with the given question and choices.
@@ -852,7 +850,7 @@ def cli_confirm(
"""
if choices is None:
choices = ['Yes', 'No']
selected = [0] # Using list to allow modification in closure
selected = [initial_selection] # Using list to allow modification in closure
def get_choice_text() -> list:
return [
+6 -12
View File
@@ -56,8 +56,7 @@ def download_latest_vsix_from_github() -> str | None:
def attempt_vscode_extension_install():
"""
Checks if running in a supported editor and attempts to install the OpenHands companion extension.
"""Checks if running in a supported editor and attempts to install the OpenHands companion extension.
This is a best-effort, one-time attempt.
"""
# 1. Check if we are in a supported editor environment
@@ -132,8 +131,7 @@ def attempt_vscode_extension_install():
def _mark_installation_successful(flag_file: pathlib.Path, editor_name: str) -> None:
"""
Mark the extension installation as successful by creating the flag file.
"""Mark the extension installation as successful by creating the flag file.
Args:
flag_file: Path to the flag file to create
@@ -147,8 +145,7 @@ def _mark_installation_successful(flag_file: pathlib.Path, editor_name: str) ->
def _is_extension_installed(editor_command: str, extension_id: str) -> bool:
"""
Check if the OpenHands extension is already installed.
"""Check if the OpenHands extension is already installed.
Args:
editor_command: The command to run the editor (e.g., 'code', 'windsurf')
@@ -174,8 +171,7 @@ def _is_extension_installed(editor_command: str, extension_id: str) -> bool:
def _attempt_github_install(editor_command: str, editor_name: str) -> bool:
"""
Attempt to install the extension from GitHub Releases.
"""Attempt to install the extension from GitHub Releases.
Downloads the latest VSIX file from GitHub releases and attempts to install it.
Ensures proper cleanup of temporary files.
@@ -227,8 +223,7 @@ def _attempt_github_install(editor_command: str, editor_name: str) -> bool:
def _attempt_bundled_install(editor_command: str, editor_name: str) -> bool:
"""
Attempt to install the extension from the bundled VSIX file.
"""Attempt to install the extension from the bundled VSIX file.
Uses the VSIX file packaged with the OpenHands installation.
@@ -280,8 +275,7 @@ def _attempt_bundled_install(editor_command: str, editor_name: str) -> bool:
def _attempt_marketplace_install(
editor_command: str, editor_name: str, extension_id: str
) -> bool:
"""
Attempt to install the extension from the marketplace.
"""Attempt to install the extension from the marketplace.
This method is currently unused as the OpenHands extension is not yet published
to the VS Code/Windsurf marketplace. It's kept here for future use when the
+1 -2
View File
@@ -55,8 +55,7 @@ class Agent(ABC):
return self._prompt_manager
def get_system_message(self) -> 'SystemMessageAction | None':
"""
Returns a SystemMessageAction containing the system message and tools.
"""Returns a SystemMessageAction containing the system message and tools.
This will be added to the event stream as the first message.
Returns:
-1
View File
@@ -142,7 +142,6 @@ class AgentController:
status_callback: Optional callback function to handle status updates.
replay_events: A list of logs to replay.
"""
self.id = sid or event_stream.sid
self.user_id = user_id
self.file_store = file_store
+1 -2
View File
@@ -57,8 +57,7 @@ class ReplayManager:
)
def should_replay(self) -> bool:
"""
Whether the controller is in trajectory replay mode, and the replay
"""Whether the controller is in trajectory replay mode, and the replay
hasn't finished. Note: after the replay is finished, the user and
the agent could continue to message/act.
+2 -6
View File
@@ -46,8 +46,7 @@ class TrafficControlState(str, Enum):
@dataclass
class State:
"""
Represents the running state of an agent in the OpenHands system, saving data of its operation and memory.
"""Represents the running state of an agent in the OpenHands system, saving data of its operation and memory.
- Multi-agent/delegate state:
- store the task (conversation between the agent and the user)
@@ -143,10 +142,7 @@ class State:
def restore_from_session(
sid: str, file_store: FileStore, user_id: str | None = None
) -> 'State':
"""
Restores the state from the previously saved session.
"""
"""Restores the state from the previously saved session."""
state: State
try:
encoded = file_store.read(
+5 -13
View File
@@ -242,41 +242,33 @@ class StateTracker:
self.state.budget_flag.increase_limit(headless_mode)
def get_metrics_snapshot(self):
"""
Deep copy of metrics
"""Deep copy of metrics
This serves as a snapshot for the parent's metrics at the time a delegate is created
It will be stored and used to compute local metrics for the delegate
(since delegates now accumulate metrics from where its parent left off)
"""
return self.state.metrics.copy()
def save_state(self):
"""
Save's current state to persistent store
"""
"""Save's current state to persistent store"""
if self.sid and self.file_store:
self.state.save_to_session(self.sid, self.file_store, self.user_id)
def run_control_flags(self):
"""
Performs one step of the control flags
"""
"""Performs one step of the control flags"""
self.state.iteration_flag.step()
if self.state.budget_flag:
self.state.budget_flag.step()
def sync_budget_flag_with_metrics(self):
"""
Ensures that budget flag is up to date with accumulated costs from llm completions
"""Ensures that budget flag is up to date with accumulated costs from llm completions
Budget flag will monitor for when budget is exceeded
"""
if self.state.budget_flag:
self.state.budget_flag.current_value = self.state.metrics.accumulated_cost
def merge_metrics(self, metrics: Metrics):
"""
Merges metrics with the state metrics
"""Merges metrics with the state metrics
NOTE: this should be refactored in the future. We should have services (draft llm, title autocomplete, condenser, etc)
use their own LLMs, but the metrics object should be shared. This way we have one source of truth for accumulated costs from
+1 -2
View File
@@ -66,8 +66,7 @@ class KubernetesConfig(BaseModel):
@classmethod
def from_toml_section(cls, data: dict) -> dict[str, 'KubernetesConfig']:
"""
Create a mapping of KubernetesConfig instances from a toml dictionary representing the [kubernetes] section.
"""Create a mapping of KubernetesConfig instances from a toml dictionary representing the [kubernetes] section.
The configuration is built from all keys in data.
+1 -3
View File
@@ -97,8 +97,7 @@ class LLMConfig(BaseModel):
@classmethod
def from_toml_section(cls, data: dict) -> dict[str, LLMConfig]:
"""
Create a mapping of LLMConfig instances from a toml dictionary representing the [llm] section.
"""Create a mapping of LLMConfig instances from a toml dictionary representing the [llm] section.
The default configuration is built from all non-dict keys in data.
Then, each key with a dict value (e.g. [llm.random_name]) is treated as a custom LLM configuration,
@@ -117,7 +116,6 @@ class LLMConfig(BaseModel):
dict[str, LLMConfig]: A mapping where the key "llm" corresponds to the default configuration
and additional keys represent custom configurations.
"""
# Initialize the result mapping
llm_mapping: dict[str, LLMConfig] = {}
-1
View File
@@ -345,7 +345,6 @@ class OpenHandsMCPConfig:
Returns:
tuple[MCPSHTTPServerConfig | None, list[MCPStdioServerConfig]]: A tuple containing the default SHTTP server configuration (or None) and a list of MCP stdio server configurations
"""
stdio_servers = []
search_engine_stdio_server = OpenHandsMCPConfig.add_search_engine(config)
if search_engine_stdio_server:
+1 -2
View File
@@ -93,8 +93,7 @@ class SandboxConfig(BaseModel):
@classmethod
def from_toml_section(cls, data: dict) -> dict[str, 'SandboxConfig']:
"""
Create a mapping of SandboxConfig instances from a toml dictionary representing the [sandbox] section.
"""Create a mapping of SandboxConfig instances from a toml dictionary representing the [sandbox] section.
The configuration is built from all keys in data.
+1 -3
View File
@@ -16,15 +16,13 @@ class SecurityConfig(BaseModel):
@classmethod
def from_toml_section(cls, data: dict) -> dict[str, 'SecurityConfig']:
"""
Create a mapping of SecurityConfig instances from a toml dictionary representing the [security] section.
"""Create a mapping of SecurityConfig instances from a toml dictionary representing the [security] section.
The configuration is built from all keys in data.
Returns:
dict[str, SecurityConfig]: A mapping where the key "security" corresponds to the [security] configuration
"""
# Initialize the result mapping
security_mapping: dict[str, SecurityConfig] = {}
+16 -11
View File
@@ -5,6 +5,7 @@ import re
import sys
import traceback
from datetime import datetime
from logging.handlers import TimedRotatingFileHandler
from types import TracebackType
from typing import Any, Literal, Mapping, MutableMapping, TextIO
@@ -294,13 +295,21 @@ def get_console_handler(log_level: int = logging.INFO) -> logging.StreamHandler:
def get_file_handler(
log_dir: str, log_level: int = logging.INFO
) -> logging.FileHandler:
log_dir: str,
log_level: int = logging.INFO,
when: str = 'd',
backup_count: int = 7,
utc: bool = False,
) -> TimedRotatingFileHandler:
"""Returns a file handler for logging."""
os.makedirs(log_dir, exist_ok=True)
timestamp = datetime.now().strftime('%Y-%m-%d')
file_name = f'openhands_{timestamp}.log'
file_handler = logging.FileHandler(os.path.join(log_dir, file_name))
file_name = 'openhands.log'
file_handler = TimedRotatingFileHandler(
os.path.join(log_dir, file_name),
when=when,
backupCount=backup_count,
utc=utc,
)
file_handler.setLevel(log_level)
if LOG_JSON:
file_handler.setFormatter(json_formatter())
@@ -322,10 +331,7 @@ def json_log_handler(
level: int = logging.INFO,
_out: TextIO = sys.stdout,
) -> logging.Handler:
"""
Configure logger instance for structured logging as json lines.
"""
"""Configure logger instance for structured logging as json lines."""
handler = logging.StreamHandler(_out)
handler.setLevel(level)
handler.setFormatter(json_formatter())
@@ -496,8 +502,7 @@ class OpenHandsLoggerAdapter(logging.LoggerAdapter):
def process(
self, msg: str, kwargs: MutableMapping[str, Any]
) -> tuple[str, MutableMapping[str, Any]]:
"""
If 'extra' is supplied in kwargs, merge it with the adapters 'extra' dict
"""If 'extra' is supplied in kwargs, merge it with the adapters 'extra' dict
Starting in Python 3.13, LoggerAdapter's merge_extra option will do this.
"""
if 'extra' in kwargs and isinstance(kwargs['extra'], dict):
+1 -2
View File
@@ -14,8 +14,7 @@ async def run_agent_until_done(
memory: Memory,
end_states: list[AgentState],
) -> None:
"""
run_agent_until_done takes a controller and a runtime, and will run
"""run_agent_until_done takes a controller and a runtime, and will run
the agent until it reaches a terminal state.
Note that runtime must be connected before being passed in here.
"""
+1 -2
View File
@@ -257,8 +257,7 @@ def auto_continue_response(
def load_replay_log(trajectory_path: str) -> tuple[list[Event] | None, Action]:
"""
Load trajectory from given path, serialize it to a list of events, and return
"""Load trajectory from given path, serialize it to a list of events, and return
two things:
1) A list of events except the first action
2) First action (user message, a.k.a. initial task)
+2 -4
View File
@@ -3,8 +3,7 @@ from openhands.llm.metrics import Metrics, TokenUsage
def get_token_usage_for_event(event: Event, metrics: Metrics) -> TokenUsage | None:
"""
Returns at most one token usage record for either:
"""Returns at most one token usage record for either:
- `tool_call_metadata.model_response.id`, if possible
- otherwise event.response_id, if set
@@ -34,8 +33,7 @@ def get_token_usage_for_event(event: Event, metrics: Metrics) -> TokenUsage | No
def get_token_usage_for_event_id(
events: list[Event], event_id: int, metrics: Metrics
) -> TokenUsage | None:
"""
Starting from the event with .id == event_id and moving backwards in `events`,
"""Starting from the event with .id == event_id and moving backwards in `events`,
find the first TokenUsage record (if any) associated either with:
- tool_call_metadata.model_response.id, or
- event.response_id
+2 -4
View File
@@ -94,8 +94,7 @@ def create_runtime(
def get_provider_tokens():
"""
Retrieve provider tokens from environment variables and return them as a dictionary.
"""Retrieve provider tokens from environment variables and return them as a dictionary.
Returns:
A dictionary mapping ProviderType to ProviderToken if tokens are found, otherwise None.
@@ -126,8 +125,7 @@ def initialize_repository_for_runtime(
immutable_provider_tokens: PROVIDER_TOKEN_TYPE | None = None,
selected_repository: str | None = None,
) -> str | None:
"""
Initialize the repository for the runtime by cloning or initializing it,
"""Initialize the repository for the runtime by cloning or initializing it,
running setup scripts, and setting up git hooks if present.
Args:
+3 -9
View File
@@ -6,25 +6,19 @@ from openhands.events import Event
class CriticResult(BaseModel):
"""
A critic result is a score and a message.
"""
"""A critic result is a score and a message."""
score: float
message: str
@property
def success(self) -> bool:
"""
Whether the agent is successful.
"""
"""Whether the agent is successful."""
return self.score >= 0.5
class BaseCritic(abc.ABC):
"""
A critic is a function that takes in a list of events, optional git patch, and returns a score about the quality of those events.
"""
"""A critic is a function that takes in a list of events, optional git patch, and returns a score about the quality of those events."""
@abc.abstractmethod
def evaluate(
+1 -2
View File
@@ -42,8 +42,7 @@ class MessageAction(Action):
@dataclass
class SystemMessageAction(Action):
"""
Action that represents a system message for an agent, including the system prompt
"""Action that represents a system message for an agent, including the system prompt
and available tools. This should be the first message in the event stream.
"""
+2 -6
View File
@@ -42,9 +42,7 @@ _DUMMY_PAGE = _CachePage(None, 1, -1)
@dataclass
class EventStore(EventStoreABC):
"""
A stored list of events backing a conversation
"""
"""A stored list of events backing a conversation"""
sid: str
file_store: FileStore
@@ -92,8 +90,7 @@ class EventStore(EventStoreABC):
filter: EventFilter | None = None,
limit: int | None = None,
) -> Iterable[Event]:
"""
Retrieve events from the event stream, optionally filtering out events of a given type
"""Retrieve events from the event stream, optionally filtering out events of a given type
and events marked as hidden.
Args:
@@ -105,7 +102,6 @@ class EventStore(EventStoreABC):
Yields:
Events from the stream that match the criteria.
"""
if end_id is None:
end_id = self.cur_id
else:
+2 -5
View File
@@ -9,9 +9,7 @@ from openhands.events.event_filter import EventFilter
class EventStoreABC:
"""
A stored list of events backing a conversation
"""
"""A stored list of events backing a conversation"""
sid: str
user_id: str | None
@@ -25,8 +23,7 @@ class EventStoreABC:
filter: EventFilter | None = None,
limit: int | None = None,
) -> Iterable[Event]:
"""
Retrieve events from the event stream, optionally excluding events using a filter
"""Retrieve events from the event stream, optionally excluding events using a filter
Args:
start_id: The ID of the first event to retrieve. Defaults to 0.
+1 -3
View File
@@ -13,9 +13,7 @@ from openhands.events.serialization.event import event_from_dict
@dataclass
class NestedEventStore(EventStoreABC):
"""
A stored list of events backing a conversation
"""
"""A stored list of events backing a conversation"""
base_url: str
sid: str
+1 -2
View File
@@ -46,8 +46,7 @@ class AgentThinkObservation(Observation):
@dataclass
class MicroagentKnowledge:
"""
Represents knowledge from a triggered microagent.
"""Represents knowledge from a triggered microagent.
Attributes:
name: The name of the microagent that was triggered
-1
View File
@@ -146,7 +146,6 @@ class CmdOutputObservation(Observation):
Returns:
Original content if not too large, or truncated content otherwise
"""
if len(content) <= max_size:
return content
+2 -1
View File
@@ -12,7 +12,8 @@ from openhands.events.observation import (
def get_pairs_from_events(events: list[Event]) -> list[tuple[Action, Observation]]:
"""Return the history as a list of tuples (action, observation).
This function is a compatibility function for evals reading and visualization working with old histories."""
This function is a compatibility function for evals reading and visualization working with old histories.
"""
tuples: list[tuple[Action, Observation]] = []
action_map: dict[int, Action] = {}
observation_map: dict[int, Observation] = {}
+35 -3
View File
@@ -1,11 +1,33 @@
import os
from pydantic import BaseModel
from openhands.core.config.openhands_config import OpenHandsConfig
from openhands.core.logger import openhands_logger as logger
from openhands.server.session.conversation_init_data import ConversationInitData
from openhands.server.shared import file_store
from openhands.storage.locations import get_experiment_config_filename
from openhands.utils.import_utils import get_impl
class ExperimentConfig(BaseModel):
config: dict[str, str] | None = None
def load_experiment_config(conversation_id: str) -> ExperimentConfig | None:
try:
file_path = get_experiment_config_filename(conversation_id)
exp_config = file_store.read(file_path)
return ExperimentConfig.model_validate_json(exp_config)
except FileNotFoundError:
pass
except Exception as e:
logger.warning(f'Failed to load experiment config: {e}')
return None
class ExperimentManager:
@staticmethod
def run_conversation_variant_test(
@@ -17,9 +39,19 @@ class ExperimentManager:
def run_config_variant_test(
user_id: str, conversation_id: str, config: OpenHandsConfig
) -> OpenHandsConfig:
logger.debug(
f'Running agent config variant test for user_id={user_id}, conversation_id={conversation_id}'
)
exp_config = load_experiment_config(conversation_id)
if exp_config and exp_config.config:
agent_cfg = config.get_agent_config(config.default_agent)
try:
for attr, value in exp_config.config.items():
if hasattr(agent_cfg, attr):
logger.info(
f'Set attrib {attr} to {value} for {conversation_id}'
)
setattr(agent_cfg, attr, value)
except Exception as e:
logger.warning(f'Error processing exp config: {e}')
return config
@@ -191,8 +191,7 @@ class BitBucketService(BaseGitService, GitService, InstallationsService):
def _parse_repository(
self, repo: dict, link_header: str | None = None
) -> Repository:
"""
Parse a Bitbucket API repository response into a Repository object.
"""Parse a Bitbucket API repository response into a Repository object.
Args:
repo: Repository data from Bitbucket API
@@ -201,7 +200,6 @@ class BitBucketService(BaseGitService, GitService, InstallationsService):
Returns:
Repository object
"""
repo_id = repo.get('uuid', '')
workspace_slug = repo.get('workspace', {}).get('slug', '')
@@ -292,8 +290,7 @@ class BitBucketService(BaseGitService, GitService, InstallationsService):
async def _fetch_paginated_data(
self, url: str, params: dict, max_items: int
) -> list[dict]:
"""
Fetch data with pagination support for Bitbucket API.
"""Fetch data with pagination support for Bitbucket API.
Args:
url: The API endpoint URL
@@ -186,8 +186,7 @@ class GitHubService(BaseGitService, GitService, InstallationsService):
async def _fetch_paginated_repos(
self, url: str, params: dict, max_repos: int, extract_key: str | None = None
) -> list[dict]:
"""
Fetch repositories with pagination support.
"""Fetch repositories with pagination support.
Args:
url: The API endpoint URL
@@ -228,8 +227,7 @@ class GitHubService(BaseGitService, GitService, InstallationsService):
def _parse_repository(
self, repo: dict, link_header: str | None = None
) -> Repository:
"""
Parse a GitHub API repository response into a Repository object.
"""Parse a GitHub API repository response into a Repository object.
Args:
repo: Repository data from GitHub API
@@ -550,8 +548,7 @@ class GitHubService(BaseGitService, GitService, InstallationsService):
draft: bool = True,
labels: list[str] | None = None,
) -> str:
"""
Creates a PR using user credentials
"""Creates a PR using user credentials
Args:
repo_name: The full name of the repository (owner/repo)
@@ -566,7 +563,6 @@ class GitHubService(BaseGitService, GitService, InstallationsService):
- PR URL when successful
- Error message when unsuccessful
"""
url = f'{self.BASE_URL}/repos/{repo_name}/pulls'
# Set default body if none provided
@@ -71,9 +71,7 @@ class GitLabService(BaseGitService, GitService):
return ProviderType.GITLAB.value
async def _get_gitlab_headers(self) -> dict[str, Any]:
"""
Retrieve the GitLab Token to construct the headers
"""
"""Retrieve the GitLab Token to construct the headers"""
if not self.token:
latest_token = await self.get_latest_token()
if latest_token:
@@ -173,8 +171,7 @@ class GitLabService(BaseGitService, GitService):
async def execute_graphql_query(
self, query: str, variables: dict[str, Any] | None = None
) -> Any:
"""
Execute a GraphQL query against the GitLab GraphQL API
"""Execute a GraphQL query against the GitLab GraphQL API
Args:
query: The GraphQL query string
@@ -244,8 +241,7 @@ class GitLabService(BaseGitService, GitService):
def _parse_repository(
self, repo: dict, link_header: str | None = None
) -> Repository:
"""
Parse a GitLab API project response into a Repository object.
"""Parse a GitLab API project response into a Repository object.
Args:
repo: Project data from GitLab API
@@ -269,8 +265,7 @@ class GitLabService(BaseGitService, GitService):
)
def _parse_gitlab_url(self, url: str) -> str | None:
"""
Parse a GitLab URL to extract the repository path.
"""Parse a GitLab URL to extract the repository path.
Expected format: https://{domain}/{group}/{possibly_subgroup}/{repo}
Returns the full path from group onwards (e.g., 'group/subgroup/repo' or 'group/repo')
@@ -588,8 +583,7 @@ class GitLabService(BaseGitService, GitService):
description: str | None = None,
labels: list[str] | None = None,
) -> str:
"""
Creates a merge request in GitLab
"""Creates a merge request in GitLab
Args:
id: The ID or URL-encoded path of the project
@@ -603,7 +597,6 @@ class GitLabService(BaseGitService, GitService):
- MR URL when successful
- Error message when unsuccessful
"""
# Convert string ID to URL-encoded path if needed
project_id = str(id).replace('/', '%2F') if isinstance(id, str) else id
url = f'{self.BASE_URL}/projects/{project_id}/merge_requests'
+8 -22
View File
@@ -191,10 +191,7 @@ class ProviderHandler:
per_page: int | None,
installation_id: str | None,
) -> list[Repository]:
"""
Get repositories from providers
"""
"""Get repositories from providers"""
"""
Get repositories from providers
"""
@@ -226,9 +223,7 @@ class ProviderHandler:
return all_repos
async def get_suggested_tasks(self) -> list[SuggestedTask]:
"""
Get suggested tasks from providers
"""
"""Get suggested tasks from providers"""
tasks: list[SuggestedTask] = []
for provider in self.provider_tokens:
try:
@@ -303,8 +298,7 @@ class ProviderHandler:
event_stream: EventStream,
env_vars: dict[ProviderType, SecretStr] | None = None,
) -> None:
"""
This ensures that the latest provider tokens are masked from the event stream
"""This ensures that the latest provider tokens are masked from the event stream
It is called when the provider tokens are first initialized in the runtime or when tokens are re-exported with the latest working ones
Args:
@@ -320,8 +314,7 @@ class ProviderHandler:
def expose_env_vars(
self, env_secrets: dict[ProviderType, SecretStr]
) -> dict[str, str]:
"""
Return string values instead of typed values for environment secrets
"""Return string values instead of typed values for environment secrets
Called just before exporting secrets to runtime, or setting secrets in the event stream
"""
exposed_envs = {}
@@ -353,8 +346,7 @@ class ProviderHandler:
providers: list[ProviderType] | None = None,
get_latest: bool = False,
) -> dict[ProviderType, SecretStr] | dict[str, str]:
"""
Retrieves the provider tokens from ProviderHandler object
"""Retrieves the provider tokens from ProviderHandler object
This is used when initializing/exporting new provider tokens in the runtime
Args:
@@ -362,7 +354,6 @@ class ProviderHandler:
providers: Return provider tokens for the list passed in, otherwise return all available providers
get_latest: Get the latest working token for the providers if True, otherwise get the existing ones
"""
if not self.provider_tokens:
return {}
@@ -393,11 +384,9 @@ class ProviderHandler:
def check_cmd_action_for_provider_token_ref(
cls, event: Action
) -> list[ProviderType]:
"""
Detect if agent run action is using a provider token (e.g $GITHUB_TOKEN)
"""Detect if agent run action is using a provider token (e.g $GITHUB_TOKEN)
Returns a list of providers which are called by the agent
"""
if not isinstance(event, CmdRunAction):
return []
@@ -410,9 +399,7 @@ class ProviderHandler:
@classmethod
def get_provider_env_key(cls, provider: ProviderType) -> str:
"""
Map ProviderType value to the environment variable name in the runtime
"""
"""Map ProviderType value to the environment variable name in the runtime"""
return f'{provider.value}_token'.lower()
async def verify_repo_provider(
@@ -443,8 +430,7 @@ class ProviderHandler:
async def get_branches(
self, repository: str, specified_provider: ProviderType | None = None
) -> list[Branch]:
"""
Get branches for a repository
"""Get branches for a repository
Args:
repository: The repository name
@@ -12,6 +12,11 @@ A comment on the issue has been addressed to you.
When you're done, make sure to
1. Re-read the issue title, body, and comments and make sure that you have successfully implemented all requirements.
2. Use the `create_pr` tool to open a new PR
3. Name the branch using `openhands/` as a prefix (e.g `openhands/update-readme`)
4. The PR description should mention that it "fixes" or "closes" the issue number
2. Create a new branch using `openhands/` as a prefix (e.g `openhands/update-readme`)
3. Commit your changes with a clear commit message
4. Push the branch to GitHub
5. Use the `create_pr` tool to open a new PR
6. The PR description should:
- Follow the repository's PR template (check `.github/pull_request_template.md` if it exists)
- Mention that it "fixes" or "closes" the issue number
- Include all required sections from the template
@@ -7,5 +7,11 @@ Your tasking is to fix an issue in your repository. Do the following
When you're done, make sure to
1. Use the `create_pr` tool to open a new PR
2. The PR description should mention that it "fixes" or "closes" the issue number
1. Create a new branch with a descriptive name (e.g., `openhands/fix-issue-123`)
2. Commit your changes with a clear commit message
3. Push the branch to GitHub
4. Use the `create_pr` tool to open a new PR
5. The PR description should:
- Follow the repository's PR template (check `.github/pull_request_template.md` if it exists)
- Mention that it "fixes" or "closes" the issue number
- Include all required sections from the template
@@ -17,5 +17,6 @@ If it's a question:
If it requests a code update:
1. Modify the code accordingly in the current branch
2. Push the changes to update the PR
3. DO NOT leave any comments on the PR
2. Commit your changes with a clear commit message
3. Push the changes to GitHub to update the PR
4. DO NOT leave any comments on the PR
@@ -12,6 +12,11 @@ A comment on the issue has been addressed to you.
When you're done, make sure to
1. Re-read the issue title, body, and comments and make sure that you have successfully implemented all requirements.
2. Use the `create_mr` tool to open a new MR
3. Name the branch using `openhands/` as a prefix (e.g `openhands/update-readme`)
4. The MR description should mention that it "fixes" or "closes" the issue number
2. Create a new branch using `openhands/` as a prefix (e.g `openhands/update-readme`)
3. Commit your changes with a clear commit message
4. Push the branch to GitLab
5. Use the `create_mr` tool to open a new MR
6. The MR description should:
- Follow the repository's MR template (check `.gitlab/merge_request_templates/` or `.github/pull_request_template.md` if it exists)
- Mention that it "fixes" or "closes" the issue number
- Include all required sections from the template
@@ -7,5 +7,11 @@ Your tasking is to fix an issue in your repository. Do the following
When you're done, make sure to
1. Use the `create_mr` tool to open a new MR
2. The MR description should mention that it "fixes" or "closes" the issue number
1. Create a new branch with a descriptive name (e.g., `openhands/fix-issue-123`)
2. Commit your changes with a clear commit message
3. Push the branch to GitLab
4. Use the `create_mr` tool to open a new MR
5. The MR description should:
- Follow the repository's MR template (check `.gitlab/merge_request_templates/` or `.github/pull_request_template.md` if it exists)
- Mention that it "fixes" or "closes" the issue number
- Include all required sections from the template
@@ -17,5 +17,6 @@ If it's a question:
If it requests a code update:
1. Modify the code accordingly in the current branch
2. Push the changes to update the MR
3. DO NOT leave any comments on the MR
2. Commit your changes with a clear commit message
3. Push the changes to GitLab to update the MR
4. DO NOT leave any comments on the MR
@@ -5,3 +5,13 @@ These are a list of text messages attached in order of most recent.
{{ message }}
{% if not loop.last %}\n\n{% endif %}
{% endfor %}
If you made code changes, when you're done make sure to:
1. Commit your changes with a clear commit message
2. **Ask the user in your summary** whether they want you to:
- Push the changes to a new branch and create a pull request (following the repository's PR template if it exists)
- Just keep the changes local for now
- Any other specific Git workflow they prefer
Do NOT automatically push or create pull requests without explicit user confirmation.
+1 -2
View File
@@ -10,8 +10,7 @@ from openhands.integrations.provider import ProviderType
async def validate_provider_token(
token: SecretStr, base_domain: str | None = None
) -> ProviderType | None:
"""
Determine whether a token is for GitHub, GitLab, or Bitbucket by attempting to get user info
"""Determine whether a token is for GitHub, GitLab, or Bitbucket by attempting to get user info
from the services.
Args:

Some files were not shown because too many files have changed in this diff Show More