github/OpenHands
1
1
Fork 0
mirror of https://github.com/All-Hands-AI/OpenHands.git synced 2026-01-09 14:57:59 -05:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md (#1825)

Browse Source
This commit is contained in:
Xingyao Wang
2024-05-16 19:06:28 +08:00
committed by GitHub
parent 7313421ae4
commit 0fdbe1ee93
1 changed files with 11 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
evaluation/swe_bench/README.md
View File

@@ -28,17 +28,9 @@ To reproduce how we pack the image, check [this doc](./BUILD_TESTBED_AND_ENV.md)
NOTE: We only support SWE-Bench lite for now. But modifying our existing scripts for full SWE-Bench should be quite straight forward.
## Test if your environment works
## Configure OpenDevin and your LLM
```bash
python3 evaluation/swe_bench/swe_env_box.py
```
If you get to the interactive shell successfully, it means success!
## Configure your LLM
Create a `config.toml` file if not exists at the root of workspace.
Create a `config.toml` file if it does not exist at the root of the workspace.
Add the following configurations:
@@ -51,7 +43,7 @@ sandbox_type = "ssh"
use_host_network = true
ssh_hostname = "localhost"
sandbox_timeout = 120
# eval specific
# SWEBench eval specific
run_as_devin = false
# TODO: Change these to the model you want to evaluate
@@ -67,6 +59,14 @@ api_key = "XXX"
temperature = 0.0
```
## Test if your environment works
```bash
python3 evaluation/swe_bench/swe_env_box.py
```
If you get to the interactive shell successfully, it means success!
## Run Inference on SWE-Bench Instances
```bash