github/OpenHands
1
1
Fork 0
mirror of https://github.com/All-Hands-AI/OpenHands.git synced 2026-01-10 07:18:10 -05:00
Code Issues Packages Projects Releases Wiki Activity

Adding LLM Based Editing capability (#8677)

Browse Source 
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Engel Nyst <engel.nyst@gmail.com>
This commit is contained in:
Leander Maben
2025-06-09 09:57:20 -04:00
committed by GitHub
parent 4eef22e04e
commit d84befe28f
7 changed files with 119 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
evaluation/README.md
View File

@@ -74,6 +74,24 @@ If no condenser configuration is specified, the 'noop' condenser will be used by
For other configurations specific to evaluation, such as `save_trajectory_path`, these are typically set in the `get_config` function of the respective `run_infer.py` file for each benchmark.
### Enabling LLM-Based Editor Tools
The LLM-Based Editor tool (currently supported only for SWE-Bench) can be enabled by setting:
```bash
export ENABLE_LLM_EDITOR=true
```
You can set the config for the Editor LLM as:
```toml
[llm.draft_editor]
base_url = "http://localhost:9002/v1"
model = "hosted_vllm/lite_coder_qwen_editor_3B"
api_key = ""
temperature = 0.7
max_input_tokens = 10500
max_output_tokens = 10500
```
## Supported Benchmarks
The OpenHands evaluation harness supports a wide variety of benchmarks across [software engineering](#software-engineering), [web browsing](#web-browsing), [miscellaneous assistance](#misc-assistance), and [real-world](#real-world) tasks.