Compare commits

...

4 Commits

Author SHA1 Message Date
Robert Brennan
7f57c2ce64 more reverst 2024-08-26 11:54:02 -04:00
Robert Brennan
9f880f7690 revert some files 2024-08-26 11:53:14 -04:00
Robert Brennan
d69228b20c fix perms 2024-08-26 11:52:13 -04:00
Robert Brennan
4340da8973 replace most references to OD 2024-08-26 11:50:23 -04:00
20 changed files with 46 additions and 41 deletions

View File

@@ -61,7 +61,7 @@ Félicitations !
Le code pertinent est défini dans [ssh_box.py](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/ssh_box.py) et [image_agnostic_util.py](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/image_agnostic_util.py).
En particulier, ssh_box.py vérifie l'objet config pour ```config.sandbox.base_container_image``` et ensuite tente de récupérer l'image à l'aide de [get_od_sandbox_image](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/image_agnostic_util.py#L72), qui est défini dans image_agnostic_util.py.
En particulier, ssh_box.py vérifie l'objet config pour ```config.sandbox.base_container_image``` et ensuite tente de récupérer l'image à l'aide de [get_openhands_sandbox_image](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/image_agnostic_util.py#L72), qui est défini dans image_agnostic_util.py.
Lorsqu'une image personnalisée est utilisée pour la première fois, elle ne sera pas trouvée et donc elle sera construite (à l'exécution ultérieure, l'image construite sera trouvée et renvoyée).

View File

@@ -60,7 +60,7 @@ base_container_image="custom_image"
相关代码定义在 [ssh_box.py](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/ssh_box.py) 和 [image_agnostic_util.py](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/image_agnostic_util.py) 中。
特别是 ssh_box.py 检查配置对象中的 ```config.sandbox.base_container_image```,然后尝试使用 [get_od_sandbox_image](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/image_agnostic_util.py#L72),在 image_agnostic_util.py 定义中进行检索。
特别是 ssh_box.py 检查配置对象中的 ```config.sandbox.base_container_image```,然后尝试使用 [get_openhands_sandbox_image](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/image_agnostic_util.py#L72),在 image_agnostic_util.py 定义中进行检索。
初次使用自定义映像时,该映像将不会被找到,因此将被构建(在后续运行中已构建的映像将被查找并返回)。

View File

@@ -91,9 +91,9 @@ OpenHands uses a dual-tagging system for its runtime images to balance reproduci
- This ensures reproducibility: the same hash always means the same image contents.
2. Generic tag: `{target_image_repo}:{target_image_tag}`
Example: `runtime:od_v0.8.3_ubuntu_tag_22.04`
Example: `runtime:openhands_v0.8.3_ubuntu_tag_22.04`
- This tag follows the format: `runtime:od_v{OD_VERSION}_{BASE_IMAGE_NAME}_tag_{BASE_IMAGE_TAG}`
- This tag follows the format: `runtime:openhands_v{OPENHANDS_VERSION}_{BASE_IMAGE_NAME}_tag_{BASE_IMAGE_TAG}`
- It represents the latest build for a particular base image and OpenHands version combination.
- This tag is updated whenever a new image is built from the same base image, even if the source code changes.
@@ -105,11 +105,11 @@ The hash-based tag ensures exact reproducibility, while the generic tag provides
- Hash-based tag: `{target_image_repo}:{target_image_hash_tag}`
Example: `runtime:abc123def456`
- Generic tag: `{target_image_repo}:{target_image_tag}`
Example: `runtime:od_v0.8.3_ubuntu_tag_22.04`
Example: `runtime:openhands_v0.8.3_ubuntu_tag_22.04`
2. Build Process:
- a. Convert the base image name to an OD runtime image name.
Example: `ubuntu:22.04` -> `runtime:od_v0.8.3_ubuntu_tag_22.04`
Example: `ubuntu:22.04` -> `runtime:openhands_v0.8.3_ubuntu_tag_22.04`
- b. Generate a build context (Dockerfile and OpenHands source code) and calculate its hash.
- c. Check for an existing image with the calculated hash.
- d. If not found, check for a recent compatible image to use as a base.
@@ -121,7 +121,7 @@ The hash-based tag ensures exact reproducibility, while the generic tag provides
a. If an image exists with the same hash (e.g., `runtime:abc123def456`), it will be reused as is.
b. If the exact hash is not found, the system will try to rebuild using the latest generic image (e.g., `runtime:od_v0.8.3_ubuntu_tag_22.04`) as a base. This saves time by leveraging existing dependencies.
b. If the exact hash is not found, the system will try to rebuild using the latest generic image (e.g., `runtime:openhands_v0.8.3_ubuntu_tag_22.04`) as a base. This saves time by leveraging existing dependencies.
c. If neither the hash-tagged nor the generic-tagged image is found, the system will build the image completely from scratch.
@@ -135,10 +135,10 @@ Here's a flowchart illustrating the build process:
```mermaid
flowchart TD
A[Start] --> B{Convert base image name}
B --> |ubuntu:22.04 -> runtime:od_v0.8.3_ubuntu_tag_22.04| C[Generate build context and hash]
B --> |ubuntu:22.04 -> runtime:openhands_v0.8.3_ubuntu_tag_22.04| C[Generate build context and hash]
C --> D{Check for existing image with hash}
D -->|Found runtime:abc123def456| E[Use existing image]
D -->|Not found| F{Check for runtime:od_v0.8.3_ubuntu_tag_22.04}
D -->|Not found| F{Check for runtime:openhands_v0.8.3_ubuntu_tag_22.04}
F -->|Found| G[Rebuild based on recent image]
F -->|Not found| H[Build from scratch]
G --> I[Tag with hash and generic tags]
@@ -151,7 +151,7 @@ This approach ensures that:
1. Identical source code and Dockerfile always produce the same image (via hash-based tags).
2. The system can quickly rebuild images when minor changes occur (by leveraging recent compatible images).
3. The generic tag (e.g., `runtime:od_v0.8.3_ubuntu_tag_22.04`) always points to the latest build for a particular base image and OpenHands version combination.
3. The generic tag (e.g., `runtime:openhands_v0.8.3_ubuntu_tag_22.04`) always points to the latest build for a particular base image and OpenHands version combination.
By using this method, OpenHands maintains an efficient and flexible system for building and managing runtime images, adapting to both development needs and production requirements.

View File

@@ -2,4 +2,4 @@ FROM python:3.11-bookworm
RUN pip install scitools-pyke
# docker build -t xingyaoww/od_logic_reasoning .
# docker build -t xingyaoww/openhands_logic_reasoning .

View File

@@ -52,7 +52,7 @@ def get_config(
base_container_image='xingyaoww/od-eval-logic-reasoning:v1.0',
enable_auto_lint=True,
use_host_network=False,
runtime_extra_deps='$OD_INTERPRETER_PATH -m pip install scitools-pyke',
runtime_extra_deps='$OPENHANDS_INTERPRETER_PATH -m pip install scitools-pyke',
),
# do not mount workspace
workspace_base=None,

View File

@@ -104,7 +104,7 @@ def get_config(
base_container_image='xingyaoww/od-eval-mint:v1.0',
enable_auto_lint=True,
use_host_network=False,
runtime_extra_deps=f'$OD_INTERPRETER_PATH -m pip install {" ".join(MINT_DEPENDENCIES)}',
runtime_extra_deps=f'$OPENHANDS_INTERPRETER_PATH -m pip install {" ".join(MINT_DEPENDENCIES)}',
),
# do not mount workspace
workspace_base=None,

0
evaluation/ml_bench/scripts/cleanup.sh Normal file → Executable file
View File

0
evaluation/ml_bench/scripts/run_analysis.sh Normal file → Executable file
View File

View File

@@ -4,14 +4,16 @@ import os
import pandas as pd
parser = argparse.ArgumentParser()
parser.add_argument('od_output_file', type=str)
parser.add_argument('openhands_output_file', type=str)
args = parser.parse_args()
output_filepath = args.od_output_file.replace('.jsonl', '.swebench.jsonl')
print(f'Converting {args.od_output_file} to {output_filepath}')
output_filepath = args.openhands_output_file.replace('.jsonl', '.swebench.jsonl')
print(f'Converting {args.openhands_output_file} to {output_filepath}')
od_format = pd.read_json(args.od_output_file, orient='records', lines=True)
# model name is the folder name of od_output_file
model_name = os.path.basename(os.path.dirname(args.od_output_file))
openhands_format = pd.read_json(
args.openhands_output_file, orient='records', lines=True
)
# model name is the folder name of openhands_output_file
model_name = os.path.basename(os.path.dirname(args.openhands_output_file))
def process_git_patch(patch):
@@ -59,5 +61,5 @@ def convert_row_to_swebench_format(row):
}
swebench_format = od_format.apply(convert_row_to_swebench_format, axis=1)
swebench_format = openhands_format.apply(convert_row_to_swebench_format, axis=1)
swebench_format.to_json(output_filepath, lines=True, orient='records')

View File

@@ -58,7 +58,7 @@ else
# ==== Convert OD format to SWE-bench format ====
echo "Merged output file with fine-grained report will be saved to $FILE_DIR"
poetry run python3 evaluation/swe_bench/scripts/eval/convert_od_output_to_swe_json.py $PROCESS_FILEPATH
poetry run python3 evaluation/swe_bench/scripts/eval/convert_openhands_output_to_swe_json.py $PROCESS_FILEPATH
# replace .jsonl with .swebench.jsonl in filename
SWEBENCH_FORMAT_JSONL=${PROCESS_FILEPATH/.jsonl/.swebench.jsonl}
echo "SWEBENCH_FORMAT_JSONL: $SWEBENCH_FORMAT_JSONL"

View File

@@ -19,10 +19,10 @@ def extract_modified_files(patch):
return modified_files
def process_report(od_output_file):
def process_report(openhands_output_file):
succ = 0
fail = 0
for line in open(od_output_file):
for line in open(openhands_output_file):
line = json.loads(line)
instance_id = line['instance_id']
gold_patch = line['swe_instance']['patch']
@@ -48,7 +48,7 @@ def process_report(od_output_file):
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--od_output_file', help='Path to the OD output file')
parser.add_argument('--openhands_output_file', help='Path to the OD output file')
args = parser.parse_args()
process_report(args.od_output_file)
process_report(args.openhands_output_file)

View File

View File

@@ -6,9 +6,9 @@ mkdir -p $EVAL_WORKSPACE
# 1. Prepare REPO
echo "==== Prepare SWE-bench repo ===="
OD_SWE_BENCH_REPO_PATH="https://github.com/All-Hands-AI/OD-SWE-bench.git"
OD_SWE_BENCH_REPO_BRANCH="eval"
git clone -b $OD_SWE_BENCH_REPO_BRANCH $OD_SWE_BENCH_REPO_PATH $EVAL_WORKSPACE/OD-SWE-bench
OPENHANDS_SWE_BENCH_REPO_PATH="https://github.com/All-Hands-AI/OD-SWE-bench.git"
OPENHANDS_SWE_BENCH_REPO_BRANCH="eval"
git clone -b $OPENHANDS_SWE_BENCH_REPO_BRANCH $OPENHANDS_SWE_BENCH_REPO_PATH $EVAL_WORKSPACE/OD-SWE-bench
# 2. Prepare DATA
echo "==== Prepare SWE-bench data ===="

View File

@@ -189,8 +189,8 @@ class SandboxConfig(metaclass=Singleton):
runtime_extra_deps: The extra dependencies to install in the runtime image (typically used for evaluation).
This will be rendered into the end of the Dockerfile that builds the runtime image.
It can contain any valid shell commands (e.g., pip install numpy).
The path to the interpreter is available as $OD_INTERPRETER_PATH,
which can be used to install dependencies for the OD-specific Python interpreter.
The path to the interpreter is available as $OPENHANDS_INTERPRETER_PATH,
which can be used to install dependencies for the OpenHands-specific Python interpreter.
runtime_startup_env_vars: The environment variables to set at the launch of the runtime.
This is a dictionary of key-value pairs.
This is useful for setting environment variables that are needed by the runtime.

View File

@@ -115,7 +115,7 @@ class EventStreamRuntime(Runtime):
self.instance_id = (
sid + '_' + str(uuid.uuid4()) if sid is not None else str(uuid.uuid4())
)
# TODO: We can switch to aiodocker when `get_od_sandbox_image` is updated to use aiodocker
# TODO: We can switch to aiodocker when `get_openhands_sandbox_image` is updated to use aiodocker
self.docker_client: docker.DockerClient = self._init_docker_client()
self.base_container_image = self.config.sandbox.base_container_image
self.runtime_container_image = self.config.sandbox.runtime_container_image

0
openhands/runtime/plugins/jupyter/execute_server.py Executable file → Normal file
View File

View File

@@ -14,7 +14,7 @@ from openhands.core.logger import openhands_logger as logger
from openhands.runtime.builder import DockerRuntimeBuilder, RuntimeBuilder
RUNTIME_IMAGE_REPO = os.getenv(
'OD_RUNTIME_RUNTIME_IMAGE_REPO', 'ghcr.io/all-hands-ai/runtime'
'OPENHANDS_RUNTIME_RUNTIME_IMAGE_REPO', 'ghcr.io/all-hands-ai/runtime'
)
@@ -200,8 +200,11 @@ def get_runtime_image_repo_and_tag(base_image: str) -> tuple[str, str]:
base_image = base_image + ':latest'
[repo, tag] = base_image.split(':')
repo = repo.replace('/', '___')
od_version = _get_package_version()
return RUNTIME_IMAGE_REPO, f'od_v{od_version}_image_{repo}_tag_{tag}'
openhands_version = _get_package_version()
return (
RUNTIME_IMAGE_REPO,
f'openhands_v{openhands_version}_image_{repo}_tag_{tag}',
)
def build_runtime_image(
@@ -360,7 +363,7 @@ def _build_sandbox_image(
on the contents of the docker build folder (source code and Dockerfile)
e.g. 1234567890abcdef
-target_image_tag (str): the tag for the target image that's generic and based on the base image name
e.g. od_v0.8.3_image_ubuntu_tag_22.04
e.g. openhands_v0.8.3_image_ubuntu_tag_22.04
"""
target_image_hash_name = f'{target_image_repo}:{target_image_hash_tag}'
target_image_generic_name = f'{target_image_repo}:{target_image_tag}'

View File

@@ -56,7 +56,7 @@ RUN cd /openhands/code && \
apt-get update && \
/openhands/miniforge3/bin/mamba run -n base poetry run pip install playwright && \
/openhands/miniforge3/bin/mamba run -n base poetry run playwright install --with-deps chromium && \
export OD_INTERPRETER_PATH=$(/openhands/miniforge3/bin/mamba run -n base poetry run python -c "import sys; print(sys.executable)") && \
export OPENHANDS_INTERPRETER_PATH=$(/openhands/miniforge3/bin/mamba run -n base poetry run python -c "import sys; print(sys.executable)") && \
{{ extra_deps }} {% if extra_deps %} && {% endif %} \
/openhands/miniforge3/bin/mamba run -n base poetry cache clear --all . && \
{% if not skip_init %}chmod -R g+rws /openhands/poetry && {% endif %} \

0
tests/integration/regenerate.sh Executable file → Normal file
View File

View File

@@ -17,7 +17,7 @@ from openhands.runtime.utils.runtime_build import (
prep_docker_build_folder,
)
OD_VERSION = f'od_v{_get_package_version()}'
OPENHANDS_VERSION = f'openhands_v{_get_package_version()}'
@pytest.fixture
@@ -176,7 +176,7 @@ def test_get_runtime_image_repo_and_tag_eventstream():
img_repo, img_tag = get_runtime_image_repo_and_tag(base_image)
assert (
img_repo == f'{RUNTIME_IMAGE_REPO}'
and img_tag == f'{OD_VERSION}_image_debian_tag_11'
and img_tag == f'{OPENHANDS_VERSION}_image_debian_tag_11'
)
base_image = 'nikolaik/python-nodejs:python3.11-nodejs22'
@@ -184,14 +184,14 @@ def test_get_runtime_image_repo_and_tag_eventstream():
assert (
img_repo == f'{RUNTIME_IMAGE_REPO}'
and img_tag
== f'{OD_VERSION}_image_nikolaik___python-nodejs_tag_python3.11-nodejs22'
== f'{OPENHANDS_VERSION}_image_nikolaik___python-nodejs_tag_python3.11-nodejs22'
)
base_image = 'ubuntu'
img_repo, img_tag = get_runtime_image_repo_and_tag(base_image)
assert (
img_repo == f'{RUNTIME_IMAGE_REPO}'
and img_tag == f'{OD_VERSION}_image_ubuntu_tag_latest'
and img_tag == f'{OPENHANDS_VERSION}_image_ubuntu_tag_latest'
)
@@ -215,7 +215,7 @@ def test_build_runtime_image_from_scratch(temp_dir):
path=ANY,
tags=[
f'{RUNTIME_IMAGE_REPO}:{from_scratch_hash}',
f'{RUNTIME_IMAGE_REPO}:{OD_VERSION}_image_debian_tag_11',
f'{RUNTIME_IMAGE_REPO}:{OPENHANDS_VERSION}_image_debian_tag_11',
],
)
assert image_name == f'{RUNTIME_IMAGE_REPO}:{from_scratch_hash}'