* [eval] increase timeout for swebench eval init/complete
* allow CmdRunAction to optionally block when .timeout is setted
* fix unit test for serialization
* fix unit tests for security analyzer
* fix integration tests
* add more timeout
* only check P2P when instances are non-empty;
convert P2P and F2P columns to string instead of list
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
* [eval] increase timeout for swebench eval init/complete
* allow CmdRunAction to optionally block when .timeout is setted
* fix unit test for serialization
* fix unit tests for security analyzer
* fix integration tests
* add more timeout
* Fail the Runtime tests check if the previous jobs were cancelled or failed
* Check only for failures
* Add test exit
* Test cancel
* fix conditions
* fix condition again
* Try another way
* Now test success
* move filematching logic into server
* wait until ready before returning
* show loading message instead of empty
* logspam
* delint
* fix type
* add a few more default ignores
* Reduce runtime tests duration by running them across CPUs
* fix hardcoded image name
* test two cpus
* Test folder change
* Up the CPU to 4 again to test
* Change to 3 CPUs
* Down to 2
* Add param to remove all openhands containers
* Add comment
* Add reruns just in case
* Fix ordering of if
* Add Handling of Cache Prompt When Formatting Messages
* Fix Value for Cache Control
* Fix Value for Cache Control
* Update openhands/core/message.py
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
* Fix lint error
* Serialize Messages if Propt Caching Is Enabled
* Remove formatting message change
---------
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: tobitege <10787084+tobitege@users.noreply.github.com>
* dummy test change
* regen yml: 1st install python 3.11, then poetry
* fix caching for poetry; old entry for python was rather useless
* fix steps order (cache before poetry)
* add poetry caching to ghcr_runtime; fix fork conditions
* ghcr_runtime: more caching actions; condition fixes
* fix interim action error (order of steps)
* cache@v4 instead of v3
* fixed interim typo for 2 fork conditions
* runtime/test_env_vars: compacted multiple tests into one to reduce time
* ugh if fork condition changes again
* (feat) making prompt caching optional instead of enabled default
At present, only the Claude models support prompt caching as a experimental feature, therefore, this feature should be implemented as an optional setting rather than being enabled by default.
Signed-off-by: Yi Lin <teroincn@gmail.com>
* handle the conflict
* fix unittest mock return value
* fix lint error in whitespace
---------
Signed-off-by: Yi Lin <teroincn@gmail.com>
* Update docs on LLM providers for consistency
* Update headless command
* minor tweaks based on feedback
---------
Co-authored-by: Robert Brennan <contact@rbren.io>
Co-authored-by: Robert Brennan <accounts@rbren.io>
* feat: add SWE-bench fullset support
* fix instance image list
* update eval script and documentation
* increase timeout for remote runtime
* add push script
* handle the case when ret push is an generator
* update pbar
* set SWE-Bench default to run SWE-Bench lite
* add script to cleanup remote runtime
* fix the cases when tag is too long
* update README
* update readme for cleanup
* rename od to oh
* Update evaluation/swe_bench/README.md
Co-authored-by: Graham Neubig <neubig@gmail.com>
* Update evaluation/swe_bench/README.md
Co-authored-by: Graham Neubig <neubig@gmail.com>
* Update evaluation/swe_bench/scripts/cleanup_remote_runtime.sh
Co-authored-by: Graham Neubig <neubig@gmail.com>
* Update evaluation/swe_bench/scripts/cleanup_remote_runtime.sh
Co-authored-by: Graham Neubig <neubig@gmail.com>
* Update evaluation/swe_bench/scripts/cleanup_remote_runtime.sh
Co-authored-by: Graham Neubig <neubig@gmail.com>
* gets API key and Runtime from env var
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
* update badges
* fix badges
* better badges
* move credits
* more badge work
* add gh logo
* update some copy
* update logo
* fix height
* update text
* emdash
* remove cruft
* move title
* update links
* add hr
* white logo
* move some stuff to getting-started
* revert logo
* more copy changes
* minor tweaks
* fix sidebar
* explicit sidebar
* words
* fix tag
* fix how-to
* more docs work
* update styles
* fix up custom sandbox docs
* change eval title
* fix up getting-started
* fix getting started
* update to 0.9.2
* update screenshot
* add company link
* fix dark mode
* minor fixes
* update image
* update headless and cli docs
* update readme
* fix links
* revert package
* rename links
* fix links
* fix link
* chagne to claude
* Add documentation for CLI mode
Fixes#3703
Add documentation for CLI mode in OpenHands.
* **New Documentation**: Add `docs/modules/usage/how-to/cli-mode.md` to document CLI mode.
- Include instructions on starting an interactive OpenHands session via the command line.
- Explain the difference between CLI mode and headless mode.
- Provide examples of CLI commands and expected outputs.
* **Update Existing Documentation**: Modify `docs/modules/usage/how-to/headless-mode.md`.
- Clarify the difference between headless mode and CLI mode.
- Add a reference to the new CLI mode documentation.
* Update cli-mode.md
* Update headless-mode.md
---------
Co-authored-by: Robert Brennan <accounts@rbren.io>
Co-authored-by: tofarr <tofarr@gmail.com>
* update badges
* fix badges
* better badges
* move credits
* more badge work
* add gh logo
* update some copy
* update logo
* fix height
* update text
* emdash
* remove cruft
* move title
* update links
* add hr
* white logo
* move some stuff to getting-started
* revert logo
* more copy changes
* minor tweaks
* words
* fix getting started
* update to 0.9.2
* update screenshot
* minor fixes
* CodeActAgent: fix message prep if prompt caching is not supported
* fix python version in regen tests workflow
* fix in conftest "mock_completion" method
* add disable_vision to LLMConfig; revert change in message parsing in llm.py
* format messages in several files for completion
* refactored message(s) formatting (llm.py); added vision_is_active()
* fix a unit test
* regenerate: added LOG_TO_FILE and FORCE_REGENERATE env flags
* try to fix path to logs folder in workflow
* llm: prevent index error
* try FORCE_USE_LLM in regenerate
* tweaks everywhere...
* fix 2 random unit test errors :(
* added FORCE_REGENERATE_TESTS=true to regenerate CLI
* fix test_lint_file_fail_typescript again
* double-quotes for env vars in workflow; llm logger set to debug
* fix typo in regenerate
* regenerate iterations now 20; applied iteration counter fix by Li
* regenerate: pass FORCE_REGENERATE flag into env
* fixes for int tests. several mock files updated.
* browsing_agent: fix response_parser.py adding ) to empty response
* test_browse_internet: fix skipif and revert obsolete mock files
* regenerate: fi bracketing for http server start/kill conditions
* disable test_browse_internet for CodeAct*Agents; mock files updated after merge
* missed to include more mock files earlier
* reverts after review feedback from Li
* forgot one
* browsing agent test, partial fixes and updated mock files
* test_browse_internet works in my WSL now!
* adapt unit test test_prompt_caching.py
* add DEBUG to regenerate workflow command
* convert regenerate workflow params to inputs
* more integration test mock files updated
* more files
* test_prompt_caching: restored test_prompt_caching_headers purpose
* file_ops: fix potential exception, like "cross device copy"; fixed mock files accordingly
* reverts/changes wrt feedback from xingyao
* updated docs and config template
* code cleanup wrt review feedback
* Catch exception and return finish action with an exception message in case of exception in llm completion
* Remove exception logs
* Raise llm response error for any exception in llm completion
* Raise LLMResponseError from async completion and async streaming completion as well
* feat: add SWE-bench fullset support
* fix instance image list
* update eval script and documentation
* increase timeout for remote runtime
* add push script
* handle the case when ret push is an generator
* update pbar
* set SWE-Bench default to run SWE-Bench lite
* feat: add SWE-bench fullset support
* fix instance image list
* update eval script and documentation
* add push script
* handle the case when ret push is an generator
* update pbar
Fix a potential issue that might lead to file corruption when edit linting is enabled
#3124 introduces a feature for editing: running linter twice before and after the change and only extract new errors introduced by the agent. This has some potential issues and I am working on #3649 to address them, but I feel like I am not gonna finish it in the next few days, and that PR has become harder and harder to review, thus this PR, which only focuses on a small improvement.
So what's the issue? When we run linters on the original file before our edits, we need to copy the original file and use a temporary file to lint, because linting may have side-effect (e.g. modifying the file in-place). I used the word "may" because:
Flake8 has no side-effect, so not a problem as of now.
We don't enforce this or document this "no side-effect" as a requirement for linter implementation, so side-effect is allowed.
Regardless, the "after-edit-linting" uses the same approach: backup the file before linting to avoid data corruption. We should keep our "before-edit-linting" consistent.
Why no new unittest that reproduces the issue? Well, as I have mentioned earlier, flake8 has no side-effect, so technically it's not a bug but a flaw. Therefore, there's no way to write a test that reproduces the issue.
* improve file editing prompts and unit test
converted most raise calls to a _output_error call in file_ops.py
* tweaks in test_agent_skill.py wrt to SEP separator
* tweaked the separator
* remove server runtime remnants and TEST_RUNTIME references
* restore use of TEST_RUNTIME args and variables
* fix integration tests
* added hint to properly escape docstrings
* revert latest prompt change
---------
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
* change runtime to be synchronous
* fix test runtime with the new interface
* fix arg
* fix eval
* fix missing config attribute
* fix plugins
* fix on_event by revert it back to async
* update upload_file endpoint
* fix argument to upload file
* remove unncessary async for eval;
fix evaluation run in parallel
* use asyncio to run controller for eval
* revert file upload
* truncate eval test result output
* feat: refactor building logic into runtime builder
* return image name
* fix testcases
* use runtime builder for eventstream runtime
* have runtime builder return str
* add api_key to sandbox config
* draft remote runtime
* remove extra if clause
* initialize runtime based on box class
* add build logic
* use base64 for file upload
* get runtime image prefix from API
* replace ___ with _s_ to make it a valid image name
* use /build to start build and /build_status to check the build progress
* update logging
* fix exit code
* always use port
* add remote runtime
* rename runtime
* fix tests import
* make dir first if work_dir does not exists;
* update debug print to remote runtime
* fix exit close_sync
* update logging
* add retry for stop
* use all box class for test keep prompt
* fix test browsing
* add retry stop
* merge init commands to save startup time
* fix await
* remove sandbox url
* support execute through specific runtime url
* fix file ops
* simplify close
* factor out runtime retry code
* fix exception handling
* fix content type error (e.g., bad gateway when runtime is not ready)
* add retry for wait until alive;
add retry for check image exists
* Revert "add retry for wait until alive;"
This reverts commit dd013cd268.
* retry when wait until alive
* clean up msg
* directly save sdist to temp dir for _put_source_code_to_dir
* support running testcases in parallel
* tweak logging;
try to close session
* try to close session even on exception
* update poetry lock
* support remote to run integration tests
* add warning for workspace base on remote runtime
* set default runtime api
* remove server runtime
* update poetry lock
* support running swe-bench (n=1) eval on remoteruntime
* add a timeout of 30 min
* add todo for docker namespace
* update poetry loc
* linter and test_aider_linter extensions for eslint
* linter tweaks
* try enabling verbose output in linter test
* one more option for linter test
* try conftest.py for tests/unit folder
* enable verbose mode in workflow; remove conftest.py again
* debug print statements of linter results
* skip some tests if eslint is not installed at all
* more tweaks
* final test skip setups
* code quality revisions
* fix test again
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
* aider-bench: add visualization to summarize script and readme
* added example cost and actions histogram images for readme
* moved dependencies to evaluation section
* added optional START_ID env flag to resume from that instance id
* prepare_dataset: fix comparisons by using instance id's as int
* aider bench complete_runtime: close runtime to close container
* added matrix display of instance id for logging
* fix typo in summarize_results.py saying summarise_results
* changed start_id to skip_num to skip rows from dataset (start_id wasn't supportable)
* doc changes about huggingface spaces to temporarily point back to OD
* Add prompt caching
* remove anthropic-version from extra_headers
* change supports_prompt_caching method to attribute
* change caching strat and log cache statistics
* add reminder as a new message to fix caching
* fix unit test
* append reminder to the end of the last message content
* move token logs to post completion function
* fix unit test failure
* fix reminder and prompt caching
* unit tests for prompt caching
* add test
* clean up tests
* separate reminder, use latest two messages
* fix tests
---------
Co-authored-by: tobitege <10787084+tobitege@users.noreply.github.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
* add eval_ids arg to run specific instance id's; fix/extend README
* fix description in parser for --eval-ids
* fix test_arg_parser.py to account for added arg
* fix typo in README to say "summarize" instead of "summarise" for script
* fix conftest.py option (#3573)
* try to fix fixture base_container_image in runtime conftest
* fix integration test mock files due to #3548
* fix test_ipython.py integration test
* Fix pypi build
The package on pypi only included opendevin/* (the poetry default). It also needs to include agenthub/*
* Bumped version so people will actually get it!
* Fix package definition
* Updated poetry lock file
* Update package name to openhands-ai
* Add py.typed to indicate that OpenHands has type annotations
* Replace package name with openhands_ai
* Fix tests to reflect new name
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
* Create helper functions
* Add map according to litellm docs
* Create ModelSelector
* Extend model selector
* use autocomplete from nextui
* Improve keys without providers
* Handle models without a provider
* Add verified section and some empty handling
* Add support for default or previously set models
* Update tests
* Lint
* Remove modifier
* Fix typescript error
* Functionality for switching to custom model
* Add verified models
* Respond to resetting to default
* Comment
* try to fix pip unavailable
* update test case for pip
* force rebuild in CI
* remove extra symlink
* fix newline
* added semi-colon to line 31
* Dockerfile.j2: activate env at the end
* Revert "Dockerfile.j2: activate env at the end"
This reverts commit cf2f565102.
* cleanup Dockerfile
* switch default python image
* remove image agnostic (no longer used)
* fix tests
* simplify integration tests default image
* add nodejs specific runtime tests
* update tests and workflows
* switch to nikolaik/python-nodejs:python3.11-nodejs22
* update build sh to output image name correctly
* increase custom images to test
* fix test
* fix test
* fix double quote
* try fixing ci
* update ghcr workflow
* fix artifact name
* try to fix ghcr again
* fix workflow
* save built image to correct dir
* remove extra -docker-image
* make last tag to be human readable image tag
* fix hyphen to underscore
* run test runtime on all tags
* revert app build
* separate ghcr workflow
* update dockerfile for eval
* fix tag for test run
* try fix tag
* try fix tag via matrix output
* try workflow again
* update comments
* try fixing test matrix
* fix artifact name
* try fix tag again
* Revert "try fix tag again"
This reverts commit b369badd8c.
* tweak filename
* try different path
* fix filepath
* try fix tag artifact path again
* save json instead of line
* update matrix
* print all tags in workflow
* support only streaming diff logs from the runtime client
* remove strip from log line to fix indentation
* get py interpreter for jupyter
* rstrip to remove newline on the rightside for logging
* fix blocking issue for stream logs
* set python interpreter path in bash ps1
* update testcase for jupyter py interpreter path
* remove accidentally added changes
* remove accidentally added changes
* only print dockerfile when debug
* add docs
* remove extra tests that weren't supposed to be in this pr
* add back missing test
* revert
* make LogBuffer synchronous to fix hang in integration tests
* fix integration tests
* Update opendevin/runtime/client/client.py
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
* fix test case
* fix integration tests
* change deque to list
* update integration tests
* rename test runtime
* fix docs
* rename opendevin to openhands in tests
---------
Co-authored-by: tobitege <tobitege@gmx.de>
Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: tobitege <10787084+tobitege@users.noreply.github.com>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
* tweaks to linter.py to prep for typescript linting (not implemented yet)
* fix 2 linter unit tests
* simpler basic_lint output; updated unit test
* fix default gpt-4o model name in aider default config
* linter.py: use tsc (typescript compiler) for linting; added more tests
* make typescript linting be more forgiving
* use npx instead of npm to install typescript in Dockerfile.j2
* Fix merge mistake
* removed npx call from Dockerfile.j2
* fix run_cmd to use code parameter; replace regex in test
* fix test_lint_file_fail_typescript to ignore leading path characters
* added TODO comment to extract_error_line_from
* fixed bug in ts_lint with wrong line number parsing
* move test_runtime to ghcr_test_runtime; adapt workflows; fix runtime AttributeError
* split test_runtime.py into multiple tests in new tests/runtime folder
* moved common fixtures to tests/runtime/conftest.py
* add anthropic dependency for Vertex to Dockerfile.j2 template
* add anthropic also to pyproject
* revert Dockerfile.j2 change
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
---------
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
* renaming more opendevin occurences
* remove DOCKER_IMAGE variable from Makefile
* Revert rename in evaluation/swe_bench/run_infer.py
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
---------
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
* Fix: Reset iteration count when resuming due to throttling
* Fix inadvertent additions
* WIP
* Changing max_iterations instead of iteration count
* Now adjusting max_iterations or max_budget_per_task as appropriate
* Fix check on iterations
* Fix linter issues
* AgentController: remember initial max_iterations and use it to extend state's iterations
* increase task budget by initial value (not doubling it)
---------
Co-authored-by: Tim O'Farrell <tofarr@gmai.com>
Co-authored-by: tobitege <10787084+tobitege@users.noreply.github.com>
Co-authored-by: mamoodi <mamoodiha@gmail.com>
* try to fix pip unavailable
* update test case for pip
* force rebuild in CI
* remove extra symlink
* fix newline
* added semi-colon to line 31
* Dockerfile.j2: activate env at the end
* Revert "Dockerfile.j2: activate env at the end"
This reverts commit cf2f565102.
* cleanup Dockerfile
* switch default python image
* remove image agnostic (no longer used)
* fix tests
* simplify integration tests default image
* add nodejs specific runtime tests
* update tests and workflows
* switch to nikolaik/python-nodejs:python3.11-nodejs22
* update build sh to output image name correctly
* increase custom images to test
* fix test
* fix test
* fix double quote
* try fixing ci
* update ghcr workflow
* fix artifact name
* try to fix ghcr again
* fix workflow
* save built image to correct dir
* remove extra -docker-image
* make last tag to be human readable image tag
* fix hyphen to underscore
* run test runtime on all tags
* revert app build
* separate ghcr workflow
* update dockerfile for eval
* fix tag for test run
* try fix tag
* try fix tag via matrix output
* try workflow again
* update comments
* try fixing test matrix
* fix artifact name
* try fix tag again
* Revert "try fix tag again"
This reverts commit b369badd8c.
* tweak filename
* try different path
* fix filepath
* try fix tag artifact path again
* save json instead of line
* update matrix
* print all tags in workflow
* fix DOCKER_IMAGE to avoid ghcr.io/opendevin/ghcr.io/opendevin/od_runtime
* fix test matrix to only load unique test image tags
* try fix matrix again!!!!!
* add all runtime tests passed
---------
Co-authored-by: tobitege <tobitege@gmx.de>
Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: tobitege <10787084+tobitege@users.noreply.github.com>
* refactor agentskills to prepare for agentless
* fix import
* fix typo
* fix imports
* fix globals
* fix import
* fix import
* disable log to file to avoid auto-created log file w/ permission issue when import od in runtime
* import agentskills from OD instead from itself directly
* add back pythonpath
* remove chown since there's no log/folder
* add projects with licenses list
* fix projects
* fix projects
* fix projects
* fix licenses, include full Apache
* move AL 2.0 text to Credits
* Update README.md
Co-authored-by: Graham Neubig <neubig@gmail.com>
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
* Split Frontend and Python Unit tests
* Extra comment for deploy docs workflow
* Simpler comment
* Add paths and paths-ignore to unit tests
* More specific comment for py unit tests
* Remove paths-ignore because jobs are required
---------
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
* fix potential flake8 miss checking
* Add unit test for edit_file_by_replace function with problematic file
* Add unit test for edit_file_by_replace function with problematic file
* Add unit test for edit_file_by_replace function with problematic file
* Add unit test for edit_file_by_replace function with problematic file
* Add unit test for edit_file function with problematic file
* Add unit test for edit_file function with problematic file
* Add unit test for edit_file function with problematic file
* Add unit test for edit_file function with problematic file
* Add unit test for edit_file function with problematic file
* Update opendevin/runtime/plugins/agent_skills/agentskills.py
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
* add test intention description
* fix potential flake8 miss checking
* fix potential flake8 miss checking
* fix potential flake8 miss checking
* fix potential flake8 miss checking
* fix potential flake8 miss checking
---------
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: tobitege <tobitege@gmx.de>
* feat: Initial work on security analyzer
* feat: Add remote invariant client
* chore: improve fault tolerance of client
* feat: Add button to enable Invariant Security Analyzer
* [feat] confirmation mode for bash actions
* feat: Add Invariant Tab with security risk outputs
* feat: Add modal setting for Confirmation Mode
* fix: frontend tests for confirmation mode switch
* fix: add missing CONFIRMATION_MODE value in SettingsModal.test.tsx
* fix: update test to integrate new setting
* feat: Initial work on security analyzer
* feat: Add remote invariant client
* chore: improve fault tolerance of client
* feat: Add button to enable Invariant Security Analyzer
* feat: Add Invariant Tab with security risk outputs
* feat: integrate security analyzer with confirmation mode
* feat: improve invariant analyzer tab
* feat: Implement user confirmation for running bash/python code
* fix: don't display rejected actions
* fix: make confirmation show only on assistant messages
* feat: download traces, update policy, implement settings, auto-approve based on defined risk
* Fix: low risk not being shown because it's 0
* fix: duplicate logs in tab
* fix: log duplication
* chore: prepare for merge, remove logging
* Merge confirmation_mode from OpenDevin main
* test: update tests to pass
* chore: finish merging changes, security analyzer now operational again
* feat: document Security Analyzers
* refactor: api, monitor
* chore: lint, fix risk None, revert policy
* fix: check security_risk for None
* refactor: rename instances of invariant to security analyzer
* feat: add /api/options/security-analyzers endpoint
* Move security analyzer from tab to modal
* Temporary fix lock when security analyzer is not chosen
* feat: don't show lock at all when security analyzer is not enabled
* refactor:
- Frontend:
* change type of SECURITY_ANALYZER from bool to string
* add combobox to select SECURITY_ANALYZER, current options are "invariant and "" (no security analyzer)
* Security is now a modal, lock in bottom right is visible only if there's a security analyzer selected
- Backend:
* add close to SecurityAnalyzer
* instantiate SecurityAnalyzer based on provided string from frontend
* fix: update close to be async, to be consistent with other close on resources
* fix: max height of modal (prevent overflow)
* feat: add logo
* small fixes
* update docs for creating a security analyzer module
* fix linting
* update timeout for http client
* fix: move security_analyzer config from agent to session
* feat: add security_risk to browser actions
* add optional remark on combobox
* fix: asdict not called on dataclass, remove invariant dependency
* fix: exclude None values when serializing
* feat: take default policy from invariant-server instead of being hardcoded
* fix: check if policy is None
* update image name
* test: fix some failing runs
* fix: security analyzer tests
* refactor: merge confirmation_mode and security_analyzer into SecurityConfig. Change invariant error message for docker
* test: add tests for invariant parsing actions / observations
* fix: python linting for test_security.py
* Apply suggestions from code review
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
* use ActionSecurityRisk | None intead of Optional
* refactor action parsing
* add extra check
* lint parser.py
* test: add field keep_prompt to test_security
* docs: add information about how to enable the analyzer
* test: Remove trailing whitespace in README.md text
---------
Co-authored-by: Mislav Balunovic <mislav.balunovic@gmail.com>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
* remove unused plugin mixin
* change the entire jupyter PWD with bash;
print jupyter pwd in obs as well;
* remove unused field
* remove unused comments
* change the entire jupyter PWD with bash;
print jupyter pwd in obs as well;
* fix runtime tests for jupyter
* update intgeration tests
* fix test again
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
* try to fix pip unavailable
* update test case for pip
* force rebuild in CI
* remove extra symlink
* fix newline
* added semi-colon to line 31
* Dockerfile.j2: activate env at the end
* Revert "Dockerfile.j2: activate env at the end"
This reverts commit cf2f565102.
* cleanup Dockerfile
* switch default python image
* remove image agnostic (no longer used)
* fix tests
* switch to nikolaik/python-nodejs:python3.11-nodejs22
* fix test
* fix test
* revert docker
* update template
---------
Co-authored-by: tobitege <tobitege@gmx.de>
Co-authored-by: Graham Neubig <neubig@gmail.com>
* Fix issue where mouse drag fails on first attempt
* Detach event correctly
* Use old variable names
---------
Co-authored-by: Tim O'Farrell <tofarr@Tims-MacBook-Pro-2.local>
Co-authored-by: tobitege <tobitege@gmx.de>
* also copy over pyproject and poetry lock
* add missing readme
* remove extra git config init since it is already done in client.py
* only chown if the /workspace dir does not exists
* Revert "remove extra git config init since it is already done in client.py"
This reverts commit e8556cd76d.
* remove extra git config init since it is already done in client.py
* fix test runtime
* print container log while reconnecting
* print log in more readable format
* print log in more readable format
* increase lines
* clean up sandbox and ssh related stuff
* remove ssh hostname
* remove ssh hostname
* fix docker app cannot access runtime API issue
* remove ssh password
* API HOSTNAME should be pre-fixed with SANDBOX
* update config
* fix typo that breaks the test
* clean up sandbox and ssh related stuff
* remove ssh hostname
* remove ssh hostname
* remove ssh password
* update config
* fix typo that breaks the test
* Add unit tests for listen.py
* Added new tests
* Improve test coverage for listen.py
* Update tests
---------
Co-authored-by: opendevin <opendevin@all-hands.dev>
* switch default to eventstream runtime
* remove pull docker from makefile
* fix unittest
* fix file store path
* try deprecate server runtime
* remove persist sandbox
* move file utils
* remove server runtime related workflow
* remove unused method
* attempt to remove the reliance on filestore for BE
* fix async for list file
* fix list_files to post
* fix list files
* add suffix to directory
* make sure list file returns abs path;
make sure other backend endpoints accpets abs path
* remove server runtime test workflow
* set git config in runtime
* chown for workspace in client;
use INIT_COMMANDS to maintain all commands that need to be run before bash start;
* fix client issue;
add test case for git;
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
* switch default to eventstream runtime
* remove pull docker from makefile
* fix unittest
* fix file store path
* try deprecate server runtime
* remove persist sandbox
* move file utils
* remove server runtime related workflow
* remove unused method
* attempt to remove the reliance on filestore for BE
* fix async for list file
* fix list_files to post
* fix list files
* add suffix to directory
* make sure list file returns abs path;
make sure other backend endpoints accpets abs path
* remove server runtime test workflow
* set git config in runtime
* move multi-line bash tests to test_runtime;
support multi-line bash for esruntime;
* add testcase to handle PS2 prompt
* use bashlex for bash parsing to handle multi-line commands;
add testcases for multi-line commands
* revert ghcr runtime change
* Apply stash
* fix run as other user;
make test async;
* fix test runtime for run as od
* add run-as-devin to all the runtime tests
* handle the case when username is root
* move all run-as-devin tests from sandbox;
only tests a few cases on different user to save time;
* move over multi-line echo related tests to test_runtime
* fix user-specific jupyter by fixing the pypoetry virtualenv folder
* make plugin's init async;
chdir at initialization of jupyter plugin;
move ipy simple testcase to test runtime;
* support agentskills import in
move tests for jupyter pwd tests;
overload `add_env_vars` for EventStreamRuntime to update env var also in Jupyter;
make agentskills read env var lazily, in case env var is updated;
* fix ServerRuntime agentskills issue
* move agnostic image test to test_runtime
* merge runtime tests in CI
* fix enable auto lint as env var
* update warning message
* update warning message
* test for different container images
* change parsing output as debug
* add exception handling for update_pwd_decorator
* fix unit test indentation
* add plugins as default input to Runtime class;
remove init_sandbox_plugins;
implement add_env_var (include jupyter) in the base class;
* fix server runtime auto lint
* Revert "add exception handling for update_pwd_decorator"
This reverts commit 2b668b1506.
* tries to print debugging info for agentskills
* explictly setting uid (try fix permission issue)
* Revert "tries to print debugging info for agentskills"
This reverts commit 8be4c86756.
* set sandbox user id during testing to hopefully fix the permission issue
* add browser tools for server runtime
* try to debug for old pwd
* update debug cmd
* only test agnostic runtime when TEST_RUNTIME is Server
* fix temp dir mkdir
* load TEST_RUNTIME at the beginning
* remove ipython tests
* only log to file when DEBUG
* default logging to project root
* temporarily remove log to file
* fix LLM logger dir
* fix logger
* make set pwd an optional aux action
* fix prev pwd
* fix infinity recursion
* simplify
* do not import the whole od library to avoid logger folder by jupyter
* fix browsing
* increase timeout
* attempt to fix agentskills yet again
* clean up in testcases, since CI maybe run as non-root
* add _cause attribute for event.id
* remove parent
* add a bunch of debugging statement again for CI :(
* fix temp_dir fixture
* change all temp dir to follow pytest's tmp_path_factory
* remove extra bracket
* clean up error printing a bit
* jupyter chdir to self.config.workspace_mount_path_in_sandbox on initialization
* jupyter chdir to self.config.workspace_mount_path_in_sandbox on initialization
* add typing for tmp dir fixture
* clear the directory before running the test to avoid weird CI temp dir
* remove agnostic test case for server runtime
* Revert "remove agnostic test case for server runtime"
This reverts commit 30e2181c3f.
* disable agnostic tests in CI
* fix test
* make sure plugin arg is not passed when no plugin is specified;
remove redundant on_event function;
* move mock prompt
* rename runtime
* remove extra logging
* refactor run_controller's interface;
support multiple runtime for integration test;
filter out hostname for prompt
* uncomment other tests
* pass the right runtime to controller
* log runtime when start
* uncomment tests
* improve symbol filters
* add intergration test prompts that seemd ok
* add integration test workflow
* add python3 to default ubuntu image
* symlink python and fix permission to jupyter pip
* add retry for jupyter execute server
* fix jupyter pip install;
add post-process for jupyter pip install;
simplify init by add agent_skills path to PYTHONPATH;
add testcase to tests jupyter pip install;
* fix bug
* use ubuntu:22.04 for eventstream integration tests
* add todo
* update testcase
* remove redundant code
* fix unit test
* reduce dependency for runtime
* try making llama-index an optional dependency that's not installed by default
* remove pip install since it seemd not needed
* log ipython execution;
await write message since it returns a future
* update ipy testcase
* do not install llama-index in CI
* do not install llama-index in the app docker as well
* set sandbox container image in the integration test script
* log plugins & env var for runtime
* update conftest for sha256
* add git
* remove all non-alphanumeric chalracters
* add working ipy module tests!
* default to use host network
* remove is_async from browser to make thing a little more reliable;
retry loading browser when error;
* add sleep to wait a bit for http server
* kill http server before regenerate browsing tests
* fix browsing
* only set sandbox container image if undefined
* skip empty config value
* update evaluation to use the latest run_controller
* revert logger in execute_server to be compatible with server runtime
* revert logging level to fix jupyter
* set logger level
* revert the logging
* chmod for workspace to fix permission
* support getting timeout from action
* update test for server runtime
* try to fix file permission
* fix test_cmd_run_action_serialization_deserialization test (added timeout)
* poetry: pip 24.2, torch 2.2.2
* revert adding pip to pyproject.toml
* add build to dependencies in pyproject.toml
* forgot poetry lock --no-update
* fix a DelegatorAgent prompt_002.log (timeout)
* fix a DelegatorAgent prompt_003.log (timeout)
* couple more timeout attribs in prompt files
* some more prompt files
* prompts galore
* add clarification comment for timeout
* default timeout to config
* add assert
* update integraton tests for eventstream
* update integration tests
* fix timeout for action<->dict
* remove redundant on_event
* default to use instance image
* update run_controller interface
* add logging for copy
* refactor swe_bench for the new design
* fix action execution timeout
* updatelock
* remove build sandbox locally
* fix runtime
* use plain for-loop for single process
* remove extra print
* get swebench inference working
* print whole `test_result` dict
* got swebench patch post-process working
* update swe-bench evaluation readme
* refactor using shared reset_logger function
* move messy swebench prompt to a different file
* support the ability to specify whether to keep prompt
* support the ability to specify whether to keep prompt
* fix dockerfile
* fix import and remove unnecessary strip logic
* fix action serialization
* get agentbench running
* remove extra ls for agent bench
* fix agentbench metric
* factor out common documentation for eval
* update biocoder doc
* remove swe_env_box since it is no longer needed
* get biocoder working
* add func timeout for bird
* fix jupyter pwd with ~ as user name
* fix jupyter pwd with ~ as user name
* get bird working
* get browsing evaluation working
* make eda runnable
* fix id column
* fix eda run_infer
* unify eval output using a structured format;
make swebench coompatible with that format;
update client source code for every swebench run;
do not inject testcmd for swebench
* standardize existing benchs for the new eval output
* set update source code = true
* get gaia standardized
* fix gaia
* gorilla refactored but stuck at language.so to test
* refactor and make gpqa work
* refactor humanevalfix and get it working
* refactor logic reasoning and get it working
* refactor browser env so it works with eventstream runtime for eval
* add initial version of miniwob refactor
* fix browsergym environment
* get miniwob working!!
* allowing injecting additional dependency to OD runtime docker image
* allowing injecting additional dependency to OD runtime docker image
* support logic reasoning with pre-injected dependency
* get mint working
* update runtime build
* fix mint docker
* add test for keep_prompt;
add missing await close for some tests
* update integration tests for eventstream runtime
* fix integration tests for server runtime
* refactor ml bench and toolqa
* refactor webarena
* fix default factory
* Update run_infer.py
* add APIError to retry
* increase timeout for swebench
* make sure to hide api key when dump eval output
* update the behavior of put source code to put files instead of tarball
* add dishash to dependency
* sendintr when timeout
* fix dockerfile copy
* reduce timeout
* use dirhash to avoid repeat building for update source
* fix runtime_build testcase
* add dir_hash to docker build pipeline
* revert api error
* update poetry lock
* add retries for swebench run infer
* fix git patch
* update poetry lock
* adjust config order
* fix mount volumns
* enforce all eval to use "instance_id"
* remove file store from runtime
* make file_store public inside eventstream
* move the runtime logic inside `main` out
* support using async function for process_instance_fn
* refactor run_infer with the create_time
* fix file store
* Update evaluation/toolqa/utils.py
Co-authored-by: Graham Neubig <neubig@gmail.com>
* fix typo
---------
Co-authored-by: tobitege <tobitege@gmx.de>
Co-authored-by: super-dainiu <78588128+super-dainiu@users.noreply.github.com>
Co-authored-by: Graham Neubig <neubig@gmail.com>
* added workflow clean-up.yml to remove old workflows and artifacts
* dispatch-only run for now
* add retention-days of 14 to upload-artifact in ghcr.yml
* update the behavior of put source code to put files instead of tarball
* add dishash to dependency
* fix dockerfile copy
* use dirhash to avoid repeat building for update source
* fix runtime_build testcase
* add dir_hash to docker build pipeline
* add additional tests for source directory
* add comment
* clear the assertion by explictly check existing files
* also assert od is a dir
* ServerRuntime: config copy in init
* revert #3233 but more logging
* get_box_classes: reset order back to previous version
* 3 logging commands switched to debug (were info)
* runtimes debug output of config on initialization
* removed unneeded logger message from _init_container
* ghcr.yml with paths/paths-ignore conditions for pull_request
* include evaluation folder
* removed paths, just paths-ignore now
* deploy-docs only for docs
* exclude evaluation folder
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
---------
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
* support the ability to specify whether to keep prompt
* fix action serialization
* fix jupyter pwd with ~ as user name
* add test for keep_prompt;
add missing await close for some tests
* update integration tests for eventstream runtime
* fix integration tests for server runtime
* Remove global config from memory
* Remove runtime global config
* Remove from storage
* Remove global config
* Fix event stream tests
* Fix sandbox issue
* Change config
* Removed transferred tests
* Add swe env box
* Fixes on testing
* Fixed some tests
* Merge with stashed changes
* Fix typing
* Fix ipython test
* Revive function
* Make temp_dir fixture
* Remove test to avoid circular import
* fix eventstream filestore for test_runtime
* fix parse arg issue that cause integration test to fail
* support swebench pull from custom namespace
* add back simple tests for runtime
* move multi-line bash tests to test_runtime;
support multi-line bash for esruntime;
* add testcase to handle PS2 prompt
* use bashlex for bash parsing to handle multi-line commands;
add testcases for multi-line commands
* revert ghcr runtime change
* Apply stash
* fix run as other user;
make test async;
* fix test runtime for run as od
* add run-as-devin to all the runtime tests
* handle the case when username is root
* move all run-as-devin tests from sandbox;
only tests a few cases on different user to save time;
* move over multi-line echo related tests to test_runtime
* fix user-specific jupyter by fixing the pypoetry virtualenv folder
* make plugin's init async;
chdir at initialization of jupyter plugin;
move ipy simple testcase to test runtime;
* support agentskills import in
move tests for jupyter pwd tests;
overload `add_env_vars` for EventStreamRuntime to update env var also in Jupyter;
make agentskills read env var lazily, in case env var is updated;
* fix ServerRuntime agentskills issue
* move agnostic image test to test_runtime
* merge runtime tests in CI
* fix enable auto lint as env var
* update warning message
* update warning message
* test for different container images
* change parsing output as debug
* add exception handling for update_pwd_decorator
* fix unit test indentation
* add plugins as default input to Runtime class;
remove init_sandbox_plugins;
implement add_env_var (include jupyter) in the base class;
* fix server runtime auto lint
* Revert "add exception handling for update_pwd_decorator"
This reverts commit 2b668b1506.
* tries to print debugging info for agentskills
* explictly setting uid (try fix permission issue)
* Revert "tries to print debugging info for agentskills"
This reverts commit 8be4c86756.
* set sandbox user id during testing to hopefully fix the permission issue
* add browser tools for server runtime
* try to debug for old pwd
* update debug cmd
* only test agnostic runtime when TEST_RUNTIME is Server
* fix temp dir mkdir
* load TEST_RUNTIME at the beginning
* remove ipython tests
* only log to file when DEBUG
* default logging to project root
* temporarily remove log to file
* fix LLM logger dir
* fix logger
* make set pwd an optional aux action
* fix prev pwd
* fix infinity recursion
* simplify
* do not import the whole od library to avoid logger folder by jupyter
* fix browsing
* increase timeout
* attempt to fix agentskills yet again
* clean up in testcases, since CI maybe run as non-root
* add _cause attribute for event.id
* remove parent
* add a bunch of debugging statement again for CI :(
* fix temp_dir fixture
* change all temp dir to follow pytest's tmp_path_factory
* remove extra bracket
* clean up error printing a bit
* jupyter chdir to self.config.workspace_mount_path_in_sandbox on initialization
* jupyter chdir to self.config.workspace_mount_path_in_sandbox on initialization
* add typing for tmp dir fixture
* clear the directory before running the test to avoid weird CI temp dir
* remove agnostic test case for server runtime
* Revert "remove agnostic test case for server runtime"
This reverts commit 30e2181c3f.
* disable agnostic tests in CI
* fix test
* make sure plugin arg is not passed when no plugin is specified;
remove redundant on_event function;
* move mock prompt
* rename runtime
* remove extra logging
* refactor run_controller's interface;
support multiple runtime for integration test;
filter out hostname for prompt
* uncomment other tests
* pass the right runtime to controller
* log runtime when start
* uncomment tests
* improve symbol filters
* add intergration test prompts that seemd ok
* add integration test workflow
* add python3 to default ubuntu image
* symlink python and fix permission to jupyter pip
* add retry for jupyter execute server
* fix jupyter pip install;
add post-process for jupyter pip install;
simplify init by add agent_skills path to PYTHONPATH;
add testcase to tests jupyter pip install;
* fix bug
* use ubuntu:22.04 for eventstream integration tests
* add todo
* update testcase
* remove redundant code
* fix unit test
* reduce dependency for runtime
* try making llama-index an optional dependency that's not installed by default
* remove pip install since it seemd not needed
* log ipython execution;
await write message since it returns a future
* update ipy testcase
* do not install llama-index in CI
* do not install llama-index in the app docker as well
* set sandbox container image in the integration test script
* log plugins & env var for runtime
* update conftest for sha256
* add git
* remove all non-alphanumeric chalracters
* add working ipy module tests!
* default to use host network
* remove is_async from browser to make thing a little more reliable;
retry loading browser when error;
* add sleep to wait a bit for http server
* kill http server before regenerate browsing tests
* fix browsing
* only set sandbox container image if undefined
* skip empty config value
* update evaluation to use the latest run_controller
* revert logger in execute_server to be compatible with server runtime
* revert logging level to fix jupyter
* set logger level
* revert the logging
* chmod for workspace to fix permission
* support getting timeout from action
* update test for server runtime
* try to fix file permission
* fix test_cmd_run_action_serialization_deserialization test (added timeout)
* poetry: pip 24.2, torch 2.2.2
* revert adding pip to pyproject.toml
* add build to dependencies in pyproject.toml
* forgot poetry lock --no-update
* fix a DelegatorAgent prompt_002.log (timeout)
* fix a DelegatorAgent prompt_003.log (timeout)
* couple more timeout attribs in prompt files
* some more prompt files
* prompts galore
* add clarification comment for timeout
* default timeout to config
* add assert
* update integraton tests for eventstream
* update integration tests
* fix timeout for action<->dict
* remove redundant on_event
* fix action execution timeout
* updatelock
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: tobitege <tobitege@gmx.de>
* LLM class: added acompletion and streaming, unit test test_acompletion.py
* LLM: cleanup of self.config defaults and their use
* added set_missing_attributes to LLMConfig
* move default checker up
* add copy to
* implement for ServerRuntime
* implement copyto for runtime (required by eval);
add tests for copy to
* fix exist file check
* unify copy_to_behavior and fix stuff
value:Thank you for taking the time to fill out this bug report. We greatly appreciate your effort to complete this template fully. Please provide as much information as possible to help us understand and address the issue effectively.
value:Thank you for taking the time to fill out this bug report. Please provide as much information as possible to help us understand and address the issue effectively.
- type:checkboxes
attributes:
label:Is there an existing issue for the same bug?
description:Please check if an issue already exists for the bug you encountered.
options:
- label:I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
required:true
- label:I have checked the existing issues.
required:true
- type:textarea
id:bug-description
attributes:
label:Describe the bug
description:Provide a short description of the problem.
label:Describe the bug and reproduction steps
description:Provide a description of the issue along with any reproduction steps.
validations:
required:true
- type:textarea
id:current-version
- type:dropdown
id:installation
attributes:
label:Current OpenDevin version
description:What version of OpenDevin are you using? If you're running in docker, tell us the tag you're using (e.g. ghcr.io/opendevin/opendevin:0.3.1).
render:bash
validations:
required:true
label:OpenHands Installation
description:How are you running OpenHands?
options:
- Docker command in README
- Development workflow
default:0
- type:textarea
id:config
- type:input
id:openhands-version
attributes:
label:Installation and Configuration
description:Please provide any commands you ran and any configuration (redacting API keys)
render:bash
validations:
required:true
label:OpenHands Version
description:What version of OpenHands are you using?
placeholder:ex. 0.9.8, main, etc.
- type:textarea
id:model-agent
attributes:
label:Model and Agent
description:What model and agent are you using? You can see these settings in the UI by clicking the settings wheel.
placeholder:|
- Model:
- Agent:
- type:textarea
id:os-version
- type:dropdown
id:os
attributes:
label:Operating System
description:What Operating System are you using? Linux, Mac OS, WSL on Windows
- type:textarea
id:repro-steps
attributes:
label:Reproduction Steps
description:Please list the steps to reproduce the issue.
placeholder:|
1.
2.
3.
options:
- MacOS
- Linux
- WSL on Windows
- type:textarea
id:additional-context
attributes:
label:Logs, Errors, Screenshots, and Additional Context
description:If you want to share the chat history you can click the thumbs-down (👎) button above the input field and you will get a shareable link (you can also click thumbs up when things are going well of course!). LLM logs will be stored in the `logs/llm/default` folder. Please add any additional context about the problem here.
description:Please provide any additional information you think might help. If you want to share the chat history
you can click the thumbs-down (👎) button above the input field and you will get a shareable link
(you can also click thumbs up when things are going well of course!). LLM logs will be stored in the
`logs/llm/default` folder. Please add any additional context about the problem here.
# Workflow that marks issues and PRs with no activity for 30 days with "Stale" and closes them after 7 more days of no activity
name:'Close stale issues'
# Runs every day at 01:30
on:
schedule:
- cron:'30 1 * * *'
@@ -9,21 +12,10 @@ jobs:
steps:
- uses:actions/stale@v9
with:
# Aggressively close issues that have been explicitly labeled `age-out`
any-of-labels:age-out
stale-issue-message:'This issue is stale because it has been open for 7 days with no activity. Remove stale label or comment or this will be closed in 1 day.'
close-issue-message:'This issue was closed because it has been stalled for over 7 days with no activity.'
stale-pr-message:'This PR is stale because it has been open for 7 days with no activity. Remove stale label or comment or this will be closed in 1 days.'
close-pr-message:'This PR was closed because it has been stalled for over 7 days with no activity.'
days-before-stale:7
days-before-close:1
- uses:actions/stale@v9
with:
# Be more lenient with other issues
stale-issue-message:'This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.'
close-issue-message:'This issue was closed because it has been stalled for over 30 days with no activity.'
stale-pr-message:'This PR is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.'
close-pr-message:'This PR was closed because it has been stalled for over 30 days with no activity.'
days-before-stale:30
exempt-issue-labels:'tracked'
close-issue-message:'This issue was closed because it has been stalled for over 30 days with no activity.'
close-pr-message:'This PR was closed because it has been stalled for over 30 days with no activity.'
OpenHands is an automated AI software engineer. It is a repo with a Python backend
(in the `openhands` directory) and TypeScript frontend (in the `frontend` directory).
General Setup:
- To set up the entire repo, including frontend and backend, run `make build`
- To run linting and type-checking before finishing the job, run `poetry run pre-commit run --all-files --config ./dev_config/python/.pre-commit-config.yaml`
Backend:
- Located in the `openhands` directory
- Testing:
- All tests are in `tests/unit/test_*.py`
- To test new code, run `poetry run pytest tests/unit/test_xxx.py` where `xxx` is the appropriate file for the current functionality
- Write all tests with pytest
Frontend:
- Located in the `frontend` directory
- Prerequisites: A recent version of NodeJS / NPM
- Setup: Run `npm install` in the frontend directory
- Testing:
- Run tests: `npm run test`
- To run specific tests: `npm run test -- -t "TestName"`
- Building:
- Build for production: `npm run build`
- Environment Variables:
- Set in `frontend/.env` or as environment variables
- Available variables: VITE_BACKEND_HOST, VITE_USE_TLS, VITE_INSECURE_SKIP_VERIFY, VITE_FRONTEND_PORT
- Internationalization:
- Generate i18n declaration file: `npm run make-i18n`
Thanks for your interest in contributing to OpenDevin! We welcome and appreciate contributions.
Thanks for your interest in contributing to OpenHands! We welcome and appreciate contributions.
## How Can I Contribute?
There are many ways that you can contribute:
1.**Download and use** OpenDevin, and send [issues](https://github.com/OpenDevin/OpenDevin/issues) when you encounter something that isn't working or a feature that you'd like to see.
2.**Send feedback** after each session by [clicking the thumbs-up thumbs-down buttons](https://docs.all-hands.dev/modules/usage/feedback), so we can see where things are working and failing, and also build an open dataset for training code agents.
3.**Improve the Codebase** by sending PRs (see details below). In particular, we have some [good first issue](https://github.com/OpenDevin/OpenDevin/labels/good%20first%20issue) issues that may be ones to start on.
## Understanding OpenDevin's CodeBase
## Understanding OpenHands's CodeBase
To understand the codebase, please refer to the README in each module:
- [frontend](./frontend/README.md)
- [agenthub](./agenthub/README.md)
- [evaluation](./evaluation/README.md)
- [opendevin](./opendevin/README.md)
- [server](./opendevin/server/README.md)
- [openhands](./openhands/README.md)
- [agenthub](./openhands/agenthub/README.md)
- [server](./openhands/server/README.md)
## Setting up your development environment
We have a separate doc [Development.md](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md) that tells you how to set up a development workflow.
## How can I contribute?
There are many ways that you can contribute:
1.**Download and use** OpenHands, and send [issues](https://github.com/All-Hands-AI/OpenHands/issues) when you encounter something that isn't working or a feature that you'd like to see.
2.**Send feedback** after each session by [clicking the thumbs-up thumbs-down buttons](https://docs.all-hands.dev/modules/usage/feedback), so we can see where things are working and failing, and also build an open dataset for training code agents.
3.**Improve the Codebase** by sending PRs (see details below). In particular, we have some [good first issues](https://github.com/All-Hands-AI/OpenHands/labels/good%20first%20issue) that may be ones to start on.
## What can I build?
Here are a few ways you can help improve the codebase.
#### UI/UX
We're always looking to improve the look and feel of the application. If you've got a small fix
for something that's bugging you, feel free to open up a PR that changes the `./frontend` directory.
If you're looking to make a bigger change, add a new UI element, or significantly alter the style
of the application, please open an issue first, or better, join the #frontend channel in our Slack
to gather consensus from our design team first.
#### Improving the agent
Our main agent is the CodeAct agent. You can [see its prompts here](https://github.com/All-Hands-AI/OpenHands/tree/main/openhands/agenthub/codeact_agent)
Changes to these prompts, and to the underlying behavior in Python, can have a huge impact on user experience.
You can try modifying the prompts to see how they change the behavior of the agent as you use the app
locally, but we will need to do an end-to-end evaluation of any changes here to ensure that the agent
is getting better over time.
We use the [SWE-bench](https://www.swebench.com/) benchmark to test our agent. You can join the #evaluation
channel in Slack to learn more.
#### Adding a new agent
You may want to experiment with building new types of agents. You can add an agent to `openhands/agenthub`
to help expand the capabilities of OpenHands.
#### Adding a new runtime
The agent needs a place to run code and commands. When you run OpenHands on your laptop, it uses a Docker container
to do this by default. But there are other ways of creating a sandbox for the agent.
If you work for a company that provides a cloud-based runtime, you could help us add support for that runtime
by implementing the [interface specified here](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/runtime.py).
#### Testing
When you write code, it is also good to write tests. Please navigate to the `tests` folder to see existing test suites.
At the moment, we have two kinds of tests: `unit` and `integration`. Please refer to the README for each test suite. These tests also run on GitHub's continuous integration to ensure quality of the project.
## Sending Pull Requests to OpenDevin
## Sending Pull Requests to OpenHands
### 1. Fork the Official Repository
Fork the [OpenDevin repository](https://github.com/OpenDevin/OpenDevin) into your own account.
Clone your own forked repository into your local environment:
You'll need to fork our repository to send us a Pull Request. You can learn more
about how to fork a GitHub repo and open a PR with your changes in [this article](https://medium.com/swlh/forks-and-pull-requests-how-to-contribute-to-github-repos-8843fac34ce8)
Set the official repository as your [upstream](https://www.atlassian.com/git/tutorials/git-forks-and-upstreams) to synchronize with the latest update in the official repository.
You should see both `origin` and `upstream` in the output.
### 3. Synchronize with Official Repository
Synchronize latest commit with official repository before coding:
```shell
git fetch upstream
git checkout main
git merge upstream/main
git push origin main
```
### 4. Set up the Development Environment
We have a separate doc [Development.md](https://github.com/OpenDevin/OpenDevin/blob/main/Development.md) that tells you how to set up a development workflow.
### 5. Write Code and Commit It
Once you have done this, you can write code, test it, and commit it to a branch (replace `my_branch` with an appropriate name):
```shell
git checkout -b my_branch
git add .
git commit
git push origin my_branch
```
### 6. Open a Pull Request
* On GitHub, go to the page of your forked repository, and create a Pull Request:
- Click on `Branches`
- Click on the `...` beside your branch and click on `New pull request`
- Set `base repository` to `OpenDevin/OpenDevin`
- Set `base` to `main`
- Click `Create pull request`
The PR should appear in [OpenDevin PRs](https://github.com/OpenDevin/OpenDevin/pulls).
Then the OpenDevin team will review your code.
## PR Rules
### 1. Pull Request title
### Pull Request title
As described [here](https://github.com/commitizen/conventional-commit-types/blob/master/index.json), a valid PR title should begin with one of the following prefixes:
-`feat`: A new feature
@@ -109,8 +84,11 @@ For example, a PR title could be:
-`refactor: modify package path`
-`feat(frontend): xxxx`, where `(frontend)` means that this PR mainly focuses on the frontend component.
You may also check out previous PRs in the [PR list](https://github.com/OpenDevin/OpenDevin/pulls).
You may also check out previous PRs in the [PR list](https://github.com/All-Hands-AI/OpenHands/pulls).
### 2. Pull Request description
### Pull Request description
- If your PR is small (such as a typo fix), you can go brief.
- If it contains a lot of changes, it's better to write more details.
If your changes are user-facing (e.g. a new feature in the UI, a change in behavior, or a bugfix)
please include a short message that we can add to our changelog.
We would like to thank all the [contributors](https://github.com/All-Hands-AI/OpenHands/graphs/contributors) who have helped make OpenHands possible. We greatly appreciate your dedication and hard work.
## Open Source Projects
OpenHands includes and adapts the following open source projects. We are grateful for their contributions to the open source community:
- Description: AI pair programming tool. OpenHands has adapted and integrated its linter module for code-related tasks in [`agentskills utilities`](https://github.com/All-Hands-AI/OpenHands/tree/main/openhands/runtime/plugins/agent_skills/utils/aider)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
### BSD 3-Clause License
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
### Apache License 2.0
Apache License
Version 2.0, January 2004
@@ -189,14 +268,45 @@
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
This guide is for people working on OpenDevin and editing the source code.
If you wish to contribute your changes, check out the [CONTRIBUTING.md](https://github.com/OpenDevin/OpenDevin/blob/main/CONTRIBUTING.md) on how to clone and setup the project initially before moving on.
Otherwise, you can clone the OpenDevin project directly.
This guide is for people working on OpenHands and editing the source code.
If you wish to contribute your changes, check out the [CONTRIBUTING.md](https://github.com/All-Hands-AI/OpenHands/blob/main/CONTRIBUTING.md) on how to clone and setup the project initially before moving on.
Otherwise, you can clone the OpenHands project directly.
## Start the server for development
### 1. Requirements
* Linux, Mac OS, or [WSL on Windows](https://learn.microsoft.com/en-us/windows/wsl/install) [Ubuntu <= 22.04]
* Linux, Mac OS, or [WSL on Windows](https://learn.microsoft.com/en-us/windows/wsl/install) [Ubuntu <= 22.04]
* [Docker](https://docs.docker.com/engine/install/) (For those on MacOS, make sure to allow the default Docker socket to be used from advanced settings!)
Begin by building the project which includes setting up the environment and installing dependencies. This step ensures that OpenDevin is ready to run on your system:
Begin by building the project which includes setting up the environment and installing dependencies. This step ensures that OpenHands is ready to run on your system:
```bash
make build
```
### 3. Configuring the Language Model
OpenDevin supports a diverse array of Language Models (LMs) through the powerful [litellm](https://docs.litellm.ai) library. By default, we've chosen the mighty GPT-4 from OpenAI as our go-to model, but the world is your oyster! You can unleash the potential of Anthropic's suave Claude, the enigmatic Llama, or any other LM that piques your interest.
OpenHands supports a diverse array of Language Models (LMs) through the powerful [litellm](https://docs.litellm.ai) library. By default, we've chosen the mighty GPT-4 from OpenAI as our go-to model, but the world is your oyster! You can unleash the potential of Anthropic's suave Claude, the enigmatic Llama, or any other LM that piques your interest.
To configure the LM of your choice, run:
@@ -44,20 +46,20 @@ To configure the LM of your choice, run:
make setup-config
```
This command will prompt you to enter the LLM API key, model name, and other variables ensuring that OpenDevin is tailored to your specific needs. Note that the model name will apply only when you run headless. If you use the UI, please set the model in the UI.
This command will prompt you to enter the LLM API key, model name, and other variables ensuring that OpenHands is tailored to your specific needs. Note that the model name will apply only when you run headless. If you use the UI, please set the model in the UI.
Note: If you have previously run OpenDevin using the docker command, you may have already set some environmental variables in your terminal. The final configurations are set from highest to lowest priority:
Note: If you have previously run OpenHands using the docker command, you may have already set some environmental variables in your terminal. The final configurations are set from highest to lowest priority:
Some alternative models may prove more challenging to tame than others. Fear not, brave adventurer! We shall soon unveil LLM-specific documentation to guide you on your quest.
And if you've already mastered the art of wielding a model other than OpenAI's GPT, we encourage you to share your setup instructions with us by creating instructions and adding it [to our documentation](https://github.com/OpenDevin/OpenDevin/tree/main/docs/modules/usage/llms).
And if you've already mastered the art of wielding a model other than OpenAI's GPT, we encourage you to share your setup instructions with us by creating instructions and adding it [to our documentation](https://github.com/All-Hands-AI/OpenHands/tree/main/docs/modules/usage/llms).
For a full list of the LM providers and models available, please consult the [litellm documentation](https://docs.litellm.ai/docs/providers).
### 4. Running the application
#### Option A: Run the Full Application
Once the setup is complete, launching OpenDevin is as simple as running a single command. This command starts both the backend and frontend servers seamlessly, allowing you to interact with OpenDevin:
Once the setup is complete, launching OpenHands is as simple as running a single command. This command starts both the backend and frontend servers seamlessly, allowing you to interact with OpenHands:
```bash
make run
```
@@ -75,10 +77,10 @@ make run
### 6. LLM Debugging
If you encounter any issues with the Language Model (LM) or you're simply curious, you can inspect the actual LLM prompts and responses. To do so, export DEBUG=1 in the environment and restart the backend.
OpenDevin will then log the prompts and responses in the logs/llm/CURRENT_DATE directory, allowing you to identify the causes.
OpenHands will then log the prompts and responses in the logs/llm/CURRENT_DATE directory, allowing you to identify the causes.
### 7. Help
Need assistance or information on available targets and commands? The help command provides all the necessary guidance to ensure a smooth experience with OpenDevin.
Need assistance or information on available targets and commands? The help command provides all the necessary guidance to ensure a smooth experience with OpenHands.
```bash
make help
```
@@ -91,9 +93,36 @@ To run tests, refer to the following:
poetry run pytest ./tests/unit/test_*.py
```
#### Integration tests
Please refer to [this README](./tests/integration/README.md) for details.
### 9. Add or update dependency
1. Add your dependency in `pyproject.toml` or use `poetry add xxx`
2. Update the poetry.lock file via `poetry lock --no-update`
### 9. Use existing Docker image
To reduce build time (e.g., if no changes were made to the client-runtime component), you can use an existing Docker container image. Follow these steps:
1. Set the SANDBOX_RUNTIME_CONTAINER_IMAGE environment variable to the desired Docker image.
<h1 align="center">OpenDevin: Code Less, Make More</h1>
<a href="https://docs.all-hands.dev/modules/usage/intro"><img src="https://img.shields.io/badge/Documentation-OpenDevin-blue?logo=googledocs&logoColor=white&style=for-the-badge" alt="Check out the documentation"></a>
<a href="https://arxiv.org/abs/2407.16741"><img src="https://img.shields.io/badge/Paper-%20on%20Arxiv-red?logo=arxiv&style=for-the-badge" alt="Paper on Arxiv"></a>
<a href="https://docs.all-hands.dev/modules/usage/getting-started"><img src="https://img.shields.io/badge/Documentation-000?logo=googledocs&logoColor=FFE165&style=for-the-badge" alt="Check out the documentation"></a>
<a href="https://arxiv.org/abs/2407.16741"><img src="https://img.shields.io/badge/Paper%20on%20Arxiv-000?logoColor=FFE165&logo=arxiv&style=for-the-badge" alt="Paper on Arxiv"></a>
> By default, this command pulls the `latest` tag, which represents the most recent release of OpenDevin. You have other options as well:
> - For a specific release version, use `ghcr.io/opendevin/opendevin:<OpenDevin_version>` (replace <OpenDevin_version> with the desired version number).
> - For the most up-to-date development version, use `ghcr.io/opendevin/opendevin:main`. This version may be **(unstable!)** and is recommended for testing or development purposes only.
>
> Choose the tag that best suits your needs based on stability requirements and desired features.
You'll find OpenHands running at [http://localhost:3000](http://localhost:3000)!
You'll find OpenDevin running at [http://localhost:3000](http://localhost:3000) with access to `./workspace`. To have OpenDevin operate on your code, place it in `./workspace`.
OpenDevin will only have access to this workspace folder. The rest of your system will not be affected as it runs in a secured docker sandbox.
You'll need a model provider and API key. One option that works well: [Claude 3.5 Sonnet](https://www.anthropic.com/api), but you have [many options](https://docs.all-hands.dev/modules/usage/llms).
Upon opening OpenDevin, you must select the appropriate `Model` and enter the `API Key` within the settings that should pop up automatically. These can be set at any time by selecting
the `Settings` button (gear icon) in the UI. If the required `Model` does not exist in the list, you can manually enter it in the text box.
---
For the development workflow, see [Development.md](https://github.com/OpenDevin/OpenDevin/blob/main/Development.md).
You can also run OpenHands in a scriptable [headless mode](https://docs.all-hands.dev/modules/usage/how-to/headless-mode),
or as an [interactive CLI](https://docs.all-hands.dev/modules/usage/how-to/cli-mode).
Are you having trouble? Check out our [Troubleshooting Guide](https://docs.all-hands.dev/modules/usage/troubleshooting).
Visit [Installation](https://docs.all-hands.dev/modules/usage/installation) for more information and setup instructions.
## 🚀 Documentation
If you want to modify the OpenHands source code, check out [Development.md](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md).
To learn more about the project, and for tips on using OpenDevin,
**check out our [documentation](https://docs.all-hands.dev/modules/usage/intro)**.
Having issues? The [Troubleshooting Guide](https://docs.all-hands.dev/modules/usage/troubleshooting) can help.
There you'll find resources on how to use different LLM providers (like ollama and Anthropic's Claude),
## 📖 Documentation
To learn more about the project, and for tips on using OpenHands,
**check out our [documentation](https://docs.all-hands.dev/modules/usage/getting-started)**.
There you'll find resources on how to use different LLM providers,
troubleshooting resources, and advanced configuration options.
## 🤝 How to Contribute
OpenDevin is a community-driven project, and we welcome contributions from everyone.
OpenHands is a community-driven project, and we welcome contributions from everyone.
Whether you're a developer, a researcher, or simply enthusiastic about advancing the field of
software engineering with AI, there are many ways to get involved:
- **Code Contributions:** Help us develop new agents, core functionality, the frontend and other interfaces, or sandboxing solutions.
- **Research and Evaluation:** Contribute to our understanding of LLMs in software engineering, participate in evaluating the models, or suggest improvements.
- **Feedback and Testing:** Use the OpenDevin toolset, report bugs, suggest features, or provide feedback on usability.
- **Feedback and Testing:** Use the OpenHands toolset, report bugs, suggest features, or provide feedback on usability.
For details, please check [CONTRIBUTING.md](./CONTRIBUTING.md).
## 🤖 Join Our Community
Whether you're a developer, a researcher, or simply enthusiastic about OpenDevin, we'd love to have you in our community.
Whether you're a developer, a researcher, or simply enthusiastic about OpenHands, we'd love to have you in our community.
Let's make software engineering better together!
- [Slack workspace](https://join.slack.com/t/opendevin/shared_invite/zt-2jsrl32uf-fTeeFjNyNYxqSZt5NPY3fA) - Here we talk about research, architecture, and future development.
- [Slack workspace](https://join.slack.com/t/opendevin/shared_invite/zt-2oikve2hu-UDxHeo8nsE69y6T7yFX_BA) - Here we talk about research, architecture, and future development.
- [Discord server](https://discord.gg/ESHStjSjD4) - This is a community-run server for general discussion, questions, and feedback.
OpenHands is built by a large number of contributors, and every contribution is greatly appreciated! We also build upon other open source projects, and we are deeply thankful fortheir work.
For a list of open source projects and licenses used in OpenHands, please see our [CREDITS.md](./CREDITS.md) file.
## 📚 Cite
```
@misc{opendevin,
title={{OpenDevin: An Open Platform for AI Software Developers as Generalist Agents}},
@misc{openhands,
title={{OpenHands: An Open Platform for AI Software Developers as Generalist Agents}},
author={Xingyao Wang and Boxuan Li and Yufan Song and Frank F. Xu and Xiangru Tang and Mingchen Zhuge and Jiayi Pan and Yueqi Song and Bowen Li and Jaskirat Singh and Hoang H. Tran and Fuqiang Li and Ren Ma and Mingzhang Zheng and Bill Qian and Yanjun Shao and Niklas Muennighoff and Yizhe Zhang and Binyuan Hui and Junyang Lin and Robert Brennan and Hao Peng and Heng Ji and Graham Neubig},
In this folder, there may exist multiple implementations of `Agent` that will be used by the framework.
For example, `agenthub/codeact_agent`, etc.
Contributors from different backgrounds and interests can choose to contribute to any (or all!) of these directions.
## Constructing an Agent
The abstraction for an agent can be found [here](../opendevin/controller/agent.py).
Agents are run inside of a loop. At each iteration, `agent.step()` is called with a
[State](../opendevin/controller/state/state.py) input, and the agent must output an [Action](../opendevin/events/action).
Every agent also has a `self.llm` which it can use to interact with the LLM configured by the user.
See the [LiteLLM docs for `self.llm.completion`](https://docs.litellm.ai/docs/completion).
## State
The `state` contains:
- A history of actions taken by the agent, as well as any observations (e.g. file content, command output) from those actions
- A list of actions/observations that have happened since the most recent step
- A [`root_task`](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/controller/state/task.py), which contains a plan of action
- The agent can add and modify subtasks through the `AddTaskAction` and `ModifyTaskAction`
## Actions
Here is a list of available Actions, which can be returned by `agent.step()`:
- [`CmdRunAction`](../opendevin/events/action/commands.py) - Runs a command inside a sandboxed terminal
- [`IPythonRunCellAction`](../opendevin/events/action/commands.py) - Execute a block of Python code interactively (in Jupyter notebook) and receives `CmdOutputObservation`. Requires setting up `jupyter` [plugin](../opendevin/runtime/plugins) as a requirement.
- [`FileReadAction`](../opendevin/events/action/files.py) - Reads the content of a file
- [`FileWriteAction`](../opendevin/events/action/files.py) - Writes new content to a file
- [`BrowseURLAction`](../opendevin/events/action/browse.py) - Gets the content of a URL
- [`AddTaskAction`](../opendevin/events/action/tasks.py) - Adds a subtask to the plan
- [`ModifyTaskAction`](../opendevin/events/action/tasks.py) - Changes the state of a subtask.
- [`AgentFinishAction`](../opendevin/events/action/agent.py) - Stops the control loop, allowing the user/delegator agent to enter a new task
- [`AgentRejectAction`](../opendevin/events/action/agent.py) - Stops the control loop, allowing the user/delegator agent to enter a new task
- [`AgentFinishAction`](../opendevin/events/action/agent.py) - Stops the control loop, allowing the user to enter a new task
- [`MessageAction`](../opendevin/events/action/message.py) - Represents a message from an agent or the user
You can use `action.to_dict()` and `action_from_dict` to serialize and deserialize actions.
## Observations
There are also several types of Observations. These are typically available in the step following the corresponding Action.
But they may also appear as a result of asynchronous events (e.g. a message from the user).
This folder implements the CodeAct idea ([paper](https://arxiv.org/abs/2402.01030), [tweet](https://twitter.com/xingyaow_/status/1754556835703751087)) that consolidates LLM agents’**act**ions into a unified **code** action space for both *simplicity* and *performance* (see paper for more details).
The conceptual idea is illustrated below. At each turn, the agent can:
1.**Converse**: Communicate with humans in natural language to ask for clarification, confirmation, etc.
2.**CodeAct**: Choose to perform the task by executing code
- Execute any valid Linux `bash` command
- Execute any valid `Python` code with [an interactive Python interpreter](https://ipython.org/). This is simulated through `bash` command, see plugin system below for more details.
To make the CodeAct agent more powerful with only access to `bash` action space, CodeAct agent leverages OpenDevin's plugin system:
- [Jupyter plugin](https://github.com/OpenDevin/OpenDevin/tree/main/opendevin/runtime/plugins/jupyter): for IPython execution via bash command
- [SWE-agent tool plugin](https://github.com/OpenDevin/OpenDevin/tree/main/opendevin/runtime/plugins/swe_agent_commands): Powerful bash command line tools for software development tasks introduced by [swe-agent](https://github.com/princeton-nlp/swe-agent).
The agent works by passing the model a list of action-observation pairs and prompting the model to take the next step.
### Overview
This agent implements the CodeAct idea ([paper](https://arxiv.org/abs/2402.01030), [tweet](https://twitter.com/xingyaow_/status/1754556835703751087)) that consolidates LLM agents’ **act**ions into a unified **code** action space for both *simplicity* and *performance* (see paper for more details).
The conceptual idea is illustrated below. At each turn, the agent can:
1. **Converse**: Communicate with humans in natural language to ask for clarification, confirmation, etc.
2. **CodeAct**: Choose to perform the task by executing code
- Execute any valid Linux `bash` command
- Execute any valid `Python` code with [an interactive Python interpreter](https://ipython.org/). This is simulated through `bash` command, see plugin system below for more details.
To make the CodeAct agent more powerful with only access to `bash` action space, CodeAct agent leverages OpenDevin's plugin system:
- [Jupyter plugin](https://github.com/OpenDevin/OpenDevin/tree/main/opendevin/runtime/plugins/jupyter): for IPython execution via bash command
- [SWE-agent tool plugin](https://github.com/OpenDevin/OpenDevin/tree/main/opendevin/runtime/plugins/swe_agent_commands): Powerful bash command line tools for software development tasks introduced by [swe-agent](https://github.com/princeton-nlp/swe-agent).
in_context_example:str=f"Here is an example of how you can interact with the environment for task solving:\n{get_in_context_example()}\n\nNOW, LET'S START!"
action_parser=CodeActResponseParser()
def__init__(
self,
llm:LLM,
)->None:
"""Initializes a new instance of the CodeActAgent class.
f'\n\nENVIRONMENT REMINDER: You have {state.max_iterations-state.iteration} turns left to complete the task. When finished reply with <finish></finish>'
'\nApart from the standard Python library, the assistant can also use the following functions (already imported) in <execute_ipython> environment:\n'
f'{_AGENT_SKILLS_DOCS}'
"Please note that THE `edit_file_by_replace`, `append_file` and `insert_content_at_line` FUNCTIONS REQUIRE PROPER INDENTATION. If the assistant would like to add the line ' print(x)', it must fully write that out, with all those spaces before the code! Indentation is important and code that is not indented correctly will fail and require fixing before it can be run."
)
# ======= SYSTEM MESSAGE =======
MINIMAL_SYSTEM_PREFIX="""A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
The assistant can use an interactive Python (Jupyter Notebook) environment, executing code with <execute_ipython>.
<execute_ipython>
print("Hello World!")
</execute_ipython>
The assistant can execute bash commands on behalf of the user by wrapping them with <execute_bash> and </execute_bash>.
For example, you can list the files in the current directory by <execute_bash> ls </execute_bash>.
Important, however: do not run interactive commands. You do not have access to stdin.
Also, you need to handle commands that may run indefinitely and not return a result. For such cases, you should redirect the output to a file and run the command in the background to avoid blocking the execution.
For example, to run a Python script that might run indefinitely without returning immediately, you can use the following format: <execute_bash> python3 app.py > server.log 2>&1 & </execute_bash>
Also, if a command execution result saying like: Command: "npm start" timed out. Sending SIGINT to the process, you should also retry with running the command in the background.
"""
BROWSING_PREFIX="""The assistant can browse the Internet with <execute_browse> and </execute_browse>.
For example, <execute_browse> Tell me the usa's president using google search </execute_browse>.
Or <execute_browse> Tell me what is in http://example.com </execute_browse>.
"""
PIP_INSTALL_PREFIX="""The assistant can install Python packages using the %pip magic command in an IPython environment by using the following syntax: <execute_ipython> %pip install [package needed] </execute_ipython> and should always import packages and define variables before starting to use them."""
If $GITHUB_TOKEN is not set, ask the user to set it."""
SYSTEM_SUFFIX="""Responses should be concise.
The assistant should attempt fewer things at a time instead of putting too many commands OR too much code in one "execute" block.
Include ONLY ONE <execute_ipython>, <execute_bash>, or <execute_browse> per response, unless the assistant is finished with the task or needs more input or action from the user in order to proceed.
If the assistant is finished with the task you MUST include <finish></finish> in your response.
IMPORTANT: Execute code using <execute_ipython>, <execute_bash>, or <execute_browse> whenever possible.
When handling files, try to use full paths and pwd to avoid errors.
"""
# ======= EXAMPLE MESSAGE =======
EXAMPLES="""
--- START OF EXAMPLE ---
USER: Create a list of numbers from 1 to 10, and display them in a web page at port 5000.
ASSISTANT:
Sure! Let me create a Python file `app.py`:
<execute_ipython>
create_file('app.py')
</execute_ipython>
USER:
OBSERVATION:
[File: /workspace/app.py (1 lines total)]
(this is the beginning of the file)
1|
(this is the end of the file)
[File app.py created.]
ASSISTANT:
Now I will write the Python code for starting a web server and save it to the file `app.py`:
<execute_ipython>
EDITED_CODE=\"\"\"from flask import Flask
app = Flask(__name__)
@app.route('/')
def index():
numbers = list(range(1, 11))
return str(numbers)
if __name__ == '__main__':
app.run(port=5000)\"\"\"
insert_content_at_line(
'app.py',
1,
EDITED_CODE,
)
</execute_ipython>
USER:
OBSERVATION:
(this is the beginning of the file)
1|from flask import Flask
2|app = Flask(__name__)
3|
4|@app.route('/')
5|def index():
6| numbers = list(range(1, 11))
7| return str(numbers)
8|
9|if __name__ == '__main__':
10| app.run(port=5000)
(this is the end of the file)
[File updated (edited at line 1). Please review the changes and make sure they are correct (correct indentation, no duplicate lines, etc). Edit the file again if necessary.]
ASSISTANT:
I have created a Python file `app.py` that will display a list of numbers from 1 to 10 when you run it. Let me run the Python file for you:
Now that Flask is installed, let me run the Python file again:
<execute_bash>
python3 app.py > server.log 2>&1 &
</execute_bash>
USER:
OBSERVATION:
[1] 124
ASSISTANT:
Let me check the server log again:
<execute_bash>
cat server.log
</execute_bash>
USER:
OBSERVATION:
* Serving Flask app 'app'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:5000
Press CTRL+C to quit
ASSISTANT:
The server is running on port 5000 with PID 124. You can access the list of numbers by visiting http://127.0.0.1:5000. If you have any further questions, feel free to ask!
USER: Now browse the newly started server's homepage and show me the content.
ASSISTANT:
Sure! Let me browse the server's homepage at http://127.0.0.1:5000:
<execute_browse>
Get the content on "http://127.0.0.1:5000"
</execute_browse>
USER:
OBSERVATION:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
ASSISTANT:
The content of the server's homepage is "[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]". If you have any further questions, feel free to ask!
USER: Now kill the server, make it display the numbers in a table format.
ASSISTANT:
Sure! Let me stop the server first:
<execute_bash>
kill 124
</execute_bash>
USER:
OBSERVATION:
[1]+ Terminated python3 app.py > server.log 2>&1
ASSISTANT:
The server has been stopped. Let me open the Python file and modify it.
<execute_ipython>
open_file('app.py')
</execute_ipython>
USER:
[File: /workspace/app.py (10 lines total)]
(this is the beginning of the file)
1|from flask import Flask
2|app = Flask(__name__)
3|
4|@app.route('/')
5|def index():
6| numbers = list(range(1, 11))
7| return str(numbers)
8|
9|if __name__ == '__main__':
10| app.run(port=5000)
(this is the end of the file)
ASSISTANT:
I should edit the file to display the numbers in a table format. I should include correct indentation. Let me update the file:
<execute_ipython>
edit_file_by_replace(
'app.py',
to_replace=" return str(numbers)",
new_content=" return '<table>' + ''.join([f'<tr><td>{i}</td></tr>' for i in numbers]) + '</table>'",
)
</execute_ipython>
USER:
Observation:
[File: /workspace/app.py (10 lines total after edit)]
(this is the beginning of the file)
1|from flask import Flask
2|app = Flask(__name__)
3|
4|@app.route('/')
5|def index():
6| numbers = list(range(1, 11))
7| return '<table>' + ''.join([f'<tr><td>{i}</td></tr>' for i in numbers]) + '</table>'
8|
9|if __name__ == '__main__':
10| app.run(port=5000)
(this is the end of the file)
[File updated (edited at line 7). Please review the changes and make sure they are correct (correct indentation, no duplicate lines, etc). Edit the file again if necessary.]
ASSISTANT:
Running the updated file:
<execute_bash>
python3 app.py > server.log 2>&1 &
</execute_bash>
USER:
Observation:
[1] 126
ASSISTANT:
The server is running on port 5000 with PID 126. You can access the list of numbers in a table format by visiting http://127.0.0.1:5000. Let me know if you have any further requests!
<finish></finish>
--- END OF EXAMPLE ---
"""
INVALID_INPUT_MESSAGE=(
"I don't understand your input. \n"
'For bash commands, use <execute_bash> YOUR_COMMAND </execute_bash>.\n'
'For Python code, use <execute_ipython> YOUR_CODE </execute_ipython>.\n'
'For browsing, use <execute_browse> YOUR_COMMAND </execute_browse>.\n'
Install [Docker](https://docs.docker.com/engine/install/) on your host machine and run:
```bash
make docker-dev
# same as:
cd ./containers/dev
./dev.sh
```
It could take some time if you are running for the first time as Docker will pull all the tools required for building OpenHands. The next time you run again, it should be instant.
## Build and run
If everything goes well, you should be inside a container after Docker finishes building the `openhands:dev` image similar to the following:
```bash
Build and run in Docker ...
root@93fc0005fcd2:/app#
```
You may now proceed with the normal [build and run](../../Development.md) workflow as if you were on the host.
## Make changes
The source code on the host is mounted as `/app` inside docker. You may edit the files as usual either inside the Docker container or on your host with your favorite IDE/editors.
The following are also mapped as readonly from your host:
[E2B](https://e2b.dev) is an [open-source](https://github.com/e2b-dev/e2b) secure cloud environment (sandbox) made for running AI-generated code and agents. E2B offers [Python](https://pypi.org/project/e2b/) and [JS/TS](https://www.npmjs.com/package/e2b) SDK to spawn and control these sandboxes.
This folder builds runtime image (sandbox), which will use a `Dockerfile` that is dynamically generated depends on the `base_image` AND a [Python source distribution](https://docs.python.org/3.10/distutils/sourcedist.html) that's based on the current commit of `opendevin`.
This folder builds a runtime image (sandbox), which will use a dynamically generated `Dockerfile`
that depends on the `base_image`**AND** a [Python source distribution](https://docs.python.org/3.10/distutils/sourcedist.html) that is based on the current commit of `openhands`.
The following command will generate Dockerfile for `ubuntu:22.04` and the source distribution `.tar` into `containers/runtime`.
The following command will generate a `Dockerfile` file for `nikolaik/python-nodejs:python3.12-nodejs22` (the default base image), an updated `config.sh` and the runtime source distribution files/folders into `containers/runtime`:
```bash
poetry run python3 opendevin/runtime/utils/runtime_build.py \
--base_image ubuntu:22.04\
poetry run python3 openhands/runtime/utils/runtime_build.py \
"message":"est un ingénieur logiciel autonome qui peut résoudre des tâches d'ingénierie logicielle et de navigation web à tout moment. Il peut exécuter des requêtes en sciences des données, telles que \"Trouver le nombre de demandes de pull à l'repository OpenDevin dans les derniers mois\", et des tâches d'ingénierie logicielle, comme \"Veuillez ajouter des tests à ce fichier et vérifier si tous les tests passent. Si ce n'est pas le cas, réparez le fichier.\"",
"description":"Description for OpenDevin"
"message":"est un ingénieur logiciel autonome qui peut résoudre des tâches d'ingénierie logicielle et de navigation web à tout moment. Il peut exécuter des requêtes en sciences des données, telles que \"Trouver le nombre de demandes de pull à l'repository OpenHands dans les derniers mois\", et des tâches d'ingénierie logicielle, comme \"Veuillez ajouter des tests à ce fichier et vérifier si tous les tests passent. Si ce n'est pas le cas, réparez le fichier.\"",
"description":"Description for OpenHands"
},
"faq.section.description.2":{
"message":"De plus, OpenDevin est une plateforme et communauté pour les développeurs d'agents qui souhaitent tester et évaluer de nouveaux agents.",
"description":"Further Description for OpenDevin"
"message":"De plus, OpenHands est une plateforme et communauté pour les développeurs d'agents qui souhaitent tester et évaluer de nouveaux agents.",
"description":"Further Description for OpenHands"
},
"faq.section.title.2":{
"message":"Support",
@@ -43,15 +43,15 @@
"description":"Support Answer"
},
"faq.section.title.3":{
"message":"Comment résoudre un problème sur GitHub avec OpenDevin ?",
"message":"Comment résoudre un problème sur GitHub avec OpenHands ?",
"description":"GitHub Issue Section Title"
},
"faq.section.github.steps.intro":{
"message":"Pour résoudre un problème sur GitHub en utilisant OpenDevin, envoyez une commande à OpenDevin demandant qu'il suit des étapes comme les suivantes :",
"message":"Pour résoudre un problème sur GitHub en utilisant OpenHands, envoyez une commande à OpenHands demandant qu'il suit des étapes comme les suivantes :",
"message":"Avant de lancer OpenDevin, vous pouvez faire :",
"message":"Avant de lancer OpenHands, vous pouvez faire :",
"description":"GitHub Steps Pre-Run"
},
"faq.section.github.steps.tokenInfo":{
"message":"où XXX est un jeton GitHub que vous avez créé et qui a les autorisations pour pousser dans le dépôt OpenDevin. Si vous n'avez pas d'autorisations de modification du dépôt OpenDevin, vous devrez peut-être changer cela en :",
"message":"où XXX est un jeton GitHub que vous avez créé et qui a les autorisations pour pousser dans le dépôt OpenHands. Si vous n'avez pas d'autorisations de modification du dépôt OpenHands, vous devrez peut-être changer cela en :",
"description":"GitHub Steps Token Info"
},
"faq.section.github.steps.usernameInfo":{
@@ -83,23 +83,23 @@
"description":"GitHub Steps Username Info"
},
"faq.section.title.4":{
"message":"Comment OpenDevin est-il différent de Devin ?",
"message":"Comment OpenHands est-il différent de Devin ?",
"description":"Devin Section Title"
},
"faq.section.devin.linkText":{
"faq.section.openhands.linkText":{
"message":"Devin",
"description":"Devin Link Text"
},
"faq.section.devin.description":{
"message":"est un produit commercial par Cognition Inc., qui a servi d'inspiration initiale pour OpenDevin. Les deux visent à bien faire le travail d'ingénierie logicielle, mais vous pouvez télécharger, utiliser et modifier OpenDevin, tandis que Devin peut être utilisé uniquement via le site de Cognition. De plus, OpenDevin a évolué au-delà de l'inspiration initiale, et est maintenant un écosystème communautaire pour le développement d'agents en général, et nous serions ravis de vous voir rejoindre et",
"faq.section.openhands.description":{
"message":"est un produit commercial par Cognition Inc., qui a servi d'inspiration initiale pour OpenHands. Les deux visent à bien faire le travail d'ingénierie logicielle, mais vous pouvez télécharger, utiliser et modifier OpenHands, tandis que Devin peut être utilisé uniquement via le site de Cognition. De plus, OpenHands a évolué au-delà de l'inspiration initiale, et est maintenant un écosystème communautaire pour le développement d'agents en général, et nous serions ravis de vous voir rejoindre et",
"description":"Devin Description"
},
"faq.section.devin.contribute":{
"faq.section.openhands.contribute":{
"message":"contribuer",
"description":"Contribute Link"
},
"faq.section.title.5":{
"message":"Comment OpenDevin est-il différent de ChatGPT ?",
"message":"Comment OpenHands est-il différent de ChatGPT ?",
"description":"ChatGPT Section Title"
},
"faq.section.chatgpt.description":{
@@ -114,7 +114,7 @@
"message":"Commencer"
},
"welcome.message":{
"message":"Bienvenue à OpenDevin, un système d'IA autonome ingénieur logiciel capable d'exécuter des tâches d'ingénierie complexes et de collaborer activement avec les utilisateurs sur les projets de développement logiciel."
"message":"Bienvenue à OpenHands, un système d'IA autonome ingénieur logiciel capable d'exécuter des tâches d'ingénierie complexes et de collaborer activement avec les utilisateurs sur les projets de développement logiciel."
@@ -19,35 +19,35 @@ La réalisation d'une réplication complète des applications de production avec
## 🤝 Comment Contribuer {#how-to-contribute}
OpenDevin est un projet communautaire, et nous accueillons les contributions de tout le monde. Que vous soyez développeur, chercheur, ou simplement enthousiaste à l'idée de faire progresser le domaine de l'ingénierie logicielle avec l'IA, il existe de nombreuses façons de vous impliquer :
OpenHands est un projet communautaire, et nous accueillons les contributions de tout le monde. Que vous soyez développeur, chercheur, ou simplement enthousiaste à l'idée de faire progresser le domaine de l'ingénierie logicielle avec l'IA, il existe de nombreuses façons de vous impliquer :
- **Contributions de Code :** Aidez-nous à développer les fonctionnalités de base, l'interface frontend ou les solutions de sandboxing.
- **Recherche et Évaluation :** Contribuez à notre compréhension des LLM en ingénierie logicielle, participez à l'évaluation des modèles ou suggérez des améliorations.
- **Retour d'Information et Tests :** Utilisez l'ensemble d'outils OpenDevin, signalez des bogues, suggérez des fonctionnalités ou fournissez des retours sur l'ergonomie.
- **Retour d'Information et Tests :** Utilisez l'ensemble d'outils OpenHands, signalez des bogues, suggérez des fonctionnalités ou fournissez des retours sur l'ergonomie.
Pour plus de détails, veuillez consulter [ce document](https://github.com/OpenDevin/OpenDevin/blob/main/CONTRIBUTING.md).
Pour plus de détails, veuillez consulter [ce document](https://github.com/All-Hands-AI/OpenHands/blob/main/CONTRIBUTING.md).
Nous avons maintenant à la fois un espace de travail Slack pour la collaboration sur la construction d'OpenDevin et un serveur Discord pour discuter de tout ce qui est lié, par exemple, à ce projet, aux LLM, aux agents, etc.
Nous avons maintenant à la fois un espace de travail Slack pour la collaboration sur la construction d'OpenHands et un serveur Discord pour discuter de tout ce qui est lié, par exemple, à ce projet, aux LLM, aux agents, etc.
- [Espace de travail Slack](https://join.slack.com/t/opendevin/shared_invite/zt-2jsrl32uf-fTeeFjNyNYxqSZt5NPY3fA)
- [Espace de travail Slack](https://join.slack.com/t/opendevin/shared_invite/zt-2oikve2hu-UDxHeo8nsE69y6T7yFX_BA)
Si vous souhaitez contribuer, n'hésitez pas à rejoindre notre communauté. Simplifions l'ingénierie logicielle ensemble !
🐚 **Codez moins, créez plus avec OpenDevin.**
🐚 **Codez moins, créez plus avec OpenHands.**
[](https://star-history.com/#OpenDevin/OpenDevin&Date)
[](https://star-history.com/#All-Hands-AI/OpenHands&Date)
## 🛠️ Construit Avec {#built-with}
OpenDevin est construit en utilisant une combinaison de cadres et de bibliothèques puissants, offrant une base robuste pour son développement. Voici les technologies clés utilisées dans le projet :
OpenHands est construit en utilisant une combinaison de cadres et de bibliothèques puissants, offrant une base robuste pour son développement. Voici les technologies clés utilisées dans le projet :
Veuillez noter que la sélection de ces technologies est en cours, et que des technologies supplémentaires peuvent être ajoutées ou des existantes supprimées au fur et à mesure de l'évolution du projet. Nous nous efforçons d'adopter les outils les plus adaptés et efficaces pour améliorer les capacités d'OpenDevin.
Veuillez noter que la sélection de ces technologies est en cours, et que des technologies supplémentaires peuvent être ajoutées ou des existantes supprimées au fur et à mesure de l'évolution du projet. Nous nous efforçons d'adopter les outils les plus adaptés et efficaces pour améliorer les capacités d'OpenHands.
## 📜 Licence {#license}
Distribué sous la licence MIT. Voir [notre licence](https://github.com/OpenDevin/OpenDevin/blob/main/LICENSE) pour plus d'informations.
Distribué sous la licence MIT. Voir [notre licence](https://github.com/All-Hands-AI/OpenHands/blob/main/LICENSE) pour plus d'informations.
@@ -18,18 +18,18 @@ L'idée conceptuelle est illustrée ci-dessous. À chaque tour, l'agent peut :
- Exécuter toute commande `bash` Linux valide
- Exécuter tout code `Python` valide avec [un interpréteur Python interactif](https://ipython.org/). Cela est simulé à travers la commande `bash`, voir le système de plugin ci-dessous pour plus de détails.
Pour rendre l'agent CodeAct plus puissant avec seulement l'accès à l'espace d'action `bash`, l'agent CodeAct exploite le système de plugins d'OpenDevin:
Pour rendre l'agent CodeAct plus puissant avec seulement l'accès à l'espace d'action `bash`, l'agent CodeAct exploite le système de plugins d'OpenHands:
- [Plugin Jupyter](https://github.com/OpenDevin/OpenDevin/tree/main/opendevin/runtime/plugins/jupyter) : pour l'exécution d'IPython via la commande bash
- [Plugin outil agent SWE](https://github.com/OpenDevin/OpenDevin/tree/main/opendevin/runtime/plugins/swe_agent_commands) : Outils de ligne de commande bash puissants pour les tâches de développement logiciel introduits par [swe-agent](https://github.com/princeton-nlp/swe-agent).
- [Plugin Jupyter](https://github.com/All-Hands-AI/OpenHands/tree/main/openhands/runtime/plugins/jupyter) : pour l'exécution d'IPython via la commande bash
- [Plugin outil agent SWE](https://github.com/All-Hands-AI/OpenHands/tree/main/openhands/runtime/plugins/swe_agent_commands) : Outils de ligne de commande bash puissants pour les tâches de développement logiciel introduits par [swe-agent](https://github.com/princeton-nlp/swe-agent).
2. Ouvrez le fichier généré dans un éditeur PlantUML, par exemple Visual Studio Code avec l'extension PlantUML ou [PlantText](https://www.planttext.com/)
Le sandbox par défaut OpenDevin est équipé d'une configuration ubuntu minimaliste. Votre cas d'utilisation pourrait nécessiter des logiciels installés par défaut. Cet article vous enseignera comment réaliser cela en utilisant une image docker personnalisée.
Le sandbox par défaut OpenHands est équipé d'une configuration ubuntu minimaliste. Votre cas d'utilisation pourrait nécessiter des logiciels installés par défaut. Cet article vous enseignera comment réaliser cela en utilisant une image docker personnalisée.
## Configuration
Assurez-vous de pouvoir utiliser OpenDevin en suivant la documentation [Development.md](https://github.com/OpenDevin/OpenDevin/blob/main/Development.md).
Assurez-vous de pouvoir utiliser OpenHands en suivant la documentation [Development.md](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md).
## Créer Votre Image Docker
Ensuite, vous devez créer votre image docker personnalisée qui doit être basée sur debian/ubuntu. Par exemple, si nous souhaitons que OpenDevin ait accès au "node" binaire, nous utiliserions ce Dockerfile:
Ensuite, vous devez créer votre image docker personnalisée qui doit être basée sur debian/ubuntu. Par exemple, si nous souhaitons que OpenHands ait accès au "node" binaire, nous utiliserions ce Dockerfile:
Cela produira une nouvelle image appelée ```image_personnalisée``` qui sera disponible dans Docker Engine.
> Remarque: Dans la configuration décrite ici, OpenDevin va fonctionner en tant que utilisateur "opendevin" à l'intérieur du sandbox et donc tous les packages installés via le Dockerfile seront disponibles pour tous les utilisateurs sur le système, pas seulement root.
> Remarque: Dans la configuration décrite ici, OpenHands va fonctionner en tant que utilisateur "openhands" à l'intérieur du sandbox et donc tous les packages installés via le Dockerfile seront disponibles pour tous les utilisateurs sur le système, pas seulement root.
>
> L'installation avec apt-get ci-dessus installe nodejs pour tous les utilisateurs.
## Spécifiez votre image personnalisée dans le fichier config.toml
La configuration OpenDevin se fait via le fichier de niveau supérieur ```config.toml``` .
Créez un fichier ```config.toml``` dans le répertoire OpenDevin et entrez ces contenus:
La configuration OpenHands se fait via le fichier de niveau supérieur ```config.toml``` .
Créez un fichier ```config.toml``` dans le répertoire OpenHands et entrez ces contenus:
> Assurez-vous que ```sandbox_container_image``` est défini sur le nom de votre image personnalisée précédente.
> Assurez-vous que ```sandbox_base_container_image``` est défini sur le nom de votre image personnalisée précédente.
## Exécution
Exécutez OpenDevin en exécutant ```make run``` dans le répertoire racine.
Exécutez OpenHands en exécutant ```make run``` dans le répertoire racine.
Naviguez vers ```localhost:3001``` et vérifiez si vos dépendances souhaitées sont disponibles.
@@ -60,41 +59,36 @@ Félicitations !
## Explication technique
Le code pertinent est défini dans [ssh_box.py](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/ssh_box.py) et [image_agnostic_util.py](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/image_agnostic_util.py).
En particulier, ssh_box.py vérifie l'objet config pour ```config.sandbox_container_image``` et ensuite tente de récupérer l'image à l'aide de [get_od_sandbox_image](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/image_agnostic_util.py#L72), qui est défini dans image_agnostic_util.py.
Lorsqu'une image personnalisée est utilisée pour la première fois, elle ne sera pas trouvée et donc elle sera construite (à l'exécution ultérieure, l'image construite sera trouvée et renvoyée).
L'image personnalisée est construite avec [_build_sandbox_image()](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/image_agnostic_util.py#L29), qui crée un fichier docker en utilisant votre image personnalisée comme base et configure ensuite l'environnement pour OpenDevin, comme ceci:
L'image personnalisée est construite avec [_build_sandbox_image()](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/image_agnostic_util.py#L29), qui crée un fichier docker en utilisant votre image personnalisée comme base et configure ensuite l'environnement pour OpenHands, comme ceci:
> Remarque: Le nom de l'image est modifié via [_get_new_image_name()](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/image_agnostic_util.py#L63) et c'est ce nom modifié qui sera recherché lors des exécutions ultérieures.
> Remarque: Le nom de l'image est modifié via [_get_new_image_name()](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/docker/image_agnostic_util.py#L63) et c'est ce nom modifié qui sera recherché lors des exécutions ultérieures.
## Dépannage / Erreurs
### Erreur: ```useradd: UID 1000 est non unique```
Si vous voyez cette erreur dans la sortie de la console, il s'agit du fait que OpenDevin essaie de créer le utilisateur opendevin dans le sandbox avec un ID d'utilisateur de 1000, cependant cet ID d'utilisateur est déjà utilisé dans l'image (pour une raison inconnue). Pour résoudre ce problème, changez la valeur du champ sandbox_user_id dans le fichier config.toml en une valeur différente:
Si vous voyez cette erreur dans la sortie de la console, il s'agit du fait que OpenHands essaie de créer le utilisateur openhands dans le sandbox avec un ID d'utilisateur de 1000, cependant cet ID d'utilisateur est déjà utilisé dans l'image (pour une raison inconnue). Pour résoudre ce problème, changez la valeur du champ sandbox_user_id dans le fichier config.toml en une valeur différente:
@@ -104,4 +98,4 @@ Si vous voyez un message d'erreur indiquant que le port est utilisé ou indispon
## Discuter
Pour d'autres problèmes ou questions rejoignez le [Slack](https://join.slack.com/t/opendevin/shared_invite/zt-2jsrl32uf-fTeeFjNyNYxqSZt5NPY3fA) ou le [Discord](https://discord.gg/ESHStjSjD4) et demandez!
Pour d'autres problèmes ou questions rejoignez le [Slack](https://join.slack.com/t/opendevin/shared_invite/zt-2oikve2hu-UDxHeo8nsE69y6T7yFX_BA) ou le [Discord](https://discord.gg/ESHStjSjD4) et demandez!
Lorsque vous utilisez OpenDevin, vous rencontrerez sans aucun doute des cas où les choses fonctionnent bien et d'autres où elles ne fonctionnent pas. Nous vous encourageons à fournir des commentaires lorsque vous utilisez OpenDevin pour aider l'équipe de développement et, peut-être plus important encore, créer un corpus ouvert d'exemples de formation pour les agents de codage -- Partagez-OpenDevin !
Lorsque vous utilisez OpenHands, vous rencontrerez sans aucun doute des cas où les choses fonctionnent bien et d'autres où elles ne fonctionnent pas. Nous vous encourageons à fournir des commentaires lorsque vous utilisez OpenHands pour aider l'équipe de développement et, peut-être plus important encore, créer un corpus ouvert d'exemples de formation pour les agents de codage -- Partagez-OpenHands !
## 📝 Comment Fournir des Commentaires
Fournir des commentaires est simple ! Lorsque vous utilisez OpenDevin, vous pouvez appuyer sur le bouton de pouce vers le haut ou vers le bas à n'importe quel moment de votre interaction. Vous serez invité à fournir votre adresse email (par exemple, afin que nous puissions vous contacter si nous voulons poser des questions de suivi), et vous pouvez choisir si vous souhaitez fournir des commentaires publiquement ou en privé.
Fournir des commentaires est simple ! Lorsque vous utilisez OpenHands, vous pouvez appuyer sur le bouton de pouce vers le haut ou vers le bas à n'importe quel moment de votre interaction. Vous serez invité à fournir votre adresse email (par exemple, afin que nous puissions vous contacter si nous voulons poser des questions de suivi), et vous pouvez choisir si vous souhaitez fournir des commentaires publiquement ou en privé.
* Les données **publiques** seront distribuées sous la licence MIT, comme OpenDevin lui-même, et pourront être utilisées par la communauté pour former et tester des modèles. Évidemment, les commentaires que vous pouvez rendre publics seront plus précieux pour la communauté dans son ensemble, donc lorsque vous ne traitez pas d'informations sensibles, nous vous encourageons à choisir cette option !
* Les données **privées** ne seront partagées qu'avec l'équipe OpenDevin dans le but d'améliorer OpenDevin.
* Les données **publiques** seront distribuées sous la licence MIT, comme OpenHands lui-même, et pourront être utilisées par la communauté pour former et tester des modèles. Évidemment, les commentaires que vous pouvez rendre publics seront plus précieux pour la communauté dans son ensemble, donc lorsque vous ne traitez pas d'informations sensibles, nous vous encourageons à choisir cette option !
* Les données **privées** ne seront partagées qu'avec l'équipe OpenHands dans le but d'améliorer OpenHands.
OpenDevin est un **ingénieur logiciel IA autonome** capable d'exécuter des tâches d'ingénierie complexes et de collaborer activement avec les utilisateurs sur des projets de développement logiciel.
OpenHands est un **ingénieur logiciel IA autonome** capable d'exécuter des tâches d'ingénierie complexes et de collaborer activement avec les utilisateurs sur des projets de développement logiciel.
Ce projet est entièrement open-source, vous pouvez donc l'utiliser et le modifier comme bon vous semble.
:::tip
Explorez le code source d'OpenDevin sur [GitHub](https://github.com/OpenDevin/OpenDevin) ou rejoignez l'une de nos communautés !
Explorez le code source d'OpenHands sur [GitHub](https://github.com/All-Hands-AI/OpenHands) ou rejoignez l'une de nos communautés !
@@ -58,10 +58,10 @@ Explorez le code source d'OpenDevin sur [GitHub](https://github.com/OpenDevin/Op
## 🛠️ Pour commencer
La manière la plus simple d'exécuter OpenDevin est à l'intérieur d'un conteneur Docker. Il fonctionne mieux avec la version la plus récente de Docker, `26.0.0`.
La manière la plus simple d'exécuter OpenHands est à l'intérieur d'un conteneur Docker. Il fonctionne mieux avec la version la plus récente de Docker, `26.0.0`.
Vous devez utiliser Linux, Mac OS ou WSL sur Windows.
Pour démarrer OpenDevin dans un conteneur docker, exécutez les commandes suivantes dans votre terminal :
Pour démarrer OpenHands dans un conteneur docker, exécutez les commandes suivantes dans votre terminal :
:::warning
Lorsque vous exécutez la commande suivante, les fichiers dans `./workspace` peuvent être modifiés ou supprimés.
Vous trouverez OpenDevin fonctionnant à l'adresse [http://localhost:3000](http://localhost:3000) avec accès à `./workspace`. Pour qu'OpenDevin fonctionne sur votre code, placez-le dans `./workspace`.
Vous trouverez OpenHands fonctionnant à l'adresse [http://localhost:3000](http://localhost:3000) avec accès à `./workspace`. Pour qu'OpenHands fonctionne sur votre code, placez-le dans `./workspace`.
OpenDevin n'aura accès qu'à ce dossier de workspace. Le reste de votre système ne sera pas affecté car il s'exécute dans un bac à sable sécurisé de docker.
OpenHands n'aura accès qu'à ce dossier de workspace. Le reste de votre système ne sera pas affecté car il s'exécute dans un bac à sable sécurisé de docker.
:::tip
Si vous souhaitez utiliser la version **(instable !)** la plus récente, vous pouvez utiliser `ghcr.io/opendevin/opendevin:main` comme image (dernière ligne).
Si vous souhaitez utiliser la version **(instable !)** la plus récente, vous pouvez utiliser `ghcr.io/all-hands-ai/openhands:main` comme image (dernière ligne).
:::
Pour le workflow de développement, consultez [Development.md](https://github.com/OpenDevin/OpenDevin/blob/main/Development.md).
Pour le workflow de développement, consultez [Development.md](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md).
Avez-vous des problèmes ? Consultez notre [Guide de dépannage](https://docs.all-hands.dev/modules/usage/troubleshooting).
:::warning
OpenDevin est actuellement en cours de développement, mais vous pouvez déjà exécuter la version alpha pour voir le système de bout en bout en action.
OpenHands est actuellement en cours de développement, mais vous pouvez déjà exécuter la version alpha pour voir le système de bout en bout en action.
OpenDevin utilise LiteLLM pour les appels de complétion. Vous pouvez trouver leur documentation sur Azure [ici](https://docs.litellm.ai/docs/providers/azure)
OpenHands utilise LiteLLM pour les appels de complétion. Vous pouvez trouver leur documentation sur Azure [ici](https://docs.litellm.ai/docs/providers/azure)
### Configurations openai Azure
Lors de l'exécution de l'image Docker OpenDevin, vous devrez définir les variables d'environnement suivantes en utilisant `-e` :
Lors de l'exécution de l'image Docker OpenHands, vous devrez définir les variables d'environnement suivantes en utilisant `-e` :
```
LLM_BASE_URL="<azure-api-base-url>" # e.g. "https://openai-gpt-4-test-v-1.openai.azure.com/"
@@ -16,19 +16,19 @@ LLM_API_VERSION = "<api-version>" # e.g. "2024-02-15-preview"
```
:::note
Vous pouvez trouver le nom de votre déploiement ChatGPT sur la page des déploiements sur Azure. Par défaut ou initialement, il pourrait être le même que le nom du modèle de chat (par exemple 'GPT4-1106-preview'), mais il n'est pas obligé de l'être. Exécutez OpenDevin, et une fois chargé dans le navigateur, allez dans Paramètres et définissez le modèle comme suit : "azure/<your-actual-gpt-deployment-name>". Si ce n'est pas dans la liste, entrez votre propre texte et enregistrez-le.
Vous pouvez trouver le nom de votre déploiement ChatGPT sur la page des déploiements sur Azure. Par défaut ou initialement, il pourrait être le même que le nom du modèle de chat (par exemple 'GPT4-1106-preview'), mais il n'est pas obligé de l'être. Exécutez OpenHands, et une fois chargé dans le navigateur, allez dans Paramètres et définissez le modèle comme suit : "azure/<your-actual-gpt-deployment-name>". Si ce n'est pas dans la liste, entrez votre propre texte et enregistrez-le.
:::
## Embeddings
OpenDevin utilise llama-index pour les embeddings. Vous pouvez trouver leur documentation sur Azure [ici](https://docs.llamaindex.ai/en/stable/api_reference/embeddings/azure_openai/)
OpenHands utilise llama-index pour les embeddings. Vous pouvez trouver leur documentation sur Azure [ici](https://docs.llamaindex.ai/en/stable/api_reference/embeddings/azure_openai/)
### Configurations openai Azure
Le modèle utilisé pour les embeddings Azure OpenAI est "text-embedding-ada-002".
Vous avez besoin du nom de déploiement correct pour ce modèle dans votre compte Azure.
Lors de l'exécution d'OpenDevin dans Docker, définissez les variables d'environnement suivantes en utilisant `-e` :
Lors de l'exécution d'OpenHands dans Docker, définissez les variables d'environnement suivantes en utilisant `-e` :
OpenDevin utilise LiteLLM pour les appels de complétion. Les ressources suivantes sont pertinentes pour utiliser OpenDevin avec les LLMs de Google :
OpenHands utilise LiteLLM pour les appels de complétion. Les ressources suivantes sont pertinentes pour utiliser OpenHands avec les LLMs de Google :
- [Gemini - Google AI Studio](https://docs.litellm.ai/docs/providers/gemini)
- [VertexAI - Google Cloud Platform](https://docs.litellm.ai/docs/providers/vertex)
### Configurations de Gemini - Google AI Studio
Pour utiliser Gemini via Google AI Studio lors de l'exécution de l'image Docker d'OpenDevin, vous devez définir les variables d'environnement suivantes en utilisant `-e` :
Pour utiliser Gemini via Google AI Studio lors de l'exécution de l'image Docker d'OpenHands, vous devez définir les variables d'environnement suivantes en utilisant `-e` :
### Configurations de Vertex AI - Google Cloud Platform
Pour utiliser Vertex AI via Google Cloud Platform lors de l'exécution de l'image Docker d'OpenDevin, vous devez définir les variables d'environnement suivantes en utilisant `-e` :
Pour utiliser Vertex AI via Google Cloud Platform lors de l'exécution de l'image Docker d'OpenHands, vous devez définir les variables d'environnement suivantes en utilisant `-e` :
OpenDevin émettra de nombreuses invitations au LLM que vous configurez. La plupart de ces LLM coûtent de l'argent -- assurez-vous de définir des limites de dépenses et de surveiller l'utilisation.
OpenHands émettra de nombreuses invitations au LLM que vous configurez. La plupart de ces LLM coûtent de l'argent -- assurez-vous de définir des limites de dépenses et de surveiller l'utilisation.
:::
La variable d'environnement `LLM_MODEL` contrôle le modèle utilisé dans les interactions programmatiques.
Mais en utilisant l'interface utilisateur OpenDevin, vous devrez choisir votre modèle dans la fenêtre des paramètres (la roue dentée en bas à gauche).
Mais en utilisant l'interface utilisateur OpenHands, vous devrez choisir votre modèle dans la fenêtre des paramètres (la roue dentée en bas à gauche).
Les variables d'environnement suivantes peuvent être nécessaires pour certains LLM :
@@ -23,10 +23,10 @@ Les variables d'environnement suivantes peuvent être nécessaires pour certains
-`LLM_EMBEDDING_DEPLOYMENT_NAME`
-`LLM_API_VERSION`
Nous avons quelques guides pour exécuter OpenDevin avec des fournisseurs de modèles spécifiques :
Nous avons quelques guides pour exécuter OpenHands avec des fournisseurs de modèles spécifiques :
- [ollama](llms/localLLMs)
- [Azure](llms/azureLLMs)
- [ollama](llms/local-llms)
- [Azure](llms/azure-llms)
Si vous utilisez un autre fournisseur, nous vous encourageons à ouvrir une PR pour partager votre configuration !
@@ -34,11 +34,11 @@ Si vous utilisez un autre fournisseur, nous vous encourageons à ouvrir une PR p
Les meilleurs modèles sont GPT-4 et Claude 3. Les modèles locaux et open source actuels ne sont pas aussi puissants.
Lors de l'utilisation d'un modèle alternatif, vous pouvez constater des temps d'attente prolongés entre les messages,
des réponses de mauvaise qualité ou des erreurs sur des JSON mal formés. OpenDevin
des réponses de mauvaise qualité ou des erreurs sur des JSON mal formés. OpenHands
ne peut être aussi puissant que les modèles qui le pilotent -- heureusement, les membres de notre équipe travaillent activement à la construction de meilleurs modèles open source !
## Réessais d'API et limites de taux
Certains LLM ont des limites de taux et peuvent nécessiter des réessais. OpenDevin réessaiera automatiquement les demandes s'il reçoit une erreur 429 ou une erreur de connexion API.
Certains LLM ont des limites de taux et peuvent nécessiter des réessais. OpenHands réessaiera automatiquement les demandes s'il reçoit une erreur 429 ou une erreur de connexion API.
Vous pouvez définir les variables d'environnement `LLM_NUM_RETRIES`, `LLM_RETRY_MIN_WAIT`, `LLM_RETRY_MAX_WAIT` pour contrôler le nombre de réessais et le temps entre les réessais.
Par défaut, `LLM_NUM_RETRIES` est 5 et `LLM_RETRY_MIN_WAIT`, `LLM_RETRY_MAX_WAIT` sont respectivement de 3 secondes et 60 secondes.
Par défaut, `LLM_NUM_RETRIES` est 8 et `LLM_RETRY_MIN_WAIT`, `LLM_RETRY_MAX_WAIT` sont respectivement de 15 secondes et 120 secondes.
starcoder2:latest f67ae0f64584 1.7 GB 19 hours ago
```
## Démarrer OpenDevin
## Démarrer OpenHands
### Docker
Utilisez les instructions [ici](../intro) pour démarrer OpenDevin en utilisant Docker.
Utilisez les instructions [ici](../intro) pour démarrer OpenHands en utilisant Docker.
Mais lors de l'exécution de `docker run`, vous devrez ajouter quelques arguments supplémentaires :
```bash
@@ -40,7 +40,7 @@ Mais lors de l'exécution de `docker run`, vous devrez ajouter quelques argument
Par exemple :
```bash
# Le répertoire que vous souhaitez qu'OpenDevin modifie. DOIT être un chemin absolu !
# Le répertoire que vous souhaitez qu'OpenHands modifie. DOIT être un chemin absolu !
exportWORKSPACE_BASE=$(pwd)/workspace
docker run \
@@ -54,14 +54,14 @@ docker run \
-v $WORKSPACE_BASE:/opt/workspace_base \
-v /var/run/docker.sock:/var/run/docker.sock \
-p 3000:3000 \
ghcr.io/opendevin/opendevin:main
ghcr.io/all-hands-ai/openhands:main
```
Vous devriez maintenant pouvoir vous connecter à `http://localhost:3000/`
### Compiler à partir des sources
Utilisez les instructions dans [Development.md](https://github.com/OpenDevin/OpenDevin/blob/main/Development.md) pour compiler OpenDevin.
Utilisez les instructions dans [Development.md](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md) pour compiler OpenHands.
Assurez-vous que `config.toml` soit présent en exécutant `make setup-config` qui en créera un pour vous. Dans `config.toml`, saisissez les éléments suivants :
Remplacez `LLM_MODEL` par celui de votre choix si nécessaire.
Fini ! Vous pouvez maintenant démarrer Devin avec : `make run` sans Docker. Vous devriez maintenant pouvoir vous connecter à `http://localhost:3000/`
Fini ! Vous pouvez maintenant démarrer OpenHands avec : `make run` sans Docker. Vous devriez maintenant pouvoir vous connecter à `http://localhost:3000/`
## Sélection de votre modèle
Dans l'interface OpenDevin, cliquez sur l'icône des paramètres en bas à gauche.
Dans l'interface OpenHands, cliquez sur l'icône des paramètres en bas à gauche.
Ensuite, dans l'entrée `Model`, saisissez `ollama/codellama:7b`, ou le nom du modèle que vous avez téléchargé précédemment.
S'il n'apparaît pas dans un menu déroulant, ce n'est pas grave, tapez-le simplement. Cliquez sur Enregistrer lorsque vous avez terminé.
@@ -87,7 +87,7 @@ Et maintenant, vous êtes prêt à démarrer !
## Configuration du service ollama (WSL){#configuring-ollama-service-fr}
La configuration par défaut pour ollama sous WSL ne sert que localhost. Cela signifie que vous ne pouvez pas l'atteindre depuis un conteneur docker, par exemple, il ne fonctionnera pas avec OpenDevin. Testons d'abord que ollama est en cours d'exécution correctement.
La configuration par défaut pour ollama sous WSL ne sert que localhost. Cela signifie que vous ne pouvez pas l'atteindre depuis un conteneur docker, par exemple, il ne fonctionnera pas avec OpenHands. Testons d'abord que ollama est en cours d'exécution correctement.
```bash
ollama list # obtenir la liste des modèles installés
OpenDevin ne supporte Windows que via [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).
OpenHands ne supporte Windows que via [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).
Veuillez vous assurer de lancer toutes les commandes à l'intérieur de votre terminal WSL.
## Dépannage
@@ -12,8 +12,8 @@ Docker doit également avoir l'option d'intégration WSL activée.
### Recommandation : Ne pas exécuter en tant qu'utilisateur root
Pour des raisons de sécurité, il est fortement recommandé de ne pas exécuter OpenDevin en tant qu'utilisateur root, mais en tant qu'utilisateur avec un UID non nul.
De plus, les sandboxes persistants ne seront pas pris en charge lors de l'exécution en tant que root et un message approprié pourrait apparaître lors du démarrage d'OpenDevin.
Pour des raisons de sécurité, il est fortement recommandé de ne pas exécuter OpenHands en tant qu'utilisateur root, mais en tant qu'utilisateur avec un UID non nul.
De plus, les sandboxes persistants ne seront pas pris en charge lors de l'exécution en tant que root et un message approprié pourrait apparaître lors du démarrage d'OpenHands.
Références :
@@ -21,12 +21,12 @@ Références :
* [Définir l'utilisateur par défaut dans WSL](https://www.tenforums.com/tutorials/128152-set-default-user-windows-subsystem-linux-distro-windows-10-a.html#option2)
Astuce pour la 2e référence : pour les utilisateurs d'Ubuntu, la commande pourrait en fait être "ubuntupreview" au lieu de "ubuntu".
### Échec de la création de l'utilisateur opendevin
### Échec de la création de l'utilisateur openhands
Si vous rencontrez l'erreur suivante lors de l'installation :
```sh
Exception: Failed to create opendevin user in sandbox: 'useradd: UID 0 is not unique'
Exception: Failed to create openhands user in sandbox: 'useradd: UID 0 is not unique'
[](https://star-history.com/#OpenDevin/OpenDevin&Date)
[](https://star-history.com/#All-Hands-AI/OpenHands&Date)
Achieving full replication of production-grade applications with LLMs is a complex endeavor. Our strategy involves:
1.**Core Technical Research:** Focusing on foundational research to understand and improve the technical aspects of code generation and handling.
2.**Specialist Abilities:** Enhancing the effectiveness of core components through data curation, training methods, and more.
3.**Task Planning:** Developing capabilities for bug detection, codebase management, and optimization.
4.**Evaluation:** Establishing comprehensive evaluation metrics to better understand and improve our models.
1.**Core Technical Research:** Focusing on foundational research to understand and improve the technical aspects of code generation and handling
2.**Specialist Abilities:** Enhancing the effectiveness of core components through data curation, training methods, and more
3.**Task Planning:** Developing capabilities for bug detection, codebase management, and optimization
4.**Evaluation:** Establishing comprehensive evaluation metrics to better understand and improve our models
## 🚧 Default Agent
-Our default Agent is currently the CodeActAgent, which is capable of generating code and handling files. We're working on other Agent implementations, including [SWE Agent](https://swe-agent.com/). You can [read about our current set of agents here](./agents).
Our default Agent is currently the [CodeActAgent](agents), which is capable of generating code and handling files.
## 🤝 How to Contribute
OpenDevin is a community-driven project, and we welcome contributions from everyone. Whether you're a developer, a researcher, or simply enthusiastic about advancing the field of software engineering with AI, there are many ways to get involved:
OpenHands is a community-driven project, and we welcome contributions from everyone. Whether you're a developer, a researcher, or simply enthusiastic about advancing the field of software engineering with AI, there are many ways to get involved:
- **Code Contributions:** Help us develop the core functionalities, frontend interface, or sandboxing solutions.
- **Research and Evaluation:** Contribute to our understanding of LLMs in software engineering, participate in evaluating the models, or suggest improvements.
- **Feedback and Testing:** Use the OpenDevin toolset, report bugs, suggest features, or provide feedback on usability.
- **Code Contributions:** Help us develop the core functionalities, frontend interface, or sandboxing solutions
- **Research and Evaluation:** Contribute to our understanding of LLMs in software engineering, participate in evaluating the models, or suggest improvements
- **Feedback and Testing:** Use the OpenHands toolset, report bugs, suggest features, or provide feedback on usability
For details, please check [this document](https://github.com/OpenDevin/OpenDevin/blob/main/CONTRIBUTING.md).
For details, please check [this document](https://github.com/All-Hands-AI/OpenHands/blob/main/CONTRIBUTING.md).
## 🤖 Join Our Community
We have both Slack workspace for the collaboration on building OpenDevin and Discord server for discussion about anything related, e.g., this project, LLM, agent, etc.
We have both Slack workspace for the collaboration on building OpenHands and Discord server for discussion about anything related, e.g., this project, LLM, agent, etc.
If you would love to contribute, feel free to join our community. Let's simplify software engineering together!
🐚 **Code less, make more with OpenDevin.**
🐚 **Code less, make more with OpenHands.**
[](https://star-history.com/#OpenDevin/OpenDevin&Date)
[](https://star-history.com/#All-Hands-AI/OpenHands&Date)
## 🛠️ Built With
OpenDevin is built using a combination of powerful frameworks and libraries, providing a robust foundation for its development. Here are the key technologies used in the project:
OpenHands is built using a combination of powerful frameworks and libraries, providing a robust foundation for its development. Here are the key technologies used in the project:
Please note that the selection of these technologies is in progress, and additional technologies may be added or existing ones may be removed as the project evolves. We strive to adopt the most suitable and efficient tools to enhance the capabilities of OpenDevin.
Please note that the selection of these technologies is in progress, and additional technologies may be added or existing ones may be removed as the project evolves. We strive to adopt the most suitable and efficient tools to enhance the capabilities of OpenHands.
## 📜 License
Distributed under the MIT License. See [our license](https://github.com/OpenDevin/OpenDevin/blob/main/LICENSE) for more information.
Distributed under the MIT License. See [our license](https://github.com/All-Hands-AI/OpenHands/blob/main/LICENSE) for more information.
This agent implements the CodeAct idea ([paper](https://arxiv.org/abs/2402.01030), [tweet](https://twitter.com/xingyaow_/status/1754556835703751087)) that consolidates LLM agents’**act**ions into a unified **code** action space for both _simplicity_ and _performance_ (see paper for more details).
This agent implements the CodeAct idea ([paper](https://arxiv.org/abs/2402.01030), [tweet](https://twitter.com/xingyaow_/status/1754556835703751087)) that consolidates LLM agents’ **act**ions into a
unified **code** action space for both _simplicity_ and _performance_.
The conceptual idea is illustrated below. At each turn, the agent can:
@@ -18,76 +15,10 @@ The conceptual idea is illustrated below. At each turn, the agent can:
- Execute any valid Linux `bash` command
- Execute any valid `Python` code with [an interactive Python interpreter](https://ipython.org/). This is simulated through `bash` command, see plugin system below for more details.
To make the CodeAct agent more powerful with only access to `bash` action space, CodeAct agent leverages OpenDevin's plugin system:
- [Jupyter plugin](https://github.com/OpenDevin/OpenDevin/tree/main/opendevin/runtime/plugins/jupyter): for IPython execution via bash command
- [SWE-agent tool plugin](https://github.com/OpenDevin/OpenDevin/tree/main/opendevin/runtime/plugins/swe_agent_commands): Powerful bash command line tools for software development tasks introduced by [swe-agent](https://github.com/princeton-nlp/swe-agent).
| `__init__` | Initializes an agent with `llm` and a list of messages `list[Mapping[str, str]]` |
| `step` | Performs one step using the CodeAct Agent. This includes gathering info on previous steps and prompting the model to make a command to execute. |
## Planner Agent
### Description
The planner agent utilizes a special prompting strategy to create long term plans for solving problems.
The agent is given its previous action-observation pairs, current task, and hint based on last action taken at every step.
| `step` | Checks to see if current step is completed, returns `AgentFinishAction` if True. Otherwise, creates a plan prompt and sends to model for inference, adding the result as the next action. |
_Example of CodeActAgent with `gpt-4-turbo-2024-04-09` performing a data science task (linear regression)_.
<img src="https://github.com/OpenDevin/OpenDevin/assets/16201837/97d747e3-29d8-4ccb-8d34-6ad1adb17f38" alt="OpenDevin System Architecture Diagram Jul 4 2024" />
<p><em>OpenDevin System Architecture Diagram (July 4, 2024)</em></p>
<img src="https://github.com/All-Hands-AI/OpenHands/assets/16201837/97d747e3-29d8-4ccb-8d34-6ad1adb17f38" alt="OpenHands System Architecture Diagram Jul 4 2024" />
<p><em>OpenHands System Architecture Diagram (July 4, 2024)</em></p>
</div>
This is a high-level overview of the system architecture. The system is divided into two main components: the frontend and the backend. The frontend is responsible for handling user interactions and displaying the results. The backend is responsible for handling the business logic and executing the agents.
@@ -33,14 +29,14 @@ _**Disclaimer**: The backend architecture is a work in progress and is subject t
## Prerequisites
- Running python environment in which opendevin is executable
- Running python environment in which openhands is executable
(according to the instructions in the README.md file in the root of the repository)
The OpenHands EventStream Runtime is the core component that enables secure and flexible execution of AI agent's action.
It creates a sandboxed environment using Docker, where arbitrary code can be run safely without risking the host system.
## Why do we need a sandboxed runtime?
OpenHands needs to execute arbitrary code in a secure, isolated environment for several reasons:
1. Security: Executing untrusted code can pose significant risks to the host system. A sandboxed environment prevents malicious code from accessing or modifying the host system's resources
2. Consistency: A sandboxed environment ensures that code execution is consistent across different machines and setups, eliminating "it works on my machine" issues
3. Resource Control: Sandboxing allows for better control over resource allocation and usage, preventing runaway processes from affecting the host system
4. Isolation: Different projects or users can work in isolated environments without interfering with each other or the host system
5. Reproducibility: Sandboxed environments make it easier to reproduce bugs and issues, as the execution environment is consistent and controllable
## How does the Runtime work?
The OpenHands Runtime system uses a client-server architecture implemented with Docker containers. Here's an overview of how it works:
1. User Input: The user provides a custom base Docker image
2. Image Building: OpenHands builds a new Docker image (the "OH runtime image") based on the user-provided image. This new image includes OpenHands-specific code, primarily the "runtime client"
3. Container Launch: When OpenHands starts, it launches a Docker container using the OH runtime image
4. Action Execution Server Initialization: The action execution server initializes an `ActionExecutor` inside the container, setting up necessary components like a bash shell and loading any specified plugins
5. Communication: The OpenHands backend (`openhands/runtime/impl/eventstream/eventstream_runtime.py`) communicates with the action execution server over RESTful API, sending actions and receiving observations
6. Action Execution: The runtime client receives actions from the backend, executes them in the sandboxed environment, and sends back observations
7. Observation Return: The action execution server sends execution results back to the OpenHands backend as observations
The role of the client:
- It acts as an intermediary between the OpenHands backend and the sandboxed environment
- It executes various types of actions (shell commands, file operations, Python code, etc.) safely within the container
- It manages the state of the sandboxed environment, including the current working directory and loaded plugins
- It formats and returns observations to the backend, ensuring a consistent interface for processing results
## How OpenHands builds and maintains OH Runtime images
OpenHands' approach to building and managing runtime images ensures efficiency, consistency, and flexibility in creating and maintaining Docker images for both production and development environments.
Check out the [relevant code](https://github.com/All-Hands-AI/OpenHands/blob/main/openhands/runtime/utils/runtime_build.py) if you are interested in more details.
### Image Tagging System
OpenHands uses a dual-tagging system for its runtime images to balance reproducibility with flexibility.
This hash is built from the first 16 digits of the MD5 of:
- The name of the base image upon which the image was built (e.g.: `nikolaik/python-nodejs:python3.12-nodejs22`)
- The content of the `pyproject.toml` included in the image.
- The content of the `poetry.lock` included in the image.
This effectively gives a hash for the dependencies of Openhands independent of the source code.
#### Source Hash
This is the first 16 digits of the MD5 of the directory hash for the source directory. This gives a hash
for only the openhands source
#### Build Process
When generating an image...
- OpenHands first checks whether an image with the same **Specific** tag exists. If there is such an image,
no build is performed - the existing image is used.
- OpenHands next checks whether an image with the **Generic** tag exists. If there is such an image,
OpenHands builds a new image based upon it, bypassing all installation steps (like `poetry install` and
`apt-get`) except a final operation to copy the current source code. The new image is tagged with a
**Specific** tag only.
- If neither a **Specific** nor **Generic** tag exists, a brand new image is built based upon the base
image (Which is a slower operation). This new image is tagged with both the **Generic** and **Specific**
tags.
This dual-tagging approach allows OpenHands to efficiently manage both development and production environments.
1. Identical source code and Dockerfile always produce the same image (via hash-based tags)
2. The system can quickly rebuild images when minor changes occur (by leveraging recent compatible images)
3. The generic tag (e.g., `runtime:oh_v0.9.3_1234567890abcdef`) always points to the latest build for a particular base image and OpenHands version combination
## Runtime Plugin System
The OpenHands Runtime supports a plugin system that allows for extending functionality and customizing the runtime environment. Plugins are initialized when the runtime client starts up.
Check [an example of Jupyter plugin here](https://github.com/All-Hands-AI/OpenHands/blob/ecf4aed28b0cf7c18d4d8ff554883ba182fc6bdd/openhands/runtime/plugins/jupyter/__init__.py#L21-L55) if you want to implement your own plugin.
*More details about the Plugin system are still under construction - contributions are welcomed!*
Key aspects of the plugin system:
1. Plugin Definition: Plugins are defined as Python classes that inherit from a base `Plugin` class
2. Plugin Registration: Available plugins are registered in an `ALL_PLUGINS` dictionary
3. Plugin Specification: Plugins are associated with `Agent.sandbox_plugins: list[PluginRequirement]`. Users can specify which plugins to load when initializing the runtime
4. Initialization: Plugins are initialized asynchronously when the runtime client starts
5. Usage: The runtime client can use initialized plugins to extend its capabilities (e.g., the JupyterPlugin for running IPython cells)
The default OpenDevin sandbox comes with a [minimal ubuntu configuration](https://github.com/OpenDevin/OpenDevin/blob/main/containers/sandbox/Dockerfile).
Your use case may need additional software installed by default.
There are two ways you can do so:
1. Use an existing image from docker hub. For instance, if you want to have `nodejs` installed, you can do so by using the `node:20` image
2. Creating your own custom docker image and using it
If you want to take the first approach, you can skip the `Create Your Docker Image` section.
For a more feature-rich environment, you might consider using pre-built images like **[nikolaik/python-nodejs](https://hub.docker.com/r/nikolaik/python-nodejs)**, which comes with both Python and Node.js pre-installed, along with many other useful tools and libraries, like:
- Node.js: 22.x
- npm: 10.x
- yarn: stable
- Python: latest
- pip: latest
- pipenv: latest
- poetry: latest
- uv: latest
## Setup
Make sure you are able to run OpenDevin using the [Development.md](https://github.com/OpenDevin/OpenDevin/blob/main/Development.md) first.
## Create Your Docker Image
To create a custom docker image, it must be debian/ubuntu based.
For example, if we want OpenDevin to have access to the `node` binary, we would use the following Dockerfile:
```dockerfile
# Start with latest ubuntu image
FROMubuntu:latest
# Run needed updates
RUN apt-get update && apt-get install -y
# Install node
RUN apt-get install -y nodejs
```
Next build your docker image with the name of your choice, for example `custom_image`.
To do this you can create a directory and put your file inside it with the name `Dockerfile`, and inside the directory run the following command:
```bash
docker build -t custom_image .
```
This will produce a new image called ```custom_image``` that will be available in Docker Engine.
> Note that in the configuration described in this document, OpenDevin will run as user "opendevin" inside the sandbox and thus all packages installed via the docker file should be available to all users on the system, not just root.
>
> Installing with apt-get above installs node for all users.
## Specify your sandbox image in config.toml file
OpenDevin configuration occurs via the top-level `config.toml` file.
Create a `config.toml` file in the OpenDevin directory and enter these contents:
```toml
[core]
workspace_base="./workspace"
persist_sandbox=false
run_as_devin=true
sandbox_container_image="custom_image"
```
For `sandbox_container_image`, you can specify either:
1. The name of your custom image that you built in the previous step (e.g., `”custom_image”`)
2. A pre-existing image from Docker Hub (e.g., `”node:20”` if you want a sandbox with Node.js pre-installed)
## Run
Run OpenDevin by running ```make run``` in the top level directory.
Navigate to ```localhost:3001``` and check if your desired dependencies are available.
In the case of the example above, running ```node -v``` in the terminal produces ```v20.15.0```
Congratulations!
## Technical Explanation
The relevant code is defined in [ssh_box.py](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/ssh_box.py) and [image_agnostic_util.py](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/image_agnostic_util.py).
In particular, `ssh_box.py` checks the config object for ```config.sandbox_container_image``` and then attempts to retrieve the image using [get_od_sandbox_image](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/image_agnostic_util.py#L72) which is defined in image_agnostic_util.py.
When first using a custom image, it will not be found and thus it will be built (on subsequent runs the built image will be found and returned).
The custom image is built using [_build_sandbox_image()](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/image_agnostic_util.py#L29), which creates a docker file using your custom_image as a base and then configures the environment for OpenDevin, like this:
> Note: the name of the image is modified via [_get_new_image_name()](https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/runtime/docker/image_agnostic_util.py#L63) and it is the modified name that is searched for on subsequent runs.
## Troubleshooting / Errors
### Error: ```useradd: UID 1000 is not unique```
If you see this error in the console output it is because OpenDevin is trying to create the opendevin user in the sandbox with a UID of 1000, however this UID is already being used in the image (for some reason). To fix this change the sandbox_user_id field in the config.toml file to a different value:
```toml
[core]
workspace_base="./workspace"
persist_sandbox=false
run_as_devin=true
sandbox_container_image="custom_image"
sandbox_user_id="1001"
```
### Port use errors
If you see an error about a port being in use or unavailable, try deleting all running Docker Containers (run `docker ps` and `docker rm` relevant containers) and then re-running ```make run``` .
## Discuss
For other issues or questions join the [Slack](https://join.slack.com/t/opendevin/shared_invite/zt-2jsrl32uf-fTeeFjNyNYxqSZt5NPY3fA) or [Discord](https://discord.gg/ESHStjSjD4) and ask!
When using OpenDevin, you will undoubtably encounter cases where things work well, and others where they don't. We encourage you to provide feedback when you use OpenDevin to help give feedback to the development team, and perhaps more importantly, create an open corpus of coding agent training examples -- Share-OpenDevin!
When using OpenHands, you will encounter cases where things work well, and others where they don't. We encourage you to provide feedback when you use OpenHands to help give feedback to the development team, and perhaps more importantly, create an open corpus of coding agent training examples -- Share-OpenHands!
## 📝 How to Provide Feedback
Providing feedback is easy! When you are using OpenDevin, you can press the thumbs-up or thumbs-down button at any point during your interaction with. You will be prompted to provide your email address (e.g. so we can contact you if we want to ask any follow-up questions), and you can choose whether you want to provide feedback publicly or privately.
Providing feedback is easy! When you are using OpenHands, you can press the thumbs-up or thumbs-down button at any point during your interaction. You will be prompted to provide your email address (e.g. so we can contact you if we want to ask any follow-up questions), and you can choose whether you want to provide feedback publicly or privately.
@@ -18,12 +14,12 @@ Providing feedback is easy! When you are using OpenDevin, you can press the thum
When you submit data, you can submit it either publicly or privately.
* **Public** data will be distributed under the MIT License, like OpenDevin itself, and can be used by the community to train and test models. Obviously, feedback that you can make public will be more valuable for the community as a whole, so when you are not dealing with sensitive information, we would encourage you to choose this option!
* **Private** data will only be shared with the OpenDevin team for the purpose of improving OpenDevin.
* **Public** data will be distributed under the MIT License, like OpenHands itself, and can be used by the community to train and test models. Obviously, feedback that you can make public will be more valuable for the community as a whole, so when you are not dealing with sensitive information, we would encourage you to choose this option!
* **Private** data will only be shared with the OpenHands team for the purpose of improving OpenHands.
### Who collects and stores the data?
The data is collected and stored by [All Hands AI](https://all-hands.dev), a company founded by OpenDevin maintainers to support and improve OpenDevin.
The data is collected and stored by [All Hands AI](https://all-hands.dev), a company founded by OpenHands maintainers to support and improve OpenHands.
### How will public data be released?
@@ -31,7 +27,7 @@ The public data will be released when we hit fixed milestones, such as 1,000 pub
At this time, we will follow the following release process:
1. All people who contributed public feedback will receive an email describing the data release and being given an opportunity to opt out.
2. The person or people in charge of the data release will perform quality control of the data, removing low-quality feedback, removing email submitter email addresses, and attempting to remove any sensitive information such as API keys.
2. The person or people in charge of the data release will perform quality control of the data, removing low-quality feedback, removing email submitter email addresses, and attempting to remove any sensitive information.
3. The data will be released publicly under the MIT license through commonly used sites such as github or Hugging Face.
The sandbox is where the agent performs its tasks. Instead of running commands directly on your computer
(which could be risky), the agent runs them inside a Docker container.
The default OpenHands sandbox (`python-nodejs:python3.12-nodejs22`
from [nikolaik/python-nodejs](https://hub.docker.com/r/nikolaik/python-nodejs)) comes with some packages installed such
as python and Node.js but may need other software installed by default.
You have two options for customization:
1. Use an existing image with the required software.
2. Create your own custom Docker image.
If you choose the first option, you can skip the `Create Your Docker Image` section.
## Create Your Docker Image
To create a custom Docker image, it must be Debian based.
For example, if you want OpenHands to have `ruby` installed, create a `Dockerfile` with the following content:
```dockerfile
FROMdebian:latest
# Install required packages
RUN apt-get update && apt-get install -y ruby
```
Save this file in a folder. Then, build your Docker image (e.g., named custom-image) by navigating to the folder in
the terminal and running::
```bash
docker build -t custom-image .
```
This will produce a new image called `custom-image`, which will be available in Docker.
> Note that in the configuration described in this document, OpenHands will run as user "openhands" inside the
> sandbox and thus all packages installed via the docker file should be available to all users on the system, not just root.
## Using the Development Workflow
### Setup
First, ensure you can run OpenHands by following the instructions in [Development.md](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md).
### Specify the Base Sandbox Image
In the `config.toml` file within the OpenHands directory, set the `sandbox_base_container_image` to the image you want to use.
This can be an image you’ve already pulled or one you’ve built:
```bash
[core]
...
sandbox_base_container_image="custom-image"
```
### Run
Run OpenHands by running ```make run``` in the top level directory.
## Technical Explanation
Please refer to [custom docker image section of the runtime documentation](https://docs.all-hands.dev/modules/usage/architecture/runtime#advanced-how-openhands-builds-and-maintains-od-runtime-images) for more details.
## Troubleshooting / Errors
### Error: ```useradd: UID 1000 is not unique```
If you see this error in the console output it is because OpenHands is trying to create the openhands user in the sandbox with a UID of 1000, however this UID is already being used in the image (for some reason). To fix this change the sandbox_user_id field in the config.toml file to a different value:
```toml
[core]
workspace_base="./workspace"
run_as_openhands=true
sandbox_base_container_image="custom_image"
sandbox_user_id="1001"
```
### Port use errors
If you see an error about a port being in use or unavailable, try deleting all running Docker Containers (run `docker ps` and `docker rm` relevant containers) and then re-running ```make run``` .
## Discuss
For other issues or questions join the [Slack](https://join.slack.com/t/opendevin/shared_invite/zt-2oikve2hu-UDxHeo8nsE69y6T7yFX_BA) or [Discord](https://discord.gg/ESHStjSjD4) and ask!
The following is intended as a primer on debugging OpenHands for Development purposes.
## Server / VSCode
The following `launch.json` will allow debugging the agent, controller and server elements, but not the sandbox (Which runs inside docker). It will ignore any changes inside the `workspace/` directory:
```
{
"version": "0.2.0",
"configurations": [
{
"name": "OpenHands CLI",
"type": "debugpy",
"request": "launch",
"module": "openhands.core.cli",
"justMyCode": false
},
{
"name": "OpenHands WebApp",
"type": "debugpy",
"request": "launch",
"module": "uvicorn",
"args": [
"openhands.server.listen:app",
"--reload",
"--reload-exclude",
"${workspaceFolder}/workspace",
"--port",
"3000"
],
"justMyCode": false
}
]
}
```
More specific debugging configurations which include more parameters may be specified:
```
...
{
"name": "Debug CodeAct",
"type": "debugpy",
"request": "launch",
"module": "openhands.core.main",
"args": [
"-t",
"Ask me what your task is.",
"-d",
"${workspaceFolder}/workspace",
"-c",
"CodeActAgent",
"-l",
"llm.o1",
"-n",
"prompts"
],
"justMyCode": false
}
...
```
Values in the snippet above can be updated such that:
* *t*: the task
* *d*: the openhands workspace directory
* *c*: the agent
* *l*: the LLM config (pre-defined in config.toml)
This guide provides an overview of how to integrate your own evaluation benchmark into the OpenHands framework.
## Setup Environment and LLM Configuration
Please follow instructions [here](https://github.com/All-Hands-AI/OpenHands/blob/main/Development.md) to setup your local development environment.
OpenHands in development mode uses `config.toml` to keep track of most configurations.
Here's an example configuration file you can use to define and use multiple LLMs:
```toml
[llm]
# IMPORTANT: add your API key here, and set the model to the one you want to evaluate
model="claude-3-5-sonnet-20240620"
api_key="sk-XXX"
[llm.eval_gpt4_1106_preview_llm]
model="gpt-4-1106-preview"
api_key="XXX"
temperature=0.0
[llm.eval_some_openai_compatible_model_llm]
model="openai/MODEL_NAME"
base_url="https://OPENAI_COMPATIBLE_URL/v1"
api_key="XXX"
temperature=0.0
```
## How to use OpenHands in the command line
OpenHands can be run from the command line using the following format:
```bash
poetry run python ./openhands/core/main.py \
-i <max_iterations> \
-t "<task_description>"\
-c <agent_class> \
-l <llm_config>
```
For example:
```bash
poetry run python ./openhands/core/main.py \
-i 10\
-t "Write me a bash script that prints hello world."\
-c CodeActAgent \
-l llm
```
This command runs OpenHands with:
- A maximum of 10 iterations
- The specified task description
- Using the CodeActAgent
- With the LLM configuration defined in the `llm` section of your `config.toml` file
## How does OpenHands work
The main entry point for OpenHands is in `openhands/core/main.py`. Here's a simplified flow of how it works:
1. Parse command-line arguments and load the configuration
2. Create a runtime environment using `create_runtime()`
3. Initialize the specified agent
4. Run the controller using `run_controller()`, which:
- Attaches the runtime to the agent
- Executes the agent's task
- Returns a final state when complete
The `run_controller()` function is the core of OpenHands's execution. It manages the interaction between the agent, the runtime, and the task, handling things like user input simulation and event processing.
## Easiest way to get started: Exploring Existing Benchmarks
We encourage you to review the various evaluation benchmarks available in the [`evaluation/` directory](https://github.com/All-Hands-AI/OpenHands/blob/main/evaluation) of our repository.
To integrate your own benchmark, we suggest starting with the one that most closely resembles your needs. This approach can significantly streamline your integration process, allowing you to build upon existing structures and adapt them to your specific requirements.
## How to create an evaluation workflow
To create an evaluation workflow for your benchmark, follow these steps:
1. Import relevant OpenHands utilities:
```python
import openhands.agenthub
from evaluation.utils.shared import (
EvalMetadata,
EvalOutput,
make_metadata,
prepare_dataset,
reset_logger_for_multiprocessing,
run_evaluation,
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
SandboxConfig,
get_llm_config_arg,
parse_arguments,
)
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime, run_controller
from openhands.events.action import CmdRunAction
from openhands.events.observation import CmdOutputObservation, ErrorObservation
This workflow sets up the configuration, initializes the runtime environment, processes each instance by running the agent and evaluating its actions, and then collects the results into an `EvalOutput` object. The `run_evaluation` function handles parallelization and progress tracking.
Remember to customize the `get_instruction`, `your_user_response_function`, and `evaluate_agent_actions` functions according to your specific benchmark requirements.
By following this structure, you can create a robust evaluation workflow for your benchmark within the OpenHands framework.
## Understanding the `user_response_fn`
The `user_response_fn` is a crucial component in OpenHands's evaluation workflow. It simulates user interaction with the agent, allowing for automated responses during the evaluation process. This function is particularly useful when you want to provide consistent, predefined responses to the agent's queries or actions.
### Workflow and Interaction
The correct workflow for handling actions and the `user_response_fn` is as follows:
1. Agent receives a task and starts processing
2. Agent emits an Action
3. If the Action is executable (e.g., CmdRunAction, IPythonRunCellAction):
- The Runtime processes the Action
- Runtime returns an Observation
4. If the Action is not executable (typically a MessageAction):
- The `user_response_fn` is called
- It returns a simulated user response
5. The agent receives either the Observation or the simulated response
6. Steps 2-5 repeat until the task is completed or max iterations are reached
Here's a more accurate visual representation:
```
[Agent]
|
v
[Emit Action]
|
v
[Is Action Executable?]
/ \
Yes No
| |
v v
[Runtime] [user_response_fn]
| |
v v
[Return Observation] [Simulated Response]
\ /
\ /
v v
[Agent receives feedback]
|
v
[Continue or Complete Task]
```
In this workflow:
- Executable actions (like running commands or executing code) are handled directly by the Runtime
- Non-executable actions (typically when the agent wants to communicate or ask for clarification) are handled by the `user_response_fn`
- The agent then processes the feedback, whether it's an Observation from the Runtime or a simulated response from the `user_response_fn`
This approach allows for automated handling of both concrete actions and simulated user interactions, making it suitable for evaluation scenarios where you want to test the agent's ability to complete tasks with minimal human intervention.
### Example Implementation
Here's an example of a `user_response_fn` used in the SWE-Bench evaluation:
```python
def codeact_user_response(state: State | None) -> str:
msg = (
'Please continue working on the task on whatever approach you think is suitable.\n'
'If you think you have solved the task, please first send your answer to user through message and then <execute_bash> exit </execute_bash>.\n'
'IMPORTANT: YOU SHOULD NEVER ASK FOR HUMAN HELP.\n'
)
if state and state.history:
# check if the agent has tried to talk to the user 3 times, if so, let the agent know it can give up
user_msgs = [
event
for event in state.history.get_events()
if isinstance(event, MessageAction) and event.source == 'user'
]
if len(user_msgs) >= 2:
# let the agent know that it can give up when it has tried 3 times
return (
msg
+ 'If you want to give up, run: <execute_bash> exit </execute_bash>.\n'
)
return msg
```
This function does the following:
1. Provides a standard message encouraging the agent to continue working
2. Checks how many times the agent has attempted to communicate with the user
3. If the agent has made multiple attempts, it provides an option to give up
By using this function, you can ensure consistent behavior across multiple evaluation runs and prevent the agent from getting stuck waiting for human input.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.