Commit Graph

59 Commits

Author SHA1 Message Date
Xingyao Wang
f80ecec772 [Arch] Add tests for EventStreamRuntime and fix bash parsing (#2933)
* deprecating recall action

* fix integration tests

* fix integration tests

* refractor runtime to use async

* remove search memory

* rename .initialize to .ainit

* draft of runtime image building (separate from img agnostic)

* refractor runtime build into separate file and add unit tests for it

* fix image agnostic tests

* move `split_bash_commands` into a separate util file

* fix bash pexcept parsing for env

* refractor add_env_var from sandbox to runtime;
add test runtime for env var, remove it from sandbox;

* remove unclear comment

* capture broader error

* make `add_env_var` handle multiple export at the same time

* add multi env var test

* fix tests with new config

* make runtime tests a separate ci to avoid full disk

* Update Runtime README with architecture diagram and detailed explanations

* update test

* remove dependency of global config in sandbox test

* fix sandbox typo

* runtime tests does not need ghcr build now

* remove download runtime img

* remove dependency of global config in sandbox test

* fix sandbox typo

* try to free disk before running the tests

* Update opendevin/runtime/client/README.md

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/client/README.md

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/client/README.md

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* try to reduce code duplication

* Update opendevin/runtime/client/README.md

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/client/README.md

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/client/README.md

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/client/README.md

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/client/README.md

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* cleanup before setup

* temporarily remove this enable lint test since env var are now handled by runtime

* linter

---------

Co-authored-by: OpenDevin <opendevin@all-hands.dev>
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
2024-07-18 06:10:45 +08:00
Graham Neubig
257698e89b Remove global config from sandbox (#2961)
* Some changes

* Fixed errors

* Remove duplicate initialize_plugins

* Fix some tests

* Fix tests
2024-07-16 18:34:04 +00:00
மனோஜ்குமார் பழனிச்சாமி
b2b6d2ac1e Fix: hostname in logging (#2914)
Co-authored-by: tobitege <tobitege@gmx.de>
2024-07-13 10:04:08 +00:00
Xingyao Wang
f2e92b2db7 move image agnostic util to shared runtime util (#2859) 2024-07-08 22:17:01 +00:00
மனோஜ்குமார் பழனிச்சாமி
9dc2d2c80f Refactor: Remove extra log (#2687) 2024-07-08 05:37:13 +09:00
Shimada666
e35c1ff74a Display real-time build logs for the agnostic image (#2830)
* Display real-time build logs for the agnostic image and improve wget's output.

* remove unused code
2024-07-08 04:35:16 +08:00
Shimada666
82f256be96 trim the sandbox image and install plugin dependencies in agnostic image (#2792)
* trim the sandbox image

* remove wrong code

* readd python

* readd python

* fix script

* readd nano
2024-07-06 17:38:37 +02:00
Shimada666
d22ff73905 Make the sandbox Python runtime completely transparent (#2796)
* Make the sandbox Python runtime completely independent

* fix source bashrc

* add pip install instruction for ipython to fix intergration tests for codeact swe

* update integration tests

* change flake8 command to (maybe) fix sandbox tests?

* make lint support both unittest & sandbox

* fix agnostic image build error

* refactor build script

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: tobitege <tobitege@gmx.de>
2024-07-06 15:22:44 +02:00
Xingyao Wang
a47713ecb0 [Arch] Remove supports for Background Commands (#2803)
* depracting docker exec box

* remove doc exec from workflow and docs

* remove background commands

* Update tests/unit/test_sandbox.py

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>

* replace for-loop with assignment

* fix integration tests

* fix integration tests for shell script

* fix integration tests

* increase max iter to fix some monologue agent issue

* fix integration test again

* fix integration tests (seems related to run_user issue)

---------

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-07-06 03:38:05 +08:00
மனோஜ்குமார் பழனிச்சாமி
143f38d25a Refactored sandbox config and added fast boot (#2455)
* Refactored sandbox config and added fastboot

* added tests

* fixed tests

* fixed tests

* intimate user about breaking change

* remove default config from eval

* check for lowercase env

* add test

* Revert Migration

* migrate old sandbox configs

* resolve merge conflict

* revert migration 2

* Revert "remove default config from eval"

This reverts commit de57c588db.

* change type to box_type

* fix var name

* linted

* lint

* lint comments

* fix tests

* fix tests

* fix typo

* fix box_type, remove fast_boot

* add tests for sandbox config

* fix test

* update eval docs

* small removal comments

* adapt toml template

* old fields shouldn't be in the app dataclass

* fix old keys in app config

* clean up exec box

---------

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-07-05 03:30:21 +00:00
Xingyao Wang
0d3b3ffbf8 [Arch] Removing docker exec box (#2802)
* depracting docker exec box

* remove doc exec from workflow and docs
2024-07-04 23:15:25 +00:00
tobitege
6246cb8e67 fix: improve exit_code processing (ssh_box) (#2726)
* improve exit_code processing

* removed debug spam
2024-07-01 19:59:31 +00:00
Xavier Vergés
cd91d45b44 Allow SANDBOX_CONTAINER_IMAGEs built from opendevin/sandbox:main (#2622) 2024-06-26 12:05:07 +08:00
Shimada666
5972498e23 No longer chown -R the miniforge3 folder (#2566)
* No longer chown -R the miniforge3 folder

* change miniforge3 group permission
2024-06-21 17:33:00 +00:00
Shimada666
64c2a783d4 Revert "Always pull sandbox image (#2538)" (#2560)
This reverts commit 6dd2491944.
2024-06-21 08:09:26 +00:00
Shimada666
1ffaed48c4 Stop always pulling the latest image. (#2558)
LGTM, thanks!
2024-06-21 05:42:49 +00:00
மனோஜ்குமார் பழனிச்சாமி
6dd2491944 Always pull sandbox image (#2538) 2024-06-21 06:52:18 +05:30
Xingyao Wang
eac05d71fa replace / for tag (#2550) 2024-06-20 11:09:29 -06:00
Shimada666
26fc3c886a Make plugins sandbox-agnostic (#2101)
* tmp

* tmp

* merge main

* feat: auto build image cache

* remove plugins

* use config file

* update mamba setup shell

* support agnostic sandbox image autobuild

* remove config

* Update .gitignore

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>

* Update opendevin/runtime/docker/ssh_box.py

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>

* update setup.sh

* readd sudo

* add sudo in dockerfile

* remove export

* move od-runtime dependencies to sandbox dockerfile

* factor out re-build logic into a separate util file

* tweak existing plugin to use OD specific sandbox

* update testcase

* attempt to fix unit test using image built in ghcr

* use cache tag

* try to fix unit tests

* add unittest

* add unittest

* add some unittests

* revert gh workflow changes

* feat: optimize sandbox image naming rule

* add pull latest image hint

* add opendevin python hint and use mamba to install gcc

* update docker image naming rule and fix mamba issue

* Update opendevin/runtime/docker/ssh_box.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* fix: opendevin user use correct pip

* fix lint issue

* fix custom sandbox base image

* rename test name

* add skipif

---------

Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
Co-authored-by: tobitege <tobitege@gmx.de>
2024-06-19 19:58:07 -07:00
மனோஜ்குமார் பழனிச்சாமி
f4c917345f Reworded port forward msg (#2478)
* Reworded port forward msg

* applied suggestions.

---------

Co-authored-by: tobitege <tobitege@gmx.de>
2024-06-17 11:40:37 +00:00
Yufan Song
85c0eae31d remove (#2465) 2024-06-17 00:45:48 +08:00
மனோஜ்குமார் பழனிச்சாமி
ac6ef8e59d stopped container to prevent port issues. (#2447) 2024-06-15 00:31:33 +00:00
Leo
759f76fab5 Fix: Properly close Docker client in DockerExecBox to prevent resource leakage (#2224) 2024-06-04 09:05:41 +08:00
Boxuan Li
1adbec6757 ssh_box: Fix Docker descriptor leak (#2212) 2024-06-03 01:22:30 +00:00
Boxuan Li
399e6fb1d1 ssh_box: Close containers before throwing exception (#2206) 2024-06-02 20:13:44 +00:00
மனோஜ்குமார் பழனிச்சாமி
4ece6fb3cc Auto started persistent container (#2151) 2024-06-01 14:46:41 +00:00
மனோஜ்குமார் பழனிச்சாமி
f9c7c3a520 Refactored logging (#2159) 2024-06-01 14:31:35 +00:00
மனோஜ்குமார் பழனிச்சாமி
aee3d506e6 Restricted persistent sandbox to opendevin user only (#2177) 2024-06-01 14:18:03 +00:00
மனோஜ்குமார் பழனிச்சாமி
04d7354501 Detailed logs for ssh_box (#2173) 2024-06-01 11:40:22 +05:30
Boxuan Li
06e45afc75 Fix ssh box hung issue (#2172)
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
2024-06-01 05:31:32 +00:00
மனோஜ்குமார் பழனிச்சாமி
3a4dc5c68c Initialized plugins only once for persistent sandboxes (#2162) 2024-06-01 10:46:09 +05:30
மனோஜ்குமார் பழனிச்சாமி
961c96a2a1 Added ssh_password to config setup (#2139)
Co-authored-by: Aleksandar <isavitaisa@gmail.com>
2024-05-31 07:26:16 +05:30
மனோஜ்குமார் பழனிச்சாமி
d4ccd48af8 Persistent docker session (#1998)
Co-authored-by: Robert Brennan <accounts@rbren.io>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Graham Neubig <neubig@gmail.com>
2024-05-29 13:22:34 +00:00
Xingyao Wang
a6b3ce866d refractor ssh login into one function (#2066) 2024-05-26 08:56:13 +00:00
Shimada666
be1c2ad60d feat: use retry decorator instead of retrying in a loop (#2058)
* feat: use retry decorator instead of retrying in a loop

* update code logic

* update poetry lock
2024-05-25 16:04:40 +00:00
Xingyao Wang
221035d39a Add retry logic to ssh login (#2053)
* add retry logic to ssh login

* Update opendevin/runtime/docker/ssh_box.py

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>

---------

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-05-25 12:16:24 +00:00
மனோஜ்குமார் பழனிச்சாமி
cfae6821fa refactored timeout (#2044) 2024-05-24 18:19:14 +02:00
Xingyao Wang
cbf4c4b4c4 fix ExceptionPxssh (#2023)
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-05-23 21:24:21 -07:00
Xingyao Wang
602ffcdffb Implement agentskills for OpenDevin to helpfully improve edit AND including more useful tools/skills (#1941)
* add draft for skills

* Implement and test agentskills functions: open_file, goto_line, scroll_down, scroll_up, create_file, search_dir, search_file, find_file

* Remove new_sample.txt file

* add some work from opendevin w/ fixes

* Add unit tests for agentskills module

* fix some issues and updated tests

* add more tests for open

* tweak and handle goto_line

* add tests for some edge cases

* add tests for scrolling

* add tests for edit

* add tests for search_dir

* update tests to use pytest

* use pytest --forked to avoid file op unit tests to interfere with each other via global var

* update doc based on swe agent tool

* update and add tests for find_file and search_file

* move agent_skills to plugins

* add agentskills as plugin and docs

* add agentskill to ssh box and fix sandbox integration

* remove extra returns in doc

* add agentskills to initial tool for jupyter

* support re-init jupyter kernel (for agentskills) after restart

* fix print window's issue with indentation and add testcases

* add prompt for codeact with the newest edit primitives

* modify the way line number is presented (remove leading space)

* change prompt to the newest display format

* support tracking of costs via metrics

* Update opendevin/runtime/plugins/agent_skills/README.md

* Update opendevin/runtime/plugins/agent_skills/README.md

* implement and add tests for py linting

* remove extra text arg for incompatible subprocess ver

* remove sample.txt

* update test_edits integration tests

* fix all integration

* Update opendevin/runtime/plugins/agent_skills/README.md

* Update opendevin/runtime/plugins/agent_skills/README.md

* Update opendevin/runtime/plugins/agent_skills/README.md

* Update agenthub/codeact_agent/prompt.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Update agenthub/codeact_agent/prompt.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Update agenthub/codeact_agent/prompt.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Update opendevin/runtime/plugins/agent_skills/agentskills.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* correctly setup plugins for swebench eval

* bump swe-bench version and add logging

* correctly setup plugins for swebench eval

* bump swe-bench version and add logging

* Revert "correctly setup plugins for swebench eval"

This reverts commit 2bd1055673.

* bump version

* remove _AGENT_SKILLS_DOCS

* move flake8 to test dep

* update poetry.lock

* remove extra arg

* reduce max iter for eval

* update poetry

* fix integration tests

---------

Co-authored-by: OpenDevin <opendevin@opendevin.ai>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-05-23 16:04:09 +00:00
Robert Brennan
5bdacf738d Refactor session management (#1810)
* refactor session mgmt

* defer file handling to runtime

* add todo

* refactor sessions a bit more

* remove messages logic from FE

* fix up socket handshake

* refactor frontend auth a bit

* first pass at redoing file explorer

* implement directory suffix

* fix up file tree

* close agent on websocket close

* remove session saving

* move file refresh

* remove getWorkspace

* plumb path/code differently

* fix build issues

* fix the tests

* fix npm build

* add session rehydration

* fix event serialization

* logspam

* fix user message rehydration

* add get_event fn

* agent state restoration

* change history tracking for codeact

* fix responsiveness of init

* fix lint

* lint

* delint

* fix prop

* update tests

* logspam

* lint

* fix test

* revert codeact

* change fileService to use API

* fix up session loading

* delint

* delint

* fix integration tests

* revert test

* fix up access to options endpoints

* fix initial files load

* delint

* fix file initialization

* fix mock server

* fixl int

* fix auth for html

* Update frontend/src/i18n/translation.json

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>

* refactor sessions and sockets

* avoid reinitializing the same session

* fix reconnect issue

* change up intro message

* more guards on reinit

* rename agent_session

* delint

* fix a bunch of tests

* delint

* fix last test

* remove code editor context

* fix build

* fix any

* fix dot notation

* Update frontend/src/services/api.ts

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* fix up error handling

* Update opendevin/server/session/agent.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Update opendevin/server/session/agent.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Update frontend/src/services/session.ts

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* fix build errs

* fix else

* add closed state

* delint

* Update opendevin/server/session/session.py

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-05-22 18:33:16 +00:00
மனோஜ்குமார் பழனிச்சாமி
d76c425b76 Refactored Logs (#1939) 2024-05-22 03:06:13 +08:00
மனோஜ்குமார் பழனிச்சாமி
4612e107c9 fix: Handle invalid exit code conversion (#1915) 2024-05-20 11:30:52 -04:00
Boxuan Li
0abc35cf57 ssh_box: Shutdown container when fail to start ssh session (#1872) 2024-05-18 17:04:38 +08:00
Boxuan Li
a57a213c7c Turn off auto linting by default, and on for swe_bench (#1861)
Disable Python linting by default, and turn it on for SWE Bench.

It is turned off by default since this behavior is weird and somewhat annoying to end users.
It is turned on for SWE Bench because linting python files gives LLM a chance to fix the indentations.
2024-05-18 04:04:38 +00:00
Xingyao Wang
0cb707bc9d fix(sandbox): ssh_box parsing (#1831)
* fix ssh_box error parsing

* Add type check

---------

Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: OpenDevinBot <bot@opendevin.com>
2024-05-16 18:03:34 -04:00
Xingyao Wang
be1aef5863 set user.name and user.email for opendevin (#1842) 2024-05-16 17:50:32 -04:00
Leo
e89cc8f19b Feat: add stream output to exec_run (#1625)
* Feat: add stream output to exec_run

* Using command timeout to control the exec_box's timeout.
* add bash -c to source command to compatible for sh.

Signed-off-by: ifuryst <ifuryst@gmail.com>

* Feat: add stream output to SSHBox execute

Signed-off-by: ifuryst <ifuryst@gmail.com>

* fix the test case fail.

Signed-off-by: ifuryst <ifuryst@gmail.com>

* fix the test case import wrong path for method.

Signed-off-by: ifuryst <ifuryst@gmail.com>

---------

Signed-off-by: ifuryst <ifuryst@gmail.com>
2024-05-16 14:37:49 +00:00
Xingyao Wang
9e59937180 fix utf-8 decoding issue (#1816) 2024-05-15 22:49:49 -07:00
Xingyao Wang
755a4072b6 Support specify timeout for long running tasks (#1756)
* support specify timeout for long running tasks

* add timeout for all existing sandbox impl

* Update opendevin/runtime/docker/local_box.py

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/docker/exec_box.py

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/docker/ssh_box.py

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/e2b/sandbox.py

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update opendevin/runtime/sandbox.py

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

---------

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
2024-05-13 10:17:03 +00:00
Xingyao Wang
00c0edae5f Re-adjust ssh_box for parallel evaluation (#1729)
* update ssh_box

* fix controller in test

---------

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-05-13 14:35:30 +08:00