86 Commits

Author SHA1 Message Date
மனோஜ்குமார் பழனிச்சாமி
143f38d25a Refactored sandbox config and added fast boot (#2455)
* Refactored sandbox config and added fastboot

* added tests

* fixed tests

* fixed tests

* intimate user about breaking change

* remove default config from eval

* check for lowercase env

* add test

* Revert Migration

* migrate old sandbox configs

* resolve merge conflict

* revert migration 2

* Revert "remove default config from eval"

This reverts commit de57c588db.

* change type to box_type

* fix var name

* linted

* lint

* lint comments

* fix tests

* fix tests

* fix typo

* fix box_type, remove fast_boot

* add tests for sandbox config

* fix test

* update eval docs

* small removal comments

* adapt toml template

* old fields shouldn't be in the app dataclass

* fix old keys in app config

* clean up exec box

---------

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-07-05 03:30:21 +00:00
Xingyao Wang
0d3b3ffbf8 [Arch] Removing docker exec box (#2802)
* depracting docker exec box

* remove doc exec from workflow and docs
2024-07-04 23:15:25 +00:00
மனோஜ்குமார் பழனிச்சாமி
cc8204e5c1 CI: For colima, modify CPU count and memory (#2712)
* CI: For colima, modify CPUcount and memory

* fix arg
2024-07-02 15:44:44 +02:00
Boxuan Li
79a3e81708 ghcr: Fix local built image name in tests (#2686)
* ghcr: Fix local built image name in tests

* Push to registry when tagged

* Fix push and experiment

* Remove debug
2024-06-29 09:53:30 -07:00
tobitege
1117dfebeb feat: update version to 0.6.2. added Action to update pyproject on Release (#2552)
* updated version; added Action to update pyproject version by current tag (if changed)

* higer pyproject version creates a tag now

* Release-only run to write tag to pyproject
2024-06-24 18:34:57 +02:00
மனோஜ்குமார் பழனிச்சாமி
c455a09e43 Remove Colima and lima directory after uninstalling for Mac OS CI (#2598)
* Remove colima dir after uninstall

* Delete lima dir
2024-06-23 08:28:29 +02:00
மனோஜ்குமார் பழனிச்சாமி
6bf1b56f06 Revert "Enable "vz" vm-type for MacOS CI (#2586)" (#2588)
This reverts commit 57b56c0536.
2024-06-22 23:45:14 +08:00
மனோஜ்குமார் பழனிச்சாமி
57b56c0536 Enable "vz" vm-type for MacOS CI (#2586) 2024-06-22 12:30:18 +00:00
மனோஜ்குமார் பழனிச்சாமி
0845d475b8 Fix Mac CI Test (#2569)
* Fix Mac CI Test

* Start colima service

* unlink colima dependency: go

* Check for colima

Co-authored-by: Graham Neubig <neubig@gmail.com>

* fix indent

Co-authored-by: Graham Neubig <neubig@gmail.com>

* Try with uninstall

---------

Co-authored-by: Graham Neubig <neubig@gmail.com>
2024-06-22 02:36:23 +00:00
Boxuan Li
01fa52d062 Enforce linter in tests folder (#2557) 2024-06-20 21:50:34 -07:00
மனோஜ்குமார் பழனிச்சாமி
f3ba2f02d5 Fix Mac OS CI - usernet unable to resolve IP for SSH forwarding (#2556) 2024-06-21 10:02:26 +05:30
Boxuan Li
bfa00932cc Enable test_agnostic_sandbox_jupyter_agentskills_fileop_pwd in CI (#2534)
* Enable test_agnostic_sandbox_jupyter_agentskills_fileop_pwd in CI

* Fix env variable value
2024-06-20 20:39:11 -06:00
மனோஜ்குமார் பழனிச்சாமி
35fc2177d0 Fix Mac CI test (#2544) 2024-06-21 00:56:23 +08:00
மனோஜ்குமார் பழனிச்சாமி
833bb50fb0 Downgrade Mac version in CI/CD Pipeline (#2499)
* downgrade mac version in CI

* Delete run-integration-tests.yml
2024-06-19 22:12:15 -07:00
Boxuan Li
89cae8d6b8 Fix Docker tagging issue with upper case (#2512)
* Fix Docker tagging issue with upper case

* Update containers/build.sh

Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>

* Use tr command which is available on both zsh and bash

* Lower image name

* Lower image name

* Update .github/workflows/ghcr.yml

Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>

* Fix shell syntax

---------

Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
2024-06-19 00:11:18 -06:00
Boxuan Li
c2fa99b4a1 Split container image build & push (#2456)
* Split container image build & push

* Code cleanup

* Cleanup

* Add back useless docker_build_success step to make CI happy

* Revert "Cleanup"

This reverts commit 2a260791a9.

* Use fresh built sandbox image in integration test

* fix dependency

* DEBUG: only build

* Attempt to fix dependency

* Change dependency

* Combine both jobs

* Fix env

* Remove Mac integration tests as they are too unstable

* Move sandbox tests to ghcr

* Use loaded image
2024-06-19 01:44:51 +08:00
மனோஜ்குமார் பழனிச்சாமி
e0ad289483 Downgraded Python version to 3.12.3 (#2331)
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-06-09 11:54:30 +05:30
Yufan Song
dc94914ad7 fix dogfood (#2313) 2024-06-07 16:35:12 +00:00
Leo
9ada36e30b fix: restore python linting. (#2228)
* fix: restore python linting.

Signed-off-by: ifuryst <ifuryst@gmail.com>

* update: extend the Python lint check to evaluation.

Signed-off-by: ifuryst <ifuryst@gmail.com>

* Update evaluation/logic_reasoning/instruction.txt

---------

Signed-off-by: ifuryst <ifuryst@gmail.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-06-04 06:36:19 +00:00
Yufan Song
2374374778 Fix python environment in review-pr dogfood action (#2237)
This doesn't completely fix the bug; it fixes the python environment, and there is more to do to fix the issue.
2024-06-03 20:31:38 -07:00
மனோஜ்குமார் பழனிச்சாமி
d4ccd48af8 Persistent docker session (#1998)
Co-authored-by: Robert Brennan <accounts@rbren.io>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Graham Neubig <neubig@gmail.com>
2024-05-29 13:22:34 +00:00
Boxuan Li
593b8d468b Fix CI workflows [mac-test] (#2025)
* Fix CI settings

* Stop saving cpu cycles for GitHub

* Conditionally run mac tests

* Random push to trigger CI checks again

---------

Co-authored-by: Graham Neubig <neubig@gmail.com>
2024-05-24 09:25:00 +00:00
Xingyao Wang
602ffcdffb Implement agentskills for OpenDevin to helpfully improve edit AND including more useful tools/skills (#1941)
* add draft for skills

* Implement and test agentskills functions: open_file, goto_line, scroll_down, scroll_up, create_file, search_dir, search_file, find_file

* Remove new_sample.txt file

* add some work from opendevin w/ fixes

* Add unit tests for agentskills module

* fix some issues and updated tests

* add more tests for open

* tweak and handle goto_line

* add tests for some edge cases

* add tests for scrolling

* add tests for edit

* add tests for search_dir

* update tests to use pytest

* use pytest --forked to avoid file op unit tests to interfere with each other via global var

* update doc based on swe agent tool

* update and add tests for find_file and search_file

* move agent_skills to plugins

* add agentskills as plugin and docs

* add agentskill to ssh box and fix sandbox integration

* remove extra returns in doc

* add agentskills to initial tool for jupyter

* support re-init jupyter kernel (for agentskills) after restart

* fix print window's issue with indentation and add testcases

* add prompt for codeact with the newest edit primitives

* modify the way line number is presented (remove leading space)

* change prompt to the newest display format

* support tracking of costs via metrics

* Update opendevin/runtime/plugins/agent_skills/README.md

* Update opendevin/runtime/plugins/agent_skills/README.md

* implement and add tests for py linting

* remove extra text arg for incompatible subprocess ver

* remove sample.txt

* update test_edits integration tests

* fix all integration

* Update opendevin/runtime/plugins/agent_skills/README.md

* Update opendevin/runtime/plugins/agent_skills/README.md

* Update opendevin/runtime/plugins/agent_skills/README.md

* Update agenthub/codeact_agent/prompt.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Update agenthub/codeact_agent/prompt.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Update agenthub/codeact_agent/prompt.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Update opendevin/runtime/plugins/agent_skills/agentskills.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* correctly setup plugins for swebench eval

* bump swe-bench version and add logging

* correctly setup plugins for swebench eval

* bump swe-bench version and add logging

* Revert "correctly setup plugins for swebench eval"

This reverts commit 2bd1055673.

* bump version

* remove _AGENT_SKILLS_DOCS

* move flake8 to test dep

* update poetry.lock

* remove extra arg

* reduce max iter for eval

* update poetry

* fix integration tests

---------

Co-authored-by: OpenDevin <opendevin@opendevin.ai>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-05-23 16:04:09 +00:00
Boxuan Li
a605e59b7e Save CI cycles for backend tests (#1985) 2024-05-23 00:10:13 -07:00
Boxuan Li
acb430eef5 Refactor integration testing CI, add optional Mac tests, and mark a few agents as deprecated (#1888)
* Add MacOS to integration tests

* Switch back to python 3.11

* Install Docker for macos pipeline

* regenerate.sh: Use environmental variable for sandbox type

* Pack different agents' tests into a single check

* Fix CodeAct tests

* Reduce file match and extensive debug logs

* Add TEST_IN_CI mode that reports codecov

* Small fix: don't quit if reusing old responses failed

* Merge codecov results

* Fix typos

* Remove coverage merge step - codecov automatically does that

* Make mac integration tests as optional - too slow

* Fix codecov args

* Add comments in yaml

* Include sandbox type in codecov report name

* Fix codecov report merge

* Revert renaming of test_matrix_success

* Remove SWEAgent and PlannerAgent from tests

* Mark planner agent and SWE agent as deprecated

* CodeCov: Ignore planner and sweagent

* Revert "Remove SWEAgent and PlannerAgent from tests"

This reverts commit 040cb3bfb9.

* Remove all tests for SWE Agent

* Only keep basic tests for MonologueAgent and PlannerAgent

* Mark SWE Agent as deprecated, and ignore code coverage for it

---------

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-05-22 20:38:57 -07:00
Rahul Anand
9a2591d0f6 fix #1960 (#1964) 2024-05-22 19:36:45 -07:00
Boxuan Li
6714000b2c CodeActAgent: Fix iteration reminder (#1803)
This PR includes three changes:
1) Iteration reminder should start with MAX_ITERATIONS from config rather than default value 100
2) In the first prompt, we should tell the LLM it has `MAX_ITERATIONS - 1` turns left, rather than `MAX_ITERATIONS - 2`
3) Remove legacy ITERATION_REMINDER config
2024-05-15 13:48:47 +08:00
Xingyao Wang
8bfae8413e Support passing sandbox as argument and iteration reminder (#1730)
* support custom sandbox;
add iteration_reminder

* Enable iteration reminder in CodeActAgent integration test

* Don't remove numbers when comparing prompts

* Update tests/integration/README.md

---------

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-05-12 07:57:33 +00:00
Xingyao Wang
1d58917bc8 remove swe-bench build workflow (#1726) 2024-05-12 06:56:20 +08:00
Xia Zhenhua
5244a34a1d feat: skip deploy-decs in folk repos. (#1703)
Co-authored-by: aaren.xzh <aaren.xzh@antfin.com>
2024-05-11 01:44:25 -07:00
Arno.Edwards
06aae67fed feat(makefile): add capability to skip Docker image pull (#1664) 2024-05-09 09:06:26 -04:00
Robert Brennan
72efa05c71 Update ghcr.yml (#1576) 2024-05-04 08:32:12 -04:00
Robert Brennan
322b23550d run ghcr on tag (#1575) 2024-05-04 12:18:21 +00:00
Alex Bäuerle
d9ba45dae8 ci(docs): only generate autogen python docs on deploy (#1501)
* ci(docs): only generate autogen python docs on deploy

* poetry

* workdir
2024-05-02 18:29:41 +00:00
Robert Brennan
fadcdc117e Migrate to new folder structure in preparation for refactor (#1531)
* fix up folder structure

* update docs

* fix imports

* fix imports

* fix imoprt

* fix imports

* fix imports

* fix imports

* fix test import

* fix tests

* fix main import
2024-05-02 17:01:54 +00:00
Robert Brennan
0bdbd8a90d disable action concurrency (#1503)
* disable action concurrency

* empty commit
2024-05-02 13:30:04 +00:00
Robert Brennan
8dbbbcf1ba dont run lint on main (#1518) 2024-05-02 09:11:11 -04:00
Frank Xu
836864fa88 [feat] Integrate BrowserGym (#1452)
* add a single-threaded server serving browsergym

* update poetry

* update browser page content

* add import to make sure browsergym environments are registered properly

* remove flask server, use multiprocess impl and Pipe

* fix

* refactor BrowserEnv

* update browser action and obs to include more complete info

* fix screenshot

* update poetry lock

* add playwright install to workflow

* update

* add better html to text conversion

* update for better text conversion to maintain parity with the current handling of browseurlaction

* update

* update poetry

* update multiprocessing mp

* fix multiprocessing

* update

* update github workflow

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-05-02 19:52:53 +08:00
Robert Brennan
24cc8ceb96 add back free storage (#1506) 2024-05-02 02:44:10 +00:00
Jirka Borovec
1b810cfbf0 ci/lint: fix calling Ruff's format (#1457)
* ci/lint: fix calling Ruff's format

* Transition for ruff lint. Only checking the modified files.

---------

Co-authored-by: ifuryst <ifuryst@gmail.com>
2024-05-01 22:19:54 -04:00
Robert Brennan
cfef3ee5c4 Fix docker push for non-forks (#1499)
* fix fork check

* minor docker tweaks

* remove caching

* try not chowinng

* fix chowns

* revert build.sh

* fix entrypoint user

* change message

* remove free disk space

* chown the entrypoint

* remove comments

* empty commit
2024-05-02 10:15:12 +08:00
Robert Brennan
c50319138e Revert "feat(makefile): add capability to skip Docker image pull (#1463)" (#1489)
This reverts commit 442ab7371c.
2024-05-01 11:00:06 -04:00
Arno.Edwards
442ab7371c feat(makefile): add capability to skip Docker image pull (#1463)
* feat(makefile): add capability to skip Docker image pull

* ci(github-actions): add conditional Docker installation based on ENV variable
2024-05-01 09:01:11 -04:00
Robert Brennan
0cda5f64af Add integration test with dummy agent (#1316)
* first pass at dummy

* add assertion to dummy

* add dummy workflow

* beef up tests

* try and fix huggingface issue

* remove newlines

* rename test

* move to pytest

* Revert " move to pytest"

This reverts commit de8121c400.

* fix lint

* delint

* Update .github/workflows/dummy-agent-test.yml

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

---------

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-04-30 16:52:00 +00:00
Xingyao Wang
fe43aeb9b6 Revert ssh box implemetation, fix multi-line command issues and add unit tests (#1460)
* update dev md for instruction of running unit tests

* add back unit tests

* revert back to the original sandbox implementation to fix testcases

* allow unit test workflow to find docker.sock

* make sandbox test working via patch

* fix arg parser that's broken for some reason

* fix integration test

* Revert "fix arg parser that's broken for some reason"

This reverts commit 6cc8961133.

* update Development.md
2024-04-30 12:46:35 -04:00
Xingyao Wang
ccbbabac1c Get RUN_AS_DEVIN working with app sandbox (#1426)
* get RUN_AS_DEVIN and network=host working with app sandbox

* attempt to fix the workspace base permission

* sandbox might failed in chown due to mounting, but it won't be fatal

* update sshbox instruction

* remove default user id since it will be passed in the instruction

* revert permission fix since it should be resolved by correct SANDBOX_USER_ID

* the permission issue can be fixed by simply provide correct env var

* remove log

* set sandbox user id to getuid by default

* move logging to initializer

* make the uid consistent across host, app container, and sandbox

* remove hostname as it causes sudo issue

* fix permission of entrypoint script

* make the uvicron app run as host user uid for jupyter plugin

* revert use host network

* get docker socket gid and usermod instead of chmod 777

* try to fix app build disk space issue
2024-04-30 12:39:52 -04:00
Robert Brennan
11d48cc2f3 Run docker on forked pull requests (#1450)
* build docker on pull request

* run docker build on PRs

* remove if

* add permissions

* change ghcr login

* empty commit

* always use opendevin as org

* lowercase

* no client token

* dont push on forks

* remove env

* only cache-to if pushing

* fix org name

* fix owner

* Update containers/build.sh

Co-authored-by: Graham Neubig <neubig@gmail.com>

* lowercase

* remove tag prefix

* lowercase

---------

Co-authored-by: Graham Neubig <neubig@gmail.com>
2024-04-29 14:24:28 -04:00
Alex Bäuerle
b3930ad195 ci: fix deploy config (#1454) 2024-04-29 17:28:30 +00:00
Alex Bäuerle
cd58194d2a docs(docs): start implementing docs website (#1372)
* docs(docs): start implementing docs website

* update video url

* add autogenerated codebase docs for backend

* precommit

* update links

* fix config and video

* gh actions

* rename

* workdirs

* path

* path

* fix doc1

* redo markdown

* docs

* change main folder name

* simplify readme

* add back architecture

* Fix lint errors

* lint

* update poetry lock

---------

Co-authored-by: Jim Su <jimsu@protonmail.com>
2024-04-29 10:00:51 -07:00
Boxuan Li
a5a1545c59 Add Codecov test coverage (#1397)
* Add pytest-cov dependency

* Add codecov to unit test CI

* Codecov: Use secrets

* Add codecov for integration tests
2024-04-29 10:33:13 -04:00