Commit Graph

168 Commits

Author SHA1 Message Date
Xingyao Wang
47774e60b0 chore: remove deprecated dockerfile (#4079) 2024-09-27 15:03:23 +00:00
tobitege
c32cec7f89 (enh) send status messages to UI during startup (#3771)
Co-authored-by: Robert Brennan <accounts@rbren.io>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Robert Brennan <contact@rbren.io>
Co-authored-by: sp.wack <83104063+amanape@users.noreply.github.com>
2024-09-24 18:46:58 +00:00
Qiang Li
f7ebc1cf1f chore: Add docker files for developing inside container. (#3911) 2024-09-17 23:19:01 +02:00
Robert Brennan
3a1c547c8c [WIP] Fix docker push issues (#3585)
* empty commit

* fix workflows for forks

* fix runtime tests

* fix repo name

* fix artifacts

* add download step
2024-08-26 19:36:28 +00:00
Robert Brennan
b63dec4b2e Add back docker caching, simplify docker builds (#3546)
* fix multiarch

* remove extra push

* add back tag file

* fix cache tag

* add login step

* fix login

* try to fix save

* fix output maybe

* rm outputs

* remove tars

* fix refs

* fix runtime dep

* force rebuild

* lowercase image

* add suffix to build tags for runtime

* update matrix

* fix cut

* fix cut again

* add back matrix

* Update containers/build.sh

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-08-23 17:01:18 +00:00
Robert Brennan
3fe7894966 update docs for headless mode (#3508)
* update docs for headless mode

* fix newline

* fix command parsing

* add docs

* Update README.md

* Update headless-mode.md

* empty commit

* update integration tests

---------

Co-authored-by: tobitege <10787084+tobitege@users.noreply.github.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-08-20 23:57:48 +00:00
Xingyao Wang
8f0f764a85 fix: CI docker image push (#3476)
* fix ghcr app

* fix ghcr runtime push

* rename od_runtime to runtime
2024-08-19 20:53:28 +00:00
Robert Brennan
01ae22ef57 Rename OpenDevin to OpenHands (#3472)
* Replace OpenDevin with OpenHands

* Update CONTRIBUTING.md

* Update README.md

* Update README.md

* update poetry lock; move opendevin folder to openhands

* fix env var

* revert image references in docs

* revert permissions

* revert permissions

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-08-20 00:44:54 +08:00
Xingyao Wang
83f36c1d66 test: build and run runtime tests on different custom docker images (#3324)
* try to fix pip unavailable

* update test case for pip

* force rebuild in CI

* remove extra symlink

* fix newline

* added semi-colon to line 31

* Dockerfile.j2: activate env at the end

* Revert "Dockerfile.j2: activate env at the end"

This reverts commit cf2f565102.

* cleanup Dockerfile

* switch default python image

* remove image agnostic (no longer used)

* fix tests

* simplify integration tests default image

* add nodejs specific runtime tests

* update tests and workflows

* switch to nikolaik/python-nodejs:python3.11-nodejs22

* update build sh to output image name correctly

* increase custom images to test

* fix test

* fix test

* fix double quote

* try fixing ci

* update ghcr workflow

* fix artifact name

* try to fix ghcr again

* fix workflow

* save built image to correct dir

* remove extra -docker-image

* make last tag to be human readable image tag

* fix hyphen to underscore

* run test runtime on all tags

* revert app build

* separate ghcr workflow

* update dockerfile for eval

* fix tag for test run

* try fix tag

* try fix tag via matrix output

* try workflow again

* update comments

* try fixing test matrix

* fix artifact name

* try fix tag again

* Revert "try fix tag again"

This reverts commit b369badd8c.

* tweak filename

* try different path

* fix filepath

* try fix tag artifact path again

* save json instead of line

* update matrix

* print all tags in workflow

* fix DOCKER_IMAGE to avoid ghcr.io/opendevin/ghcr.io/opendevin/od_runtime

* fix test matrix to only load unique test image tags

* try fix matrix again!!!!!

* add all runtime tests passed

---------

Co-authored-by: tobitege <tobitege@gmx.de>
Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: tobitege <10787084+tobitege@users.noreply.github.com>
2024-08-19 21:12:00 +08:00
Xingyao Wang
8d7bf83224 refactor: break agentskills single file to multiple composable modules (#3429)
* refactor agentskills to prepare for agentless

* fix import

* fix typo

* fix imports

* fix globals

* fix import

* fix import

* disable log to file to avoid auto-created log file w/ permission issue when import od in runtime

* import agentskills from OD instead from itself directly

* add back pythonpath

* remove chown since there's no log/folder
2024-08-18 05:18:36 +08:00
Xingyao Wang
1cf44ef854 fix app docker again! (#3427) 2024-08-16 21:43:56 +00:00
Xingyao Wang
3f20e4edc6 attempt to fix docker issue (#3426) 2024-08-16 19:41:24 +00:00
tobitege
3a77af8a22 copy manifest file into container (#3396) 2024-08-14 21:46:56 -04:00
mamoodi
e2b2f74737 Add comments to the runtime_build script (#3333) 2024-08-12 15:17:12 -04:00
Xingyao Wang
2e6b08db4f fix: workspace folder permission & app container cannot access client API (#3300)
* also copy over pyproject and poetry lock

* add missing readme

* remove extra git config init since it is already done in client.py

* only chown if the /workspace dir does not exists

* Revert "remove extra git config init since it is already done in client.py"

This reverts commit e8556cd76d.

* remove extra git config init since it is already done in client.py

* fix test runtime

* print container log while reconnecting

* print log in more readable format

* print log in more readable format

* increase lines

* clean up sandbox and ssh related stuff

* remove ssh hostname

* remove ssh hostname

* fix docker app cannot access runtime API issue

* remove ssh password

* API HOSTNAME should be pre-fixed with SANDBOX

* update config

* fix typo that breaks the test
2024-08-08 19:28:34 -04:00
Xingyao Wang
a5195b0e65 chore: clean up sandbox and ssh related configs (#3301)
* clean up sandbox and ssh related stuff

* remove ssh hostname

* remove ssh hostname

* remove ssh password

* update config

* fix typo that breaks the test
2024-08-08 22:15:40 +00:00
Xingyao Wang
4915168da2 fix: copy over pyproject and poetry lock for App docker (#3299)
* also copy over pyproject and poetry lock

* add missing readme
2024-08-08 21:06:26 +00:00
tobitege
f561b48f03 Dockerfile for app: re-declare ARG OPEN_DEVIN_BUILD_VERSION (#3257) 2024-08-06 04:12:30 +08:00
Xingyao Wang
a69120d399 [Arch] Use hash to avoid repeat building EventStreamRuntime image (#3243)
* update the behavior of put source code to put files instead of tarball

* add dishash to dependency

* fix dockerfile copy

* use dirhash to avoid repeat building for update source

* fix runtime_build testcase

* add dir_hash to docker build pipeline

* add additional tests for source directory

* add comment

* clear the assertion by explictly check existing files

* also assert od is a dir
2024-08-05 03:13:32 +00:00
Xingyao Wang
1d49ef253b [Runtime] Reduce dependency to speed up CI and reduce image size (#3195)
* reduce dependency for runtime

* try making llama-index an optional dependency that's not installed by default

* do not install llama-index in CI

* do not install llama-index in the app docker as well
2024-07-31 13:55:09 -04:00
tobitege
a52df6a272 (Docker) Fix passing of POETRY_CACHE_DIR into RUN command (#3163)
* Fix RUN command to get correct value of POETRY_CACHE_DIR

* Fix passing of POETRY_CACHE_DIR into RUN command

* remove test copy of Dockerfile
2024-07-29 23:16:46 +08:00
Xingyao Wang
405c8a0456 [Arch] Add runtime image build CI & clean up runtime build using jinja2 template (#3055)
* test_runtime_client.py to test _execute_bash()

* runtime_build and runtime tweaks

* fix in docker script

* revert bash changes

* use sandbox_config.update_source_code to control source code update

* add od_version to the sandbox tag

* add doc instruction for update source code

* do not remove whole poetry folder;
add mamba clean

* add missing newlines

* cleanup runtime dockerfile into jinja template

* make prep temp file a separate function;
make that function accessible through cli

* modify `runtime_build.py` so it can generate directory for building docker img

* add dockerfile and sdist of runtime to gitignore since it will be dynamically generated

* add runtime to build

* do not rebuild new image when an `od_runtime` is provided

* use default container_image for testing if possible

* move runtime tests to ghcr runtime workflow

* update docker base dir for runtime

* fix unittest

* fix image name

* fix image name for test case

* rename to make it consistent

---------

Co-authored-by: tobitege <tobitege@gmx.de>
2024-07-24 21:56:12 +08:00
Zheng Chaojian
15697bed5a Update Dockerfile casing (#3045)
* Update Dockerfile casing

* Update Dockerfile
2024-07-20 08:00:37 -04:00
மனோஜ்குமார் பழனிச்சாமி
b60890c064 Fix playwright cache error during container restart (#3011) 2024-07-19 09:30:19 +02:00
Yufan Song
351127db55 add od runtime clinet dependencies (#2858) 2024-07-08 18:20:44 +00:00
Shimada666
e35c1ff74a Display real-time build logs for the agnostic image (#2830)
* Display real-time build logs for the agnostic image and improve wget's output.

* remove unused code
2024-07-08 04:35:16 +08:00
Shimada666
82f256be96 trim the sandbox image and install plugin dependencies in agnostic image (#2792)
* trim the sandbox image

* remove wrong code

* readd python

* readd python

* fix script

* readd nano
2024-07-06 17:38:37 +02:00
Shimada666
d22ff73905 Make the sandbox Python runtime completely transparent (#2796)
* Make the sandbox Python runtime completely independent

* fix source bashrc

* add pip install instruction for ipython to fix intergration tests for codeact swe

* update integration tests

* change flake8 command to (maybe) fix sandbox tests?

* make lint support both unittest & sandbox

* fix agnostic image build error

* refactor build script

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: tobitege <tobitege@gmx.de>
2024-07-06 15:22:44 +02:00
r.e.e.c.h.e.e
d894347f9b Add 'latest' tag to Docker builds for stable releases only (#2781)
- Ensure 'latest' always points to the most recent stable version
- Address issue #2730: Release "latest" tag when pushing image to DockerHub
2024-07-04 18:06:54 +08:00
Shimada666
5972498e23 No longer chown -R the miniforge3 folder (#2566)
* No longer chown -R the miniforge3 folder

* change miniforge3 group permission
2024-06-21 17:33:00 +00:00
Shimada666
26fc3c886a Make plugins sandbox-agnostic (#2101)
* tmp

* tmp

* merge main

* feat: auto build image cache

* remove plugins

* use config file

* update mamba setup shell

* support agnostic sandbox image autobuild

* remove config

* Update .gitignore

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>

* Update opendevin/runtime/docker/ssh_box.py

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>

* update setup.sh

* readd sudo

* add sudo in dockerfile

* remove export

* move od-runtime dependencies to sandbox dockerfile

* factor out re-build logic into a separate util file

* tweak existing plugin to use OD specific sandbox

* update testcase

* attempt to fix unit test using image built in ghcr

* use cache tag

* try to fix unit tests

* add unittest

* add unittest

* add some unittests

* revert gh workflow changes

* feat: optimize sandbox image naming rule

* add pull latest image hint

* add opendevin python hint and use mamba to install gcc

* update docker image naming rule and fix mamba issue

* Update opendevin/runtime/docker/ssh_box.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* fix: opendevin user use correct pip

* fix lint issue

* fix custom sandbox base image

* rename test name

* add skipif

---------

Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
Co-authored-by: tobitege <tobitege@gmx.de>
2024-06-19 19:58:07 -07:00
Boxuan Li
89cae8d6b8 Fix Docker tagging issue with upper case (#2512)
* Fix Docker tagging issue with upper case

* Update containers/build.sh

Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>

* Use tr command which is available on both zsh and bash

* Lower image name

* Lower image name

* Update .github/workflows/ghcr.yml

Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>

* Fix shell syntax

---------

Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
2024-06-19 00:11:18 -06:00
Boxuan Li
c2fa99b4a1 Split container image build & push (#2456)
* Split container image build & push

* Code cleanup

* Cleanup

* Add back useless docker_build_success step to make CI happy

* Revert "Cleanup"

This reverts commit 2a260791a9.

* Use fresh built sandbox image in integration test

* fix dependency

* DEBUG: only build

* Attempt to fix dependency

* Change dependency

* Combine both jobs

* Fix env

* Remove Mac integration tests as they are too unstable

* Move sandbox tests to ghcr

* Use loaded image
2024-06-19 01:44:51 +08:00
Shimada666
e4d78153fe feat: add environments in global bashrc file (#2486) 2024-06-18 02:36:35 +00:00
tobitege
823298e0d0 fix: Agentskills enhancements (#2384)
* avoid repeat logging of unneeded messages

* refactored append/edit_file (tests next)

* agentskills and unit test fixes

* testing

* more changes and test prompts

* smaller changes

* final test fixes

* remove dead code from test_agent.py

* reverting unneeded changes

* updated tests, more tweaks to skills

* refactor (#2442)

* chores: fix DelegatorAgent description (#2446)

* change

* change comments

* fix

* stopped container to prevent port issues. (#2447)

* chore: remove useless browsing code in CodeActSWEAgent (#2438)

* remove useless

* fix integration test

* Regenerate test_ipython_module artifacts for CodeActSWEAgent

---------

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Merge remote-tracking branch 'upstream/main' into agent-fileops

* unneeded tweak

* * fix edit_file to not introduce extra newline
* updated docstrings with more details for LLM
* fix legacy typo in prompts causing ]] instead of ]
* several mock files regenerated

* Regen'ed CodeActSWEAgent integration tests

* fix _print_window signature; explicit exception type in _is_valid_path

* splitlines with named param

---------

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-06-16 15:06:46 -04:00
Yufan Song
e04bde851e test (#2409)
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-06-13 21:10:17 +08:00
மனோஜ்குமார் பழனிச்சாமி
e0ad289483 Downgraded Python version to 3.12.3 (#2331)
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-06-09 11:54:30 +05:30
மனோஜ்குமார் பழனிச்சாமி
971ad68431 Solved Hugging Face cache issue. (#2277) 2024-06-05 21:18:33 +05:30
Shimada666
b31f7701eb Integrate Multimodal tools to agentskills. (#2016)
* suport reading multimodal files

* move file

* update dependency

* remove useless pip install

* add comments

* update the comment

* Apply suggestions from code review

* Add unit test for TXTReader

* pre-commit hook corrupted utf16 test txt

* Revert unnecessary dependency upgrades

* feat: import some readers for agentskill

* add dependencies

* Integrate some multimodal tools

* add shell pip dependency

* update dependencies

* update dependencies

* update print window

* remove __main__

* locally import cv2

* add c library for opencv

* update lock file

* update prompt

* remove unuseful file

* add some unittest

* add unittest & remove excel-related parser

* rollback poetry lock

* remove markdown

* remove requests

* optimize parse_video output

* Fix integration tests for CodeActAgent

* remove test_parse_image unittest

* Add a TODO to containers/sandbox/Dockerfile

* update dependencies

* remove pyproject.toml useless package

* change document via openai key

* Fix prompts after removing some actions

---------

Co-authored-by: Mingchen Zhuge <mczhuge@gmail.com>
Co-authored-by: yufansong <yufan@risingwave-labs.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
Co-authored-by: Mingchen Zhuge <64179323+mczhuge@users.noreply.github.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-05-25 18:58:49 +08:00
மனோஜ்குமார் பழனிச்சாமி
6ef7e6eb0f Checked for enduser in Docker (#1899) 2024-05-20 08:50:42 +05:30
John Tackman
171764469d Update Dockerfile to assign workspace directory properly to user (#1830)
* Update Dockerfile

creating the workspace directory after switching user to fix permission errors
https://github.com/OpenDevin/OpenDevin/issues/1560

* assign $workspace_base to user

instead of creating directory later, just fix the permissions so user can write to it

* another sudo

---------

Co-authored-by: Leo <ifuryst@gmail.com>
2024-05-17 20:11:23 -04:00
Xingyao Wang
c320d908e5 add iproute2 to sandbox (#1859) 2024-05-17 12:57:58 +08:00
Robert Brennan
135320861c set a higer UID_MAX (#1788)
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
2024-05-15 00:28:37 +00:00
Xingyao Wang
1d58917bc8 remove swe-bench build workflow (#1726) 2024-05-12 06:56:20 +08:00
Robert Brennan
1cbb16cfc2 allow running app as root (#1651)
* allow running app as root

* better entrypoint mgmt

* add nosetup option

* remove comments

* create docker group if it doesnt exist

* better docker group mgmt

* cast bools better

* fix playwright

* fix playwright for root

* fix root source ~/.bashrc hangs by create clean bashrc

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-05-10 20:41:25 +00:00
Robert Brennan
7b6b4e3a11 be more dynamic around uid generation (#1584)
* be more dynamic around uid generation

* fix comment

* fix second uid add
2024-05-04 22:26:55 -04:00
Robert Brennan
1d92e9a27b Fix user permissions in docker (#1565)
* permissions mostly working

* fix browser
2024-05-04 13:08:43 +08:00
Robert Brennan
7d8e10aa18 Remove v prefix on tags (#1555) 2024-05-03 10:36:09 -04:00
Robert Brennan
7914258945 Fix docker build, integration test (#1533)
* fix docker build

* revert plugins

* fix imports

* fix import

* fix one more import
2024-05-02 13:49:25 -04:00
Robert Brennan
cfef3ee5c4 Fix docker push for non-forks (#1499)
* fix fork check

* minor docker tweaks

* remove caching

* try not chowinng

* fix chowns

* revert build.sh

* fix entrypoint user

* change message

* remove free disk space

* chown the entrypoint

* remove comments

* empty commit
2024-05-02 10:15:12 +08:00