* deprecating recall action
* fix integration tests
* fix integration tests
* refractor runtime to use async
* remove search memory
* rename .initialize to .ainit
* draft of runtime image building (separate from img agnostic)
* refractor runtime build into separate file and add unit tests for it
* fix image agnostic tests
* move `split_bash_commands` into a separate util file
* fix bash pexcept parsing for env
* refractor add_env_var from sandbox to runtime;
add test runtime for env var, remove it from sandbox;
* remove unclear comment
* capture broader error
* make `add_env_var` handle multiple export at the same time
* add multi env var test
* fix tests with new config
* make runtime tests a separate ci to avoid full disk
* Update Runtime README with architecture diagram and detailed explanations
* update test
* remove dependency of global config in sandbox test
* fix sandbox typo
* runtime tests does not need ghcr build now
* remove download runtime img
* remove dependency of global config in sandbox test
* fix sandbox typo
* try to free disk before running the tests
* Update opendevin/runtime/client/README.md
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* Update opendevin/runtime/client/README.md
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* Update opendevin/runtime/client/README.md
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* try to reduce code duplication
* Update opendevin/runtime/client/README.md
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* Update opendevin/runtime/client/README.md
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* Update opendevin/runtime/client/README.md
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* Update opendevin/runtime/client/README.md
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* Update opendevin/runtime/client/README.md
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* cleanup before setup
* temporarily remove this enable lint test since env var are now handled by runtime
* linter
---------
Co-authored-by: OpenDevin <opendevin@all-hands.dev>
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* Updated documentation using ruff's autofix feature
* Updated pyproject.toml to include docstring validations
* Updated documentation using ruff's autofix feature
* Updated pyproject.toml to include docstring validations
* Updated docstrings using ruff's autfix feature
* Deleted opendevin/runtime/utils/soource.py, Keeping in sync with main
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
* support loading a particular runtime class via config.runtime (default to server to not break things)
* move image agnostic util to shared runtime util
* move dependency
* include poetry.lock in sdist
* accept port as arg for client
* make client start server with specified port
* update image agnostic utility for eventstream runtime
* make client and runtime working with REST API
* rename execute_server
* add plugin to initialize stuff inside es-runtime;
cleanup runtime methods to delegate everything to container
* remove redundant ls -alh
* fix jupyter
* improve logging in agnostic sandbox
* improve logging of test function
* add read & edit
* update agnostic sandbox
* support setting work dir at start
* fix file read/write test
* fix unit test
* update tescase
* Fix unit test again
* fix unit test again again
Currently, OpenDevin uses a global singleton LLM config and a global singleton agent config. This PR allows customers to configure an LLM config for each agent. A hypothetically useful scenario is to use a cheaper LLM for repo exploration / code search, and a more powerful LLM to actually do the problem solving (CodeActAgent).
Partially solves #2075 (web GUI improvement is not the goal of this PR)
* refactor: Enhance file handling and code editing functionality
# PR Summary
**refactor: Enhance file handling and code editing functionality**
## PR Description
This pull request includes improvements to file handling, error management, and code editing functionality across multiple files. The changes enhance the robustness, security, and user experience of the application.
### Changes in `listen.py`
1. **Imports and Error Handling**:
- Removed `warnings` import and its usage with `litellm`.
- More consistent use of `JSONResponse` and `HTTPException` for error handling.
2. **WebSocket Endpoint (`/ws`)**:
- Simplified logic for handling events using a single `isinstance` check.
3. **New Endpoint**:
- Added `/api/save-file` POST endpoint for saving file contents.
- Implemented checks for agent state before allowing file edits.
4. **Code Style and Organization**:
- Improved code formatting and organization.
- Refactored some functions for better readability and consistency.
### Changes in `fileService.ts`
1. **Error Handling**:
- Added try-catch blocks to all functions for better error handling and logging.
2. **Input Sanitization**:
- Implemented `encodeURIComponent()` for file names and paths in API requests.
3. **Type Checking**:
- Added type checks for API responses to ensure data format consistency.
4. **File Upload Improvement**:
- Refactored `uploadFiles()` to use `Array.from(files)` instead of a for loop.
5. **New Functionality**:
- Added `saveFile()` function to allow saving file content to a specified path.
### Changes in `CodeEditor.tsx`
1. **New Dependencies**:
- Added imports for state management, UI components, and file operations.
2. **State Management**:
- Introduced new state variables for tracking save status and last saved time.
- Implemented Redux state management for code and agent state.
3. **UI Enhancements**:
- Added a save button with dynamic colors based on save status.
- Implemented a save notification system.
- Added a "Last saved" timestamp display.
4. **File Saving Functionality**:
- Implemented complete file saving feature with error handling and user feedback.
5. **Code Structure**:
- Improved structure with additional hooks and memoized values for optimization.
### Testing Performed
- Manually tested new file saving functionality.
- Verified error handling and user feedback mechanisms.
- Checked integration between backend (`listen.py`) and frontend (`fileService.ts`, `CodeEditor.tsx`).
### Next Steps
- Conduct thorough testing of the file saving feature across different scenarios.
- Update documentation to reflect new file handling capabilities.
- Consider adding unit tests for new functions and components.
* Added Docstrings back
Added Docstrings back
* Fix
# Allow Code Editing in AWAITING_USER_INPUT State
## Description
This pull request extends the functionality of the code editor to allow editing when the agent is in the AWAITING_USER_INPUT state, in addition to the existing PAUSED and FINISHED states.
## Changes
1. Backend (`listen.py`):
- Updated the `save_file` function to allow saving when the agent state is AWAITING_USER_INPUT.
2. Frontend (`CodeEditor.tsx`):
- Modified the `isEditingAllowed` condition to include the AWAITING_USER_INPUT state.
## Files Changed
- `listen.py`
- `CodeEditor.tsx`
## Testing
- Verified that the save button appears when the agent is in the AWAITING_USER_INPUT state.
- Tested saving files in all three allowed states (PAUSED, FINISHED, AWAITING_USER_INPUT).
- Ensured that saving is still prohibited in other agent states.
## Additional Notes
This change improves the user experience by allowing code edits while the agent is waiting for user input, which is a common scenario in interactive coding sessions.
* Add internationalization for 'File saved successfully' message
# Add internationalization for 'File saved successfully' message
## Description
This PR adds internationalization support for the "File saved successfully" message in the CodeEditor component. It updates the translation.json file to include translations for multiple languages and modifies the CodeEditor.tsx file to use the new translation key.
## Changes
1. Updated `translation.json`:
- Added a new key `CODE_EDITOR$FILE_SAVED_SUCCESSFULLY` with translations for multiple languages.
- Ensured the file structure supports multiple languages per key.
2. Modified `CodeEditor.tsx`:
- Updated the success message to use the new translation key.
- Applied the translation to both the toast notification and the on-screen notification.
## Why
These changes improve the user experience for non-English speakers by providing localized feedback when a file is successfully saved. This aligns with our goal of making the application more accessible to a global audience.
## How to Test
1. Change the application language to different supported languages.
2. Open the CodeEditor, make changes to a file, and save it.
3. Verify that the "File saved successfully" message appears in the correct language for both the toast and on-screen notifications.
## Additional Notes
Please pay special attention to the structure of the translation.json file to ensure it follows our established patterns for internationalization.
* Add toast notifications for error handling in fileService
# Add toast notifications for error handling in fileService
## Description
This PR enhances the error handling in the `fileService.ts` file by adding toast notifications for user feedback. It maintains the existing console error logging for debugging purposes while improving the user experience by providing visible error messages in the UI.
## Changes
- Added import for the toast utility
- Implemented toast.error() calls in catch blocks for all file operations
- Kept console.error() calls for detailed logging
- Updated error messages to be more user-friendly
## Files Changed
- `src/services/fileService.ts`
## Testing
- Tested all file operations (select, upload, list, save) to ensure proper error handling
- Verified that toast notifications appear when errors are simulated
- Confirmed that console errors are still logged for debugging
## Additional Notes
This change improves error visibility for users without altering the underlying error handling logic. It should make troubleshooting easier for both users and developers.
* Add file path safety check and improve error handling in file services
# Add file path safety check and improve error handling in file services
## Description
This PR enhances the `fileService.ts` by adding a safety check for file paths in the `saveFile` function and improves error handling across all file operations. It also includes new translations for various file-related error messages.
## Changes
1. Updated `src/services/fileService.ts`:
- Added a validation check for file paths in the saveFile function
- Improved error handling for all file operations (select, upload, list, save)
- Implemented toast error messages with translation support
2. Updated `src/i18n/translations.json`:
- Added new translation keys for file service error messages:
- FILE_SERVICE$SELECT_FILE_ERROR
- FILE_SERVICE$UPLOAD_FILES_ERROR
- FILE_SERVICE$LIST_FILES_ERROR
- FILE_SERVICE$SAVE_FILE_ERROR
- FILE_SERVICE$INVALID_FILE_PATH
## Files Changed
- `src/services/fileService.ts`
- `src/i18n/translations.json`
## Key Implementation Details
```typescript
export async function saveFile(filePath: string, content: string): Promise<void> {
const { t } = useTranslation();
if (!filePath || filePath.includes('..')) {
toast.error(t(I18nKey.FILE_SERVICE$INVALID_FILE_PATH));
throw new Error('Invalid file path');
}
try {
// Existing implementation...
} catch (error) {
console.error('Error saving file:', error);
toast.error(t(I18nKey.FILE_SERVICE$SAVE_FILE_ERROR), 'File Save Error');
throw error;
}
}
```
## Testing
- Verified that the saveFile function rejects invalid file paths (empty or containing '..')
- Confirmed that appropriate error messages are displayed using toast notifications for all file operations
- Tested with different languages to ensure translated messages appear correctly
## Security Implications
The file path check in saveFile enhances security by preventing potential directory traversal attacks.
## Next Steps
- Consider adding similar safety checks to other file operations if applicable
- Ensure thorough testing of error scenarios across all supported languages
* Add docstrings to listen.py
# Add docstrings to listen.py
## Description
This PR adds comprehensive docstrings to all functions in the `listen.py` file. These additions improve code documentation, making the file more readable and maintainable for current and future developers.
## Changes
- Added docstrings to all functions in `listen.py`
- Docstrings follow the Google Python Style Guide format
- Included descriptions, parameters, return values, and potential exceptions for each function
## Files Changed
- `src/listen.py`
## Docstring Example
Here's an example of one of the added docstrings:
```python
@app.post('/api/save-file')
async def save_file(request: Request):
"""
Save a file to the agent's runtime file store.
This endpoint allows saving a file when the agent is in a paused, finished,
or awaiting user input state. It checks the agent's state before proceeding
with the file save operation.
Args:
request (Request): The incoming FastAPI request object.
Returns:
JSONResponse: A JSON response indicating the success of the operation.
Raises:
HTTPException:
- 403 error if the agent is not in an allowed state for editing.
- 400 error if the file path or content is missing.
- 500 error if there's an unexpected error during the save operation.
"""
# Function implementation...
```
## Impact
- Improved code readability and maintainability
- Better understanding of function purposes, inputs, outputs, and potential errors
- Easier onboarding for new developers working on this file
- Enhanced IDE support for function descriptions and parameter information
## Testing
- No functional changes were made, so existing tests should pass without modification
- Manual review of docstrings for accuracy and completeness is recommended
## Next Steps
- Consider adding similar docstrings to other files in the project for consistency
- Review the added docstrings to ensure they accurately describe the current functionality
- Update docstrings as needed when function implementations change in the future
## Additional Notes
The existing code structure and functionality remain unchanged. This PR focuses solely on improving documentation through the addition of docstrings.
* Revert exclude_list formatting and add docstrings in listen.py
# Revert exclude_list formatting and add docstrings in listen.py
## Description
This PR makes two main changes to the `listen.py` file:
1. Reverts the `exclude_list` in the `list_files` function to its original format, with each item on a separate line.
2. Adds comprehensive docstrings to all functions in the file.
These changes improve code readability, maintain consistency with project standards, and enhance documentation for better maintainability.
## Changes
1. Updated `opendevin/server/listen.py`:
- Reverted `exclude_list` formatting in `list_files` function
- Added docstrings to all functions
## Detailed Changes
### 1. Reverted exclude_list formatting
```python
exclude_list = (
'.git',
'.DS_Store',
'.svn',
'.hg',
'.idea',
'.vscode',
'.settings',
'.pytest_cache',
'__pycache__',
'node_modules',
'vendor',
'build',
'dist',
'bin',
'logs',
'log',
'tmp',
'temp',
'coverage',
'venv',
'env',
)
```
### 2. Added docstrings (example)
```python
@app.get('/api/list-files')
def list_files(request: Request, path: str = '/'):
"""
List files in the specified path.
This function retrieves a list of files from the agent's runtime file store,
excluding certain system and hidden files/directories.
Args:
request (Request): The incoming request object.
path (str, optional): The path to list files from. Defaults to '/'.
Returns:
list: A list of file names in the specified path.
Raises:
HTTPException: If there's an error listing the files.
"""
# Function implementation...
```
## Rationale
- Reverting `exclude_list` formatting maintains consistency with the project's coding style and ensures proper functioning of pre-commit hooks.
- Adding docstrings improves code documentation, making it easier for developers to understand and maintain the codebase.
## Impact
- Improved code readability and consistency
- Enhanced documentation for all functions in `listen.py`
- Easier onboarding for new developers
- Better IDE support for function descriptions and parameter information
## Testing
- No functional changes were made, so existing tests should pass without modification
- Manual review of the reverted `exclude_list` and new docstrings is recommended
## Additional Notes
- The existing code functionality remains unchanged
- All functions in `listen.py` now have detailed docstrings following the Google Python Style Guide format
## Next Steps
- Review the added docstrings to ensure they accurately describe the current functionality
- Consider adding similar docstrings to other files in the project for consistency
- Update docstrings as needed when function implementations change in the future
* made code reviewable
* fixed ruff issues
* Update listen.py docstrings
* final tweaks
* re-added encodedURIComponent in selectFile
---------
Co-authored-by: tobitege <tobitege@gmx.de>
Co-authored-by: sp.wack <83104063+amanape@users.noreply.github.com>
* feat: lazy launching browser; browser optional for diffrent agents.
* style: lint
* fix: integration test fail due to browser not started.
* fix: run by cli and integration test failed.
* fix: lint
* fix: lint
---------
Co-authored-by: Graham Neubig <neubig@gmail.com>
* Fix: Feedback should be sent through the backend to avoid CORS issues
* Update
* Fix merge error
* Revert unnecessary change
* Lint
* Moved to services
* Fixed bugs
---------
Co-authored-by: OpenDevin <opendevin@opendevin.ai>
* Refactor monologue to use the messages in state history
* add messages, clean up
* fix monologue
* update integration tests
* move private method
* update SWE agent to use the history from State
* integration tests for SWE agent
* rename monologue to initial_thoughts, since that is what it is
* modify api endpoint
* update frontend for backend
* fix fileservice
* rm file
* unskip test
* fix some more tests
* fix another test
* fix another test
* fix api call
* fix refresh for subdirs
* more tests passing
* more tests
* more tests
* another test
* logspam
* lint
* fix import
* logspam
* code review feedback
---------
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
* Feat: add stream output to exec_run
* Using command timeout to control the exec_box's timeout.
* add bash -c to source command to compatible for sh.
Signed-off-by: ifuryst <ifuryst@gmail.com>
* Feat: add stream output to SSHBox execute
Signed-off-by: ifuryst <ifuryst@gmail.com>
* fix the test case fail.
Signed-off-by: ifuryst <ifuryst@gmail.com>
* fix the test case import wrong path for method.
Signed-off-by: ifuryst <ifuryst@gmail.com>
---------
Signed-off-by: ifuryst <ifuryst@gmail.com>
* Update msg_stack.py
1、[msg.to_dict() for msg in msgs], msg is not instanse of Message, it not has a func of to_dict(), so msg.to_dict() will accur JSONDecodeError;
2、json.dump(new_data, file), it appends new_data to the end of the file instead of overwriting from the beginning, Hence, it's necessary to first perform file.seek(0) and file.truncate().
* Update opendevin/server/session/msg_stack.py
---------
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* create test todos
* extend to support uploading directories
* remove dir-upload logic and feature drag-and-drop
---------
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
* fix plugin use in jupyter
* fix jupyter plugin potential port conflict
* update integration test
* wait a bit for jupyter execution
* add one unit tests for sandbox
* fix integration test
* fix integration
* fix integration yet again
* init sandbox plugins in the server
