Compare commits

..

44 Commits

Author SHA1 Message Date
openhands 489e32c2c0 Fix email update to use /api/settings endpoint 2025-05-31 19:10:03 +00:00
openhands c189012f0a Fix email update to use query parameter instead of form data 2025-05-31 19:05:02 +00:00
openhands 2407420e17 Make email field editable and add save button in user settings 2025-05-31 18:59:14 +00:00
chuckbutkus bb0c47c41a Merge branch 'main' into display-email 2025-05-31 01:14:24 -04:00
Chuck Butkus 83e5276de5 Update User Setting tab 2025-05-31 01:13:38 -04:00
openhands 816082a55b Update User tab to display email from settings instead of git user 2025-05-31 04:12:04 +00:00
Chuck Butkus 82d72b145d Add email to Setting class 2025-05-30 23:59:47 -04:00
Xingyao Wang 277b87413b chore: log stack trace when failed to connect in mcp (#8804) 2025-05-30 18:35:19 -04:00
Rohit Malhotra 972837ff73 [Feat]: Gitlab resolver (#8770) 2025-05-30 22:18:34 +00:00
Chuck Butkus f8c3470c91 Add get_user_email from UserAuth 2025-05-30 15:43:08 -04:00
tofarr de196d6da1 Allowing local runtimes to have domains (#8798) 2025-05-29 15:15:27 -06:00
Rohit Malhotra 10a4ab9199 [Fix]: inconsistent microagent descriptions (#8800) 2025-05-29 20:09:01 +00:00
Robert Brennan 3f0d8e7f47 add more logging to debug runtime restarts (#8799) 2025-05-29 19:52:13 +00:00
Engel Nyst 0e0687efc3 Rename service (#8791) 2025-05-29 19:43:42 +00:00
tofarr 30d53f8df0 Fix openapi authorize (#8794) 2025-05-29 13:20:00 -06:00
sp.wack fd52841842 Add git_provider and selected_branch to conversation response (#8795)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-29 22:59:30 +04:00
Graham Neubig d770edc010 Add images and example commands to cloud issue resolver documentation (#8785)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-29 18:13:49 +00:00
Engel Nyst df2374bd85 Refactor agent controller's filter (#8792) 2025-05-29 19:59:34 +02:00
sp.wack 542be7338e fix(frontend): Replace actions with their observations pairs in the chat interface (#8760)
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2025-05-29 19:22:10 +02:00
Rohit Malhotra e4d84946db (Chore): deprecate alias (#8704) 2025-05-29 11:01:28 -04:00
tofarr cc881a6bcb Added Webhooks to event store (#8763)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-29 07:39:08 -06:00
Howie Zhou cb0a1c91e4 docs: change to existing testing files (#8779) 2025-05-29 12:23:24 +00:00
Graham Neubig e208bffade Fix localization issue with hardcoded English strings (#8736)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-29 00:40:57 +02:00
Graham Neubig 6491142364 Fix KeyError on router error logging (#8769) 2025-05-28 19:59:18 +00:00
Robert Brennan 205f0234e8 Rename Conversation to ServerConversation and AppConfig to OpenHandsConfig (#8754)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-28 21:48:34 +02:00
Robert Brennan c76809a766 Revert "Add username parameter to AsyncBashSession" (#8767) 2025-05-28 14:28:26 -04:00
chuckbutkus 9f86f731a7 Update login (#8743)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-28 17:53:35 +00:00
sp.wack 6fe5da810b fix(frontend): Handle assistant messages at the top (#8766) 2025-05-28 17:33:05 +00:00
dependabot[bot] 52a1e94335 chore(deps): bump the docusaurus group in /docs with 7 updates (#8758)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-28 21:24:54 +04:00
sp.wack 3e0532e8b9 fix(frontend): Only clear UI messages on cid change (#8762) 2025-05-28 15:31:34 +00:00
tofarr 90c440d709 Add HTTP FileStore implementation (#8751)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-28 08:17:26 -06:00
Robert Brennan 82657b7ba1 Add username parameter to AsyncBashSession (#8746)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-28 10:16:45 -04:00
Engel Nyst 3c51600260 Add vscode rules/ignores to .gitignore (#8755) 2025-05-28 15:42:11 +02:00
sp.wack b5f2a04ea2 Add refill link to out-of-credits error message (#8737)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-28 17:23:14 +04:00
sp.wack 155615bbb1 hotfix(frontend): Invalidate and refetch git changes if messages aren't being received (#8752) 2025-05-28 13:22:15 +00:00
Kent Johnson 4b6f2aeb4d docs: Mention dev container in Development.md (#8726) 2025-05-27 18:29:05 -04:00
Rohit Malhotra 0023eb0982 (Hotfix): Handle cases where user secrets store doesn't exist (#8745)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-27 18:26:36 -04:00
Robert Brennan c3ab4b480b Fix TypeError in list_files endpoint while preserving router_error_log functionality (#8744)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-27 18:25:07 -04:00
Xingyao Wang 35f7efb9d7 Fix: Remove strip() from parameter value extraction to preserve indentation (#8739)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-27 20:24:00 +00:00
Xuhui Zhou 14498c5e25 Feature/swe run interact (#8714)
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
2025-05-27 19:35:21 +00:00
sp.wack cdb9aeb9ba fix(frontend): Don't show terminal commands in chat interface that are from the user (#8729) 2025-05-27 18:59:32 +00:00
Robert Brennan 318883e5e0 Fix VS Code tab and other runtime-dependent features showing null (#8734)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-27 18:41:31 +00:00
Rohit Malhotra 767b6ce600 [Refactor]: separate args setup logic for restarting conversations (#8679)
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2025-05-27 13:16:33 -04:00
Xingyao Wang 3ccc96d794 Fix(docs): volumes configuration under [sandbox] in config.toml (#8724)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-28 00:30:07 +08:00
223 changed files with 4422 additions and 2791 deletions
+14 -1
View File
@@ -161,7 +161,6 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
.idea/
.cursorignore
# VS Code: Ignore all but certain files that specify repo-specific settings.
# https://stackoverflow.com/questions/32964920/should-i-commit-the-vscode-folder-to-source-control
@@ -171,6 +170,20 @@ cython_debug/
!.vscode/settings.json
!.vscode/tasks.json
# VS Code extensions/forks:
.cursorignore
.rooignore
.clineignore
.windsurfignore
.cursorrules
.roorules
.clinerules
.windsurfrules
.cursor/rules
.roo/rules
.cline/rules
.windsurf/rules
# evaluation
evaluation/evaluation_outputs
evaluation/outputs
+23 -10
View File
@@ -1,8 +1,10 @@
# Development Guide
This guide is for people working on OpenHands and editing the source code.
If you wish to contribute your changes, check out the [CONTRIBUTING.md](https://github.com/All-Hands-AI/OpenHands/blob/main/CONTRIBUTING.md) on how to clone and setup the project
initially before moving on. Otherwise, you can clone the OpenHands project directly.
If you wish to contribute your changes, check out the
[CONTRIBUTING.md](https://github.com/All-Hands-AI/OpenHands/blob/main/CONTRIBUTING.md)
on how to clone and setup the project initially before moving on. Otherwise,
you can clone the OpenHands project directly.
## Start the Server for Development
@@ -19,9 +21,20 @@ initially before moving on. Otherwise, you can clone the OpenHands project direc
Make sure you have all these dependencies installed before moving on to `make build`.
#### Dev container
There is a [dev container](https://containers.dev/) available which provides a
pre-configured environment with all the necessary dependencies installed if you
are using a [supported editor or tool](https://containers.dev/supporting). For
example, if you are using Visual Studio Code (VS Code) with the
[Dev Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
extension installed, you can open the project in a dev container by using the
_Dev Container: Reopen in Container_ command from the Command Palette
(Ctrl+Shift+P).
#### Develop without sudo access
If you want to develop without system admin/sudo access to upgrade/install `Python` and/or `NodeJs`, you can use
If you want to develop without system admin/sudo access to upgrade/install `Python` and/or `NodeJs`, you can use
`conda` or `mamba` to manage the packages for you:
```bash
@@ -37,7 +50,7 @@ mamba install conda-forge::poetry
### 2. Build and Setup The Environment
Begin by building the project which includes setting up the environment and installing dependencies. This step ensures
Begin by building the project which includes setting up the environment and installing dependencies. This step ensures
that OpenHands is ready to run on your system:
```bash
@@ -54,11 +67,11 @@ To configure the LM of your choice, run:
make setup-config
```
This command will prompt you to enter the LLM API key, model name, and other variables ensuring that OpenHands is
tailored to your specific needs. Note that the model name will apply only when you run headless. If you use the UI,
This command will prompt you to enter the LLM API key, model name, and other variables ensuring that OpenHands is
tailored to your specific needs. Note that the model name will apply only when you run headless. If you use the UI,
please set the model in the UI.
Note: If you have previously run OpenHands using the docker command, you may have already set some environmental
Note: If you have previously run OpenHands using the docker command, you may have already set some environmental
variables in your terminal. The final configurations are set from highest to lowest priority:
Environment variables > config.toml variables > default variables
@@ -77,14 +90,14 @@ make run
#### Option B: Individual Server Startup
- **Start the Backend Server:** If you prefer, you can start the backend server independently to focus on
- **Start the Backend Server:** If you prefer, you can start the backend server independently to focus on
backend-related tasks or configurations.
```bash
make start-backend
```
- **Start the Frontend Server:** Similarly, you can start the frontend server on its own to work on frontend-related
- **Start the Frontend Server:** Similarly, you can start the frontend server on its own to work on frontend-related
components or interface enhancements.
```bash
make start-frontend
@@ -120,7 +133,7 @@ poetry run pytest ./tests/unit/test_*.py
### 9. Use existing Docker image
To reduce build time (e.g., if no changes were made to the client-runtime component), you can use an existing Docker
To reduce build time (e.g., if no changes were made to the client-runtime component), you can use an existing Docker
container image by setting the SANDBOX_RUNTIME_CONTAINER_IMAGE environment variable to the desired Docker image.
Example: `export SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/all-hands-ai/runtime:0.39-nikolaik`
+9
View File
@@ -325,6 +325,15 @@ classpath = "my_package.my_module.MyCustomAgent"
# Useful when deploying OpenHands in a remote machine where you need to expose a specific port.
#vscode_port = 41234
# Volume mounts in the format 'host_path:container_path[:mode]'
# e.g. '/my/host/dir:/workspace:rw'
# Multiple mounts can be specified using commas
# e.g. '/path1:/workspace/path1,/path2:/workspace/path2:ro'
# Configure volumes under the [sandbox] section:
# [sandbox]
# volumes = "/my/host/dir:/workspace:rw,/path2:/workspace/path2:ro"
#################################### Security ###################################
# Configuration for security features
##############################################################################
@@ -8,18 +8,30 @@ Le Résolveur de Problèmes Cloud est disponible automatiquement lorsque vous ac
- [Accès au dépôt GitHub](./github-installation#adding-repository-access)
- [Accès au dépôt GitLab](./gitlab-installation#adding-repository-access)
![Ajout d'accès au dépôt à OpenHands](/img/cloud/add-repo.png)
## Utilisation
Après avoir accordé l'accès au dépôt OpenHands Cloud, vous pouvez utiliser le Résolveur de Problèmes Cloud sur les problèmes et les pull/merge requests dans vos dépôts.
### Travailler avec les Problèmes
Sur votre dépôt, étiquetez un problème avec `openhands`. OpenHands va :
Sur votre dépôt, étiquetez un problème avec `openhands` ou ajoutez un message commençant par `@openhands`. OpenHands va :
1. Commenter le problème pour vous faire savoir qu'il y travaille
- Vous pouvez cliquer sur le lien pour suivre la progression sur OpenHands Cloud
2. Ouvrir une pull request (GitHub) ou une merge request (GitLab) s'il détermine que le problème a été résolu avec succès
3. Commenter le problème avec un résumé des tâches effectuées et un lien vers la PR/MR
![Résolveur de problèmes OpenHands en action](/img/cloud/issue-resolver.png)
#### Exemples de Commandes pour les Problèmes
Voici quelques exemples de commandes que vous pouvez utiliser avec le résolveur de problèmes :
```
@openhands lisez la description du problème et corrigez-le
```
### Travailler avec les Pull/Merge Requests
Pour qu'OpenHands travaille sur les pull requests (GitHub) ou les merge requests (GitLab), mentionnez `@openhands` dans les commentaires pour :
@@ -30,3 +42,15 @@ Pour qu'OpenHands travaille sur les pull requests (GitHub) ou les merge requests
OpenHands va :
1. Commenter pour vous faire savoir qu'il y travaille
2. Effectuer la tâche demandée
#### Exemples de Commandes pour les Pull/Merge Requests
Voici quelques exemples de commandes que vous pouvez utiliser avec les pull/merge requests :
```
@openhands reflétez les commentaires de la revue
```
```
@openhands corrigez les conflits de fusion et assurez-vous que le CI passe
```
@@ -36,6 +36,8 @@ Vous pouvez accorder à OpenHands l'accès à des dépôts spécifiques :
- Permissions GitHub de l'utilisateur (propriétaire/collaborateur)
3. Cliquez sur `Installer & Autoriser`
![Ajout de l'accès au dépôt à OpenHands](/img/cloud/add-repo.png)
## Modification de l'Accès au Dépôt
Vous pouvez modifier l'accès au dépôt à tout moment :
@@ -8,18 +8,30 @@
- [GitHubリポジトリアクセス](./github-installation#adding-repository-access)
- [GitLabリポジトリアクセス](./gitlab-installation#adding-repository-access)
![OpenHandsにリポジトリアクセスを追加する](/img/cloud/add-repo.png)
## 使用方法
OpenHands Cloudリポジトリアクセスを許可した後、リポジトリの課題やプルリクエスト/マージリクエストでクラウド課題リゾルバーを使用できます。
### 課題の操作
リポジトリで、課題に`openhands`というラベルを付けます。OpenHandsは以下を行います:
リポジトリで、課題に`openhands`というラベルを付けるか、`@openhands`で始まるメッセージを追加します。OpenHandsは以下を行います:
1. 課題にコメントして、作業中であることを知らせます
- リンクをクリックすると、OpenHands Cloudで進捗状況を追跡できます
2. 課題が正常に解決されたと判断した場合、プルリクエスト(GitHub)またはマージリクエスト(GitLab)を開きます
3. 実行されたタスクの概要とPR/MRへのリンクを含むコメントを課題に残します
![OpenHands課題リゾルバーの動作](/img/cloud/issue-resolver.png)
#### 課題用のコマンド例
以下は、課題リゾルバーで使用できるコマンドの例です:
```
@openhands 課題の説明を読んで修正してください
```
### プルリクエスト/マージリクエストの操作
プルリクエスト(GitHub)またはマージリクエスト(GitLab)でOpenHandsを動作させるには、コメントで`@openhands`を言及して以下を行います:
@@ -30,3 +42,15 @@ OpenHands Cloudリポジトリアクセスを許可した後、リポジトリ
OpenHandsは以下を行います:
1. 作業中であることを知らせるコメントをします
2. リクエストされたタスクを実行します
#### プルリクエスト/マージリクエスト用のコマンド例
以下は、プルリクエスト/マージリクエストで使用できるコマンドの例です:
```
@openhands レビューコメントを反映してください
```
```
@openhands マージの競合を修正し、CIが通過することを確認してください
```
@@ -36,6 +36,8 @@
- ユーザーのGitHub権限(所有者/コラボレーター)
3. `インストール&承認`をクリックします
![OpenHandsへのリポジトリアクセスの追加](/img/cloud/add-repo.png)
## リポジトリアクセスの変更
リポジトリアクセスはいつでも変更できます:
@@ -8,18 +8,30 @@ O Resolvedor de Problemas na Nuvem está disponível automaticamente quando voc
- [Acesso ao repositório GitHub](./github-installation#adding-repository-access)
- [Acesso ao repositório GitLab](./gitlab-installation#adding-repository-access)
![Adicionando acesso ao repositório ao OpenHands](/img/cloud/add-repo.png)
## Uso
Após conceder acesso ao repositório OpenHands Cloud, você pode usar o Resolvedor de Problemas na Nuvem em issues e pull/merge requests em seus repositórios.
### Trabalhando com Issues
No seu repositório, rotule uma issue com `openhands`. O OpenHands irá:
No seu repositório, rotule uma issue com `openhands` ou adicione uma mensagem começando com `@openhands`. O OpenHands irá:
1. Comentar na issue para informar que está trabalhando nela
- Você pode clicar no link para acompanhar o progresso no OpenHands Cloud
2. Abrir um pull request (GitHub) ou merge request (GitLab) se determinar que o problema foi resolvido com sucesso
3. Comentar na issue com um resumo das tarefas realizadas e um link para o PR/MR
![OpenHands resolvedor de problemas em ação](/img/cloud/issue-resolver.png)
#### Exemplos de Comandos para Issues
Aqui estão alguns exemplos de comandos que você pode usar com o resolvedor de problemas:
```
@openhands leia a descrição do problema e corrija-o
```
### Trabalhando com Pull/Merge Requests
Para fazer o OpenHands trabalhar em pull requests (GitHub) ou merge requests (GitLab), mencione `@openhands` nos comentários para:
@@ -30,3 +42,15 @@ Para fazer o OpenHands trabalhar em pull requests (GitHub) ou merge requests (Gi
O OpenHands irá:
1. Comentar para informar que está trabalhando nisso
2. Realizar a tarefa solicitada
#### Exemplos de Comandos para Pull/Merge Requests
Aqui estão alguns exemplos de comandos que você pode usar com pull/merge requests:
```
@openhands reflita os comentários da revisão
```
```
@openhands corrija os conflitos de merge e certifique-se de que o CI passa
```
@@ -36,6 +36,8 @@ Você pode conceder ao OpenHands acesso a repositórios específicos:
- Permissões do GitHub do usuário (proprietário/colaborador)
3. Clique em `Instalar e Autorizar`
![Adicionando acesso ao repositório ao OpenHands](/img/cloud/add-repo.png)
## Modificando o Acesso ao Repositório
Você pode modificar o acesso ao repositório a qualquer momento:
@@ -8,18 +8,30 @@
- [GitHub仓库访问](./github-installation#adding-repository-access)
- [GitLab仓库访问](./gitlab-installation#adding-repository-access)
![向OpenHands添加仓库访问权限](/img/cloud/add-repo.png)
## 使用方法
授予OpenHands Cloud仓库访问权限后,您可以在仓库的问题和拉取/合并请求中使用云问题解决器。
### 处理问题
在您的仓库中,给问题添加`openhands`标签。OpenHands将会:
在您的仓库中,给问题添加`openhands`标签或添加以`@openhands`开头的消息。OpenHands将会:
1. 在问题上发表评论,让您知道它正在处理
- 您可以点击链接在OpenHands Cloud上跟踪进度
2. 如果确定问题已成功解决,则打开拉取请求(GitHub)或合并请求(GitLab
3. 在问题上发表评论,总结已执行的任务并提供PR/MR的链接
![OpenHands问题解决器运行中](/img/cloud/issue-resolver.png)
#### 问题命令示例
以下是您可以在问题解决器中使用的命令示例:
```
@openhands 阅读问题描述并修复它
```
### 处理拉取/合并请求
要让OpenHands处理拉取请求(GitHub)或合并请求(GitLab),在评论中提及`@openhands`以:
@@ -30,3 +42,15 @@
OpenHands将会:
1. 发表评论让您知道它正在处理
2. 执行请求的任务
#### 拉取/合并请求命令示例
以下是您可以在拉取/合并请求中使用的命令示例:
```
@openhands 反映审查评论
```
```
@openhands 修复合并冲突并确保CI通过
```
@@ -36,6 +36,8 @@
- 用户的GitHub权限(所有者/协作者)
3. 点击`安装并授权`
![向OpenHands添加仓库访问权限](/img/cloud/add-repo.png)
## 修改仓库访问权限
您可以随时修改仓库访问权限:
@@ -7,6 +7,8 @@ The Cloud Issue Resolver automates code fixes and provides intelligent assistanc
The Cloud Issue Resolver is available automatically when you grant OpenHands Cloud repository access:
- [GitHub repository access](./github-installation#adding-repository-access)
![Adding repository access to OpenHands](/img/cloud/add-repo.png)
## Usage
After granting OpenHands Cloud repository access, you can use the Cloud Issue Resolver on issues and pull requests in your repositories.
@@ -20,6 +22,16 @@ On your repository, label an issue with `openhands` or add a message starting wi
2. Open a pull request if it determines that the issue has been successfully resolved
3. Comment on the issue with a summary of the performed tasks and a link to the PR
![OpenHands issue resolver in action](/img/cloud/issue-resolver.png)
#### Example Commands for Issues
Here are some examples of commands you can use with the issue resolver:
```
@openhands read the issue description and fix it
```
### Working with Pull Requests
To get OpenHands to work on pull requests, mention `@openhands` in comments to:
@@ -30,3 +42,15 @@ To get OpenHands to work on pull requests, mention `@openhands` in comments to:
OpenHands will:
1. Comment to let you know it is working on it
2. Perform the requested task
#### Example Commands for Pull Requests
Here are some examples of commands you can use with pull requests:
```
@openhands reflect the review comments
```
```
@openhands fix the merge conflicts and make sure that CI passes
```
@@ -36,6 +36,8 @@ You can grant OpenHands access to specific repositories:
- User's GitHub permissions (owner/collaborator)
3. Click `Install & Authorize`
![Adding repository access to OpenHands](/img/cloud/add-repo.png)
## Modifying Repository Access
You can modify repository access at any time by visiting the Settings page and selecting `Configure GitHub Repositories` under the `Git` tab.
+13 -1
View File
@@ -66,9 +66,19 @@ The core configuration options are defined in the `[core]` section of the `confi
- `file_store`
- Type: `str`
- Default: `"memory"`
- Default: `"local"`
- Description: File store type
- `file_store_web_hook_url`
- Type: `str`
- Default: `None`
- Description: Optional url for a webhook to invoke after file store writes / deletes
- `file_store_web_hook_headers`
- Type: `str`
- Default: `None`
- Description: HTTP Headers to include in web hook requests.
- `file_uploads_allowed_extensions`
- Type: `list of str`
- Default: `[".*"]`
@@ -331,6 +341,8 @@ The agent configuration options are defined in the `[agent]` and `[agent.<agent_
The sandbox configuration options are defined in the `[sandbox]` section of the `config.toml` file.
To use these with the docker command, pass in `-e SANDBOX_<option>`. Example: `-e SANDBOX_TIMEOUT`.
### Execution
+1687 -1907
View File
File diff suppressed because it is too large Load Diff
+5 -5
View File
@@ -17,10 +17,10 @@
},
"// Note": "The OpenAPI spec is stored in docs/static/openapi.json so it's accessible at /openapi.json in the deployed site",
"dependencies": {
"@docusaurus/core": "^3.7.0",
"@docusaurus/plugin-content-pages": "^3.7.0",
"@docusaurus/preset-classic": "^3.7.0",
"@docusaurus/theme-mermaid": "^3.7.0",
"@docusaurus/core": "^3.8.0",
"@docusaurus/plugin-content-pages": "^3.8.0",
"@docusaurus/preset-classic": "^3.8.0",
"@docusaurus/theme-mermaid": "^3.8.0",
"@mdx-js/react": "^3.1.0",
"@node-rs/jieba": "^2.0.1",
"clsx": "^2.0.0",
@@ -33,7 +33,7 @@
},
"devDependencies": {
"@docusaurus/module-type-aliases": "^3.5.1",
"@docusaurus/tsconfig": "^3.7.0",
"@docusaurus/tsconfig": "^3.8.0",
"@docusaurus/types": "^3.5.1",
"swagger-cli": "^4.0.4",
"swagger-ui-dist": "^5.22.0",
Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 74 KiB

+3 -3
View File
@@ -17,7 +17,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -59,10 +59,10 @@ AGENT_CLS_TO_INST_SUFFIX = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
@@ -25,7 +25,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -39,11 +39,11 @@ from openhands.utils.async_utils import call_async_from_sync
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-slim'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime=os.environ.get('RUNTIME', 'docker'),
@@ -24,7 +24,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
load_from_toml,
parse_arguments,
@@ -46,10 +46,10 @@ SKIP_NUM = (
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.11-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime=os.environ.get('RUNTIME', 'docker'),
+3 -3
View File
@@ -22,7 +22,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -55,12 +55,12 @@ FILE_EXT_MAP = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
BIOCODER_BENCH_CONTAINER_IMAGE = 'public.ecr.aws/i5g0m1f6/eval_biocoder:v1.0'
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = BIOCODER_BENCH_CONTAINER_IMAGE
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
+3 -3
View File
@@ -25,7 +25,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -70,11 +70,11 @@ AGENT_CLS_TO_INST_SUFFIX = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
@@ -18,7 +18,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -33,13 +33,13 @@ SUPPORTED_AGENT_CLS = {'CodeActAgent'}
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
assert metadata.max_iterations == 1, (
'max_iterations must be 1 for browsing delegation evaluation.'
)
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
+3 -3
View File
@@ -25,7 +25,7 @@ from evaluation.utils.shared import (
from openhands.controller.state.state import State
from openhands.core.config import (
AgentConfig,
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -101,7 +101,7 @@ def get_instance_docker_image(repo_name: str) -> str:
def get_config(
instance: pd.Series,
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
repo_name = instance['repo'].split('/')[1]
base_container_image = get_instance_docker_image(repo_name)
logger.info(
@@ -113,7 +113,7 @@ def get_config(
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = base_container_image
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
max_iterations=metadata.max_iterations,
@@ -25,7 +25,7 @@ from evaluation.utils.shared import (
from openhands.controller.state.state import State
from openhands.core.config import (
AgentConfig,
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -61,10 +61,10 @@ AGENT_CLS_TO_INST_SUFFIX = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
+3 -3
View File
@@ -21,7 +21,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -47,10 +47,10 @@ AGENT_CLS_TO_INST_SUFFIX = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
+3 -3
View File
@@ -19,7 +19,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -39,10 +39,10 @@ AGENT_CLS_TO_INST_SUFFIX = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
+3 -3
View File
@@ -37,7 +37,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -60,10 +60,10 @@ ACTION_FORMAT = """
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
@@ -30,7 +30,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -81,10 +81,10 @@ AGENT_CLS_TO_INST_SUFFIX = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
@@ -19,10 +19,10 @@ from evaluation.utils.shared import (
make_metadata,
)
from openhands.core.config import (
AppConfig,
LLMConfig,
OpenHandsConfig,
get_parser,
load_app_config,
load_openhands_config,
)
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime
@@ -34,10 +34,10 @@ from openhands.utils.async_utils import call_async_from_sync
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
@@ -53,7 +53,7 @@ def get_config(
return config
config = load_app_config()
config = load_openhands_config()
def load_bench_config():
@@ -29,10 +29,10 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
load_app_config,
load_openhands_config,
)
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime, run_controller
@@ -44,10 +44,10 @@ from openhands.utils.async_utils import call_async_from_sync
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
@@ -63,7 +63,7 @@ def get_config(
return config
config = load_app_config()
config = load_openhands_config()
def load_bench_config():
@@ -17,7 +17,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -44,14 +44,14 @@ AGENT_CLS_TO_INST_SUFFIX = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'xingyaoww/od-eval-logic-reasoning:v1.0'
sandbox_config.runtime_extra_deps = (
'$OH_INTERPRETER_PATH -m pip install scitools-pyke'
)
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
+3 -3
View File
@@ -21,7 +21,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -54,10 +54,10 @@ AGENT_CLS_TO_FAKE_USER_RESPONSE_FN = {
def get_config(
metadata: EvalMetadata,
env_id: str,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'xingyaoww/od-eval-miniwob:v1.0'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime=os.environ.get('RUNTIME', 'docker'),
+3 -3
View File
@@ -22,7 +22,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -102,14 +102,14 @@ def load_incontext_example(task_name: str, with_tool: bool = True):
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'xingyaoww/od-eval-mint:v1.0'
sandbox_config.runtime_extra_deps = (
f'$OH_INTERPRETER_PATH -m pip install {" ".join(MINT_DEPENDENCIES)}'
)
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
@@ -4,11 +4,11 @@ import pprint
import tqdm
from openhands.core.config import get_llm_config_arg, get_parser, load_app_config
from openhands.core.config import get_llm_config_arg, get_parser, load_openhands_config
from openhands.core.logger import openhands_logger as logger
from openhands.llm.llm import LLM
config = load_app_config()
config = load_openhands_config()
def extract_test_results(res_file_path: str) -> tuple[list[str], list[str]]:
+5 -5
View File
@@ -33,10 +33,10 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
load_app_config,
load_openhands_config,
)
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime, run_controller
@@ -45,7 +45,7 @@ from openhands.events.observation import CmdOutputObservation
from openhands.runtime.base import Runtime
from openhands.utils.async_utils import call_async_from_sync
config = load_app_config()
config = load_openhands_config()
AGENT_CLS_TO_FAKE_USER_RESPONSE_FN = {
'CodeActAgent': codeact_user_response,
@@ -76,10 +76,10 @@ ID2CONDA = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'public.ecr.aws/i5g0m1f6/ml-bench'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
@@ -28,8 +28,8 @@ from evaluation.utils.shared import (
run_evaluation,
)
from openhands.core.config import (
AppConfig,
LLMConfig,
OpenHandsConfig,
get_parser,
)
from openhands.core.logger import openhands_logger as logger
@@ -73,7 +73,7 @@ def process_git_patch(patch):
return patch
def get_config(metadata: EvalMetadata, instance: pd.Series) -> AppConfig:
def get_config(metadata: EvalMetadata, instance: pd.Series) -> OpenHandsConfig:
# We use a different instance image for the each instance of swe-bench eval
base_container_image = get_instance_docker_image(instance['instance_id'])
logger.info(
@@ -87,7 +87,7 @@ def get_config(metadata: EvalMetadata, instance: pd.Series) -> AppConfig:
dataset_name=metadata.dataset,
instance_id=instance['instance_id'],
)
config = AppConfig(
config = OpenHandsConfig(
run_as_openhands=False,
runtime=os.environ.get('RUNTIME', 'docker'),
sandbox=sandbox_config,
@@ -30,7 +30,7 @@ from evaluation.utils.shared import (
from openhands.controller.state.state import State
from openhands.core.config import (
AgentConfig,
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -314,7 +314,7 @@ def get_instance_docker_image(instance: pd.Series):
def get_config(
instance: pd.Series,
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
SWE_BENCH_CONTAINER_IMAGE = 'ghcr.io/opendevin/eval-swe-bench:full-v1.2.1'
if USE_INSTANCE_IMAGE:
# We use a different instance image for the each instance of swe-bench eval
@@ -340,7 +340,7 @@ def get_config(
instance_id=instance['instance_id'],
)
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
max_iterations=metadata.max_iterations,
@@ -20,7 +20,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -58,12 +58,12 @@ def format_task_dict(example, use_knowledge):
def get_config(
metadata: EvalMetadata,
instance_id: str,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = (
'docker.io/xingyaoww/openhands-eval-scienceagentbench'
)
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime=os.environ.get('RUNTIME', 'docker'),
@@ -2,6 +2,8 @@
This folder contains the evaluation harness that we built on top of the original [SWE-Bench benchmark](https://www.swebench.com/) ([paper](https://arxiv.org/abs/2310.06770)).
**UPDATE (5/26/2025): We now support running interactive SWE-Bench evaluation (see the paper [here](https://arxiv.org/abs/2502.13069))! For how to run it, checkout [this README](./SWE-Interact.md).**
**UPDATE (4/8/2025): We now support running SWT-Bench evaluation! For more details, checkout [the corresponding section](#SWT-Bench-Evaluation).**
**UPDATE (03/27/2025): We now support SWE-Bench multimodal evaluation! Simply use "princeton-nlp/SWE-bench_Multimodal" as the dataset name in the `run_infer.sh` script to evaluate on multimodal instances.**
@@ -0,0 +1,92 @@
# SWE-Interact Benchmark
This document explains how to use the [Interactive SWE-Bench](https://arxiv.org/abs/2502.13069) benchmark scripts for running and evaluating interactive software engineering tasks.
## Setting things up
After following the [README](./README.md) to set up the environment, you would need to additionally add LLM configurations for simulated human users. In the original [paper](https://arxiv.org/abs/2502.13069), we use gpt-4o as the simulated human user. You can add the following to your `config.toml` file:
```toml
[llm.fake_user]
model="litellm_proxy/gpt-4o-2024-08-06"
api_key="<your-api-key>"
temperature = 0.0
base_url = "https://llm-proxy.eval.all-hands.dev"
```
## Running the Benchmark
The main script for running the benchmark is `run_infer_interact.sh`. Here's how to use it:
```bash
bash ./evaluation/benchmarks/swe_bench/scripts/run_infer_interact.sh <model_config> <commit_hash> <agent> <eval_limit> <max_iter> <num_workers> <split>
```
### Parameters:
- `model_config`: Path to the LLM configuration file (e.g., `llm.claude-3-7-sonnet`)
- `commit_hash`: Git commit hash to use (e.g., `HEAD`)
- `agent`: The agent class to use (e.g., `CodeActAgent`)
- `eval_limit`: Number of examples to evaluate (e.g., `500`)
- `max_iter`: Maximum number of iterations per task (e.g., `100`)
- `num_workers`: Number of parallel workers (e.g., `1`)
- `split`: Dataset split to use (e.g., `test`)
### Example:
```bash
bash ./evaluation/benchmarks/swe_bench/scripts/run_infer_interact.sh llm.claude-3-7-sonnet HEAD CodeActAgent 500 100 1 test
```
### Additional Environment Variables:
You can customize the behavior using these environment variables:
- `RUN_WITH_BROWSING`: Enable/disable web browsing (default: false)
- `USE_HINT_TEXT`: Enable/disable hint text (default: false)
- `EVAL_CONDENSER`: Specify a condenser configuration
- `EXP_NAME`: Add a custom experiment name to the output
- `N_RUNS`: Number of runs to perform (default: 1)
- `SKIP_RUNS`: Comma-separated list of run numbers to skip
## Evaluating Results
After running the benchmark, you can evaluate the results using `eval_infer.sh`:
```bash
./evaluation/benchmarks/swe_bench/scripts/eval_infer.sh <output_file> <instance_id> <dataset> <split>
```
### Parameters:
- `output_file`: Path to the output JSONL file
- `instance_id`: The specific instance ID to evaluate
- `dataset`: Dataset name (e.g., `cmu-lti/interactive-swe`)
- `split`: Dataset split (e.g., `test`)
### Example:
```bash
./evaluation/benchmarks/swe_bench/scripts/eval_infer.sh evaluation/evaluation_outputs/outputs/cmu-lti__interactive-swe-test/CodeActAgent/claude-3-7-sonnet-20250219_maxiter_100_N_v0.39.0-no-hint-run_1/output.jsonl sphinx-doc__sphinx-8721 cmu-lti/interactive-swe test
```
## Output Structure
The benchmark outputs are stored in the `evaluation/evaluation_outputs/outputs/` directory with the following structure:
```
evaluation/evaluation_outputs/outputs/
└── cmu-lti__interactive-swe-{split}/
└── {agent}/
└── {model}-{date}_maxiter_{max_iter}_N_{version}-{options}-run_{run_number}/
└── output.jsonl
```
Where:
- `{split}` is the dataset split (e.g., test)
- `{agent}` is the agent class name
- `{model}` is the model name
- `{date}` is the run date
- `{max_iter}` is the maximum iterations
- `{version}` is the OpenHands version
- `{options}` includes any additional options (e.g., no-hint, with-browsing)
- `{run_number}` is the run number
@@ -24,8 +24,8 @@ from evaluation.utils.shared import (
run_evaluation,
)
from openhands.core.config import (
AppConfig,
LLMConfig,
OpenHandsConfig,
get_parser,
)
from openhands.core.logger import openhands_logger as logger
@@ -69,7 +69,7 @@ def process_git_patch(patch):
return patch
def get_config(metadata: EvalMetadata, instance: pd.Series) -> AppConfig:
def get_config(metadata: EvalMetadata, instance: pd.Series) -> OpenHandsConfig:
# We use a different instance image for the each instance of swe-bench eval
base_container_image = get_instance_docker_image(instance['instance_id'])
logger.info(
@@ -83,7 +83,7 @@ def get_config(metadata: EvalMetadata, instance: pd.Series) -> AppConfig:
dataset_name=metadata.dataset,
instance_id=instance['instance_id'],
)
config = AppConfig(
config = OpenHandsConfig(
run_as_openhands=False,
runtime=os.environ.get('RUNTIME', 'docker'),
sandbox=sandbox_config,
+11 -8
View File
@@ -40,12 +40,12 @@ from evaluation.utils.shared import (
from openhands.controller.state.state import State
from openhands.core.config import (
AgentConfig,
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
from openhands.core.config.utils import get_condenser_config_arg
from openhands.core.config.condenser_config import NoOpCondenserConfig
from openhands.core.config.utils import get_condenser_config_arg
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime, run_controller
from openhands.critic import AgentFinishedCritic
@@ -220,7 +220,7 @@ def get_instance_docker_image(
def get_config(
instance: pd.Series,
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
# We use a different instance image for the each instance of swe-bench eval
use_swebench_official_image = 'swe-gym' not in metadata.dataset.lower()
base_container_image = get_instance_docker_image(
@@ -244,7 +244,7 @@ def get_config(
instance_id=instance['instance_id'],
)
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
max_iterations=metadata.max_iterations,
@@ -721,15 +721,16 @@ def filter_dataset(dataset: pd.DataFrame, filter_column: str) -> pd.DataFrame:
# repos for the swe-bench instances:
# ['astropy/astropy', 'django/django', 'matplotlib/matplotlib', 'mwaskom/seaborn', 'pallets/flask', 'psf/requests', 'pydata/xarray', 'pylint-dev/pylint', 'pytest-dev/pytest', 'scikit-learn/scikit-learn', 'sphinx-doc/sphinx', 'sympy/sympy']
selected_repos = data['selected_repos']
if isinstance(selected_repos, str): selected_repos = [selected_repos]
if isinstance(selected_repos, str):
selected_repos = [selected_repos]
assert isinstance(selected_repos, list)
logger.info(
f'Filtering {selected_repos} tasks from "selected_repos"...'
)
subset = dataset[dataset["repo"].isin(selected_repos)]
subset = dataset[dataset['repo'].isin(selected_repos)]
logger.info(f'Retained {subset.shape[0]} tasks after filtering')
return subset
skip_ids = os.environ.get('SKIP_IDS', '').split(',')
if len(skip_ids) > 0:
logger.info(f'Filtering {len(skip_ids)} tasks from "SKIP_IDS"...')
@@ -806,7 +807,9 @@ if __name__ == '__main__':
else:
# If no specific condenser config is provided via env var, default to NoOpCondenser
condenser_config = NoOpCondenserConfig()
logger.debug('No Condenser config provided via EVAL_CONDENSER, using NoOpCondenser.')
logger.debug(
'No Condenser config provided via EVAL_CONDENSER, using NoOpCondenser.'
)
details = {'mode': args.mode}
_agent_cls = openhands.agenthub.Agent.get_cls(args.agent_cls)
+411
View File
@@ -0,0 +1,411 @@
import asyncio
import json
import os
import pandas as pd
from datasets import load_dataset
from litellm import completion as litellm_completion
import openhands.agenthub
from evaluation.benchmarks.swe_bench.run_infer import (
AgentFinishedCritic,
complete_runtime,
filter_dataset,
get_config,
initialize_runtime,
)
from evaluation.benchmarks.swe_bench.run_infer import (
get_instruction as base_get_instruction,
)
from evaluation.utils.shared import (
EvalException,
EvalMetadata,
EvalOutput,
make_metadata,
prepare_dataset,
reset_logger_for_multiprocessing,
run_evaluation,
)
from openhands.controller.state.state import State
from openhands.core.config import (
get_llm_config_arg,
get_parser,
)
from openhands.core.config.condenser_config import NoOpCondenserConfig
from openhands.core.config.utils import get_condenser_config_arg
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime, run_controller
from openhands.events.action import MessageAction
from openhands.events.serialization.event import event_from_dict, event_to_dict
from openhands.utils.async_utils import call_async_from_sync
USE_HINT_TEXT = os.environ.get('USE_HINT_TEXT', 'false').lower() == 'true'
USE_INSTANCE_IMAGE = os.environ.get('USE_INSTANCE_IMAGE', 'false').lower() == 'true'
RUN_WITH_BROWSING = os.environ.get('RUN_WITH_BROWSING', 'false').lower() == 'false'
class FakeUser:
def __init__(self, issue, hints, files):
self.system_message = f"""
You are a GitHub user reporting an issue. Here are the details of your issue and environment:
Issue: {issue}
Hints: {hints}
Files relative to your current directory: {files}
Your task is to respond to questions from a coder who is trying to solve your issue. The coder has a summarized version of the issue you have. Follow these rules:
1. If the coder asks a question that is directly related to the information in the issue you have, provide that information.
2. Always stay in character as a user reporting an issue, not as an AI assistant.
3. Keep your responses concise and to the point.
4. The coder has limited turns to solve the issue. Do not interact with the coder beyond 3 turns.
Respond with "I don't have that information" if the question is unrelated or you're unsure.
"""
self.chat_history = [{'role': 'system', 'content': self.system_message}]
self.turns = 0
# Get LLM config from config.toml
self.llm_config = get_llm_config_arg(
'llm.fake_user'
) # You can change 'fake_user' to any config name you want
def generate_reply(self, question):
if self.turns > 3:
return 'Please continue working on the task. Do NOT ask for more help.'
self.chat_history.append({'role': 'user', 'content': question.content})
response = litellm_completion(
model=self.llm_config.model,
messages=self.chat_history,
api_key=self.llm_config.api_key.get_secret_value(),
temperature=self.llm_config.temperature,
base_url=self.llm_config.base_url,
)
reply = response.choices[0].message.content
self.chat_history.append({'role': 'assistant', 'content': reply})
self.turns += 1
return reply
# Global variable for fake user
fake_user = None
def get_fake_user_response(state: State) -> str:
global fake_user
if not fake_user:
return 'Please continue working on the task.'
last_agent_message = state.get_last_agent_message()
if last_agent_message:
return fake_user.generate_reply(last_agent_message)
return 'Please continue working on the task.'
AGENT_CLS_TO_FAKE_USER_RESPONSE_FN = {
'CodeActAgent': get_fake_user_response,
}
def get_instruction(instance: pd.Series, metadata: EvalMetadata) -> MessageAction:
instance_copy = instance.copy()
instance_copy.problem_statement = f'{instance.problem_statement}\n\nHints:\nThe user has not provided all the necessary details about the issue, and there are some hidden details that are helpful. Please ask the user specific questions using non-code commands to gather the relevant information that the user has to help you solve the issue. Ensure you have all the details you require to solve the issue.'
return base_get_instruction(instance_copy, metadata)
def process_instance(
instance: pd.Series,
metadata: EvalMetadata,
reset_logger: bool = True,
) -> EvalOutput:
config = get_config(instance, metadata)
global fake_user
original_issue = instance.original_issue
issue = str(original_issue)
fake_user = FakeUser(issue=issue, hints=instance.hints_text, files=instance.files)
# Setup the logger properly, so you can run multi-processing to parallelize the evaluation
if reset_logger:
log_dir = os.path.join(metadata.eval_output_dir, 'infer_logs')
reset_logger_for_multiprocessing(logger, instance.instance_id, log_dir)
else:
logger.info(f'Starting evaluation for instance {instance.instance_id}.')
runtime = create_runtime(config)
call_async_from_sync(runtime.connect)
try:
initialize_runtime(runtime, instance, metadata)
message_action = get_instruction(instance, metadata)
# Here's how you can run the agent (similar to the `main` function) and get the final task state
state: State | None = asyncio.run(
run_controller(
config=config,
initial_user_action=message_action,
runtime=runtime,
fake_user_response_fn=AGENT_CLS_TO_FAKE_USER_RESPONSE_FN[
metadata.agent_class
],
)
)
# if fatal error, throw EvalError to trigger re-run
if (
state
and state.last_error
and 'fatal error during agent execution' in state.last_error
and 'stuck in a loop' not in state.last_error
):
raise EvalException('Fatal error detected: ' + state.last_error)
# Get git patch
return_val = complete_runtime(runtime, instance)
git_patch = return_val['git_patch']
logger.info(
f'Got git diff for instance {instance.instance_id}:\n--------\n{git_patch}\n--------'
)
finally:
runtime.close()
# Prepare test result
test_result = {
'git_patch': git_patch,
}
if state is None:
raise ValueError('State should not be None.')
histories = [event_to_dict(event) for event in state.history]
metrics = state.metrics.get() if state.metrics else None
# Save the output
instruction = message_action.content
if message_action.image_urls:
instruction += (
'\n\n<image_urls>' + '\n'.join(message_action.image_urls) + '</image_urls>'
)
output = EvalOutput(
instance_id=instance.instance_id,
instruction=instruction,
instance=instance.to_dict(),
test_result=test_result,
metadata=metadata,
history=histories,
metrics=metrics,
error=state.last_error if state and state.last_error else None,
)
return output
if __name__ == '__main__':
parser = get_parser()
parser.add_argument(
'--dataset',
type=str,
default='cmu-lti/interactive-swe',
help='dataset to evaluate on',
)
parser.add_argument(
'--split',
type=str,
default='test',
help='split to evaluate on',
)
args, _ = parser.parse_known_args()
# Load dataset from huggingface datasets
dataset = load_dataset(args.dataset, split=args.split)
swe_bench_tests = filter_dataset(dataset.to_pandas(), 'instance_id')
logger.info(
f'Loaded dataset {args.dataset} with split {args.split}: {len(swe_bench_tests)} tasks'
)
llm_config = None
if args.llm_config:
llm_config = get_llm_config_arg(args.llm_config)
llm_config.log_completions = True
# modify_params must be False for evaluation purpose, for reproducibility and accurancy of results
llm_config.modify_params = False
if llm_config is None:
raise ValueError(f'Could not find LLM config: --llm_config {args.llm_config}')
# Get condenser config from environment variable
condenser_name = os.environ.get('EVAL_CONDENSER')
if condenser_name:
condenser_config = get_condenser_config_arg(condenser_name)
if condenser_config is None:
raise ValueError(
f'Could not find Condenser config: EVAL_CONDENSER={condenser_name}'
)
else:
# If no specific condenser config is provided via env var, default to NoOpCondenser
condenser_config = NoOpCondenserConfig()
logger.debug(
'No Condenser config provided via EVAL_CONDENSER, using NoOpCondenser.'
)
details = {'mode': 'interact'}
_agent_cls = openhands.agenthub.Agent.get_cls(args.agent_cls)
dataset_descrption = (
args.dataset.replace('/', '__') + '-' + args.split.replace('/', '__')
)
metadata = make_metadata(
llm_config,
dataset_descrption,
args.agent_cls,
args.max_iterations,
args.eval_note,
args.eval_output_dir,
details=details,
condenser_config=condenser_config,
)
output_file = os.path.join(metadata.eval_output_dir, 'output.jsonl')
print(f'### OUTPUT FILE: {output_file} ###')
# Run evaluation in iterative mode:
# If a rollout fails to output AgentFinishAction, we will try again until it succeeds OR total 3 attempts have been made.
ITERATIVE_EVAL_MODE = (
os.environ.get('ITERATIVE_EVAL_MODE', 'false').lower() == 'true'
)
ITERATIVE_EVAL_MODE_MAX_ATTEMPTS = int(
os.environ.get('ITERATIVE_EVAL_MODE_MAX_ATTEMPTS', '3')
)
if not ITERATIVE_EVAL_MODE:
# load the dataset
instances = prepare_dataset(swe_bench_tests, output_file, args.eval_n_limit)
if len(instances) > 0 and not isinstance(
instances['PASS_TO_PASS'][instances['PASS_TO_PASS'].index[0]], str
):
for col in ['PASS_TO_PASS', 'FAIL_TO_PASS']:
instances[col] = instances[col].apply(lambda x: str(x))
run_evaluation(
instances,
metadata,
output_file,
args.eval_num_workers,
process_instance,
timeout_seconds=8
* 60
* 60, # 8 hour PER instance should be more than enough
max_retries=5,
)
else:
critic = AgentFinishedCritic()
def get_cur_output_file_path(attempt: int) -> str:
return (
f'{output_file.removesuffix(".jsonl")}.critic_attempt_{attempt}.jsonl'
)
eval_ids = None
for attempt in range(1, ITERATIVE_EVAL_MODE_MAX_ATTEMPTS + 1):
cur_output_file = get_cur_output_file_path(attempt)
logger.info(
f'Running evaluation with critic {critic.__class__.__name__} for attempt {attempt} of {ITERATIVE_EVAL_MODE_MAX_ATTEMPTS}.'
)
# For deterministic eval, we set temperature to 0.1 for (>1) attempt
# so hopefully we get slightly different results
if attempt > 1 and metadata.llm_config.temperature == 0:
logger.info(
f'Detected temperature is 0 for (>1) attempt {attempt}. Setting temperature to 0.1...'
)
metadata.llm_config.temperature = 0.1
# Load instances - at first attempt, we evaluate all instances
# On subsequent attempts, we only evaluate the instances that failed the previous attempt determined by critic
instances = prepare_dataset(
swe_bench_tests, cur_output_file, args.eval_n_limit, eval_ids=eval_ids
)
if len(instances) > 0 and not isinstance(
instances['PASS_TO_PASS'][instances['PASS_TO_PASS'].index[0]], str
):
for col in ['PASS_TO_PASS', 'FAIL_TO_PASS']:
instances[col] = instances[col].apply(lambda x: str(x))
# Run evaluation - but save them to cur_output_file
logger.info(
f'Evaluating {len(instances)} instances for attempt {attempt}...'
)
run_evaluation(
instances,
metadata,
cur_output_file,
args.eval_num_workers,
process_instance,
timeout_seconds=8
* 60
* 60, # 8 hour PER instance should be more than enough
max_retries=5,
)
# When eval is done, we update eval_ids to the instances that failed the current attempt
instances_failed = []
logger.info(
f'Use critic {critic.__class__.__name__} to check {len(instances)} instances for attempt {attempt}...'
)
with open(cur_output_file, 'r') as f:
for line in f:
instance = json.loads(line)
try:
history = [
event_from_dict(event) for event in instance['history']
]
critic_result = critic.evaluate(
history, instance['test_result'].get('git_patch', '')
)
if not critic_result.success:
instances_failed.append(instance['instance_id'])
except Exception as e:
logger.error(
f'Error loading history for instance {instance["instance_id"]}: {e}'
)
instances_failed.append(instance['instance_id'])
logger.info(
f'{len(instances_failed)} instances failed the current attempt {attempt}: {instances_failed}'
)
eval_ids = instances_failed
# If no instances failed, we break
if len(instances_failed) == 0:
break
# Then we should aggregate the results from all attempts into the original output file
# and remove the intermediate files
logger.info(
'Aggregating results from all attempts into the original output file...'
)
fout = open(output_file, 'w')
added_instance_ids = set()
for attempt in reversed(range(1, ITERATIVE_EVAL_MODE_MAX_ATTEMPTS + 1)):
cur_output_file = get_cur_output_file_path(attempt)
if not os.path.exists(cur_output_file):
logger.warning(
f'Intermediate output file {cur_output_file} does not exist. Skipping...'
)
continue
with open(cur_output_file, 'r') as f:
for line in f:
instance = json.loads(line)
# Also make sure git_patch is not empty - otherwise we fall back to previous attempt (empty patch is worse than anything else)
if (
instance['instance_id'] not in added_instance_ids
and instance['test_result'].get('git_patch', '').strip()
):
fout.write(line)
added_instance_ids.add(instance['instance_id'])
logger.info(
f'Aggregated instances from {cur_output_file}. Total instances added so far: {len(added_instance_ids)}'
)
fout.close()
logger.info(
f'Done! Total {len(added_instance_ids)} instances added to {output_file}'
)
@@ -30,7 +30,7 @@ from evaluation.utils.shared import (
from openhands.controller.state.state import State
from openhands.core.config import (
AgentConfig,
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -58,7 +58,7 @@ def _get_swebench_workspace_dir_name(instance: pd.Series) -> str:
def get_instruction(instance: pd.Series, metadata: EvalMetadata):
workspace_dir_name = _get_swebench_workspace_dir_name(instance)
_get_swebench_workspace_dir_name(instance)
instruction = f"""
Consider the following issue description:
@@ -168,7 +168,7 @@ def get_instance_docker_image(instance_id: str, official_image: bool = False) ->
def get_config(
instance: pd.Series,
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
# We use a different instance image for the each instance of swe-bench eval
use_official_image = bool(
'verified' in metadata.dataset.lower() or 'lite' in metadata.dataset.lower()
@@ -197,7 +197,7 @@ def get_config(
'REPO_PATH': f'/workspace/{workspace_dir_name}/',
}
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
max_iterations=metadata.max_iterations,
@@ -348,13 +348,13 @@ def initialize_runtime(
# Check if an existing graph index file is available
graph_index_file_path = os.path.join(
INDEX_BASE_DIR, 'graph_index_v2.3', f"{instance['instance_id']}.pkl"
INDEX_BASE_DIR, 'graph_index_v2.3', f'{instance["instance_id"]}.pkl'
)
if INDEX_BASE_DIR and os.path.exists(graph_index_file_path):
logger.info(
f"Copying graph index from {graph_index_file_path} to /workspace/{workspace_dir_name}/_index_data/graph_index_v2.3"
f'Copying graph index from {graph_index_file_path} to /workspace/{workspace_dir_name}/_index_data/graph_index_v2.3'
)
runtime.copy_to(
graph_index_file_path,
f'/workspace/{workspace_dir_name}/_index_data/graph_index_v2.3',
@@ -364,9 +364,13 @@ def initialize_runtime(
)
obs = runtime.run_action(action)
bm25_index_dir = os.path.join(INDEX_BASE_DIR, 'BM25_index', instance['instance_id'])
bm25_index_dir = os.path.join(
INDEX_BASE_DIR, 'BM25_index', instance['instance_id']
)
runtime.copy_to(
bm25_index_dir, f'/workspace/{workspace_dir_name}/_index_data', recursive=True
bm25_index_dir,
f'/workspace/{workspace_dir_name}/_index_data',
recursive=True,
)
action = CmdRunAction(
command=f'mv _index_data/{instance["instance_id"]} _index_data/bm25_index'
@@ -0,0 +1,131 @@
#!/usr/bin/env bash
set -eo pipefail
source "evaluation/utils/version_control.sh"
MODEL_CONFIG=$1
COMMIT_HASH=$2
AGENT=$3
EVAL_LIMIT=$4
MAX_ITER=$5
NUM_WORKERS=$6
SPLIT=$8
N_RUNS=$9
if [ -z "$NUM_WORKERS" ]; then
NUM_WORKERS=1
echo "Number of workers not specified, use default $NUM_WORKERS"
fi
checkout_eval_branch
if [ -z "$AGENT" ]; then
echo "Agent not specified, use default CodeActAgent"
AGENT="CodeActAgent"
fi
if [ -z "$MAX_ITER" ]; then
echo "MAX_ITER not specified, use default 100"
MAX_ITER=100
fi
if [ -z "$RUN_WITH_BROWSING" ]; then
echo "RUN_WITH_BROWSING not specified, use default false"
RUN_WITH_BROWSING=false
fi
if [ -z "$DATASET" ]; then
echo "DATASET not specified, use default cmu-lti/interactive-swe"
DATASET="cmu-lti/interactive-swe"
fi
if [ -z "$SPLIT" ]; then
echo "SPLIT not specified, use default test"
SPLIT="test"
fi
if [ -n "$EVAL_CONDENSER" ]; then
echo "Using Condenser Config: $EVAL_CONDENSER"
else
echo "No Condenser Config provided via EVAL_CONDENSER, use default (NoOpCondenser)."
fi
export RUN_WITH_BROWSING=$RUN_WITH_BROWSING
echo "RUN_WITH_BROWSING: $RUN_WITH_BROWSING"
get_openhands_version
echo "AGENT: $AGENT"
echo "OPENHANDS_VERSION: $OPENHANDS_VERSION"
echo "MODEL_CONFIG: $MODEL_CONFIG"
echo "DATASET: $DATASET"
echo "SPLIT: $SPLIT"
echo "MAX_ITER: $MAX_ITER"
echo "NUM_WORKERS: $NUM_WORKERS"
echo "COMMIT_HASH: $COMMIT_HASH"
echo "EVAL_CONDENSER: $EVAL_CONDENSER"
# Default to NOT use Hint
if [ -z "$USE_HINT_TEXT" ]; then
export USE_HINT_TEXT=false
fi
echo "USE_HINT_TEXT: $USE_HINT_TEXT"
EVAL_NOTE="$OPENHANDS_VERSION"
# if not using Hint, add -no-hint to the eval note
if [ "$USE_HINT_TEXT" = false ]; then
EVAL_NOTE="$EVAL_NOTE-no-hint"
fi
if [ "$RUN_WITH_BROWSING" = true ]; then
EVAL_NOTE="$EVAL_NOTE-with-browsing"
fi
if [ -n "$EXP_NAME" ]; then
EVAL_NOTE="$EVAL_NOTE-$EXP_NAME"
fi
# Add condenser config to eval note if provided
if [ -n "$EVAL_CONDENSER" ]; then
EVAL_NOTE="${EVAL_NOTE}-${EVAL_CONDENSER}"
fi
function run_eval() {
local eval_note="${1}"
COMMAND="poetry run python evaluation/benchmarks/swe_bench/run_infer_interact.py \
--agent-cls $AGENT \
--llm-config $MODEL_CONFIG \
--max-iterations $MAX_ITER \
--eval-num-workers $NUM_WORKERS \
--eval-note $eval_note \
--dataset $DATASET \
--split $SPLIT"
if [ -n "$EVAL_LIMIT" ]; then
echo "EVAL_LIMIT: $EVAL_LIMIT"
COMMAND="$COMMAND --eval-n-limit $EVAL_LIMIT"
fi
# Run the command
eval $COMMAND
}
unset SANDBOX_ENV_GITHUB_TOKEN # prevent the agent from using the github token to push
if [ -z "$N_RUNS" ]; then
N_RUNS=1
echo "N_RUNS not specified, use default $N_RUNS"
fi
# Skip runs if the run number is in the SKIP_RUNS list
# read from env variable SKIP_RUNS as a comma separated list of run numbers
SKIP_RUNS=(${SKIP_RUNS//,/ })
for i in $(seq 1 $N_RUNS); do
if [[ " ${SKIP_RUNS[@]} " =~ " $i " ]]; then
echo "Skipping run $i"
continue
fi
current_eval_note="$EVAL_NOTE-run_$i"
echo "EVAL_NOTE: $current_eval_note"
run_eval $current_eval_note
done
checkout_original_branch
@@ -41,7 +41,7 @@ from evaluation.utils.shared import (
reset_logger_for_multiprocessing,
run_evaluation,
)
from openhands.core.config import AppConfig, SandboxConfig, get_parser
from openhands.core.config import OpenHandsConfig, SandboxConfig, get_parser
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime
from openhands.events.action import CmdRunAction
@@ -52,13 +52,13 @@ DOCKER_IMAGE_PREFIX = os.environ.get('EVAL_DOCKER_IMAGE_PREFIX', 'docker.io/kdja
logger.info(f'Using docker image prefix: {DOCKER_IMAGE_PREFIX}')
def get_config(instance: pd.Series) -> AppConfig:
def get_config(instance: pd.Series) -> OpenHandsConfig:
base_container_image = get_instance_docker_image(instance['instance_id_swebench'])
assert base_container_image, (
f'Invalid container image for instance {instance["instance_id_swebench"]}.'
)
logger.info(f'Using instance container image: {base_container_image}.')
return AppConfig(
return OpenHandsConfig(
run_as_openhands=False,
runtime=os.environ.get('RUNTIME', 'eventstream'),
sandbox=SandboxConfig(
@@ -35,7 +35,7 @@ from evaluation.utils.shared import (
from openhands.controller.state.state import State
from openhands.core.config import (
AgentConfig,
AppConfig,
OpenHandsConfig,
SandboxConfig,
get_llm_config_arg,
get_parser,
@@ -117,7 +117,7 @@ def get_instance_docker_image(instance_id: str) -> str:
def get_config(
instance: pd.Series,
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
# We use a different instance image for the each instance of TestGenEval
base_container_image = get_instance_docker_image(instance['instance_id_swebench'])
logger.info(
@@ -126,7 +126,7 @@ def get_config(
f'Submit an issue on https://github.com/All-Hands-AI/OpenHands if you run into any issues.'
)
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
max_iterations=metadata.max_iterations,
@@ -15,8 +15,8 @@ from browsing import pre_login
from evaluation.utils.shared import get_default_sandbox_config_for_eval
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
LLMConfig,
OpenHandsConfig,
get_agent_config_arg,
get_llm_config_arg,
get_parser,
@@ -36,13 +36,13 @@ def get_config(
mount_path_on_host: str,
llm_config: LLMConfig,
agent_config: AgentConfig | None,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = base_container_image
sandbox_config.enable_auto_lint = True
# If the web services are running on the host machine, this must be set to True
sandbox_config.use_host_network = True
config = AppConfig(
config = OpenHandsConfig(
run_as_openhands=False,
max_budget_per_task=4,
max_iterations=100,
@@ -126,7 +126,7 @@ def codeact_user_response(state: State) -> str:
def run_solver(
runtime: Runtime,
task_name: str,
config: AppConfig,
config: OpenHandsConfig,
dependencies: list[str],
save_final_state: bool,
state_dir: str,
@@ -274,7 +274,7 @@ if __name__ == '__main__':
temp_dir = os.path.abspath(os.getenv('TMPDIR'))
else:
temp_dir = tempfile.mkdtemp()
config: AppConfig = get_config(
config: OpenHandsConfig = get_config(
args.task_image_name, task_short_name, temp_dir, agent_llm_config, agent_config
)
runtime: Runtime = create_runtime(config)
+3 -3
View File
@@ -18,7 +18,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -40,10 +40,10 @@ AGENT_CLS_TO_INST_SUFFIX = {
def get_config(
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.12-bookworm'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
@@ -30,7 +30,7 @@ from evaluation.utils.shared import (
from openhands.controller.state.state import State
from openhands.core.config import (
AgentConfig,
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
get_parser,
)
@@ -135,7 +135,7 @@ def get_instance_docker_image(instance_id: str, official_image: bool = False) ->
def get_config(
instance: pd.Series,
metadata: EvalMetadata,
) -> AppConfig:
) -> OpenHandsConfig:
# We use a different instance image for the each instance of swe-bench eval
use_official_image = bool(
'verified' in metadata.dataset.lower() or 'lite' in metadata.dataset.lower()
@@ -160,7 +160,7 @@ def get_config(
instance_id=instance['instance_id'],
)
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
max_iterations=metadata.max_iterations,
@@ -20,7 +20,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -48,7 +48,7 @@ AGENT_CLS_TO_FAKE_USER_RESPONSE_FN = {
def get_config(
metadata: EvalMetadata,
env_id: str,
) -> AppConfig:
) -> OpenHandsConfig:
base_url = os.environ.get('VISUALWEBARENA_BASE_URL', None)
openai_api_key = os.environ.get('OPENAI_API_KEY', None)
openai_base_url = os.environ.get('OPENAI_BASE_URL', None)
@@ -72,7 +72,7 @@ def get_config(
'VWA_WIKIPEDIA': f'{base_url}:8888',
'VWA_HOMEPAGE': f'{base_url}:4399',
}
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
+3 -3
View File
@@ -19,7 +19,7 @@ from evaluation.utils.shared import (
)
from openhands.controller.state.state import State
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -44,7 +44,7 @@ SUPPORTED_AGENT_CLS = {'BrowsingAgent'}
def get_config(
metadata: EvalMetadata,
env_id: str,
) -> AppConfig:
) -> OpenHandsConfig:
base_url = os.environ.get('WEBARENA_BASE_URL', None)
openai_api_key = os.environ.get('OPENAI_API_KEY', None)
assert base_url is not None, 'WEBARENA_BASE_URL must be set'
@@ -64,7 +64,7 @@ def get_config(
'MAP': f'{base_url}:3000',
'HOMEPAGE': f'{base_url}:4399',
}
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime='docker',
+3 -3
View File
@@ -21,7 +21,7 @@ from evaluation.utils.shared import (
from openhands.controller.state.state import State
from openhands.core.config import (
AgentConfig,
AppConfig,
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
@@ -41,10 +41,10 @@ FAKE_RESPONSES = {
def get_config(
metadata: EvalMetadata,
instance_id: str,
) -> AppConfig:
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.platform = 'linux/amd64'
config = AppConfig(
config = OpenHandsConfig(
default_agent=metadata.agent_class,
run_as_openhands=False,
runtime=os.environ.get('RUNTIME', 'docker'),
+2 -2
View File
@@ -2,9 +2,9 @@ import argparse
import pytest
from openhands.config import load_app_config
from openhands.config import load_openhands_config
config = load_app_config()
config = load_openhands_config()
if __name__ == '__main__':
"""Main entry point of the script.
@@ -7,6 +7,7 @@ import React from "react";
import { renderWithProviders } from "test-utils";
import { ConversationPanel } from "#/components/features/conversation-panel/conversation-panel";
import OpenHands from "#/api/open-hands";
import { Conversation } from "#/api/open-hands.types";
describe("ConversationPanel", () => {
const onCloseMock = vi.fn();
@@ -37,11 +38,13 @@ describe("ConversationPanel", () => {
}));
});
const mockConversations = [
const mockConversations: Conversation[] = [
{
conversation_id: "1",
title: "Conversation 1",
selected_repository: null,
git_provider: null,
selected_branch: null,
last_updated_at: "2021-10-01T12:00:00Z",
created_at: "2021-10-01T12:00:00Z",
status: "STOPPED" as const,
@@ -52,6 +55,8 @@ describe("ConversationPanel", () => {
conversation_id: "2",
title: "Conversation 2",
selected_repository: null,
git_provider: null,
selected_branch: null,
last_updated_at: "2021-10-02T12:00:00Z",
created_at: "2021-10-02T12:00:00Z",
status: "STOPPED" as const,
@@ -62,6 +67,8 @@ describe("ConversationPanel", () => {
conversation_id: "3",
title: "Conversation 3",
selected_repository: null,
git_provider: null,
selected_branch: null,
last_updated_at: "2021-10-03T12:00:00Z",
created_at: "2021-10-03T12:00:00Z",
status: "STOPPED" as const,
@@ -141,11 +148,13 @@ describe("ConversationPanel", () => {
it("should delete a conversation", async () => {
const user = userEvent.setup();
const mockData = [
const mockData: Conversation[] = [
{
conversation_id: "1",
title: "Conversation 1",
selected_repository: null,
git_provider: null,
selected_branch: null,
last_updated_at: "2021-10-01T12:00:00Z",
created_at: "2021-10-01T12:00:00Z",
status: "STOPPED" as const,
@@ -156,6 +165,8 @@ describe("ConversationPanel", () => {
conversation_id: "2",
title: "Conversation 2",
selected_repository: null,
git_provider: null,
selected_branch: null,
last_updated_at: "2021-10-02T12:00:00Z",
created_at: "2021-10-02T12:00:00Z",
status: "STOPPED" as const,
@@ -166,6 +177,8 @@ describe("ConversationPanel", () => {
conversation_id: "3",
title: "Conversation 3",
selected_repository: null,
git_provider: null,
selected_branch: null,
last_updated_at: "2021-10-03T12:00:00Z",
created_at: "2021-10-03T12:00:00Z",
status: "STOPPED" as const,
@@ -8,6 +8,29 @@ import userEvent from "@testing-library/user-event";
import { HomeHeader } from "#/components/features/home/home-header";
import OpenHands from "#/api/open-hands";
// Mock the translation function
vi.mock("react-i18next", async () => {
const actual = await vi.importActual("react-i18next");
return {
...actual,
useTranslation: () => ({
t: (key: string) => {
// Return a mock translation for the test
const translations: Record<string, string> = {
"HOME$LETS_START_BUILDING": "Let's start building",
"HOME$LAUNCH_FROM_SCRATCH": "Launch from Scratch",
"HOME$LOADING": "Loading...",
"HOME$OPENHANDS_DESCRIPTION": "OpenHands is an AI software engineer",
"HOME$NOT_SURE_HOW_TO_START": "Not sure how to start?",
"HOME$READ_THIS": "Read this"
};
return translations[key] || key;
},
i18n: { language: "en" },
}),
};
});
const renderHomeHeader = () => {
const RouterStub = createRoutesStub([
{
@@ -38,7 +61,7 @@ describe("HomeHeader", () => {
renderHomeHeader();
const launchButton = screen.getByRole("button", {
name: /launch from scratch/i,
name: /Launch from Scratch/i,
});
await userEvent.click(launchButton);
@@ -60,11 +83,11 @@ describe("HomeHeader", () => {
renderHomeHeader();
const launchButton = screen.getByRole("button", {
name: /launch from scratch/i,
name: /Launch from Scratch/i,
});
await userEvent.click(launchButton);
expect(launchButton).toHaveTextContent(/Loading/i);
expect(launchButton).toHaveTextContent(/Loading.../i);
expect(launchButton).toBeDisabled();
});
});
@@ -19,7 +19,11 @@ describe("Check for hardcoded English strings", () => {
const text = container.textContent;
// List of English strings that should be translated
const hardcodedStrings = ["What do you want to build?"];
const hardcodedStrings = [
"What do you want to build?",
"Launch from Scratch",
"Read this"
];
// Check each string
hardcodedStrings.forEach((str) => {
@@ -0,0 +1,42 @@
import { render } from "@testing-library/react";
import { test, expect, describe, vi } from "vitest";
import { HomeHeader } from "#/components/features/home/home-header";
// Mock dependencies
vi.mock("#/hooks/mutation/use-create-conversation", () => ({
useCreateConversation: () => ({
mutate: vi.fn(),
isPending: false,
isSuccess: false,
}),
}));
vi.mock("#/hooks/use-is-creating-conversation", () => ({
useIsCreatingConversation: () => false,
}));
vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string) => key,
}),
}));
describe("Check for hardcoded English strings in Home components", () => {
test("HomeHeader should not have hardcoded English strings", () => {
const { container } = render(<HomeHeader />);
// Get all text content
const text = container.textContent;
// List of English strings that should be translated
const hardcodedStrings = [
"Launch from Scratch",
"Read this",
];
// Check each string
hardcodedStrings.forEach((str) => {
expect(text).not.toContain(str);
});
});
});
+2
View File
@@ -76,6 +76,8 @@ export interface Conversation {
conversation_id: string;
title: string;
selected_repository: string | null;
selected_branch: string | null;
git_provider: string | null;
last_updated_at: string;
created_at: string;
status: ProjectStatus;
@@ -26,7 +26,6 @@ import { downloadTrajectory } from "#/utils/download-trajectory";
import { displayErrorToast } from "#/utils/custom-toast-handlers";
import { useOptimisticUserMessage } from "#/hooks/use-optimistic-user-message";
import { useWSErrorMessage } from "#/hooks/use-ws-error-message";
import i18n from "#/i18n";
import { ErrorMessageBanner } from "./error-message-banner";
import { shouldRenderEvent } from "./event-content-helpers/should-render-event";
@@ -181,11 +180,7 @@ export function ChatInterface() {
{!hitBottom && <ScrollToBottomButton onClick={scrollDomToBottom} />}
</div>
{errorMessage && (
<ErrorMessageBanner
message={i18n.exists(errorMessage) ? t(errorMessage) : errorMessage}
/>
)}
{errorMessage && <ErrorMessageBanner message={errorMessage} />}
<InteractiveChatBox
onSubmit={handleSendMessage}
@@ -1,3 +1,7 @@
import { Trans } from "react-i18next";
import { Link } from "react-router";
import i18n from "#/i18n";
interface ErrorMessageBannerProps {
message: string;
}
@@ -5,7 +9,23 @@ interface ErrorMessageBannerProps {
export function ErrorMessageBanner({ message }: ErrorMessageBannerProps) {
return (
<div className="w-full rounded-lg p-2 text-black border border-red-800 bg-red-500">
{message}
{i18n.exists(message) ? (
<Trans
i18nKey={message}
components={{
a: (
<Link
className="underline font-bold cursor-pointer"
to="/settings/billing"
>
link
</Link>
),
}}
/>
) : (
message
)}
</div>
);
}
@@ -1,6 +1,11 @@
import { OpenHandsAction } from "#/types/core/actions";
import { OpenHandsEventType } from "#/types/core/base";
import { isOpenHandsAction, isOpenHandsObservation } from "#/types/core/guards";
import {
isCommandAction,
isCommandObservation,
isOpenHandsAction,
isOpenHandsObservation,
} from "#/types/core/guards";
import { OpenHandsObservation } from "#/types/core/observations";
const COMMON_NO_RENDER_LIST: OpenHandsEventType[] = [
@@ -15,11 +20,21 @@ export const shouldRenderEvent = (
event: OpenHandsAction | OpenHandsObservation,
) => {
if (isOpenHandsAction(event)) {
if (isCommandAction(event) && event.source === "user") {
// For user commands, we always hide them from the chat interface
return false;
}
const noRenderList = COMMON_NO_RENDER_LIST.concat(ACTION_NO_RENDER_LIST);
return !noRenderList.includes(event.action);
}
if (isOpenHandsObservation(event)) {
if (isCommandObservation(event) && event.source === "user") {
// For user commands, we always hide them from the chat interface
return false;
}
return !COMMON_NO_RENDER_LIST.includes(event.observation);
}
@@ -2,32 +2,10 @@ import React from "react";
import { OpenHandsAction } from "#/types/core/actions";
import { OpenHandsObservation } from "#/types/core/observations";
import { isOpenHandsAction, isOpenHandsObservation } from "#/types/core/guards";
import { OpenHandsEventType } from "#/types/core/base";
import { EventMessage } from "./event-message";
import { ChatMessage } from "./chat-message";
import { useOptimisticUserMessage } from "#/hooks/use-optimistic-user-message";
const COMMON_NO_RENDER_LIST: OpenHandsEventType[] = [
"system",
"agent_state_changed",
"change_agent_state",
];
const ACTION_NO_RENDER_LIST: OpenHandsEventType[] = ["recall"];
const shouldRenderEvent = (event: OpenHandsAction | OpenHandsObservation) => {
if (isOpenHandsAction(event)) {
const noRenderList = COMMON_NO_RENDER_LIST.concat(ACTION_NO_RENDER_LIST);
return !noRenderList.includes(event.action);
}
if (isOpenHandsObservation(event)) {
return !COMMON_NO_RENDER_LIST.includes(event.observation);
}
return true;
};
interface MessagesProps {
messages: (OpenHandsAction | OpenHandsObservation)[];
isAwaitingUserConfirmation: boolean;
@@ -49,12 +27,12 @@ export const Messages: React.FC<MessagesProps> = React.memo(
return false;
},
[],
[messages],
);
return (
<>
{messages.filter(shouldRenderEvent).map((message, index) => (
{messages.map((message, index) => (
<EventMessage
key={index}
event={message}
@@ -129,7 +129,7 @@ export function FeedbackForm({ onClose, polarity }: FeedbackFormProps) {
isDisabled={isPending}
>
{isPending
? t(I18nKey.FEEDBACK$SUBMITTING_LABEL) || "Submitting..."
? t(I18nKey.FEEDBACK$SUBMITTING_LABEL)
: t(I18nKey.FEEDBACK$SHARE_LABEL)}
</BrandButton>
<BrandButton
@@ -144,8 +144,7 @@ export function FeedbackForm({ onClose, polarity }: FeedbackFormProps) {
</div>
{isPending && (
<p className="text-sm text-center text-neutral-400">
{t(I18nKey.FEEDBACK$SUBMITTING_MESSAGE) ||
"Submitting your feedback, please wait..."}
{t(I18nKey.FEEDBACK$SUBMITTING_MESSAGE)}
</p>
)}
</form>
@@ -31,7 +31,7 @@ export function HomeHeader() {
onClick={() => createConversation({})}
isDisabled={isCreatingConversation}
>
{!isCreatingConversation && "Launch from Scratch"}
{!isCreatingConversation && t("HOME$LAUNCH_FROM_SCRATCH")}
{isCreatingConversation && t("HOME$LOADING")}
</BrandButton>
</div>
@@ -48,7 +48,7 @@ export function HomeHeader() {
rel="noopener noreferrer"
className="underline underline-offset-2"
>
Read this
{t("HOME$READ_THIS")}
</a>
</p>
</div>
@@ -9,7 +9,6 @@ import GitHubLogo from "#/assets/branding/github-logo.svg?react";
import GitLabLogo from "#/assets/branding/gitlab-logo.svg?react";
import { useAuthUrl } from "#/hooks/use-auth-url";
import { GetConfigResponse } from "#/api/open-hands.types";
import { LoginMethod, setLoginMethod } from "#/utils/local-storage";
interface AuthModalProps {
githubAuthUrl: string | null;
@@ -26,10 +25,6 @@ export function AuthModal({ githubAuthUrl, appMode }: AuthModalProps) {
const handleGitHubAuth = () => {
if (githubAuthUrl) {
// Store the login method in local storage (only in SAAS mode)
if (appMode === "saas") {
setLoginMethod(LoginMethod.GITHUB);
}
// Always start the OIDC flow, let the backend handle TOS check
window.location.href = githubAuthUrl;
}
@@ -37,10 +32,6 @@ export function AuthModal({ githubAuthUrl, appMode }: AuthModalProps) {
const handleGitLabAuth = () => {
if (gitlabAuthUrl) {
// Store the login method in local storage (only in SAAS mode)
if (appMode === "saas") {
setLoginMethod(LoginMethod.GITLAB);
}
// Always start the OIDC flow, let the backend handle TOS check
window.location.href = gitlabAuthUrl;
}
+15 -11
View File
@@ -166,6 +166,8 @@ export function WsClientProvider({
}
function handleMessage(event: Record<string, unknown>) {
handleAssistantMessage(event);
if (isOpenHandsEvent(event)) {
const isStatusUpdateError =
isStatusUpdate(event) && event.type === "error";
@@ -213,14 +215,18 @@ export function WsClientProvider({
// Invalidate diffs cache when a file is edited or written
if (
!messageRateHandler.isUnderThreshold &&
(isFileEditAction(event) ||
isFileWriteAction(event) ||
isCommandAction(event))
isFileEditAction(event) ||
isFileWriteAction(event) ||
isCommandAction(event)
) {
queryClient.invalidateQueries({
queryKey: ["file_changes", conversationId],
});
queryClient.invalidateQueries(
{
queryKey: ["file_changes", conversationId],
},
// Do not refetch if we are still receiving messages at a high rate (e.g., loading an existing conversation)
// This prevents unnecessary refetches when the user is still receiving messages
{ cancelRefetch: false },
);
// Invalidate file diff cache when a file is edited or written
if (!isCommandAction(event)) {
@@ -251,8 +257,6 @@ export function WsClientProvider({
if (!Number.isNaN(parseInt(event.id as string, 10))) {
lastEventRef.current = event;
}
handleAssistantMessage(event);
}
function handleDisconnect(data: unknown) {
@@ -285,14 +289,14 @@ export function WsClientProvider({
React.useEffect(() => {
lastEventRef.current = null;
}, [conversationId]);
React.useEffect(() => {
// reset events when conversationId changes
setEvents([]);
setParsedEvents([]);
setStatus(WsClientProviderStatus.DISCONNECTED);
}, [conversationId]);
React.useEffect(() => {
if (!conversationId) {
throw new Error("No conversation ID provided");
}
@@ -1,5 +1,4 @@
import { useMutation, useQueryClient } from "@tanstack/react-query";
import { useNavigate } from "react-router";
import posthog from "posthog-js";
import OpenHands from "#/api/open-hands";
import { useConfig } from "../query/use-config";
@@ -8,7 +7,6 @@ import { clearLoginData } from "#/utils/local-storage";
export const useLogout = () => {
const queryClient = useQueryClient();
const { data: config } = useConfig();
const navigate = useNavigate();
return useMutation({
mutationFn: () => OpenHands.logout(config?.APP_MODE ?? "oss"),
@@ -24,7 +22,6 @@ export const useLogout = () => {
}
posthog.reset();
await navigate("/");
// Refresh the page after all logout logic is completed
window.location.reload();
+3 -10
View File
@@ -1,21 +1,14 @@
import { useQueries, useQuery } from "@tanstack/react-query";
import axios from "axios";
import React from "react";
import { useSelector } from "react-redux";
import OpenHands from "#/api/open-hands";
import { RUNTIME_INACTIVE_STATES } from "#/types/agent-state";
import { RootState } from "#/store";
import { useConversationId } from "#/hooks/use-conversation-id";
import { useActiveConversation } from "./use-active-conversation";
import { useRuntimeIsReady } from "#/hooks/use-runtime-is-ready";
export const useActiveHost = () => {
const { curAgentState } = useSelector((state: RootState) => state.agent);
const [activeHost, setActiveHost] = React.useState<string | null>(null);
const { conversationId } = useConversationId();
const { data: conversation } = useActiveConversation();
const enabled =
conversation?.status === "RUNNING" &&
RUNTIME_INACTIVE_STATES.includes(curAgentState);
const runtimeIsReady = useRuntimeIsReady();
const { data } = useQuery({
queryKey: [conversationId, "hosts"],
@@ -23,7 +16,7 @@ export const useActiveHost = () => {
const hosts = await OpenHands.getWebHosts(conversationId);
return { hosts };
},
enabled,
enabled: runtimeIsReady && !!conversationId,
initialData: { hosts: [] },
meta: {
disableToast: true,
@@ -1,23 +1,15 @@
import { useQuery } from "@tanstack/react-query";
import React from "react";
import { useSelector } from "react-redux";
import OpenHands from "#/api/open-hands";
import { useConversationId } from "#/hooks/use-conversation-id";
import { GitChange } from "#/api/open-hands.types";
import { RootState } from "#/store";
import { RUNTIME_INACTIVE_STATES } from "#/types/agent-state";
import { useActiveConversation } from "./use-active-conversation";
import { useRuntimeIsReady } from "#/hooks/use-runtime-is-ready";
export const useGetGitChanges = () => {
const { conversationId } = useConversationId();
const { data: conversation } = useActiveConversation();
const [orderedChanges, setOrderedChanges] = React.useState<GitChange[]>([]);
const previousDataRef = React.useRef<GitChange[]>(null);
const { curAgentState } = useSelector((state: RootState) => state.agent);
const enabled =
conversation?.status === "RUNNING" &&
RUNTIME_INACTIVE_STATES.includes(curAgentState);
const runtimeIsReady = useRuntimeIsReady();
const result = useQuery({
queryKey: ["file_changes", conversationId],
@@ -25,7 +17,7 @@ export const useGetGitChanges = () => {
retry: false,
staleTime: 1000 * 60 * 5, // 5 minutes
gcTime: 1000 * 60 * 15, // 15 minutes
enabled,
enabled: runtimeIsReady && !!conversationId,
meta: {
disableToast: true,
},
+1 -1
View File
@@ -27,7 +27,7 @@ const getSettingsQueryFn = async (): Promise<Settings> => {
apiSettings.enable_proactive_conversation_starters,
USER_CONSENTS_TO_ANALYTICS: apiSettings.user_consents_to_analytics,
SEARCH_API_KEY: apiSettings.search_api_key || "",
EMAIL: apiSettings.email || "",
MCP_CONFIG: apiSettings.mcp_config,
IS_NEW_USER: false,
};
+3 -10
View File
@@ -1,13 +1,10 @@
import { useQuery } from "@tanstack/react-query";
import { useTranslation } from "react-i18next";
import { useSelector } from "react-redux";
import OpenHands from "#/api/open-hands";
import { useConversationId } from "#/hooks/use-conversation-id";
import { I18nKey } from "#/i18n/declaration";
import { RootState } from "#/store";
import { RUNTIME_INACTIVE_STATES } from "#/types/agent-state";
import { transformVSCodeUrl } from "#/utils/vscode-url-helper";
import { useActiveConversation } from "./use-active-conversation";
import { useRuntimeIsReady } from "#/hooks/use-runtime-is-ready";
// Define the return type for the VS Code URL query
interface VSCodeUrlResult {
@@ -18,11 +15,7 @@ interface VSCodeUrlResult {
export const useVSCodeUrl = () => {
const { t } = useTranslation();
const { conversationId } = useConversationId();
const { data: conversation } = useActiveConversation();
const { curAgentState } = useSelector((state: RootState) => state.agent);
const enabled =
conversation?.status === "RUNNING" &&
RUNTIME_INACTIVE_STATES.includes(curAgentState);
const runtimeIsReady = useRuntimeIsReady();
return useQuery<VSCodeUrlResult>({
queryKey: ["vscode_url", conversationId],
@@ -40,7 +33,7 @@ export const useVSCodeUrl = () => {
error: t(I18nKey.VSCODE$URL_NOT_AVAILABLE),
};
},
enabled,
enabled: runtimeIsReady && !!conversationId,
refetchOnMount: true,
retry: 3,
});
+49
View File
@@ -0,0 +1,49 @@
import { useEffect } from "react";
import { useLocation, useNavigate } from "react-router";
import { useIsAuthed } from "./query/use-is-authed";
import { LoginMethod, setLoginMethod } from "#/utils/local-storage";
import { useConfig } from "./query/use-config";
/**
* Hook to handle authentication callback and set login method after successful authentication
*/
export const useAuthCallback = () => {
const location = useLocation();
const { data: isAuthed, isLoading: isAuthLoading } = useIsAuthed();
const { data: config } = useConfig();
const navigate = useNavigate();
useEffect(() => {
// Only run in SAAS mode
if (config?.APP_MODE !== "saas") {
return;
}
// Wait for auth to load
if (isAuthLoading) {
return;
}
// Only set login method if authentication was successful
if (!isAuthed) {
return;
}
// Check if we have a login_method query parameter
const searchParams = new URLSearchParams(location.search);
const loginMethod = searchParams.get("login_method");
// Set the login method if it's valid
if (
loginMethod === LoginMethod.GITHUB ||
loginMethod === LoginMethod.GITLAB
) {
setLoginMethod(loginMethod as LoginMethod);
// Clean up the URL by removing the login_method parameter
searchParams.delete("login_method");
const newUrl = `${location.pathname}${searchParams.toString() ? `?${searchParams.toString()}` : ""}`;
navigate(newUrl, { replace: true });
}
}, [isAuthed, isAuthLoading, location.search, config?.APP_MODE]);
};
+5 -1
View File
@@ -53,8 +53,12 @@ export const useAutoLogin = () => {
// If we have an auth URL, redirect to it
if (authUrl) {
// Add the login method as a query parameter
const url = new URL(authUrl);
url.searchParams.append("login_method", loginMethod);
// After successful login, the user will be redirected back and can navigate to the last page
window.location.href = authUrl;
window.location.href = url.toString();
}
}, [
config?.APP_MODE,
@@ -0,0 +1,19 @@
import { useSelector } from "react-redux";
import { RootState } from "#/store";
import { RUNTIME_INACTIVE_STATES } from "#/types/agent-state";
import { useActiveConversation } from "./query/use-active-conversation";
/**
* Hook to determine if the runtime is ready for operations
*
* @returns boolean indicating if the runtime is ready
*/
export const useRuntimeIsReady = (): boolean => {
const { data: conversation } = useActiveConversation();
const { curAgentState } = useSelector((state: RootState) => state.agent);
return (
conversation?.status === "RUNNING" &&
!RUNTIME_INACTIVE_STATES.includes(curAgentState)
);
};
+8
View File
@@ -1,5 +1,7 @@
// this file generate by script, don't modify it manually!!!
export enum I18nKey {
HOME$LAUNCH_FROM_SCRATCH = "HOME$LAUNCH_FROM_SCRATCH",
HOME$READ_THIS = "HOME$READ_THIS",
AUTH$LOGGING_BACK_IN = "AUTH$LOGGING_BACK_IN",
SECURITY$LOW_RISK = "SECURITY$LOW_RISK",
SECURITY$MEDIUM_RISK = "SECURITY$MEDIUM_RISK",
@@ -550,4 +552,10 @@ export enum I18nKey {
TIPS$PROTIP = "TIPS$PROTIP",
FEEDBACK$SUBMITTING_LABEL = "FEEDBACK$SUBMITTING_LABEL",
FEEDBACK$SUBMITTING_MESSAGE = "FEEDBACK$SUBMITTING_MESSAGE",
SETTINGS$NAV_USER = "SETTINGS$NAV_USER",
SETTINGS$USER_TITLE = "SETTINGS$USER_TITLE",
SETTINGS$USER_EMAIL = "SETTINGS$USER_EMAIL",
SETTINGS$USER_EMAIL_NOT_AVAILABLE = "SETTINGS$USER_EMAIL_NOT_AVAILABLE",
SETTINGS$SAVE = "SETTINGS$SAVE",
SETTINGS$EMAIL_SAVED_SUCCESSFULLY = "SETTINGS$EMAIL_SAVED_SUCCESSFULLY",
}
+159 -15
View File
@@ -1,4 +1,36 @@
{
"HOME$LAUNCH_FROM_SCRATCH": {
"en": "Launch from Scratch",
"ja": "ゼロから始める",
"zh-CN": "从零开始",
"zh-TW": "從零開始",
"ko-KR": "처음부터 시작",
"no": "Start fra bunnen",
"it": "Inizia da zero",
"pt": "Começar do zero",
"es": "Comenzar desde cero",
"ar": "البدء من الصفر",
"fr": "Démarrer de zéro",
"tr": "Sıfırdan başla",
"de": "Von Grund auf starten",
"uk": "Почати з нуля"
},
"HOME$READ_THIS": {
"en": "Read this",
"ja": "こちらを読む",
"zh-CN": "阅读此内容",
"zh-TW": "閱讀此內容",
"ko-KR": "이것을 읽어보세요",
"no": "Les dette",
"it": "Leggi questo",
"pt": "Leia isto",
"es": "Leer esto",
"ar": "اقرأ هذا",
"fr": "Lire ceci",
"tr": "Bunu oku",
"de": "Lies dies",
"uk": "Прочитайте це"
},
"AUTH$LOGGING_BACK_IN": {
"en": "Logging back into OpenHands...",
"ja": "OpenHandsに再ログインしています...",
@@ -6400,20 +6432,20 @@
"uk": "Запит не вдалося виконати через внутрішню помилку сервера."
},
"STATUS$ERROR_LLM_OUT_OF_CREDITS": {
"en": "You're out of OpenHands Credits",
"ja": "OpenHandsクレジットが不足しています",
"zh-CN": "您的OpenHands点数已用完",
"zh-TW": "您的OpenHands點數已用完",
"ko-KR": "OpenHands 크레딧이 소진되었습니다",
"no": "Du er tom for OpenHands-kreditter",
"it": "Hai esaurito i crediti OpenHands",
"pt": "Você está sem créditos OpenHands",
"es": "Te has quedado sin créditos de OpenHands",
"ar": "لقد نفدت رصيدك من OpenHands",
"fr": "Vous n'avez plus de crédits OpenHands",
"tr": "OpenHands kredileriniz tükendi",
"de": "Ihre OpenHands-Guthaben sind aufgebraucht",
"uk": "У вас закінчилися кредити OpenHands"
"en": "You're out of OpenHands Credits. <a>Add funds</a>",
"ja": "OpenHandsクレジットが不足しています。<a>資金を追加</a>",
"zh-CN": "您的OpenHands点数已用完。<a>添加资金</a>",
"zh-TW": "您的OpenHands點數已用完。<a>添加資金</a>",
"ko-KR": "OpenHands 크레딧이 소진되었습니다. <a>자금 추가</a>",
"no": "Du er tom for OpenHands-kreditter. <a>Legg til midler</a>",
"it": "Hai esaurito i crediti OpenHands. <a>Aggiungi fondi</a>",
"pt": "Você está sem créditos OpenHands. <a>Adicionar fundos</a>",
"es": "Te has quedado sin créditos de OpenHands. <a>Añadir fondos</a>",
"ar": "لقد نفدت رصيدك من OpenHands. <a>إضافة رصيد</a>",
"fr": "Vous n'avez plus de crédits OpenHands. <a>Ajouter des fonds</a>",
"tr": "OpenHands kredileriniz tükendi. <a>Bakiye ekle</a>",
"de": "Ihre OpenHands-Guthaben sind aufgebraucht. <a>Guthaben hinzufügen</a>",
"uk": "У вас закінчилися кредити OpenHands. <a>Додати кошти</a>"
},
"STATUS$ERROR_LLM_CONTENT_POLICY_VIOLATION": {
"en": "Content policy violation. The output was blocked by content filtering policy.",
@@ -8780,7 +8812,7 @@
"ar": "إرسال...",
"fr": "Envoi...",
"tr": "Gönderiliyor...",
"de": "Senden...",
"de": "Senden...",
"uk": "Відправляємо..."
},
"FEEDBACK$SUBMITTING_MESSAGE": {
@@ -8798,5 +8830,117 @@
"tr": "Geri bildirim gönderiliyor, lütfen bekleyin...",
"de": "Feedback senden, bitte warten...",
"uk": "Відправляємо відгук, будь ласка, почекайте..."
},
"SETTINGS$NAV_USER": {
"en": "User",
"ja": "ユーザー",
"zh-CN": "用户",
"zh-TW": "用戶",
"ko-KR": "사용자",
"no": "Bruker",
"it": "Utente",
"pt": "Usuário",
"es": "Usuario",
"ar": "المستخدم",
"fr": "Utilisateur",
"tr": "Kullanıcı",
"de": "Benutzer",
"uk": "Користувач"
},
"SETTINGS$USER_TITLE": {
"en": "User Information",
"ja": "ユーザー情報",
"zh-CN": "用户信息",
"zh-TW": "用戶信息",
"ko-KR": "사용자 정보",
"no": "Brukerinformasjon",
"it": "Informazioni utente",
"pt": "Informações do usuário",
"es": "Información del usuario",
"ar": "معلومات المستخدم",
"fr": "Informations utilisateur",
"tr": "Kullanıcı Bilgileri",
"de": "Benutzerinformationen",
"uk": "Інформація про користувача"
},
"SETTINGS$USER_EMAIL": {
"en": "Email",
"ja": "メール",
"zh-CN": "邮箱",
"zh-TW": "郵箱",
"ko-KR": "이메일",
"no": "E-post",
"it": "Email",
"pt": "Email",
"es": "Correo electrónico",
"ar": "البريد الإلكتروني",
"fr": "Email",
"tr": "E-posta",
"de": "E-Mail",
"uk": "Електронна пошта"
},
"SETTINGS$USER_EMAIL_NOT_AVAILABLE": {
"en": "Email not available",
"ja": "メールは利用できません",
"zh-CN": "邮箱不可用",
"zh-TW": "郵箱不可用",
"ko-KR": "이메일을 사용할 수 없습니다",
"no": "E-post ikke tilgjengelig",
"it": "Email non disponibile",
"pt": "Email não disponível",
"es": "Correo electrónico no disponible",
"ar": "البريد الإلكتروني غير متوفر",
"fr": "Email non disponible",
"tr": "E-posta mevcut değil",
"de": "E-Mail nicht verfügbar",
"uk": "Електронна пошта недоступна"
},
"SETTINGS$SAVE": {
"en": "Save",
"ja": "保存",
"zh-CN": "保存",
"zh-TW": "儲存",
"ko-KR": "저장",
"no": "Lagre",
"it": "Salva",
"pt": "Salvar",
"es": "Guardar",
"ar": "حفظ",
"fr": "Enregistrer",
"tr": "Kaydet",
"de": "Speichern",
"uk": "Зберегти"
},
"SETTINGS$EMAIL_SAVED_SUCCESSFULLY": {
"en": "Email saved successfully",
"ja": "メールが正常に保存されました",
"zh-CN": "邮箱保存成功",
"zh-TW": "郵箱儲存成功",
"ko-KR": "이메일이 성공적으로 저장되었습니다",
"no": "E-post lagret",
"it": "Email salvata con successo",
"pt": "Email salvo com sucesso",
"es": "Correo electrónico guardado con éxito",
"ar": "تم حفظ البريد الإلكتروني بنجاح",
"fr": "Email enregistré avec succès",
"tr": "E-posta başarıyla kaydedildi",
"de": "E-Mail erfolgreich gespeichert",
"uk": "Електронну пошту успішно збережено"
},
"SETTINGS$FAILED_TO_SAVE_EMAIL": {
"en": "Failed to save email",
"ja": "メールの保存に失敗しました",
"zh-CN": "保存邮箱失败",
"zh-TW": "儲存郵箱失敗",
"ko-KR": "이메일 저장 실패",
"no": "Kunne ikke lagre e-post",
"it": "Impossibile salvare l'email",
"pt": "Falha ao salvar email",
"es": "Error al guardar el correo electrónico",
"ar": "فشل في حفظ البريد الإلكتروني",
"fr": "Échec de l'enregistrement de l'email",
"tr": "E-posta kaydedilemedi",
"de": "E-Mail konnte nicht gespeichert werden",
"uk": "Не вдалося зберегти електронну пошту"
}
}
+8
View File
@@ -52,6 +52,8 @@ const conversations: Conversation[] = [
conversation_id: "1",
title: "My New Project",
selected_repository: null,
git_provider: null,
selected_branch: null,
last_updated_at: new Date().toISOString(),
created_at: new Date().toISOString(),
status: "RUNNING",
@@ -62,6 +64,8 @@ const conversations: Conversation[] = [
conversation_id: "2",
title: "Repo Testing",
selected_repository: "octocat/hello-world",
git_provider: "github",
selected_branch: null,
// 2 days ago
last_updated_at: new Date(
Date.now() - 2 * 24 * 60 * 60 * 1000,
@@ -75,6 +79,8 @@ const conversations: Conversation[] = [
conversation_id: "3",
title: "Another Project",
selected_repository: "octocat/earth",
git_provider: null,
selected_branch: "main",
// 5 days ago
last_updated_at: new Date(
Date.now() - 5 * 24 * 60 * 60 * 1000,
@@ -271,6 +277,8 @@ export const handlers = [
conversation_id: (Math.random() * 100).toString(),
title: "New Conversation",
selected_repository: null,
git_provider: null,
selected_branch: null,
last_updated_at: new Date().toISOString(),
created_at: new Date().toISOString(),
status: "RUNNING",
+1
View File
@@ -12,6 +12,7 @@ export default [
route("settings", "routes/settings.tsx", [
index("routes/llm-settings.tsx"),
route("mcp", "routes/mcp-settings.tsx"),
route("user", "routes/user-settings.tsx"),
route("git", "routes/git-settings.tsx"),
route("app", "routes/app-settings.tsx"),
route("billing", "routes/billing.tsx"),
+4 -2
View File
@@ -36,6 +36,7 @@ import { useDocumentTitleFromState } from "#/hooks/use-document-title-from-state
import { transformVSCodeUrl } from "#/utils/vscode-url-helper";
import OpenHands from "#/api/open-hands";
import { TabContent } from "#/components/layout/tab-content";
import { useIsAuthed } from "#/hooks/query/use-is-authed";
function AppContent() {
useConversationConfig();
@@ -43,6 +44,7 @@ function AppContent() {
const { data: settings } = useSettings();
const { conversationId } = useConversationId();
const { data: conversation, isFetched } = useActiveConversation();
const { data: isAuthed } = useIsAuthed();
const { curAgentState } = useSelector((state: RootState) => state.agent);
const dispatch = useDispatch();
@@ -54,13 +56,13 @@ function AppContent() {
const [width, setWidth] = React.useState(window.innerWidth);
React.useEffect(() => {
if (isFetched && !conversation) {
if (isFetched && !conversation && isAuthed) {
displayErrorToast(
"This conversation does not exist, or you do not have permission to access it.",
);
navigate("/");
}
}, [conversation, isFetched]);
}, [conversation, isFetched, isAuthed]);
React.useEffect(() => {
dispatch(clearTerminal());
+39 -2
View File
@@ -23,6 +23,7 @@ import { SetupPaymentModal } from "#/components/features/payment/setup-payment-m
import { displaySuccessToast } from "#/utils/custom-toast-handlers";
import { useIsOnTosPage } from "#/hooks/use-is-on-tos-page";
import { useAutoLogin } from "#/hooks/use-auto-login";
import { useAuthCallback } from "#/hooks/use-auth-callback";
import { LOCAL_STORAGE_KEYS } from "#/utils/local-storage";
export function ErrorBoundary() {
@@ -88,6 +89,9 @@ export default function MainApp() {
// Auto-login if login method is stored in local storage
useAutoLogin();
// Handle authentication callback and set login method after successful authentication
useAuthCallback();
React.useEffect(() => {
// Don't change language when on TOS page
if (!isOnTosPage && settings?.LANGUAGE) {
@@ -131,8 +135,8 @@ export default function MainApp() {
}
}, [error?.status, pathname, isOnTosPage]);
// Check if login method exists in local storage
const loginMethodExists = React.useMemo(() => {
// Function to check if login method exists in local storage
const checkLoginMethodExists = React.useCallback(() => {
// Only check localStorage if we're in a browser environment
if (typeof window !== "undefined" && window.localStorage) {
return localStorage.getItem(LOCAL_STORAGE_KEYS.LOGIN_METHOD) !== null;
@@ -140,6 +144,39 @@ export default function MainApp() {
return false;
}, []);
// State to track if login method exists
const [loginMethodExists, setLoginMethodExists] = React.useState(
checkLoginMethodExists(),
);
// Listen for storage events to update loginMethodExists when logout happens
React.useEffect(() => {
const handleStorageChange = (event: StorageEvent) => {
if (event.key === LOCAL_STORAGE_KEYS.LOGIN_METHOD) {
setLoginMethodExists(checkLoginMethodExists());
}
};
// Also check on window focus, as logout might happen in another tab
const handleWindowFocus = () => {
setLoginMethodExists(checkLoginMethodExists());
};
window.addEventListener("storage", handleStorageChange);
window.addEventListener("focus", handleWindowFocus);
return () => {
window.removeEventListener("storage", handleStorageChange);
window.removeEventListener("focus", handleWindowFocus);
};
}, [checkLoginMethodExists]);
// Check login method status when auth status changes
React.useEffect(() => {
// When auth status changes (especially on logout), recheck login method
setLoginMethodExists(checkLoginMethodExists());
}, [isAuthed, checkLoginMethodExists]);
const renderAuthModal =
!isAuthed &&
!isAuthError &&
+3 -1
View File
@@ -15,6 +15,7 @@ function SettingsScreen() {
const isSaas = config?.APP_MODE === "saas";
const saasNavItems = [
{ to: "/settings/user", text: t("SETTINGS$NAV_USER") },
{ to: "/settings/git", text: t("SETTINGS$NAV_GIT") },
{ to: "/settings/app", text: t("SETTINGS$NAV_APPLICATION") },
{ to: "/settings/billing", text: t("SETTINGS$NAV_CREDITS") },
@@ -33,10 +34,11 @@ function SettingsScreen() {
React.useEffect(() => {
if (isSaas) {
if (pathname === "/settings") {
navigate("/settings/git");
navigate("/settings/user");
}
} else {
const noEnteringPaths = [
"/settings/user",
"/settings/billing",
"/settings/credits",
"/settings/api-keys",
+98
View File
@@ -0,0 +1,98 @@
import React, { useState, useEffect } from "react";
import { useTranslation } from "react-i18next";
import { useQueryClient } from "@tanstack/react-query";
import { useSettings } from "#/hooks/query/use-settings";
import { openHands } from "#/api/open-hands-axios";
function UserSettingsScreen() {
const { t } = useTranslation();
const { data: settings, isLoading } = useSettings();
const [email, setEmail] = useState("");
const [originalEmail, setOriginalEmail] = useState("");
const [isSaving, setIsSaving] = useState(false);
const [saveSuccess, setSaveSuccess] = useState(false);
const queryClient = useQueryClient();
useEffect(() => {
if (settings?.EMAIL) {
setEmail(settings.EMAIL);
setOriginalEmail(settings.EMAIL);
}
}, [settings?.EMAIL]);
const handleEmailChange = (e: React.ChangeEvent<HTMLInputElement>) => {
setEmail(e.target.value);
setSaveSuccess(false);
};
const handleSaveEmail = async () => {
if (email === originalEmail) return;
try {
setIsSaving(true);
// Send email as part of settings update
await openHands.post('/api/settings', { email });
setOriginalEmail(email);
setSaveSuccess(true);
// Invalidate settings query to refresh data
queryClient.invalidateQueries({ queryKey: ["settings"] });
} catch (error) {
// Log error but don't show to user
// eslint-disable-next-line no-console
console.error(t("SETTINGS$FAILED_TO_SAVE_EMAIL"), error);
} finally {
setIsSaving(false);
}
};
const isEmailChanged = email !== originalEmail;
return (
<div data-testid="user-settings-screen" className="flex flex-col h-full">
<div className="p-9 flex flex-col gap-6">
{isLoading ? (
<div className="animate-pulse h-8 w-64 bg-tertiary rounded" />
) : (
<div className="flex flex-col gap-4">
<div className="flex flex-col gap-2">
<label className="text-sm">{t("SETTINGS$USER_EMAIL")}</label>
<div className="flex items-center gap-3">
<input
type="email"
value={email}
onChange={handleEmailChange}
className="text-base text-primary p-2 bg-base-tertiary rounded border border-tertiary flex-grow"
placeholder={t("SETTINGS$USER_EMAIL_NOT_AVAILABLE")}
data-testid="email-input"
/>
<button
type="button"
onClick={handleSaveEmail}
disabled={!isEmailChanged || isSaving}
className={`px-4 py-2 rounded ${
isEmailChanged && !isSaving
? "bg-primary text-white hover:bg-primary-dark"
: "bg-tertiary text-secondary cursor-not-allowed"
}`}
data-testid="save-email-button"
>
{isSaving ? t("SETTINGS$SAVING") : t("SETTINGS$SAVE")}
</button>
</div>
{saveSuccess && (
<div className="text-sm text-green-500 mt-1">
{t("SETTINGS$EMAIL_SAVED_SUCCESSFULLY")}
</div>
)}
</div>
</div>
)}
</div>
</div>
);
}
export default UserSettingsScreen;
+1 -1
View File
@@ -20,7 +20,7 @@ export interface SystemMessageAction extends OpenHandsActionEvent<"system"> {
}
export interface CommandAction extends OpenHandsActionEvent<"run"> {
source: "agent";
source: "agent" | "user";
args: {
command: string;
security_risk: ActionSecurityRisk;
+5
View File
@@ -4,6 +4,7 @@ import {
AssistantMessageAction,
OpenHandsAction,
SystemMessageAction,
CommandAction,
} from "./actions";
import {
AgentStateChangeObservation,
@@ -41,6 +42,10 @@ export const isErrorObservation = (
): event is ErrorObservation =>
isOpenHandsObservation(event) && event.observation === "error";
export const isCommandAction = (
event: OpenHandsParsedEvent,
): event is CommandAction => isOpenHandsAction(event) && event.action === "run";
export const isAgentStateChangeObservation = (
event: OpenHandsParsedEvent,
): event is AgentStateChangeObservation =>
+1 -1
View File
@@ -11,7 +11,7 @@ export interface AgentStateChangeObservation
}
export interface CommandObservation extends OpenHandsObservationEvent<"run"> {
source: "agent";
source: "agent" | "user";
extras: {
command: string;
hidden?: boolean;
+2
View File
@@ -45,6 +45,7 @@ export type Settings = {
SEARCH_API_KEY?: string;
IS_NEW_USER?: boolean;
MCP_CONFIG?: MCPConfig;
EMAIL?: string;
};
export type ApiSettings = {
@@ -68,6 +69,7 @@ export type ApiSettings = {
sse_servers: (string | MCPSSEServer)[];
stdio_servers: MCPStdioServer[];
};
email?: string;
};
export type PostSettings = Settings & {
+4 -1
View File
@@ -16,5 +16,8 @@ export const generateAuthUrl = (identityProvider: string, requestUrl: URL) => {
authUrl = `auth.${requestUrl.hostname}`;
}
const scope = "openid email profile"; // OAuth scope - not user-facing
return `https://${authUrl}/realms/allhands/protocol/openid-connect/auth?client_id=allhands&kc_idp_hint=${identityProvider}&response_type=code&redirect_uri=${encodeURIComponent(redirectUri)}&scope=${encodeURIComponent(scope)}&state=${encodeURIComponent(requestUrl.href)}`;
const separator = requestUrl.search ? "&" : "?";
const cleanHref = requestUrl.href.replace(/\/$/, "");
const state = `${cleanHref}${separator}login_method=${identityProvider}`;
return `https://${authUrl}/realms/allhands/protocol/openid-connect/auth?client_id=allhands&kc_idp_hint=${identityProvider}&response_type=code&redirect_uri=${encodeURIComponent(redirectUri)}&scope=${encodeURIComponent(scope)}&state=${encodeURIComponent(state)}`;
};
+4 -5
View File
@@ -11,10 +11,11 @@ triggers:
You have access to an environment variable, `GITHUB_TOKEN`, which allows you to interact with
the GitHub API.
<IMPORTANT>
You can use `curl` with the `GITHUB_TOKEN` to interact with GitHub's API.
ALWAYS use the GitHub API for operations instead of a web browser.
To open a pull request, always use the `create_pr` tool
ALWAYS use the `create_pr` tool to open a pull request
</IMPORTANT>
If you encounter authentication issues when pushing to GitHub (such as password prompts or permission errors), the old token may have expired. In such case, update the remote URL to include the current token: `git remote set-url origin https://${GITHUB_TOKEN}@github.com/username/repo.git`
@@ -26,11 +27,9 @@ Here are some instructions for pushing, but ONLY do this if the user asks you to
* Once you've created your own branch or a pull request, continue to update it. Do NOT create a new one unless you are explicitly asked to. Update the PR title and description as necessary, but don't change the branch name.
* Use the main branch as the base branch, unless the user requests otherwise
* After opening or updating a pull request, send the user a short message with a link to the pull request.
* Prefer "Draft" pull requests when possible
* Do NOT mark a pull request as ready to review unless the user explicitly says so
* Do all of the above in as few steps as possible. E.g. you could open a PR with one step by running the following bash commands:
* Do all of the above in as few steps as possible. E.g. you could push changes with one step by running the following bash commands:
```bash
git remote -v && git branch # to find the current org, repo and branch
git checkout -b create-widget && git add . && git commit -m "Create widget" && git push -u origin create-widget
# Then use the MCP tool to create the PR instead of directly using the GitHub API
```
+4 -5
View File
@@ -11,10 +11,11 @@ triggers:
You have access to an environment variable, `GITLAB_TOKEN`, which allows you to interact with
the GitLab API.
<IMPORTANT>
You can use `curl` with the `GITLAB_TOKEN` to interact with GitLab's API.
ALWAYS use the GitLab API for operations instead of a web browser.
To open a merge request, always use the `create_mr` tool
ALWAYS use the `create_mr` tool to open a merge request
</IMPORTANT>
If you encounter authentication issues when pushing to GitLab (such as password prompts or permission errors), the old token may have expired. In such case, update the remote URL to include the current token: `git remote set-url origin https://oauth2:${GITLAB_TOKEN}@gitlab.com/username/repo.git`
@@ -26,10 +27,8 @@ Here are some instructions for pushing, but ONLY do this if the user asks you to
* Once you've created your own branch or a merge request, continue to update it. Do NOT create a new one unless you are explicitly asked to. Update the PR title and description as necessary, but don't change the branch name.
* Use the main branch as the base branch, unless the user requests otherwise
* After opening or updating a merge request, send the user a short message with a link to the merge request.
* Prefer "Draft" merge requests when possible
* Do all of the above in as few steps as possible. E.g. you could open a PR with one step by running the following bash commands:
* Do all of the above in as few steps as possible. E.g. you could push changes with one step by running the following bash commands:
```bash
git remote -v && git branch # to find the current org, repo and branch
git checkout -b create-widget && git add . && git commit -m "Create widget" && git push -u origin create-widget
# Then use the MCP tool to create the MR instead of directly using the GitLab API
```
@@ -266,5 +266,6 @@ class CodeActAgent(Agent):
def response_to_actions(self, response: 'ModelResponse') -> list['Action']:
return codeact_function_calling.response_to_actions(
response, mcp_tool_names=list(self.mcp_tools.keys()),
response,
mcp_tool_names=list(self.mcp_tools.keys()),
)
@@ -5,14 +5,13 @@ This is similar to the functionality of `CodeActResponseParser`.
import json
from litellm import (
ChatCompletionToolParam,
ModelResponse,
)
from openhands.agenthub.codeact_agent.tools import FinishTool
from openhands.agenthub.codeact_agent.function_calling import combine_thought
from openhands.agenthub.codeact_agent.tools import FinishTool
from openhands.agenthub.loc_agent.tools import (
SearchEntityTool,
SearchRepoTool,
@@ -32,7 +31,8 @@ from openhands.events.tool import ToolCallMetadata
def response_to_actions(
response: ModelResponse, mcp_tool_names: list[str] | None = None,
response: ModelResponse,
mcp_tool_names: list[str] | None = None,
) -> list[Action]:
actions: list[Action] = []
assert len(response.choices) == 1, 'Only one choice is supported for now'
@@ -87,7 +87,7 @@ def response_to_actions(
raise FunctionCallNotExistsError(
f'Tool {tool_call.function.name} is not registered. (arguments: {arguments}). Please check the tool name and retry with an existing tool.'
)
# We only add thought to the first action
if i == 0:
action = combine_thought(action, thought)
@@ -106,7 +106,7 @@ def response_to_actions(
wait_for_response=True,
)
)
# Add response id to actions
# This will ensure we can match both actions without tool calls (e.g. MessageAction)
# and actions with tool calls (e.g. CmdRunAction, IPythonRunCellAction, etc.)
@@ -116,7 +116,7 @@ def response_to_actions(
assert len(actions) >= 1
return actions
def get_tools() -> list[ChatCompletionToolParam]:
tools = [FinishTool]
+5 -5
View File
@@ -1,13 +1,12 @@
from openhands.agenthub.codeact_agent import CodeActAgent
from typing import TYPE_CHECKING
import openhands.agenthub.loc_agent.function_calling as locagent_function_calling
from openhands.agenthub.codeact_agent import CodeActAgent
from openhands.core.config import AgentConfig
from openhands.core.logger import openhands_logger as logger
from openhands.llm.llm import LLM
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from openhands.events.action import Action
from openhands.llm.llm import ModelResponse
@@ -35,5 +34,6 @@ class LocAgent(CodeActAgent):
def response_to_actions(self, response: 'ModelResponse') -> list['Action']:
return locagent_function_calling.response_to_actions(
response, mcp_tool_names=list(self.mcp_tools.keys()),
response,
mcp_tool_names=list(self.mcp_tools.keys()),
)
+5 -5
View File
@@ -25,7 +25,7 @@ from openhands.cli.utils import (
write_to_file,
)
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
)
from openhands.core.schema import AgentState
from openhands.events import EventSource
@@ -42,7 +42,7 @@ async def handle_commands(
event_stream: EventStream,
usage_metrics: UsageMetrics,
sid: str,
config: AppConfig,
config: OpenHandsConfig,
current_dir: str,
settings_store: FileSettingsStore,
) -> tuple[bool, bool, bool]:
@@ -105,7 +105,7 @@ def handle_help_command() -> None:
async def handle_init_command(
config: AppConfig, event_stream: EventStream, current_dir: str
config: OpenHandsConfig, event_stream: EventStream, current_dir: str
) -> tuple[bool, bool]:
REPO_MD_CREATE_PROMPT = """
Please explore this repository. Create the file .openhands/microagents/repo.md with:
@@ -166,7 +166,7 @@ def handle_new_command(
async def handle_settings_command(
config: AppConfig,
config: OpenHandsConfig,
settings_store: FileSettingsStore,
) -> None:
display_settings(config)
@@ -264,7 +264,7 @@ async def init_repository(current_dir: str) -> bool:
return init_repo
def check_folder_security_agreement(config: AppConfig, current_dir: str) -> bool:
def check_folder_security_agreement(config: OpenHandsConfig, current_dir: str) -> bool:
# Directories trusted by user for the CLI to use as workspace
# Config from ~/.openhands/config.toml overrides the app config
+3 -3
View File
@@ -30,7 +30,7 @@ from openhands.cli.utils import (
from openhands.controller import AgentController
from openhands.controller.agent import Agent
from openhands.core.config import (
AppConfig,
OpenHandsConfig,
parse_arguments,
setup_config_from_args,
)
@@ -103,7 +103,7 @@ async def cleanup_session(
async def run_session(
loop: asyncio.AbstractEventLoop,
config: AppConfig,
config: OpenHandsConfig,
settings_store: FileSettingsStore,
current_dir: str,
task_content: str | None = None,
@@ -334,7 +334,7 @@ async def main(loop: asyncio.AbstractEventLoop) -> None:
logger.setLevel(logging.WARNING)
# Load config from toml and override with command line arguments
config: AppConfig = setup_config_from_args(args)
config: OpenHandsConfig = setup_config_from_args(args)
# Load settings from Settings Store
# TODO: Make this generic?
+4 -4
View File
@@ -18,7 +18,7 @@ from openhands.cli.utils import (
organize_models_and_providers,
)
from openhands.controller.agent import Agent
from openhands.core.config import AppConfig
from openhands.core.config import OpenHandsConfig
from openhands.core.config.condenser_config import NoOpCondenserConfig
from openhands.core.config.utils import OH_DEFAULT_AGENT
from openhands.memory.condenser.impl.llm_summarizing_condenser import (
@@ -29,7 +29,7 @@ from openhands.storage.settings.file_settings_store import FileSettingsStore
from openhands.utils.llm import get_supported_llm_models
def display_settings(config: AppConfig) -> None:
def display_settings(config: OpenHandsConfig) -> None:
llm_config = config.get_llm_config()
advanced_llm_settings = True if llm_config.base_url else False
@@ -145,7 +145,7 @@ def save_settings_confirmation() -> bool:
async def modify_llm_settings_basic(
config: AppConfig, settings_store: FileSettingsStore
config: OpenHandsConfig, settings_store: FileSettingsStore
) -> None:
model_list = get_supported_llm_models(config)
organized_models = organize_models_and_providers(model_list)
@@ -243,7 +243,7 @@ async def modify_llm_settings_basic(
async def modify_llm_settings_advanced(
config: AppConfig, settings_store: FileSettingsStore
config: OpenHandsConfig, settings_store: FileSettingsStore
) -> None:
session = PromptSession(key_bindings=kb_cancel())
+2 -2
View File
@@ -27,7 +27,7 @@ from prompt_toolkit.styles import Style
from prompt_toolkit.widgets import Frame, TextArea
from openhands import __version__
from openhands.core.config import AppConfig
from openhands.core.config import OpenHandsConfig
from openhands.core.schema import AgentState
from openhands.events import EventSource, EventStream
from openhands.events.action import (
@@ -180,7 +180,7 @@ def display_initial_user_prompt(prompt: str) -> None:
# Prompt output display functions
def display_event(event: Event, config: AppConfig) -> None:
def display_event(event: Event, config: OpenHandsConfig) -> None:
global streaming_output_text_area
with print_lock:
if isinstance(event, Action):
+19 -14
View File
@@ -5,7 +5,7 @@ import copy
import os
import time
import traceback
from typing import Callable, ClassVar
from typing import Callable
import litellm # noqa
from litellm.exceptions import ( # noqa
@@ -61,6 +61,7 @@ from openhands.events.action import (
)
from openhands.events.action.agent import CondensationAction, RecallAction
from openhands.events.event import Event
from openhands.events.event_filter import EventFilter
from openhands.events.observation import (
AgentDelegateObservation,
AgentStateChangedObservation,
@@ -93,12 +94,6 @@ class AgentController:
delegate: 'AgentController | None' = None
_pending_action_info: tuple[Action, float] | None = None # (action, timestamp)
_closed: bool = False
filter_out: ClassVar[tuple[type[Event], ...]] = (
NullAction,
NullObservation,
ChangeAgentStateAction,
AgentStateChangedObservation,
)
_cached_first_user_message: MessageAction | None = None
def __init__(
@@ -150,6 +145,18 @@ class AgentController:
EventStreamSubscriber.AGENT_CONTROLLER, self.on_event, self.id
)
# filter out events that are not relevant to the agent
# so they will not be included in the agent history
self.agent_history_filter = EventFilter(
exclude_types=(
NullAction,
NullObservation,
ChangeAgentStateAction,
AgentStateChangedObservation,
),
exclude_hidden=True,
)
# state from the previous session, state from a parent agent, or a fresh state
self.set_initial_state(
state=initial_state,
@@ -219,12 +226,11 @@ class AgentController:
else self.event_stream.get_latest_event_id()
)
self.state.history = list(
self.event_stream.get_events(
self.event_stream.search_events(
start_id=start_id,
end_id=end_id,
reverse=False,
filter_out_type=self.filter_out,
filter_hidden=True,
filter=self.agent_history_filter,
)
)
@@ -406,7 +412,7 @@ class AgentController:
return
# if the event is not filtered out, add it to the history
if not any(isinstance(event, filter_type) for filter_type in self.filter_out):
if self.agent_history_filter.include(event):
self.state.history.append(event)
if isinstance(event, Action):
@@ -1091,12 +1097,11 @@ class AgentController:
# Get rest of history
events_to_add = list(
self.event_stream.get_events(
self.event_stream.search_events(
start_id=start_id,
end_id=end_id,
reverse=False,
filter_out_type=self.filter_out,
filter_hidden=True,
filter=self.agent_history_filter,
)
)
events.extend(events_to_add)

Some files were not shown because too many files have changed in this diff Show More