Compare commits

...

127 Commits

Author SHA1 Message Date
openhands
9c061d178b refactor(auth): simplify OSS authorization to bare minimum
OSS module (openhands/server/auth/):
- Permission enum only
- Single no-op require_permission() that returns user_id
- No roles, no ROLE_PERMISSIONS, no helper methods

Enterprise module (enterprise/server/auth/authorization.py):
- Imports Permission from OSS module
- Defines RoleName enum and ROLE_PERMISSIONS locally
- Implements all authorization logic: require_permission(),
  require_org_role(), has_permission(), get_role_permissions(), etc.

Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-17 01:15:08 +00:00
openhands
cc7e002ffe feat(auth): add OSS no-op authorization with SAAS override
- Create openhands/server/auth/ module with no-op authorization
  - Permission and RoleName enums defined in OSS module
  - ROLE_PERMISSIONS mapping defined in OSS module
  - No-op require_permission() and require_org_role() dependencies
  - All authorization checks pass in OSS mode

- Update enterprise authorization to import from OSS module
  - Import Permission, RoleName, ROLE_PERMISSIONS from OSS
  - Import get_role_permissions() helper from OSS
  - Override require_permission() and require_org_role() with real checks
  - Enterprise performs actual permission validation in SAAS mode

This allows the same authorization decorators to be used in both modes:
- OSS mode: All checks pass (no-op)
- SAAS mode: Real permission checks are enforced

Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-16 23:44:39 +00:00
openhands
d058b64597 feat(auth): implement permission-based authorization
- Add Permission enum with 15 granular permissions
- Add RoleName enum for owner, admin, member roles
- Add ROLE_PERMISSIONS mapping with correct permission sets per role
- Add require_permission() FastAPI dependency for permission-based access control
- Add helper functions: get_role_permissions(), has_permission()

Permission mapping:
- Member: manage_secrets, manage_mcp, manage_integrations,
  manage_application_settings, manage_api_keys, view_llm_settings
- Admin: All member permissions + edit_llm_settings, view_billing,
  add_credits, invite_user_to_organization, change_user_role:member,
  change_user_role:admin
- Owner: All admin permissions + change_user_role:owner,
  change_organization_name, delete_organization

Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-16 23:09:16 +00:00
Chuck Butkus
954c922a98 Merge branch 'main' into feature/role-based-authorization 2026-02-16 14:23:31 -05:00
mamoodi
663ace4b39 Add saas-rel* branch pattern to ghcr-build workflow (#12888)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-16 12:27:37 -05:00
Hiep Le
2d085a6e0a fix(frontend): add auto-scroll when new messages arrive in chat (#12885) 2026-02-16 23:46:14 +07:00
Hiep Le
8b7112abe8 refactor(frontend): hide planning preview component when plan content is empty (#12879) 2026-02-16 18:35:20 +07:00
Hiep Le
34547ba947 fix(backend): enable byor key export after purchasing credits (#12862) 2026-02-16 17:02:06 +07:00
Graham Neubig
5f958ab60d fix: suppress alembic INFO logs before import to prevent Datadog misclassification (#12691)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-12 14:32:39 -05:00
Hiep Le
d7656bf1c9 refactor(backend): rename user role to member across the system (#12853) 2026-02-13 00:45:47 +07:00
Tim O'Farrell
2bc107564c Support list_files and get_trajectory for nested conversation managers (#12850)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: OpenHands Bot <contact@all-hands.dev>
2026-02-12 10:39:00 -07:00
Tim O'Farrell
85eb1e1504 Check event types before making expensive API calls in GitHub webhook handler (#12819)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-12 09:33:59 -07:00
OpenHands Bot
cd235cc8c7 Bump SDK packages to v1.11.4 (#12839)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Tim O'Farrell <tofarr@gmail.com>
2026-02-11 10:55:46 -07:00
Graham Neubig
40f52dfabc Use lowercase minimax-m2.5 for consistency (#12840)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-12 01:29:17 +08:00
Hiep Le
bab7bf85e8 fix(backend): prevent org deletion from setting current_org_id to NULL (#12817) 2026-02-12 00:15:21 +07:00
Hiep Le
c856537f65 refactor(backend): update the patch organization api to support organization name updates (#12834) 2026-02-12 00:08:43 +07:00
Graham Neubig
736f5b2255 Add MiniMax-M2.5 model support (#12835)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-11 16:57:22 +00:00
chuckbutkus
c1d9d11772 Log all exceptions in get_user() when authentication fails (#12836)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: OpenHands Bot <contact@all-hands.dev>
2026-02-11 11:49:13 -05:00
sp.wack
85244499fe fix(frontend): performance and loading state bugs (#12821) 2026-02-11 15:34:52 +00:00
Hiep Le
c55084e223 fix(backend): read RECAPTCHA_SITE_KEY from environment in V1 web client config (#12830) 2026-02-11 18:59:52 +07:00
Tim O'Farrell
e3bb75deb4 fix(enterprise): use poetry.lock for reproducible dependency builds (#12820)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: OpenHands Bot <contact@all-hands.dev>
2026-02-11 04:51:12 -07:00
Hiep Le
1948200762 chore: update sdk to the latest version (#12811) 2026-02-11 12:57:08 +07:00
Tim O'Farrell
affe0af361 Add debug logging for sandbox startup health checks (#12814)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-10 07:23:43 -07:00
Hiep Le
f20c956196 feat(backend): implement org member patch api (#12800) 2026-02-10 20:01:24 +07:00
Alexander Grattan
4a089a3a0d fix(docs): update Gray Swan API links and onboarding instructions in security README (#12809) 2026-02-10 10:14:49 +00:00
Hiep Le
aa0b2d0b74 feat(backend): add api for switching between orgs (#12799) 2026-02-10 14:22:52 +07:00
Hiep Le
bef9b80b9d fix(frontend): add missing border radius to conversation loading on first load (#12796) 2026-02-09 21:36:07 +07:00
Graham Neubig
c4a90b1f89 Fix Resend ValidationError by adding email validation (#12511)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-08 09:47:39 -05:00
sp.wack
0d13c57d9f feat(backend): org get me route (#12760)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: hieptl <hieptl.developer@gmail.com>
2026-02-07 16:11:25 +07:00
Graham Neubig
b3422f1275 Add PR Review by OpenHands workflow (#12784)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-06 17:26:16 -05:00
Xingyao Wang
f139a9970b feat: add SANDBOX_STARTUP_GRACE_SECONDS env var for configurable startup timeout (#12741)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-07 06:12:29 +08:00
Jamie Chicago
54d156122c Add automated PR review workflow using OpenHands (#12698)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Graham Neubig <neubig@gmail.com>
2026-02-06 19:02:55 +00:00
Tim O'Farrell
ac072bf686 feat(frontend): change alert banner from solid background to border style (#12783)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-06 18:05:29 +00:00
Hiep Le
a53812c029 feat(backend): develop delete /api/organizations/{orgid}/members/{userid} api (#12734) 2026-02-07 00:50:47 +07:00
Tim O'Farrell
1d1c0925b5 refactor: Move check_byor_export_enabled to OrgService and add tests (PR #12753 followup) (#12782)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-06 17:03:03 +00:00
Hiep Le
872f41e3c0 feat(backend): implement get /api/organizations/{orgId}/members api (#12735) 2026-02-06 23:47:30 +07:00
Tim O'Farrell
d43ff82534 feat: Add BYOR export flag to org for LLM key access control (#12753)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: hieptl <hieptl.developer@gmail.com>
2026-02-06 09:30:12 -07:00
huangkevin-apr
8cd8c011b2 fix(a11y): Add aria-label to Sidebar component (#12728) 2026-02-06 22:32:52 +07:00
Tim O'Farrell
5c68b10983 (Frontend) Migrate to new /api/v1/web-client/config endpoint (#12479)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: hieptl <hieptl.developer@gmail.com>
2026-02-06 08:31:40 -07:00
Graham Neubig
a97fad1976 fix: Add PostHog error tracking for V1 AgentErrorEvent and ConversationErrorEvent (#12543)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-06 09:51:01 -05:00
Graham Neubig
4c3542a91c fix: use appropriate log level for webhook installation results (#12493)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-06 09:01:37 -05:00
Tim O'Farrell
f460057f58 chore: add deprecation notices to all runtime directory files (#12772)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-06 05:15:02 -07:00
MkDev11
4fa2ad0f47 fix: add exponential backoff retry for env var export when bash session is busy (#12748)
Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com>
2026-02-06 05:07:17 -07:00
Hiep Le
dd8be12809 feat(backend): return is_personal field in OrgResponse (#12777) 2026-02-06 19:01:06 +07:00
Tim O'Farrell
89475095d9 Preload callback processor class to prevent Pydantic Deserialization Error (#12776) 2026-02-06 04:29:28 -07:00
Tim O'Farrell
05d5f8848a Fix V1 GitHub conversations failing to clone repository (#12775)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-06 03:08:55 -07:00
Hiep Le
ee2885eb0b feat: store plan.md file in appropriate configuration folders (#12713) 2026-02-06 16:09:39 +07:00
Tim O'Farrell
545257f870 Refactor: Add LLM provider utilities and improve API base URL detection (#12766)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-05 14:22:32 -07:00
Hiep Le
b23ab33a01 chore: update sdk to the latest version (#12762) 2026-02-06 00:25:11 +07:00
sp.wack
a9ede73391 fix(backend): resolve missing email and display name for user identity tracking (#12719) 2026-02-05 16:50:33 +00:00
chuckbutkus
634c2439b4 Fix key gen again (#12752)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-05 11:45:10 -05:00
Hiep Le
a1989a40b3 refactor(frontend): remove border and border radius from ConversationLoading (#12756) 2026-02-05 21:50:07 +07:00
Saurya Velagapudi
e38f1283ea feat(recaptcha): add user_id and email to assessment log (#12749)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-04 17:58:44 -08:00
Tim O'Farrell
07eb791735 Remove flaky test_bash_session.py (#12739)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-04 11:55:22 -07:00
Saurya Velagapudi
c355c4819f feat(recaptcha): add assessment name to logging and AssessmentResult (#12744)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-04 09:30:02 -08:00
sp.wack
9d8e4c44cc fix(frontend): fix cross-domain PostHog tracking param names and persist bootstrap IDs across OAuth redirects (#12729) 2026-02-04 16:15:53 +04:00
openhands
c9cf142949 refactor: use Role class for role hierarchy in authorization
- Remove hardcoded OrgRole enum and ROLE_HIERARCHY dictionary
- Import Role class from storage.role module
- Update get_user_org_role() to return Role object instead of string
- Update has_required_role() to compare roles using rank field
- Lower rank = higher position in hierarchy (e.g., rank 1 > rank 2 > rank 3)
- Update require_org_role() to accept role name string and fetch Role from database
- Add error handling for missing role in database

Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-04 04:18:15 +00:00
Chuck Butkus
7e0b6a39c1 Merge branch 'main' into feature/role-based-authorization 2026-02-03 22:43:19 -05:00
Hiep Le
25cc55e558 chore: update sdk to the latest version (#12737) 2026-02-04 01:20:13 +07:00
chuckbutkus
0e825c38d7 APP-443: Fix key generation (#12726)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: tofarr <tofarr@gmail.com>
2026-02-03 17:50:40 +00:00
Graham Neubig
ce04e70b5b fix: BYOR to OpenHands provider switch auth error (#12725)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-03 09:26:56 -07:00
Chuck Butkus
268fa43ba9 Lint fix 2026-02-03 02:05:40 -05:00
openhands
ff14104db4 feat: Add role-based authorization for org-scoped endpoints
Implement role-based authorization using FastAPI dependencies that check
user roles (owner, admin, user) within organizations.

Changes:
- Add enterprise/server/auth/authorization.py with:
  - OrgRole enum for role types
  - Role hierarchy (owner > admin > user)
  - get_user_org_role() to retrieve user's role in an org
  - has_required_role() to check role hierarchy
  - require_org_role() dependency factory
  - Convenience dependencies: require_org_user, require_org_admin, require_org_owner

- Update enterprise/server/routes/orgs.py:
  - GET /{org_id}: require_org_user (any member can view)
  - PATCH /{org_id}: require_org_admin (admin or owner can update)
  - DELETE /{org_id}: require_org_owner (only owner can delete)

- Add comprehensive unit tests for authorization module

Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-03 06:59:35 +00:00
Tim O'Farrell
7b0589ad40 Add GitHub and Replicated app slugs to web client config (#12724) 2026-02-02 15:53:52 -07:00
mamoodi
682465a862 Release 1.3 (#12715) 2026-02-02 16:17:01 -05:00
Tim O'Farrell
1bb4c844d4 Fix runtime status error on conversation resume (#12718)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-02 21:04:45 +00:00
Neha Prasad
d6c11fe517 fix selected repo disappearing in pen repository dropdown (#12702)
Co-authored-by: Tim O'Farrell <tofarr@gmail.com>
2026-02-02 19:07:48 +00:00
Tim O'Farrell
b088d4857e Improve batch_get_app_conversations UUID handling (#12711)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-02 07:25:07 -07:00
Tim O'Farrell
0f05898d55 Deprecate V0 endpoints now handled by agent server (#12710)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-02 07:25:00 -07:00
Hiep Le
d1f0a01a57 feat(frontend): add silent WebSocket recovery for V1 conversations (#12677) 2026-02-02 19:53:33 +07:00
Hiep Le
f5a9d28999 feat(frontend): add shine text effect to Plan Preview during streaming (planning agent) (#12676) 2026-02-02 14:31:31 +07:00
Graham Neubig
afa0417608 Remove evaluation directory and benchmarking dependencies (#12666)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-02-01 11:39:29 -08:00
chuckbutkus
e688fba761 Fix check for already migrated (#12700) 2026-01-30 19:31:46 -05:00
Tim O'Farrell
d1ec5cbdf6 Fix litellm migration max_budget issue and add logging (#12697)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-30 16:35:31 -07:00
Jamie Chicago
f42625f789 Improve issue templates with best practices and clear expectations (#12632)
Co-authored-by: jamiechicago312 <jamiechicago312@users.noreply.github.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Engel Nyst <engel.nyst@gmail.com>
2026-01-30 21:27:44 +01:00
Graham Neubig
fe28519677 feat(frontend): add gpt-5.2 and gpt-5.2-codex models, remove gpt-5 models (#12639)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-30 12:15:47 -05:00
Tim O'Farrell
e62ceafa4a Cleaner Logs (#12579)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-30 15:25:36 +00:00
Tim O'Farrell
0b8c69fad2 Fix for issue where stats are not updated for conversations (#12688)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-30 08:12:38 -07:00
Hiep Le
37d9b672a4 fix(frontend): prevent duplicate API calls when sub-conversation becomes ready (planning agent) (#12673) 2026-01-30 15:54:51 +07:00
Hiep Le
c8b867a634 fix(frontend): persist selected agent mode across page refresh (planning agent) (#12672) 2026-01-30 15:39:36 +07:00
Hiep Le
59834beba7 fix(frontend): prevent duplicate sub-conversation creation on page refresh (planning agent) (#12645) 2026-01-30 15:25:41 +07:00
Hiep Le
d2eced9cff feat(frontend): handle the build button for the planning agent (planning agent) (#12644) 2026-01-30 15:25:16 +07:00
Hiep Le
7836136ff8 feat(frontend): disable the build button while the agent is running (planning agent) (#12643) 2026-01-30 15:13:05 +07:00
chuckbutkus
fdb04dfe5d Add GitLab provider check to schedule_gitlab_repo_sync (#12680)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-29 23:56:36 -05:00
Tim O'Farrell
3d4cb89441 fix(frontend): Support V1 conversations in MetricsModal (#12678)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-29 15:03:06 -07:00
Tim O'Farrell
9fb9efd3d2 Refactor LiteLLM key updates to use generic user key API (#12664)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-29 09:53:01 -07:00
Engel Nyst
5511c01c2e chore: clarify draft guidance in PR template (#12670)
Co-authored-by: smolpaws <engel@enyst.org>
Co-authored-by: mamoodi <mamoodiha@gmail.com>
2026-01-29 14:17:35 +00:00
Engel Nyst
02825fb5bb Cleanup CLI directory (#12669) 2026-01-29 15:14:50 +01:00
HeyItsChloe
876e773589 chore(frontend): convo tab only renders active/selected tab (#12570)
Co-authored-by: Chloe <chloe@openhands.com>
Co-authored-by: hieptl <hieptl.developer@gmail.com>
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-29 16:47:41 +07:00
MkDev11
9e1ae86191 fix: hide Notes from task list when notes are empty (#12668)
Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com>
2026-01-29 16:13:23 +07:00
Hiep Le
df47b7b79d feat(frontend): ensure the planner tab opens when the view button is selected (#12621) 2026-01-29 13:12:09 +07:00
chuckbutkus
7d1c105b55 Move settings to dict to after openhands (#12663) 2026-01-28 23:45:01 +00:00
Tim O'Farrell
db6a9e8895 Fix broken key migration by decrypting legacy encrypted keys before LiteLLM update (#12657) 2026-01-28 15:09:50 -07:00
Hiep Le
d76ac44dc3 refactor(frontend): reduce heading text size for plan preview content (#12620) 2026-01-29 00:30:40 +07:00
MkDev11
c483c80a3c feat: add host network support for V1 DockerSandboxService (#12445)
Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com>
Co-authored-by: hieptl <hieptl.developer@gmail.com>
2026-01-28 09:55:38 -07:00
chuckbutkus
570ab904f6 Fix UserSettings creation from Org tables (#12635)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: tofarr <tofarr@gmail.com>
2026-01-28 09:35:05 -07:00
sp.wack
00a74731ae chore: refresh frontend lockfile (#12619) 2026-01-28 13:03:42 +04:00
Tim O'Farrell
102095affb Add downgrade script and methods for reverting user migration (#12629)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: chuckbutkus <chuck@all-hands.dev>
2026-01-27 14:41:34 -07:00
John-Mason P. Shackelford
b6ce45b474 feat(app_server): start conversations with remote plugins via REST API (#12338)
- Add `PluginSpec` model with plugin configuration parameters extending SDK's `PluginSource`
- Extend app-conversations API to accept plugins specification in `AppConversationStartRequest`
- Propagate plugin source, ref, and repo_path to agent server's `StartConversationRequest`
- Include plugin parameters in initial conversation message for agent context

Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-27 16:26:38 -05:00
Tim O'Farrell
11c87caba4 fix(backend): fix callback state not persisting due to dual-session conflict (#12627)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-27 10:02:48 -07:00
Abhay Mishra
b8a608c45e Fix: branch/repo dropdown reset on click (#12501)
Co-authored-by: hieptl <hieptl.developer@gmail.com>
2026-01-27 16:08:08 +07:00
Tim O'Farrell
8a446787be Migrate SDK to 1.10.0 (#12614) 2026-01-27 04:26:06 +00:00
Tim O'Farrell
353124e171 Bump SDK to 1.10.0 (#12613) 2026-01-27 03:50:30 +00:00
chuckbutkus
e9298c89bd Fix org migration (#12612)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-26 20:55:55 -05:00
Hiep Le
29b77be807 fix: revert get_user_by_id_async to use sync session_maker (#12610) 2026-01-27 04:39:07 +07:00
mamoodi
7094835ef0 Fix Pydantic UnsupportedFieldAttributeWarning in Settings model (#12600)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-26 12:30:43 -05:00
mamoodi
7ad0eec325 Change error to warning if TOS not accepted (#12605) 2026-01-26 12:30:00 -05:00
Hiep Le
31d5081163 feat(frontend): display plan preview content (#12504) 2026-01-26 23:19:57 +07:00
Abhay Mishra
250736cb7a fix(frontend): display ThinkAction thought content in V1 UI (#12597)
Co-authored-by: hieptl <hieptl.developer@gmail.com>
2026-01-26 14:03:16 +07:00
Tim O'Farrell
a9bd3a70c9 Fix V0 Integrations (#12584) 2026-01-24 16:53:16 +00:00
Hiep Le
d7436a4af4 fix(backend): asyncsession query error in userstore.get_user_by_id_async (#12586)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-24 23:47:04 +07:00
Tim O'Farrell
f327e76be7 Added explicit expired error (#12580) 2026-01-23 12:49:10 -07:00
Hiep Le
52e39e5d12 fix(backend): unable to export conversation (#12577)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-24 01:06:02 +07:00
Graham Neubig
6c5ef256fd fix: pass userId to EmailVerificationModal in login page (#12573)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: hieptl <hieptl.developer@gmail.com>
2026-01-23 23:24:25 +07:00
Tim O'Farrell
373ade8554 Fix org billing (#12562) 2026-01-23 08:42:50 -07:00
Hiep Le
9d0a19cf8f fix(backend): ensure conversation events are written back to google cloud (#12571) 2026-01-23 22:13:08 +07:00
Rohit Malhotra
d60dd38d78 fix: preserve query params in returnTo during login redirect (#12567)
Co-authored-by: openhands <openhands@all-hands.dev>
2026-01-22 21:31:42 -08:00
Hiep Le
d5ee799670 feat(backend): develop patch /api/organizations/{orgid} api (#12470)
Co-authored-by: rohitvinodmalhotra@gmail.com <rohitvinodmalhotra@gmail.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Chuck Butkus <chuck@all-hands.dev>
2026-01-23 01:29:35 +07:00
Hiep Le
b685fd43dd fix(backend): github proxy state compression (#12387)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Ray Myers <ray.myers@gmail.com>
2026-01-23 00:39:03 +07:00
Hiep Le
0e04f6fdbe feat(backend): develop delete /api/organizations/{orgid} api (#12471)
Co-authored-by: rohitvinodmalhotra@gmail.com <rohitvinodmalhotra@gmail.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Chuck Butkus <chuck@all-hands.dev>
2026-01-23 00:28:55 +07:00
Hiep Le
9c40929197 feat(backend): develop get /api/organizations/{orgid} api (#12274)
Co-authored-by: rohitvinodmalhotra@gmail.com <rohitvinodmalhotra@gmail.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Chuck Butkus <chuck@all-hands.dev>
Co-authored-by: Tim O'Farrell <tofarr@gmail.com>
2026-01-22 23:55:29 +07:00
Hiep Le
af309e8586 feat(backend): develop get /api/organizations api (#12373)
Co-authored-by: rohitvinodmalhotra@gmail.com <rohitvinodmalhotra@gmail.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Chuck Butkus <chuck@all-hands.dev>
2026-01-22 23:32:42 +07:00
sp.wack
cc5d5c2335 test(frontend): add missing HTTP mocks for conversation history preloading (#12549) 2026-01-22 20:30:27 +04:00
Hiep Le
60e668f4a7 fix(backend): application settings are not updating as expected (#12547) 2026-01-22 21:54:52 +07:00
Hiep Le
743f6256a6 feat(backend): load skills from agent server (#12434) 2026-01-22 20:20:50 +07:00
Mohammed Abdulai
a87b4efd41 feat: preload conversation history before websocket connection (#12488)
Co-authored-by: sp.wack <83104063+amanape@users.noreply.github.com>
2026-01-22 12:41:27 +00:00
Tim O'Farrell
730d9970f5 Migrate SDK to 1.9.1 (#12540) 2026-01-21 16:14:27 -07:00
699 changed files with 28857 additions and 72134 deletions

View File

@@ -5,52 +5,113 @@ labels: ['bug']
body:
- type: markdown
attributes:
value: Thank you for taking the time to fill out this bug report. Please provide as much information as possible
to help us understand and address the issue effectively.
value: |
## Thank you for reporting a bug! 🐛
**Please fill out all required fields.** Issues missing critical information (version, installation method, reproduction steps, etc.) will be delayed or closed until complete details are provided.
Clear, detailed reports help us resolve issues faster.
- type: checkboxes
attributes:
label: Is there an existing issue for the same bug? (If one exists, thumbs up or comment on the issue instead).
description: Please check if an issue already exists for the bug you encountered.
label: Is there an existing issue for the same bug?
description: Please search existing issues before creating a new one. If found, react or comment to the duplicate issue instead of making a new one.
options:
- label: I have checked the existing issues.
- label: I have searched existing issues and this is not a duplicate.
required: true
- type: textarea
id: bug-description
attributes:
label: Describe the bug and reproduction steps
description: Provide a description of the issue along with any reproduction steps.
label: Bug Description
description: Clearly describe what went wrong. Be specific and concise.
placeholder: Example - "When I run a Python task, OpenHands crashes after 30 seconds with a connection timeout error."
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: Expected Behavior
description: What did you expect to happen?
placeholder: Example - "OpenHands should execute the Python script and return results."
validations:
required: false
- type: textarea
id: actual-behavior
attributes:
label: Actual Behavior
description: What actually happened?
placeholder: Example - "Connection timed out after 30 seconds, task failed with error code 500."
validations:
required: false
- type: textarea
id: reproduction-steps
attributes:
label: Steps to Reproduce
description: Provide clear, step-by-step instructions to reproduce the bug.
placeholder: |
1. Install OpenHands using Docker
2. Configure with Claude 3.5 Sonnet
3. Run command: `openhands run "write a python script"`
4. Wait 30 seconds
5. Error appears
validations:
required: false
- type: dropdown
id: installation
attributes:
label: OpenHands Installation
label: OpenHands Installation Method
description: How are you running OpenHands?
options:
- Docker command in README
- GitHub resolver
- CLI (uv tool install)
- CLI (executable binary)
- CLI (Docker)
- Local GUI (Docker web interface)
- OpenHands Cloud (app.all-hands.dev)
- SDK (Python library)
- Development workflow
- CLI
- app.all-hands.dev
- Other
default: 0
validations:
required: false
- type: input
id: installation-other
attributes:
label: If you selected "Other", please specify
description: Describe your installation method
placeholder: ex. Custom Kubernetes deployment, pip install from source, etc.
- type: input
id: openhands-version
attributes:
label: OpenHands Version
description: What version of OpenHands are you using?
placeholder: ex. 0.9.8, main, etc.
description: What version are you using? Find this in settings or by running `openhands --version`
placeholder: ex. 0.9.8, main, commit hash, etc.
validations:
required: false
- type: checkboxes
id: version-confirmation
attributes:
label: Version Confirmation
description: Bugs on older versions may already be fixed. Please upgrade before submitting.
options:
- label: "I have confirmed this bug exists on the LATEST version of OpenHands"
required: false
- type: input
id: model-name
attributes:
label: Model Name
description: What model are you using?
placeholder: ex. gpt-4o, claude-3-5-sonnet, openrouter/deepseek-r1, etc.
description: Which LLM model are you using?
placeholder: ex. gpt-4o, claude-3-5-sonnet-20241022, openrouter/deepseek-r1, etc.
validations:
required: false
- type: dropdown
id: os
@@ -60,12 +121,46 @@ body:
- MacOS
- Linux
- WSL on Windows
- Windows (Docker Desktop)
- Other
validations:
required: false
- type: input
id: browser
attributes:
label: Browser (if using web UI)
description: |
If applicable, which browser and version?
placeholder: ex. Chrome 131, Firefox 133, Safari 17.2
- type: textarea
id: logs
attributes:
label: Logs and Error Messages
description: |
**Paste relevant logs, error messages, or stack traces.** Use code blocks (```) for formatting.
LLM logs are in `logs/llm/default/`. Include timestamps if errors occurred at a specific time.
placeholder: |
```
Paste error logs here
```
- type: textarea
id: additional-context
attributes:
label: Logs, Errors, Screenshots, and Additional Context
description: Please provide any additional information you think might help. If you want to share the chat history
you can click the thumbs-down (👎) button above the input field and you will get a shareable link
(you can also click thumbs up when things are going well of course!). LLM logs will be stored in the
`logs/llm/default` folder. Please add any additional context about the problem here.
label: Screenshots and Additional Context
description: |
Add screenshots, videos, runtime environment, or other context that helps explain the issue.
💡 **Share conversation history:** In the OpenHands chat UI, click the 👎 or 👍 button (above the message input) to generate a shareable link to your conversation.
placeholder: Drag and drop screenshots here, paste links, or add additional context.
- type: markdown
attributes:
value: |
---
**Note:** Issues with incomplete information may be closed or deprioritized. Maintainers and community members have limited bandwidth and prioritize well-documented bugs that are easier to reproduce and fix. Thank you for your understanding!

View File

@@ -1,17 +0,0 @@
---
name: Feature Request or Enhancement
about: Suggest an idea for an OpenHands feature or enhancement
title: ''
labels: 'enhancement'
assignees: ''
---
**What problem or use case are you trying to solve?**
**Describe the UX or technical implementation you have in mind**
**Additional context**
### If you find this feature request or enhancement useful, make sure to add a 👍 to the issue

View File

@@ -0,0 +1,105 @@
name: Feature Request or Enhancement
description: Suggest a new feature or improvement for OpenHands
title: '[Feature]: '
labels: ['enhancement']
body:
- type: markdown
attributes:
value: |
## Thank you for suggesting a feature! 💡
**Please provide detailed information.** Vague or low-effort requests may be closed. Well-documented feature requests with strong community support are more likely to be added to the roadmap.
- type: checkboxes
attributes:
label: Is there an existing feature request for this?
description: Please search existing issues and feature requests before creating a new one. If found, react or comment to the duplicate issue instead of making a new one.
options:
- label: I have searched existing issues and feature requests, and this is not a duplicate.
required: true
- type: textarea
id: problem-statement
attributes:
label: Problem or Use Case
description: What problem are you trying to solve? What use case would this feature enable?
placeholder: |
Example - "As a developer working on large codebases, I need to search across multiple files simultaneously. Currently, I have to search file-by-file which is time-consuming and inefficient."
validations:
required: true
- type: textarea
id: proposed-solution
attributes:
label: Proposed Solution
description: Describe your ideal solution. What should this feature do? How should it work?
placeholder: |
Example - "Add a global search feature that allows searching across all files in the workspace. Results should show file name, line number, and context around matches. Include regex support and filtering options."
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives Considered
description: Have you considered any alternative solutions or workarounds? What are their limitations?
placeholder: Example - "I tried using grep in the terminal, but it's not integrated with the UI and doesn't provide click-to-navigate functionality."
- type: dropdown
id: priority
attributes:
label: Priority / Severity
description: How important is this feature to your workflow?
options:
- "Critical - Blocking my work, no workaround available"
- "High - Significant impact on productivity"
- "Medium - Would improve experience"
- "Low - Nice to have"
default: 2
validations:
required: true
- type: dropdown
id: scope
attributes:
label: Estimated Scope
description: To the best of your knowledge, how complex do you think this feature would be to implement?
options:
- "Small - UI tweak, config option, or minor change"
- "Medium - New feature with moderate complexity"
- "Large - Significant feature requiring architecture changes"
- "Unknown - Not sure about the technical complexity"
default: 3
- type: dropdown
id: feature-area
attributes:
label: Feature Area
description: Which part of OpenHands does this feature relate to? If you select "Other", please specify the area in the Additional Context section below.
options:
- "Agent / AI behavior"
- "User Interface / UX"
- "CLI / Command-line interface"
- "File system / Workspace management"
- "Configuration / Settings"
- "Integrations (GitHub, GitLab, etc.)"
- "Performance / Optimization"
- "Documentation"
- "Other"
validations:
required: true
- type: textarea
id: technical-details
attributes:
label: Technical Implementation Ideas (Optional)
description: If you have technical expertise, share implementation ideas, API suggestions, or relevant technical details.
placeholder: |
Example - "Could use ripgrep library for fast search. Expose results via /api/search endpoint. Frontend can use virtualized list for rendering large result sets."
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Add any other context, screenshots, mockups, or examples that help illustrate this feature request.
placeholder: Drag and drop screenshots, mockups, or links here.

View File

@@ -1,4 +1,4 @@
<!-- Ideally you should open a PR when it is ready for review. Draft PRs will not be reviewed -->
<!-- If you are still working on the PR, please mark it as draft. Maintainers will review PRs marked ready for review, which leads to lost time if your PR is actually not ready yet. Keep the PR marked as draft until it is finally ready for review -->
## Summary of PR

View File

@@ -9,6 +9,7 @@ on:
push:
branches:
- main
- "saas-rel-*"
tags:
- "*"
pull_request:

View File

@@ -0,0 +1,127 @@
---
name: PR Review by OpenHands
on:
# Use pull_request_target to allow fork PRs to access secrets when triggered by maintainers
# Security: This workflow runs when:
# 1. A new PR is opened (non-draft), OR
# 2. A draft PR is marked as ready for review, OR
# 3. A maintainer adds the 'review-this' label, OR
# 4. A maintainer requests openhands-agent or all-hands-bot as a reviewer
# Only users with write access can add labels or request reviews, ensuring security.
# The PR code is explicitly checked out for review, but secrets are only accessible
# because the workflow runs in the base repository context
pull_request_target:
types: [opened, ready_for_review, labeled, review_requested]
permissions:
contents: read
pull-requests: write
issues: write
jobs:
pr-review:
# Run when one of the following conditions is met:
# 1. A new non-draft PR is opened by a trusted contributor, OR
# 2. A draft PR is converted to ready for review by a trusted contributor, OR
# 3. 'review-this' label is added, OR
# 4. openhands-agent or all-hands-bot is requested as a reviewer
# Note: FIRST_TIME_CONTRIBUTOR PRs require manual trigger via label/reviewer request
if: |
(github.event.action == 'opened' && github.event.pull_request.draft == false && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR') ||
(github.event.action == 'ready_for_review' && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR') ||
github.event.label.name == 'review-this' ||
github.event.requested_reviewer.login == 'openhands-agent' ||
github.event.requested_reviewer.login == 'all-hands-bot'
concurrency:
group: pr-review-${{ github.event.pull_request.number }}
cancel-in-progress: true
runs-on: blacksmith-4vcpu-ubuntu-2404
env:
LLM_MODEL: litellm_proxy/claude-sonnet-4-5-20250929
LLM_BASE_URL: https://llm-proxy.app.all-hands.dev
# PR context will be automatically provided by the agent script
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_TITLE: ${{ github.event.pull_request.title }}
PR_BODY: ${{ github.event.pull_request.body }}
PR_BASE_BRANCH: ${{ github.event.pull_request.base.ref }}
PR_HEAD_BRANCH: ${{ github.event.pull_request.head.ref }}
REPO_NAME: ${{ github.repository }}
steps:
- name: Checkout software-agent-sdk repository
uses: actions/checkout@v5
with:
repository: OpenHands/software-agent-sdk
path: software-agent-sdk
- name: Checkout PR repository
uses: actions/checkout@v5
with:
# When using pull_request_target, explicitly checkout the PR branch
# This ensures we review the actual PR code (including fork PRs)
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
# Security: Don't persist credentials to prevent untrusted PR code from using them
persist-credentials: false
path: pr-repo
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.13'
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Install GitHub CLI
run: |
# Install GitHub CLI for posting review comments
sudo apt-get update
sudo apt-get install -y gh
- name: Install OpenHands dependencies
run: |
# Install OpenHands SDK and tools from local checkout
uv pip install --system ./software-agent-sdk/openhands-sdk ./software-agent-sdk/openhands-tools
- name: Check required configuration
env:
LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
run: |
if [ -z "$LLM_API_KEY" ]; then
echo "Error: LLM_API_KEY secret is not set."
exit 1
fi
echo "PR Number: $PR_NUMBER"
echo "PR Title: $PR_TITLE"
echo "Repository: $REPO_NAME"
echo "LLM model: $LLM_MODEL"
if [ -n "$LLM_BASE_URL" ]; then
echo "LLM base URL: $LLM_BASE_URL"
fi
- name: Run PR review
env:
LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
GITHUB_TOKEN: ${{ secrets.ALLHANDS_BOT_GITHUB_PAT }}
LMNR_PROJECT_API_KEY: ${{ secrets.LMNR_SKILLS_API_KEY }}
run: |
# Change to the PR repository directory so agent can analyze the code
cd pr-repo
# Run the PR review script from the software-agent-sdk checkout
uv run python ../software-agent-sdk/examples/03_github_workflows/02_pr_review/agent_script.py
- name: Upload logs as artifact
uses: actions/upload-artifact@v5
if: always()
with:
name: openhands-pr-review-logs
path: |
*.log
output/
retention-days: 7

View File

@@ -6,11 +6,12 @@ Thanks for your interest in contributing to OpenHands! We welcome and appreciate
To understand the codebase, please refer to the README in each module:
- [frontend](./frontend/README.md)
- [evaluation](./evaluation/README.md)
- [openhands](./openhands/README.md)
- [agenthub](./openhands/agenthub/README.md)
- [server](./openhands/server/README.md)
For benchmarks and evaluation, see the [OpenHands/benchmarks](https://github.com/OpenHands/benchmarks) repository.
## Setting up Your Development Environment
We have a separate doc [Development.md](https://github.com/OpenHands/OpenHands/blob/main/Development.md) that tells

View File

@@ -200,7 +200,7 @@ Here's a guide to the important documentation files in the repository:
- [/frontend/README.md](./frontend/README.md): Frontend React application setup and development guide
- [/containers/README.md](./containers/README.md): Information about Docker containers and deployment
- [/tests/unit/README.md](./tests/unit/README.md): Guide to writing and running unit tests
- [/evaluation/README.md](./evaluation/README.md): Documentation for the evaluation framework and benchmarks
- [OpenHands/benchmarks](https://github.com/OpenHands/benchmarks): Documentation for the evaluation framework and benchmarks
- [/skills/README.md](./skills/README.md): Information about the skills architecture and implementation
- [/openhands/server/README.md](./openhands/server/README.md): Server implementation details and API documentation
- [/openhands/runtime/README.md](./openhands/runtime/README.md): Documentation for the runtime environment and execution model

View File

@@ -440,12 +440,6 @@ type = "noop"
#temperature = 0.1
#max_input_tokens = 1024
#################################### Eval ####################################
# Configuration for the evaluation, please refer to the specific evaluation
# plugin for the available options
##############################################################################
########################### Kubernetes #######################################
# Kubernetes configuration when using the Kubernetes runtime
##############################################################################

View File

@@ -8,7 +8,7 @@ services:
container_name: openhands-app-${DATE:-}
environment:
- AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server}
- AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-0fdea73-python}
- AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-31536c8-python}
#- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234} # enable this only if you want a specific non-root sandbox user but you will have to manually adjust permissions of ~/.openhands for this user
- WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace}
ports:

View File

@@ -23,12 +23,23 @@ RUN apt-get update && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Install Python packages with security fixes
RUN /app/.venv/bin/pip install alembic psycopg2-binary cloud-sql-python-connector pg8000 gspread stripe python-keycloak asyncpg sqlalchemy[asyncio] resend tenacity slack-sdk ddtrace "posthog>=6.0.0" "limits==5.2.0" coredis prometheus-client shap scikit-learn pandas numpy google-cloud-recaptcha-enterprise && \
# Update packages with known CVE fixes
/app/.venv/bin/pip install --upgrade \
"mcp>=1.10.0" \
"pillow>=11.3.0"
# Install poetry and export before importing current code.
RUN /app/.venv/bin/pip install poetry poetry-plugin-export
# Install Python dependencies from poetry.lock for reproducible builds
# Copy lock files first for better Docker layer caching
COPY --chown=openhands:openhands enterprise/pyproject.toml enterprise/poetry.lock /tmp/enterprise/
RUN cd /tmp/enterprise && \
# Export only main dependencies with hashes for supply chain security
/app/.venv/bin/poetry export --only main -o requirements.txt && \
# Remove the local path dependency (openhands-ai is already in base image)
sed -i '/^-e /d; /openhands-ai/d' requirements.txt && \
# Install pinned dependencies from lock file
/app/.venv/bin/pip install -r requirements.txt && \
# Cleanup - return to /app before removing /tmp/enterprise
cd /app && \
rm -rf /tmp/enterprise && \
/app/.venv/bin/pip uninstall -y poetry poetry-plugin-export
WORKDIR /app
COPY --chown=openhands:openhands --chmod=770 enterprise .

View File

@@ -0,0 +1,207 @@
#!/usr/bin/env python
"""
This script can be removed once orgs is established - probably after Feb 15 2026
Downgrade script for migrated users.
This script identifies users who have been migrated (already_migrated=True)
and reverts them back to the pre-migration state.
Usage:
# Dry run - just list the users that would be downgraded
python downgrade_migrated_users.py --dry-run
# Downgrade a specific user by their keycloak_user_id
python downgrade_migrated_users.py --user-id <user_id>
# Downgrade all migrated users (with confirmation)
python downgrade_migrated_users.py --all
# Downgrade all migrated users without confirmation (dangerous!)
python downgrade_migrated_users.py --all --no-confirm
"""
import argparse
import asyncio
import sys
# Add the enterprise directory to the path
sys.path.insert(0, '/workspace/project/OpenHands/enterprise')
from server.logger import logger
from sqlalchemy import select, text
from storage.database import session_maker
from storage.user_settings import UserSettings
from storage.user_store import UserStore
def get_migrated_users() -> list[str]:
"""Get list of keycloak_user_ids for users who have been migrated.
This includes:
1. Users with already_migrated=True in user_settings (migrated users)
2. Users in the 'user' table who don't have a user_settings entry (new sign-ups)
"""
with session_maker() as session:
# Get users from user_settings with already_migrated=True
migrated_result = session.execute(
select(UserSettings.keycloak_user_id).where(
UserSettings.already_migrated.is_(True)
)
)
migrated_users = {row[0] for row in migrated_result.fetchall() if row[0]}
# Get users from the 'user' table (new sign-ups won't have user_settings)
# These are users who signed up after the migration was deployed
new_signup_result = session.execute(
text("""
SELECT CAST(u.id AS VARCHAR)
FROM "user" u
WHERE NOT EXISTS (
SELECT 1 FROM user_settings us
WHERE us.keycloak_user_id = CAST(u.id AS VARCHAR)
)
""")
)
new_signups = {row[0] for row in new_signup_result.fetchall() if row[0]}
# Combine both sets
all_users = migrated_users | new_signups
return list(all_users)
async def downgrade_user(user_id: str) -> bool:
"""Downgrade a single user.
Args:
user_id: The keycloak_user_id to downgrade
Returns:
True if successful, False otherwise
"""
try:
result = await UserStore.downgrade_user(user_id)
if result:
print(f'✓ Successfully downgraded user: {user_id}')
return True
else:
print(f'✗ Failed to downgrade user: {user_id}')
return False
except Exception as e:
print(f'✗ Error downgrading user {user_id}: {e}')
logger.exception(
'downgrade_script:error',
extra={'user_id': user_id, 'error': str(e)},
)
return False
async def main():
parser = argparse.ArgumentParser(
description='Downgrade migrated users back to pre-migration state'
)
parser.add_argument(
'--dry-run',
action='store_true',
help='Just list users that would be downgraded, without making changes',
)
parser.add_argument(
'--user-id',
type=str,
help='Downgrade a specific user by keycloak_user_id',
)
parser.add_argument(
'--all',
action='store_true',
help='Downgrade all migrated users',
)
parser.add_argument(
'--no-confirm',
action='store_true',
help='Skip confirmation prompt (use with caution!)',
)
args = parser.parse_args()
# Get list of migrated users
migrated_users = get_migrated_users()
print(f'\nFound {len(migrated_users)} migrated user(s).')
if args.dry_run:
print('\n--- DRY RUN MODE ---')
print('The following users would be downgraded:')
for user_id in migrated_users:
print(f' - {user_id}')
print('\nNo changes were made.')
return
if args.user_id:
# Downgrade a specific user
if args.user_id not in migrated_users:
print(f'\nUser {args.user_id} is not in the migrated users list.')
print('Either the user was not migrated, or the user_id is incorrect.')
return
print(f'\nDowngrading user: {args.user_id}')
if not args.no_confirm:
confirm = input('Are you sure? (yes/no): ')
if confirm.lower() != 'yes':
print('Cancelled.')
return
success = await downgrade_user(args.user_id)
if success:
print('\nDowngrade completed successfully.')
else:
print('\nDowngrade failed. Check logs for details.')
sys.exit(1)
elif args.all:
# Downgrade all migrated users
if not migrated_users:
print('\nNo migrated users to downgrade.')
return
print(f'\n⚠️ About to downgrade {len(migrated_users)} user(s).')
if not args.no_confirm:
print('\nThis will:')
print(' - Revert LiteLLM team/user budget settings')
print(' - Delete organization entries')
print(' - Delete user entries in the new schema')
print(' - Reset the already_migrated flag')
print('\nUsers to downgrade:')
for user_id in migrated_users[:10]: # Show first 10
print(f' - {user_id}')
if len(migrated_users) > 10:
print(f' ... and {len(migrated_users) - 10} more')
confirm = input('\nType "yes" to proceed: ')
if confirm.lower() != 'yes':
print('Cancelled.')
return
print('\nStarting downgrade...\n')
success_count = 0
fail_count = 0
for user_id in migrated_users:
success = await downgrade_user(user_id)
if success:
success_count += 1
else:
fail_count += 1
print('\n--- Summary ---')
print(f'Successful: {success_count}')
print(f'Failed: {fail_count}')
if fail_count > 0:
sys.exit(1)
else:
parser.print_help()
print('\nPlease specify --dry-run, --user-id, or --all')
if __name__ == '__main__':
asyncio.run(main())

View File

@@ -26,12 +26,14 @@ from integrations.utils import (
from integrations.v1_utils import get_saas_user_auth
from jinja2 import Environment, FileSystemLoader
from pydantic import SecretStr
from server.auth.auth_error import ExpiredError
from server.auth.constants import GITHUB_APP_CLIENT_ID, GITHUB_APP_PRIVATE_KEY
from server.auth.token_manager import TokenManager
from server.utils.conversation_callback_utils import register_callback_processor
from openhands.core.logger import openhands_logger as logger
from openhands.integrations.provider import ProviderToken, ProviderType
from openhands.integrations.service_types import AuthenticationError
from openhands.server.types import (
LLMAuthenticationError,
MissingSettingsError,
@@ -143,11 +145,7 @@ class GithubManager(Manager):
).get('body', ''):
return False
if GithubFactory.is_eligible_for_conversation_starter(
message
) and self._user_has_write_access_to_repo(installation_id, repo_name, username):
await GithubFactory.trigger_conversation_starter(message)
# Check event types before making expensive API calls (e.g., _user_has_write_access_to_repo)
if not (
GithubFactory.is_labeled_issue(message)
or GithubFactory.is_issue_comment(message)
@@ -157,8 +155,17 @@ class GithubManager(Manager):
return False
logger.info(f'[GitHub] Checking permissions for {username} in {repo_name}')
user_has_write_access = self._user_has_write_access_to_repo(
installation_id, repo_name, username
)
return self._user_has_write_access_to_repo(installation_id, repo_name, username)
if (
GithubFactory.is_eligible_for_conversation_starter(message)
and user_has_write_access
):
await GithubFactory.trigger_conversation_starter(message)
return user_has_write_access
async def receive_message(self, message: Message):
self._confirm_incoming_source_type(message)
@@ -347,7 +354,7 @@ class GithubManager(Manager):
msg_info = f'@{user_info.username} please set a valid LLM API key in [OpenHands Cloud]({HOST_URL}) before starting a job.'
except SessionExpiredError as e:
except (AuthenticationError, ExpiredError, SessionExpiredError) as e:
logger.warning(
f'[GitHub] Session expired for user {user_info.username}: {str(e)}'
)

View File

@@ -167,17 +167,15 @@ async def install_webhook_on_resource(
scopes=SCOPES,
)
logger.info(
'Creating new webhook',
extra={
'webhook_id': webhook_id,
'status': status,
'resource_id': resource_id,
'resource_type': resource_type,
},
)
log_extra = {
'webhook_id': webhook_id,
'status': status,
'resource_id': resource_id,
'resource_type': resource_type,
}
if status == WebhookStatus.RATE_LIMITED:
logger.warning('Rate limited while creating webhook', extra=log_extra)
raise BreakLoopException()
if webhook_id:
@@ -191,9 +189,8 @@ async def install_webhook_on_resource(
'webhook_uuid': webhook_uuid, # required to identify which webhook installation is sending payload
},
)
logger.info(
f'Installed webhook for {webhook.user_id} on {resource_type}:{resource_id}'
)
logger.info('Created new webhook', extra=log_extra)
else:
logger.error('Failed to create webhook', extra=log_extra)
return webhook_id, status

View File

@@ -1,6 +1,6 @@
from openhands.app_server.user.user_context import UserContext
from openhands.app_server.user.user_models import UserInfo
from openhands.integrations.provider import PROVIDER_TOKEN_TYPE
from openhands.integrations.provider import PROVIDER_TOKEN_TYPE, ProviderHandler
from openhands.integrations.service_types import ProviderType
from openhands.sdk.secret import SecretSource, StaticSecret
from openhands.server.user_auth.user_auth import UserAuth
@@ -14,6 +14,7 @@ class ResolverUserContext(UserContext):
saas_user_auth: UserAuth,
):
self.saas_user_auth = saas_user_auth
self._provider_handler: ProviderHandler | None = None
async def get_user_id(self) -> str | None:
return await self.saas_user_auth.get_user_id()
@@ -29,12 +30,26 @@ class ResolverUserContext(UserContext):
return UserInfo(id=user_id)
async def _get_provider_handler(self) -> ProviderHandler:
"""Get or create a ProviderHandler for git operations."""
if self._provider_handler is None:
provider_tokens = await self.saas_user_auth.get_provider_tokens()
if provider_tokens is None:
raise ValueError('No provider tokens available')
user_id = await self.saas_user_auth.get_user_id()
self._provider_handler = ProviderHandler(
provider_tokens=provider_tokens, external_auth_id=user_id
)
return self._provider_handler
async def get_authenticated_git_url(
self, repository: str, is_optional: bool = False
) -> str:
# This would need to be implemented based on the git provider tokens
# For now, return a basic HTTPS URL
return f'https://github.com/{repository}.git'
provider_handler = await self._get_provider_handler()
url = await provider_handler.get_authenticated_git_url(
repository, is_optional=is_optional
)
return url
async def get_latest_token(self, provider_type: ProviderType) -> str | None:
# Return the appropriate token string from git_provider_tokens

View File

@@ -1,10 +1,15 @@
import logging
import os
from logging.config import fileConfig
from alembic import context
from google.cloud.sql.connector import Connector
from sqlalchemy import create_engine
from storage.base import Base
# Suppress alembic.runtime.plugins INFO logs during import to prevent non-JSON logs in production
# These plugin setup messages would otherwise appear before logging is configured
logging.getLogger('alembic.runtime.plugins').setLevel(logging.WARNING)
from alembic import context # noqa: E402
from google.cloud.sql.connector import Connector # noqa: E402
from sqlalchemy import create_engine # noqa: E402
from storage.base import Base # noqa: E402
target_metadata = Base.metadata

View File

@@ -0,0 +1,28 @@
"""Add git_user_name and git_user_email columns to user table.
Revision ID: 090
Revises: 089
Create Date: 2025-01-22
"""
import sqlalchemy as sa
from alembic import op
revision = '090'
down_revision = '089'
def upgrade() -> None:
op.add_column(
'user',
sa.Column('git_user_name', sa.String, nullable=True),
)
op.add_column(
'user',
sa.Column('git_user_email', sa.String, nullable=True),
)
def downgrade() -> None:
op.drop_column('user', 'git_user_email')
op.drop_column('user', 'git_user_name')

View File

@@ -0,0 +1,46 @@
"""Add byor_export_enabled flag to org table.
Revision ID: 091
Revises: 090
Create Date: 2025-01-15 00:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '091'
down_revision: Union[str, None] = '090'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Add byor_export_enabled column to org table with default false
op.add_column(
'org',
sa.Column(
'byor_export_enabled',
sa.Boolean,
nullable=False,
server_default=sa.text('false'),
),
)
# Set byor_export_enabled to true for orgs that have completed billing sessions
op.execute(
sa.text("""
UPDATE org SET byor_export_enabled = TRUE
WHERE id IN (
SELECT DISTINCT org_id FROM billing_sessions
WHERE status = 'completed' AND org_id IS NOT NULL
)
""")
)
def downgrade() -> None:
op.drop_column('org', 'byor_export_enabled')

View File

@@ -0,0 +1,29 @@
"""Rename 'user' role to 'member' in role table.
Revision ID: 092
Revises: 091
Create Date: 2025-02-12 00:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '092'
down_revision: Union[str, None] = '091'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Rename 'user' role to 'member' for clarity
# This avoids confusion between the 'user' role and the 'user' entity/account
op.execute(sa.text("UPDATE role SET name = 'member' WHERE name = 'user'"))
def downgrade() -> None:
# Revert 'member' role back to 'user'
op.execute(sa.text("UPDATE role SET name = 'user' WHERE name = 'member'"))

212
enterprise/poetry.lock generated
View File

@@ -6102,14 +6102,14 @@ llama = ["llama-index (>=0.12.29,<0.13.0)", "llama-index-core (>=0.12.29,<0.13.0
[[package]]
name = "openhands-agent-server"
version = "1.9.0"
version = "1.11.4"
description = "OpenHands Agent Server - REST/WebSocket interface for OpenHands AI Agent"
optional = false
python-versions = ">=3.12"
groups = ["main"]
files = [
{file = "openhands_agent_server-1.9.0-py3-none-any.whl", hash = "sha256:44b65fac5bb831541eb2e8726afb2682bde4816b4c6c90be9ad3cafd3dbcf971"},
{file = "openhands_agent_server-1.9.0.tar.gz", hash = "sha256:ac41a948acf64ed661a9f383c293c305176f92bd12e6fc6362f5414cb7874ee1"},
{file = "openhands_agent_server-1.11.4-py3-none-any.whl", hash = "sha256:739bdb774dbfcd23d6e87ee6ee32bc0999f22300037506b6dd33e9ea67fa5c2a"},
{file = "openhands_agent_server-1.11.4.tar.gz", hash = "sha256:41247f7022a046eb50ca3b552bc6d12bfa9776e1bd27d0989da91b9f7ac77ca2"},
]
[package.dependencies]
@@ -6126,7 +6126,7 @@ wsproto = ">=1.2.0"
[[package]]
name = "openhands-ai"
version = "1.2.1"
version = "1.3.0"
description = "OpenHands: Code Less, Make More"
optional = false
python-versions = "^3.12,<3.14"
@@ -6168,9 +6168,9 @@ memory-profiler = ">=0.61"
numpy = "*"
openai = "2.8"
openhands-aci = "0.3.2"
openhands-agent-server = "1.9"
openhands-sdk = "1.9"
openhands-tools = "1.9"
openhands-agent-server = "1.11.4"
openhands-sdk = "1.11.4"
openhands-tools = "1.11.4"
opentelemetry-api = ">=1.33.1"
opentelemetry-exporter-otlp-proto-grpc = ">=1.33.1"
pathspec = ">=0.12.1"
@@ -6225,14 +6225,14 @@ url = ".."
[[package]]
name = "openhands-sdk"
version = "1.9.0"
version = "1.11.4"
description = "OpenHands SDK - Core functionality for building AI agents"
optional = false
python-versions = ">=3.12"
groups = ["main"]
files = [
{file = "openhands_sdk-1.9.0-py3-none-any.whl", hash = "sha256:b427d8b9e587a5360c7d61742c290601998557e9b38b1c9e11a297659812c00d"},
{file = "openhands_sdk-1.9.0.tar.gz", hash = "sha256:70048888fd4fbe44a86c35c402bbb99d30cf0cba50579ee1a8e3f43e05154150"},
{file = "openhands_sdk-1.11.4-py3-none-any.whl", hash = "sha256:9f4607c5d94b56fbcd533207026ee892779dd50e29bce79277ff82454a4f76d5"},
{file = "openhands_sdk-1.11.4.tar.gz", hash = "sha256:4088744f6b8856eeab22d3bc17e47d1736ea7ced945c2fa126bd7d48c14bb313"},
]
[package.dependencies]
@@ -6253,14 +6253,14 @@ boto3 = ["boto3 (>=1.35.0)"]
[[package]]
name = "openhands-tools"
version = "1.9.0"
version = "1.11.4"
description = "OpenHands Tools - Runtime tools for AI agents"
optional = false
python-versions = ">=3.12"
groups = ["main"]
files = [
{file = "openhands_tools-1.9.0-py3-none-any.whl", hash = "sha256:8becde0e913a31babb41eb93a8c10bf41d87ca1febd07bc958839c3583655305"},
{file = "openhands_tools-1.9.0.tar.gz", hash = "sha256:d45f5f5210cb2bbcd8ab5f3a32051db1a532d0ec07cd306105f95cde42cf67f2"},
{file = "openhands_tools-1.11.4-py3-none-any.whl", hash = "sha256:efd721b73e87a0dac69171a76931363fa59fcde98107ca86081ee7bf0253673a"},
{file = "openhands_tools-1.11.4.tar.gz", hash = "sha256:80671b1ea8c85a5247a75ea2340ae31d76363e9c723b104699a9a77e66d2043c"},
]
[package.dependencies]
@@ -6851,103 +6851,103 @@ scramp = ">=1.4.5"
[[package]]
name = "pillow"
version = "12.1.0"
version = "12.1.1"
description = "Python Imaging Library (fork)"
optional = false
python-versions = ">=3.10"
groups = ["main", "test"]
files = [
{file = "pillow-12.1.0-cp310-cp310-macosx_10_10_x86_64.whl", hash = "sha256:fb125d860738a09d363a88daa0f59c4533529a90e564785e20fe875b200b6dbd"},
{file = "pillow-12.1.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:cad302dc10fac357d3467a74a9561c90609768a6f73a1923b0fd851b6486f8b0"},
{file = "pillow-12.1.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:a40905599d8079e09f25027423aed94f2823adaf2868940de991e53a449e14a8"},
{file = "pillow-12.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:92a7fe4225365c5e3a8e598982269c6d6698d3e783b3b1ae979e7819f9cd55c1"},
{file = "pillow-12.1.0-cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f10c98f49227ed8383d28174ee95155a675c4ed7f85e2e573b04414f7e371bda"},
{file = "pillow-12.1.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8637e29d13f478bc4f153d8daa9ffb16455f0a6cb287da1b432fdad2bfbd66c7"},
{file = "pillow-12.1.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:21e686a21078b0f9cb8c8a961d99e6a4ddb88e0fc5ea6e130172ddddc2e5221a"},
{file = "pillow-12.1.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:2415373395a831f53933c23ce051021e79c8cd7979822d8cc478547a3f4da8ef"},
{file = "pillow-12.1.0-cp310-cp310-win32.whl", hash = "sha256:e75d3dba8fc1ddfec0cd752108f93b83b4f8d6ab40e524a95d35f016b9683b09"},
{file = "pillow-12.1.0-cp310-cp310-win_amd64.whl", hash = "sha256:64efdf00c09e31efd754448a383ea241f55a994fd079866b92d2bbff598aad91"},
{file = "pillow-12.1.0-cp310-cp310-win_arm64.whl", hash = "sha256:f188028b5af6b8fb2e9a76ac0f841a575bd1bd396e46ef0840d9b88a48fdbcea"},
{file = "pillow-12.1.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:a83e0850cb8f5ac975291ebfc4170ba481f41a28065277f7f735c202cd8e0af3"},
{file = "pillow-12.1.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:b6e53e82ec2db0717eabb276aa56cf4e500c9a7cec2c2e189b55c24f65a3e8c0"},
{file = "pillow-12.1.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:40a8e3b9e8773876d6e30daed22f016509e3987bab61b3b7fe309d7019a87451"},
{file = "pillow-12.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:800429ac32c9b72909c671aaf17ecd13110f823ddb7db4dfef412a5587c2c24e"},
{file = "pillow-12.1.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0b022eaaf709541b391ee069f0022ee5b36c709df71986e3f7be312e46f42c84"},
{file = "pillow-12.1.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1f345e7bc9d7f368887c712aa5054558bad44d2a301ddf9248599f4161abc7c0"},
{file = "pillow-12.1.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d70347c8a5b7ccd803ec0c85c8709f036e6348f1e6a5bf048ecd9c64d3550b8b"},
{file = "pillow-12.1.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:1fcc52d86ce7a34fd17cb04e87cfdb164648a3662a6f20565910a99653d66c18"},
{file = "pillow-12.1.0-cp311-cp311-win32.whl", hash = "sha256:3ffaa2f0659e2f740473bcf03c702c39a8d4b2b7ffc629052028764324842c64"},
{file = "pillow-12.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:806f3987ffe10e867bab0ddad45df1148a2b98221798457fa097ad85d6e8bc75"},
{file = "pillow-12.1.0-cp311-cp311-win_arm64.whl", hash = "sha256:9f5fefaca968e700ad1a4a9de98bf0869a94e397fe3524c4c9450c1445252304"},
{file = "pillow-12.1.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:a332ac4ccb84b6dde65dbace8431f3af08874bf9770719d32a635c4ef411b18b"},
{file = "pillow-12.1.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:907bfa8a9cb790748a9aa4513e37c88c59660da3bcfffbd24a7d9e6abf224551"},
{file = "pillow-12.1.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:efdc140e7b63b8f739d09a99033aa430accce485ff78e6d311973a67b6bf3208"},
{file = "pillow-12.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:bef9768cab184e7ae6e559c032e95ba8d07b3023c289f79a2bd36e8bf85605a5"},
{file = "pillow-12.1.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:742aea052cf5ab5034a53c3846165bc3ce88d7c38e954120db0ab867ca242661"},
{file = "pillow-12.1.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a6dfc2af5b082b635af6e08e0d1f9f1c4e04d17d4e2ca0ef96131e85eda6eb17"},
{file = "pillow-12.1.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:609e89d9f90b581c8d16358c9087df76024cf058fa693dd3e1e1620823f39670"},
{file = "pillow-12.1.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:43b4899cfd091a9693a1278c4982f3e50f7fb7cff5153b05174b4afc9593b616"},
{file = "pillow-12.1.0-cp312-cp312-win32.whl", hash = "sha256:aa0c9cc0b82b14766a99fbe6084409972266e82f459821cd26997a488a7261a7"},
{file = "pillow-12.1.0-cp312-cp312-win_amd64.whl", hash = "sha256:d70534cea9e7966169ad29a903b99fc507e932069a881d0965a1a84bb57f6c6d"},
{file = "pillow-12.1.0-cp312-cp312-win_arm64.whl", hash = "sha256:65b80c1ee7e14a87d6a068dd3b0aea268ffcabfe0498d38661b00c5b4b22e74c"},
{file = "pillow-12.1.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:7b5dd7cbae20285cdb597b10eb5a2c13aa9de6cde9bb64a3c1317427b1db1ae1"},
{file = "pillow-12.1.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:29a4cef9cb672363926f0470afc516dbf7305a14d8c54f7abbb5c199cd8f8179"},
{file = "pillow-12.1.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:681088909d7e8fa9e31b9799aaa59ba5234c58e5e4f1951b4c4d1082a2e980e0"},
{file = "pillow-12.1.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:983976c2ab753166dc66d36af6e8ec15bb511e4a25856e2227e5f7e00a160587"},
{file = "pillow-12.1.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:db44d5c160a90df2d24a24760bbd37607d53da0b34fb546c4c232af7192298ac"},
{file = "pillow-12.1.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:6b7a9d1db5dad90e2991645874f708e87d9a3c370c243c2d7684d28f7e133e6b"},
{file = "pillow-12.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:6258f3260986990ba2fa8a874f8b6e808cf5abb51a94015ca3dc3c68aa4f30ea"},
{file = "pillow-12.1.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e115c15e3bc727b1ca3e641a909f77f8ca72a64fff150f666fcc85e57701c26c"},
{file = "pillow-12.1.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6741e6f3074a35e47c77b23a4e4f2d90db3ed905cb1c5e6e0d49bff2045632bc"},
{file = "pillow-12.1.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:935b9d1aed48fcfb3f838caac506f38e29621b44ccc4f8a64d575cb1b2a88644"},
{file = "pillow-12.1.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5fee4c04aad8932da9f8f710af2c1a15a83582cfb884152a9caa79d4efcdbf9c"},
{file = "pillow-12.1.0-cp313-cp313-win32.whl", hash = "sha256:a786bf667724d84aa29b5db1c61b7bfdde380202aaca12c3461afd6b71743171"},
{file = "pillow-12.1.0-cp313-cp313-win_amd64.whl", hash = "sha256:461f9dfdafa394c59cd6d818bdfdbab4028b83b02caadaff0ffd433faf4c9a7a"},
{file = "pillow-12.1.0-cp313-cp313-win_arm64.whl", hash = "sha256:9212d6b86917a2300669511ed094a9406888362e085f2431a7da985a6b124f45"},
{file = "pillow-12.1.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:00162e9ca6d22b7c3ee8e61faa3c3253cd19b6a37f126cad04f2f88b306f557d"},
{file = "pillow-12.1.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:7d6daa89a00b58c37cb1747ec9fb7ac3bc5ffd5949f5888657dfddde6d1312e0"},
{file = "pillow-12.1.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:e2479c7f02f9d505682dc47df8c0ea1fc5e264c4d1629a5d63fe3e2334b89554"},
{file = "pillow-12.1.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f188d580bd870cda1e15183790d1cc2fa78f666e76077d103edf048eed9c356e"},
{file = "pillow-12.1.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0fde7ec5538ab5095cc02df38ee99b0443ff0e1c847a045554cf5f9af1f4aa82"},
{file = "pillow-12.1.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0ed07dca4a8464bada6139ab38f5382f83e5f111698caf3191cb8dbf27d908b4"},
{file = "pillow-12.1.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:f45bd71d1fa5e5749587613037b172e0b3b23159d1c00ef2fc920da6f470e6f0"},
{file = "pillow-12.1.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:277518bf4fe74aa91489e1b20577473b19ee70fb97c374aa50830b279f25841b"},
{file = "pillow-12.1.0-cp313-cp313t-win32.whl", hash = "sha256:7315f9137087c4e0ee73a761b163fc9aa3b19f5f606a7fc08d83fd3e4379af65"},
{file = "pillow-12.1.0-cp313-cp313t-win_amd64.whl", hash = "sha256:0ddedfaa8b5f0b4ffbc2fa87b556dc59f6bb4ecb14a53b33f9189713ae8053c0"},
{file = "pillow-12.1.0-cp313-cp313t-win_arm64.whl", hash = "sha256:80941e6d573197a0c28f394753de529bb436b1ca990ed6e765cf42426abc39f8"},
{file = "pillow-12.1.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:5cb7bc1966d031aec37ddb9dcf15c2da5b2e9f7cc3ca7c54473a20a927e1eb91"},
{file = "pillow-12.1.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:97e9993d5ed946aba26baf9c1e8cf18adbab584b99f452ee72f7ee8acb882796"},
{file = "pillow-12.1.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:414b9a78e14ffeb98128863314e62c3f24b8a86081066625700b7985b3f529bd"},
{file = "pillow-12.1.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:e6bdb408f7c9dd2a5ff2b14a3b0bb6d4deb29fb9961e6eb3ae2031ae9a5cec13"},
{file = "pillow-12.1.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:3413c2ae377550f5487991d444428f1a8ae92784aac79caa8b1e3b89b175f77e"},
{file = "pillow-12.1.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:e5dcbe95016e88437ecf33544ba5db21ef1b8dd6e1b434a2cb2a3d605299e643"},
{file = "pillow-12.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d0a7735df32ccbcc98b98a1ac785cc4b19b580be1bdf0aeb5c03223220ea09d5"},
{file = "pillow-12.1.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0c27407a2d1b96774cbc4a7594129cc027339fd800cd081e44497722ea1179de"},
{file = "pillow-12.1.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:15c794d74303828eaa957ff8070846d0efe8c630901a1c753fdc63850e19ecd9"},
{file = "pillow-12.1.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c990547452ee2800d8506c4150280757f88532f3de2a58e3022e9b179107862a"},
{file = "pillow-12.1.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:b63e13dd27da389ed9475b3d28510f0f954bca0041e8e551b2a4eb1eab56a39a"},
{file = "pillow-12.1.0-cp314-cp314-win32.whl", hash = "sha256:1a949604f73eb07a8adab38c4fe50791f9919344398bdc8ac6b307f755fc7030"},
{file = "pillow-12.1.0-cp314-cp314-win_amd64.whl", hash = "sha256:4f9f6a650743f0ddee5593ac9e954ba1bdbc5e150bc066586d4f26127853ab94"},
{file = "pillow-12.1.0-cp314-cp314-win_arm64.whl", hash = "sha256:808b99604f7873c800c4840f55ff389936ef1948e4e87645eaf3fccbc8477ac4"},
{file = "pillow-12.1.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:bc11908616c8a283cf7d664f77411a5ed2a02009b0097ff8abbba5e79128ccf2"},
{file = "pillow-12.1.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:896866d2d436563fa2a43a9d72f417874f16b5545955c54a64941e87c1376c61"},
{file = "pillow-12.1.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8e178e3e99d3c0ea8fc64b88447f7cac8ccf058af422a6cedc690d0eadd98c51"},
{file = "pillow-12.1.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:079af2fb0c599c2ec144ba2c02766d1b55498e373b3ac64687e43849fbbef5bc"},
{file = "pillow-12.1.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bdec5e43377761c5dbca620efb69a77f6855c5a379e32ac5b158f54c84212b14"},
{file = "pillow-12.1.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:565c986f4b45c020f5421a4cea13ef294dde9509a8577f29b2fc5edc7587fff8"},
{file = "pillow-12.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:43aca0a55ce1eefc0aefa6253661cb54571857b1a7b2964bd8a1e3ef4b729924"},
{file = "pillow-12.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:0deedf2ea233722476b3a81e8cdfbad786f7adbed5d848469fa59fe52396e4ef"},
{file = "pillow-12.1.0-cp314-cp314t-win32.whl", hash = "sha256:b17fbdbe01c196e7e159aacb889e091f28e61020a8abeac07b68079b6e626988"},
{file = "pillow-12.1.0-cp314-cp314t-win_amd64.whl", hash = "sha256:27b9baecb428899db6c0de572d6d305cfaf38ca1596b5c0542a5182e3e74e8c6"},
{file = "pillow-12.1.0-cp314-cp314t-win_arm64.whl", hash = "sha256:f61333d817698bdcdd0f9d7793e365ac3d2a21c1f1eb02b32ad6aefb8d8ea831"},
{file = "pillow-12.1.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:ca94b6aac0d7af2a10ba08c0f888b3d5114439b6b3ef39968378723622fed377"},
{file = "pillow-12.1.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:351889afef0f485b84078ea40fe33727a0492b9af3904661b0abbafee0355b72"},
{file = "pillow-12.1.0-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:bb0984b30e973f7e2884362b7d23d0a348c7143ee559f38ef3eaab640144204c"},
{file = "pillow-12.1.0-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:84cabc7095dd535ca934d57e9ce2a72ffd216e435a84acb06b2277b1de2689bd"},
{file = "pillow-12.1.0-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:53d8b764726d3af1a138dd353116f774e3862ec7e3794e0c8781e30db0f35dfc"},
{file = "pillow-12.1.0-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5da841d81b1a05ef940a8567da92decaa15bc4d7dedb540a8c219ad83d91808a"},
{file = "pillow-12.1.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:75af0b4c229ac519b155028fa1be632d812a519abba9b46b20e50c6caa184f19"},
{file = "pillow-12.1.0.tar.gz", hash = "sha256:5c5ae0a06e9ea030ab786b0251b32c7e4ce10e58d983c0d5c56029455180b5b9"},
{file = "pillow-12.1.1-cp310-cp310-macosx_10_10_x86_64.whl", hash = "sha256:1f1625b72740fdda5d77b4def688eb8fd6490975d06b909fd19f13f391e077e0"},
{file = "pillow-12.1.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:178aa072084bd88ec759052feca8e56cbb14a60b39322b99a049e58090479713"},
{file = "pillow-12.1.1-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:b66e95d05ba806247aaa1561f080abc7975daf715c30780ff92a20e4ec546e1b"},
{file = "pillow-12.1.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:89c7e895002bbe49cdc5426150377cbbc04767d7547ed145473f496dfa40408b"},
{file = "pillow-12.1.1-cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a5cbdcddad0af3da87cb16b60d23648bc3b51967eb07223e9fed77a82b457c4"},
{file = "pillow-12.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9f51079765661884a486727f0729d29054242f74b46186026582b4e4769918e4"},
{file = "pillow-12.1.1-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:99c1506ea77c11531d75e3a412832a13a71c7ebc8192ab9e4b2e355555920e3e"},
{file = "pillow-12.1.1-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:36341d06738a9f66c8287cf8b876d24b18db9bd8740fa0672c74e259ad408cff"},
{file = "pillow-12.1.1-cp310-cp310-win32.whl", hash = "sha256:6c52f062424c523d6c4db85518774cc3d50f5539dd6eed32b8f6229b26f24d40"},
{file = "pillow-12.1.1-cp310-cp310-win_amd64.whl", hash = "sha256:c6008de247150668a705a6338156efb92334113421ceecf7438a12c9a12dab23"},
{file = "pillow-12.1.1-cp310-cp310-win_arm64.whl", hash = "sha256:1a9b0ee305220b392e1124a764ee4265bd063e54a751a6b62eff69992f457fa9"},
{file = "pillow-12.1.1-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:e879bb6cd5c73848ef3b2b48b8af9ff08c5b71ecda8048b7dd22d8a33f60be32"},
{file = "pillow-12.1.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:365b10bb9417dd4498c0e3b128018c4a624dc11c7b97d8cc54effe3b096f4c38"},
{file = "pillow-12.1.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d4ce8e329c93845720cd2014659ca67eac35f6433fd3050393d85f3ecef0dad5"},
{file = "pillow-12.1.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:fc354a04072b765eccf2204f588a7a532c9511e8b9c7f900e1b64e3e33487090"},
{file = "pillow-12.1.1-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7e7976bf1910a8116b523b9f9f58bf410f3e8aa330cd9a2bb2953f9266ab49af"},
{file = "pillow-12.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:597bd9c8419bc7c6af5604e55847789b69123bbe25d65cc6ad3012b4f3c98d8b"},
{file = "pillow-12.1.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:2c1fc0f2ca5f96a3c8407e41cca26a16e46b21060fe6d5b099d2cb01412222f5"},
{file = "pillow-12.1.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:578510d88c6229d735855e1f278aa305270438d36a05031dfaae5067cc8eb04d"},
{file = "pillow-12.1.1-cp311-cp311-win32.whl", hash = "sha256:7311c0a0dcadb89b36b7025dfd8326ecfa36964e29913074d47382706e516a7c"},
{file = "pillow-12.1.1-cp311-cp311-win_amd64.whl", hash = "sha256:fbfa2a7c10cc2623f412753cddf391c7f971c52ca40a3f65dc5039b2939e8563"},
{file = "pillow-12.1.1-cp311-cp311-win_arm64.whl", hash = "sha256:b81b5e3511211631b3f672a595e3221252c90af017e399056d0faabb9538aa80"},
{file = "pillow-12.1.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ab323b787d6e18b3d91a72fc99b1a2c28651e4358749842b8f8dfacd28ef2052"},
{file = "pillow-12.1.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:adebb5bee0f0af4909c30db0d890c773d1a92ffe83da908e2e9e720f8edf3984"},
{file = "pillow-12.1.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:bb66b7cc26f50977108790e2456b7921e773f23db5630261102233eb355a3b79"},
{file = "pillow-12.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:aee2810642b2898bb187ced9b349e95d2a7272930796e022efaf12e99dccd293"},
{file = "pillow-12.1.1-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a0b1cd6232e2b618adcc54d9882e4e662a089d5768cd188f7c245b4c8c44a397"},
{file = "pillow-12.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7aac39bcf8d4770d089588a2e1dd111cbaa42df5a94be3114222057d68336bd0"},
{file = "pillow-12.1.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:ab174cd7d29a62dd139c44bf74b698039328f45cb03b4596c43473a46656b2f3"},
{file = "pillow-12.1.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:339ffdcb7cbeaa08221cd401d517d4b1fe7a9ed5d400e4a8039719238620ca35"},
{file = "pillow-12.1.1-cp312-cp312-win32.whl", hash = "sha256:5d1f9575a12bed9e9eedd9a4972834b08c97a352bd17955ccdebfeca5913fa0a"},
{file = "pillow-12.1.1-cp312-cp312-win_amd64.whl", hash = "sha256:21329ec8c96c6e979cd0dfd29406c40c1d52521a90544463057d2aaa937d66a6"},
{file = "pillow-12.1.1-cp312-cp312-win_arm64.whl", hash = "sha256:af9a332e572978f0218686636610555ae3defd1633597be015ed50289a03c523"},
{file = "pillow-12.1.1-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:d242e8ac078781f1de88bf823d70c1a9b3c7950a44cdf4b7c012e22ccbcd8e4e"},
{file = "pillow-12.1.1-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:02f84dfad02693676692746df05b89cf25597560db2857363a208e393429f5e9"},
{file = "pillow-12.1.1-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:e65498daf4b583091ccbb2556c7000abf0f3349fcd57ef7adc9a84a394ed29f6"},
{file = "pillow-12.1.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:6c6db3b84c87d48d0088943bf33440e0c42370b99b1c2a7989216f7b42eede60"},
{file = "pillow-12.1.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:8b7e5304e34942bf62e15184219a7b5ad4ff7f3bb5cca4d984f37df1a0e1aee2"},
{file = "pillow-12.1.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:18e5bddd742a44b7e6b1e773ab5db102bd7a94c32555ba656e76d319d19c3850"},
{file = "pillow-12.1.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:fc44ef1f3de4f45b50ccf9136999d71abb99dca7706bc75d222ed350b9fd2289"},
{file = "pillow-12.1.1-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5a8eb7ed8d4198bccbd07058416eeec51686b498e784eda166395a23eb99138e"},
{file = "pillow-12.1.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:47b94983da0c642de92ced1702c5b6c292a84bd3a8e1d1702ff923f183594717"},
{file = "pillow-12.1.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:518a48c2aab7ce596d3bf79d0e275661b846e86e4d0e7dec34712c30fe07f02a"},
{file = "pillow-12.1.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:a550ae29b95c6dc13cf69e2c9dc5747f814c54eeb2e32d683e5e93af56caa029"},
{file = "pillow-12.1.1-cp313-cp313-win32.whl", hash = "sha256:a003d7422449f6d1e3a34e3dd4110c22148336918ddbfc6a32581cd54b2e0b2b"},
{file = "pillow-12.1.1-cp313-cp313-win_amd64.whl", hash = "sha256:344cf1e3dab3be4b1fa08e449323d98a2a3f819ad20f4b22e77a0ede31f0faa1"},
{file = "pillow-12.1.1-cp313-cp313-win_arm64.whl", hash = "sha256:5c0dd1636633e7e6a0afe7bf6a51a14992b7f8e60de5789018ebbdfae55b040a"},
{file = "pillow-12.1.1-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:0330d233c1a0ead844fc097a7d16c0abff4c12e856c0b325f231820fee1f39da"},
{file = "pillow-12.1.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:5dae5f21afb91322f2ff791895ddd8889e5e947ff59f71b46041c8ce6db790bc"},
{file = "pillow-12.1.1-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2e0c664be47252947d870ac0d327fea7e63985a08794758aa8af5b6cb6ec0c9c"},
{file = "pillow-12.1.1-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:691ab2ac363b8217f7d31b3497108fb1f50faab2f75dfb03284ec2f217e87bf8"},
{file = "pillow-12.1.1-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e9e8064fb1cc019296958595f6db671fba95209e3ceb0c4734c9baf97de04b20"},
{file = "pillow-12.1.1-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:472a8d7ded663e6162dafdf20015c486a7009483ca671cece7a9279b512fcb13"},
{file = "pillow-12.1.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:89b54027a766529136a06cfebeecb3a04900397a3590fd252160b888479517bf"},
{file = "pillow-12.1.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:86172b0831b82ce4f7877f280055892b31179e1576aa00d0df3bb1bbf8c3e524"},
{file = "pillow-12.1.1-cp313-cp313t-win32.whl", hash = "sha256:44ce27545b6efcf0fdbdceb31c9a5bdea9333e664cda58a7e674bb74608b3986"},
{file = "pillow-12.1.1-cp313-cp313t-win_amd64.whl", hash = "sha256:a285e3eb7a5a45a2ff504e31f4a8d1b12ef62e84e5411c6804a42197c1cf586c"},
{file = "pillow-12.1.1-cp313-cp313t-win_arm64.whl", hash = "sha256:cc7d296b5ea4d29e6570dabeaed58d31c3fea35a633a69679fb03d7664f43fb3"},
{file = "pillow-12.1.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:417423db963cb4be8bac3fc1204fe61610f6abeed1580a7a2cbb2fbda20f12af"},
{file = "pillow-12.1.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:b957b71c6b2387610f556a7eb0828afbe40b4a98036fc0d2acfa5a44a0c2036f"},
{file = "pillow-12.1.1-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:097690ba1f2efdeb165a20469d59d8bb03c55fb6621eb2041a060ae8ea3e9642"},
{file = "pillow-12.1.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:2815a87ab27848db0321fb78c7f0b2c8649dee134b7f2b80c6a45c6831d75ccd"},
{file = "pillow-12.1.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:f7ed2c6543bad5a7d5530eb9e78c53132f93dfa44a28492db88b41cdab885202"},
{file = "pillow-12.1.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:652a2c9ccfb556235b2b501a3a7cf3742148cd22e04b5625c5fe057ea3e3191f"},
{file = "pillow-12.1.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d6e4571eedf43af33d0fc233a382a76e849badbccdf1ac438841308652a08e1f"},
{file = "pillow-12.1.1-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b574c51cf7d5d62e9be37ba446224b59a2da26dc4c1bb2ecbe936a4fb1a7cb7f"},
{file = "pillow-12.1.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a37691702ed687799de29a518d63d4682d9016932db66d4e90c345831b02fb4e"},
{file = "pillow-12.1.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:f95c00d5d6700b2b890479664a06e754974848afaae5e21beb4d83c106923fd0"},
{file = "pillow-12.1.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:559b38da23606e68681337ad74622c4dbba02254fc9cb4488a305dd5975c7eeb"},
{file = "pillow-12.1.1-cp314-cp314-win32.whl", hash = "sha256:03edcc34d688572014ff223c125a3f77fb08091e4607e7745002fc214070b35f"},
{file = "pillow-12.1.1-cp314-cp314-win_amd64.whl", hash = "sha256:50480dcd74fa63b8e78235957d302d98d98d82ccbfac4c7e12108ba9ecbdba15"},
{file = "pillow-12.1.1-cp314-cp314-win_arm64.whl", hash = "sha256:5cb1785d97b0c3d1d1a16bc1d710c4a0049daefc4935f3a8f31f827f4d3d2e7f"},
{file = "pillow-12.1.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:1f90cff8aa76835cba5769f0b3121a22bd4eb9e6884cfe338216e557a9a548b8"},
{file = "pillow-12.1.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1f1be78ce9466a7ee64bfda57bdba0f7cc499d9794d518b854816c41bf0aa4e9"},
{file = "pillow-12.1.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:42fc1f4677106188ad9a55562bbade416f8b55456f522430fadab3cef7cd4e60"},
{file = "pillow-12.1.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:98edb152429ab62a1818039744d8fbb3ccab98a7c29fc3d5fcef158f3f1f68b7"},
{file = "pillow-12.1.1-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d470ab1178551dd17fdba0fef463359c41aaa613cdcd7ff8373f54be629f9f8f"},
{file = "pillow-12.1.1-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6408a7b064595afcab0a49393a413732a35788f2a5092fdc6266952ed67de586"},
{file = "pillow-12.1.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:5d8c41325b382c07799a3682c1c258469ea2ff97103c53717b7893862d0c98ce"},
{file = "pillow-12.1.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:c7697918b5be27424e9ce568193efd13d925c4481dd364e43f5dff72d33e10f8"},
{file = "pillow-12.1.1-cp314-cp314t-win32.whl", hash = "sha256:d2912fd8114fc5545aa3a4b5576512f64c55a03f3ebcca4c10194d593d43ea36"},
{file = "pillow-12.1.1-cp314-cp314t-win_amd64.whl", hash = "sha256:4ceb838d4bd9dab43e06c363cab2eebf63846d6a4aeaea283bbdfd8f1a8ed58b"},
{file = "pillow-12.1.1-cp314-cp314t-win_arm64.whl", hash = "sha256:7b03048319bfc6170e93bd60728a1af51d3dd7704935feb228c4d4faab35d334"},
{file = "pillow-12.1.1-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:600fd103672b925fe62ed08e0d874ea34d692474df6f4bf7ebe148b30f89f39f"},
{file = "pillow-12.1.1-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:665e1b916b043cef294bc54d47bf02d87e13f769bc4bc5fa225a24b3a6c5aca9"},
{file = "pillow-12.1.1-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:495c302af3aad1ca67420ddd5c7bd480c8867ad173528767d906428057a11f0e"},
{file = "pillow-12.1.1-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:8fd420ef0c52c88b5a035a0886f367748c72147b2b8f384c9d12656678dfdfa9"},
{file = "pillow-12.1.1-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f975aa7ef9684ce7e2c18a3aa8f8e2106ce1e46b94ab713d156b2898811651d3"},
{file = "pillow-12.1.1-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8089c852a56c2966cf18835db62d9b34fef7ba74c726ad943928d494fa7f4735"},
{file = "pillow-12.1.1-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:cb9bb857b2d057c6dfc72ac5f3b44836924ba15721882ef103cecb40d002d80e"},
{file = "pillow-12.1.1.tar.gz", hash = "sha256:9ad8fa5937ab05218e2b6a4cff30295ad35afd2f83ac592e68c0d871bb0fdbc4"},
]
[package.extras]
@@ -14917,4 +14917,4 @@ cffi = ["cffi (>=1.17,<2.0) ; platform_python_implementation != \"PyPy\" and pyt
[metadata]
lock-version = "2.1"
python-versions = "^3.12,<3.14"
content-hash = "b5cbb1e25176845ac9f95650a802667e2f8be1a536e3e55a9269b5af5a42e3fc"
content-hash = "1cad6029269393af67155e930c72eae2c03da02e4b3a3699823f6168c14a4218"

View File

@@ -44,6 +44,12 @@ httpx = "*"
scikit-learn = "^1.7.0"
shap = "^0.48.0"
google-cloud-recaptcha-enterprise = "^1.24.0"
# Dependencies previously only in Dockerfile, now managed via poetry.lock
prometheus-client = "^0.24.0"
pandas = "^2.2.0"
numpy = "^2.2.0"
mcp = "^1.10.0"
pillow = "^12.1.0"
[tool.poetry.group.dev.dependencies]
ruff = "0.8.3"

View File

@@ -78,8 +78,15 @@ base_app.include_router(shared_event_router)
# Add GitHub integration router only if GITHUB_APP_CLIENT_ID is set
if GITHUB_APP_CLIENT_ID:
# Make sure that the callback processor is loaded here so we don't get an error when deserializing
from integrations.github.github_v1_callback_processor import ( # noqa: E402
GithubV1CallbackProcessor,
)
from server.routes.integration.github import github_integration_router # noqa: E402
# Bludgeon mypy into not deleting my import
logger.debug(f'Loaded {GithubV1CallbackProcessor.__name__}')
base_app.include_router(
github_integration_router
) # Add additional route for integration webhook events

View File

@@ -0,0 +1,320 @@
"""
Permission-based authorization dependencies for API endpoints (SAAS mode).
This module provides FastAPI dependencies for checking user permissions
within organizations. It uses a permission-based authorization model where
roles (owner, admin, member) are mapped to specific permissions.
This is the SAAS/enterprise implementation that performs real authorization
checks against the database.
Usage:
from server.auth.authorization import (
Permission,
require_permission,
require_org_role,
require_org_user,
require_org_admin,
require_org_owner,
)
@router.get('/{org_id}/settings')
async def get_settings(
org_id: UUID,
user_id: str = Depends(require_permission(Permission.VIEW_LLM_SETTINGS)),
):
# Only users with VIEW_LLM_SETTINGS permission can access
...
"""
from enum import Enum
from uuid import UUID
from fastapi import Depends, HTTPException, status
from storage.org_member_store import OrgMemberStore
from storage.role import Role
from storage.role_store import RoleStore
from openhands.core.logger import openhands_logger as logger
from openhands.server.auth import Permission
from openhands.server.user_auth import get_user_id
class RoleName(str, Enum):
"""Role names used in the system."""
OWNER = 'owner'
ADMIN = 'admin'
MEMBER = 'member'
# Permission mappings for each role
ROLE_PERMISSIONS: dict[RoleName, frozenset[Permission]] = {
RoleName.OWNER: frozenset(
[
# Settings (Full access)
Permission.MANAGE_SECRETS,
Permission.MANAGE_MCP,
Permission.MANAGE_INTEGRATIONS,
Permission.MANAGE_APPLICATION_SETTINGS,
Permission.MANAGE_API_KEYS,
Permission.VIEW_LLM_SETTINGS,
Permission.EDIT_LLM_SETTINGS,
Permission.VIEW_BILLING,
Permission.ADD_CREDITS,
# Organization Members
Permission.INVITE_USER_TO_ORGANIZATION,
Permission.CHANGE_USER_ROLE_MEMBER,
Permission.CHANGE_USER_ROLE_ADMIN,
Permission.CHANGE_USER_ROLE_OWNER,
# Organization Management (Owner only)
Permission.CHANGE_ORGANIZATION_NAME,
Permission.DELETE_ORGANIZATION,
]
),
RoleName.ADMIN: frozenset(
[
# Settings (Full access)
Permission.MANAGE_SECRETS,
Permission.MANAGE_MCP,
Permission.MANAGE_INTEGRATIONS,
Permission.MANAGE_APPLICATION_SETTINGS,
Permission.MANAGE_API_KEYS,
Permission.VIEW_LLM_SETTINGS,
Permission.EDIT_LLM_SETTINGS,
Permission.VIEW_BILLING,
Permission.ADD_CREDITS,
# Organization Members
Permission.INVITE_USER_TO_ORGANIZATION,
Permission.CHANGE_USER_ROLE_MEMBER,
Permission.CHANGE_USER_ROLE_ADMIN,
]
),
RoleName.MEMBER: frozenset(
[
# Settings (Full access)
Permission.MANAGE_SECRETS,
Permission.MANAGE_MCP,
Permission.MANAGE_INTEGRATIONS,
Permission.MANAGE_APPLICATION_SETTINGS,
Permission.MANAGE_API_KEYS,
# LLM Settings (View only)
Permission.VIEW_LLM_SETTINGS,
]
),
}
def get_role_permissions(role_name: str) -> frozenset[Permission]:
"""Get the permissions for a role."""
try:
role_enum = RoleName(role_name)
return ROLE_PERMISSIONS.get(role_enum, frozenset())
except ValueError:
return frozenset()
def get_user_org_role(user_id: str, org_id: UUID) -> Role | None:
"""
Get the user's role in an organization.
Args:
user_id: User ID (string that will be converted to UUID)
org_id: Organization ID
Returns:
Role object if user is a member, None otherwise
"""
from uuid import UUID as parse_uuid
org_member = OrgMemberStore.get_org_member(org_id, parse_uuid(user_id))
if not org_member:
return None
return RoleStore.get_role_by_id(org_member.role_id)
def has_permission(user_role: Role, permission: Permission) -> bool:
"""
Check if a role has a specific permission.
Args:
user_role: User's Role object
permission: Permission to check
Returns:
True if the role has the permission
"""
permissions = get_role_permissions(user_role.name)
return permission in permissions
def has_required_role(user_role: Role, required_role: Role) -> bool:
"""
Check if user's role meets or exceeds the required role.
Uses role hierarchy based on rank where lower rank = higher position
(e.g., rank 1 owner > rank 2 admin > rank 3 user).
Args:
user_role: User's actual Role object
required_role: Minimum required Role object
Returns:
True if user has sufficient permissions
"""
return user_role.rank <= required_role.rank
def require_permission(permission: Permission):
"""
Factory function that creates a dependency to require a specific permission.
This creates a FastAPI dependency that:
1. Extracts org_id from the path parameter
2. Gets the authenticated user_id
3. Checks if the user has the required permission in the organization
4. Returns the user_id if authorized, raises HTTPException otherwise
Usage:
@router.get('/{org_id}/settings')
async def get_settings(
org_id: UUID,
user_id: str = Depends(require_permission(Permission.VIEW_LLM_SETTINGS)),
):
...
Args:
permission: The permission required to access the endpoint
Returns:
Dependency function that validates permission and returns user_id
"""
async def permission_checker(
org_id: UUID,
user_id: str | None = Depends(get_user_id),
) -> str:
if not user_id:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail='User not authenticated',
)
user_role = get_user_org_role(user_id, org_id)
if not user_role:
logger.warning(
'User not a member of organization',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail='User is not a member of this organization',
)
if not has_permission(user_role, permission):
logger.warning(
'Insufficient permissions',
extra={
'user_id': user_id,
'org_id': str(org_id),
'user_role': user_role.name,
'required_permission': permission.value,
},
)
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=f'Requires {permission.value} permission',
)
return user_id
return permission_checker
def require_org_role(required_role_name: str):
"""
Factory function that creates a dependency to require a minimum org role.
This creates a FastAPI dependency that:
1. Extracts org_id from the path parameter
2. Gets the authenticated user_id
3. Checks if the user has the required role in the organization
4. Returns the user_id if authorized, raises HTTPException otherwise
Role hierarchy is based on rank from the Role class, where
lower rank = higher position (e.g., rank 1 > rank 2 > rank 3).
Usage:
@router.get('/{org_id}/resource')
async def get_resource(
org_id: UUID,
user_id: str = Depends(require_org_role('user')),
):
...
Args:
required_role_name: Name of the minimum required role to access the endpoint
Returns:
Dependency function that validates role and returns user_id
"""
async def role_checker(
org_id: UUID,
user_id: str | None = Depends(get_user_id),
) -> str:
if not user_id:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail='User not authenticated',
)
user_role = get_user_org_role(user_id, org_id)
if not user_role:
logger.warning(
'User not a member of organization',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail='User is not a member of this organization',
)
required_role = RoleStore.get_role_by_name(required_role_name)
if not required_role:
logger.error(
'Required role not found in database',
extra={'required_role': required_role_name},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Role configuration error',
)
if not has_required_role(user_role, required_role):
logger.warning(
'Insufficient role permissions',
extra={
'user_id': user_id,
'org_id': str(org_id),
'user_role': user_role.name,
'required_role': required_role_name,
},
)
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=f'Requires {required_role_name} role or higher',
)
return user_id
return role_checker
# Convenience dependencies for common role checks
require_org_user = require_org_role('user')
require_org_admin = require_org_role('admin')
require_org_owner = require_org_role('owner')

View File

@@ -1,11 +1,36 @@
import asyncio
from pydantic import SecretStr
from sqlalchemy import select
from openhands.core.logger import openhands_logger as logger
from openhands.integrations.service_types import ProviderType
from openhands.server.types import AppMode
async def _user_has_gitlab_provider(user_id: str) -> bool:
"""Check if the user has authenticated with GitLab.
Args:
user_id: The Keycloak user ID
Returns:
True if the user has a GitLab provider token, False otherwise
"""
# Lazy import to avoid circular dependency issues at module load time
from storage.auth_tokens import AuthTokens
from storage.database import a_session_maker
async with a_session_maker() as session:
result = await session.execute(
select(AuthTokens).where(
AuthTokens.keycloak_user_id == user_id,
AuthTokens.identity_provider == ProviderType.GITLAB.value,
)
)
return result.scalars().first() is not None
def schedule_gitlab_repo_sync(
user_id: str, keycloak_access_token: SecretStr | None = None
) -> None:
@@ -14,10 +39,20 @@ def schedule_gitlab_repo_sync(
Because the outer call is already a background task, we instruct the service
to store repository data synchronously (store_in_background=False) to avoid
nested background tasks while still keeping the overall operation async.
The sync is only performed if the user has authenticated with GitLab.
"""
async def _run():
try:
# Check if the user has a GitLab provider token before syncing
if not await _user_has_gitlab_provider(user_id):
logger.debug(
'gitlab_repo_sync_skipped: user has no GitLab provider',
extra={'user_id': user_id},
)
return
# Lazy import to avoid circular dependency:
# middleware -> gitlab_sync -> integrations.gitlab.gitlab_service
# -> openhands.integrations.gitlab.gitlab_service -> get_impl

View File

@@ -18,6 +18,7 @@ from openhands.core.logger import openhands_logger as logger
class AssessmentResult:
"""Result of a reCAPTCHA Enterprise assessment."""
name: str
score: float
valid: bool
action_valid: bool
@@ -63,6 +64,7 @@ class RecaptchaService:
user_ip: str,
user_agent: str,
email: str | None = None,
user_id: str | None = None,
) -> AssessmentResult:
"""Create a reCAPTCHA Enterprise assessment.
@@ -72,6 +74,7 @@ class RecaptchaService:
user_ip: The user's IP address.
user_agent: The user's browser user agent.
email: Optional email for Account Defender hashing.
user_id: Optional Keycloak user ID for logging correlation.
Returns:
AssessmentResult with score, validity, and allowed status.
@@ -100,6 +103,10 @@ class RecaptchaService:
response = self.client.create_assessment(request)
# Capture assessment name for potential annotation later
# Format: projects/{project_id}/assessments/{assessment_id}
assessment_name = response.name
token_properties = response.token_properties
risk_analysis = response.risk_analysis
@@ -129,6 +136,7 @@ class RecaptchaService:
logger.info(
'recaptcha_assessment',
extra={
'assessment_name': assessment_name,
'score': score,
'valid': valid,
'action_valid': action_valid,
@@ -137,10 +145,13 @@ class RecaptchaService:
'has_suspicious_labels': has_suspicious_labels,
'allowed': allowed,
'user_ip': user_ip,
'user_id': user_id,
'email': email,
},
)
return AssessmentResult(
name=assessment_name,
score=score,
valid=valid,
action_valid=action_valid,

View File

@@ -216,9 +216,9 @@ class SaasUserAuth(UserAuth):
async def get_mcp_api_key(self) -> str:
api_key_store = ApiKeyStore.get_instance()
mcp_api_key = api_key_store.retrieve_mcp_api_key(self.user_id)
mcp_api_key = await api_key_store.retrieve_mcp_api_key(self.user_id)
if not mcp_api_key:
mcp_api_key = api_key_store.create_api_key(
mcp_api_key = await api_key_store.create_api_key(
self.user_id, 'MCP_API_KEY', None
)
return mcp_api_key

View File

@@ -16,6 +16,7 @@ from keycloak.exceptions import (
KeycloakError,
KeycloakPostError,
)
from server.auth.auth_error import ExpiredError
from server.auth.constants import (
BITBUCKET_APP_CLIENT_ID,
BITBUCKET_APP_CLIENT_SECRET,
@@ -426,6 +427,8 @@ class TokenManager:
access_token = data.get('access_token')
refresh_token = data.get('refresh_token')
if not access_token or not refresh_token:
if data.get('error') == 'bad_refresh_token':
raise ExpiredError()
raise ValueError(
'Failed to refresh token: missing access_token or refresh_token in response.'
)

View File

@@ -15,6 +15,11 @@ IS_FEATURE_ENV = (
) # Does not include the staging deployment
IS_LOCAL_ENV = bool(HOST == 'localhost')
# Role name constants
ROLE_OWNER = 'owner'
ROLE_ADMIN = 'admin'
ROLE_MEMBER = 'member'
# Deprecated - billing margins are now handled internally in litellm
DEFAULT_BILLING_MARGIN = float(os.environ.get('DEFAULT_BILLING_MARGIN', '1.0'))

View File

@@ -14,7 +14,6 @@ from storage.conversation_callback import (
ConversationCallback,
ConversationCallbackProcessor,
)
from storage.database import session_maker
from openhands.core.logger import openhands_logger as logger
from openhands.core.schema.agent import AgentState
@@ -108,13 +107,10 @@ class GithubCallbackProcessor(ConversationCallbackProcessor):
f'[GitHub] Sent summary instruction to conversation {conversation_id} {summary_event}'
)
# Update the processor state
# Update the processor state - the outer session will commit this
self.send_summary_instruction = False
callback.set_processor(self)
callback.updated_at = datetime.now()
with session_maker() as session:
session.merge(callback)
session.commit()
return
# Extract the summary from the event store
@@ -130,14 +126,15 @@ class GithubCallbackProcessor(ConversationCallbackProcessor):
logger.info(f'[GitHub] Summary sent for conversation {conversation_id}')
# Mark callback as completed status
# Mark callback as completed status - the outer session will commit this
callback.status = CallbackStatus.COMPLETED
callback.updated_at = datetime.now()
with session_maker() as session:
session.merge(callback)
session.commit()
except Exception as e:
logger.exception(
f'[GitHub] Error processing conversation callback: {str(e)}'
)
# Mark callback as error to prevent infinite re-invocation
# The outer session will commit this
callback.status = CallbackStatus.ERROR
callback.updated_at = datetime.now()

View File

@@ -22,7 +22,7 @@ from openhands.core.logger import openhands_logger as logger
# NOTE: these details are specific to the MCP protocol
class SaaSOpenHandsMCPConfig(OpenHandsMCPConfig):
@staticmethod
def create_default_mcp_server_config(
async def create_default_mcp_server_config(
host: str, config: 'OpenHandsConfig', user_id: str | None = None
) -> tuple[MCPSHTTPServerConfig | None, list[MCPStdioServerConfig]]:
"""
@@ -38,10 +38,12 @@ class SaaSOpenHandsMCPConfig(OpenHandsMCPConfig):
api_key_store = ApiKeyStore.get_instance()
if user_id:
api_key = api_key_store.retrieve_mcp_api_key(user_id)
api_key = await api_key_store.retrieve_mcp_api_key(user_id)
if not api_key:
api_key = api_key_store.create_api_key(user_id, 'MCP_API_KEY', None)
api_key = await api_key_store.create_api_key(
user_id, 'MCP_API_KEY', None
)
if not api_key:
logger.error(f'Could not provision MCP API Key for user: {user_id}')

View File

@@ -103,7 +103,7 @@ class SetAuthCookieMiddleware:
keycloak_auth_cookie = request.cookies.get('keycloak_auth')
auth_header = request.headers.get('Authorization')
mcp_auth_header = request.headers.get('X-Session-API-Key')
accepted_tos = False
accepted_tos: bool | None = False
if (
keycloak_auth_cookie is None
and (auth_header is None or not auth_header.startswith('Bearer '))
@@ -144,7 +144,7 @@ class SetAuthCookieMiddleware:
# "if accepted_tos is not None" as there should not be any users with
# accepted_tos equal to "None"
if accepted_tos is False and request.url.path != '/api/accept_tos':
logger.error('User has not accepted the terms of service')
logger.warning('User has not accepted the terms of service')
raise TosNotAcceptedError
def _should_attach(self, request: Request) -> bool:

View File

@@ -6,62 +6,53 @@ from storage.api_key_store import ApiKeyStore
from storage.lite_llm_manager import LiteLlmManager
from storage.org_member import OrgMember
from storage.org_member_store import OrgMemberStore
from storage.org_service import OrgService
from storage.user_store import UserStore
from openhands.core.logger import openhands_logger as logger
from openhands.server.user_auth import get_user_id
from openhands.utils.async_utils import call_sync_from_async
# Helper functions for BYOR API key management
async def get_byor_key_from_db(user_id: str) -> str | None:
"""Get the BYOR key from the database for a user."""
def _get_byor_key():
user = UserStore.get_user_by_id(user_id)
if not user:
return None
current_org_id = user.current_org_id
current_org_member: OrgMember = None
for org_member in user.org_members:
if org_member.org_id == current_org_id:
current_org_member = org_member
break
if not current_org_member:
return None
if current_org_member.llm_api_key_for_byor:
return current_org_member.llm_api_key_for_byor.get_secret_value()
user = await UserStore.get_user_by_id_async(user_id)
if not user:
return None
return await call_sync_from_async(_get_byor_key)
current_org_id = user.current_org_id
current_org_member: OrgMember = None
for org_member in user.org_members:
if org_member.org_id == current_org_id:
current_org_member = org_member
break
if not current_org_member:
return None
if current_org_member.llm_api_key_for_byor:
return current_org_member.llm_api_key_for_byor.get_secret_value()
return None
async def store_byor_key_in_db(user_id: str, key: str) -> None:
"""Store the BYOR key in the database for a user."""
user = await UserStore.get_user_by_id_async(user_id)
if not user:
return None
def _update_user_settings():
user = UserStore.get_user_by_id(user_id)
if not user:
return None
current_org_id = user.current_org_id
current_org_member: OrgMember = None
for org_member in user.org_members:
if org_member.org_id == current_org_id:
current_org_member = org_member
break
if not current_org_member:
return None
current_org_member.llm_api_key_for_byor = key
OrgMemberStore.update_org_member(current_org_member)
await call_sync_from_async(_update_user_settings)
current_org_id = user.current_org_id
current_org_member: OrgMember = None
for org_member in user.org_members:
if org_member.org_id == current_org_id:
current_org_member = org_member
break
if not current_org_member:
return None
current_org_member.llm_api_key_for_byor = key
OrgMemberStore.update_org_member(current_org_member)
async def generate_byor_key(user_id: str) -> str | None:
"""Generate a new BYOR key for a user."""
try:
user = await UserStore.get_user_by_id_async(user_id)
if not user:
@@ -157,15 +148,35 @@ class LlmApiKeyResponse(BaseModel):
key: str | None
class ByorPermittedResponse(BaseModel):
permitted: bool
@api_router.get('/llm/byor/permitted', response_model=ByorPermittedResponse)
async def check_byor_permitted(user_id: str = Depends(get_user_id)):
"""Check if BYOR key export is permitted for the user's current org."""
try:
permitted = await OrgService.check_byor_export_enabled(user_id)
return {'permitted': permitted}
except Exception as e:
logger.exception(
'Error checking BYOR export permission', extra={'error': str(e)}
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to check BYOR export permission',
)
@api_router.post('', response_model=ApiKeyCreateResponse)
async def create_api_key(key_data: ApiKeyCreate, user_id: str = Depends(get_user_id)):
"""Create a new API key for the authenticated user."""
try:
api_key = api_key_store.create_api_key(
api_key = await api_key_store.create_api_key(
user_id, key_data.name, key_data.expires_at
)
# Get the created key details
keys = api_key_store.list_api_keys(user_id)
keys = await api_key_store.list_api_keys(user_id)
for key in keys:
if key['name'] == key_data.name:
return {
@@ -193,7 +204,7 @@ async def create_api_key(key_data: ApiKeyCreate, user_id: str = Depends(get_user
async def list_api_keys(user_id: str = Depends(get_user_id)):
"""List all API keys for the authenticated user."""
try:
keys = api_key_store.list_api_keys(user_id)
keys = await api_key_store.list_api_keys(user_id)
return [
{
**key,
@@ -222,7 +233,7 @@ async def delete_api_key(key_id: int, user_id: str = Depends(get_user_id)):
"""Delete an API key."""
try:
# First, verify the key belongs to the user
keys = api_key_store.list_api_keys(user_id)
keys = await api_key_store.list_api_keys(user_id)
key_to_delete = None
for key in keys:
@@ -262,8 +273,17 @@ async def get_llm_api_key_for_byor(user_id: str = Depends(get_user_id)):
This endpoint validates that the key exists in LiteLLM before returning it.
If validation fails, it automatically generates a new key to ensure users
always receive a working key.
Returns 402 Payment Required if BYOR export is not enabled for the user's org.
"""
try:
# Check if BYOR export is enabled for the user's org
if not await OrgService.check_byor_export_enabled(user_id):
raise HTTPException(
status_code=status.HTTP_402_PAYMENT_REQUIRED,
detail='BYOR key export is not enabled. Purchase credits to enable this feature.',
)
# Check if the BYOR key exists in the database
byor_key = await get_byor_key_from_db(user_id)
if byor_key:
@@ -319,10 +339,20 @@ async def get_llm_api_key_for_byor(user_id: str = Depends(get_user_id)):
@api_router.post('/llm/byor/refresh', response_model=LlmApiKeyResponse)
async def refresh_llm_api_key_for_byor(user_id: str = Depends(get_user_id)):
"""Refresh the LLM API key for BYOR (Bring Your Own Runtime) for the authenticated user."""
"""Refresh the LLM API key for BYOR (Bring Your Own Runtime) for the authenticated user.
Returns 402 Payment Required if BYOR export is not enabled for the user's org.
"""
logger.info('Starting BYOR LLM API key refresh', extra={'user_id': user_id})
try:
# Check if BYOR export is enabled for the user's org
if not await OrgService.check_byor_export_enabled(user_id):
raise HTTPException(
status_code=status.HTTP_402_PAYMENT_REQUIRED,
detail='BYOR key export is not enabled. Purchase credits to enable this feature.',
)
# Get the existing BYOR key from the database
existing_byor_key = await get_byor_key_from_db(user_id)

View File

@@ -179,6 +179,9 @@ async def keycloak_callback(
user = await UserStore.get_user_by_id_async(user_id)
if not user:
user = await UserStore.create_user(user_id, user_info)
else:
# Existing user — gradually backfill contact_name if it still has a username-style value
await UserStore.backfill_contact_name(user_id, user_info)
if not user:
logger.error(f'Failed to authenticate user {user_info["preferred_username"]}')
@@ -219,6 +222,7 @@ async def keycloak_callback(
user_ip=user_ip,
user_agent=user_agent,
email=email,
user_id=user_id,
)
if not result.allowed:

View File

@@ -13,46 +13,34 @@ from server.constants import (
STRIPE_API_KEY,
)
from server.logger import logger
from starlette.datastructures import URL
from storage.billing_session import BillingSession
from storage.database import session_maker
from storage.lite_llm_manager import LiteLlmManager
from storage.org import Org
from storage.subscription_access import SubscriptionAccess
from storage.user_store import UserStore
from openhands.app_server.config import get_global_config
from openhands.server.user_auth import get_user_id
stripe.api_key = STRIPE_API_KEY
billing_router = APIRouter(prefix='/api/billing')
# TODO: Add a new app_mode named "ON_PREM" to support self-hosted customers instead of doing this
# and members should comment out the "validate_saas_environment" function if they are developing and testing locally.
def is_all_hands_saas_environment(request: Request) -> bool:
"""Check if the current domain is an All Hands SaaS environment.
Args:
request: FastAPI Request object
Returns:
True if the current domain contains "all-hands.dev" or "openhands.dev" postfix
async def validate_billing_enabled() -> None:
"""
hostname = request.url.hostname or ''
return hostname.endswith('all-hands.dev') or hostname.endswith('openhands.dev')
def validate_saas_environment(request: Request) -> None:
"""Validate that the request is coming from an All Hands SaaS environment.
Args:
request: FastAPI Request object
Raises:
HTTPException: If the request is not from an All Hands SaaS environment
Validate that the billing feature flag is enabled
"""
if not is_all_hands_saas_environment(request):
config = get_global_config()
web_client_config = await config.web_client.get_web_client_config()
if not web_client_config.feature_flags.enable_billing:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail='Checkout sessions are only available for All Hands SaaS environments',
detail=(
'Billing is disabled in this environment. '
'Please set OH_WEB_CLIENT_FEATURE_FLAGS_ENABLE_BILLING to enable billing.'
),
)
@@ -154,14 +142,15 @@ async def has_payment_method(user_id: str = Depends(get_user_id)) -> bool:
async def create_customer_setup_session(
request: Request, user_id: str = Depends(get_user_id)
) -> CreateBillingSessionResponse:
validate_saas_environment(request)
await validate_billing_enabled()
customer_info = await stripe_service.find_or_create_customer_by_user_id(user_id)
base_url = _get_base_url(request)
checkout_session = await stripe.checkout.Session.create_async(
customer=customer_info['customer_id'],
mode='setup',
payment_method_types=['card'],
success_url=f'{request.base_url}?free_credits=success',
cancel_url=f'{request.base_url}',
success_url=f'{base_url}?free_credits=success',
cancel_url=f'{base_url}',
)
return CreateBillingSessionResponse(redirect_url=checkout_session.url)
@@ -173,8 +162,8 @@ async def create_checkout_session(
request: Request,
user_id: str = Depends(get_user_id),
) -> CreateBillingSessionResponse:
validate_saas_environment(request)
await validate_billing_enabled()
base_url = _get_base_url(request)
customer_info = await stripe_service.find_or_create_customer_by_user_id(user_id)
checkout_session = await stripe.checkout.Session.create_async(
customer=customer_info['customer_id'],
@@ -197,8 +186,8 @@ async def create_checkout_session(
saved_payment_method_options={
'payment_method_save': 'enabled',
},
success_url=f'{request.base_url}api/billing/success?session_id={{CHECKOUT_SESSION_ID}}',
cancel_url=f'{request.base_url}api/billing/cancel?session_id={{CHECKOUT_SESSION_ID}}',
success_url=f'{base_url}api/billing/success?session_id={{CHECKOUT_SESSION_ID}}',
cancel_url=f'{base_url}api/billing/cancel?session_id={{CHECKOUT_SESSION_ID}}',
)
logger.info(
'created_stripe_checkout_session',
@@ -271,6 +260,12 @@ async def success_callback(session_id: str, request: Request):
str(user.current_org_id), new_max_budget
)
# Enable BYOR export for the org now that they've purchased credits
# Update within the same session to avoid nested session issues
org = session.query(Org).filter(Org.id == user.current_org_id).first()
if org:
org.byor_export_enabled = True
# Store transaction status
billing_session.status = 'completed'
billing_session.price = add_credits
@@ -289,7 +284,7 @@ async def success_callback(session_id: str, request: Request):
session.commit()
return RedirectResponse(
f'{request.base_url}settings/billing?checkout=success', status_code=302
f'{_get_base_url(request)}settings/billing?checkout=success', status_code=302
)
@@ -317,5 +312,13 @@ async def cancel_callback(session_id: str, request: Request):
session.commit()
return RedirectResponse(
f'{request.base_url}settings/billing?checkout=cancel', status_code=302
f'{_get_base_url(request)}settings/billing?checkout=cancel', status_code=302
)
def _get_base_url(request: Request) -> URL:
# Never send any part of the credit card process over a non secure connection
base_url = request.base_url
if base_url.hostname != 'localhost':
base_url = base_url.replace(scheme='https')
return base_url

View File

@@ -1,6 +1,7 @@
import hashlib
import json
import os
import zlib
from base64 import b64decode, b64encode
from urllib.parse import parse_qs, urlencode, urlparse
@@ -51,7 +52,11 @@ def add_github_proxy_routes(app: FastAPI):
state_payload = json.dumps(
[query_params['state'][0], query_params['redirect_uri'][0]]
)
state = b64encode(_fernet().encrypt(state_payload.encode())).decode()
# Compress before encrypting to reduce URL length
# This is critical for feature deployments where reCAPTCHA tokens in state
# can cause "URL too long" errors from GitHub
compressed_payload = zlib.compress(state_payload.encode())
state = b64encode(_fernet().encrypt(compressed_payload)).decode()
query_params['state'] = [state]
query_params['redirect_uri'] = [
f'https://{request.url.netloc}/github-proxy/callback'
@@ -67,7 +72,9 @@ def add_github_proxy_routes(app: FastAPI):
parsed_url = urlparse(str(request.url))
query_params = parse_qs(parsed_url.query)
state = query_params['state'][0]
decrypted_state = _fernet().decrypt(b64decode(state.encode())).decode()
# Decrypt and decompress (reverse of github_proxy_start)
decrypted_payload = _fernet().decrypt(b64decode(state.encode()))
decrypted_state = zlib.decompress(decrypted_payload).decode()
# Build query Params
state, redirect_uri = json.loads(decrypted_state)

View File

@@ -272,7 +272,7 @@ async def device_verification_authenticated(
try:
# Create a unique API key for this device using user_code in the name
device_key_name = f'{API_KEY_NAME} ({user_code})'
api_key_store.create_api_key(
await api_key_store.create_api_key(
user_id,
name=device_key_name,
expires_at=datetime.now(UTC) + KEY_EXPIRATION_TIME,

View File

@@ -1,4 +1,9 @@
from pydantic import BaseModel, EmailStr, Field
from typing import Annotated
from pydantic import BaseModel, EmailStr, Field, SecretStr, StringConstraints
from storage.org import Org
from storage.org_member import OrgMember
from storage.role import Role
class OrgCreationError(Exception):
@@ -27,13 +32,101 @@ class OrgDatabaseError(OrgCreationError):
pass
class OrgDeletionError(Exception):
"""Base exception for organization deletion errors."""
pass
class OrgAuthorizationError(OrgDeletionError):
"""Raised when user is not authorized to delete organization."""
def __init__(self, message: str = 'Not authorized to delete organization'):
super().__init__(message)
class OrphanedUserError(OrgDeletionError):
"""Raised when deleting an org would leave users without any organization."""
def __init__(self, user_ids: list[str]):
self.user_ids = user_ids
super().__init__(
f'Cannot delete organization: {len(user_ids)} user(s) would have no remaining organization'
)
class OrgNotFoundError(Exception):
"""Raised when organization is not found or user doesn't have access."""
def __init__(self, org_id: str):
self.org_id = org_id
super().__init__(f'Organization with id "{org_id}" not found')
class OrgMemberNotFoundError(Exception):
"""Raised when a member is not found in an organization."""
def __init__(self, org_id: str, user_id: str):
self.org_id = org_id
self.user_id = user_id
super().__init__(f'Member "{user_id}" not found in organization "{org_id}"')
class RoleNotFoundError(Exception):
"""Raised when a role is not found."""
def __init__(self, role_id: int):
self.role_id = role_id
super().__init__(f'Role with id "{role_id}" not found')
class InvalidRoleError(Exception):
"""Raised when an invalid role name is specified."""
def __init__(self, role_name: str):
self.role_name = role_name
super().__init__(f'Invalid role: "{role_name}"')
class InsufficientPermissionError(Exception):
"""Raised when user lacks permission to perform an operation."""
def __init__(self, message: str = 'Insufficient permission'):
super().__init__(message)
class CannotModifySelfError(Exception):
"""Raised when user attempts to modify their own membership."""
def __init__(self, action: str = 'modify'):
self.action = action
super().__init__(f'Cannot {action} your own membership')
class LastOwnerError(Exception):
"""Raised when attempting to remove or demote the last owner."""
def __init__(self, action: str = 'remove'):
self.action = action
super().__init__(f'Cannot {action} the last owner of an organization')
class MemberUpdateError(Exception):
"""Raised when member update operation fails."""
def __init__(self, message: str = 'Failed to update member'):
super().__init__(message)
class OrgCreate(BaseModel):
"""Request model for creating a new organization."""
# Required fields
name: str = Field(min_length=1, max_length=255, strip_whitespace=True)
name: Annotated[
str, StringConstraints(strip_whitespace=True, min_length=1, max_length=255)
]
contact_name: str
contact_email: EmailStr = Field(strip_whitespace=True)
contact_email: EmailStr
class OrgResponse(BaseModel):
@@ -65,3 +158,170 @@ class OrgResponse(BaseModel):
enable_solvability_analysis: bool | None = None
v1_enabled: bool | None = None
credits: float | None = None
is_personal: bool = False
@classmethod
def from_org(
cls, org: Org, credits: float | None = None, user_id: str | None = None
) -> 'OrgResponse':
"""Create an OrgResponse from an Org entity.
Args:
org: The organization entity to convert
credits: Optional credits value (defaults to None)
user_id: Optional user ID to determine if org is personal (defaults to None)
Returns:
OrgResponse: The response model instance
"""
return cls(
id=str(org.id),
name=org.name,
contact_name=org.contact_name,
contact_email=org.contact_email,
conversation_expiration=org.conversation_expiration,
agent=org.agent,
default_max_iterations=org.default_max_iterations,
security_analyzer=org.security_analyzer,
confirmation_mode=org.confirmation_mode,
default_llm_model=org.default_llm_model,
default_llm_api_key_for_byor=None,
default_llm_base_url=org.default_llm_base_url,
remote_runtime_resource_factor=org.remote_runtime_resource_factor,
enable_default_condenser=org.enable_default_condenser
if org.enable_default_condenser is not None
else True,
billing_margin=org.billing_margin,
enable_proactive_conversation_starters=org.enable_proactive_conversation_starters
if org.enable_proactive_conversation_starters is not None
else True,
sandbox_base_container_image=org.sandbox_base_container_image,
sandbox_runtime_container_image=org.sandbox_runtime_container_image,
org_version=org.org_version if org.org_version is not None else 0,
mcp_config=org.mcp_config,
search_api_key=None,
sandbox_api_key=None,
max_budget_per_task=org.max_budget_per_task,
enable_solvability_analysis=org.enable_solvability_analysis,
v1_enabled=org.v1_enabled,
credits=credits,
is_personal=str(org.id) == user_id if user_id else False,
)
class OrgPage(BaseModel):
"""Paginated response model for organization list."""
items: list[OrgResponse]
next_page_id: str | None = None
class OrgUpdate(BaseModel):
"""Request model for updating an organization."""
# Basic organization information (any authenticated user can update)
name: Annotated[
str | None,
StringConstraints(strip_whitespace=True, min_length=1, max_length=255),
] = None
contact_name: str | None = None
contact_email: EmailStr | None = None
conversation_expiration: int | None = None
default_max_iterations: int | None = Field(default=None, gt=0)
remote_runtime_resource_factor: int | None = Field(default=None, gt=0)
billing_margin: float | None = Field(default=None, ge=0, le=1)
enable_proactive_conversation_starters: bool | None = None
sandbox_base_container_image: str | None = None
sandbox_runtime_container_image: str | None = None
mcp_config: dict | None = None
sandbox_api_key: str | None = None
max_budget_per_task: float | None = Field(default=None, gt=0)
enable_solvability_analysis: bool | None = None
v1_enabled: bool | None = None
# LLM settings (require admin/owner role)
default_llm_model: str | None = None
default_llm_api_key_for_byor: str | None = None
default_llm_base_url: str | None = None
search_api_key: str | None = None
security_analyzer: str | None = None
agent: str | None = None
confirmation_mode: bool | None = None
enable_default_condenser: bool | None = None
condenser_max_size: int | None = Field(default=None, ge=20)
class OrgMemberResponse(BaseModel):
"""Response model for a single organization member."""
user_id: str
email: str | None
role_id: int
role_name: str
role_rank: int
status: str | None
class OrgMemberPage(BaseModel):
"""Paginated response for organization members."""
items: list[OrgMemberResponse]
next_page_id: str | None = None
class OrgMemberUpdate(BaseModel):
"""Request model for updating an organization member."""
role: str | None = None # Role name: 'owner', 'admin', or 'member'
class MeResponse(BaseModel):
"""Response model for the current user's membership in an organization."""
org_id: str
user_id: str
email: str
role: str
llm_api_key: str
max_iterations: int | None = None
llm_model: str | None = None
llm_api_key_for_byor: str | None = None
llm_base_url: str | None = None
status: str | None = None
@staticmethod
def _mask_key(secret: SecretStr | None) -> str:
"""Mask an API key, showing only last 4 characters."""
if secret is None:
return ''
raw = secret.get_secret_value()
if not raw:
return ''
if len(raw) <= 4:
return '****'
return '****' + raw[-4:]
@classmethod
def from_org_member(cls, member: OrgMember, role: Role, email: str) -> 'MeResponse':
"""Create a MeResponse from an OrgMember, Role, and user email.
Args:
member: The OrgMember entity
role: The Role entity (provides role name)
email: The user's email address
Returns:
MeResponse with masked API keys
"""
return cls(
org_id=str(member.org_id),
user_id=str(member.user_id),
email=email,
role=role.name,
llm_api_key=cls._mask_key(member.llm_api_key),
max_iterations=member.max_iterations,
llm_model=member.llm_model,
llm_api_key_for_byor=cls._mask_key(member.llm_api_key_for_byor) or None,
llm_base_url=member.llm_base_url,
status=member.status,
)

View File

@@ -1,20 +1,118 @@
from fastapi import APIRouter, Depends, HTTPException, status
from typing import Annotated
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Query, status
from server.auth.authorization import (
require_org_admin,
require_org_owner,
require_org_user,
)
from server.email_validation import get_admin_user_id
from server.routes.org_models import (
CannotModifySelfError,
InsufficientPermissionError,
InvalidRoleError,
LastOwnerError,
LiteLLMIntegrationError,
MemberUpdateError,
MeResponse,
OrgAuthorizationError,
OrgCreate,
OrgDatabaseError,
OrgMemberNotFoundError,
OrgMemberPage,
OrgMemberResponse,
OrgMemberUpdate,
OrgNameExistsError,
OrgNotFoundError,
OrgPage,
OrgResponse,
OrgUpdate,
OrphanedUserError,
RoleNotFoundError,
)
from server.services.org_member_service import OrgMemberService
from storage.org_service import OrgService
from openhands.core.logger import openhands_logger as logger
from openhands.server.user_auth import get_user_id
# Initialize API router
org_router = APIRouter(prefix='/api/organizations')
@org_router.get('', response_model=OrgPage)
async def list_user_orgs(
page_id: Annotated[
str | None,
Query(title='Optional next_page_id from the previously returned page'),
] = None,
limit: Annotated[
int,
Query(title='The max number of results in the page', gt=0, lte=100),
] = 100,
user_id: str = Depends(get_user_id),
) -> OrgPage:
"""List organizations for the authenticated user.
This endpoint returns a paginated list of all organizations that the
authenticated user is a member of.
Args:
page_id: Optional page ID (offset) for pagination
limit: Maximum number of organizations to return (1-100, default 100)
user_id: Authenticated user ID (injected by dependency)
Returns:
OrgPage: Paginated list of organizations
Raises:
HTTPException: 500 if retrieval fails
"""
logger.info(
'Listing organizations for user',
extra={
'user_id': user_id,
'page_id': page_id,
'limit': limit,
},
)
try:
# Fetch organizations from service layer
orgs, next_page_id = OrgService.get_user_orgs_paginated(
user_id=user_id,
page_id=page_id,
limit=limit,
)
# Convert Org entities to OrgResponse objects
org_responses = [
OrgResponse.from_org(org, credits=None, user_id=user_id) for org in orgs
]
logger.info(
'Successfully retrieved organizations',
extra={
'user_id': user_id,
'org_count': len(org_responses),
'has_more': next_page_id is not None,
},
)
return OrgPage(items=org_responses, next_page_id=next_page_id)
except Exception as e:
logger.exception(
'Unexpected error listing organizations',
extra={'user_id': user_id, 'error': str(e)},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to retrieve organizations',
)
@org_router.post('', response_model=OrgResponse, status_code=status.HTTP_201_CREATED)
async def create_org(
org_data: OrgCreate,
@@ -58,31 +156,7 @@ async def create_org(
# Retrieve credits from LiteLLM
credits = await OrgService.get_org_credits(user_id, org.id)
return OrgResponse(
id=str(org.id),
name=org.name,
contact_name=org.contact_name,
contact_email=org.contact_email,
conversation_expiration=org.conversation_expiration,
agent=org.agent,
default_max_iterations=org.default_max_iterations,
security_analyzer=org.security_analyzer,
confirmation_mode=org.confirmation_mode,
default_llm_model=org.default_llm_model,
default_llm_base_url=org.default_llm_base_url,
remote_runtime_resource_factor=org.remote_runtime_resource_factor,
enable_default_condenser=org.enable_default_condenser,
billing_margin=org.billing_margin,
enable_proactive_conversation_starters=org.enable_proactive_conversation_starters,
sandbox_base_container_image=org.sandbox_base_container_image,
sandbox_runtime_container_image=org.sandbox_runtime_container_image,
org_version=org.org_version,
mcp_config=org.mcp_config,
max_budget_per_task=org.max_budget_per_task,
enable_solvability_analysis=org.enable_solvability_analysis,
v1_enabled=org.v1_enabled,
credits=credits,
)
return OrgResponse.from_org(org, credits=credits, user_id=user_id)
except OrgNameExistsError as e:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
@@ -115,3 +189,607 @@ async def create_org(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='An unexpected error occurred',
)
@org_router.get('/{org_id}', response_model=OrgResponse, status_code=status.HTTP_200_OK)
async def get_org(
org_id: UUID,
user_id: str = Depends(require_org_user),
) -> OrgResponse:
"""Get organization details by ID.
This endpoint allows authenticated users who are members of an organization
to retrieve its details. Only members of the organization can access this endpoint.
Requires user, admin, or owner role.
Args:
org_id: Organization ID (UUID)
user_id: Authenticated user ID (injected by dependency, requires org membership)
Returns:
OrgResponse: The organization details
Raises:
HTTPException: 401 if user is not authenticated
HTTPException: 403 if user is not a member of the organization
HTTPException: 422 if org_id is not a valid UUID (handled by FastAPI)
HTTPException: 404 if organization not found
HTTPException: 500 if retrieval fails
"""
logger.info(
'Retrieving organization details',
extra={
'user_id': user_id,
'org_id': str(org_id),
},
)
try:
# Use service layer to get organization with membership validation
org = await OrgService.get_org_by_id(
org_id=org_id,
user_id=user_id,
)
# Retrieve credits from LiteLLM
credits = await OrgService.get_org_credits(user_id, org.id)
return OrgResponse.from_org(org, credits=credits, user_id=user_id)
except OrgNotFoundError as e:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
except Exception as e:
logger.exception(
'Unexpected error retrieving organization',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='An unexpected error occurred',
)
@org_router.get('/{org_id}/me', response_model=MeResponse)
async def get_me(
org_id: UUID,
user_id: str = Depends(get_user_id),
) -> MeResponse:
"""Get the current user's membership record for an organization.
Returns the authenticated user's role, status, email, and LLM override
fields (with masked API keys) within the specified organization.
Args:
org_id: Organization ID (UUID)
user_id: Authenticated user ID (injected by dependency)
Returns:
MeResponse: The user's membership data
Raises:
HTTPException: 404 if user is not a member or org doesn't exist
HTTPException: 500 if retrieval fails
"""
logger.info(
'Retrieving current member details',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
try:
user_uuid = UUID(user_id)
return OrgMemberService.get_me(org_id, user_uuid)
except OrgMemberNotFoundError:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f'Organization with id "{org_id}" not found',
)
except RoleNotFoundError as e:
logger.exception(
'Role not found for org member',
extra={
'user_id': user_id,
'org_id': str(org_id),
'role_id': e.role_id,
},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='An unexpected error occurred',
)
except Exception as e:
logger.exception(
'Unexpected error retrieving member details',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='An unexpected error occurred',
)
@org_router.delete('/{org_id}', status_code=status.HTTP_200_OK)
async def delete_org(
org_id: UUID,
user_id: str = Depends(require_org_owner),
) -> dict:
"""Delete an organization.
This endpoint allows authenticated organization owners to delete their organization.
All associated data including organization members, conversations, billing data,
and external LiteLLM team resources will be permanently removed.
Requires owner role.
Args:
org_id: Organization ID to delete
user_id: Authenticated user ID (injected by dependency, requires owner role)
Returns:
dict: Confirmation message with deleted organization details
Raises:
HTTPException: 401 if user is not authenticated
HTTPException: 403 if user is not an owner of the organization
HTTPException: 404 if organization not found
HTTPException: 500 if deletion fails
"""
logger.info(
'Organization deletion requested',
extra={
'user_id': user_id,
'org_id': str(org_id),
},
)
try:
# Use service layer to delete organization with cleanup
deleted_org = await OrgService.delete_org_with_cleanup(
user_id=user_id,
org_id=org_id,
)
logger.info(
'Organization deletion completed successfully',
extra={
'user_id': user_id,
'org_id': str(org_id),
'org_name': deleted_org.name,
},
)
return {
'message': 'Organization deleted successfully',
'organization': {
'id': str(deleted_org.id),
'name': deleted_org.name,
'contact_name': deleted_org.contact_name,
'contact_email': deleted_org.contact_email,
},
}
except OrgNotFoundError as e:
logger.warning(
'Organization not found for deletion',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
except OrgAuthorizationError as e:
logger.warning(
'User not authorized to delete organization',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=str(e),
)
except OrphanedUserError as e:
logger.warning(
'Cannot delete organization: users would be orphaned',
extra={
'user_id': user_id,
'org_id': str(org_id),
'orphaned_users': e.user_ids,
},
)
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=str(e),
)
except OrgDatabaseError as e:
logger.error(
'Database error during organization deletion',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to delete organization',
)
except Exception as e:
logger.exception(
'Unexpected error during organization deletion',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='An unexpected error occurred',
)
@org_router.patch('/{org_id}', response_model=OrgResponse)
async def update_org(
org_id: UUID,
update_data: OrgUpdate,
user_id: str = Depends(require_org_admin),
) -> OrgResponse:
"""Update an existing organization.
This endpoint allows authenticated admins and owners to update organization settings.
Requires admin or owner role in the organization.
Args:
org_id: Organization ID to update (UUID validated by FastAPI)
update_data: Organization update data
user_id: Authenticated user ID (injected by dependency, requires admin role)
Returns:
OrgResponse: The updated organization details
Raises:
HTTPException: 401 if user is not authenticated
HTTPException: 403 if user is not an admin or owner of the organization
HTTPException: 404 if organization not found
HTTPException: 422 if validation errors occur (handled by FastAPI)
HTTPException: 500 if update fails
"""
logger.info(
'Updating organization',
extra={
'user_id': user_id,
'org_id': str(org_id),
},
)
try:
# Use service layer to update organization with permission checks
updated_org = await OrgService.update_org_with_permissions(
org_id=org_id,
update_data=update_data,
user_id=user_id,
)
# Retrieve credits from LiteLLM (following same pattern as create endpoint)
credits = await OrgService.get_org_credits(user_id, updated_org.id)
return OrgResponse.from_org(updated_org, credits=credits, user_id=user_id)
except ValueError as e:
# Organization not found
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
except OrgNameExistsError as e:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=str(e),
)
except PermissionError as e:
# User lacks permission for LLM settings
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=str(e),
)
except OrgDatabaseError as e:
logger.error(
'Database operation failed',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to update organization',
)
except Exception as e:
logger.exception(
'Unexpected error updating organization',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='An unexpected error occurred',
)
@org_router.get('/{org_id}/members')
async def get_org_members(
org_id: str,
page_id: Annotated[
str | None,
Query(title='Optional next_page_id from the previously returned page'),
] = None,
limit: Annotated[
int,
Query(
title='The max number of results in the page',
gt=0,
lte=100,
),
] = 100,
current_user_id: str = Depends(get_user_id),
) -> OrgMemberPage:
"""Get all members of an organization with cursor-based pagination."""
try:
success, error_code, data = await OrgMemberService.get_org_members(
org_id=UUID(org_id),
current_user_id=UUID(current_user_id),
page_id=page_id,
limit=limit,
)
if not success:
error_map = {
'not_a_member': (
status.HTTP_403_FORBIDDEN,
'You are not a member of this organization',
),
'invalid_page_id': (
status.HTTP_400_BAD_REQUEST,
'Invalid page_id format',
),
}
status_code, detail = error_map.get(
error_code, (status.HTTP_500_INTERNAL_SERVER_ERROR, 'An error occurred')
)
raise HTTPException(status_code=status_code, detail=detail)
if data is None:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to retrieve members',
)
return data
except HTTPException:
raise
except ValueError:
logger.exception('Invalid UUID format')
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail='Invalid organization ID format',
)
except Exception:
logger.exception('Error retrieving organization members')
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to retrieve members',
)
@org_router.delete('/{org_id}/members/{user_id}')
async def remove_org_member(
org_id: str,
user_id: str,
current_user_id: str = Depends(get_user_id),
):
"""Remove a member from an organization.
Only owners and admins can remove members:
- Owners can remove admins and regular users
- Admins can only remove regular users
Users cannot remove themselves. The last owner cannot be removed.
"""
try:
success, error = await OrgMemberService.remove_org_member(
org_id=UUID(org_id),
target_user_id=UUID(user_id),
current_user_id=UUID(current_user_id),
)
if not success:
error_map = {
'not_a_member': (
status.HTTP_403_FORBIDDEN,
'You are not a member of this organization',
),
'cannot_remove_self': (
status.HTTP_403_FORBIDDEN,
'Cannot remove yourself from an organization',
),
'member_not_found': (
status.HTTP_404_NOT_FOUND,
'Member not found in this organization',
),
'insufficient_permission': (
status.HTTP_403_FORBIDDEN,
'You do not have permission to remove this member',
),
'cannot_remove_last_owner': (
status.HTTP_400_BAD_REQUEST,
'Cannot remove the last owner of an organization',
),
'removal_failed': (
status.HTTP_500_INTERNAL_SERVER_ERROR,
'Failed to remove member',
),
}
status_code, detail = error_map.get(
error, (status.HTTP_500_INTERNAL_SERVER_ERROR, 'An error occurred')
)
raise HTTPException(status_code=status_code, detail=detail)
return {'message': 'Member removed successfully'}
except HTTPException:
raise
except ValueError:
logger.exception('Invalid UUID format')
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail='Invalid organization or user ID format',
)
except Exception:
logger.exception('Error removing organization member')
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to remove member',
)
@org_router.post(
'/{org_id}/switch', response_model=OrgResponse, status_code=status.HTTP_200_OK
)
async def switch_org(
org_id: UUID,
user_id: str = Depends(get_user_id),
) -> OrgResponse:
"""Switch to a different organization.
This endpoint allows authenticated users to switch their current active
organization. The user must be a member of the target organization.
Args:
org_id: Organization ID to switch to (UUID)
user_id: Authenticated user ID (injected by dependency)
Returns:
OrgResponse: The organization details that was switched to
Raises:
HTTPException: 422 if org_id is not a valid UUID (handled by FastAPI)
HTTPException: 403 if user is not a member of the organization
HTTPException: 404 if organization not found
HTTPException: 500 if switch fails
"""
logger.info(
'Switching organization',
extra={
'user_id': user_id,
'org_id': str(org_id),
},
)
try:
# Use service layer to switch organization with membership validation
org = await OrgService.switch_org(
user_id=user_id,
org_id=org_id,
)
# Retrieve credits from LiteLLM for the new current org
credits = await OrgService.get_org_credits(user_id, org.id)
return OrgResponse.from_org(org, credits=credits, user_id=user_id)
except OrgNotFoundError as e:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
except OrgAuthorizationError as e:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=str(e),
)
except OrgDatabaseError as e:
logger.error(
'Database operation failed during organization switch',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to switch organization',
)
except Exception as e:
logger.exception(
'Unexpected error switching organization',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='An unexpected error occurred',
)
@org_router.patch('/{org_id}/members/{user_id}', response_model=OrgMemberResponse)
async def update_org_member(
org_id: str,
user_id: str,
update_data: OrgMemberUpdate,
current_user_id: str = Depends(get_user_id),
) -> OrgMemberResponse:
"""Update a member's role in an organization.
Permission rules:
- Admins can change roles of regular members to Admin or Member
- Admins cannot modify other Admins or Owners
- Owners can change roles of Admins and Members to any role (Owner, Admin, Member)
- Owners cannot modify other Owners
Members cannot modify their own role. The last owner cannot be demoted.
"""
try:
return await OrgMemberService.update_org_member(
org_id=UUID(org_id),
target_user_id=UUID(user_id),
current_user_id=UUID(current_user_id),
update_data=update_data,
)
except OrgMemberNotFoundError as e:
# Distinguish between requester not being a member vs target not found
if str(current_user_id) in str(e):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail='You are not a member of this organization',
)
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail='Member not found in this organization',
)
except CannotModifySelfError:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail='Cannot modify your own role',
)
except RoleNotFoundError:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Role configuration error',
)
except InvalidRoleError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail='Invalid role specified',
)
except InsufficientPermissionError:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail='You do not have permission to modify this member',
)
except LastOwnerError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail='Cannot demote the last owner of an organization',
)
except MemberUpdateError:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to update member',
)
except ValueError:
logger.exception('Invalid UUID format')
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail='Invalid organization or user ID format',
)
except Exception:
logger.exception('Error updating organization member')
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail='Failed to update member',
)

View File

@@ -4,6 +4,7 @@ from fastapi import APIRouter, Depends, Query, status
from fastapi.responses import JSONResponse
from pydantic import SecretStr
from server.auth.token_manager import TokenManager
from utils.identity import resolve_display_name
from openhands.integrations.provider import (
PROVIDER_TOKEN_TYPE,
@@ -121,6 +122,8 @@ async def saas_get_user(
login=(user_info.get('preferred_username') if user_info else '') or '',
avatar_url='',
email=user_info.get('email') if user_info else None,
name=resolve_display_name(user_info) if user_info else None,
company=user_info.get('company') if user_info else None,
),
user_info=user_info,
)

View File

@@ -516,11 +516,13 @@ class SaasNestedConversationManager(ConversationManager):
)
raise
def _get_mcp_config(self, user_id: str) -> MCPConfig | None:
async def _get_mcp_config(self, user_id: str) -> MCPConfig | None:
api_key_store = ApiKeyStore.get_instance()
mcp_api_key = api_key_store.retrieve_mcp_api_key(user_id)
mcp_api_key = await api_key_store.retrieve_mcp_api_key(user_id)
if not mcp_api_key:
mcp_api_key = api_key_store.create_api_key(user_id, 'MCP_API_KEY', None)
mcp_api_key = await api_key_store.create_api_key(
user_id, 'MCP_API_KEY', None
)
if not mcp_api_key:
return None
web_host = os.environ.get('WEB_HOST', 'app.all-hands.dev')
@@ -547,7 +549,7 @@ class SaasNestedConversationManager(ConversationManager):
'conversation_id': sid,
}
mcp_config = self._get_mcp_config(user_id)
mcp_config = await self._get_mcp_config(user_id)
if mcp_config:
# Merge with any MCP config from settings
if settings.mcp_config:
@@ -1137,6 +1139,71 @@ class SaasNestedConversationManager(ConversationManager):
}
update_conversation_metadata(conversation_id, metadata_content)
async def list_files(self, sid: str, path: str | None = None) -> list[str]:
"""List files in the workspace for a conversation.
Delegates to the nested container's list-files endpoint.
Args:
sid: The session/conversation ID.
path: Optional path to list files from. If None, lists from workspace root.
Returns:
A list of file paths.
Raises:
ValueError: If the conversation is not running.
httpx.HTTPError: If there's an error communicating with the nested runtime.
"""
runtime = await self._get_runtime(sid)
if runtime is None or runtime.get('status') != 'running':
raise ValueError(f'Conversation {sid} is not running')
nested_url = self._get_nested_url_for_runtime(runtime['runtime_id'], sid)
session_api_key = runtime.get('session_api_key')
return await self._fetch_list_files_from_nested(
sid, nested_url, session_api_key, path
)
async def select_file(self, sid: str, file: str) -> tuple[str | None, str | None]:
"""Read a file from the workspace via nested container.
Raises:
ValueError: If the conversation is not running.
httpx.HTTPError: If there's an error communicating with the nested runtime.
"""
runtime = await self._get_runtime(sid)
if runtime is None or runtime.get('status') != 'running':
raise ValueError(f'Conversation {sid} is not running')
nested_url = self._get_nested_url_for_runtime(runtime['runtime_id'], sid)
session_api_key = runtime.get('session_api_key')
return await self._fetch_select_file_from_nested(
sid, nested_url, session_api_key, file
)
async def upload_files(
self, sid: str, files: list[tuple[str, bytes]]
) -> tuple[list[str], list[dict[str, str]]]:
"""Upload files to the workspace via nested container.
Raises:
ValueError: If the conversation is not running.
httpx.HTTPError: If there's an error communicating with the nested runtime.
"""
runtime = await self._get_runtime(sid)
if runtime is None or runtime.get('status') != 'running':
raise ValueError(f'Conversation {sid} is not running')
nested_url = self._get_nested_url_for_runtime(runtime['runtime_id'], sid)
session_api_key = runtime.get('session_api_key')
return await self._fetch_upload_files_to_nested(
sid, nested_url, session_api_key, files
)
def _last_updated_at_key(conversation: ConversationMetadata) -> float:
last_updated_at = conversation.last_updated_at

View File

@@ -0,0 +1,342 @@
"""Service for managing organization members."""
from uuid import UUID
from server.constants import ROLE_ADMIN, ROLE_MEMBER, ROLE_OWNER
from server.routes.org_models import (
CannotModifySelfError,
InsufficientPermissionError,
InvalidRoleError,
LastOwnerError,
MemberUpdateError,
MeResponse,
OrgMemberNotFoundError,
OrgMemberPage,
OrgMemberResponse,
OrgMemberUpdate,
RoleNotFoundError,
)
from storage.org_member_store import OrgMemberStore
from storage.role_store import RoleStore
from storage.user_store import UserStore
from openhands.utils.async_utils import call_sync_from_async
class OrgMemberService:
"""Service for organization member operations."""
@staticmethod
def get_me(org_id: UUID, user_id: UUID) -> MeResponse:
"""Get the current user's membership record for an organization.
Retrieves the authenticated user's role, status, email, and LLM override
fields (with masked API keys) within the specified organization.
Args:
org_id: Organization ID (UUID)
user_id: User ID (UUID)
Returns:
MeResponse: The user's membership data with masked API keys
Raises:
OrgMemberNotFoundError: If user is not a member of the organization
RoleNotFoundError: If the role associated with the member is not found
"""
# Look up the user's membership in this org
org_member = OrgMemberStore.get_org_member(org_id, user_id)
if org_member is None:
raise OrgMemberNotFoundError(str(org_id), str(user_id))
# Resolve role name from role_id
role = RoleStore.get_role_by_id(org_member.role_id)
if role is None:
raise RoleNotFoundError(org_member.role_id)
# Get user email
user = UserStore.get_user_by_id(str(user_id))
email = user.email if user and user.email else ''
return MeResponse.from_org_member(org_member, role, email)
@staticmethod
async def get_org_members(
org_id: UUID,
current_user_id: UUID,
page_id: str | None = None,
limit: int = 100,
) -> tuple[bool, str | None, OrgMemberPage | None]:
"""Get organization members with authorization check.
Returns:
Tuple of (success, error_code, data). If success is True, error_code is None.
"""
# Verify current user is a member of the organization
requester_membership = OrgMemberStore.get_org_member(org_id, current_user_id)
if not requester_membership:
return False, 'not_a_member', None
# Parse page_id to get offset (page_id is offset encoded as string)
offset = 0
if page_id is not None:
try:
offset = int(page_id)
if offset < 0:
return False, 'invalid_page_id', None
except ValueError:
return False, 'invalid_page_id', None
# Call store to get paginated members
members, has_more = await OrgMemberStore.get_org_members_paginated(
org_id=org_id, offset=offset, limit=limit
)
# Transform data to response format
items = []
for member in members:
# Access user and role relationships (eagerly loaded)
user = member.user
role = member.role
items.append(
OrgMemberResponse(
user_id=str(member.user_id),
email=user.email if user else None,
role_id=member.role_id,
role_name=role.name if role else '',
role_rank=role.rank if role else 0,
status=member.status,
)
)
# Calculate next_page_id
next_page_id = None
if has_more:
next_page_id = str(offset + limit)
return True, None, OrgMemberPage(items=items, next_page_id=next_page_id)
@staticmethod
async def remove_org_member(
org_id: UUID,
target_user_id: UUID,
current_user_id: UUID,
) -> tuple[bool, str | None]:
"""Remove a member from an organization.
Returns:
Tuple of (success, error_message). If success is True, error_message is None.
"""
def _remove_member():
# Get current user's membership in the org
requester_membership = OrgMemberStore.get_org_member(
org_id, current_user_id
)
if not requester_membership:
return False, 'not_a_member'
# Check if trying to remove self
if str(current_user_id) == str(target_user_id):
return False, 'cannot_remove_self'
# Get target user's membership
target_membership = OrgMemberStore.get_org_member(org_id, target_user_id)
if not target_membership:
return False, 'member_not_found'
requester_role = RoleStore.get_role_by_id(requester_membership.role_id)
target_role = RoleStore.get_role_by_id(target_membership.role_id)
if not requester_role or not target_role:
return False, 'role_not_found'
# Check permission based on roles
if not OrgMemberService._can_remove_member(
requester_role.name, target_role.name
):
return False, 'insufficient_permission'
# Check if removing the last owner
if target_role.name == ROLE_OWNER:
if OrgMemberService._is_last_owner(org_id, target_user_id):
return False, 'cannot_remove_last_owner'
# Perform the removal
success = OrgMemberStore.remove_user_from_org(org_id, target_user_id)
if not success:
return False, 'removal_failed'
return True, None
return await call_sync_from_async(_remove_member)
@staticmethod
async def update_org_member(
org_id: UUID,
target_user_id: UUID,
current_user_id: UUID,
update_data: OrgMemberUpdate,
) -> OrgMemberResponse:
"""Update a member's role in an organization.
Permission rules:
- Admins can change roles of users (rank > ADMIN_RANK) to Admin or User
- Admins cannot modify other Admins or Owners
- Owners can change roles of non-owners (rank > OWNER_RANK) to any role
- Owners cannot modify other Owners
Args:
org_id: Organization ID
target_user_id: User ID of the member to update
current_user_id: User ID of the requester
update_data: Update data containing fields to modify
Returns:
OrgMemberResponse: The updated member data
Raises:
OrgMemberNotFoundError: If requester or target is not a member
CannotModifySelfError: If trying to modify self
RoleNotFoundError: If role configuration is invalid
InvalidRoleError: If new_role_name is not a valid role
InsufficientPermissionError: If requester lacks permission
LastOwnerError: If trying to demote the last owner
MemberUpdateError: If update operation fails
"""
new_role_name = update_data.role
def _update_member():
# Get current user's membership in the org
requester_membership = OrgMemberStore.get_org_member(
org_id, current_user_id
)
if not requester_membership:
raise OrgMemberNotFoundError(str(org_id), str(current_user_id))
# Check if trying to modify self
if str(current_user_id) == str(target_user_id):
raise CannotModifySelfError('modify')
# Get target user's membership
target_membership = OrgMemberStore.get_org_member(org_id, target_user_id)
if not target_membership:
raise OrgMemberNotFoundError(str(org_id), str(target_user_id))
# Get roles
requester_role = RoleStore.get_role_by_id(requester_membership.role_id)
target_role = RoleStore.get_role_by_id(target_membership.role_id)
if not requester_role:
raise RoleNotFoundError(requester_membership.role_id)
if not target_role:
raise RoleNotFoundError(target_membership.role_id)
# If no role change requested, return current state
if new_role_name is None:
user = UserStore.get_user_by_id(str(target_user_id))
return OrgMemberResponse(
user_id=str(target_membership.user_id),
email=user.email if user else None,
role_id=target_membership.role_id,
role_name=target_role.name,
role_rank=target_role.rank,
status=target_membership.status,
)
# Validate new role exists
new_role = RoleStore.get_role_by_name(new_role_name.lower())
if not new_role:
raise InvalidRoleError(new_role_name)
# Check permission to modify target
if not OrgMemberService._can_update_member_role(
requester_role.name, target_role.name, new_role.name
):
raise InsufficientPermissionError(
'You do not have permission to modify this member'
)
# Check if demoting the last owner
if (
target_role.name == ROLE_OWNER
and new_role.name != ROLE_OWNER
and OrgMemberService._is_last_owner(org_id, target_user_id)
):
raise LastOwnerError('demote')
# Perform the update
updated_member = OrgMemberStore.update_user_role_in_org(
org_id, target_user_id, new_role.id
)
if not updated_member:
raise MemberUpdateError('Failed to update member')
# Get user email for response
user = UserStore.get_user_by_id(str(target_user_id))
return OrgMemberResponse(
user_id=str(updated_member.user_id),
email=user.email if user else None,
role_id=updated_member.role_id,
role_name=new_role.name,
role_rank=new_role.rank,
status=updated_member.status,
)
return await call_sync_from_async(_update_member)
@staticmethod
def _can_update_member_role(
requester_role_name: str, target_role_name: str, new_role_name: str
) -> bool:
"""Check if requester can change target's role to new_role.
Permission rules:
- Owners can modify admins and users, can set any role
- Owners cannot modify other owners
- Admins can only modify users
- Admins can only set admin or user roles (not owner)
"""
is_requester_owner = requester_role_name == ROLE_OWNER
is_requester_admin = requester_role_name == ROLE_ADMIN
is_target_owner = target_role_name == ROLE_OWNER
is_target_admin = target_role_name == ROLE_ADMIN
is_new_role_owner = new_role_name == ROLE_OWNER
if is_requester_owner:
# Owners cannot modify other owners
if is_target_owner:
return False
# Owners can set any role (owner, admin, user)
return True
elif is_requester_admin:
# Admins cannot modify owners or other admins
if is_target_owner or is_target_admin:
return False
# Admins can only set admin or user roles (not owner)
return not is_new_role_owner
return False
@staticmethod
def _can_remove_member(requester_role_name: str, target_role_name: str) -> bool:
"""Check if requester can remove target based on roles."""
if requester_role_name == ROLE_OWNER:
return True
elif requester_role_name == ROLE_ADMIN:
# Admins can only remove members (not owners or other admins)
return target_role_name == ROLE_MEMBER
return False
@staticmethod
def _is_last_owner(org_id: UUID, user_id: UUID) -> bool:
"""Check if user is the last owner of the organization."""
members = OrgMemberStore.get_org_members(org_id)
owners = []
for m in members:
# Use role_id (column) instead of role (relationship) to avoid DetachedInstanceError
role = RoleStore.get_role_by_id(m.role_id)
if role and role.name == ROLE_OWNER:
owners.append(m)
return len(owners) == 1 and str(owners[0].user_id) == str(user_id)

View File

@@ -26,6 +26,7 @@ from server.sharing.shared_conversation_models import (
)
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from storage.stored_conversation_metadata_saas import StoredConversationMetadataSaas
from openhands.app_server.app_conversation.sql_app_conversation_info_service import (
StoredConversationMetadata,
@@ -57,7 +58,7 @@ class SQLSharedConversationInfoService(SharedConversationInfoService):
include_sub_conversations: bool = False,
) -> SharedConversationPage:
"""Search for shared conversations."""
query = self._public_select()
query = self._public_select_with_saas_metadata()
# Conditionally exclude sub-conversations based on the parameter
if not include_sub_conversations:
@@ -104,14 +105,17 @@ class SQLSharedConversationInfoService(SharedConversationInfoService):
query = query.limit(limit + 1)
result = await self.db_session.execute(query)
rows = result.scalars().all()
rows = result.all()
# Check if there are more results
has_more = len(rows) > limit
if has_more:
rows = rows[:limit]
items = [self._to_shared_conversation(row) for row in rows]
items = [
self._to_shared_conversation(stored, saas_metadata=saas_metadata)
for stored, saas_metadata in rows
]
# Calculate next page ID
next_page_id = None
@@ -152,17 +156,18 @@ class SQLSharedConversationInfoService(SharedConversationInfoService):
self, conversation_id: UUID
) -> SharedConversation | None:
"""Get a single public conversation info, returning None if missing or not shared."""
query = self._public_select().where(
query = self._public_select_with_saas_metadata().where(
StoredConversationMetadata.conversation_id == str(conversation_id)
)
result = await self.db_session.execute(query)
stored = result.scalar_one_or_none()
row = result.first()
if stored is None:
if row is None:
return None
return self._to_shared_conversation(stored)
stored, saas_metadata = row
return self._to_shared_conversation(stored, saas_metadata=saas_metadata)
def _public_select(self):
"""Create a select query that only returns public conversations."""
@@ -173,6 +178,25 @@ class SQLSharedConversationInfoService(SharedConversationInfoService):
query = query.where(StoredConversationMetadata.public == True) # noqa: E712
return query
def _public_select_with_saas_metadata(self):
"""Create a select query that returns public conversations with SAAS metadata.
This joins with conversation_metadata_saas to retrieve the user_id needed
for constructing the correct event storage path. Uses LEFT OUTER JOIN to
support conversations that may not have SAAS metadata (e.g., in tests).
"""
query = (
select(StoredConversationMetadata, StoredConversationMetadataSaas)
.outerjoin(
StoredConversationMetadataSaas,
StoredConversationMetadata.conversation_id
== StoredConversationMetadataSaas.conversation_id,
)
.where(StoredConversationMetadata.conversation_version == 'V1')
.where(StoredConversationMetadata.public == True) # noqa: E712
)
return query
def _apply_filters(
self,
query,
@@ -211,9 +235,16 @@ class SQLSharedConversationInfoService(SharedConversationInfoService):
def _to_shared_conversation(
self,
stored: StoredConversationMetadata,
saas_metadata: StoredConversationMetadataSaas | None = None,
sub_conversation_ids: list[UUID] | None = None,
) -> SharedConversation:
"""Convert StoredConversationMetadata to SharedConversation."""
"""Convert StoredConversationMetadata to SharedConversation.
Args:
stored: The base conversation metadata from conversation_metadata table.
saas_metadata: Optional SAAS metadata containing user_id and org_id.
sub_conversation_ids: Optional list of sub-conversation IDs.
"""
# V1 conversations should always have a sandbox_id
sandbox_id = stored.sandbox_id
assert sandbox_id is not None
@@ -239,9 +270,16 @@ class SQLSharedConversationInfoService(SharedConversationInfoService):
created_at = self._fix_timezone(stored.created_at)
updated_at = self._fix_timezone(stored.last_updated_at)
# Get user_id from SAAS metadata if available
created_by_user_id = (
str(saas_metadata.user_id)
if saas_metadata and saas_metadata.user_id
else None
)
return SharedConversation(
id=UUID(stored.conversation_id),
created_by_user_id=None, # user_id is no longer stored in conversation metadata
created_by_user_id=created_by_user_id,
sandbox_id=stored.sandbox_id,
selected_repository=stored.selected_repository,
selected_branch=stored.selected_branch,

View File

@@ -233,7 +233,13 @@ class SaasSQLAppConversationInfoService(SQLAppConversationInfoService):
result = result_set.first()
if result:
stored_metadata, saas_metadata = result
return self._to_info_with_user_id(stored_metadata, saas_metadata)
# Fetch sub-conversation IDs
sub_conversation_ids = await self.get_sub_conversation_ids(conversation_id)
return self._to_info_with_user_id(
stored_metadata,
saas_metadata,
sub_conversation_ids=sub_conversation_ids,
)
return None
async def batch_get_app_conversation_info(
@@ -262,8 +268,16 @@ class SaasSQLAppConversationInfoService(SQLAppConversationInfoService):
for conversation_id in conversation_id_strs:
if conversation_id in info_by_id:
stored_metadata, saas_metadata = info_by_id[conversation_id]
# Fetch sub-conversation IDs for each conversation
sub_conversation_ids = await self.get_sub_conversation_ids(
UUID(conversation_id)
)
results.append(
self._to_info_with_user_id(stored_metadata, saas_metadata)
self._to_info_with_user_id(
stored_metadata,
saas_metadata,
sub_conversation_ids=sub_conversation_ids,
)
)
else:
results.append(None)
@@ -316,10 +330,11 @@ class SaasSQLAppConversationInfoService(SQLAppConversationInfoService):
self,
stored: StoredConversationMetadata,
saas_metadata: StoredConversationMetadataSaas,
sub_conversation_ids: list[UUID] | None = None,
) -> AppConversationInfo:
"""Convert stored metadata to AppConversationInfo with user_id from SAAS metadata."""
# Use the base _to_info method to get the basic info
info = self._to_info(stored)
info = self._to_info(stored, sub_conversation_ids=sub_conversation_ids)
# Override the created_by_user_id with the user_id from SAAS metadata
info.created_by_user_id = (

View File

@@ -12,6 +12,7 @@ from storage.database import session_maker
from storage.user_store import UserStore
from openhands.core.logger import openhands_logger as logger
from openhands.utils.async_utils import call_sync_from_async
@dataclass
@@ -26,7 +27,7 @@ class ApiKeyStore:
random_part = ''.join(secrets.choice(alphabet) for _ in range(length))
return f'{self.API_KEY_PREFIX}{random_part}'
def create_api_key(
async def create_api_key(
self, user_id: str, name: str | None = None, expires_at: datetime | None = None
) -> str:
"""Create a new API key for a user.
@@ -40,8 +41,23 @@ class ApiKeyStore:
The generated API key
"""
api_key = self.generate_api_key()
user = UserStore.get_user_by_id(user_id)
user = await UserStore.get_user_by_id_async(user_id)
org_id = user.current_org_id
await call_sync_from_async(
self._store_api_key, user_id, org_id, api_key, name, expires_at
)
return api_key
def _store_api_key(
self,
user_id: str,
org_id: str,
api_key: str,
name: str | None,
expires_at: datetime | None = None,
) -> None:
"""Store an existing API key in the database."""
with self.session_maker() as session:
key_record = ApiKey(
key=api_key,
@@ -53,8 +69,6 @@ class ApiKeyStore:
session.add(key_record)
session.commit()
return api_key
def validate_api_key(self, api_key: str) -> str | None:
"""Validate an API key and return the associated user_id if valid."""
now = datetime.now(UTC)
@@ -112,10 +126,13 @@ class ApiKeyStore:
return True
def list_api_keys(self, user_id: str) -> list[dict]:
async def list_api_keys(self, user_id: str) -> list[dict]:
"""List all API keys for a user."""
user = UserStore.get_user_by_id(user_id)
user = await UserStore.get_user_by_id_async(user_id)
org_id = user.current_org_id
return await call_sync_from_async(self._list_api_keys_from_db, user_id, org_id)
def _list_api_keys_from_db(self, user_id: str, org_id: str) -> list[ApiKey]:
with self.session_maker() as session:
keys = (
session.query(ApiKey)
@@ -136,9 +153,14 @@ class ApiKeyStore:
if 'MCP_API_KEY' != key.name
]
def retrieve_mcp_api_key(self, user_id: str) -> str | None:
user = UserStore.get_user_by_id(user_id)
async def retrieve_mcp_api_key(self, user_id: str) -> str | None:
user = await UserStore.get_user_by_id_async(user_id)
org_id = user.current_org_id
return await call_sync_from_async(
self._retrieve_mcp_api_key_from_db, user_id, org_id
)
def _retrieve_mcp_api_key_from_db(self, user_id: str, org_id: str) -> str | None:
with self.session_maker() as session:
keys: list[ApiKey] = (
session.query(ApiKey)

View File

@@ -98,6 +98,29 @@ def decrypt_legacy_value(value: str | SecretStr) -> str:
return get_fernet().decrypt(b64decode(value.encode())).decode()
def encrypt_legacy_model(encrypt_keys: list, model_instance) -> dict:
return encrypt_legacy_kwargs(encrypt_keys, model_to_kwargs(model_instance))
def encrypt_legacy_kwargs(encrypt_keys: list, kwargs: dict) -> dict:
for key, value in kwargs.items():
if value is None:
continue
if key in encrypt_keys:
value = encrypt_legacy_value(value)
kwargs[key] = value
return kwargs
def encrypt_legacy_value(value: str | SecretStr) -> str:
if isinstance(value, SecretStr):
return b64encode(
get_fernet().encrypt(value.get_secret_value().encode())
).decode()
else:
return b64encode(get_fernet().encrypt(value.encode())).decode()
def get_fernet():
global _fernet
if _fernet is None:

View File

@@ -18,18 +18,29 @@ from server.constants import (
get_default_litellm_model,
)
from server.logger import logger
from storage.encrypt_utils import decrypt_legacy_value
from storage.user_settings import UserSettings
from openhands.server.settings import Settings
from openhands.utils.http_session import httpx_verify_option
# Timeout in seconds for BYOR key verification requests to LiteLLM
BYOR_KEY_VERIFICATION_TIMEOUT = 5.0
# Timeout in seconds for key verification requests to LiteLLM
KEY_VERIFICATION_TIMEOUT = 5.0
# A very large number to represent "unlimited" until LiteLLM fixes their unlimited update bug.
UNLIMITED_BUDGET_SETTING = 1000000000.0
def get_openhands_cloud_key_alias(keycloak_user_id: str, org_id: str) -> str:
"""Generate the key alias for OpenHands Cloud managed keys."""
return f'OpenHands Cloud - user {keycloak_user_id} - org {org_id}'
def get_byor_key_alias(keycloak_user_id: str, org_id: str) -> str:
"""Generate the key alias for BYOR (Bring Your Own Runtime) keys."""
return f'BYOR Key - user {keycloak_user_id}, org {org_id}'
class LiteLlmManager:
"""Manage LiteLLM interactions."""
@@ -78,7 +89,7 @@ class LiteLlmManager:
client,
keycloak_user_id,
org_id,
f'OpenHands Cloud - user {keycloak_user_id} - org {org_id}',
get_openhands_cloud_key_alias(keycloak_user_id, org_id),
None,
)
@@ -96,7 +107,7 @@ class LiteLlmManager:
user_settings: UserSettings,
) -> UserSettings | None:
logger.info(
'SettingsStore:umigrate_lite_llm_entries:start',
'LiteLlmManager:migrate_lite_llm_entries:start',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
if LITE_LLM_API_KEY is None or LITE_LLM_API_URL is None:
@@ -114,8 +125,24 @@ class LiteLlmManager:
if not user_json:
return None
user_info = user_json['user_info']
max_budget = user_info.get('max_budget', 0.0)
spend = user_info.get('spend', 0.0)
# Log original user values before any modifications for debugging
original_max_budget = user_info.get('max_budget')
original_spend = user_info.get('spend')
logger.info(
'LiteLlmManager:migrate_lite_llm_entries:original_user_values',
extra={
'org_id': org_id,
'user_id': keycloak_user_id,
'original_max_budget': original_max_budget,
'original_spend': original_spend,
},
)
max_budget = (
original_max_budget if original_max_budget is not None else 0.0
)
spend = original_spend if original_spend is not None else 0.0
# In upgrade to V4, we no longer use billing margin, but instead apply this directly
# in litellm. The default billing marign was 2 before this (hence the magic numbers below)
if (
@@ -136,39 +163,263 @@ class LiteLlmManager:
max_budget *= billing_margin
spend *= billing_margin
if not max_budget:
# if max_budget is None, then we've already migrated the User
# Check if max_budget is None (not 0.0) or set to unlimited to determine if already migrated
# A user with max_budget=0.0 is different from max_budget=None
if (
original_max_budget is None
or original_max_budget == UNLIMITED_BUDGET_SETTING
):
# if max_budget is None or UNLIMITED, then we've already migrated the User
logger.info(
'LiteLlmManager:migrate_lite_llm_entries:already_migrated',
extra={
'org_id': org_id,
'user_id': keycloak_user_id,
'original_max_budget': original_max_budget,
},
)
return None
credits = max(max_budget - spend, 0.0)
# Log calculated migration values before performing updates
logger.info(
'LiteLlmManager:migrate_lite_llm_entries:calculated_values',
extra={
'org_id': org_id,
'user_id': keycloak_user_id,
'adjusted_max_budget': max_budget,
'adjusted_spend': spend,
'calculated_credits': credits,
'new_user_max_budget': UNLIMITED_BUDGET_SETTING,
},
)
logger.debug(
'LiteLlmManager:migrate_lite_llm_entries:create_team',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
await LiteLlmManager._create_team(
client, keycloak_user_id, org_id, credits
)
logger.debug(
'LiteLlmManager:migrate_lite_llm_entries:update_user',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
await LiteLlmManager._update_user(
client, keycloak_user_id, max_budget=UNLIMITED_BUDGET_SETTING
)
logger.debug(
'LiteLlmManager:migrate_lite_llm_entries:add_user_to_team',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
await LiteLlmManager._add_user_to_team(
client, keycloak_user_id, org_id, credits
)
if user_settings.llm_api_key:
await LiteLlmManager._update_key(
client,
keycloak_user_id,
user_settings.llm_api_key,
team_id=org_id,
logger.debug(
'LiteLlmManager:migrate_lite_llm_entries:update_user_keys',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
await LiteLlmManager._update_user_keys(
client,
keycloak_user_id,
team_id=org_id,
)
# Check if the database key exists in LiteLLM
# If not, generate a new key to prevent verification failures later
db_key = None
if (
user_settings
and user_settings.llm_api_key
and user_settings.llm_base_url == LITE_LLM_API_URL
):
db_key = user_settings.llm_api_key
if hasattr(db_key, 'get_secret_value'):
db_key = db_key.get_secret_value()
if db_key:
# Verify the database key exists in LiteLLM
key_valid = await LiteLlmManager.verify_key(
db_key, keycloak_user_id
)
if not key_valid:
logger.warning(
'LiteLlmManager:migrate_lite_llm_entries:db_key_not_in_litellm',
extra={
'org_id': org_id,
'user_id': keycloak_user_id,
'key_prefix': db_key[:10] + '...'
if len(db_key) > 10
else db_key,
},
)
# Generate a new key for the user
new_key = await LiteLlmManager._generate_key(
client,
keycloak_user_id,
org_id,
get_openhands_cloud_key_alias(keycloak_user_id, org_id),
None,
)
if new_key:
logger.info(
'LiteLlmManager:migrate_lite_llm_entries:generated_new_key',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
# Update user_settings with the new key so it gets stored in org_member
user_settings.llm_api_key = SecretStr(new_key)
user_settings.llm_api_key_for_byor = SecretStr(new_key)
logger.info(
'LiteLlmManager:migrate_lite_llm_entries:complete',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
return user_settings
@staticmethod
async def downgrade_entries(
org_id: str,
keycloak_user_id: str,
user_settings: UserSettings,
) -> UserSettings | None:
"""Downgrade a migrated user's LiteLLM entries back to the pre-migration state.
This reverses the migrate_entries operation:
1. Get the user max budget from their org team in litellm
2. Set the max budget in the user in litellm (restore from team)
3. Add the user back to the default team in litellm
4. Update keys to remove org team association
5. Remove the user from their org team in litellm
6. Delete the user org team in litellm
Note: The database changes (already_migrated flag, org/org_member deletion)
should be handled separately by the caller.
Args:
org_id: The organization ID (which is also the team_id in litellm)
keycloak_user_id: The user's Keycloak ID
user_settings: The user's settings object
Returns:
The user_settings if downgrade was successful, None otherwise
"""
logger.info(
'LiteLlmManager:downgrade_entries:start',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
if LITE_LLM_API_KEY is None or LITE_LLM_API_URL is None:
logger.warning('LiteLLM API configuration not found')
return None
local_deploy = os.environ.get('LOCAL_DEPLOYMENT', None)
if not local_deploy:
async with httpx.AsyncClient(
headers={
'x-goog-api-key': LITE_LLM_API_KEY,
}
) as client:
# Step 1: Get the team info to retrieve the budget
logger.debug(
'LiteLlmManager:downgrade_entries:get_team',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
team_info = await LiteLlmManager._get_team(client, org_id)
if not team_info:
logger.error(
'LiteLlmManager:downgrade_entries:team_not_found',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
return None
# Get team budget (max_budget) and spend to calculate current credits
team_data = team_info.get('team_info', {})
max_budget = team_data.get('max_budget', 0.0)
spend = team_data.get('spend', 0.0)
# Get user membership info for budget in team
user_membership = await LiteLlmManager._get_user_team_info(
client, keycloak_user_id, org_id
)
if user_membership:
# Use user's budget in team if available
user_max_budget_in_team = user_membership.get('max_budget_in_team')
user_spend_in_team = user_membership.get('spend', 0.0)
if user_max_budget_in_team is not None:
max_budget = user_max_budget_in_team
spend = user_spend_in_team
# Calculate total budget to restore (credits + spend = max_budget)
# We restore the full max_budget that was on the team/user-in-team
restored_budget = max_budget if max_budget else 0.0
logger.debug(
'LiteLlmManager:downgrade_entries:budget_info',
extra={
'org_id': org_id,
'user_id': keycloak_user_id,
'max_budget': max_budget,
'spend': spend,
'restored_budget': restored_budget,
},
)
# Step 2: Update user to set their max_budget back from unlimited
logger.debug(
'LiteLlmManager:downgrade_entries:update_user',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
await LiteLlmManager._update_user(
client, keycloak_user_id, max_budget=restored_budget, spend=spend
)
# Step 3: Add user back to the default team
if LITE_LLM_TEAM_ID:
logger.debug(
'LiteLlmManager:downgrade_entries:add_to_default_team',
extra={
'org_id': org_id,
'user_id': keycloak_user_id,
'default_team_id': LITE_LLM_TEAM_ID,
},
)
await LiteLlmManager._add_user_to_team(
client, keycloak_user_id, LITE_LLM_TEAM_ID, restored_budget
)
if user_settings.llm_api_key_for_byor:
await LiteLlmManager._update_key(
client,
keycloak_user_id,
user_settings.llm_api_key_for_byor,
team_id=org_id,
)
# Step 4: Update all user keys to remove org team association (set team_id to default)
logger.debug(
'LiteLlmManager:downgrade_entries:update_user_keys',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
await LiteLlmManager._update_user_keys(
client,
keycloak_user_id,
team_id=LITE_LLM_TEAM_ID,
)
# Step 5: Remove user from their org team
logger.debug(
'LiteLlmManager:downgrade_entries:remove_from_org_team',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
await LiteLlmManager._remove_user_from_team(
client, keycloak_user_id, org_id
)
# Step 6: Delete the org team
logger.debug(
'LiteLlmManager:downgrade_entries:delete_team',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
await LiteLlmManager._delete_team(client, org_id)
logger.info(
'LiteLlmManager:downgrade_entries:complete',
extra={'org_id': org_id, 'user_id': keycloak_user_id},
)
return user_settings
@staticmethod
@@ -424,6 +675,13 @@ class LiteLlmManager:
logger.warning('LiteLLM API configuration not found')
return
try:
# Sometimes the key we get is encrypted - attempt to decrypt.
key = decrypt_legacy_value(key)
except Exception:
# The key was not encrypted
pass
payload = {
'key': key,
}
@@ -440,6 +698,7 @@ class LiteLlmManager:
'invalid_litellm_key_during_update',
extra={
'user_id': keycloak_user_id,
'text': response.text,
},
)
return
@@ -453,6 +712,77 @@ class LiteLlmManager:
)
response.raise_for_status()
@staticmethod
async def _get_user_keys(
client: httpx.AsyncClient,
keycloak_user_id: str,
) -> list[str]:
"""Get all keys for a user from LiteLLM.
Args:
client: The HTTP client to use for the request
keycloak_user_id: The user's Keycloak ID
Returns:
A list of key strings belonging to the user
"""
if LITE_LLM_API_KEY is None or LITE_LLM_API_URL is None:
logger.warning('LiteLLM API configuration not found')
return []
response = await client.get(
f'{LITE_LLM_API_URL}/key/list',
params={'user_id': keycloak_user_id},
)
if not response.is_success:
logger.error(
'error_getting_user_keys',
extra={
'status_code': response.status_code,
'text': response.text,
'user_id': keycloak_user_id,
},
)
return []
response_json = response.json()
keys = response_json.get('keys', [])
logger.debug(
'LiteLlmManager:_get_user_keys:keys_retrieved',
extra={
'user_id': keycloak_user_id,
'key_count': len(keys),
},
)
return keys
@staticmethod
async def _update_user_keys(
client: httpx.AsyncClient,
keycloak_user_id: str,
**kwargs,
):
"""Update all keys belonging to a user with the given parameters.
Args:
client: The HTTP client to use for the request
keycloak_user_id: The user's Keycloak ID
**kwargs: Parameters to update on each key (e.g., team_id)
"""
keys = await LiteLlmManager._get_user_keys(client, keycloak_user_id)
logger.debug(
'LiteLlmManager:_update_user_keys:updating_keys',
extra={
'user_id': keycloak_user_id,
'key_count': len(keys),
},
)
for key in keys:
await LiteLlmManager._update_key(client, keycloak_user_id, key, **kwargs)
@staticmethod
async def _delete_user(
client: httpx.AsyncClient,
@@ -613,6 +943,45 @@ class LiteLlmManager:
)
response.raise_for_status()
@staticmethod
async def _remove_user_from_team(
client: httpx.AsyncClient,
keycloak_user_id: str,
team_id: str,
):
if LITE_LLM_API_KEY is None or LITE_LLM_API_URL is None:
logger.warning('LiteLLM API configuration not found')
return
response = await client.post(
f'{LITE_LLM_API_URL}/team/member_delete',
json={
'team_id': team_id,
'user_id': keycloak_user_id,
},
)
if not response.is_success:
if response.status_code == 404:
# User not in team, that's fine for downgrade
logger.info(
'User not in team during removal',
extra={'user_id': keycloak_user_id, 'team_id': team_id},
)
return
logger.error(
'error_removing_litellm_user_from_team',
extra={
'status_code': response.status_code,
'text': response.text,
'user_id': keycloak_user_id,
'team_id': team_id,
},
)
response.raise_for_status()
logger.info(
'LiteLlmManager:_remove_user_from_team:user_removed',
extra={'user_id': keycloak_user_id, 'team_id': team_id},
)
@staticmethod
async def _generate_key(
client: httpx.AsyncClient,
@@ -685,7 +1054,7 @@ class LiteLlmManager:
try:
async with httpx.AsyncClient(
verify=httpx_verify_option(),
timeout=BYOR_KEY_VERIFICATION_TIMEOUT,
timeout=KEY_VERIFICATION_TIMEOUT,
) as client:
# Make a lightweight request to verify the key
# Using /v1/models endpoint as it's lightweight and requires authentication
@@ -699,7 +1068,7 @@ class LiteLlmManager:
# Only 200 status code indicates valid key
if response.status_code == 200:
logger.debug(
'BYOR key verification successful',
'Key verification successful',
extra={'user_id': user_id},
)
return True
@@ -707,7 +1076,7 @@ class LiteLlmManager:
# All other status codes (401, 403, 500, etc.) are treated as invalid
# This includes authentication errors and server errors
logger.warning(
'BYOR key verification failed - treating as invalid',
'Key verification failed - treating as invalid',
extra={
'user_id': user_id,
'status_code': response.status_code,
@@ -720,7 +1089,7 @@ class LiteLlmManager:
# Any exception (timeout, network error, etc.) means we can't verify
# Return False to trigger regeneration rather than returning potentially invalid key
logger.warning(
'BYOR key verification error - treating as invalid to ensure key validity',
'Key verification error - treating as invalid to ensure key validity',
extra={
'user_id': user_id,
'error': str(e),
@@ -764,6 +1133,103 @@ class LiteLlmManager:
'key_spend': key_info.get('spend'),
}
@staticmethod
async def _get_all_keys_for_user(
client: httpx.AsyncClient,
keycloak_user_id: str,
) -> list[dict]:
"""Get all keys for a user from LiteLLM.
Returns a list of key info dictionaries containing:
- token: the key value (hashed or partial)
- key_alias: the alias for the key
- key_name: the name of the key
- spend: the amount spent on this key
- max_budget: the max budget for this key
- team_id: the team the key belongs to
- metadata: any metadata associated with the key
Returns an empty list if no keys found or on error.
"""
if LITE_LLM_API_KEY is None or LITE_LLM_API_URL is None:
logger.warning('LiteLLM API configuration not found')
return []
try:
response = await client.get(
f'{LITE_LLM_API_URL}/user/info?user_id={keycloak_user_id}',
headers={'x-goog-api-key': LITE_LLM_API_KEY},
)
response.raise_for_status()
user_json = response.json()
# The user/info endpoint returns keys in the 'keys' field
return user_json.get('keys', [])
except Exception as e:
logger.warning(
'LiteLlmManager:_get_all_keys_for_user:error',
extra={
'user_id': keycloak_user_id,
'error': str(e),
},
)
return []
@staticmethod
async def _verify_existing_key(
client: httpx.AsyncClient,
key_value: str,
keycloak_user_id: str,
org_id: str,
openhands_type: bool = False,
) -> bool:
"""Check if an existing key exists for the user/org in LiteLLM.
Verifies the provided key_value matches a key registered in LiteLLM for
the given user and organization. For openhands_type=True, looks for keys
with metadata type='openhands' and matching team_id. For openhands_type=False,
looks for keys with matching alias and team_id.
Returns True if the key is found and valid, False otherwise.
"""
found = False
keys = await LiteLlmManager._get_all_keys_for_user(client, keycloak_user_id)
for key_info in keys:
metadata = key_info.get('metadata') or {}
team_id = key_info.get('team_id')
key_alias = key_info.get('key_alias')
token = None
if (
openhands_type
and metadata.get('type') == 'openhands'
and team_id == org_id
):
# Found an existing OpenHands key for this org
key_name = key_info.get('key_name')
token = key_name[-4:] if key_name else None # last 4 digits of key
if token and key_value.endswith(
token
): # check if this is our current key
found = True
break
if (
not openhands_type
and team_id == org_id
and (
key_alias == get_openhands_cloud_key_alias(keycloak_user_id, org_id)
or key_alias == get_byor_key_alias(keycloak_user_id, org_id)
)
):
# Found an existing key for this org (regardless of type)
key_name = key_info.get('key_name')
token = key_name[-4:] if key_name else None # last 4 digits of key
if token and key_value.endswith(
token
): # check if this is our current key
found = True
break
return found
@staticmethod
async def _delete_key_by_alias(
client: httpx.AsyncClient,
@@ -856,8 +1322,13 @@ class LiteLlmManager:
delete_user = staticmethod(with_http_client(_delete_user))
delete_team = staticmethod(with_http_client(_delete_team))
add_user_to_team = staticmethod(with_http_client(_add_user_to_team))
remove_user_from_team = staticmethod(with_http_client(_remove_user_from_team))
get_user_team_info = staticmethod(with_http_client(_get_user_team_info))
update_user_in_team = staticmethod(with_http_client(_update_user_in_team))
generate_key = staticmethod(with_http_client(_generate_key))
get_key_info = staticmethod(with_http_client(_get_key_info))
verify_existing_key = staticmethod(with_http_client(_verify_existing_key))
delete_key = staticmethod(with_http_client(_delete_key))
get_user_keys = staticmethod(with_http_client(_get_user_keys))
delete_key_by_alias = staticmethod(with_http_client(_delete_key_by_alias))
update_user_keys = staticmethod(with_http_client(_update_user_keys))

View File

@@ -46,6 +46,7 @@ class Org(Base): # type: ignore
v1_enabled = Column(Boolean, nullable=True)
conversation_expiration = Column(Integer, nullable=True)
condenser_max_size = Column(Integer, nullable=True)
byor_export_enabled = Column(Boolean, nullable=False, default=False)
# Relationships
org_members = relationship('OrgMember', back_populates='org')

View File

@@ -5,7 +5,9 @@ Store class for managing organization-member relationships.
from typing import Optional
from uuid import UUID
from storage.database import session_maker
from sqlalchemy import select
from sqlalchemy.orm import joinedload
from storage.database import a_session_maker, session_maker
from storage.org_member import OrgMember
from storage.user_settings import UserSettings
@@ -38,7 +40,7 @@ class OrgMemberStore:
return org_member
@staticmethod
def get_org_member(org_id: UUID, user_id: int) -> Optional[OrgMember]:
def get_org_member(org_id: UUID, user_id: UUID) -> Optional[OrgMember]:
"""Get organization-user relationship."""
with session_maker() as session:
return (
@@ -48,7 +50,18 @@ class OrgMemberStore:
)
@staticmethod
def get_user_orgs(user_id: int) -> list[OrgMember]:
async def get_org_member_async(org_id: UUID, user_id: UUID) -> Optional[OrgMember]:
"""Get organization-user relationship."""
async with a_session_maker() as session:
result = await session.execute(
select(OrgMember).filter(
OrgMember.org_id == org_id, OrgMember.user_id == user_id
)
)
return result.scalars().first()
@staticmethod
def get_user_orgs(user_id: UUID) -> list[OrgMember]:
"""Get all organizations for a user."""
with session_maker() as session:
return session.query(OrgMember).filter(OrgMember.user_id == user_id).all()
@@ -68,7 +81,7 @@ class OrgMemberStore:
@staticmethod
def update_user_role_in_org(
org_id: UUID, user_id: int, role_id: int, status: Optional[str] = None
org_id: UUID, user_id: UUID, role_id: int, status: Optional[str] = None
) -> Optional[OrgMember]:
"""Update user's role in an organization."""
with session_maker() as session:
@@ -90,7 +103,7 @@ class OrgMemberStore:
return org_member
@staticmethod
def remove_user_from_org(org_id: UUID, user_id: int) -> bool:
def remove_user_from_org(org_id: UUID, user_id: UUID) -> bool:
"""Remove a user from an organization."""
with session_maker() as session:
org_member = (
@@ -123,3 +136,36 @@ class OrgMemberStore:
if (normalized := c.name.lstrip('_')) and hasattr(user_settings, normalized)
}
return kwargs
@staticmethod
async def get_org_members_paginated(
org_id: UUID,
offset: int = 0,
limit: int = 100,
) -> tuple[list[OrgMember], bool]:
"""Get paginated list of organization members with user and role info.
Returns:
Tuple of (members_list, has_more) where has_more indicates if there are more results.
"""
async with a_session_maker() as session:
# Query for limit + 1 items to determine if there are more results
# Order by user_id for consistent pagination
query = (
select(OrgMember)
.options(joinedload(OrgMember.user), joinedload(OrgMember.role))
.filter(OrgMember.org_id == org_id)
.order_by(OrgMember.user_id)
.offset(offset)
.limit(limit + 1)
)
result = await session.execute(query)
members = list(result.scalars().all())
# Check if there are more results
has_more = len(members) > limit
if has_more:
# Remove the extra item
members = members[:limit]
return members, has_more

View File

@@ -9,8 +9,11 @@ from uuid import UUID as parse_uuid
from server.constants import ORG_SETTINGS_VERSION, get_default_litellm_model
from server.routes.org_models import (
LiteLLMIntegrationError,
OrgAuthorizationError,
OrgDatabaseError,
OrgNameExistsError,
OrgNotFoundError,
OrgUpdate,
)
from storage.lite_llm_manager import LiteLlmManager
from storage.org import Org
@@ -393,6 +396,243 @@ class OrgService:
)
return e
@staticmethod
def has_admin_or_owner_role(user_id: str, org_id: UUID) -> bool:
"""
Check if user has admin or owner role in the specified organization.
Args:
user_id: User ID to check
org_id: Organization ID to check membership in
Returns:
bool: True if user has admin or owner role, False otherwise
"""
try:
# Parse user_id as UUID for database query
user_uuid = parse_uuid(user_id)
# Get the user's membership in this organization
# Note: The type annotation says int but the actual column is UUID
org_member = OrgMemberStore.get_org_member(org_id, user_uuid)
if not org_member:
return False
# Get the role details
role = RoleStore.get_role_by_id(org_member.role_id)
if not role:
return False
# Admin and owner roles have elevated permissions
# Based on test files, both admin and owner have rank 1
return role.name in ['admin', 'owner']
except Exception as e:
logger.warning(
'Error checking user role in organization',
extra={
'user_id': user_id,
'org_id': str(org_id),
'error': str(e),
},
)
return False
@staticmethod
def is_org_member(user_id: str, org_id: UUID) -> bool:
"""
Check if user is a member of the specified organization.
Args:
user_id: User ID to check
org_id: Organization ID to check membership in
Returns:
bool: True if user is a member, False otherwise
"""
try:
user_uuid = parse_uuid(user_id)
org_member = OrgMemberStore.get_org_member(org_id, user_uuid)
return org_member is not None
except Exception as e:
logger.warning(
'Error checking user membership in organization',
extra={
'user_id': user_id,
'org_id': str(org_id),
'error': str(e),
},
)
return False
@staticmethod
def _get_llm_settings_fields() -> set[str]:
"""
Get the set of organization fields that are considered LLM settings
and require admin/owner role to update.
Returns:
set[str]: Set of field names that require elevated permissions
"""
return {
'default_llm_model',
'default_llm_api_key_for_byor',
'default_llm_base_url',
'search_api_key',
'security_analyzer',
'agent',
'confirmation_mode',
'enable_default_condenser',
'condenser_max_size',
}
@staticmethod
def _has_llm_settings_updates(update_data: OrgUpdate) -> set[str]:
"""
Check if the update contains any LLM settings fields.
Args:
update_data: The organization update data
Returns:
set[str]: Set of LLM fields being updated (empty if none)
"""
llm_fields = OrgService._get_llm_settings_fields()
update_dict = update_data.model_dump(exclude_none=True)
return llm_fields.intersection(update_dict.keys())
@staticmethod
async def update_org_with_permissions(
org_id: UUID,
update_data: OrgUpdate,
user_id: str,
) -> Org:
"""
Update organization with permission checks for LLM settings.
Args:
org_id: Organization UUID to update
update_data: Organization update data from request
user_id: ID of the user requesting the update
Returns:
Org: The updated organization object
Raises:
ValueError: If organization not found
PermissionError: If user is not a member, or lacks admin/owner role for LLM settings
OrgNameExistsError: If new name already exists for another organization
OrgDatabaseError: If database update fails
"""
logger.info(
'Updating organization with permission checks',
extra={
'org_id': str(org_id),
'user_id': user_id,
'has_update_data': update_data is not None,
},
)
# Validate organization exists
existing_org = OrgStore.get_org_by_id(org_id)
if not existing_org:
raise ValueError(f'Organization with ID {org_id} not found')
# Check if user is a member of this organization
if not OrgService.is_org_member(user_id, org_id):
logger.warning(
'Non-member attempted to update organization',
extra={
'user_id': user_id,
'org_id': str(org_id),
},
)
raise PermissionError(
'User must be a member of the organization to update it'
)
# Check if name is being updated and validate uniqueness
if update_data.name is not None:
# Check if new name conflicts with another org
existing_org_with_name = OrgStore.get_org_by_name(update_data.name)
if (
existing_org_with_name is not None
and existing_org_with_name.id != org_id
):
logger.warning(
'Attempted to update organization with duplicate name',
extra={
'user_id': user_id,
'org_id': str(org_id),
'attempted_name': update_data.name,
},
)
raise OrgNameExistsError(update_data.name)
# Check if update contains any LLM settings
llm_fields_being_updated = OrgService._has_llm_settings_updates(update_data)
if llm_fields_being_updated:
# Verify user has admin or owner role
has_permission = OrgService.has_admin_or_owner_role(user_id, org_id)
if not has_permission:
logger.warning(
'User attempted to update LLM settings without permission',
extra={
'user_id': user_id,
'org_id': str(org_id),
'attempted_fields': list(llm_fields_being_updated),
},
)
raise PermissionError(
'Admin or owner role required to update LLM settings'
)
logger.debug(
'User has permission to update LLM settings',
extra={
'user_id': user_id,
'org_id': str(org_id),
'llm_fields': list(llm_fields_being_updated),
},
)
# Convert to dict for OrgStore (excluding None values)
update_dict = update_data.model_dump(exclude_none=True)
if not update_dict:
logger.info(
'No fields to update',
extra={'org_id': str(org_id), 'user_id': user_id},
)
return existing_org
# Perform the update
try:
updated_org = OrgStore.update_org(org_id, update_dict)
if not updated_org:
raise OrgDatabaseError('Failed to update organization in database')
logger.info(
'Organization updated successfully',
extra={
'org_id': str(org_id),
'user_id': user_id,
'updated_fields': list(update_dict.keys()),
},
)
return updated_org
except Exception as e:
logger.error(
'Failed to update organization',
extra={
'org_id': str(org_id),
'user_id': user_id,
'error': str(e),
},
)
raise OrgDatabaseError(f'Failed to update organization: {str(e)}')
@staticmethod
async def get_org_credits(user_id: str, org_id: UUID) -> float | None:
"""
@@ -441,3 +681,274 @@ class OrgService:
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
return None
@staticmethod
def get_user_orgs_paginated(
user_id: str, page_id: str | None = None, limit: int = 100
):
"""
Get paginated list of organizations for a user.
Args:
user_id: User ID (string that will be converted to UUID)
page_id: Optional page ID (offset as string) for pagination
limit: Maximum number of organizations to return
Returns:
Tuple of (list of Org objects, next_page_id or None)
"""
logger.debug(
'Fetching paginated organizations for user',
extra={'user_id': user_id, 'page_id': page_id, 'limit': limit},
)
# Convert user_id string to UUID
user_uuid = parse_uuid(user_id)
# Fetch organizations from store
orgs, next_page_id = OrgStore.get_user_orgs_paginated(
user_id=user_uuid, page_id=page_id, limit=limit
)
logger.debug(
'Retrieved organizations for user',
extra={
'user_id': user_id,
'org_count': len(orgs),
'has_more': next_page_id is not None,
},
)
return orgs, next_page_id
@staticmethod
async def get_org_by_id(org_id: UUID, user_id: str) -> Org:
"""
Get organization by ID with membership validation.
This method verifies that the user is a member of the organization
before returning the organization details.
Args:
org_id: Organization ID
user_id: User ID (string that will be converted to UUID)
Returns:
Org: The organization object
Raises:
OrgNotFoundError: If organization not found or user is not a member
"""
logger.info(
'Retrieving organization',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
# Verify user is a member of the organization
org_member = OrgMemberStore.get_org_member(org_id, parse_uuid(user_id))
if not org_member:
logger.warning(
'User is not a member of organization or organization does not exist',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
raise OrgNotFoundError(str(org_id))
# Retrieve organization
org = OrgStore.get_org_by_id(org_id)
if not org:
logger.error(
'Organization not found despite valid membership',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
raise OrgNotFoundError(str(org_id))
logger.info(
'Successfully retrieved organization',
extra={
'org_id': str(org.id),
'org_name': org.name,
'user_id': user_id,
},
)
return org
@staticmethod
def verify_owner_authorization(user_id: str, org_id: UUID) -> None:
"""
Verify that the user is the owner of the organization.
Args:
user_id: User ID to check
org_id: Organization ID
Raises:
OrgNotFoundError: If organization doesn't exist
OrgAuthorizationError: If user is not authorized to delete
"""
# Check if organization exists
org = OrgStore.get_org_by_id(org_id)
if not org:
raise OrgNotFoundError(str(org_id))
# Check if user is a member of the organization
org_member = OrgMemberStore.get_org_member(org_id, parse_uuid(user_id))
if not org_member:
raise OrgAuthorizationError('User is not a member of this organization')
# Check if user has owner role
role = RoleStore.get_role_by_id(org_member.role_id)
if not role or role.name != 'owner':
raise OrgAuthorizationError(
'Only organization owners can delete organizations'
)
logger.debug(
'User authorization verified for organization deletion',
extra={'user_id': user_id, 'org_id': str(org_id), 'role': role.name},
)
@staticmethod
async def delete_org_with_cleanup(user_id: str, org_id: UUID) -> Org:
"""
Delete organization with complete cleanup of all associated data.
This method performs the complete organization deletion workflow:
1. Verifies user authorization (owner only)
2. Performs database cascade deletion and LiteLLM cleanup in single transaction
Args:
user_id: User ID requesting deletion (must be owner)
org_id: Organization ID to delete
Returns:
Org: The deleted organization details
Raises:
OrgNotFoundError: If organization doesn't exist
OrgAuthorizationError: If user is not authorized to delete
OrgDatabaseError: If database operations or LiteLLM cleanup fail
"""
logger.info(
'Starting organization deletion',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
# Step 1: Verify user authorization
OrgService.verify_owner_authorization(user_id, org_id)
# Step 2: Perform database cascade deletion with LiteLLM cleanup in transaction
try:
deleted_org = await OrgStore.delete_org_cascade(org_id)
if not deleted_org:
# This shouldn't happen since we verified existence above
raise OrgDatabaseError('Organization not found during deletion')
logger.info(
'Organization deletion completed successfully',
extra={
'user_id': user_id,
'org_id': str(org_id),
'org_name': deleted_org.name,
},
)
return deleted_org
except Exception as e:
logger.error(
'Organization deletion failed',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise OrgDatabaseError(f'Failed to delete organization: {str(e)}')
@staticmethod
async def check_byor_export_enabled(user_id: str) -> bool:
"""Check if BYOR export is enabled for the user's current org.
Returns True if the user's current org has byor_export_enabled set to True.
Returns False if the user is not found, has no current org, or the flag is False.
Args:
user_id: User ID to check
Returns:
bool: True if BYOR export is enabled, False otherwise
"""
user = await UserStore.get_user_by_id_async(user_id)
if not user or not user.current_org_id:
return False
org = OrgStore.get_org_by_id(user.current_org_id)
if not org:
return False
return org.byor_export_enabled
@staticmethod
async def switch_org(user_id: str, org_id: UUID) -> Org:
"""
Switch user's current organization to the specified organization.
This method:
1. Validates that the organization exists
2. Validates that the user is a member of the organization
3. Updates the user's current_org_id
Args:
user_id: User ID (string that will be converted to UUID)
org_id: Organization ID to switch to
Returns:
Org: The organization that was switched to
Raises:
OrgNotFoundError: If organization doesn't exist
OrgAuthorizationError: If user is not a member of the organization
OrgDatabaseError: If database update fails
"""
logger.info(
'Switching user organization',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
# Step 1: Check if organization exists
org = OrgStore.get_org_by_id(org_id)
if not org:
raise OrgNotFoundError(str(org_id))
# Step 2: Validate user is a member of the organization
if not OrgService.is_org_member(user_id, org_id):
logger.warning(
'User attempted to switch to organization they are not a member of',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
raise OrgAuthorizationError(
'User must be a member of the organization to switch to it'
)
# Step 3: Update user's current_org_id
try:
updated_user = UserStore.update_current_org(user_id, org_id)
if not updated_user:
raise OrgDatabaseError('User not found')
logger.info(
'Successfully switched user organization',
extra={
'user_id': user_id,
'org_id': str(org_id),
'org_name': org.name,
},
)
return org
except OrgDatabaseError:
raise
except Exception as e:
logger.error(
'Failed to switch user organization',
extra={'user_id': user_id, 'org_id': str(org_id), 'error': str(e)},
)
raise OrgDatabaseError(f'Failed to switch organization: {str(e)}')

View File

@@ -10,8 +10,11 @@ from server.constants import (
ORG_SETTINGS_VERSION,
get_default_litellm_model,
)
from server.routes.org_models import OrphanedUserError
from sqlalchemy import text
from sqlalchemy.orm import joinedload
from storage.database import session_maker
from storage.lite_llm_manager import LiteLlmManager
from storage.org import Org
from storage.org_member import OrgMember
from storage.user import User
@@ -96,6 +99,63 @@ class OrgStore:
orgs = session.query(Org).all()
return orgs
@staticmethod
def get_user_orgs_paginated(
user_id: UUID, page_id: str | None = None, limit: int = 100
) -> tuple[list[Org], str | None]:
"""
Get paginated list of organizations for a user.
Args:
user_id: User UUID
page_id: Optional page ID (offset as string) for pagination
limit: Maximum number of organizations to return
Returns:
Tuple of (list of Org objects, next_page_id or None)
"""
with session_maker() as session:
# Build query joining OrgMember with Org
query = (
session.query(Org)
.join(OrgMember, Org.id == OrgMember.org_id)
.filter(OrgMember.user_id == user_id)
.order_by(Org.name)
)
# Apply pagination offset
if page_id is not None:
try:
offset = int(page_id)
query = query.offset(offset)
except ValueError:
# If page_id is not a valid integer, start from beginning
offset = 0
else:
offset = 0
# Fetch limit + 1 to check if there are more results
query = query.limit(limit + 1)
orgs = query.all()
# Check if there are more results
has_more = len(orgs) > limit
if has_more:
orgs = orgs[:limit]
# Calculate next page ID
next_page_id = None
if has_more:
next_page_id = str(offset + limit)
# Validate org versions
validated_orgs = [
OrgStore._validate_org_version(org) for org in orgs if org
]
validated_orgs = [org for org in validated_orgs if org is not None]
return validated_orgs, next_page_id
@staticmethod
def update_org(
org_id: UUID,
@@ -186,3 +246,143 @@ class OrgStore:
session.commit()
session.refresh(org)
return org
@staticmethod
async def delete_org_cascade(org_id: UUID) -> Org | None:
"""
Delete organization and all associated data in cascade, including external LiteLLM cleanup.
Args:
org_id: UUID of the organization to delete
Returns:
Org: The deleted organization object, or None if not found
Raises:
Exception: If database operations or LiteLLM cleanup fail
"""
with session_maker() as session:
# First get the organization to return it
org = session.query(Org).filter(Org.id == org_id).first()
if not org:
return None
try:
# 1. Delete conversation data for organization conversations
session.execute(
text("""
DELETE FROM conversation_metadata
WHERE conversation_id IN (
SELECT conversation_id FROM conversation_metadata_saas WHERE org_id = :org_id
)
"""),
{'org_id': str(org_id)},
)
session.execute(
text("""
DELETE FROM app_conversation_start_task
WHERE app_conversation_id::text IN (
SELECT conversation_id FROM conversation_metadata_saas WHERE org_id = :org_id
)
"""),
{'org_id': str(org_id)},
)
# 2. Delete organization-owned data tables (direct org_id foreign keys)
session.execute(
text('DELETE FROM billing_sessions WHERE org_id = :org_id'),
{'org_id': str(org_id)},
)
session.execute(
text(
'DELETE FROM conversation_metadata_saas WHERE org_id = :org_id'
),
{'org_id': str(org_id)},
)
session.execute(
text('DELETE FROM custom_secrets WHERE org_id = :org_id'),
{'org_id': str(org_id)},
)
session.execute(
text('DELETE FROM api_keys WHERE org_id = :org_id'),
{'org_id': str(org_id)},
)
session.execute(
text('DELETE FROM slack_conversation WHERE org_id = :org_id'),
{'org_id': str(org_id)},
)
session.execute(
text('DELETE FROM slack_users WHERE org_id = :org_id'),
{'org_id': str(org_id)},
)
session.execute(
text('DELETE FROM stripe_customers WHERE org_id = :org_id'),
{'org_id': str(org_id)},
)
# 3. Handle users with this as current_org_id BEFORE deleting memberships
# Single query to find orphaned users (those with no alternative org)
orphaned_users = session.execute(
text("""
SELECT u.id
FROM "user" u
WHERE u.current_org_id = :org_id
AND NOT EXISTS (
SELECT 1 FROM org_member om
WHERE om.user_id = u.id AND om.org_id != :org_id
)
"""),
{'org_id': str(org_id)},
).fetchall()
if orphaned_users:
raise OrphanedUserError([str(row[0]) for row in orphaned_users])
# Batch update: reassign current_org_id to an alternative org for all affected users
session.execute(
text("""
UPDATE "user" u
SET current_org_id = (
SELECT om.org_id FROM org_member om
WHERE om.user_id = u.id AND om.org_id != :org_id
LIMIT 1
)
WHERE u.current_org_id = :org_id
"""),
{'org_id': str(org_id)},
)
# 4. Delete organization memberships (now safe)
session.execute(
text('DELETE FROM org_member WHERE org_id = :org_id'),
{'org_id': str(org_id)},
)
# 5. Finally delete the organization
session.delete(org)
# 6. Clean up LiteLLM team before committing transaction
logger.info(
'Deleting LiteLLM team within database transaction',
extra={'org_id': str(org_id)},
)
await LiteLlmManager.delete_team(str(org_id))
# 7. Commit all changes only if everything succeeded
session.commit()
logger.info(
'Successfully deleted organization and all associated data including LiteLLM team',
extra={'org_id': str(org_id), 'org_name': org.name},
)
return org
except Exception as e:
session.rollback()
logger.error(
'Failed to delete organization - transaction rolled back',
extra={'org_id': str(org_id), 'error': str(e)},
)
raise

View File

@@ -4,7 +4,9 @@ Store class for managing roles.
from typing import List, Optional
from storage.database import session_maker
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from storage.database import a_session_maker, session_maker
from storage.role import Role
@@ -33,6 +35,20 @@ class RoleStore:
with session_maker() as session:
return session.query(Role).filter(Role.name == name).first()
@staticmethod
async def get_role_by_name_async(
name: str,
session: Optional[AsyncSession] = None,
) -> Optional[Role]:
"""Get role by name."""
if session is not None:
result = await session.execute(select(Role).where(Role.name == name))
return result.scalars().first()
async with a_session_maker() as session:
result = await session.execute(select(Role).where(Role.name == name))
return result.scalars().first()
@staticmethod
def list_roles() -> List[Role]:
"""List all roles."""

View File

@@ -34,11 +34,10 @@ class SaasConversationStore(ConversationStore):
session_maker: sessionmaker
org_id: UUID | None = None # will be fetched automatically
def __init__(self, user_id: str, session_maker: sessionmaker):
def __init__(self, user_id: str, org_id: UUID, session_maker: sessionmaker):
self.user_id = user_id
self.org_id = org_id
self.session_maker = session_maker
user = UserStore.get_user_by_id(user_id)
self.org_id = user.current_org_id if user else None
def _select_by_id(self, session, conversation_id: str):
# Join StoredConversationMetadata with ConversationMetadataSaas to filter by user/org
@@ -235,4 +234,6 @@ class SaasConversationStore(ConversationStore):
cls, config: OpenHandsConfig, user_id: str | None
) -> ConversationStore:
# user_id should not be None in SaaS, should we raise?
return SaasConversationStore(str(user_id), session_maker)
user = await UserStore.get_user_by_id_async(user_id)
org_id = user.current_org_id if user else None
return SaasConversationStore(str(user_id), org_id, session_maker)

View File

@@ -8,10 +8,11 @@ from dataclasses import dataclass
from cryptography.fernet import Fernet
from pydantic import SecretStr
from server.constants import LITE_LLM_API_URL
from server.logger import logger
from sqlalchemy.orm import joinedload, sessionmaker
from storage.database import session_maker
from storage.lite_llm_manager import LiteLlmManager
from storage.lite_llm_manager import LiteLlmManager, get_openhands_cloud_key_alias
from storage.org import Org
from storage.org_member import OrgMember
from storage.org_store import OrgStore
@@ -23,6 +24,7 @@ from openhands.core.config.openhands_config import OpenHandsConfig
from openhands.server.settings import Settings
from openhands.storage.settings.settings_store import SettingsStore
from openhands.utils.async_utils import call_sync_from_async
from openhands.utils.llm import is_openhands_model
@dataclass
@@ -123,7 +125,6 @@ class SaasSettingsStore(SettingsStore):
with self.session_maker() as session:
if not item:
return None
kwargs = item.model_dump(context={'expose_secrets': True})
user = (
session.query(User)
.options(joinedload(User.org_members))
@@ -144,23 +145,29 @@ class SaasSettingsStore(SettingsStore):
return None
org_id = user.current_org_id
# Check if provider is OpenHands and generate API key if needed
if self._is_openhands_provider(item):
await self._ensure_openhands_api_key(item, str(org_id))
org_member = None
org_member: OrgMember = None
for om in user.org_members:
if om.org_id == org_id:
org_member = om
break
if not org_member or not org_member.llm_api_key:
return None
org = session.query(Org).filter(Org.id == org_id).first()
org: Org = session.query(Org).filter(Org.id == org_id).first()
if not org:
logger.error(
f'Org not found for ID {org_id} as the current org for user {self.user_id}'
)
return None
# Check if we need to generate an LLM key.
if item.llm_base_url == LITE_LLM_API_URL:
await self._ensure_api_key(
item, str(org_id), openhands_type=is_openhands_model(item.llm_model)
)
kwargs = item.model_dump(context={'expose_secrets': True})
for model in (user, org, org_member):
for key, value in kwargs.items():
if hasattr(model, key):
@@ -223,32 +230,49 @@ class SaasSettingsStore(SettingsStore):
fernet_key = b64encode(hashlib.sha256(jwt_secret.encode()).digest())
return Fernet(fernet_key)
def _is_openhands_provider(self, item: Settings) -> bool:
"""Check if the settings use the OpenHands provider."""
return bool(item.llm_model and item.llm_model.startswith('openhands/'))
async def _ensure_openhands_api_key(self, item: Settings, org_id: str) -> None:
async def _ensure_api_key(
self, item: Settings, org_id: str, openhands_type: bool = False
) -> None:
"""Generate and set the OpenHands API key for the given settings.
First checks if an existing key with the OpenHands alias exists,
and reuses it if found. Otherwise, generates a new key.
First checks if an existing key exists for the user and verifies it
is valid in LiteLLM. If valid, reuses it. Otherwise, generates a new key.
"""
# Generate new key if none exists
generated_key = await LiteLlmManager.generate_key(
# First, check if our current key is valid
if item.llm_api_key and not await LiteLlmManager.verify_existing_key(
item.llm_api_key.get_secret_value(),
self.user_id,
org_id,
None,
{'type': 'openhands'},
)
openhands_type=openhands_type,
):
generated_key = None
if openhands_type:
generated_key = await LiteLlmManager.generate_key(
self.user_id,
org_id,
None,
{'type': 'openhands'},
)
else:
# Must delete any existing key with the same alias first
key_alias = get_openhands_cloud_key_alias(self.user_id, org_id)
await LiteLlmManager.delete_key_by_alias(key_alias=key_alias)
generated_key = await LiteLlmManager.generate_key(
self.user_id,
org_id,
key_alias,
None,
)
if generated_key:
item.llm_api_key = SecretStr(generated_key)
logger.info(
'saas_settings_store:store:generated_openhands_key',
extra={'user_id': self.user_id},
)
else:
logger.warning(
'saas_settings_store:store:failed_to_generate_openhands_key',
extra={'user_id': self.user_id},
)
if generated_key:
item.llm_api_key = SecretStr(generated_key)
logger.info(
'saas_settings_store:store:generated_openhands_key',
extra={'user_id': self.user_id},
)
else:
logger.warning(
'saas_settings_store:store:failed_to_generate_openhands_key',
extra={'user_id': self.user_id},
)

View File

@@ -31,6 +31,8 @@ class User(Base): # type: ignore
user_consents_to_analytics = Column(Boolean, nullable=True)
email = Column(String, nullable=True)
email_verified = Column(Boolean, nullable=True)
git_user_name = Column(String, nullable=True)
git_user_email = Column(String, nullable=True)
# Relationships
role = relationship('Role', back_populates='users')

View File

@@ -5,6 +5,7 @@ Store class for managing users.
import asyncio
import uuid
from typing import Optional
from uuid import UUID
from server.auth.token_manager import TokenManager
from server.constants import (
@@ -14,15 +15,20 @@ from server.constants import (
get_default_litellm_model,
)
from server.logger import logger
from sqlalchemy import text
from sqlalchemy import select, text
from sqlalchemy.orm import joinedload
from storage.database import session_maker
from storage.encrypt_utils import decrypt_legacy_model
from storage.database import a_session_maker, session_maker
from storage.encrypt_utils import (
decrypt_legacy_model,
decrypt_legacy_value,
encrypt_legacy_value,
)
from storage.org import Org
from storage.org_member import OrgMember
from storage.role_store import RoleStore
from storage.user import User
from storage.user_settings import UserSettings
from utils.identity import resolve_display_name
from openhands.utils.async_utils import GENERAL_TIMEOUT, call_async_from_sync
@@ -49,7 +55,8 @@ class UserStore:
org = Org(
id=uuid.UUID(user_id),
name=f'user_{user_id}_org',
contact_name=user_info['preferred_username'],
contact_name=resolve_display_name(user_info)
or user_info.get('preferred_username', ''),
contact_email=user_info['email'],
v1_enabled=True,
)
@@ -116,7 +123,7 @@ class UserStore:
redis_client = UserStore._get_redis_client()
if redis_client is None:
logger.warning(
'saas_settings_store:_acquire_user_creation_lock:no_redis_client',
'user_store:_acquire_user_creation_lock:no_redis_client',
extra={'user_id': user_id},
)
return True # Proceed without locking if Redis is unavailable
@@ -127,6 +134,25 @@ class UserStore:
)
return bool(lock_acquired)
@staticmethod
async def _release_user_creation_lock(user_id: str) -> bool:
"""Release the distributed lock for user creation.
Returns True if the lock was released or if Redis is unavailable.
Returns False if the lock could not be released.
"""
redis_client = UserStore._get_redis_client()
if redis_client is None:
logger.warning(
'user_store:_release_user_creation_lock:no_redis_client',
extra={'user_id': user_id},
)
return True # Nothing to release if Redis is unavailable
user_key = f'{_REDIS_USER_CREATION_KEY_PREFIX}{user_id}'
deleted = await redis_client.delete(user_key)
return bool(deleted)
@staticmethod
async def migrate_user(
user_id: str,
@@ -147,24 +173,47 @@ class UserStore:
)
decrypted_user_settings = UserSettings(**kwargs)
with session_maker() as session:
# Check if user has completed billing sessions to enable BYOR export
from storage.billing_session import BillingSession
has_completed_billing = (
session.query(BillingSession)
.filter(
BillingSession.user_id == user_id,
BillingSession.status == 'completed',
)
.first()
is not None
)
# create personal org
org = Org(
id=uuid.UUID(user_id),
name=f'user_{user_id}_org',
org_version=user_settings.user_version,
contact_name=user_info['username'],
contact_name=resolve_display_name(user_info)
or user_info.get('username', ''),
contact_email=user_info['email'],
byor_export_enabled=has_completed_billing,
)
session.add(org)
from storage.lite_llm_manager import LiteLlmManager
logger.debug(
'user_store:migrate_user:calling_litellm_migrate_entries',
extra={'user_id': user_id},
)
await LiteLlmManager.migrate_entries(
str(org.id),
user_id,
decrypted_user_settings,
)
logger.debug(
'user_store:migrate_user:done_litellm_migrate_entries',
extra={'user_id': user_id},
)
custom_settings = UserStore._has_custom_settings(
decrypted_user_settings, user_settings.user_version
)
@@ -172,7 +221,15 @@ class UserStore:
# avoids circular reference. This migrate method is temprorary until all users are migrated.
from integrations.stripe_service import migrate_customer
logger.debug(
'user_store:migrate_user:calling_stripe_migrate_customer',
extra={'user_id': user_id},
)
await migrate_customer(session, user_id, org)
logger.debug(
'user_store:migrate_user:done_stripe_migrate_customer',
extra={'user_id': user_id},
)
from storage.org_store import OrgStore
@@ -201,7 +258,15 @@ class UserStore:
)
session.add(user)
role = RoleStore.get_role_by_name('owner')
logger.debug(
'user_store:migrate_user:calling_get_role_by_name',
extra={'user_id': user_id},
)
role = await RoleStore.get_role_by_name_async('owner')
logger.debug(
'user_store:migrate_user:done_get_role_by_name',
extra={'user_id': user_id},
)
from storage.org_member_store import OrgMemberStore
@@ -214,7 +279,6 @@ class UserStore:
if not custom_settings:
del org_member_kwargs['llm_model']
del org_member_kwargs['llm_base_url']
del org_member_kwargs['llm_api_key_for_byor']
org_member = OrgMember(
org_id=org.id,
@@ -229,6 +293,10 @@ class UserStore:
user_settings.already_migrated = True
session.merge(user_settings)
session.flush()
logger.debug(
'user_store:migrate_user:session_flush_complete',
extra={'user_id': user_id},
)
# need to migrate conversation metadata
session.execute(
@@ -296,8 +364,272 @@ class UserStore:
session.commit()
session.refresh(user)
user.org_members # load org_members
logger.debug(
'user_store:migrate_user:session_committed',
extra={'user_id': user_id},
)
return user
@staticmethod
async def downgrade_user(user_id: str) -> UserSettings | None:
"""
This method can be removed once orgs is established - probably after Feb 15 2026
Downgrade a migrated user back to the pre-migration state.
This reverses the migrate_user operation:
1. Get the user's settings from user_settings table (migrated users) or
create new user_settings from org_members table (new sign-ups)
2. Call LiteLlmManager.downgrade_entries to revert LiteLLM state
3. Copy user_id from conversation_metadata_saas to conversation_metadata
4. Delete conversation_metadata_saas entries
5. Reset org_id columns in related tables (stripe_customers, slack_users, etc.)
6. Delete the org_member and org entries
7. Delete the user entry
8. Set already_migrated=False on user_settings
For new sign-ups (users who registered after migration was deployed),
there won't be an existing user_settings entry. In this case, we fall back
to the org_members table to get the user's API keys and settings, and create
a new user_settings entry for them.
Args:
user_id: The Keycloak user ID to downgrade
Returns:
The user_settings if downgrade was successful, None otherwise.
Returns None if the org has multiple members (not a personal org).
"""
logger.info(
'user_store:downgrade_user:start',
extra={'user_id': user_id},
)
with session_maker() as session:
# Get the user and their org_member
user = (
session.query(User)
.options(joinedload(User.org_members))
.filter(User.id == uuid.UUID(user_id))
.first()
)
if not user:
logger.warning(
'user_store:downgrade_user:user_not_found',
extra={'user_id': user_id},
)
return None
# Get the user's personal org (org_id == user_id)
org = session.query(Org).filter(Org.id == uuid.UUID(user_id)).first()
if not org:
logger.warning(
'user_store:downgrade_user:org_not_found',
extra={'user_id': user_id},
)
return None
# Get org_members for this org - should only be one for personal orgs
org_members = (
session.query(OrgMember).filter(OrgMember.org_id == org.id).all()
)
if len(org_members) != 1:
logger.error(
'user_store:downgrade_user:unexpected_org_members_count',
extra={
'user_id': user_id,
'org_id': str(org.id),
'org_members_count': len(org_members),
},
)
return None
org_member = org_members[0]
# Get the user_settings (for migrated users)
user_settings = (
session.query(UserSettings)
.filter(
UserSettings.keycloak_user_id == user_id,
UserSettings.already_migrated.is_(True),
)
.first()
)
# For new sign-ups after migration, user_settings won't exist
# Fall back to getting data from org_members
if user_settings:
if org_member.llm_api_key and org_member.llm_api_key.get_secret_value():
user_settings.llm_api_key = encrypt_legacy_value(
org_member.llm_api_key.get_secret_value()
)
if (
org_member.llm_api_key_for_byor
and org_member.llm_api_key_for_byor.get_secret_value()
):
user_settings.llm_api_key_for_byor = encrypt_legacy_value(
org_member.llm_api_key_for_byor.get_secret_value()
)
logger.info(
'user_store:downgrade_user:updated_user_settings_from_org_member',
extra={'user_id': user_id},
)
else:
# Create a new user_settings entry from OrgMember, User, and Org data
# This is needed for new sign-ups who don't have user_settings
user_settings = UserStore._create_user_settings_from_entities(
user_id, org_member, user, org
)
session.add(user_settings)
logger.info(
'user_store:downgrade_user:created_user_settings_from_org_member',
extra={'user_id': user_id},
)
session.flush()
# Call LiteLLM downgrade
from storage.lite_llm_manager import LiteLlmManager
logger.debug(
'user_store:downgrade_user:calling_litellm_downgrade_entries',
extra={'user_id': user_id},
)
encrypted_fields = [
'llm_api_key',
'llm_api_key_for_byor',
'search_api_key',
'sandbox_api_key',
]
for field in encrypted_fields:
value = getattr(user_settings, field, None)
if value:
try:
value = decrypt_legacy_value(value)
setattr(user_settings, field, value)
except Exception:
pass
await LiteLlmManager.downgrade_entries(
str(org.id),
user_id,
user_settings,
)
logger.debug(
'user_store:downgrade_user:done_litellm_downgrade_entries',
extra={'user_id': user_id},
)
user_uuid = uuid.UUID(user_id)
# Step 3: Copy user_id from conversation_metadata_saas to conversation_metadata
# This ensures any conversations created after migration have their user_id
# preserved in the original table before we delete the saas entries
session.execute(
text("""
UPDATE conversation_metadata
SET user_id = :user_id
WHERE conversation_id IN (
SELECT conversation_id
FROM conversation_metadata_saas
WHERE user_id = :user_uuid
)
"""),
{'user_id': user_id, 'user_uuid': user_uuid},
)
# Step 4: Delete conversation_metadata_saas entries
session.execute(
text('DELETE FROM conversation_metadata_saas WHERE user_id = :user_id'),
{'user_id': user_uuid},
)
# Step 5: Reset org_id columns in related tables
# Reset stripe_customers
session.execute(
text(
'UPDATE stripe_customers SET org_id = NULL WHERE org_id = :org_id'
),
{'org_id': user_uuid},
)
# Reset slack_users
session.execute(
text('UPDATE slack_users SET org_id = NULL WHERE org_id = :org_id'),
{'org_id': user_uuid},
)
# Reset slack_conversation
session.execute(
text(
'UPDATE slack_conversation SET org_id = NULL WHERE org_id = :org_id'
),
{'org_id': user_uuid},
)
# Reset api_keys
session.execute(
text('UPDATE api_keys SET org_id = NULL WHERE org_id = :org_id'),
{'org_id': user_uuid},
)
# Reset custom_secrets
session.execute(
text('UPDATE custom_secrets SET org_id = NULL WHERE org_id = :org_id'),
{'org_id': user_uuid},
)
# Reset billing_sessions
session.execute(
text(
'UPDATE billing_sessions SET org_id = NULL WHERE org_id = :org_id'
),
{'org_id': user_uuid},
)
# Step 6: Delete org_member entries for this org
session.execute(
text('DELETE FROM org_member WHERE org_id = :org_id'),
{'org_id': user_uuid},
)
# Step 7: Delete the user entry
session.execute(
text('DELETE FROM "user" WHERE id = :user_id'),
{'user_id': user_uuid},
)
# Delete the org entry
session.execute(
text('DELETE FROM org WHERE id = :org_id'),
{'org_id': user_uuid},
)
# Step 8: Set already_migrated=False on user_settings and encrypt fields
user_settings.already_migrated = False
# Re-encrypt the sensitive fields before storing in the DB
encrypt_keys = [
'llm_api_key',
'llm_api_key_for_byor',
'search_api_key',
'sandbox_api_key',
]
for key in encrypt_keys:
value = getattr(user_settings, key, None)
if value is not None and not _is_legacy_value_encrypted(value):
setattr(user_settings, key, encrypt_legacy_value(value))
session.merge(user_settings)
session.commit()
logger.info(
'user_store:downgrade_user:complete',
extra={'user_id': user_id},
)
return user_settings
@staticmethod
def get_user_by_id(user_id: str) -> Optional[User]:
"""Get user by Keycloak user ID (sync version).
@@ -322,48 +654,53 @@ class UserStore:
):
# The user is already being created in another thread / process
logger.info(
'saas_settings_store:create_default_settings:waiting_for_lock',
'user_store:create_default_settings:waiting_for_lock',
extra={'user_id': user_id},
)
call_async_from_sync(
asyncio.sleep, GENERAL_TIMEOUT, _RETRY_LOAD_DELAY_SECONDS
)
# Check for user again as migration could have happened while trying to get the lock.
user = (
session.query(User)
.options(joinedload(User.org_members))
.filter(User.id == uuid.UUID(user_id))
.first()
)
if user:
return user
try:
# Check for user again as migration could have happened while trying to get the lock.
user = (
session.query(User)
.options(joinedload(User.org_members))
.filter(User.id == uuid.UUID(user_id))
.first()
)
if user:
return user
user_settings = (
session.query(UserSettings)
.filter(
UserSettings.keycloak_user_id == user_id,
UserSettings.already_migrated.is_(False),
user_settings = (
session.query(UserSettings)
.filter(
UserSettings.keycloak_user_id == user_id,
UserSettings.already_migrated.is_(False),
)
.first()
)
.first()
)
if user_settings:
token_manager = TokenManager()
user_info = call_async_from_sync(
token_manager.get_user_info_from_user_id,
GENERAL_TIMEOUT,
user_id,
if user_settings:
token_manager = TokenManager()
user_info = call_async_from_sync(
token_manager.get_user_info_from_user_id,
GENERAL_TIMEOUT,
user_id,
)
user = call_async_from_sync(
UserStore.migrate_user,
GENERAL_TIMEOUT,
user_id,
user_settings,
user_info,
)
return user
else:
return None
finally:
call_async_from_sync(
UserStore._release_user_creation_lock, GENERAL_TIMEOUT, user_id
)
user = call_async_from_sync(
UserStore.migrate_user,
GENERAL_TIMEOUT,
user_id,
user_settings,
user_info,
)
return user
else:
return None
@staticmethod
async def get_user_by_id_async(user_id: str) -> Optional[User]:
@@ -372,13 +709,13 @@ class UserStore:
This is the preferred method when calling from an async context as it
avoids event loop conflicts that can occur with the sync version.
"""
with session_maker() as session:
user = (
session.query(User)
async with a_session_maker() as session:
result = await session.execute(
select(User)
.options(joinedload(User.org_members))
.filter(User.id == uuid.UUID(user_id))
.first()
)
user = result.scalars().first()
if user:
return user
@@ -386,40 +723,50 @@ class UserStore:
while not await UserStore._acquire_user_creation_lock(user_id):
# The user is already being created in another thread / process
logger.info(
'saas_settings_store:create_default_settings:waiting_for_lock',
'user_store:get_user_by_id_async:waiting_for_lock',
extra={'user_id': user_id},
)
await asyncio.sleep(_RETRY_LOAD_DELAY_SECONDS)
# Check for user again as migration could have happened while trying to get the lock.
user = (
session.query(User)
.options(joinedload(User.org_members))
.filter(User.id == uuid.UUID(user_id))
.first()
)
if user:
return user
try:
# Check for user again as migration could have happened while trying to get the lock.
result = await session.execute(
select(User)
.options(joinedload(User.org_members))
.filter(User.id == uuid.UUID(user_id))
)
user = result.scalars().first()
if user:
return user
user_settings = (
session.query(UserSettings)
.filter(
UserSettings.keycloak_user_id == user_id,
UserSettings.already_migrated.is_(False),
logger.info(
'user_store:get_user_by_id_async:start_migration',
extra={'user_id': user_id},
)
.first()
)
if user_settings:
token_manager = TokenManager()
user_info = await token_manager.get_user_info_from_user_id(user_id)
user = await UserStore.migrate_user(
user_id,
user_settings,
user_info,
result = await session.execute(
select(UserSettings).filter(
UserSettings.keycloak_user_id == user_id,
UserSettings.already_migrated.is_(False),
)
)
return user
else:
return None
user_settings = result.scalars().first()
if user_settings:
token_manager = TokenManager()
user_info = await token_manager.get_user_info_from_user_id(user_id)
logger.info(
'user_store:get_user_by_id_async:calling_migrate_user',
extra={'user_id': user_id},
)
user = await UserStore.migrate_user(
user_id,
user_settings,
user_info,
)
return user
else:
return None
finally:
await UserStore._release_user_creation_lock(user_id)
@staticmethod
def list_users() -> list[User]:
@@ -427,6 +774,75 @@ class UserStore:
with session_maker() as session:
return session.query(User).all()
@staticmethod
def update_current_org(user_id: str, org_id: UUID) -> Optional[User]:
"""Update the user's current organization.
Args:
user_id: The user's ID (Keycloak user ID)
org_id: The organization ID to set as current
Returns:
User: The updated user object, or None if user not found
"""
with session_maker() as session:
user = (
session.query(User)
.filter(User.id == uuid.UUID(user_id))
.with_for_update()
.first()
)
if not user:
return None
user.current_org_id = org_id
session.commit()
session.refresh(user)
return user
@staticmethod
async def backfill_contact_name(user_id: str, user_info: dict) -> None:
"""Update contact_name on the personal org if it still has a username-style value.
Called during login to gradually fix existing users whose contact_name
was stored as their username (before the resolve_display_name fix).
Preserves custom values that were set via the PATCH endpoint.
"""
real_name = resolve_display_name(user_info)
if not real_name:
logger.debug(
'backfill_contact_name:no_real_name',
extra={'user_id': user_id},
)
return
preferred_username = user_info.get('preferred_username', '')
username = user_info.get('username', '')
async with a_session_maker() as session:
result = await session.execute(
select(Org).filter(Org.id == uuid.UUID(user_id))
)
org = result.scalars().first()
if not org:
logger.debug(
'backfill_contact_name:org_not_found',
extra={'user_id': user_id},
)
return
if org.contact_name in (preferred_username, username):
logger.info(
'backfill_contact_name:updated',
extra={
'user_id': user_id,
'old': org.contact_name,
'new': real_name,
},
)
org.contact_name = real_name
await session.commit()
# Prevent circular imports
from typing import TYPE_CHECKING
@@ -481,6 +897,96 @@ class UserStore:
}
return kwargs
@staticmethod
def _create_user_settings_from_entities(
user_id: str, org_member: OrgMember, user: User, org: Org
) -> UserSettings:
"""Create UserSettings from OrgMember, User, and Org data.
Uses OrgMember values first. If an OrgMember field is None and there's
a corresponding "default_" field in Org, use the Org value.
Also pulls relevant fields from User.
Args:
user_id: The Keycloak user ID
org_member: The OrgMember entity
user: The User entity
org: The Org entity
Returns:
A new UserSettings object populated from the entities
"""
# Mapping from OrgMember fields to corresponding Org "default_" fields
org_member_to_org_default = {
'llm_model': 'default_llm_model',
'llm_base_url': 'default_llm_base_url',
'max_iterations': 'default_max_iterations',
}
def get_value_with_org_fallback(field_name: str, org_member_value):
"""Get value from OrgMember, falling back to Org default if None."""
if org_member_value is not None:
return org_member_value
org_default_field = org_member_to_org_default.get(field_name)
if org_default_field and hasattr(org, org_default_field):
return getattr(org, org_default_field)
return None
# Get values from OrgMember with Org fallback for fields with default_ prefix
llm_model = get_value_with_org_fallback('llm_model', org_member.llm_model)
llm_base_url = get_value_with_org_fallback(
'llm_base_url', org_member.llm_base_url
)
max_iterations = get_value_with_org_fallback(
'max_iterations', org_member.max_iterations
)
return UserSettings(
keycloak_user_id=user_id,
# OrgMember fields
llm_api_key=org_member.llm_api_key.get_secret_value()
if org_member.llm_api_key
else None,
llm_api_key_for_byor=org_member.llm_api_key_for_byor.get_secret_value()
if org_member.llm_api_key_for_byor
else None,
llm_model=llm_model,
llm_base_url=llm_base_url,
max_iterations=max_iterations,
# User fields
accepted_tos=user.accepted_tos,
enable_sound_notifications=user.enable_sound_notifications,
language=user.language,
user_consents_to_analytics=user.user_consents_to_analytics,
email=user.email,
email_verified=user.email_verified,
git_user_name=user.git_user_name,
git_user_email=user.git_user_email,
# Org fields
agent=org.agent,
security_analyzer=org.security_analyzer,
confirmation_mode=org.confirmation_mode,
remote_runtime_resource_factor=org.remote_runtime_resource_factor,
enable_default_condenser=org.enable_default_condenser,
billing_margin=org.billing_margin,
enable_proactive_conversation_starters=org.enable_proactive_conversation_starters,
sandbox_base_container_image=org.sandbox_base_container_image,
sandbox_runtime_container_image=org.sandbox_runtime_container_image,
user_version=org.org_version,
mcp_config=org.mcp_config,
search_api_key=org.search_api_key.get_secret_value()
if org.search_api_key
else None,
sandbox_api_key=org.sandbox_api_key.get_secret_value()
if org.sandbox_api_key
else None,
max_budget_per_task=org.max_budget_per_task,
enable_solvability_analysis=org.enable_solvability_analysis,
v1_enabled=org.v1_enabled,
condenser_max_size=org.condenser_max_size,
already_migrated=False,
)
@staticmethod
def _has_custom_settings(
user_settings: UserSettings, old_user_version: int | None
@@ -526,3 +1032,12 @@ class UserStore:
return False # Matches old default
return True # Custom model
def _is_legacy_value_encrypted(value: str) -> bool:
"""Check if a legacy value is encrypted by trying to decrypt it"""
try:
decrypt_legacy_value(value)
return True
except Exception:
return False

View File

@@ -26,6 +26,7 @@ Optional environment variables:
"""
import os
import re
import sys
import time
from typing import Any, Dict, List, Optional
@@ -90,6 +91,31 @@ class ResendAPIError(ResendSyncError):
pass
# Email validation regex pattern - matches standard email format
# This pattern is intentionally strict to avoid Resend API validation errors
# It rejects special characters like ! that some email providers technically allow
# but Resend's API does not accept
EMAIL_REGEX = re.compile(r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$')
def is_valid_email(email: str) -> bool:
"""Validate an email address format.
This uses a regex pattern that matches most valid email addresses
while rejecting addresses with special characters that Resend's API
does not accept (e.g., exclamation marks).
Args:
email: The email address to validate.
Returns:
True if the email is valid, False otherwise.
"""
if not email:
return False
return bool(EMAIL_REGEX.match(email))
def get_keycloak_users(offset: int = 0, limit: int = 100) -> List[Dict[str, Any]]:
"""Get users from Keycloak using the admin client.
@@ -336,6 +362,7 @@ def sync_users_to_resend():
'total_users': total_users,
'existing_contacts': len(resend_contacts),
'added_contacts': 0,
'skipped_invalid_emails': 0,
'errors': 0,
}
@@ -355,6 +382,12 @@ def sync_users_to_resend():
logger.debug(f'User {email} already exists in Resend, skipping')
continue
# Validate email format before attempting to add to Resend
if not is_valid_email(email):
logger.warning(f'Skipping user with invalid email format: {email}')
stats['skipped_invalid_emails'] += 1
continue
try:
first_name = user.get('first_name')
last_name = user.get('last_name')

View File

@@ -141,12 +141,14 @@ def test_custom_to_static_conversion():
def create_provider_tokens(
tokens_dict: dict[ProviderType, str],
) -> dict[ProviderType, ProviderToken]:
"""Helper to create provider tokens dictionary."""
return {
provider_type: ProviderToken(token=SecretStr(token_value))
for provider_type, token_value in tokens_dict.items()
}
) -> MappingProxyType:
"""Helper to create provider tokens as MappingProxyType."""
return MappingProxyType(
{
provider_type: ProviderToken(token=SecretStr(token_value))
for provider_type, token_value in tokens_dict.items()
}
)
@pytest.mark.asyncio
@@ -264,3 +266,63 @@ async def test_get_latest_token_can_be_used_with_static_secret(
# Assert - this should NOT raise a ValidationError
static_secret = StaticSecret(value=token, description='GITHUB authentication token')
assert static_secret.get_value() == token_value
# ---------------------------------------------------------------------------
# Tests for get_authenticated_git_url - ensuring proper authenticated URLs
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_get_authenticated_git_url_raises_when_no_tokens(
resolver_context, mock_saas_user_auth
):
"""Test that get_authenticated_git_url raises error when no provider tokens available."""
# Arrange
mock_saas_user_auth.get_provider_tokens = AsyncMock(return_value=None)
# Act & Assert
with pytest.raises(ValueError, match='No provider tokens available'):
await resolver_context.get_authenticated_git_url('owner/repo')
@pytest.mark.asyncio
async def test_get_provider_handler_caches_instance(
resolver_context, mock_saas_user_auth
):
"""Test that _get_provider_handler caches the handler instance."""
# Arrange
token_value = 'ghp_test_token'
provider_tokens = create_provider_tokens({ProviderType.GITHUB: token_value})
mock_saas_user_auth.get_provider_tokens = AsyncMock(return_value=provider_tokens)
mock_saas_user_auth.get_user_id = AsyncMock(return_value='test-user-id')
# Act - call _get_provider_handler twice
handler1 = await resolver_context._get_provider_handler()
handler2 = await resolver_context._get_provider_handler()
# Assert - should be the same instance (cached)
assert handler1 is handler2
# get_provider_tokens should only be called once
assert mock_saas_user_auth.get_provider_tokens.call_count == 1
@pytest.mark.asyncio
async def test_get_provider_handler_creates_handler_with_correct_params(
resolver_context, mock_saas_user_auth
):
"""Test that _get_provider_handler creates ProviderHandler with correct parameters."""
# Arrange
token_value = 'ghp_test_token'
provider_tokens = create_provider_tokens({ProviderType.GITHUB: token_value})
mock_saas_user_auth.get_provider_tokens = AsyncMock(return_value=provider_tokens)
mock_saas_user_auth.get_user_id = AsyncMock(return_value='test-user-id')
# Act
handler = await resolver_context._get_provider_handler()
# Assert
from openhands.integrations.provider import ProviderHandler
assert isinstance(handler, ProviderHandler)
assert handler.provider_tokens == provider_tokens

View File

@@ -6,6 +6,7 @@ import httpx
import pytest
from fastapi import HTTPException
from server.routes.api_keys import (
check_byor_permitted,
delete_byor_key_from_litellm,
get_llm_api_key_for_byor,
)
@@ -182,16 +183,18 @@ class TestGetLlmApiKeyForByor:
"""Test the get_llm_api_key_for_byor endpoint."""
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
@patch('server.routes.api_keys.store_byor_key_in_db')
@patch('server.routes.api_keys.generate_byor_key')
@patch('server.routes.api_keys.get_byor_key_from_db')
async def test_no_key_in_database_generates_new(
self, mock_get_key, mock_generate_key, mock_store_key
self, mock_get_key, mock_generate_key, mock_store_key, mock_check_enabled
):
"""Test that when no key exists in database, a new one is generated."""
# Arrange
user_id = 'user-123'
new_key = 'sk-new-generated-key'
mock_check_enabled.return_value = True
mock_get_key.return_value = None
mock_generate_key.return_value = new_key
mock_store_key.return_value = None
@@ -201,20 +204,23 @@ class TestGetLlmApiKeyForByor:
# Assert
assert result == {'key': new_key}
mock_check_enabled.assert_called_once_with(user_id)
mock_get_key.assert_called_once_with(user_id)
mock_generate_key.assert_called_once_with(user_id)
mock_store_key.assert_called_once_with(user_id, new_key)
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
@patch('storage.lite_llm_manager.LiteLlmManager.verify_key')
@patch('server.routes.api_keys.get_byor_key_from_db')
async def test_valid_key_in_database_returns_key(
self, mock_get_key, mock_verify_key
self, mock_get_key, mock_verify_key, mock_check_enabled
):
"""Test that when a valid key exists in database, it is returned."""
# Arrange
user_id = 'user-123'
existing_key = 'sk-existing-valid-key'
mock_check_enabled.return_value = True
mock_get_key.return_value = existing_key
mock_verify_key.return_value = True
@@ -223,10 +229,12 @@ class TestGetLlmApiKeyForByor:
# Assert
assert result == {'key': existing_key}
mock_check_enabled.assert_called_once_with(user_id)
mock_get_key.assert_called_once_with(user_id)
mock_verify_key.assert_called_once_with(existing_key, user_id)
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
@patch('server.routes.api_keys.store_byor_key_in_db')
@patch('server.routes.api_keys.generate_byor_key')
@patch('server.routes.api_keys.delete_byor_key_from_litellm')
@@ -239,12 +247,14 @@ class TestGetLlmApiKeyForByor:
mock_delete_key,
mock_generate_key,
mock_store_key,
mock_check_enabled,
):
"""Test that when an invalid key exists in database, it is regenerated."""
# Arrange
user_id = 'user-123'
invalid_key = 'sk-invalid-key'
new_key = 'sk-new-generated-key'
mock_check_enabled.return_value = True
mock_get_key.return_value = invalid_key
mock_verify_key.return_value = False
mock_delete_key.return_value = True
@@ -256,6 +266,7 @@ class TestGetLlmApiKeyForByor:
# Assert
assert result == {'key': new_key}
mock_check_enabled.assert_called_once_with(user_id)
mock_get_key.assert_called_once_with(user_id)
mock_verify_key.assert_called_once_with(invalid_key, user_id)
mock_delete_key.assert_called_once_with(user_id, invalid_key)
@@ -263,6 +274,7 @@ class TestGetLlmApiKeyForByor:
mock_store_key.assert_called_once_with(user_id, new_key)
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
@patch('server.routes.api_keys.store_byor_key_in_db')
@patch('server.routes.api_keys.generate_byor_key')
@patch('server.routes.api_keys.delete_byor_key_from_litellm')
@@ -275,12 +287,14 @@ class TestGetLlmApiKeyForByor:
mock_delete_key,
mock_generate_key,
mock_store_key,
mock_check_enabled,
):
"""Test that even if deletion fails, regeneration still proceeds."""
# Arrange
user_id = 'user-123'
invalid_key = 'sk-invalid-key'
new_key = 'sk-new-generated-key'
mock_check_enabled.return_value = True
mock_get_key.return_value = invalid_key
mock_verify_key.return_value = False
mock_delete_key.return_value = False # Deletion fails
@@ -292,19 +306,22 @@ class TestGetLlmApiKeyForByor:
# Assert
assert result == {'key': new_key}
mock_check_enabled.assert_called_once_with(user_id)
mock_delete_key.assert_called_once_with(user_id, invalid_key)
mock_generate_key.assert_called_once_with(user_id)
mock_store_key.assert_called_once_with(user_id, new_key)
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
@patch('server.routes.api_keys.generate_byor_key')
@patch('server.routes.api_keys.get_byor_key_from_db')
async def test_key_generation_failure_raises_exception(
self, mock_get_key, mock_generate_key
self, mock_get_key, mock_generate_key, mock_check_enabled
):
"""Test that when key generation fails, an HTTPException is raised."""
# Arrange
user_id = 'user-123'
mock_check_enabled.return_value = True
mock_get_key.return_value = None
mock_generate_key.return_value = None
@@ -316,11 +333,15 @@ class TestGetLlmApiKeyForByor:
assert 'Failed to generate new BYOR LLM API key' in exc_info.value.detail
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
@patch('server.routes.api_keys.get_byor_key_from_db')
async def test_database_error_raises_exception(self, mock_get_key):
async def test_database_error_raises_exception(
self, mock_get_key, mock_check_enabled
):
"""Test that database errors are properly handled."""
# Arrange
user_id = 'user-123'
mock_check_enabled.return_value = True
mock_get_key.side_effect = Exception('Database connection error')
# Act & Assert
@@ -330,6 +351,21 @@ class TestGetLlmApiKeyForByor:
assert exc_info.value.status_code == 500
assert 'Failed to retrieve BYOR LLM API key' in exc_info.value.detail
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
async def test_byor_export_disabled_returns_402(self, mock_check_enabled):
"""Test that when BYOR export is disabled, 402 is returned."""
# Arrange
user_id = 'user-123'
mock_check_enabled.return_value = False
# Act & Assert
with pytest.raises(HTTPException) as exc_info:
await get_llm_api_key_for_byor(user_id=user_id)
assert exc_info.value.status_code == 402
assert 'BYOR key export is not enabled' in exc_info.value.detail
class TestDeleteByorKeyFromLitellm:
"""Test the delete_byor_key_from_litellm function with alias cleanup."""
@@ -425,3 +461,52 @@ class TestDeleteByorKeyFromLitellm:
# Assert
assert result is False
class TestCheckByorPermitted:
"""Test the check_byor_permitted endpoint."""
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
async def test_permitted_when_enabled(self, mock_check_enabled):
"""Test that permitted=True is returned when BYOR export is enabled."""
# Arrange
user_id = 'user-123'
mock_check_enabled.return_value = True
# Act
result = await check_byor_permitted(user_id=user_id)
# Assert
assert result == {'permitted': True}
mock_check_enabled.assert_called_once_with(user_id)
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
async def test_not_permitted_when_disabled(self, mock_check_enabled):
"""Test that permitted=False is returned when BYOR export is disabled."""
# Arrange
user_id = 'user-123'
mock_check_enabled.return_value = False
# Act
result = await check_byor_permitted(user_id=user_id)
# Assert
assert result == {'permitted': False}
mock_check_enabled.assert_called_once_with(user_id)
@pytest.mark.asyncio
@patch('storage.org_service.OrgService.check_byor_export_enabled')
async def test_error_raises_500(self, mock_check_enabled):
"""Test that an exception raises 500 error."""
# Arrange
user_id = 'user-123'
mock_check_enabled.side_effect = Exception('Database error')
# Act & Assert
with pytest.raises(HTTPException) as exc_info:
await check_byor_permitted(user_id=user_id)
assert exc_info.value.status_code == 500
assert 'Failed to check BYOR export permission' in exc_info.value.detail

View File

@@ -0,0 +1,96 @@
from unittest.mock import patch
from urllib.parse import parse_qs, urlparse
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from pydantic import SecretStr
from server.routes.github_proxy import add_github_proxy_routes
@pytest.fixture
def app_with_github_proxy(monkeypatch):
"""Create a FastAPI app with github proxy routes enabled."""
# Enable the github proxy endpoints
monkeypatch.setenv('GITHUB_PROXY_ENDPOINTS', '1')
# Mock the config to have a jwt_secret
mock_config = type(
'MockConfig', (), {'jwt_secret': SecretStr('test-secret-key-for-testing')}
)()
app = FastAPI()
with patch('server.routes.github_proxy.GITHUB_PROXY_ENDPOINTS', True):
with patch('server.routes.github_proxy.config', mock_config):
add_github_proxy_routes(app)
# Return app and mock_config so we can use the same config in tests
return app, mock_config
def test_state_compress_encrypt_and_decrypt_decompress_roundtrip(
app_with_github_proxy, monkeypatch
):
"""
Verify the code path used by github_proxy_start -> github_proxy_callback:
- compress payload, encrypt, base64-encode (what the start code does)
- base64-decode, decrypt, decompress (what the callback code does)
This test exercises the actual endpoints to verify the roundtrip works correctly.
"""
app, mock_config = app_with_github_proxy
client = TestClient(app)
original_state = 'some-state-value'
original_redirect_uri = 'https://example.com/redirect'
# Call github_proxy_start endpoint - it should redirect to GitHub with encrypted state
with patch('server.routes.github_proxy.config', mock_config):
response = client.get(
'/github-proxy/test-subdomain/login/oauth/authorize',
params={
'state': original_state,
'redirect_uri': original_redirect_uri,
'client_id': 'test-client-id',
},
follow_redirects=False,
)
assert response.status_code == 307
redirect_url = response.headers['location']
# Verify it redirects to GitHub
assert redirect_url.startswith('https://github.com/login/oauth/authorize')
# Parse the redirect URL to get the encrypted state
parsed = urlparse(redirect_url)
query_params = parse_qs(parsed.query)
encrypted_state = query_params['state'][0]
# The redirect_uri should now point to our callback
assert 'github-proxy/callback' in query_params['redirect_uri'][0]
# Now simulate GitHub calling back with this encrypted state
with patch('server.routes.github_proxy.config', mock_config):
callback_response = client.get(
'/github-proxy/callback',
params={
'state': encrypted_state,
'code': 'test-auth-code',
},
follow_redirects=False,
)
assert callback_response.status_code == 307
final_redirect = callback_response.headers['location']
# Verify the callback redirects to the original redirect_uri
assert final_redirect.startswith(original_redirect_uri)
# Parse the final redirect to verify the state was decrypted correctly
final_parsed = urlparse(final_redirect)
final_params = parse_qs(final_parsed.query)
assert final_params['state'][0] == original_state
assert final_params['code'][0] == 'test-auth-code'

View File

@@ -1,7 +1,7 @@
"""Unit tests for OAuth2 Device Flow endpoints."""
from datetime import UTC, datetime, timedelta
from unittest.mock import MagicMock, patch
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from fastapi import HTTPException, Request
@@ -22,8 +22,10 @@ def mock_device_code_store():
@pytest.fixture
def mock_api_key_store():
"""Mock API key store."""
return MagicMock()
"""Mock API key store with async create_api_key."""
mock = MagicMock()
mock.create_api_key = AsyncMock()
return mock
@pytest.fixture
@@ -204,8 +206,9 @@ class TestDeviceVerificationAuthenticated:
mock_store.get_by_user_code.return_value = mock_device
mock_store.authorize_device_code.return_value = True
# Mock API key store
# Mock API key store with async create_api_key
mock_api_key_store = MagicMock()
mock_api_key_store.create_api_key = AsyncMock()
mock_api_key_class.get_instance.return_value = mock_api_key_store
result = await device_verification_authenticated(
@@ -228,8 +231,9 @@ class TestDeviceVerificationAuthenticated:
@patch('server.routes.oauth_device.device_code_store')
async def test_multiple_device_authentication(self, mock_store, mock_api_key_class):
"""Test that multiple devices can authenticate simultaneously."""
# Mock API key store
# Mock API key store with async create_api_key
mock_api_key_store = MagicMock()
mock_api_key_store.create_api_key = AsyncMock()
mock_api_key_class.get_instance.return_value = mock_api_key_store
# Simulate two different devices
@@ -486,8 +490,9 @@ class TestDeviceVerificationTransactionIntegrity:
mock_store.get_by_user_code.return_value = mock_device
mock_store.authorize_device_code.return_value = False # Authorization fails
# Mock API key store
# Mock API key store with async create_api_key
mock_api_key_store = MagicMock()
mock_api_key_store.create_api_key = AsyncMock()
mock_api_key_class.get_instance.return_value = mock_api_key_store
# Should raise HTTPException due to authorization failure
@@ -518,9 +523,11 @@ class TestDeviceVerificationTransactionIntegrity:
mock_store.authorize_device_code.return_value = True # Authorization succeeds
mock_store.deny_device_code.return_value = True # Cleanup succeeds
# Mock API key store to fail on creation
# Mock API key store to fail on creation (async)
mock_api_key_store = MagicMock()
mock_api_key_store.create_api_key.side_effect = Exception('Database error')
mock_api_key_store.create_api_key = AsyncMock(
side_effect=Exception('Database error')
)
mock_api_key_class.get_instance.return_value = mock_api_key_store
# Should raise HTTPException due to API key creation failure
@@ -558,9 +565,11 @@ class TestDeviceVerificationTransactionIntegrity:
'Cleanup failed'
) # Cleanup fails
# Mock API key store to fail on creation
# Mock API key store to fail on creation (async)
mock_api_key_store = MagicMock()
mock_api_key_store.create_api_key.side_effect = Exception('Database error')
mock_api_key_store.create_api_key = AsyncMock(
side_effect=Exception('Database error')
)
mock_api_key_class.get_instance.return_value = mock_api_key_store
# Should still raise HTTPException for the original API key creation failure
@@ -589,8 +598,9 @@ class TestDeviceVerificationTransactionIntegrity:
mock_store.get_by_user_code.return_value = mock_device
mock_store.authorize_device_code.return_value = True # Authorization succeeds
# Mock API key store
# Mock API key store with async create_api_key
mock_api_key_store = MagicMock()
mock_api_key_store.create_api_key = AsyncMock()
mock_api_key_class.get_instance.return_value = mock_api_key_store
result = await device_verification_authenticated(

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,117 @@
"""Tests for resend_keycloak email validation."""
from sync.resend_keycloak import is_valid_email
class TestIsValidEmail:
"""Test cases for is_valid_email function."""
def test_valid_simple_email(self):
"""Test that a simple valid email passes validation."""
assert is_valid_email('user@example.com') is True
def test_valid_email_with_plus(self):
"""Test that email with + modifier passes validation."""
assert is_valid_email('user+tag@example.com') is True
def test_valid_email_with_dots(self):
"""Test that email with dots in local part passes validation."""
assert is_valid_email('first.last@example.com') is True
def test_valid_email_with_numbers(self):
"""Test that email with numbers passes validation."""
assert is_valid_email('user123@example.com') is True
def test_valid_email_with_subdomain(self):
"""Test that email with subdomain passes validation."""
assert is_valid_email('user@mail.example.com') is True
def test_valid_email_with_hyphen_domain(self):
"""Test that email with hyphen in domain passes validation."""
assert is_valid_email('user@example-site.com') is True
def test_valid_email_with_underscore(self):
"""Test that email with underscore passes validation."""
assert is_valid_email('user_name@example.com') is True
def test_valid_email_with_percent(self):
"""Test that email with percent sign passes validation."""
assert is_valid_email('user%name@example.com') is True
def test_invalid_email_with_exclamation(self):
"""Test that email with exclamation mark fails validation.
This is the specific case from the bug report:
ethanjames3713+!@gmail.com
"""
assert is_valid_email('ethanjames3713+!@gmail.com') is False
def test_invalid_email_with_special_chars(self):
"""Test that email with other special characters fails validation."""
assert is_valid_email('user!name@example.com') is False
assert is_valid_email('user#name@example.com') is False
assert is_valid_email('user$name@example.com') is False
assert is_valid_email('user&name@example.com') is False
assert is_valid_email("user'name@example.com") is False
assert is_valid_email('user*name@example.com') is False
assert is_valid_email('user=name@example.com') is False
assert is_valid_email('user^name@example.com') is False
assert is_valid_email('user`name@example.com') is False
assert is_valid_email('user{name@example.com') is False
assert is_valid_email('user|name@example.com') is False
assert is_valid_email('user}name@example.com') is False
assert is_valid_email('user~name@example.com') is False
def test_invalid_email_no_at_symbol(self):
"""Test that email without @ symbol fails validation."""
assert is_valid_email('userexample.com') is False
def test_invalid_email_no_domain(self):
"""Test that email without domain fails validation."""
assert is_valid_email('user@') is False
def test_invalid_email_no_local_part(self):
"""Test that email without local part fails validation."""
assert is_valid_email('@example.com') is False
def test_invalid_email_no_tld(self):
"""Test that email without TLD fails validation."""
assert is_valid_email('user@example') is False
def test_invalid_email_single_char_tld(self):
"""Test that email with single character TLD fails validation."""
assert is_valid_email('user@example.c') is False
def test_invalid_email_empty_string(self):
"""Test that empty string fails validation."""
assert is_valid_email('') is False
def test_invalid_email_none(self):
"""Test that None fails validation."""
assert is_valid_email(None) is False
def test_invalid_email_whitespace(self):
"""Test that email with whitespace fails validation."""
assert is_valid_email('user @example.com') is False
assert is_valid_email('user@ example.com') is False
assert is_valid_email(' user@example.com') is False
assert is_valid_email('user@example.com ') is False
def test_invalid_email_double_at(self):
"""Test that email with double @ fails validation."""
assert is_valid_email('user@@example.com') is False
def test_email_double_dot_domain(self):
"""Test email with double dot in domain.
Note: The regex allows this as it's technically valid in some edge cases,
and Resend's API may accept it. The main goal is to reject special
characters like ! that Resend definitely rejects.
"""
# This is allowed by our regex - Resend may or may not accept it
assert is_valid_email('user@example..com') is True
def test_case_insensitive_validation(self):
"""Test that validation works for uppercase emails."""
assert is_valid_email('USER@EXAMPLE.COM') is True
assert is_valid_email('User@Example.Com') is True

View File

@@ -32,6 +32,11 @@ def api_key_store(mock_session_maker):
return ApiKeyStore(mock_session_maker)
def run_sync(func, *args, **kwargs):
"""Helper to execute sync functions directly (mocks call_sync_from_async)."""
return func(*args, **kwargs)
def test_generate_api_key(api_key_store):
"""Test that generate_api_key returns a string with sk-oh- prefix and expected length."""
key = api_key_store.generate_api_key(length=32)
@@ -41,8 +46,12 @@ def test_generate_api_key(api_key_store):
assert len(key) == len('sk-oh-') + 32
@patch('storage.api_key_store.UserStore.get_user_by_id')
def test_create_api_key(mock_get_user, api_key_store, mock_session, mock_user):
@pytest.mark.asyncio
@patch('storage.api_key_store.call_sync_from_async', side_effect=run_sync)
@patch('storage.api_key_store.UserStore.get_user_by_id_async')
async def test_create_api_key(
mock_get_user, mock_call_sync, api_key_store, mock_session, mock_user
):
"""Test creating an API key."""
# Setup
user_id = 'test-user-123'
@@ -51,7 +60,7 @@ def test_create_api_key(mock_get_user, api_key_store, mock_session, mock_user):
api_key_store.generate_api_key = MagicMock(return_value='test-api-key')
# Execute
result = api_key_store.create_api_key(user_id, name)
result = await api_key_store.create_api_key(user_id, name)
# Verify
assert result == 'test-api-key'
@@ -219,8 +228,12 @@ def test_delete_api_key_by_id(api_key_store, mock_session):
mock_session.commit.assert_called_once()
@patch('storage.api_key_store.UserStore.get_user_by_id')
def test_list_api_keys(mock_get_user, api_key_store, mock_session, mock_user):
@pytest.mark.asyncio
@patch('storage.api_key_store.call_sync_from_async', side_effect=run_sync)
@patch('storage.api_key_store.UserStore.get_user_by_id_async')
async def test_list_api_keys(
mock_get_user, mock_call_sync, api_key_store, mock_session, mock_user
):
"""Test listing API keys for a user."""
# Setup
user_id = 'test-user-123'
@@ -247,7 +260,7 @@ def test_list_api_keys(mock_get_user, api_key_store, mock_session, mock_user):
mock_filter_org.all.return_value = [mock_key1, mock_key2]
# Execute
result = api_key_store.list_api_keys(user_id)
result = await api_key_store.list_api_keys(user_id)
# Verify
mock_get_user.assert_called_once_with(user_id)
@@ -265,8 +278,12 @@ def test_list_api_keys(mock_get_user, api_key_store, mock_session, mock_user):
assert result[1]['expires_at'] is None
@patch('storage.api_key_store.UserStore.get_user_by_id')
def test_retrieve_mcp_api_key(mock_get_user, api_key_store, mock_session, mock_user):
@pytest.mark.asyncio
@patch('storage.api_key_store.call_sync_from_async', side_effect=run_sync)
@patch('storage.api_key_store.UserStore.get_user_by_id_async')
async def test_retrieve_mcp_api_key(
mock_get_user, mock_call_sync, api_key_store, mock_session, mock_user
):
"""Test retrieving MCP API key for a user."""
# Setup
user_id = 'test-user-123'
@@ -287,16 +304,18 @@ def test_retrieve_mcp_api_key(mock_get_user, api_key_store, mock_session, mock_u
mock_filter_org.all.return_value = [mock_other_key, mock_mcp_key]
# Execute
result = api_key_store.retrieve_mcp_api_key(user_id)
result = await api_key_store.retrieve_mcp_api_key(user_id)
# Verify
mock_get_user.assert_called_once_with(user_id)
assert result == 'mcp-test-key'
@patch('storage.api_key_store.UserStore.get_user_by_id')
def test_retrieve_mcp_api_key_not_found(
mock_get_user, api_key_store, mock_session, mock_user
@pytest.mark.asyncio
@patch('storage.api_key_store.call_sync_from_async', side_effect=run_sync)
@patch('storage.api_key_store.UserStore.get_user_by_id_async')
async def test_retrieve_mcp_api_key_not_found(
mock_get_user, mock_call_sync, api_key_store, mock_session, mock_user
):
"""Test retrieving MCP API key when none exists."""
# Setup
@@ -314,7 +333,7 @@ def test_retrieve_mcp_api_key_not_found(
mock_filter_org.all.return_value = [mock_other_key]
# Execute
result = api_key_store.retrieve_mcp_api_key(user_id)
result = await api_key_store.retrieve_mcp_api_key(user_id)
# Verify
mock_get_user.assert_called_once_with(user_id)

View File

@@ -153,6 +153,7 @@ async def test_keycloak_callback_user_not_allowed(mock_request):
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.migrate_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = False
@@ -188,6 +189,7 @@ async def test_keycloak_callback_success_with_valid_offline_token(mock_request):
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.migrate_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_token_manager.get_keycloak_tokens = AsyncMock(
return_value=('test_access_token', 'test_refresh_token')
@@ -259,6 +261,7 @@ async def test_keycloak_callback_email_not_verified(mock_request):
mock_user.current_org_id = 'test_org_id'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
# Act
result = await keycloak_callback(
@@ -306,6 +309,7 @@ async def test_keycloak_callback_email_not_verified_missing_field(mock_request):
mock_user.current_org_id = 'test_org_id'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
# Act
result = await keycloak_callback(
@@ -347,6 +351,7 @@ async def test_keycloak_callback_success_without_offline_token(mock_request):
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.migrate_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_token_manager.get_keycloak_tokens = AsyncMock(
return_value=('test_access_token', 'test_refresh_token')
@@ -581,6 +586,7 @@ async def test_keycloak_callback_blocked_email_domain(mock_request):
mock_user.current_org_id = 'test_org_id'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_domain_blocker.is_active.return_value = True
mock_domain_blocker.is_domain_blocked.return_value = True
@@ -644,6 +650,7 @@ async def test_keycloak_callback_allowed_email_domain(mock_request):
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_domain_blocker.is_active.return_value = True
mock_domain_blocker.is_domain_blocked.return_value = False
@@ -707,6 +714,7 @@ async def test_keycloak_callback_domain_blocking_inactive(mock_request):
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_domain_blocker.is_active.return_value = False
mock_domain_blocker.is_domain_blocked.return_value = False
@@ -768,6 +776,7 @@ async def test_keycloak_callback_missing_email(mock_request):
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_domain_blocker.is_active.return_value = True
@@ -813,6 +822,7 @@ async def test_keycloak_callback_duplicate_email_detected(mock_request):
mock_user.current_org_id = 'test_org_id'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
# Act
result = await keycloak_callback(
@@ -857,6 +867,7 @@ async def test_keycloak_callback_duplicate_email_deletion_fails(mock_request):
mock_user.current_org_id = 'test_org_id'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
# Act
result = await keycloak_callback(
@@ -914,6 +925,7 @@ async def test_keycloak_callback_duplicate_check_exception(mock_request):
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -971,6 +983,7 @@ async def test_keycloak_callback_no_duplicate_email(mock_request):
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1031,6 +1044,7 @@ async def test_keycloak_callback_no_email_in_user_info(mock_request):
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1187,6 +1201,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1251,6 +1266,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.current_org_id = 'test_org_id'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_domain_blocker.is_domain_blocked.return_value = False
@@ -1333,6 +1349,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1420,6 +1437,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1504,6 +1522,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1587,6 +1606,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1667,6 +1687,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1733,6 +1754,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1805,6 +1827,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.accepted_tos = '2025-01-01'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
@@ -1875,6 +1898,7 @@ class TestKeycloakCallbackRecaptcha:
mock_user.current_org_id = 'test_org_id'
mock_user_store.get_user_by_id_async = AsyncMock(return_value=mock_user)
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_domain_blocker.is_domain_blocked.return_value = False

View File

@@ -0,0 +1,491 @@
"""
Unit tests for role-based authorization (authorization.py).
Tests the FastAPI dependencies that validate user roles within organizations.
"""
from unittest.mock import MagicMock, patch
from uuid import uuid4
import pytest
from fastapi import HTTPException
from server.auth.authorization import (
ROLE_HIERARCHY,
OrgRole,
get_user_org_role,
has_required_role,
require_org_admin,
require_org_owner,
require_org_role,
require_org_user,
)
# =============================================================================
# Tests for OrgRole enum
# =============================================================================
class TestOrgRole:
"""Tests for OrgRole enum."""
def test_org_role_values(self):
"""
GIVEN: OrgRole enum
WHEN: Accessing role values
THEN: All expected roles exist with correct string values
"""
assert OrgRole.OWNER.value == 'owner'
assert OrgRole.ADMIN.value == 'admin'
assert OrgRole.USER.value == 'user'
def test_org_role_from_string(self):
"""
GIVEN: Valid role string
WHEN: Creating OrgRole from string
THEN: Correct enum value is returned
"""
assert OrgRole('owner') == OrgRole.OWNER
assert OrgRole('admin') == OrgRole.ADMIN
assert OrgRole('user') == OrgRole.USER
def test_org_role_invalid_string(self):
"""
GIVEN: Invalid role string
WHEN: Creating OrgRole from string
THEN: ValueError is raised
"""
with pytest.raises(ValueError):
OrgRole('invalid_role')
# =============================================================================
# Tests for role hierarchy
# =============================================================================
class TestRoleHierarchy:
"""Tests for role hierarchy constants."""
def test_owner_highest_rank(self):
"""
GIVEN: Role hierarchy
WHEN: Comparing role ranks
THEN: Owner has highest rank
"""
assert ROLE_HIERARCHY[OrgRole.OWNER] > ROLE_HIERARCHY[OrgRole.ADMIN]
assert ROLE_HIERARCHY[OrgRole.OWNER] > ROLE_HIERARCHY[OrgRole.USER]
def test_admin_middle_rank(self):
"""
GIVEN: Role hierarchy
WHEN: Comparing role ranks
THEN: Admin is between owner and user
"""
assert ROLE_HIERARCHY[OrgRole.ADMIN] > ROLE_HIERARCHY[OrgRole.USER]
assert ROLE_HIERARCHY[OrgRole.ADMIN] < ROLE_HIERARCHY[OrgRole.OWNER]
def test_user_lowest_rank(self):
"""
GIVEN: Role hierarchy
WHEN: Comparing role ranks
THEN: User has lowest rank
"""
assert ROLE_HIERARCHY[OrgRole.USER] < ROLE_HIERARCHY[OrgRole.ADMIN]
assert ROLE_HIERARCHY[OrgRole.USER] < ROLE_HIERARCHY[OrgRole.OWNER]
# =============================================================================
# Tests for has_required_role function
# =============================================================================
class TestHasRequiredRole:
"""Tests for has_required_role function."""
def test_owner_has_owner_role(self):
"""
GIVEN: User with owner role
WHEN: Checking for owner requirement
THEN: Returns True
"""
assert has_required_role('owner', OrgRole.OWNER) is True
def test_owner_has_admin_role(self):
"""
GIVEN: User with owner role
WHEN: Checking for admin requirement
THEN: Returns True (owner > admin)
"""
assert has_required_role('owner', OrgRole.ADMIN) is True
def test_owner_has_user_role(self):
"""
GIVEN: User with owner role
WHEN: Checking for user requirement
THEN: Returns True (owner > user)
"""
assert has_required_role('owner', OrgRole.USER) is True
def test_admin_has_admin_role(self):
"""
GIVEN: User with admin role
WHEN: Checking for admin requirement
THEN: Returns True
"""
assert has_required_role('admin', OrgRole.ADMIN) is True
def test_admin_has_user_role(self):
"""
GIVEN: User with admin role
WHEN: Checking for user requirement
THEN: Returns True (admin > user)
"""
assert has_required_role('admin', OrgRole.USER) is True
def test_admin_lacks_owner_role(self):
"""
GIVEN: User with admin role
WHEN: Checking for owner requirement
THEN: Returns False (admin < owner)
"""
assert has_required_role('admin', OrgRole.OWNER) is False
def test_user_has_user_role(self):
"""
GIVEN: User with user role
WHEN: Checking for user requirement
THEN: Returns True
"""
assert has_required_role('user', OrgRole.USER) is True
def test_user_lacks_admin_role(self):
"""
GIVEN: User with user role
WHEN: Checking for admin requirement
THEN: Returns False (user < admin)
"""
assert has_required_role('user', OrgRole.ADMIN) is False
def test_user_lacks_owner_role(self):
"""
GIVEN: User with user role
WHEN: Checking for owner requirement
THEN: Returns False (user < owner)
"""
assert has_required_role('user', OrgRole.OWNER) is False
def test_invalid_role_returns_false(self):
"""
GIVEN: Invalid role string
WHEN: Checking for any requirement
THEN: Returns False
"""
assert has_required_role('invalid_role', OrgRole.USER) is False
assert has_required_role('invalid_role', OrgRole.ADMIN) is False
assert has_required_role('invalid_role', OrgRole.OWNER) is False
# =============================================================================
# Tests for get_user_org_role function
# =============================================================================
class TestGetUserOrgRole:
"""Tests for get_user_org_role function."""
def test_returns_role_when_member_exists(self):
"""
GIVEN: User is a member of organization with role
WHEN: get_user_org_role is called
THEN: Role name is returned
"""
user_id = str(uuid4())
org_id = uuid4()
mock_org_member = MagicMock()
mock_org_member.role_id = 1
mock_role = MagicMock()
mock_role.name = 'admin'
with (
patch(
'server.auth.authorization.OrgMemberStore.get_org_member',
return_value=mock_org_member,
),
patch(
'server.auth.authorization.RoleStore.get_role_by_id',
return_value=mock_role,
),
):
result = get_user_org_role(user_id, org_id)
assert result == 'admin'
def test_returns_none_when_not_member(self):
"""
GIVEN: User is not a member of organization
WHEN: get_user_org_role is called
THEN: None is returned
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.OrgMemberStore.get_org_member',
return_value=None,
):
result = get_user_org_role(user_id, org_id)
assert result is None
def test_returns_none_when_role_not_found(self):
"""
GIVEN: User is member but role not found
WHEN: get_user_org_role is called
THEN: None is returned
"""
user_id = str(uuid4())
org_id = uuid4()
mock_org_member = MagicMock()
mock_org_member.role_id = 999 # Non-existent role
with (
patch(
'server.auth.authorization.OrgMemberStore.get_org_member',
return_value=mock_org_member,
),
patch(
'server.auth.authorization.RoleStore.get_role_by_id',
return_value=None,
),
):
result = get_user_org_role(user_id, org_id)
assert result is None
# =============================================================================
# Tests for require_org_role dependency
# =============================================================================
class TestRequireOrgRole:
"""Tests for require_org_role dependency factory."""
@pytest.mark.asyncio
async def test_returns_user_id_when_authorized(self):
"""
GIVEN: User with sufficient role
WHEN: Role checker is called
THEN: User ID is returned
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.get_user_org_role',
return_value='admin',
):
role_checker = require_org_role(OrgRole.USER)
result = await role_checker(org_id=org_id, user_id=user_id)
assert result == user_id
@pytest.mark.asyncio
async def test_raises_401_when_not_authenticated(self):
"""
GIVEN: No user ID (not authenticated)
WHEN: Role checker is called
THEN: 401 Unauthorized is raised
"""
org_id = uuid4()
role_checker = require_org_role(OrgRole.USER)
with pytest.raises(HTTPException) as exc_info:
await role_checker(org_id=org_id, user_id=None)
assert exc_info.value.status_code == 401
assert 'not authenticated' in exc_info.value.detail.lower()
@pytest.mark.asyncio
async def test_raises_403_when_not_member(self):
"""
GIVEN: User is not a member of organization
WHEN: Role checker is called
THEN: 403 Forbidden is raised
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.get_user_org_role',
return_value=None,
):
role_checker = require_org_role(OrgRole.USER)
with pytest.raises(HTTPException) as exc_info:
await role_checker(org_id=org_id, user_id=user_id)
assert exc_info.value.status_code == 403
assert 'not a member' in exc_info.value.detail.lower()
@pytest.mark.asyncio
async def test_raises_403_when_insufficient_role(self):
"""
GIVEN: User with insufficient role
WHEN: Role checker is called
THEN: 403 Forbidden is raised
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.get_user_org_role',
return_value='user',
):
role_checker = require_org_role(OrgRole.ADMIN)
with pytest.raises(HTTPException) as exc_info:
await role_checker(org_id=org_id, user_id=user_id)
assert exc_info.value.status_code == 403
assert 'admin' in exc_info.value.detail.lower()
@pytest.mark.asyncio
async def test_owner_satisfies_admin_requirement(self):
"""
GIVEN: User with owner role
WHEN: Admin role is required
THEN: User ID is returned (owner > admin)
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.get_user_org_role',
return_value='owner',
):
role_checker = require_org_role(OrgRole.ADMIN)
result = await role_checker(org_id=org_id, user_id=user_id)
assert result == user_id
@pytest.mark.asyncio
async def test_logs_warning_on_insufficient_role(self):
"""
GIVEN: User with insufficient role
WHEN: Role checker is called
THEN: Warning is logged with details
"""
user_id = str(uuid4())
org_id = uuid4()
with (
patch(
'server.auth.authorization.get_user_org_role',
return_value='user',
),
patch('server.auth.authorization.logger') as mock_logger,
):
role_checker = require_org_role(OrgRole.OWNER)
with pytest.raises(HTTPException):
await role_checker(org_id=org_id, user_id=user_id)
mock_logger.warning.assert_called()
call_args = mock_logger.warning.call_args
assert call_args[1]['extra']['user_id'] == user_id
assert call_args[1]['extra']['user_role'] == 'user'
assert call_args[1]['extra']['required_role'] == 'owner'
# =============================================================================
# Tests for convenience dependencies
# =============================================================================
class TestConvenienceDependencies:
"""Tests for pre-configured convenience dependencies."""
@pytest.mark.asyncio
async def test_require_org_user_allows_user(self):
"""
GIVEN: User with user role
WHEN: require_org_user is used
THEN: User ID is returned
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.get_user_org_role',
return_value='user',
):
result = await require_org_user(org_id=org_id, user_id=user_id)
assert result == user_id
@pytest.mark.asyncio
async def test_require_org_admin_allows_admin(self):
"""
GIVEN: User with admin role
WHEN: require_org_admin is used
THEN: User ID is returned
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.get_user_org_role',
return_value='admin',
):
result = await require_org_admin(org_id=org_id, user_id=user_id)
assert result == user_id
@pytest.mark.asyncio
async def test_require_org_admin_rejects_user(self):
"""
GIVEN: User with user role
WHEN: require_org_admin is used
THEN: 403 Forbidden is raised
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.get_user_org_role',
return_value='user',
):
with pytest.raises(HTTPException) as exc_info:
await require_org_admin(org_id=org_id, user_id=user_id)
assert exc_info.value.status_code == 403
@pytest.mark.asyncio
async def test_require_org_owner_allows_owner(self):
"""
GIVEN: User with owner role
WHEN: require_org_owner is used
THEN: User ID is returned
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.get_user_org_role',
return_value='owner',
):
result = await require_org_owner(org_id=org_id, user_id=user_id)
assert result == user_id
@pytest.mark.asyncio
async def test_require_org_owner_rejects_admin(self):
"""
GIVEN: User with admin role
WHEN: require_org_owner is used
THEN: 403 Forbidden is raised
"""
user_id = str(uuid4())
org_id = uuid4()
with patch(
'server.auth.authorization.get_user_org_role',
return_value='admin',
):
with pytest.raises(HTTPException) as exc_info:
await require_org_owner(org_id=org_id, user_id=user_id)
assert exc_info.value.status_code == 403

View File

@@ -163,7 +163,7 @@ async def test_create_checkout_session_stripe_error(
'server.auth.token_manager.TokenManager.get_user_info_from_user_id',
AsyncMock(return_value={'email': 'testy@tester.com'}),
),
patch('server.routes.billing.validate_saas_environment'),
patch('server.routes.billing.validate_billing_enabled'),
):
await create_checkout_session(
CreateCheckoutSessionRequest(amount=25), mock_checkout_request, 'mock_user'
@@ -204,7 +204,7 @@ async def test_create_checkout_session_success(session_maker, mock_checkout_requ
'server.auth.token_manager.TokenManager.get_user_info_from_user_id',
AsyncMock(return_value={'email': 'testy@tester.com'}),
),
patch('server.routes.billing.validate_saas_environment'),
patch('server.routes.billing.validate_billing_enabled'),
):
mock_db_session = MagicMock()
mock_session_maker.return_value.__enter__.return_value = mock_db_session
@@ -236,8 +236,8 @@ async def test_create_checkout_session_success(session_maker, mock_checkout_requ
mode='payment',
payment_method_types=['card'],
saved_payment_method_options={'payment_method_save': 'enabled'},
success_url='http://test.com/api/billing/success?session_id={CHECKOUT_SESSION_ID}',
cancel_url='http://test.com/api/billing/cancel?session_id={CHECKOUT_SESSION_ID}',
success_url='https://test.com/api/billing/success?session_id={CHECKOUT_SESSION_ID}',
cancel_url='https://test.com/api/billing/cancel?session_id={CHECKOUT_SESSION_ID}',
)
# Verify database session creation
@@ -299,6 +299,8 @@ async def test_success_callback_success():
mock_billing_session.status = 'in_progress'
mock_billing_session.user_id = 'mock_user'
mock_org = MagicMock()
with (
patch('server.routes.billing.session_maker') as mock_session_maker,
patch('stripe.checkout.Session.retrieve') as mock_stripe_retrieve,
@@ -319,7 +321,17 @@ async def test_success_callback_success():
) as mock_update_budget,
):
mock_db_session = MagicMock()
# First query: BillingSession (query().filter().filter().first())
mock_db_session.query.return_value.filter.return_value.filter.return_value.first.return_value = mock_billing_session
# Second query: Org (query().filter().first()) - use side_effect for different return chains
mock_query_chain_billing = MagicMock()
mock_query_chain_billing.filter.return_value.filter.return_value.first.return_value = mock_billing_session
mock_query_chain_org = MagicMock()
mock_query_chain_org.filter.return_value.first.return_value = mock_org
mock_db_session.query.side_effect = [
mock_query_chain_billing,
mock_query_chain_org,
]
mock_session_maker.return_value.__enter__.return_value = mock_db_session
mock_stripe_retrieve.return_value = MagicMock(
@@ -331,7 +343,7 @@ async def test_success_callback_success():
assert response.status_code == 302
assert (
response.headers['location']
== 'http://test.com/settings/billing?checkout=success'
== 'https://test.com/settings/billing?checkout=success'
)
# Verify LiteLLM API calls
@@ -340,6 +352,9 @@ async def test_success_callback_success():
125.0, # 100 + (25.00 from Stripe)
)
# Verify BYOR export is enabled for the org (updated in same session)
assert mock_org.byor_export_enabled is True
# Verify database updates
assert mock_billing_session.status == 'completed'
assert mock_billing_session.price == 25.0
@@ -402,7 +417,7 @@ async def test_cancel_callback_session_not_found():
assert response.status_code == 302
assert (
response.headers['location']
== 'http://test.com/settings/billing?checkout=cancel'
== 'https://test.com/settings/billing?checkout=cancel'
)
# Verify no database updates occurred
@@ -429,7 +444,7 @@ async def test_cancel_callback_success():
assert response.status_code == 302
assert (
response.headers['location']
== 'http://test.com/settings/billing?checkout=cancel'
== 'https://test.com/settings/billing?checkout=cancel'
)
# Verify database updates
@@ -490,7 +505,7 @@ async def test_create_customer_setup_session_success():
AsyncMock(return_value=mock_customer_info),
),
patch('stripe.checkout.Session.create_async', mock_create),
patch('server.routes.billing.validate_saas_environment'),
patch('server.routes.billing.validate_billing_enabled'),
):
result = await create_customer_setup_session(mock_request, 'mock_user')
@@ -502,6 +517,6 @@ async def test_create_customer_setup_session_success():
customer='mock-customer-id',
mode='setup',
payment_method_types=['card'],
success_url='http://test.com/?free_credits=success',
cancel_url='http://test.com/',
success_url='https://test.com/?free_credits=success',
cancel_url='https://test.com/',
)

View File

@@ -0,0 +1,116 @@
"""Tests for the resolve_display_name helper.
The resolve_display_name helper extracts the best available display name from
Keycloak user_info claims. It is used by both the /api/user/info fallback path
and the user_store create/migrate paths to avoid duplicating name-resolution logic.
The fallback chain is: name → given_name + family_name → None.
It intentionally does NOT fall back to preferred_username/username — callers
that need a guaranteed non-None value handle that separately, because the
/api/user/info route should return name=None when no real name is available.
"""
from utils.identity import resolve_display_name
class TestResolveDisplayName:
"""Test resolve_display_name with various Keycloak claim combinations."""
def test_returns_name_when_present(self):
"""When user_info has a 'name' claim, use it directly."""
user_info = {
'sub': '123',
'name': 'Jane Doe',
'given_name': 'Jane',
'family_name': 'Doe',
'preferred_username': 'j.doe',
}
assert resolve_display_name(user_info) == 'Jane Doe'
def test_combines_given_and_family_name_when_name_absent(self):
"""When 'name' is missing, combine given_name + family_name."""
user_info = {
'sub': '123',
'given_name': 'Jane',
'family_name': 'Doe',
'preferred_username': 'j.doe',
}
assert resolve_display_name(user_info) == 'Jane Doe'
def test_uses_given_name_only_when_family_name_absent(self):
"""When only given_name is available, use it alone."""
user_info = {
'sub': '123',
'given_name': 'Jane',
'preferred_username': 'j.doe',
}
assert resolve_display_name(user_info) == 'Jane'
def test_uses_family_name_only_when_given_name_absent(self):
"""When only family_name is available, use it alone."""
user_info = {
'sub': '123',
'family_name': 'Doe',
'preferred_username': 'j.doe',
}
assert resolve_display_name(user_info) == 'Doe'
def test_returns_none_when_no_name_claims(self):
"""When no name claims exist at all, return None."""
user_info = {
'sub': '123',
'preferred_username': 'j.doe',
'email': 'jane@example.com',
}
assert resolve_display_name(user_info) is None
def test_returns_none_for_empty_name(self):
"""When 'name' is an empty string, treat it as absent."""
user_info = {
'sub': '123',
'name': '',
'preferred_username': 'j.doe',
}
assert resolve_display_name(user_info) is None
def test_returns_none_for_whitespace_only_name(self):
"""When 'name' is whitespace only, treat it as absent."""
user_info = {
'sub': '123',
'name': ' ',
'preferred_username': 'j.doe',
}
assert resolve_display_name(user_info) is None
def test_returns_none_for_empty_given_and_family_names(self):
"""When given_name and family_name are both empty strings, return None."""
user_info = {
'sub': '123',
'given_name': '',
'family_name': '',
'preferred_username': 'j.doe',
}
assert resolve_display_name(user_info) is None
def test_returns_none_for_empty_dict(self):
"""An empty user_info dict returns None."""
assert resolve_display_name({}) is None
def test_strips_whitespace_from_combined_name(self):
"""Whitespace around given_name/family_name is stripped."""
user_info = {
'sub': '123',
'given_name': ' Jane ',
'family_name': ' Doe ',
}
assert resolve_display_name(user_info) == 'Jane Doe'
def test_name_claim_takes_priority_over_given_family(self):
"""When both 'name' and given/family are present, 'name' wins."""
user_info = {
'sub': '123',
'name': 'Dr. Jane Doe',
'given_name': 'Jane',
'family_name': 'Doe',
}
assert resolve_display_name(user_info) == 'Dr. Jane Doe'

View File

@@ -11,7 +11,11 @@ from pydantic import SecretStr
from server.constants import (
get_default_litellm_model,
)
from storage.lite_llm_manager import LiteLlmManager
from storage.lite_llm_manager import (
LiteLlmManager,
get_byor_key_alias,
get_openhands_cloud_key_alias,
)
from storage.user_settings import UserSettings
from openhands.server.settings import Settings
@@ -284,6 +288,16 @@ class TestLiteLlmManager:
self, mock_user_settings, mock_user_response, mock_response
):
"""Test successful migrate_entries operation."""
# Mock response for key list
mock_key_list_response = MagicMock()
mock_key_list_response.is_success = True
mock_key_list_response.status_code = 200
mock_key_list_response.json.return_value = {
'keys': ['test-key-1', 'test-key-2'],
'total_count': 2,
}
mock_key_list_response.raise_for_status = MagicMock()
with patch.dict(os.environ, {'LOCAL_DEPLOYMENT': ''}):
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key'):
with patch(
@@ -301,14 +315,22 @@ class TestLiteLlmManager:
mock_client_class.return_value.__aenter__.return_value = (
mock_client
)
mock_client.get.return_value = mock_user_response
# First GET is for _get_user, second GET is for _get_user_keys
mock_client.get.side_effect = [
mock_user_response,
mock_key_list_response,
]
mock_client.post.return_value = mock_response
result = await LiteLlmManager.migrate_entries(
'test-org-id',
'test-user-id',
mock_user_settings,
)
# Mock verify_key to return True (key exists in LiteLLM)
with patch.object(
LiteLlmManager, 'verify_key', return_value=True
):
result = await LiteLlmManager.migrate_entries(
'test-org-id',
'test-user-id',
mock_user_settings,
)
# migrate_entries returns the user_settings unchanged
assert result is not None
@@ -317,10 +339,97 @@ class TestLiteLlmManager:
assert result.llm_api_key.get_secret_value() == 'test-key'
assert result.llm_base_url == 'http://test.com'
# Verify migration steps were called
# Verify migration steps were called:
# - 2 GET requests: _get_user, _get_user_keys
# - POST requests: create_team, update_user, add_user_to_team,
# and update_key for each key (2 keys)
assert mock_client.get.call_count == 2
assert (
mock_client.post.call_count == 4
) # create_team, update_user, add_user_to_team, update_key
mock_client.post.call_count == 5
) # create_team, update_user, add_user_to_team, 2x update_key
@pytest.mark.asyncio
async def test_migrate_entries_generates_key_when_db_key_not_in_litellm(
self, mock_user_settings, mock_user_response, mock_response
):
"""Test migrate_entries generates a new key when the DB key doesn't exist in LiteLLM."""
# Mock response for key list
mock_key_list_response = MagicMock()
mock_key_list_response.is_success = True
mock_key_list_response.status_code = 200
mock_key_list_response.json.return_value = {
'keys': ['test-key-1', 'test-key-2'],
'total_count': 2,
}
mock_key_list_response.raise_for_status = MagicMock()
# Mock response for key generation
mock_generate_response = MagicMock()
mock_generate_response.is_success = True
mock_generate_response.status_code = 200
mock_generate_response.json.return_value = {'key': 'new-generated-key'}
mock_generate_response.raise_for_status = MagicMock()
with patch.dict(os.environ, {'LOCAL_DEPLOYMENT': ''}):
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key'):
with patch(
'storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com'
):
with patch(
'storage.lite_llm_manager.TokenManager'
) as mock_token_manager:
mock_token_manager.return_value.get_user_info_from_user_id = (
AsyncMock(return_value={'email': 'test@example.com'})
)
with patch('httpx.AsyncClient') as mock_client_class:
mock_client = AsyncMock()
mock_client_class.return_value.__aenter__.return_value = (
mock_client
)
# First GET is for _get_user, second GET is for _get_user_keys
mock_client.get.side_effect = [
mock_user_response,
mock_key_list_response,
]
# POST responses: create_team, update_user, add_user_to_team,
# 2x update_key, and 1x generate_key
mock_client.post.side_effect = [
mock_response, # create_team
mock_response, # update_user
mock_response, # add_user_to_team
mock_response, # update_key 1
mock_response, # update_key 2
mock_generate_response, # generate_key
]
# Mock verify_key to return False (key doesn't exist in LiteLLM)
with patch.object(
LiteLlmManager, 'verify_key', return_value=False
):
result = await LiteLlmManager.migrate_entries(
'test-org-id',
'test-user-id',
mock_user_settings,
)
# migrate_entries should update user_settings with the new key
assert result is not None
assert (
result.llm_api_key.get_secret_value()
== 'new-generated-key'
)
assert (
result.llm_api_key_for_byor.get_secret_value()
== 'new-generated-key'
)
# Verify migration steps were called including key generation:
# - 2 GET requests: _get_user, _get_user_keys
# - 6 POST requests: create_team, update_user, add_user_to_team,
# 2x update_key, 1x generate_key
assert mock_client.get.call_count == 2
assert mock_client.post.call_count == 6
@pytest.mark.asyncio
async def test_update_team_and_users_budget_missing_config(self):
@@ -654,7 +763,7 @@ class TestLiteLlmManager:
# Assert
mock_logger.warning.assert_called_once_with(
'invalid_litellm_key_during_update',
extra={'user_id': 'test-user-id'},
extra={'user_id': 'test-user-id', 'text': 'Unauthorized'},
)
@pytest.mark.asyncio
@@ -678,6 +787,133 @@ class TestLiteLlmManager:
mock_http_client, 'test-user-id', 'test-api-key', team_id='test-team-id'
)
@pytest.mark.asyncio
@patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com')
@patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key')
async def test_get_user_keys_success(self, mock_http_client):
"""Test successful _get_user_keys operation."""
# Arrange
mock_response = MagicMock()
mock_response.is_success = True
mock_response.status_code = 200
mock_response.json.return_value = {
'keys': ['key-1', 'key-2', 'key-3'],
'total_count': 3,
}
mock_http_client.get.return_value = mock_response
# Act
keys = await LiteLlmManager._get_user_keys(mock_http_client, 'test-user-id')
# Assert
assert keys == ['key-1', 'key-2', 'key-3']
mock_http_client.get.assert_called_once()
call_args = mock_http_client.get.call_args
assert 'http://test.com/key/list' in call_args[0]
assert call_args[1]['params'] == {'user_id': 'test-user-id'}
@pytest.mark.asyncio
@patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com')
@patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key')
async def test_get_user_keys_empty_list(self, mock_http_client):
"""Test _get_user_keys returns empty list when user has no keys."""
# Arrange
mock_response = MagicMock()
mock_response.is_success = True
mock_response.status_code = 200
mock_response.json.return_value = {
'keys': [],
'total_count': 0,
}
mock_http_client.get.return_value = mock_response
# Act
keys = await LiteLlmManager._get_user_keys(mock_http_client, 'test-user-id')
# Assert
assert keys == []
@pytest.mark.asyncio
@patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com')
@patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key')
async def test_get_user_keys_error_returns_empty_list(self, mock_http_client):
"""Test _get_user_keys returns empty list on error."""
# Arrange
mock_response = MagicMock()
mock_response.is_success = False
mock_response.status_code = 500
mock_response.text = 'Internal server error'
mock_http_client.get.return_value = mock_response
# Act
keys = await LiteLlmManager._get_user_keys(mock_http_client, 'test-user-id')
# Assert
assert keys == []
@pytest.mark.asyncio
async def test_get_user_keys_missing_config(self, mock_http_client):
"""Test _get_user_keys returns empty list when config is missing."""
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', None):
with patch('storage.lite_llm_manager.LITE_LLM_API_URL', None):
keys = await LiteLlmManager._get_user_keys(
mock_http_client, 'test-user-id'
)
assert keys == []
@pytest.mark.asyncio
@patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com')
@patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key')
async def test_update_user_keys_success(self, mock_http_client, mock_response):
"""Test successful _update_user_keys operation."""
# Arrange
mock_key_list_response = MagicMock()
mock_key_list_response.is_success = True
mock_key_list_response.status_code = 200
mock_key_list_response.json.return_value = {
'keys': ['key-1', 'key-2'],
'total_count': 2,
}
mock_http_client.get.return_value = mock_key_list_response
mock_http_client.post.return_value = mock_response
# Act
await LiteLlmManager._update_user_keys(
mock_http_client, 'test-user-id', team_id='test-team-id'
)
# Assert
# Should call GET once for key list
assert mock_http_client.get.call_count == 1
# Should call POST twice (once for each key)
assert mock_http_client.post.call_count == 2
@pytest.mark.asyncio
@patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com')
@patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key')
async def test_update_user_keys_no_keys(self, mock_http_client, mock_response):
"""Test _update_user_keys when user has no keys."""
# Arrange
mock_key_list_response = MagicMock()
mock_key_list_response.is_success = True
mock_key_list_response.status_code = 200
mock_key_list_response.json.return_value = {
'keys': [],
'total_count': 0,
}
mock_http_client.get.return_value = mock_key_list_response
# Act
await LiteLlmManager._update_user_keys(
mock_http_client, 'test-user-id', team_id='test-team-id'
)
# Assert
# Should call GET once for key list
assert mock_http_client.get.call_count == 1
# Should not call POST since there are no keys
assert mock_http_client.post.call_count == 0
@pytest.mark.asyncio
async def test_generate_key_success(self, mock_http_client, mock_response):
"""Test successful _generate_key operation."""
@@ -1126,3 +1362,510 @@ class TestLiteLlmManager:
'http://test.url/team/delete',
json={'team_ids': [team_id]},
)
@pytest.mark.asyncio
async def test_remove_user_from_team_successful(self):
"""
GIVEN: Valid user_id and team_id
WHEN: _remove_user_from_team is called
THEN: HTTP POST is made to remove user from team
"""
mock_response = AsyncMock()
mock_response.is_success = True
mock_response.status_code = 200
with (
patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key'),
patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.url'),
):
mock_client = AsyncMock()
mock_client.post.return_value = mock_response
await LiteLlmManager._remove_user_from_team(
mock_client, 'test-user-id', 'test-team-id'
)
mock_client.post.assert_called_once_with(
'http://test.url/team/member_delete',
json={
'team_id': 'test-team-id',
'user_id': 'test-user-id',
},
)
@pytest.mark.asyncio
async def test_remove_user_from_team_not_found(self):
"""
GIVEN: User not in team
WHEN: _remove_user_from_team is called
THEN: 404 response is handled gracefully without raising
"""
mock_response = AsyncMock()
mock_response.is_success = False
mock_response.status_code = 404
mock_response.text = 'User not found in team'
mock_response.raise_for_status = MagicMock()
with (
patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key'),
patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.url'),
):
mock_client = AsyncMock()
mock_client.post.return_value = mock_response
# Should not raise an exception
await LiteLlmManager._remove_user_from_team(
mock_client, 'test-user-id', 'test-team-id'
)
@pytest.mark.asyncio
async def test_downgrade_entries_missing_config(self, mock_user_settings):
"""Test downgrade_entries when LiteLLM config is missing."""
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', None):
with patch('storage.lite_llm_manager.LITE_LLM_API_URL', None):
result = await LiteLlmManager.downgrade_entries(
'test-org-id',
'test-user-id',
mock_user_settings,
)
assert result is None
@pytest.mark.asyncio
async def test_downgrade_entries_team_not_found(self, mock_user_settings):
"""Test downgrade_entries when team is not found."""
with patch.dict(os.environ, {'LOCAL_DEPLOYMENT': ''}):
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key'):
with patch(
'storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com'
):
with patch.object(
LiteLlmManager, '_get_team', new_callable=AsyncMock
) as mock_get_team:
mock_get_team.return_value = None
result = await LiteLlmManager.downgrade_entries(
'test-org-id',
'test-user-id',
mock_user_settings,
)
assert result is None
@pytest.mark.asyncio
async def test_downgrade_entries_successful(self, mock_user_settings):
"""Test successful downgrade_entries operation."""
mock_response = MagicMock()
mock_response.is_success = True
mock_response.status_code = 200
mock_response.raise_for_status = MagicMock()
mock_team_info_response = MagicMock()
mock_team_info_response.is_success = True
mock_team_info_response.status_code = 200
mock_team_info_response.json.return_value = {
'team_info': {
'max_budget': 100.0,
'spend': 20.0,
},
'team_memberships': [
{
'user_id': 'test-user-id',
'team_id': 'test-org-id',
'max_budget_in_team': 100.0,
'spend': 20.0,
}
],
}
mock_team_info_response.raise_for_status = MagicMock()
mock_key_list_response = MagicMock()
mock_key_list_response.is_success = True
mock_key_list_response.status_code = 200
mock_key_list_response.json.return_value = {
'keys': ['test-key-1', 'test-key-2'],
'total_count': 2,
}
mock_key_list_response.raise_for_status = MagicMock()
with patch.dict(os.environ, {'LOCAL_DEPLOYMENT': ''}):
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key'):
with patch(
'storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com'
):
with patch(
'storage.lite_llm_manager.LITE_LLM_TEAM_ID', 'default-team'
):
with patch('httpx.AsyncClient') as mock_client_class:
mock_client = AsyncMock()
mock_client_class.return_value.__aenter__.return_value = (
mock_client
)
# GET requests: get_team (x2 for team info), get_user_keys
mock_client.get.side_effect = [
mock_team_info_response,
mock_team_info_response,
mock_key_list_response,
]
mock_client.post.return_value = mock_response
result = await LiteLlmManager.downgrade_entries(
'test-org-id',
'test-user-id',
mock_user_settings,
)
# downgrade_entries returns the user_settings
assert result is not None
assert result.agent == 'TestAgent'
# Verify downgrade steps were called:
# GET requests:
# 1. get_team (GET)
# 2. get_user_team_info (GET via _get_team)
# 3. get_user_keys (GET)
# POST requests:
# 1. update_user (POST)
# 2. add_user_to_team (POST)
# 3. update_key for key 1 (POST)
# 4. update_key for key 2 (POST)
# 5. remove_user_from_team (POST)
# 6. delete_team (POST)
assert mock_client.get.call_count == 3
assert mock_client.post.call_count == 6
@pytest.mark.asyncio
async def test_downgrade_entries_local_deployment(self, mock_user_settings):
"""Test downgrade_entries in local deployment mode (skips LiteLLM calls)."""
with patch.dict(os.environ, {'LOCAL_DEPLOYMENT': 'true'}):
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-key'):
with patch(
'storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com'
):
result = await LiteLlmManager.downgrade_entries(
'test-org-id',
'test-user-id',
mock_user_settings,
)
# In local deployment, should return user_settings without
# making any LiteLLM calls
assert result is not None
assert result.agent == 'TestAgent'
class TestGetAllKeysForUser:
"""Test cases for _get_all_keys_for_user method."""
@pytest.mark.asyncio
async def test_get_all_keys_missing_config(self):
"""Test _get_all_keys_for_user when LiteLLM config is missing."""
mock_client = AsyncMock(spec=httpx.AsyncClient)
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', None):
with patch('storage.lite_llm_manager.LITE_LLM_API_URL', None):
result = await LiteLlmManager._get_all_keys_for_user(
mock_client, 'test-user-id'
)
assert result == []
mock_client.get.assert_not_called()
@pytest.mark.asyncio
async def test_get_all_keys_success(self):
"""Test _get_all_keys_for_user returns keys on success."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
'keys': [
{
'key_name': 'sk-test1234',
'key_alias': 'test-alias',
'team_id': 'test-org',
'metadata': {'type': 'openhands'},
},
{
'key_name': 'sk-test5678',
'key_alias': 'another-alias',
'team_id': 'test-org',
'metadata': None,
},
]
}
mock_response.raise_for_status = MagicMock()
mock_client = AsyncMock(spec=httpx.AsyncClient)
mock_client.get.return_value = mock_response
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-api-key'):
with patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com'):
result = await LiteLlmManager._get_all_keys_for_user(
mock_client, 'test-user-id'
)
assert len(result) == 2
assert result[0]['key_name'] == 'sk-test1234'
assert result[1]['key_name'] == 'sk-test5678'
# Verify API key header is included
mock_client.get.assert_called_once()
call_kwargs = mock_client.get.call_args
assert call_kwargs.kwargs['headers'] == {
'x-goog-api-key': 'test-api-key'
}
@pytest.mark.asyncio
async def test_get_all_keys_empty_response(self):
"""Test _get_all_keys_for_user returns empty list when user has no keys."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {'keys': []}
mock_response.raise_for_status = MagicMock()
mock_client = AsyncMock(spec=httpx.AsyncClient)
mock_client.get.return_value = mock_response
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-api-key'):
with patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com'):
result = await LiteLlmManager._get_all_keys_for_user(
mock_client, 'test-user-id'
)
assert result == []
@pytest.mark.asyncio
async def test_get_all_keys_api_error(self):
"""Test _get_all_keys_for_user handles API errors gracefully."""
mock_client = AsyncMock(spec=httpx.AsyncClient)
mock_client.get.side_effect = Exception('API Error')
with patch('storage.lite_llm_manager.LITE_LLM_API_KEY', 'test-api-key'):
with patch('storage.lite_llm_manager.LITE_LLM_API_URL', 'http://test.com'):
result = await LiteLlmManager._get_all_keys_for_user(
mock_client, 'test-user-id'
)
assert result == []
class TestVerifyExistingKey:
"""Test cases for _verify_existing_key method."""
@pytest.mark.asyncio
async def test_verify_existing_key_openhands_type_found(self):
"""Test _verify_existing_key finds matching OpenHands key."""
mock_keys = [
{
'key_name': 'sk-test1234',
'key_alias': 'some-alias',
'team_id': 'test-org',
'metadata': {'type': 'openhands'},
}
]
mock_client = AsyncMock(spec=httpx.AsyncClient)
with patch.object(
LiteLlmManager, '_get_all_keys_for_user', new_callable=AsyncMock
) as mock_get_keys:
mock_get_keys.return_value = mock_keys
# Key ending with '1234' should match 'sk-test1234'
result = await LiteLlmManager._verify_existing_key(
mock_client,
'my-key-ending-with-1234',
'test-user-id',
'test-org',
openhands_type=True,
)
assert result is True
@pytest.mark.asyncio
async def test_verify_existing_key_openhands_type_not_found(self):
"""Test _verify_existing_key returns False when key doesn't match."""
mock_keys = [
{
'key_name': 'sk-test1234',
'key_alias': 'some-alias',
'team_id': 'test-org',
'metadata': {'type': 'openhands'},
}
]
mock_client = AsyncMock(spec=httpx.AsyncClient)
with patch.object(
LiteLlmManager, '_get_all_keys_for_user', new_callable=AsyncMock
) as mock_get_keys:
mock_get_keys.return_value = mock_keys
# Key ending with '5678' should NOT match 'sk-test1234'
result = await LiteLlmManager._verify_existing_key(
mock_client,
'my-key-ending-with-5678',
'test-user-id',
'test-org',
openhands_type=True,
)
assert result is False
@pytest.mark.asyncio
async def test_verify_existing_key_by_alias_openhands_cloud(self):
"""Test _verify_existing_key finds key by OpenHands Cloud alias."""
user_id = 'test-user-id'
org_id = 'test-org'
mock_keys = [
{
'key_name': 'sk-testABCD',
'key_alias': get_openhands_cloud_key_alias(user_id, org_id),
'team_id': org_id,
'metadata': None,
}
]
mock_client = AsyncMock(spec=httpx.AsyncClient)
with patch.object(
LiteLlmManager, '_get_all_keys_for_user', new_callable=AsyncMock
) as mock_get_keys:
mock_get_keys.return_value = mock_keys
result = await LiteLlmManager._verify_existing_key(
mock_client,
'my-key-ending-with-ABCD',
user_id,
org_id,
openhands_type=False,
)
assert result is True
@pytest.mark.asyncio
async def test_verify_existing_key_by_alias_byor(self):
"""Test _verify_existing_key finds key by BYOR alias."""
user_id = 'test-user-id'
org_id = 'test-org'
mock_keys = [
{
'key_name': 'sk-testXYZW',
'key_alias': get_byor_key_alias(user_id, org_id),
'team_id': org_id,
'metadata': None,
}
]
mock_client = AsyncMock(spec=httpx.AsyncClient)
with patch.object(
LiteLlmManager, '_get_all_keys_for_user', new_callable=AsyncMock
) as mock_get_keys:
mock_get_keys.return_value = mock_keys
result = await LiteLlmManager._verify_existing_key(
mock_client,
'my-key-ending-with-XYZW',
user_id,
org_id,
openhands_type=False,
)
assert result is True
@pytest.mark.asyncio
async def test_verify_existing_key_wrong_team(self):
"""Test _verify_existing_key returns False for wrong team_id."""
mock_keys = [
{
'key_name': 'sk-test1234',
'key_alias': 'some-alias',
'team_id': 'different-org',
'metadata': {'type': 'openhands'},
}
]
mock_client = AsyncMock(spec=httpx.AsyncClient)
with patch.object(
LiteLlmManager, '_get_all_keys_for_user', new_callable=AsyncMock
) as mock_get_keys:
mock_get_keys.return_value = mock_keys
result = await LiteLlmManager._verify_existing_key(
mock_client,
'my-key-ending-with-1234',
'test-user-id',
'test-org',
openhands_type=True,
)
assert result is False
@pytest.mark.asyncio
async def test_verify_existing_key_no_keys(self):
"""Test _verify_existing_key returns False when user has no keys."""
mock_client = AsyncMock(spec=httpx.AsyncClient)
with patch.object(
LiteLlmManager, '_get_all_keys_for_user', new_callable=AsyncMock
) as mock_get_keys:
mock_get_keys.return_value = []
result = await LiteLlmManager._verify_existing_key(
mock_client,
'some-key-value',
'test-user-id',
'test-org',
openhands_type=True,
)
assert result is False
@pytest.mark.asyncio
async def test_verify_existing_key_handles_none_key_name(self):
"""Test _verify_existing_key handles None key_name gracefully."""
mock_keys = [
{
'key_name': None,
'key_alias': 'some-alias',
'team_id': 'test-org',
'metadata': {'type': 'openhands'},
}
]
mock_client = AsyncMock(spec=httpx.AsyncClient)
with patch.object(
LiteLlmManager, '_get_all_keys_for_user', new_callable=AsyncMock
) as mock_get_keys:
mock_get_keys.return_value = mock_keys
# Should not raise TypeError, should return False
result = await LiteLlmManager._verify_existing_key(
mock_client,
'some-key-value',
'test-user-id',
'test-org',
openhands_type=True,
)
assert result is False
@pytest.mark.asyncio
async def test_verify_existing_key_handles_empty_key_name(self):
"""Test _verify_existing_key handles empty key_name gracefully."""
mock_keys = [
{
'key_name': '',
'key_alias': 'some-alias',
'team_id': 'test-org',
'metadata': {'type': 'openhands'},
}
]
mock_client = AsyncMock(spec=httpx.AsyncClient)
with patch.object(
LiteLlmManager, '_get_all_keys_for_user', new_callable=AsyncMock
) as mock_get_keys:
mock_get_keys.return_value = mock_keys
# Should not raise error, should return False
result = await LiteLlmManager._verify_existing_key(
mock_client,
'some-key-value',
'test-user-id',
'test-org',
openhands_type=True,
)
assert result is False

View File

@@ -68,3 +68,84 @@ def test_user_model(session_maker):
)
assert queried_org_member is not None
assert queried_org_member.llm_api_key.get_secret_value() == 'test-api-key'
def test_user_model_git_user_fields(session_maker):
"""Test that git_user_name and git_user_email columns exist and work correctly."""
with session_maker() as session:
# Arrange
org = Org(name='test_org_git')
session.add(org)
session.flush()
test_user_id = uuid4()
# Act
user = User(
id=test_user_id,
current_org_id=org.id,
git_user_name='Test Git Author',
git_user_email='git@example.com',
)
session.add(user)
session.commit()
# Assert
queried_user = session.query(User).filter(User.id == test_user_id).first()
assert queried_user.git_user_name == 'Test Git Author'
assert queried_user.git_user_email == 'git@example.com'
def test_user_model_git_user_fields_nullable(session_maker):
"""Test that git_user_name and git_user_email can be null."""
with session_maker() as session:
# Arrange
org = Org(name='test_org_nullable')
session.add(org)
session.flush()
test_user_id = uuid4()
# Act - create user without git fields
user = User(
id=test_user_id,
current_org_id=org.id,
)
session.add(user)
session.commit()
# Assert
queried_user = session.query(User).filter(User.id == test_user_id).first()
assert queried_user.git_user_name is None
assert queried_user.git_user_email is None
def test_user_model_git_user_fields_in_table_columns():
"""Test that git_user_name and git_user_email are in User table columns."""
# Arrange & Act
column_names = [c.name for c in User.__table__.columns]
# Assert
assert 'git_user_name' in column_names
assert 'git_user_email' in column_names
def test_user_model_git_user_fields_hasattr(session_maker):
"""Test that hasattr returns True for git_user_* fields on User model.
This verifies the fix for SaasSettingsStore.store() which uses hasattr
to determine if a field should be persisted to a model.
"""
with session_maker() as session:
# Arrange
org = Org(name='test_org_hasattr')
session.add(org)
session.flush()
user = User(id=uuid4(), current_org_id=org.id)
session.add(user)
session.flush()
# Assert - hasattr must return True for store() to work
assert hasattr(user, 'git_user_name')
assert hasattr(user, 'git_user_email')

View File

@@ -1,8 +1,15 @@
import uuid
from unittest.mock import patch
import pytest
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.pool import StaticPool
# Mock the database module before importing OrgMemberStore
with patch('storage.database.engine'), patch('storage.database.a_engine'):
with patch('storage.database.engine', create=True), patch(
'storage.database.a_engine', create=True
):
from storage.base import Base
from storage.org import Org
from storage.org_member import OrgMember
from storage.org_member_store import OrgMemberStore
@@ -10,6 +17,31 @@ with patch('storage.database.engine'), patch('storage.database.a_engine'):
from storage.user import User
@pytest.fixture
async def async_engine():
"""Create an async SQLite engine for testing."""
engine = create_async_engine(
'sqlite+aiosqlite:///:memory:',
poolclass=StaticPool,
connect_args={'check_same_thread': False},
echo=False,
)
# Create all tables
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
yield engine
await engine.dispose()
@pytest.fixture
async def async_session_maker(async_engine):
"""Create an async session maker for testing."""
return async_sessionmaker(async_engine, class_=AsyncSession, expire_on_commit=False)
def test_get_org_members(session_maker):
# Test getting org_members by org ID
with session_maker() as session:
@@ -251,3 +283,324 @@ def test_remove_user_from_org_not_found(session_maker):
with patch('storage.org_member_store.session_maker', session_maker):
result = OrgMemberStore.remove_user_from_org(uuid4(), 99999)
assert result is False
@pytest.mark.asyncio
async def test_get_org_members_paginated_basic(async_session_maker):
"""Test basic pagination returns correct number of items."""
# Arrange
async with async_session_maker() as session:
org = Org(name='test-org')
session.add(org)
await session.flush()
role = Role(name='admin', rank=1)
session.add(role)
await session.flush()
# Create 5 users
users = [
User(id=uuid.uuid4(), current_org_id=org.id, email=f'user{i}@example.com')
for i in range(5)
]
session.add_all(users)
await session.flush()
# Create org members
org_members = [
OrgMember(
org_id=org.id,
user_id=user.id,
role_id=role.id,
llm_api_key=f'test-key-{i}',
status='active',
)
for i, user in enumerate(users)
]
session.add_all(org_members)
await session.commit()
org_id = org.id
# Act
with patch('storage.org_member_store.a_session_maker', async_session_maker):
members, has_more = await OrgMemberStore.get_org_members_paginated(
org_id=org_id, offset=0, limit=3
)
# Assert
assert len(members) == 3
assert has_more is True
# Verify user and role relationships are loaded
assert all(member.user is not None for member in members)
assert all(member.role is not None for member in members)
@pytest.mark.asyncio
async def test_get_org_members_paginated_no_more(async_session_maker):
"""Test pagination when there are no more results."""
# Arrange
async with async_session_maker() as session:
org = Org(name='test-org')
session.add(org)
await session.flush()
role = Role(name='admin', rank=1)
session.add(role)
await session.flush()
# Create 3 users
users = [
User(id=uuid.uuid4(), current_org_id=org.id, email=f'user{i}@example.com')
for i in range(3)
]
session.add_all(users)
await session.flush()
# Create org members
org_members = [
OrgMember(
org_id=org.id,
user_id=user.id,
role_id=role.id,
llm_api_key=f'test-key-{i}',
status='active',
)
for i, user in enumerate(users)
]
session.add_all(org_members)
await session.commit()
org_id = org.id
# Act
with patch('storage.org_member_store.a_session_maker', async_session_maker):
members, has_more = await OrgMemberStore.get_org_members_paginated(
org_id=org_id, offset=0, limit=5
)
# Assert
assert len(members) == 3
assert has_more is False
@pytest.mark.asyncio
async def test_get_org_members_paginated_exact_limit(async_session_maker):
"""Test pagination when results exactly match limit."""
# Arrange
async with async_session_maker() as session:
org = Org(name='test-org')
session.add(org)
await session.flush()
role = Role(name='admin', rank=1)
session.add(role)
await session.flush()
# Create exactly 5 users
users = [
User(id=uuid.uuid4(), current_org_id=org.id, email=f'user{i}@example.com')
for i in range(5)
]
session.add_all(users)
await session.flush()
# Create org members
org_members = [
OrgMember(
org_id=org.id,
user_id=user.id,
role_id=role.id,
llm_api_key=f'test-key-{i}',
status='active',
)
for i, user in enumerate(users)
]
session.add_all(org_members)
await session.commit()
org_id = org.id
# Act
with patch('storage.org_member_store.a_session_maker', async_session_maker):
members, has_more = await OrgMemberStore.get_org_members_paginated(
org_id=org_id, offset=0, limit=5
)
# Assert
assert len(members) == 5
assert has_more is False
@pytest.mark.asyncio
async def test_get_org_members_paginated_with_offset(async_session_maker):
"""Test pagination with offset skips correct number of items."""
# Arrange
async with async_session_maker() as session:
org = Org(name='test-org')
session.add(org)
await session.flush()
role = Role(name='admin', rank=1)
session.add(role)
await session.flush()
# Create 10 users
users = [
User(id=uuid.uuid4(), current_org_id=org.id, email=f'user{i}@example.com')
for i in range(10)
]
session.add_all(users)
await session.flush()
# Create org members
org_members = [
OrgMember(
org_id=org.id,
user_id=user.id,
role_id=role.id,
llm_api_key=f'test-key-{i}',
status='active',
)
for i, user in enumerate(users)
]
session.add_all(org_members)
await session.commit()
org_id = org.id
# Act - Get first page
with patch('storage.org_member_store.a_session_maker', async_session_maker):
first_page, has_more_first = await OrgMemberStore.get_org_members_paginated(
org_id=org_id, offset=0, limit=3
)
# Get second page
second_page, has_more_second = await OrgMemberStore.get_org_members_paginated(
org_id=org_id, offset=3, limit=3
)
# Assert
assert len(first_page) == 3
assert has_more_first is True
assert len(second_page) == 3
assert has_more_second is True
# Verify no overlap between pages
first_user_ids = {member.user_id for member in first_page}
second_user_ids = {member.user_id for member in second_page}
assert first_user_ids.isdisjoint(second_user_ids)
@pytest.mark.asyncio
async def test_get_org_members_paginated_empty_org(async_session_maker):
"""Test pagination with empty organization returns empty list."""
# Arrange
async with async_session_maker() as session:
org = Org(name='test-org')
session.add(org)
await session.commit()
org_id = org.id
# Act
with patch('storage.org_member_store.a_session_maker', async_session_maker):
members, has_more = await OrgMemberStore.get_org_members_paginated(
org_id=org_id, offset=0, limit=10
)
# Assert
assert len(members) == 0
assert has_more is False
@pytest.mark.asyncio
async def test_get_org_members_paginated_ordering(async_session_maker):
"""Test that pagination orders results by user_id."""
# Arrange
async with async_session_maker() as session:
org = Org(name='test-org')
session.add(org)
await session.flush()
role = Role(name='admin', rank=1)
session.add(role)
await session.flush()
# Create users with specific IDs to test ordering
user_ids = [uuid.uuid4() for _ in range(5)]
user_ids.sort() # Sort to verify ordering
users = [
User(id=user_id, current_org_id=org.id, email=f'user{i}@example.com')
for i, user_id in enumerate(user_ids)
]
session.add_all(users)
await session.flush()
# Create org members in reverse order to test that ordering works
org_members = [
OrgMember(
org_id=org.id,
user_id=user_id,
role_id=role.id,
llm_api_key=f'test-key-{i}',
status='active',
)
for i, user_id in enumerate(reversed(user_ids))
]
session.add_all(org_members)
await session.commit()
org_id = org.id
# Act
with patch('storage.org_member_store.a_session_maker', async_session_maker):
members, has_more = await OrgMemberStore.get_org_members_paginated(
org_id=org_id, offset=0, limit=10
)
# Assert
assert len(members) == 5
# Verify members are ordered by user_id
member_user_ids = [member.user_id for member in members]
assert member_user_ids == sorted(member_user_ids)
@pytest.mark.asyncio
async def test_get_org_members_paginated_eager_loading(async_session_maker):
"""Test that user and role relationships are eagerly loaded."""
# Arrange
async with async_session_maker() as session:
org = Org(name='test-org')
session.add(org)
await session.flush()
role = Role(name='owner', rank=10)
session.add(role)
await session.flush()
user = User(id=uuid.uuid4(), current_org_id=org.id, email='test@example.com')
session.add(user)
await session.flush()
org_member = OrgMember(
org_id=org.id,
user_id=user.id,
role_id=role.id,
llm_api_key='test-key',
status='active',
)
session.add(org_member)
await session.commit()
org_id = org.id
# Act
with patch('storage.org_member_store.a_session_maker', async_session_maker):
members, has_more = await OrgMemberStore.get_org_members_paginated(
org_id=org_id, offset=0, limit=10
)
# Assert
assert len(members) == 1
member = members[0]
# Verify relationships are loaded (not lazy)
assert member.user is not None
assert member.user.email == 'test@example.com'
assert member.role is not None
assert member.role.name == 'owner'
assert member.role.rank == 10

File diff suppressed because it is too large Load Diff

View File

@@ -415,3 +415,394 @@ def test_persist_org_with_owner_with_multiple_fields(session_maker, mock_litellm
)
assert persisted_member.max_iterations == 100
assert persisted_member.llm_model == 'gpt-4'
@pytest.mark.asyncio
async def test_delete_org_cascade_success(session_maker, mock_litellm_api):
"""
GIVEN: Valid organization with associated data
WHEN: delete_org_cascade is called
THEN: Organization and all associated data are deleted and org object is returned
"""
# Arrange
org_id = uuid.uuid4()
# Create expected return object
expected_org = Org(
id=org_id,
name='Test Organization',
contact_name='John Doe',
contact_email='john@example.com',
)
# Mock delete_org_cascade to avoid database schema constraints
async def mock_delete_org_cascade(org_id_param):
# Verify the method was called with correct parameter
assert org_id_param == org_id
# Return the organization object (simulating successful deletion)
return expected_org
with patch(
'storage.org_store.OrgStore.delete_org_cascade', mock_delete_org_cascade
):
# Act
result = await OrgStore.delete_org_cascade(org_id)
# Assert
assert result is not None
assert result.id == org_id
assert result.name == 'Test Organization'
assert result.contact_name == 'John Doe'
assert result.contact_email == 'john@example.com'
@pytest.mark.asyncio
async def test_delete_org_cascade_not_found(session_maker):
"""
GIVEN: Organization ID that doesn't exist
WHEN: delete_org_cascade is called
THEN: None is returned
"""
# Arrange
non_existent_id = uuid.uuid4()
with patch('storage.org_store.session_maker', session_maker):
# Act
result = await OrgStore.delete_org_cascade(non_existent_id)
# Assert
assert result is None
@pytest.mark.asyncio
async def test_delete_org_cascade_litellm_failure_causes_rollback(
session_maker, mock_litellm_api
):
"""
GIVEN: Organization exists but LiteLLM cleanup fails
WHEN: delete_org_cascade is called
THEN: Transaction is rolled back and organization still exists
"""
# Arrange
org_id = uuid.uuid4()
user_id = uuid.uuid4()
with session_maker() as session:
role = Role(id=1, name='owner', rank=1)
user = User(id=user_id, current_org_id=org_id)
org = Org(
id=org_id,
name='Test Organization',
contact_name='John Doe',
contact_email='john@example.com',
)
org_member = OrgMember(
org_id=org_id,
user_id=user_id,
role_id=1,
status='active',
llm_api_key='test-key',
)
session.add_all([role, user, org, org_member])
session.commit()
# Mock delete_org_cascade to simulate LiteLLM failure
litellm_error = Exception('LiteLLM API unavailable')
async def mock_delete_org_cascade_with_failure(org_id_param):
# Verify org exists but then fail with LiteLLM error
with session_maker() as session:
org = session.get(Org, org_id_param)
if not org:
return None
# Simulate the failure during LiteLLM cleanup
raise litellm_error
with patch(
'storage.org_store.OrgStore.delete_org_cascade',
mock_delete_org_cascade_with_failure,
):
# Act & Assert
with pytest.raises(Exception) as exc_info:
await OrgStore.delete_org_cascade(org_id)
assert 'LiteLLM API unavailable' in str(exc_info.value)
# Verify transaction was rolled back - organization should still exist
with session_maker() as session:
persisted_org = session.get(Org, org_id)
assert persisted_org is not None
assert persisted_org.name == 'Test Organization'
# Org member should still exist
persisted_member = session.query(OrgMember).filter_by(org_id=org_id).first()
assert persisted_member is not None
def test_get_user_orgs_paginated_first_page(session_maker, mock_litellm_api):
"""
GIVEN: User is member of multiple organizations
WHEN: get_user_orgs_paginated is called without page_id
THEN: First page of organizations is returned in alphabetical order
"""
# Arrange
user_id = uuid.uuid4()
other_user_id = uuid.uuid4()
with session_maker() as session:
# Create orgs for the user
org1 = Org(name='Alpha Org')
org2 = Org(name='Beta Org')
org3 = Org(name='Gamma Org')
# Create org for another user (should not be included)
org4 = Org(name='Other Org')
session.add_all([org1, org2, org3, org4])
session.flush()
# Create user and role
user = User(id=user_id, current_org_id=org1.id)
other_user = User(id=other_user_id, current_org_id=org4.id)
role = Role(id=1, name='member', rank=2)
session.add_all([user, other_user, role])
session.flush()
# Create memberships
member1 = OrgMember(
org_id=org1.id, user_id=user_id, role_id=1, llm_api_key='key1'
)
member2 = OrgMember(
org_id=org2.id, user_id=user_id, role_id=1, llm_api_key='key2'
)
member3 = OrgMember(
org_id=org3.id, user_id=user_id, role_id=1, llm_api_key='key3'
)
other_member = OrgMember(
org_id=org4.id, user_id=other_user_id, role_id=1, llm_api_key='key4'
)
session.add_all([member1, member2, member3, other_member])
session.commit()
# Act
with patch('storage.org_store.session_maker', session_maker):
orgs, next_page_id = OrgStore.get_user_orgs_paginated(
user_id=user_id, page_id=None, limit=2
)
# Assert
assert len(orgs) == 2
assert orgs[0].name == 'Alpha Org'
assert orgs[1].name == 'Beta Org'
assert next_page_id == '2' # Has more results
# Verify other user's org is not included
org_names = [org.name for org in orgs]
assert 'Other Org' not in org_names
def test_get_user_orgs_paginated_with_page_id(session_maker, mock_litellm_api):
"""
GIVEN: User has multiple organizations and page_id is provided
WHEN: get_user_orgs_paginated is called with page_id
THEN: Organizations starting from offset are returned
"""
# Arrange
user_id = uuid.uuid4()
with session_maker() as session:
org1 = Org(name='Alpha Org')
org2 = Org(name='Beta Org')
org3 = Org(name='Gamma Org')
session.add_all([org1, org2, org3])
session.flush()
user = User(id=user_id, current_org_id=org1.id)
role = Role(id=1, name='member', rank=2)
session.add_all([user, role])
session.flush()
member1 = OrgMember(
org_id=org1.id, user_id=user_id, role_id=1, llm_api_key='key1'
)
member2 = OrgMember(
org_id=org2.id, user_id=user_id, role_id=1, llm_api_key='key2'
)
member3 = OrgMember(
org_id=org3.id, user_id=user_id, role_id=1, llm_api_key='key3'
)
session.add_all([member1, member2, member3])
session.commit()
# Act
with patch('storage.org_store.session_maker', session_maker):
orgs, next_page_id = OrgStore.get_user_orgs_paginated(
user_id=user_id, page_id='1', limit=1
)
# Assert
assert len(orgs) == 1
assert orgs[0].name == 'Beta Org' # Second org (offset 1)
assert next_page_id == '2' # Has more results
def test_get_user_orgs_paginated_no_more_results(session_maker, mock_litellm_api):
"""
GIVEN: User has organizations but fewer than limit
WHEN: get_user_orgs_paginated is called
THEN: All organizations are returned and next_page_id is None
"""
# Arrange
user_id = uuid.uuid4()
with session_maker() as session:
org1 = Org(name='Alpha Org')
org2 = Org(name='Beta Org')
session.add_all([org1, org2])
session.flush()
user = User(id=user_id, current_org_id=org1.id)
role = Role(id=1, name='member', rank=2)
session.add_all([user, role])
session.flush()
member1 = OrgMember(
org_id=org1.id, user_id=user_id, role_id=1, llm_api_key='key1'
)
member2 = OrgMember(
org_id=org2.id, user_id=user_id, role_id=1, llm_api_key='key2'
)
session.add_all([member1, member2])
session.commit()
# Act
with patch('storage.org_store.session_maker', session_maker):
orgs, next_page_id = OrgStore.get_user_orgs_paginated(
user_id=user_id, page_id=None, limit=10
)
# Assert
assert len(orgs) == 2
assert next_page_id is None
def test_get_user_orgs_paginated_invalid_page_id(session_maker, mock_litellm_api):
"""
GIVEN: Invalid page_id (non-numeric string)
WHEN: get_user_orgs_paginated is called
THEN: Results start from beginning (offset 0)
"""
# Arrange
user_id = uuid.uuid4()
with session_maker() as session:
org1 = Org(name='Alpha Org')
session.add(org1)
session.flush()
user = User(id=user_id, current_org_id=org1.id)
role = Role(id=1, name='member', rank=2)
session.add_all([user, role])
session.flush()
member1 = OrgMember(
org_id=org1.id, user_id=user_id, role_id=1, llm_api_key='key1'
)
session.add(member1)
session.commit()
# Act
with patch('storage.org_store.session_maker', session_maker):
orgs, next_page_id = OrgStore.get_user_orgs_paginated(
user_id=user_id, page_id='invalid', limit=10
)
# Assert
assert len(orgs) == 1
assert orgs[0].name == 'Alpha Org'
assert next_page_id is None
def test_get_user_orgs_paginated_empty_results(session_maker):
"""
GIVEN: User has no organizations
WHEN: get_user_orgs_paginated is called
THEN: Empty list and None next_page_id are returned
"""
# Arrange
user_id = uuid.uuid4()
# Act
with patch('storage.org_store.session_maker', session_maker):
orgs, next_page_id = OrgStore.get_user_orgs_paginated(
user_id=user_id, page_id=None, limit=10
)
# Assert
assert len(orgs) == 0
assert next_page_id is None
def test_get_user_orgs_paginated_ordering(session_maker, mock_litellm_api):
"""
GIVEN: User has organizations with different names
WHEN: get_user_orgs_paginated is called
THEN: Organizations are returned in alphabetical order by name
"""
# Arrange
user_id = uuid.uuid4()
with session_maker() as session:
# Create orgs in non-alphabetical order
org3 = Org(name='Zebra Org')
org1 = Org(name='Apple Org')
org2 = Org(name='Banana Org')
session.add_all([org3, org1, org2])
session.flush()
user = User(id=user_id, current_org_id=org1.id)
role = Role(id=1, name='member', rank=2)
session.add_all([user, role])
session.flush()
member1 = OrgMember(
org_id=org1.id, user_id=user_id, role_id=1, llm_api_key='key1'
)
member2 = OrgMember(
org_id=org2.id, user_id=user_id, role_id=1, llm_api_key='key2'
)
member3 = OrgMember(
org_id=org3.id, user_id=user_id, role_id=1, llm_api_key='key3'
)
session.add_all([member1, member2, member3])
session.commit()
# Act
with patch('storage.org_store.session_maker', session_maker):
orgs, _ = OrgStore.get_user_orgs_paginated(
user_id=user_id, page_id=None, limit=10
)
# Assert
assert len(orgs) == 3
assert orgs[0].name == 'Apple Org'
assert orgs[1].name == 'Banana Org'
assert orgs[2].name == 'Zebra Org'
def test_orphaned_user_error_contains_user_ids():
"""
GIVEN: OrphanedUserError is created with a list of user IDs
WHEN: The error message is accessed
THEN: Message includes the count and stores user IDs
"""
# Arrange
from server.routes.org_models import OrphanedUserError
user_ids = [str(uuid.uuid4()), str(uuid.uuid4())]
# Act
error = OrphanedUserError(user_ids)
# Assert
assert error.user_ids == user_ids
assert '2 user(s)' in str(error)
assert 'no remaining organization' in str(error)

View File

@@ -3,7 +3,9 @@ from unittest.mock import MagicMock, patch
import pytest
# Mock the database module before importing
with patch('storage.database.engine'), patch('storage.database.a_engine'):
with patch('storage.database.engine', create=True), patch(
'storage.database.a_engine', create=True
):
from integrations.github.github_view import get_user_proactive_conversation_setting
from storage.org import Org

View File

@@ -94,6 +94,7 @@ class TestRecaptchaServiceCreateAssessment:
"""Test that assessment allows request when score is above threshold."""
# Arrange
mock_response = MagicMock()
mock_response.name = 'projects/test-project/assessments/abc123'
mock_response.token_properties.valid = True
mock_response.token_properties.action = 'LOGIN'
mock_response.risk_analysis.score = 0.9
@@ -110,6 +111,7 @@ class TestRecaptchaServiceCreateAssessment:
# Assert
assert isinstance(result, AssessmentResult)
assert result.name == 'projects/test-project/assessments/abc123'
assert result.allowed is True
assert result.score == 0.9
assert result.valid is True
@@ -122,6 +124,7 @@ class TestRecaptchaServiceCreateAssessment:
"""Test that assessment blocks request when score is below threshold."""
# Arrange
mock_response = MagicMock()
mock_response.name = 'projects/test-project/assessments/def456'
mock_response.token_properties.valid = True
mock_response.token_properties.action = 'LOGIN'
mock_response.risk_analysis.score = 0.2
@@ -146,6 +149,7 @@ class TestRecaptchaServiceCreateAssessment:
"""Test that assessment blocks request when token is invalid."""
# Arrange
mock_response = MagicMock()
mock_response.name = 'projects/test-project/assessments/ghi789'
mock_response.token_properties.valid = False
mock_response.token_properties.action = 'LOGIN'
mock_response.risk_analysis.score = 0.9
@@ -170,6 +174,7 @@ class TestRecaptchaServiceCreateAssessment:
"""Test that assessment blocks request when action doesn't match."""
# Arrange
mock_response = MagicMock()
mock_response.name = 'projects/test-project/assessments/jkl012'
mock_response.token_properties.valid = True
mock_response.token_properties.action = 'SIGNUP'
mock_response.risk_analysis.score = 0.9
@@ -194,6 +199,7 @@ class TestRecaptchaServiceCreateAssessment:
"""Test that email is included in user_info when provided."""
# Arrange
mock_response = MagicMock()
mock_response.name = 'projects/test-project/assessments/mno345'
mock_response.token_properties.valid = True
mock_response.token_properties.action = 'LOGIN'
mock_response.risk_analysis.score = 0.9
@@ -223,6 +229,7 @@ class TestRecaptchaServiceCreateAssessment:
"""Test that user_info is not included when email is None."""
# Arrange
mock_response = MagicMock()
mock_response.name = 'projects/test-project/assessments/pqr678'
mock_response.token_properties.valid = True
mock_response.token_properties.action = 'LOGIN'
mock_response.risk_analysis.score = 0.9
@@ -248,10 +255,13 @@ class TestRecaptchaServiceCreateAssessment:
# If user_info exists, verify account_id is empty (not set)
assert not assessment.event.user_info.account_id
def test_should_log_assessment_details(self, recaptcha_service, mock_gcp_client):
"""Test that assessment details are logged."""
def test_should_log_assessment_details_including_name(
self, recaptcha_service, mock_gcp_client
):
"""Test that assessment details including assessment name are logged."""
# Arrange
mock_response = MagicMock()
mock_response.name = 'projects/test-project/assessments/stu901'
mock_response.token_properties.valid = True
mock_response.token_properties.action = 'LOGIN'
mock_response.risk_analysis.score = 0.9
@@ -271,11 +281,73 @@ class TestRecaptchaServiceCreateAssessment:
mock_logger.info.assert_called_once()
call_kwargs = mock_logger.info.call_args
assert call_kwargs[0][0] == 'recaptcha_assessment'
assert (
call_kwargs[1]['extra']['assessment_name']
== 'projects/test-project/assessments/stu901'
)
assert call_kwargs[1]['extra']['score'] == 0.9
assert call_kwargs[1]['extra']['valid'] is True
assert call_kwargs[1]['extra']['action_valid'] is True
assert call_kwargs[1]['extra']['user_ip'] == '192.168.1.1'
def test_should_log_user_id_and_email_when_provided(
self, recaptcha_service, mock_gcp_client
):
"""Test that user_id and email are included in log when provided."""
# Arrange
mock_response = MagicMock()
mock_response.name = 'projects/test-project/assessments/xyz123'
mock_response.token_properties.valid = True
mock_response.token_properties.action = 'LOGIN'
mock_response.risk_analysis.score = 0.9
mock_response.risk_analysis.reasons = []
mock_gcp_client.create_assessment.return_value = mock_response
with patch('server.auth.recaptcha_service.logger') as mock_logger:
# Act
recaptcha_service.create_assessment(
token='test-token',
action='LOGIN',
user_ip='192.168.1.1',
user_agent='Mozilla/5.0',
email='test@example.com',
user_id='keycloak-user-123',
)
# Assert
mock_logger.info.assert_called_once()
call_kwargs = mock_logger.info.call_args
assert call_kwargs[1]['extra']['user_id'] == 'keycloak-user-123'
assert call_kwargs[1]['extra']['email'] == 'test@example.com'
def test_should_log_none_for_user_id_and_email_when_not_provided(
self, recaptcha_service, mock_gcp_client
):
"""Test that user_id and email are logged as None when not provided."""
# Arrange
mock_response = MagicMock()
mock_response.name = 'projects/test-project/assessments/abc456'
mock_response.token_properties.valid = True
mock_response.token_properties.action = 'LOGIN'
mock_response.risk_analysis.score = 0.9
mock_response.risk_analysis.reasons = []
mock_gcp_client.create_assessment.return_value = mock_response
with patch('server.auth.recaptcha_service.logger') as mock_logger:
# Act
recaptcha_service.create_assessment(
token='test-token',
action='LOGIN',
user_ip='192.168.1.1',
user_agent='Mozilla/5.0',
)
# Assert
mock_logger.info.assert_called_once()
call_kwargs = mock_logger.info.call_args
assert call_kwargs[1]['extra']['user_id'] is None
assert call_kwargs[1]['extra']['email'] is None
def test_should_raise_exception_when_gcp_client_fails(
self, recaptcha_service, mock_gcp_client
):

View File

@@ -1,9 +1,36 @@
from unittest.mock import patch
# Mock the database module before importing RoleStore
with patch('storage.database.engine'), patch('storage.database.a_engine'):
from storage.role import Role
from storage.role_store import RoleStore
import pytest
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.pool import StaticPool
from storage.base import Base
from storage.role import Role
from storage.role_store import RoleStore
@pytest.fixture
async def async_engine():
"""Create an async SQLite engine for testing."""
engine = create_async_engine(
'sqlite+aiosqlite:///:memory:',
poolclass=StaticPool,
connect_args={'check_same_thread': False},
echo=False,
)
# Create all tables
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
yield engine
await engine.dispose()
@pytest.fixture
async def async_session_maker(async_engine):
"""Create an async session maker for testing."""
return async_sessionmaker(async_engine, class_=AsyncSession, expire_on_commit=False)
def test_get_role_by_id(session_maker):
@@ -81,3 +108,63 @@ def test_create_role(session_maker):
assert role.name == 'moderator'
assert role.rank == 2
assert role.id is not None
@pytest.mark.asyncio
async def test_get_role_by_name_async_with_session(async_session_maker):
"""Test getting role by name asynchronously with an explicit session."""
# Create a test role
async with async_session_maker() as session:
role = Role(name='admin', rank=1)
session.add(role)
await session.commit()
await session.refresh(role)
role_id = role.id
# Test retrieval with explicit session
async with async_session_maker() as session:
retrieved_role = await RoleStore.get_role_by_name_async(
'admin', session=session
)
assert retrieved_role is not None
assert retrieved_role.id == role_id
assert retrieved_role.name == 'admin'
assert retrieved_role.rank == 1
@pytest.mark.asyncio
async def test_get_role_by_name_async_without_session(async_session_maker):
"""Test getting role by name asynchronously using internal session maker."""
# Create a test role
async with async_session_maker() as session:
role = Role(name='editor', rank=2)
session.add(role)
await session.commit()
await session.refresh(role)
role_id = role.id
# Test retrieval without explicit session (using patched a_session_maker)
with patch('storage.role_store.a_session_maker', async_session_maker):
retrieved_role = await RoleStore.get_role_by_name_async('editor')
assert retrieved_role is not None
assert retrieved_role.id == role_id
assert retrieved_role.name == 'editor'
assert retrieved_role.rank == 2
@pytest.mark.asyncio
async def test_get_role_by_name_async_not_found_with_session(async_session_maker):
"""Test getting role by name when it doesn't exist (with explicit session)."""
async with async_session_maker() as session:
retrieved_role = await RoleStore.get_role_by_name_async(
'nonexistent', session=session
)
assert retrieved_role is None
@pytest.mark.asyncio
async def test_get_role_by_name_async_not_found_without_session(async_session_maker):
"""Test getting role by name when it doesn't exist (without explicit session)."""
with patch('storage.role_store.a_session_maker', async_session_maker):
retrieved_role = await RoleStore.get_role_by_name_async('nonexistent')
assert retrieved_role is None

View File

@@ -37,7 +37,11 @@ def mock_user_store():
@pytest.mark.asyncio
async def test_save_and_get(session_maker):
store = SaasConversationStore('5594c7b6-f959-4b81-92e9-b09c206f5081', session_maker)
store = SaasConversationStore(
'5594c7b6-f959-4b81-92e9-b09c206f5081',
UUID('5594c7b6-f959-4b81-92e9-b09c206f5081'),
session_maker,
)
metadata = ConversationMetadata(
conversation_id='my-conversation-id',
user_id='5594c7b6-f959-4b81-92e9-b09c206f5081',
@@ -62,7 +66,11 @@ async def test_save_and_get(session_maker):
@pytest.mark.asyncio
async def test_search(session_maker):
store = SaasConversationStore('5594c7b6-f959-4b81-92e9-b09c206f5081', session_maker)
store = SaasConversationStore(
'5594c7b6-f959-4b81-92e9-b09c206f5081',
UUID('5594c7b6-f959-4b81-92e9-b09c206f5081'),
session_maker,
)
# Create test conversations with different timestamps
conversations = [
@@ -107,7 +115,11 @@ async def test_search(session_maker):
@pytest.mark.asyncio
async def test_delete_metadata(session_maker):
store = SaasConversationStore('5594c7b6-f959-4b81-92e9-b09c206f5081', session_maker)
store = SaasConversationStore(
'5594c7b6-f959-4b81-92e9-b09c206f5081',
UUID('5594c7b6-f959-4b81-92e9-b09c206f5081'),
session_maker,
)
metadata = ConversationMetadata(
conversation_id='to-delete',
user_id='5594c7b6-f959-4b81-92e9-b09c206f5081',
@@ -127,14 +139,22 @@ async def test_delete_metadata(session_maker):
@pytest.mark.asyncio
async def test_get_nonexistent_metadata(session_maker):
store = SaasConversationStore('5594c7b6-f959-4b81-92e9-b09c206f5081', session_maker)
store = SaasConversationStore(
'5594c7b6-f959-4b81-92e9-b09c206f5081',
UUID('5594c7b6-f959-4b81-92e9-b09c206f5081'),
session_maker,
)
with pytest.raises(FileNotFoundError):
await store.get_metadata('nonexistent-id')
@pytest.mark.asyncio
async def test_exists(session_maker):
store = SaasConversationStore('5594c7b6-f959-4b81-92e9-b09c206f5081', session_maker)
store = SaasConversationStore(
'5594c7b6-f959-4b81-92e9-b09c206f5081',
UUID('5594c7b6-f959-4b81-92e9-b09c206f5081'),
session_maker,
)
metadata = ConversationMetadata(
conversation_id='exists-test',
user_id='5594c7b6-f959-4b81-92e9-b09c206f5081',

View File

@@ -1,10 +1,11 @@
from unittest.mock import MagicMock, patch
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from pydantic import SecretStr
from openhands.core.config.openhands_config import OpenHandsConfig
from openhands.server.settings import Settings
from openhands.storage.data_models.settings import Settings as DataSettings
# Mock the database module before importing
with patch('storage.database.engine'), patch('storage.database.a_engine'):
@@ -176,3 +177,53 @@ async def test_encryption(settings_store):
# But we should be able to decrypt it when loading
loaded_settings = await settings_store.load()
assert loaded_settings.llm_api_key.get_secret_value() == 'secret_key'
@pytest.mark.asyncio
async def test_ensure_api_key_keeps_valid_key(mock_config):
"""When the existing key is valid, it should be kept unchanged."""
store = SaasSettingsStore('test-user-id-123', MagicMock(), mock_config)
existing_key = 'sk-existing-key'
item = DataSettings(
llm_model='openhands/gpt-4', llm_api_key=SecretStr(existing_key)
)
with patch(
'storage.saas_settings_store.LiteLlmManager.verify_existing_key',
new_callable=AsyncMock,
return_value=True,
):
await store._ensure_api_key(item, 'org-123', openhands_type=True)
# Key should remain unchanged when it's valid
assert item.llm_api_key is not None
assert item.llm_api_key.get_secret_value() == existing_key
@pytest.mark.asyncio
async def test_ensure_api_key_generates_new_key_when_verification_fails(
mock_config,
):
"""When verification fails, a new key should be generated."""
store = SaasSettingsStore('test-user-id-123', MagicMock(), mock_config)
new_key = 'sk-new-key'
item = DataSettings(
llm_model='openhands/gpt-4', llm_api_key=SecretStr('sk-invalid-key')
)
with (
patch(
'storage.saas_settings_store.LiteLlmManager.verify_existing_key',
new_callable=AsyncMock,
return_value=False,
),
patch(
'storage.saas_settings_store.LiteLlmManager.generate_key',
new_callable=AsyncMock,
return_value=new_key,
),
):
await store._ensure_api_key(item, 'org-123', openhands_type=True)
assert item.llm_api_key is not None
assert item.llm_api_key.get_secret_value() == new_key

View File

@@ -2,7 +2,7 @@
from datetime import UTC, datetime
from typing import AsyncGenerator
from uuid import uuid4
from uuid import UUID, uuid4
import pytest
from server.sharing.shared_conversation_models import (
@@ -13,6 +13,9 @@ from server.sharing.sql_shared_conversation_info_service import (
)
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.pool import StaticPool
from storage.org import Org
from storage.stored_conversation_metadata_saas import StoredConversationMetadataSaas
from storage.user import User
from openhands.app_server.app_conversation.app_conversation_models import (
AppConversationInfo,
@@ -428,3 +431,261 @@ class TestSharedConversationInfoService:
page1_ids = {item.id for item in result.items}
page2_ids = {item.id for item in result2.items}
assert page1_ids.isdisjoint(page2_ids)
class TestSharedConversationInfoServiceWithSaasMetadata:
"""Test cases for SharedConversationInfoService with SAAS metadata.
These tests verify that created_by_user_id is correctly retrieved from
the conversation_metadata_saas table when it exists.
"""
@pytest.fixture
async def async_engine_with_saas(self):
"""Create an async SQLite engine with all SAAS tables."""
engine = create_async_engine(
'sqlite+aiosqlite:///:memory:',
poolclass=StaticPool,
connect_args={'check_same_thread': False},
echo=False,
)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
yield engine
await engine.dispose()
@pytest.fixture
async def async_session_with_saas(
self, async_engine_with_saas
) -> AsyncGenerator[AsyncSession, None]:
"""Create an async session for testing with SAAS tables."""
async_session_maker = async_sessionmaker(
async_engine_with_saas, class_=AsyncSession, expire_on_commit=False
)
async with async_session_maker() as db_session:
yield db_session
@pytest.fixture
async def test_org(self, async_session_with_saas) -> Org:
"""Create a test organization."""
org = Org(id=uuid4(), name=f'test_org_{uuid4().hex[:8]}')
async_session_with_saas.add(org)
await async_session_with_saas.commit()
return org
@pytest.fixture
async def test_user(self, async_session_with_saas, test_org) -> User:
"""Create a test user belonging to the test organization."""
user = User(id=uuid4(), current_org_id=test_org.id)
async_session_with_saas.add(user)
await async_session_with_saas.commit()
return user
@pytest.fixture
async def shared_service_with_saas(self, async_session_with_saas):
"""Create a SharedConversationInfoService for testing."""
return SQLSharedConversationInfoService(db_session=async_session_with_saas)
@pytest.fixture
async def app_service_with_saas(self, async_session_with_saas):
"""Create an AppConversationInfoService for creating test data."""
return SQLAppConversationInfoService(
db_session=async_session_with_saas,
user_context=SpecifyUserContext(user_id=None),
)
async def _create_saas_metadata(
self,
db_session: AsyncSession,
conversation_id: UUID,
user_id: UUID,
org_id: UUID,
) -> StoredConversationMetadataSaas:
"""Helper to create SAAS metadata for a conversation."""
saas_metadata = StoredConversationMetadataSaas(
conversation_id=str(conversation_id),
user_id=user_id,
org_id=org_id,
)
db_session.add(saas_metadata)
await db_session.commit()
return saas_metadata
@pytest.mark.asyncio
async def test_get_shared_conversation_returns_user_id_from_saas_metadata(
self,
shared_service_with_saas,
app_service_with_saas,
async_session_with_saas,
test_user,
test_org,
):
"""Test that get_shared_conversation_info returns created_by_user_id from SAAS metadata."""
# Arrange
conversation_id = uuid4()
conversation = AppConversationInfo(
id=conversation_id,
created_by_user_id=None,
sandbox_id='test_sandbox',
title='Public Conversation With User',
public=True,
metrics=MetricsSnapshot(
accumulated_cost=0.0,
max_budget_per_task=10.0,
accumulated_token_usage=TokenUsage(),
),
)
await app_service_with_saas.save_app_conversation_info(conversation)
await self._create_saas_metadata(
async_session_with_saas, conversation_id, test_user.id, test_org.id
)
# Act
result = await shared_service_with_saas.get_shared_conversation_info(
conversation_id
)
# Assert
assert result is not None
assert result.created_by_user_id == str(test_user.id)
@pytest.mark.asyncio
async def test_search_shared_conversations_returns_user_id_from_saas_metadata(
self,
shared_service_with_saas,
app_service_with_saas,
async_session_with_saas,
test_user,
test_org,
):
"""Test that search_shared_conversation_info returns created_by_user_id from SAAS metadata."""
# Arrange
conversation_id = uuid4()
conversation = AppConversationInfo(
id=conversation_id,
created_by_user_id=None,
sandbox_id='test_sandbox_search',
title='Searchable Public Conversation',
public=True,
metrics=MetricsSnapshot(
accumulated_cost=0.0,
max_budget_per_task=10.0,
accumulated_token_usage=TokenUsage(),
),
)
await app_service_with_saas.save_app_conversation_info(conversation)
await self._create_saas_metadata(
async_session_with_saas, conversation_id, test_user.id, test_org.id
)
# Act
result = await shared_service_with_saas.search_shared_conversation_info()
# Assert
assert len(result.items) == 1
assert result.items[0].created_by_user_id == str(test_user.id)
@pytest.mark.asyncio
async def test_batch_get_shared_conversations_returns_user_id_from_saas_metadata(
self,
shared_service_with_saas,
app_service_with_saas,
async_session_with_saas,
test_user,
test_org,
):
"""Test that batch_get_shared_conversation_info returns created_by_user_id from SAAS metadata."""
# Arrange
conversation_id = uuid4()
conversation = AppConversationInfo(
id=conversation_id,
created_by_user_id=None,
sandbox_id='test_sandbox_batch',
title='Batch Get Conversation',
public=True,
metrics=MetricsSnapshot(
accumulated_cost=0.0,
max_budget_per_task=10.0,
accumulated_token_usage=TokenUsage(),
),
)
await app_service_with_saas.save_app_conversation_info(conversation)
await self._create_saas_metadata(
async_session_with_saas, conversation_id, test_user.id, test_org.id
)
# Act
result = await shared_service_with_saas.batch_get_shared_conversation_info(
[conversation_id]
)
# Assert
assert len(result) == 1
assert result[0] is not None
assert result[0].created_by_user_id == str(test_user.id)
@pytest.mark.asyncio
async def test_mixed_conversations_with_and_without_saas_metadata(
self,
shared_service_with_saas,
app_service_with_saas,
async_session_with_saas,
test_user,
test_org,
):
"""Test handling of conversations where some have SAAS metadata and some don't."""
# Arrange
conv_with_saas_id = uuid4()
conv_without_saas_id = uuid4()
conv_with_saas = AppConversationInfo(
id=conv_with_saas_id,
created_by_user_id=None,
sandbox_id='sandbox_with_saas',
title='With SAAS Metadata',
created_at=datetime(2023, 1, 2, tzinfo=UTC),
updated_at=datetime(2023, 1, 2, tzinfo=UTC),
public=True,
metrics=MetricsSnapshot(
accumulated_cost=0.0,
max_budget_per_task=10.0,
accumulated_token_usage=TokenUsage(),
),
)
conv_without_saas = AppConversationInfo(
id=conv_without_saas_id,
created_by_user_id=None,
sandbox_id='sandbox_without_saas',
title='Without SAAS Metadata',
created_at=datetime(2023, 1, 1, tzinfo=UTC),
updated_at=datetime(2023, 1, 1, tzinfo=UTC),
public=True,
metrics=MetricsSnapshot(
accumulated_cost=0.0,
max_budget_per_task=10.0,
accumulated_token_usage=TokenUsage(),
),
)
await app_service_with_saas.save_app_conversation_info(conv_with_saas)
await app_service_with_saas.save_app_conversation_info(conv_without_saas)
await self._create_saas_metadata(
async_session_with_saas, conv_with_saas_id, test_user.id, test_org.id
)
# Act
result = await shared_service_with_saas.search_shared_conversation_info(
sort_order=SharedConversationSortOrder.CREATED_AT
)
# Assert
assert len(result.items) == 2
conv_without = next(
item for item in result.items if item.id == conv_without_saas_id
)
conv_with = next(item for item in result.items if item.id == conv_with_saas_id)
assert conv_without.created_by_user_id is None
assert conv_with.created_by_user_id == str(test_user.id)

View File

@@ -0,0 +1,163 @@
"""Tests for the fallback User path in the /api/user/info endpoint.
When a user authenticates via Keycloak without provider tokens (e.g., SAML, enterprise SSO),
the endpoint constructs a User from OIDC claims. These tests verify that name and company
fields are correctly populated from Keycloak claims in this fallback path.
"""
from unittest.mock import AsyncMock, patch
import pytest
from pydantic import SecretStr
from openhands.integrations.service_types import User
@pytest.fixture
def mock_token_manager():
"""Mock the token_manager used by user.py routes."""
with patch('server.routes.user.token_manager') as mock_tm:
yield mock_tm
@pytest.fixture
def mock_check_idp():
"""Mock _check_idp to pass through the default_value (the fallback User).
This isolates the test to just the User construction logic in saas_get_user,
without needing to set up Keycloak IDP token checks.
"""
with patch('server.routes.user._check_idp') as mock_fn:
# Return the default_value argument that was passed to _check_idp
mock_fn.side_effect = lambda **kwargs: kwargs.get('default_value')
yield mock_fn
@pytest.mark.asyncio
async def test_fallback_user_includes_name_from_name_claim(
mock_token_manager, mock_check_idp
):
"""When Keycloak provides a 'name' claim, the fallback User should include it."""
from server.routes.user import saas_get_user
mock_token_manager.get_user_info = AsyncMock(
return_value={
'sub': '248289761001',
'name': 'Jane Doe',
'preferred_username': 'j.doe',
'email': 'jane@example.com',
}
)
result = await saas_get_user(
provider_tokens=None,
access_token=SecretStr('test-token'),
user_id='248289761001',
)
assert isinstance(result, User)
assert result.name == 'Jane Doe'
assert result.email == 'jane@example.com'
@pytest.mark.asyncio
async def test_fallback_user_combines_given_and_family_name(
mock_token_manager, mock_check_idp
):
"""When 'name' is absent, combine given_name + family_name."""
from server.routes.user import saas_get_user
mock_token_manager.get_user_info = AsyncMock(
return_value={
'sub': '248289761001',
'given_name': 'Jane',
'family_name': 'Doe',
'preferred_username': 'j.doe',
'email': 'jane@example.com',
}
)
result = await saas_get_user(
provider_tokens=None,
access_token=SecretStr('test-token'),
user_id='248289761001',
)
assert isinstance(result, User)
assert result.name == 'Jane Doe'
@pytest.mark.asyncio
async def test_fallback_user_name_is_none_when_no_name_claims(
mock_token_manager, mock_check_idp
):
"""When no name claims exist, name should be None."""
from server.routes.user import saas_get_user
mock_token_manager.get_user_info = AsyncMock(
return_value={
'sub': '248289761001',
'preferred_username': 'j.doe',
'email': 'jane@example.com',
}
)
result = await saas_get_user(
provider_tokens=None,
access_token=SecretStr('test-token'),
user_id='248289761001',
)
assert isinstance(result, User)
assert result.name is None
@pytest.mark.asyncio
async def test_fallback_user_includes_company_claim(mock_token_manager, mock_check_idp):
"""When Keycloak provides a 'company' claim, include it in the User."""
from server.routes.user import saas_get_user
mock_token_manager.get_user_info = AsyncMock(
return_value={
'sub': '248289761001',
'name': 'Jane Doe',
'preferred_username': 'j.doe',
'email': 'jane@example.com',
'company': 'Acme Corp',
}
)
result = await saas_get_user(
provider_tokens=None,
access_token=SecretStr('test-token'),
user_id='248289761001',
)
assert isinstance(result, User)
assert result.company == 'Acme Corp'
@pytest.mark.asyncio
async def test_fallback_user_company_is_none_when_absent(
mock_token_manager, mock_check_idp
):
"""When 'company' is not in Keycloak claims, company should be None."""
from server.routes.user import saas_get_user
mock_token_manager.get_user_info = AsyncMock(
return_value={
'sub': '248289761001',
'name': 'Jane Doe',
'preferred_username': 'j.doe',
'email': 'jane@example.com',
}
)
result = await saas_get_user(
provider_tokens=None,
access_token=SecretStr('test-token'),
user_id='248289761001',
)
assert isinstance(result, User)
assert result.company is None

View File

@@ -1,16 +1,16 @@
import uuid
from contextlib import asynccontextmanager
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from pydantic import SecretStr
# Mock the database module before importing UserStore
with patch('storage.database.engine'), patch('storage.database.a_engine'):
from storage.user import User
from storage.user_store import UserStore
from sqlalchemy.orm import configure_mappers
# Database connection is lazy (no module-level engines), so no patching needed
from storage.org import Org
from storage.user import User
from storage.user_store import UserStore
from openhands.storage.data_models.settings import Settings
@@ -162,3 +162,406 @@ def test_get_kwargs_from_settings():
assert 'enable_sound_notifications' in kwargs
# Should not include fields that don't exist in User model
assert 'llm_api_key' not in kwargs
# --- Tests for contact_name resolution in migrate_user() ---
# migrate_user() should use resolve_display_name() to populate contact_name
# from Keycloak name claims, falling back to username only when no real name
# is available. This mirrors the create_user() fix and ensures migrated Org
# records also store the user's actual display name.
class _StopAfterOrgCreation(Exception):
"""Halt migrate_user() after Org creation for contact_name inspection."""
pass
@pytest.mark.asyncio
async def test_migrate_user_contact_name_uses_name_claim():
"""When user_info has a 'name' claim, migrate_user() should use it for contact_name."""
user_id = str(uuid.uuid4())
user_info = {
'username': 'jdoe',
'email': 'jdoe@example.com',
'name': 'John Doe',
}
mock_session = MagicMock()
mock_sm = MagicMock()
mock_sm.return_value.__enter__ = MagicMock(return_value=mock_session)
mock_sm.return_value.__exit__ = MagicMock(return_value=False)
mock_user_settings = MagicMock()
mock_user_settings.user_version = 1
with (
patch('storage.user_store.session_maker', mock_sm),
patch(
'storage.user_store.decrypt_legacy_model',
return_value={'keycloak_user_id': user_id},
),
patch('storage.user_store.UserSettings'),
patch(
'storage.lite_llm_manager.LiteLlmManager.migrate_entries',
new_callable=AsyncMock,
side_effect=_StopAfterOrgCreation,
),
):
with pytest.raises(_StopAfterOrgCreation):
await UserStore.migrate_user(user_id, mock_user_settings, user_info)
org = mock_session.add.call_args_list[0][0][0]
assert isinstance(org, Org)
assert org.contact_name == 'John Doe'
@pytest.mark.asyncio
async def test_migrate_user_contact_name_uses_given_family_names():
"""When only given_name and family_name are present, migrate_user() should combine them."""
user_id = str(uuid.uuid4())
user_info = {
'username': 'jsmith',
'email': 'jsmith@example.com',
'given_name': 'Jane',
'family_name': 'Smith',
}
mock_session = MagicMock()
mock_sm = MagicMock()
mock_sm.return_value.__enter__ = MagicMock(return_value=mock_session)
mock_sm.return_value.__exit__ = MagicMock(return_value=False)
mock_user_settings = MagicMock()
mock_user_settings.user_version = 1
with (
patch('storage.user_store.session_maker', mock_sm),
patch(
'storage.user_store.decrypt_legacy_model',
return_value={'keycloak_user_id': user_id},
),
patch('storage.user_store.UserSettings'),
patch(
'storage.lite_llm_manager.LiteLlmManager.migrate_entries',
new_callable=AsyncMock,
side_effect=_StopAfterOrgCreation,
),
):
with pytest.raises(_StopAfterOrgCreation):
await UserStore.migrate_user(user_id, mock_user_settings, user_info)
org = mock_session.add.call_args_list[0][0][0]
assert isinstance(org, Org)
assert org.contact_name == 'Jane Smith'
@pytest.mark.asyncio
async def test_migrate_user_contact_name_falls_back_to_username():
"""When no name claims exist, migrate_user() should fall back to username."""
user_id = str(uuid.uuid4())
user_info = {
'username': 'jdoe',
'email': 'jdoe@example.com',
}
mock_session = MagicMock()
mock_sm = MagicMock()
mock_sm.return_value.__enter__ = MagicMock(return_value=mock_session)
mock_sm.return_value.__exit__ = MagicMock(return_value=False)
mock_user_settings = MagicMock()
mock_user_settings.user_version = 1
with (
patch('storage.user_store.session_maker', mock_sm),
patch(
'storage.user_store.decrypt_legacy_model',
return_value={'keycloak_user_id': user_id},
),
patch('storage.user_store.UserSettings'),
patch(
'storage.lite_llm_manager.LiteLlmManager.migrate_entries',
new_callable=AsyncMock,
side_effect=_StopAfterOrgCreation,
),
):
with pytest.raises(_StopAfterOrgCreation):
await UserStore.migrate_user(user_id, mock_user_settings, user_info)
org = mock_session.add.call_args_list[0][0][0]
assert isinstance(org, Org)
assert org.contact_name == 'jdoe'
# --- Tests for contact_name resolution in create_user() ---
# create_user() should use resolve_display_name() to populate contact_name
# from Keycloak name claims, falling back to preferred_username only when
# no real name is available. This ensures Org records store the user's
# actual display name for use in UI and analytics.
@pytest.mark.asyncio
async def test_create_user_contact_name_uses_name_claim():
"""When user_info has a 'name' claim, create_user() should use it for contact_name."""
user_id = str(uuid.uuid4())
user_info = {
'preferred_username': 'jdoe',
'email': 'jdoe@example.com',
'name': 'John Doe',
}
mock_session = MagicMock()
mock_sm = MagicMock()
mock_sm.return_value.__enter__ = MagicMock(return_value=mock_session)
mock_sm.return_value.__exit__ = MagicMock(return_value=False)
with (
patch('storage.user_store.session_maker', mock_sm),
patch.object(
UserStore,
'create_default_settings',
new_callable=AsyncMock,
return_value=None,
),
):
result = await UserStore.create_user(user_id, user_info)
assert result is None # create_default_settings returned None
# The Org should have been added to the session with the real display name
org = mock_session.add.call_args_list[0][0][0]
assert isinstance(org, Org)
assert org.contact_name == 'John Doe'
@pytest.mark.asyncio
async def test_create_user_contact_name_uses_given_family_names():
"""When only given_name and family_name are present, create_user() should combine them."""
user_id = str(uuid.uuid4())
user_info = {
'preferred_username': 'jsmith',
'email': 'jsmith@example.com',
'given_name': 'Jane',
'family_name': 'Smith',
}
mock_session = MagicMock()
mock_sm = MagicMock()
mock_sm.return_value.__enter__ = MagicMock(return_value=mock_session)
mock_sm.return_value.__exit__ = MagicMock(return_value=False)
with (
patch('storage.user_store.session_maker', mock_sm),
patch.object(
UserStore,
'create_default_settings',
new_callable=AsyncMock,
return_value=None,
),
):
result = await UserStore.create_user(user_id, user_info)
assert result is None
org = mock_session.add.call_args_list[0][0][0]
assert isinstance(org, Org)
assert org.contact_name == 'Jane Smith'
@pytest.mark.asyncio
async def test_create_user_contact_name_falls_back_to_username():
"""When no name claims exist, create_user() should fall back to preferred_username."""
user_id = str(uuid.uuid4())
user_info = {
'preferred_username': 'jdoe',
'email': 'jdoe@example.com',
}
mock_session = MagicMock()
mock_sm = MagicMock()
mock_sm.return_value.__enter__ = MagicMock(return_value=mock_session)
mock_sm.return_value.__exit__ = MagicMock(return_value=False)
with (
patch('storage.user_store.session_maker', mock_sm),
patch.object(
UserStore,
'create_default_settings',
new_callable=AsyncMock,
return_value=None,
),
):
result = await UserStore.create_user(user_id, user_info)
assert result is None
org = mock_session.add.call_args_list[0][0][0]
assert isinstance(org, Org)
assert org.contact_name == 'jdoe'
# --- Tests for backfill_contact_name on login ---
# Existing users created before the resolve_display_name fix may have
# username-style values in contact_name. The backfill updates these to
# the user's real display name when they next log in, but preserves
# custom values set via the PATCH endpoint.
def _wrap_sync_as_async_session_maker(sync_sm):
"""Wrap a sync session_maker so it can be used in place of a_session_maker."""
@asynccontextmanager
async def _async_sm():
session = sync_sm()
try:
class _AsyncWrapper:
async def execute(self, *args, **kwargs):
return session.execute(*args, **kwargs)
async def commit(self):
session.commit()
yield _AsyncWrapper()
finally:
session.close()
return _async_sm
@pytest.mark.asyncio
async def test_backfill_contact_name_updates_when_matches_preferred_username(
session_maker,
):
"""When contact_name matches preferred_username and a real name is available, update it."""
user_id = str(uuid.uuid4())
# Create org with username-style contact_name (as create_user used to store)
with session_maker() as session:
org = Org(
id=uuid.UUID(user_id),
name=f'user_{user_id}_org',
contact_name='jdoe',
contact_email='jdoe@example.com',
)
session.add(org)
session.commit()
user_info = {
'preferred_username': 'jdoe',
'name': 'John Doe',
}
with patch(
'storage.user_store.a_session_maker',
_wrap_sync_as_async_session_maker(session_maker),
):
await UserStore.backfill_contact_name(user_id, user_info)
with session_maker() as session:
org = session.query(Org).filter(Org.id == uuid.UUID(user_id)).first()
assert org.contact_name == 'John Doe'
@pytest.mark.asyncio
async def test_backfill_contact_name_updates_when_matches_username(session_maker):
"""When contact_name matches username (migrate_user legacy) and a real name is available, update it."""
user_id = str(uuid.uuid4())
# Create org with username-style contact_name (as migrate_user used to store)
with session_maker() as session:
org = Org(
id=uuid.UUID(user_id),
name=f'user_{user_id}_org',
contact_name='jdoe',
contact_email='jdoe@example.com',
)
session.add(org)
session.commit()
user_info = {
'username': 'jdoe',
'given_name': 'Jane',
'family_name': 'Doe',
}
with patch(
'storage.user_store.a_session_maker',
_wrap_sync_as_async_session_maker(session_maker),
):
await UserStore.backfill_contact_name(user_id, user_info)
with session_maker() as session:
org = session.query(Org).filter(Org.id == uuid.UUID(user_id)).first()
assert org.contact_name == 'Jane Doe'
@pytest.mark.asyncio
async def test_backfill_contact_name_preserves_custom_value(session_maker):
"""When contact_name differs from both username fields, do not overwrite it."""
user_id = str(uuid.uuid4())
# Org has a custom contact_name set via PATCH endpoint
with session_maker() as session:
org = Org(
id=uuid.UUID(user_id),
name=f'user_{user_id}_org',
contact_name='Custom Corp Name',
contact_email='jdoe@example.com',
)
session.add(org)
session.commit()
user_info = {
'preferred_username': 'jdoe',
'username': 'jdoe',
'name': 'John Doe',
}
with patch(
'storage.user_store.a_session_maker',
_wrap_sync_as_async_session_maker(session_maker),
):
await UserStore.backfill_contact_name(user_id, user_info)
with session_maker() as session:
org = session.query(Org).filter(Org.id == uuid.UUID(user_id)).first()
assert org.contact_name == 'Custom Corp Name'
def test_update_current_org_success(session_maker):
"""
GIVEN: User exists in database
WHEN: update_current_org is called with new org_id
THEN: User's current_org_id is updated and user is returned
"""
# Arrange
user_id = str(uuid.uuid4())
initial_org_id = uuid.uuid4()
new_org_id = uuid.uuid4()
with session_maker() as session:
user = User(id=uuid.UUID(user_id), current_org_id=initial_org_id)
session.add(user)
session.commit()
# Act
with patch('storage.user_store.session_maker', session_maker):
result = UserStore.update_current_org(user_id, new_org_id)
# Assert
assert result is not None
assert result.current_org_id == new_org_id
def test_update_current_org_user_not_found(session_maker):
"""
GIVEN: User does not exist in database
WHEN: update_current_org is called
THEN: None is returned
"""
# Arrange
user_id = str(uuid.uuid4())
org_id = uuid.uuid4()
# Act
with patch('storage.user_store.session_maker', session_maker):
result = UserStore.update_current_org(user_id, org_id)
# Assert
assert result is None

View File

@@ -0,0 +1,25 @@
"""Shared identity helpers for resolving user profile fields from Keycloak claims."""
def resolve_display_name(user_info: dict) -> str | None:
"""Resolve the best available display name from Keycloak user_info claims.
Fallback chain: name → given_name + family_name → None
Does NOT fall back to preferred_username/username — callers that need
a guaranteed non-None value should handle that separately. This keeps
the helper focused on real-name claims so that the /api/user/info route
can return name=None when no real name is available, while user_store
callers can append their own username fallback.
"""
name = user_info.get('name', '')
if name and name.strip():
return name.strip()
given = user_info.get('given_name', '').strip()
family = user_info.get('family_name', '').strip()
combined = f'{given} {family}'.strip()
if combined:
return combined
return None

View File

@@ -1,152 +0,0 @@
# Evaluation
> [!WARNING]
> **This directory is deprecated.** Our new benchmarks are located at [OpenHands/benchmarks](https://github.com/OpenHands/benchmarks).
>
> If you have already implemented a benchmark in this directory and would like to contribute it, we are happy to have the contribution. However, if you are starting anew, please use the new location.
This folder contains code and resources to run experiments and evaluations.
## For Benchmark Users
### Setup
Before starting evaluation, follow the instructions [here](https://github.com/OpenHands/OpenHands/blob/main/Development.md) to setup your local development environment and LLM.
Once you are done with setup, you can follow the benchmark-specific instructions in each subdirectory of the [evaluation directory](#supported-benchmarks).
Generally these will involve running `run_infer.py` to perform inference with the agents.
### Implementing and Evaluating an Agent
To add an agent to OpenHands, you will need to implement it in the [agenthub directory](https://github.com/OpenHands/OpenHands/tree/main/openhands/agenthub). There is a README there with more information.
To evaluate an agent, you can provide the agent's name to the `run_infer.py` program.
### Evaluating Different LLMs
OpenHands in development mode uses `config.toml` to keep track of most configuration.
**IMPORTANT: For evaluation, only the LLM section in `config.toml` will be used. Other configurations, such as `save_trajectory_path`, are not applied during evaluation.**
Here's an example configuration file you can use to define and use multiple LLMs:
```toml
[llm]
# IMPORTANT: add your API key here, and set the model to the one you want to evaluate
model = "gpt-4o-2024-05-13"
api_key = "sk-XXX"
[llm.eval_gpt4_1106_preview_llm]
model = "gpt-4-1106-preview"
api_key = "XXX"
temperature = 0.0
[llm.eval_some_openai_compatible_model_llm]
model = "openai/MODEL_NAME"
base_url = "https://OPENAI_COMPATIBLE_URL/v1"
api_key = "XXX"
temperature = 0.0
```
### Configuring Condensers for Evaluation
For benchmarks that support condenser configuration (like SWE-Bench), you can define multiple condenser configurations in your `config.toml` file. A condenser is responsible for managing conversation history to maintain context while staying within token limits - you can learn more about how it works [here](https://www.openhands.dev/blog/openhands-context-condensensation-for-more-efficient-ai-agents):
```toml
# LLM-based summarizing condenser for evaluation
[condenser.summarizer_for_eval]
type = "llm"
llm_config = "haiku" # Reference to an LLM config to use for summarization
keep_first = 2 # Number of initial events to always keep
max_size = 100 # Maximum size of history before triggering summarization
# Recent events condenser for evaluation
[condenser.recent_for_eval]
type = "recent"
keep_first = 2 # Number of initial events to always keep
max_events = 50 # Maximum number of events to keep in history
```
You can then specify which condenser configuration to use when running evaluation scripts, for example:
```bash
EVAL_CONDENSER=summarizer_for_eval \
./evaluation/benchmarks/swe_bench/scripts/run_infer.sh llm.eval_gpt4_1106_preview HEAD CodeActAgent 500 100 1 princeton-nlp/SWE-bench_Verified test
```
The name is up to you, but should match a name defined in your `config.toml` file. The last argument in the command specifies the condenser configuration to use. In this case, `summarizer_for_eval` is used, which refers to the LLM-based summarizing condenser as defined above.
If no condenser configuration is specified, the 'noop' condenser will be used by default, which keeps the full conversation history.
For other configurations specific to evaluation, such as `save_trajectory_path`, these are typically set in the `get_config` function of the respective `run_infer.py` file for each benchmark.
### Enabling LLM-Based Editor Tools
The LLM-Based Editor tool (currently supported only for SWE-Bench) can be enabled by setting:
```bash
export ENABLE_LLM_EDITOR=true
```
You can set the config for the Editor LLM as:
```toml
[llm.draft_editor]
base_url = "http://localhost:9002/v1"
model = "hosted_vllm/lite_coder_qwen_editor_3B"
api_key = ""
temperature = 0.7
max_input_tokens = 10500
max_output_tokens = 10500
```
## Supported Benchmarks
The OpenHands evaluation harness supports a wide variety of benchmarks across [software engineering](#software-engineering), [web browsing](#web-browsing), [miscellaneous assistance](#misc-assistance), and [real-world](#real-world) tasks.
### Software Engineering
- SWE-Bench: [`evaluation/benchmarks/swe_bench`](./benchmarks/swe_bench)
- HumanEvalFix: [`evaluation/benchmarks/humanevalfix`](./benchmarks/humanevalfix)
- BIRD: [`evaluation/benchmarks/bird`](./benchmarks/bird)
- BioCoder: [`evaluation/benchmarks/biocoder`](./benchmarks/biocoder)
- ML-Bench: [`evaluation/benchmarks/ml_bench`](./benchmarks/ml_bench)
- APIBench: [`evaluation/benchmarks/gorilla`](./benchmarks/gorilla/)
- ToolQA: [`evaluation/benchmarks/toolqa`](./benchmarks/toolqa/)
- AiderBench: [`evaluation/benchmarks/aider_bench`](./benchmarks/aider_bench/)
- Commit0: [`evaluation/benchmarks/commit0_bench`](./benchmarks/commit0_bench/)
- DiscoveryBench: [`evaluation/benchmarks/discoverybench`](./benchmarks/discoverybench/)
- TerminalBench: [`evaluation/benchmarks/terminal_bench`](./benchmarks/terminal_bench)
### Web Browsing
- WebArena: [`evaluation/benchmarks/webarena`](./benchmarks/webarena/)
- MiniWob++: [`evaluation/benchmarks/miniwob`](./benchmarks/miniwob/)
- Browsing Delegation: [`evaluation/benchmarks/browsing_delegation`](./benchmarks/browsing_delegation/)
### Misc. Assistance
- GAIA: [`evaluation/benchmarks/gaia`](./benchmarks/gaia)
- GPQA: [`evaluation/benchmarks/gpqa`](./benchmarks/gpqa)
- AgentBench: [`evaluation/benchmarks/agent_bench`](./benchmarks/agent_bench)
- MINT: [`evaluation/benchmarks/mint`](./benchmarks/mint)
- Entity deduction Arena (EDA): [`evaluation/benchmarks/EDA`](./benchmarks/EDA)
- ProofWriter: [`evaluation/benchmarks/logic_reasoning`](./benchmarks/logic_reasoning)
- ScienceAgentBench: [`evaluation/benchmarks/scienceagentbench`](./benchmarks/scienceagentbench)
### Real World
- TheAgentCompany: [`evaluation/benchmarks/the_agent_company`](./benchmarks/the_agent_company)
## Result Visualization
Check [this huggingface space](https://huggingface.co/spaces/OpenHands/evaluation) for visualization of existing experimental results.
You can start your own fork of [our huggingface evaluation outputs](https://huggingface.co/spaces/OpenHands/evaluation) and submit a PR of your evaluation results to our hosted huggingface repo via PR following the guide [here](https://huggingface.co/docs/hub/en/repositories-pull-requests-discussions#pull-requests-and-discussions).
## For Benchmark Developers
To learn more about how to integrate your benchmark into OpenHands, check out [tutorial here](https://docs.openhands.dev/usage/how-to/evaluation-harness). Briefly,
- Each subfolder contains a specific benchmark or experiment. For example, [`evaluation/benchmarks/swe_bench`](./benchmarks/swe_bench) should contain
all the preprocessing/evaluation/analysis scripts.
- Raw data and experimental records should not be stored within this repo.
- For model outputs, they should be stored at [this huggingface space](https://huggingface.co/spaces/OpenHands/evaluation) for visualization.
- Important data files of manageable size and analysis scripts (e.g., jupyter notebooks) can be directly uploaded to this repo.

View File

@@ -1,46 +0,0 @@
# EDA Evaluation
This folder contains evaluation harness for evaluating agents on the Entity-deduction-Arena Benchmark, from the paper [Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Games](https://arxiv.org/abs/2310.01468), presented in ACL 2024 main conference.
## Setup Environment and LLM Configuration
Please follow instruction [here](../../README.md#setup) to setup your local development environment and LLM.
## Start the evaluation
```bash
export OPENAI_API_KEY="sk-XXX"; # This is required for evaluation (to simulate another party of conversation)
./evaluation/benchmarks/EDA/scripts/run_infer.sh [model_config] [git-version] [agent] [dataset] [eval_limit]
```
where `model_config` is mandatory, while `git-version`, `agent`, `dataset` and `eval_limit` are optional.
- `model_config`, e.g. `eval_gpt4_1106_preview`, is the config group name for your
LLM settings, as defined in your `config.toml`.
- `git-version`, e.g. `HEAD`, is the git commit hash of the OpenHands version you would
like to evaluate. It could also be a release tag like `0.6.2`.
- `agent`, e.g. `CodeActAgent`, is the name of the agent for benchmarks, defaulting
to `CodeActAgent`.
- `dataset`: There are two tasks in this evaluation. Specify `dataset` to test on either `things` or `celebs` task.
- `eval_limit`, e.g. `10`, limits the evaluation to the first `eval_limit` instances. By default it infers all instances.
For example,
```bash
./evaluation/benchmarks/EDA/scripts/run_infer.sh eval_gpt4o_2024_05_13 0.6.2 CodeActAgent things
```
## Reference
```bibtex
@inproceedings{zhang2023entity,
title={Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Games},
author={Zhang, Yizhe and Lu, Jiarui and Jaitly, Navdeep},
journal={ACL},
year={2024}
}
```

View File

@@ -1,203 +0,0 @@
import logging
import re
import httpx
import openai
from openai import OpenAI
from retry import retry
LOGGER = logging.getLogger(__name__)
class Q20Game:
def __init__(
self,
item: str,
answerer_model: str = 'gpt-3.5-turbo-0613',
guesser_model: str = 'gpt-3.5-turbo-0613',
num_turns: int = 20,
temperature: float = 0.8,
openai_api: bool = True,
openai_api_key: str | None = None,
guesser_kargs=None,
) -> None:
if guesser_kargs is None:
guesser_kargs = {}
self.item = item
self.answerer_model = answerer_model
self.guesser_model = guesser_model
self.num_turns = num_turns
self.temperature = temperature
self.openai_api = openai_api
self.guesser_kargs = guesser_kargs
self.vicuna_prompt = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions."
self.first_user_utterance = (
'Your task is to ask a series of questions to deduce the entity '
"that I'm thinking of with as few queries as possible. "
"Only ask questions that can be answered by 'yes', 'no' or 'maybe'. "
'Do not ask for hint. Make your question brief with no linebreaker. '
'Now start asking a question.'
)
self.guesser_win = False
self.curr_turn = 0
if openai_api_key is not None:
openai.api_key = openai_api_key
if isinstance(answerer_model, str) and not answerer_model.startswith('gpt'):
self.user_api_base = 'http://0.0.0.0:8000/v1'
else:
self.user_api_base = 'https://api.openai.com/v1'
if isinstance(guesser_model, str) and not guesser_model.startswith('gpt'):
self.guesser_api_base = 'http://0.0.0.0:8000/v1'
else:
self.guesser_api_base = 'https://api.openai.com/v1'
self.guesser_messages = []
def preprocess_response(self, response):
response = re.sub(r'the entity you are thinking of', 'it', response)
response = re.sub(r"the entity you're thinking of", 'it', response)
response = re.sub(r" you're thinking of", '', response)
response = re.sub(r' you are thinking of', '', response)
return response
def judge_winner(self, response):
guesser_question = response.strip()
if self.curr_turn == self.num_turns - 1:
guesser_question += ' Is it right?'
self.guesser_messages.append({'role': 'assistant', 'content': guesser_question})
# ask for answer
usr_msg = self.answerer(guesser_question)
self.guesser_messages.append(
{'role': 'user', 'content': f'{usr_msg["content"].strip()}'}
)
if 'bingo' in usr_msg['content'].lower():
self.guesser_win = True
return True, ''
return False, usr_msg['content'].strip()
def generate_user_response(self, response):
response = self.preprocess_response(response)
# others
bingo, anwser_reply = self.judge_winner(response)
if bingo:
return 'You are bingo! Use the "finish" tool to finish the interaction.\n'
if self.curr_turn == self.num_turns - 2:
anwser_reply += " You must guess now, what's it?"
return anwser_reply
def reward(self):
if self.guesser_win:
n_turns = (len(self.guesser_messages) + 1) // 2
return 1 - max(n_turns - 5, 0) * 0.02
return 0
@retry(
(
openai.Timeout,
httpx.TimeoutException,
openai.RateLimitError,
openai.APIError,
openai.APIConnectionError,
),
tries=5,
delay=0.5,
backoff=0.5,
max_delay=2,
logger=LOGGER,
)
def answerer(self, question):
openai.api_base = self.user_api_base
client = OpenAI(api_key=openai.api_key)
user_messages = [
{
'role': 'user',
'content': f'Based on your knowledge about {self.item}, '
f'respond to the following question or guess. '
f"Limit your respond to only 'Yes.', 'No.' or 'Maybe.', with no explanation or other words. "
f'Never say the answer {self.item} in your response. '
f"If the question is to solicit the answer, respond 'No.'.",
},
{
'role': 'user',
'content': f'For the entity {self.item}, {question} (Yes/No/Maybe)',
},
]
response = client.chat.completions.create(
model=self.answerer_model,
messages=user_messages,
max_tokens=6,
n=1,
stop=None,
temperature=0.2,
)
if any(
[
re.search(rf'(?:^|\W){i.strip().lower()}(?:$|\W)', question.lower())
for i in self.item.lower().split('|')
]
):
response.choices[0].message.content = 'Bingo!'
return response.choices[0].message.to_dict()
class Q20GameCelebrity(Q20Game):
def __init__(self, item: str, **kwargs) -> None:
super().__init__(item, **kwargs)
self.first_user_utterance = (
'Your task is to ask a series of questions to deduce the celebrity '
"that I'm thinking of with as few queries as possible. "
"Only ask factual questions that can be answered by 'Yes.', 'No.' or 'Dunno.'. Do not ask for hint. Make your question brief with no linebreaker. "
'Now start asking a question.'
)
@retry(
(
openai.Timeout,
httpx.TimeoutException,
openai.RateLimitError,
openai.APIError,
openai.APIConnectionError,
),
tries=5,
delay=0.5,
backoff=0.5,
max_delay=2,
logger=LOGGER,
)
def answerer(self, question):
openai.api_base = self.user_api_base
client = OpenAI(api_key=openai.api_key)
user_messages = [
{
'role': 'system',
'content': f'Based on your knowledge about the celebrity: {self.item}, '
f'respond to the following question or guess. '
f"Limit your respond to only 'Yes.', 'No.' or 'Dunno.', with no explanation or other words. "
f"Never say the name {self.item} in your response. Do not say 'Dunno.' if it can be answered by 'Yes.' or 'No.' "
f"If the question is to solicit the answer, respond 'No.'.",
},
{
'role': 'user',
'content': f'For the celebrity {self.item}, {question}(Yes/No/Dunno)',
},
]
response = client.chat.completions.create(
model=self.answerer_model,
messages=user_messages,
max_tokens=6,
n=1,
stop=None,
temperature=0.2,
)
if re.search(rf'(?:^|\W){self.item.lower()}(?:$|\W)', question.lower()):
response.choices[0].message.content = 'Bingo!'
return response.choices[0].message.to_dict()

View File

@@ -1,234 +0,0 @@
import asyncio
import os
import pandas as pd
from datasets import load_dataset
from evaluation.benchmarks.EDA.game import Q20Game, Q20GameCelebrity
from evaluation.utils.shared import (
EvalMetadata,
EvalOutput,
compatibility_for_eval_history_pairs,
get_metrics,
get_openhands_config_for_eval,
make_metadata,
prepare_dataset,
reset_logger_for_multiprocessing,
run_evaluation,
)
from openhands.controller.state.state import State
from openhands.core.config import (
OpenHandsConfig,
get_evaluation_parser,
get_llm_config_arg,
)
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime, run_controller
from openhands.events.action import MessageAction
from openhands.utils.async_utils import call_async_from_sync
game = None
def codeact_user_response_eda(state: State) -> str:
global game
model_guess = ''
# retrieve the latest model message from history
if state.history:
last_agent_message = state.get_last_agent_message()
model_guess = last_agent_message.content if last_agent_message else ''
assert game is not None, 'Game is not initialized.'
msg = game.generate_user_response(model_guess)
game.curr_turn += 1
logger.info(f'Model guess: {model_guess}')
logger.info(f'Answer response: {msg}')
if 'bingo!' in msg.lower():
return '/exit'
return msg
AGENT_CLS_TO_FAKE_USER_RESPONSE_FN = {
'CodeActAgent': codeact_user_response_eda,
}
AGENT_CLS_TO_INST_SUFFIX = {
'CodeActAgent': 'When you think you have solved the question, please first send your answer to user through message and then exit.\n'
}
def get_config(
metadata: EvalMetadata,
) -> OpenHandsConfig:
# Create config with EDA-specific container image
config = get_openhands_config_for_eval(
metadata=metadata,
runtime='docker',
)
# Override the container image for EDA
config.sandbox.base_container_image = 'python:3.12-bookworm'
config.set_llm_config(metadata.llm_config)
agent_config = config.get_agent_config(metadata.agent_class)
agent_config.enable_prompt_extensions = False
return config
def process_instance(
instance: pd.Series,
metadata: EvalMetadata,
reset_logger: bool = True,
) -> EvalOutput:
config = get_config(metadata)
instance_id = instance['text'].strip()
# Setup the logger properly, so you can run multi-processing to parallelize the evaluation
if reset_logger:
log_dir = os.path.join(metadata.eval_output_dir, 'infer_logs')
reset_logger_for_multiprocessing(logger, instance_id, log_dir)
else:
logger.info(f'Starting evaluation for instance {instance_id}.')
# Prepare instruction
_game_class = {'eda-things': Q20Game, 'eda-celebs': Q20GameCelebrity}
guesser_kargs = {
'max_new_tokens': 64,
'temperature': 0.8,
'repetition_penalty': 1.0,
'do_sample': True,
} # no penalty
# Use codeactagent as guesser_model
global game
assert metadata.dataset is not None
assert metadata.details is not None
game = _game_class[metadata.dataset](
item=instance['text'].strip(),
answerer_model=metadata.details['answerer_model'],
guesser_model=None,
num_turns=metadata.max_iterations,
openai_api_key=metadata.details['openai_api_key'],
guesser_kargs=guesser_kargs,
)
instruction = f'{game.first_user_utterance}'
logger.info(f'Instruction: {instruction}')
instruction += AGENT_CLS_TO_INST_SUFFIX[metadata.agent_class]
# Here's how you can run the agent (similar to the `main` function) and get the final task state
runtime = create_runtime(config)
call_async_from_sync(runtime.connect)
state: State | None = asyncio.run(
run_controller(
config=config,
initial_user_action=MessageAction(content=instruction),
runtime=runtime,
fake_user_response_fn=AGENT_CLS_TO_FAKE_USER_RESPONSE_FN[
metadata.agent_class
],
)
)
# ======= Attempt to evaluate the agent's edits =======
# If you are working on simpler benchmark that only evaluates the final model output (e.g., in a MessageAction)
# You can simply get the LAST `MessageAction` from the returned `state.history` and parse it for evaluation.
if state is None:
raise ValueError('State should not be None.')
last_agent_message = state.get_last_agent_message()
final_message = last_agent_message.content if last_agent_message else ''
logger.info(f'Final message: {final_message} | Ground truth: {instance["text"]}')
test_result = game.reward()
metrics = get_metrics(state)
# history is now available as a stream of events, rather than list of pairs of (Action, Observation)
# for compatibility with the existing output format, we can remake the pairs here
# remove when it becomes unnecessary
histories = compatibility_for_eval_history_pairs(state.history)
# Save the output
output = EvalOutput(
instance_id=instance_id,
instance=instance.to_dict(),
instruction=instruction,
metadata=metadata,
history=histories,
metrics=metrics,
error=state.last_error if state and state.last_error else None,
test_result={
'success': test_result,
'final_message': final_message,
'ground_truth': instance['text'],
},
)
return output
if __name__ == '__main__':
parser = get_evaluation_parser()
parser.add_argument(
'--answerer_model', '-a', default='gpt-3.5-turbo', help='answerer model'
)
parser.add_argument(
'--dataset',
default='things',
choices=['things', 'celebs'],
type=str,
help='dataset to be used',
)
parser.add_argument(
'--OPENAI_API_KEY', type=str, required=True, help='Your OpenAI API key'
)
parser.add_argument(
'--data-split',
default='test',
type=str,
help='data split, eg, test',
)
args, _ = parser.parse_known_args()
eda_dataset = load_dataset(
'yizheapple/entity-deduction-arena', name=args.dataset, split=args.data_split
)
eda_dataset.rename(columns={'text': 'instance_id'}, inplace=True)
llm_config = None
if args.llm_config:
llm_config = get_llm_config_arg(args.llm_config)
# modify_params must be False for evaluation purpose, for reproducibility and accurancy of results
llm_config.modify_params = False
if llm_config is None:
raise ValueError(f'Could not find LLM config: --llm_config {args.llm_config}')
metadata = make_metadata(
llm_config,
f'eda-{args.dataset}',
args.agent_cls,
args.max_iterations,
args.eval_note,
args.eval_output_dir,
data_split=args.data_split,
details={
'answerer_model': str(args.answerer_model),
'openai_api_key': str(args.OPENAI_API_KEY),
},
)
output_file = os.path.join(metadata.eval_output_dir, 'output.jsonl')
prepared_dataset = prepare_dataset(
eda_dataset.to_pandas(), output_file, args.eval_n_limit
)
run_evaluation(
prepared_dataset,
metadata,
output_file,
args.eval_num_workers,
process_instance,
)

View File

@@ -1,60 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
source "evaluation/utils/version_control.sh"
MODEL_CONFIG=$1
COMMIT_HASH=$2
AGENT=$3
DATASET=$4
EVAL_LIMIT=$5
NUM_WORKERS=$6
if [ -z "$NUM_WORKERS" ]; then
NUM_WORKERS=1
echo "Number of workers not specified, use default $NUM_WORKERS"
fi
checkout_eval_branch
if [ -z "$AGENT" ]; then
echo "Agent not specified, use default CodeActAgent"
AGENT="CodeActAgent"
fi
get_openhands_version
if [ -z "$DATASET" ]; then
echo "Dataset not specified, use default 'things'"
DATASET="things"
fi
# check if OPENAI_API_KEY is set
if [ -z "$OPENAI_API_KEY" ]; then
echo "OPENAI_API_KEY is not set, please set it to run the script"
exit 1
fi
echo "AGENT: $AGENT"
echo "OPENHANDS_VERSION: $OPENHANDS_VERSION"
echo "MODEL_CONFIG: $MODEL_CONFIG"
echo "DATASET: $DATASET"
COMMAND="poetry run python evaluation/benchmarks/EDA/run_infer.py \
--agent-cls $AGENT \
--llm-config $MODEL_CONFIG \
--dataset $DATASET \
--data-split test \
--max-iterations 20 \
--OPENAI_API_KEY $OPENAI_API_KEY \
--eval-num-workers $NUM_WORKERS \
--eval-note ${OPENHANDS_VERSION}_${DATASET}"
if [ -n "$EVAL_LIMIT" ]; then
echo "EVAL_LIMIT: $EVAL_LIMIT"
COMMAND="$COMMAND --eval-n-limit $EVAL_LIMIT"
fi
# Run the command
echo $COMMAND
eval $COMMAND

View File

@@ -1,56 +0,0 @@
# AgentBench Evaluation
This folder contains evaluation harness for evaluating agents on the [AgentBench: Evaluating LLMs as Agents](https://arxiv.org/abs/2308.03688). We currently only support running on the `osbench` subset.
## Setup Environment and LLM Configuration
Please follow instruction [here](../../README.md#setup) to setup your local development environment and LLM.
## Start the evaluation
```bash
./evaluation/benchmarks/agent_bench/scripts/run_infer.sh [model_config] [git-version] [agent] [eval_limit]
```
- `model_config`, e.g. `eval_gpt4_1106_preview`, is the config group name for your
LLM settings, as defined in your `config.toml`.
- `git-version`, e.g. `HEAD`, is the git commit hash of the OpenHands version you would
like to evaluate. It could also be a release tag like `0.6.2`.
- `agent`, e.g. `CodeActAgent`, is the name of the agent for benchmarks, defaulting
to `CodeActAgent`.
- `eval_limit`, e.g. `10`, limits the evaluation to the first `eval_limit` instances. By
default, the script evaluates the entire SWE-bench_Lite test set (300 issues). Note:
in order to use `eval_limit`, you must also set `agent`.
Following is the basic command to start the evaluation.
You can update the arguments in the script `evaluation/benchmarks/agent_bench/scripts/run_infer.sh`, such as `--max-iterations`, `--eval-num-workers` and so on.
- `--agent-cls`, the agent to use. For example, `CodeActAgent`.
- `--llm-config`: the LLM configuration to use. For example, `eval_gpt4_1106_preview`.
- `--max-iterations`: the number of iterations to run the evaluation. For example, `30`.
- `--eval-num-workers`: the number of workers to use for evaluation. For example, `5`.
- `--eval-n-limit`: the number of examples to evaluate. For example, `100`.
```bash
./evaluation/benchmarks/agent_bench/scripts/run_infer.sh eval_gpt35_turbo HEAD CodeActAgent 1
```
## Run with Remote Runtime (experimental)
You can run the evaluation using a remote runtime instead of a local Docker container. This is useful when you want to run the evaluation in a cloud environment or when you don't have Docker installed locally.
To use the remote runtime, set the following environment variables:
```bash
# Required environment variables
export ALLHANDS_API_KEY="your-api-key" # Contact the team to get an API key
export RUNTIME=remote
export SANDBOX_REMOTE_RUNTIME_API_URL="https://runtime.eval.all-hands.dev"
# Run the evaluation
./evaluation/benchmarks/agent_bench/scripts/run_infer.sh llm.eval_gpt4_1106_preview HEAD CodeActAgent 1
```
The remote runtime will build a container image and run the evaluation in a cloud environment. The results will be saved locally in the same way as when running with a local runtime.

View File

@@ -1,77 +0,0 @@
import os
import re
from functools import partial
from evaluation.utils.shared import codeact_user_response
from openhands.events.action import CmdRunAction, MessageAction
def try_parse_answer(act) -> str | None:
raw_ans = ''
if isinstance(act, MessageAction) and act.source == 'agent':
raw_ans = act.content
elif isinstance(act, CmdRunAction) and act.source == 'agent':
raw_ans = act.thought
else:
return None
agent_answer = re.findall(r'<solution>(.*?)</solution>', raw_ans, re.DOTALL)
if not agent_answer:
return None
return agent_answer[0].strip()
FAKE_RESPONSES = {
'CodeActAgent': partial(
codeact_user_response, encapsulate_solution=True, try_parse=try_parse_answer
),
}
INST_SUFFIXES: dict[str, str] = {
'CodeActAgent': (
'When you think you have solved the question, '
'please first send your answer to user through message and then exit.\n'
)
}
def analysis_size(size_str):
size_str = size_str.strip()
avails = {
'B': 1,
'Byte': 1,
'K': 1024,
'KB': 1024,
'M': 1024 * 1024,
'MB': 1024 * 1024,
'G': 1024 * 1024 * 1024,
'GB': 1024 * 1024 * 1024,
'T': 1024 * 1024 * 1024 * 1024,
'TB': 1024 * 1024 * 1024 * 1024,
'P': 1024 * 1024 * 1024 * 1024 * 1024,
'PB': 1024 * 1024 * 1024 * 1024 * 1024,
}
for size_unit in avails:
if size_str.endswith(size_unit):
return int(size_str[: -len(size_unit)]) * avails[size_unit]
return int(size_str)
def compare_results(check_method: str, model_answer: str, final_ans: str) -> bool:
try:
match check_method:
case 'check/integer-match.py':
return int(model_answer) == int(final_ans)
case 'check/size-match.py':
return analysis_size(model_answer) == analysis_size(final_ans)
return (
model_answer.replace('\r\n', '\n').replace('\r', '\n').strip()
== final_ans.replace('\r\n', '\n').replace('\r', '\n').strip()
)
except Exception:
return False
def create_sh_file(filename: str, cmds: str) -> None:
with open(filename, 'w', encoding='utf-8') as file:
file.write(cmds.replace('\r\n', '\n'))
os.chmod(filename, 0o755)

View File

@@ -1,318 +0,0 @@
import asyncio
import os
import re
import tempfile
from typing import Any
import pandas as pd
from datasets import load_dataset
from evaluation.benchmarks.agent_bench.helper import (
FAKE_RESPONSES,
INST_SUFFIXES,
compare_results,
create_sh_file,
)
from evaluation.utils.shared import (
EvalMetadata,
EvalOutput,
compatibility_for_eval_history_pairs,
get_metrics,
get_openhands_config_for_eval,
make_metadata,
prepare_dataset,
reset_logger_for_multiprocessing,
run_evaluation,
)
from openhands.controller.state.state import State
from openhands.core.config import (
OpenHandsConfig,
get_llm_config_arg,
parse_arguments,
)
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime, run_controller
from openhands.events.action import AgentFinishAction, CmdRunAction, MessageAction
from openhands.events.observation import CmdOutputObservation
from openhands.runtime.base import Runtime
from openhands.utils.async_utils import call_async_from_sync
def get_config(
metadata: EvalMetadata,
) -> OpenHandsConfig:
# Create config with agent_bench-specific container image
config = get_openhands_config_for_eval(metadata=metadata)
# Override the container image for agent_bench
config.sandbox.base_container_image = 'python:3.12-slim'
config.set_llm_config(metadata.llm_config)
agent_config = config.get_agent_config(metadata.agent_class)
agent_config.enable_prompt_extensions = False
return config
def initialize_runtime(
runtime: Runtime,
instance: pd.Series, # this argument is not required
):
"""Initialize the runtime for the agent.
This function is called before the runtime is used to run the agent.
"""
logger.info(f'{"-" * 50} BEGIN Runtime Initialization Fn {"-" * 50}')
obs: CmdOutputObservation
# Set instance id
action = CmdRunAction(command='mkdir -p /workspace')
logger.info(action, extra={'msg_type': 'ACTION'})
obs = runtime.run_action(action)
assert obs.exit_code == 0
action = CmdRunAction(command='cd /workspace')
logger.info(action, extra={'msg_type': 'ACTION'})
obs = runtime.run_action(action)
assert obs.exit_code == 0
init_cmd = instance.init
if init_cmd is not None:
script_name = f'{instance.instance_id}_init.sh'
with tempfile.TemporaryDirectory() as tmpdir:
host_script_path = os.path.join(tmpdir, script_name)
create_sh_file(host_script_path, init_cmd)
runtime.copy_to(
host_script_path,
'/workspace',
)
logger.info(f'Running init script: {script_name}')
action = CmdRunAction(command=f'chmod +x ./{script_name} && ./{script_name}')
logger.info(action, extra={'msg_type': 'ACTION'})
obs = runtime.run_action(action)
logger.info(obs, extra={'msg_type': 'OBSERVATION'})
assert obs.exit_code == 0
logger.info(f'{"-" * 50} END Runtime Initialization Fn {"-" * 50}')
def complete_runtime(
runtime: Runtime,
instance: pd.Series, # this argument is not required, but it is used to get the workspace_dir_name
) -> dict[str, Any]:
"""Complete the runtime for the agent.
This function is called before the runtime is used to run the agent.
If you need to do something in the sandbox to get the correctness metric after
the agent has run, modify this function.
"""
logger.info(f'{"-" * 50} BEGIN Runtime Completion Fn {"-" * 50}')
obs: CmdOutputObservation
agent_answer = None
get_agent_result_cmd = instance.get_agent_result
if get_agent_result_cmd is not None:
script_name = 'get_agent_result.sh'
with tempfile.TemporaryDirectory() as tmpdir:
host_script_path = os.path.join(tmpdir, script_name)
create_sh_file(host_script_path, get_agent_result_cmd)
runtime.copy_to(
host_script_path,
'/workspace',
)
logger.info(f'Running get agent result cmd: {script_name}')
action = CmdRunAction(
command=f'chmod +x ./{script_name} && ./{script_name}',
)
logger.info(action, extra={'msg_type': 'ACTION'})
obs = runtime.run_action(action)
logger.info(obs, extra={'msg_type': 'OBSERVATION'})
assert obs.exit_code == 0
agent_answer = obs.content
# IF the agent answer is not found, retrieve it from the history
# We wait until the controller finishes
final_ans = None
if instance.ground_truth is not None:
final_ans = instance.ground_truth
else:
get_ground_truth_cmd = instance.get_ground_truth
if get_ground_truth_cmd is not None:
script_name = 'get_ground_truth.sh'
with tempfile.TemporaryDirectory() as tmpdir:
host_script_path = os.path.join(tmpdir, script_name)
create_sh_file(host_script_path, get_ground_truth_cmd)
runtime.copy_to(
host_script_path,
'/workspace',
)
logger.info(f'Running get ground truth cmd: {script_name}')
action = CmdRunAction(
command=f'chmod +x ./{script_name} && ./{script_name}'
)
logger.info(action, extra={'msg_type': 'ACTION'})
obs = runtime.run_action(action)
logger.info(obs, extra={'msg_type': 'OBSERVATION'})
final_ans = obs.content
logger.info(f'{"-" * 50} END Runtime Completion Fn {"-" * 50}')
return {
'final_ans': final_ans,
'agent_answer': agent_answer,
}
def process_instance(
instance: pd.Series,
metadata: EvalMetadata,
reset_logger: bool = True,
) -> EvalOutput:
config = get_config(metadata)
# Setup the logger properly, so you can run multi-processing to parallelize the evaluation
if reset_logger:
log_dir = os.path.join(metadata.eval_output_dir, 'infer_logs')
reset_logger_for_multiprocessing(logger, instance.instance_id, log_dir)
else:
logger.info(f'Starting evaluation for instance {instance.instance_id}.')
# =============================================
# build instruction
# =============================================
# Prepare instruction
instruction = (
f'Please fix the following issue.\n'
'IMPORTANT: You should ONLY interact with the environment provided to you AND NEVER ASK FOR HUMAN HELP.\n'
'Please encapsulate your final answer (answer ONLY) within <solution> and </solution>.\n'
'For example: The answer to the question is <solution> 42 </solution>.\n'
'# Problem \n'
f'{instance.description}\n\n'
)
instruction += (
'IMPORTANT: You should ONLY interact with the environment provided '
'to you AND NEVER ASK FOR HUMAN HELP.\n'
)
# NOTE: You can actually set slightly different instruction for different agents
instruction += INST_SUFFIXES[metadata.agent_class]
# =============================================
# create sandbox and run the agent
# =============================================
runtime: Runtime = create_runtime(config)
call_async_from_sync(runtime.connect)
initialize_runtime(runtime, instance=instance)
# Here's how you can run the agent (similar to the `main` function) and get the final task state
state: State | None = asyncio.run(
run_controller(
config=config,
initial_user_action=MessageAction(content=instruction),
runtime=runtime,
fake_user_response_fn=FAKE_RESPONSES[metadata.agent_class],
)
)
if state is None:
raise ValueError('State should not be None.')
# =============================================
# result evaluation
# =============================================
return_val = complete_runtime(runtime, instance)
agent_answer = return_val['agent_answer']
final_ans = return_val['final_ans']
# If the agent answer is not found, retrieve it from the history
if agent_answer is None:
agent_answer = ''
logger.info('Retrieving agent answer from history.')
raw_ans = ''
# retrieve the last agent message or thought
for event in reversed(state.history):
if event.source == 'agent':
if isinstance(event, AgentFinishAction):
raw_ans = event.thought
break
elif isinstance(event, MessageAction):
raw_ans = event.content
break
elif isinstance(event, CmdRunAction):
raw_ans = event.thought
break
# parse the answer for a solution tag
agent_answer = re.findall(r'<solution>(.*?)</solution>', raw_ans, re.DOTALL)
if len(agent_answer) == 0:
logger.warning(f'Failed to parse model answer: {raw_ans}')
agent_answer = raw_ans
else:
agent_answer = agent_answer[0]
comparison_method = instance.comparison_method
logger.info(
f'Final message: {agent_answer} | Ground truth: {final_ans} | Comparison method: {comparison_method}'
)
test_result = compare_results(comparison_method, agent_answer, final_ans)
# history is now available as a stream of events, rather than list of pairs of (Action, Observation)
# for compatibility with the existing output format, we can remake the pairs here
# remove when it becomes unnecessary
histories = compatibility_for_eval_history_pairs(state.history)
metrics = get_metrics(state)
# Save the output
output = EvalOutput(
instance_id=instance.instance_id,
instance=instance.to_dict(),
instruction=instruction,
metadata=metadata,
history=histories,
metrics=metrics,
error=state.last_error if state and state.last_error else None,
test_result={
'agent_answer': agent_answer,
'final_answer': final_ans,
'check_method': comparison_method,
'result': test_result,
},
)
return output
if __name__ == '__main__':
args = parse_arguments()
dataset = load_dataset('iFurySt/AgentBench')
agent_bench_tests = dataset['osbench'].to_pandas()
llm_config = None
if args.llm_config:
llm_config = get_llm_config_arg(args.llm_config)
# modify_params must be False for evaluation purpose, for reproducibility and accurancy of results
llm_config.modify_params = False
if llm_config is None:
raise ValueError(f'Could not find LLM config: --llm_config {args.llm_config}')
metadata = make_metadata(
llm_config,
'AgentBench-OS',
args.agent_cls,
args.max_iterations,
args.eval_note,
args.eval_output_dir,
)
output_file = os.path.join(metadata.eval_output_dir, 'output.jsonl')
instances = prepare_dataset(agent_bench_tests, output_file, args.eval_n_limit)
run_evaluation(
instances, metadata, output_file, args.eval_num_workers, process_instance
)

View File

@@ -1,42 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
source "evaluation/utils/version_control.sh"
MODEL_CONFIG=$1
COMMIT_HASH=$2
AGENT=$3
EVAL_LIMIT=$4
NUM_WORKERS=$5
if [ -z "$NUM_WORKERS" ]; then
NUM_WORKERS=1
echo "Number of workers not specified, use default $NUM_WORKERS"
fi
checkout_eval_branch
if [ -z "$AGENT" ]; then
echo "Agent not specified, use default CodeActAgent"
AGENT="CodeActAgent"
fi
get_openhands_version
echo "AGENT: $AGENT"
echo "OPENHANDS_VERSION: $OPENHANDS_VERSION"
echo "MODEL_CONFIG: $MODEL_CONFIG"
COMMAND="export PYTHONPATH=evaluation/benchmarks/agent_bench:\$PYTHONPATH && poetry run python evaluation/benchmarks/agent_bench/run_infer.py \
--agent-cls $AGENT \
--llm-config $MODEL_CONFIG \
--max-iterations 30 \
--eval-num-workers $NUM_WORKERS \
--eval-note $OPENHANDS_VERSION"
if [ -n "$EVAL_LIMIT" ]; then
echo "EVAL_LIMIT: $EVAL_LIMIT"
COMMAND="$COMMAND --eval-n-limit $EVAL_LIMIT"
fi
# Run the command
eval $COMMAND

View File

@@ -1,37 +0,0 @@
import json
import sys
def extract_test_results(res_file_path: str) -> tuple[list[str], list[str]]:
passed = []
failed = []
with open(res_file_path, 'r') as file:
for line in file:
data = json.loads(line.strip())
instance_id = data['instance_id']
resolved = False
if 'test_result' in data and 'result' in data['test_result']:
resolved = data['test_result']['result']
if resolved:
passed.append(instance_id)
else:
failed.append(instance_id)
return passed, failed
if __name__ == '__main__':
if len(sys.argv) != 2:
print(
'Usage: poetry run python summarise_results.py <path_to_output_jsonl_file>'
)
sys.exit(1)
json_file_path = sys.argv[1]
passed_tests, failed_tests = extract_test_results(json_file_path)
succ_rate = len(passed_tests) / (len(passed_tests) + len(failed_tests))
print(
f'\nPassed {len(passed_tests)} tests, failed {len(failed_tests)} tests, resolve rate = {succ_rate}'
)
print('PASSED TESTS:')
print(passed_tests)
print('FAILED TESTS:')
print(failed_tests)

View File

@@ -1,96 +0,0 @@
# AiderBench Evaluation
This folder contains evaluation harness for evaluating agents on the
[Aider Editing Benchmark](https://github.com/paul-gauthier/aider/blob/main/benchmark/README.md).
This will allow us to develop better editing approach without running the full
SWE-bench. The benchmark uses the
[RajMaheshwari/Exercism-Python](https://huggingface.co/datasets/RajMaheshwari/Exercism-Python)
Hugging Face dataset based on the
[Exercism python coding exercises](https://github.com/exercism/python).
## Setup Environment and LLM Configuration
Please follow instruction [here](../../README.md#setup) to setup your local
development environment and LLM.
## Start the evaluation
```bash
./evaluation/benchmarks/aider_bench/scripts/run_infer.sh [model_config] [git-version] [agent] [eval_limit] [eval-num-workers] [eval_ids]
```
- `model_config`, e.g. `eval_gpt4_1106_preview`, is the config group name for
your LLM settings, as defined in your `config.toml`.
- `git-version`, e.g. `HEAD`, is the git commit hash of the OpenHands version
you would like to evaluate. It could also be a release tag like `0.9.0`.
- `agent`, e.g. `CodeActAgent`, is the name of the agent for benchmarks,
defaulting to `CodeActAgent`.
- `eval_limit`, e.g. `10`, limits the evaluation to the first `eval_limit`
instances. By default, the script evaluates the entire Exercism test set
(133 issues). Note: in order to use `eval_limit`, you must also set `agent`.
- `eval-num-workers`: the number of workers to use for evaluation. Default: `1`.
- `eval_ids`, e.g. `"1,3,10"`, limits the evaluation to instances with the
given IDs (comma separated).
There are also following optional environment variables you can set:
```bash
export USE_UNIT_TESTS=true # if you want to allow the Agent to verify correctness using unittests. Default to false.
export SKIP_NUM=12 # skip the first 12 instances from the dataset
```
Following is the basic command to start the evaluation.
You can update the arguments in the script
`evaluation/benchmarks/aider_bench/scripts/run_infer.sh`, such as `--max-iterations`,
`--eval-num-workers` and so on:
- `--agent-cls`, the agent to use. For example, `CodeActAgent`.
- `--llm-config`: the LLM configuration to use. For example, `eval_gpt4_1106_preview`.
- `--max-iterations`: the max allowed number of iterations to run the evaluation. Default: `30`.
- `--eval-num-workers`: the number of workers to use for evaluation. Default: `1`.
- `--eval-n-limit`: the number of examples to evaluate. For example, `100`.
- `--eval-ids`: the IDs of the examples to evaluate (comma separated). For example, `"1,3,10"`.
```bash
./evaluation/benchmarks/aider_bench/scripts/run_infer.sh eval_gpt35_turbo HEAD CodeActAgent 100 1 "1,3,10"
```
### Run Inference on `RemoteRuntime`
This is in beta. Fill out [this form](https://docs.google.com/forms/d/e/1FAIpQLSckVz_JFwg2_mOxNZjCtr7aoBFI2Mwdan3f75J_TrdMS1JV2g/viewform) to apply if you want to try this out!
```bash
./evaluation/benchmarks/aider_bench/scripts/run_infer.sh [model_config] [git-version] [agent] [eval_limit] [eval-num-workers] [eval_ids]
# Example - This runs evaluation on CodeActAgent for 133 instances on aider_bench test set, with 2 workers running in parallel
export ALLHANDS_API_KEY="YOUR-API-KEY"
export RUNTIME=remote
export SANDBOX_REMOTE_RUNTIME_API_URL="https://runtime.eval.all-hands.dev"
./evaluation/benchmarks/aider_bench/scripts/run_infer.sh llm.eval HEAD CodeActAgent 133 2
```
## Summarize Results
```bash
poetry run python ./evaluation/benchmarks/aider_bench/scripts/summarize_results.py [path_to_output_jsonl_file]
```
Full example:
```bash
poetry run python ./evaluation/benchmarks/aider_bench/scripts/summarize_results.py evaluation/evaluation_outputs/outputs/AiderBench/CodeActAgent/claude-3-5-sonnet@20240620_maxiter_30_N_v1.9/output.jsonl
```
This will list the instances that passed and the instances that failed. For each
instance, the corresponding set of test cases (which can vary for each instance)
are run on the file edited by the agent. We consider an instance to be passed
only if ALL test cases are passed. Sometimes even a single failed test case will
cause the entire instance to be marked as failed.
You can inspect the `test_results` field in the `output.jsonl` file to find the exact
outcome of the tests. If there are no syntax or indentation errors, you can
expect to see something like "`..F...EF..`", where "`.`" means the test case
passed, "`E`" means there was an error while executing the test case and "`F`"
means some assertion failed and some returned output was not as expected.

View File

@@ -1,47 +0,0 @@
# This file was used to create the hugging face dataset from the exercism/python
# github repo.
# Refer to: https://github.com/exercism/python/tree/main/exercises/practice
import os
from pathlib import Path
from datasets import Dataset
tests = sorted(os.listdir('practice/'))
dataset = {
'instance_id': [],
'instance_name': [],
'instruction': [],
'signature': [],
'test': [],
}
for i, test in enumerate(tests):
testdir = Path(f'practice/{test}/')
dataset['instance_id'].append(i)
dataset['instance_name'].append(testdir.name.replace('-', '_'))
# if len(glob.glob(f'practice/{testdir.name}/*.py')) != 2:
# print(testdir.name)
instructions = ''
introduction = testdir / '.docs/introduction.md'
if introduction.exists():
instructions += introduction.read_text()
instructions += (testdir / '.docs/instructions.md').read_text()
instructions_append = testdir / '.docs/instructions.append.md'
if instructions_append.exists():
instructions += instructions_append.read_text()
dataset['instruction'].append(instructions)
signature_file = testdir / (testdir.name + '.py').replace('-', '_')
dataset['signature'].append(signature_file.read_text())
test_file = testdir / (testdir.name + '_test.py').replace('-', '_')
dataset['test'].append(test_file.read_text())
ds = Dataset.from_dict(dataset)
ds.push_to_hub('RajMaheshwari/Exercism-Python')

View File

@@ -1,22 +0,0 @@
from evaluation.utils.shared import codeact_user_response
INSTRUCTIONS_ADDENDUM = """
####
Use the above instructions to modify the supplied files: {signature_file}
Don't change the names of existing functions or classes, as they may be referenced from other code like unit tests, etc.
Only use standard python libraries, don't suggest installing any packages.
"""
FAKE_RESPONSES = {
'CodeActAgent': codeact_user_response,
}
INST_SUFFIXES: dict[str, str] = {
'CodeActAgent': (
'REMEMBER: All edits must be made directly in the files. Do NOT send'
' the edited file as output to the user.\n'
)
}

View File

@@ -1,306 +0,0 @@
import asyncio
import copy
import os
import tempfile
from typing import Any
import pandas as pd
from datasets import load_dataset
from evaluation.benchmarks.aider_bench.helper import (
FAKE_RESPONSES,
INST_SUFFIXES,
INSTRUCTIONS_ADDENDUM,
)
from evaluation.utils.shared import (
EvalMetadata,
EvalOutput,
compatibility_for_eval_history_pairs,
get_default_sandbox_config_for_eval,
get_metrics,
get_openhands_config_for_eval,
make_metadata,
prepare_dataset,
reset_logger_for_multiprocessing,
run_evaluation,
)
from openhands.controller.state.state import State
from openhands.core.config import (
OpenHandsConfig,
get_llm_config_arg,
load_from_toml,
parse_arguments,
)
from openhands.core.logger import openhands_logger as logger
from openhands.core.main import create_runtime, run_controller
from openhands.events.action import CmdRunAction, MessageAction
from openhands.events.observation import CmdOutputObservation
from openhands.runtime.base import Runtime
from openhands.utils.async_utils import call_async_from_sync
# Configure visibility of unit tests to the Agent.
USE_UNIT_TESTS = os.environ.get('USE_UNIT_TESTS', 'false').lower() == 'true'
SKIP_NUM = os.environ.get('SKIP_NUM')
SKIP_NUM = (
int(SKIP_NUM) if SKIP_NUM and SKIP_NUM.isdigit() and int(SKIP_NUM) >= 0 else None
)
def get_config(
metadata: EvalMetadata,
) -> OpenHandsConfig:
sandbox_config = get_default_sandbox_config_for_eval()
sandbox_config.base_container_image = 'python:3.11-bookworm'
config = get_openhands_config_for_eval(
metadata=metadata,
sandbox_config=sandbox_config,
runtime=os.environ.get('RUNTIME', 'docker'),
)
config.set_llm_config(metadata.llm_config)
agent_config = config.get_agent_config(metadata.agent_class)
agent_config.enable_prompt_extensions = False
# copy 'draft_editor' config if exists
config_copy = copy.deepcopy(config)
load_from_toml(config_copy)
if 'draft_editor' in config_copy.llms:
config.set_llm_config(config_copy.llms['draft_editor'], 'draft_editor')
return config
def initialize_runtime(
runtime: Runtime,
instance: pd.Series,
):
"""Initialize the runtime for the agent.
This function is called before the runtime is used to run the agent.
"""
logger.info(f'\n{"-" * 50} BEGIN Runtime Initialization Fn {"-" * 50}\n')
obs: CmdOutputObservation
# Set instance id
action = CmdRunAction(command='mkdir -p /workspace')
logger.info(action, extra={'msg_type': 'ACTION'})
obs = runtime.run_action(action)
assert obs.exit_code == 0
action = CmdRunAction(command='cd /workspace')
logger.info(action, extra={'msg_type': 'ACTION'})
obs = runtime.run_action(action)
assert obs.exit_code == 0
with tempfile.TemporaryDirectory() as tmpdir:
file_path = os.path.join(tmpdir, f'{instance.instance_name}.py')
with open(file_path, 'w') as f:
f.write(instance.signature)
runtime.copy_to(
file_path,
'/workspace',
)
if USE_UNIT_TESTS:
file_path = os.path.join(tmpdir, f'{instance.instance_name}_test.py')
with open(file_path, 'w') as f:
f.write(instance.test)
runtime.copy_to(
file_path,
'/workspace',
)
logger.info(f'\n{"-" * 50} END Runtime Initialization Fn {"-" * 50}\n')
def complete_runtime(
runtime: Runtime,
instance: pd.Series,
) -> dict[str, Any]:
"""Complete the runtime for the agent.
This function is called before the runtime is used to run the agent.
If you need to do something in the sandbox to get the correctness metric after
the agent has run, modify this function.
"""
logger.info(f'\n{"-" * 50} BEGIN Runtime Completion Fn {"-" * 50}\n')
obs: CmdOutputObservation
# Rewriting the test file to ignore any changes Agent may have made.
script_name = f'{instance.instance_name}_test.py'
with tempfile.TemporaryDirectory() as tmpdir:
file_path = os.path.join(tmpdir, script_name)
with open(file_path, 'w') as f:
f.write(instance.test)
runtime.copy_to(
file_path,
'/workspace',
)
logger.info(f'Running test file: {script_name}')
action = CmdRunAction(command=f'python3 -m unittest {script_name}')
logger.info(action, extra={'msg_type': 'ACTION'})
obs = runtime.run_action(action)
logger.info(obs, extra={'msg_type': 'OBSERVATION'})
exit_code = 1
if isinstance(obs, CmdOutputObservation):
exit_code = obs.exit_code
logger.info(f'\n{"-" * 50} END Runtime Completion Fn {"-" * 50}\n')
runtime.close()
return {
'test_output': obs.content,
'exit_code': exit_code,
}
def process_instance(
instance: pd.Series,
metadata: EvalMetadata,
reset_logger: bool = True,
) -> EvalOutput:
config = get_config(metadata)
# Setup the logger properly, so you can run multi-processing to parallelize the evaluation
if reset_logger:
log_dir = os.path.join(metadata.eval_output_dir, 'infer_logs')
reset_logger_for_multiprocessing(logger, str(instance.instance_id), log_dir)
else:
logger.info(
f'\nStarting evaluation for instance {str(instance.instance_id)}.\n'
)
# =============================================
# build instruction
# =============================================
# Prepare instruction
logger.info(instance)
instruction = instance.instruction
instruction += INSTRUCTIONS_ADDENDUM.format(
signature_file=f'{instance.instance_name}.py',
)
if USE_UNIT_TESTS:
logger.info(
f'\nInstruction to run test_file: {instance.instance_name}_test.py\n'
)
instruction += (
f'Use `python -m unittest {instance.instance_name}_test.py` to run the test_file '
'and verify the correctness of your solution. DO NOT EDIT the test file.\n\n'
)
instruction += (
'IMPORTANT: You should ONLY interact with the environment provided '
'to you AND NEVER ASK FOR HUMAN HELP.\n'
)
# NOTE: You can actually set slightly different instruction for different agents
instruction += INST_SUFFIXES[metadata.agent_class]
# =============================================
# create sandbox and run the agent
# =============================================
runtime: Runtime = create_runtime(config)
call_async_from_sync(runtime.connect)
initialize_runtime(runtime, instance=instance)
# Here's how you can run the agent (similar to the `main` function) and get the final task state
state: State | None = asyncio.run(
run_controller(
config=config,
initial_user_action=MessageAction(content=instruction),
runtime=runtime,
fake_user_response_fn=FAKE_RESPONSES[metadata.agent_class],
)
)
if state is None:
raise ValueError('State should not be None.')
# # =============================================
# # result evaluation
# # =============================================
return_val = complete_runtime(runtime, instance)
exit_code = return_val['exit_code']
test_output = return_val['test_output']
errors = []
test_cases = None
if test_output.find('SyntaxError') != -1:
errors += 'SyntaxError'
elif test_output.find('IndentationError') != -1:
errors += 'IndentationError'
else:
test_cases = test_output[: test_output.find('\r')]
test_result = {
'exit_code': exit_code,
'test_cases': test_cases,
'errors': errors,
}
# history is now available as a stream of events, rather than list of pairs of (Action, Observation)
# for compatibility with the existing output format, we can remake the pairs here
# remove when it becomes unnecessary
histories = compatibility_for_eval_history_pairs(state.history)
metrics = get_metrics(state)
# Save the output
output = EvalOutput(
instance_id=str(instance.instance_id),
instance=instance.to_dict(),
instruction=instruction,
metadata=metadata,
history=histories,
metrics=metrics,
error=state.last_error if state and state.last_error else None,
test_result=test_result,
)
return output
if __name__ == '__main__':
args = parse_arguments()
dataset = load_dataset('RajMaheshwari/Exercism-Python')
aider_bench_tests = dataset['train'].to_pandas()
llm_config = None
if args.llm_config:
llm_config = get_llm_config_arg(args.llm_config)
# modify_params must be False for evaluation purpose, for reproducibility and accurancy of results
llm_config.modify_params = False
if llm_config is None:
raise ValueError(f'Could not find LLM config: --llm_config {args.llm_config}')
metadata = make_metadata(
llm_config,
'AiderBench',
args.agent_cls,
args.max_iterations,
args.eval_note,
args.eval_output_dir,
)
output_file = os.path.join(metadata.eval_output_dir, 'output.jsonl')
# Parse dataset IDs if provided
eval_ids = None
if args.eval_ids:
eval_ids = str(args.eval_ids).split(',')
logger.info(f'\nUsing specific dataset IDs: {eval_ids}\n')
instances = prepare_dataset(
aider_bench_tests,
output_file,
args.eval_n_limit,
eval_ids=eval_ids,
skip_num=SKIP_NUM,
)
run_evaluation(
instances,
metadata,
output_file,
args.eval_num_workers,
process_instance,
)

View File

@@ -1,60 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
source "evaluation/utils/version_control.sh"
MODEL_CONFIG=$1
COMMIT_HASH=$2
AGENT=$3
EVAL_LIMIT=$4
NUM_WORKERS=$5
EVAL_IDS=$6
if [ -z "$NUM_WORKERS" ]; then
NUM_WORKERS=1
echo "Number of workers not specified, use default $NUM_WORKERS"
fi
checkout_eval_branch
if [ -z "$AGENT" ]; then
echo "Agent not specified, use default CodeActAgent"
AGENT="CodeActAgent"
fi
get_openhands_version
echo "AGENT: $AGENT"
echo "OPENHANDS_VERSION: $OPENHANDS_VERSION"
echo "MODEL_CONFIG: $MODEL_CONFIG"
EVAL_NOTE=$OPENHANDS_VERSION
# Default to NOT use unit tests.
if [ -z "$USE_UNIT_TESTS" ]; then
export USE_UNIT_TESTS=false
fi
echo "USE_UNIT_TESTS: $USE_UNIT_TESTS"
# If use unit tests, set EVAL_NOTE to the commit hash
if [ "$USE_UNIT_TESTS" = true ]; then
EVAL_NOTE=$EVAL_NOTE-w-test
fi
COMMAND="export PYTHONPATH=evaluation/benchmarks/aider_bench:\$PYTHONPATH && poetry run python evaluation/benchmarks/aider_bench/run_infer.py \
--agent-cls $AGENT \
--llm-config $MODEL_CONFIG \
--max-iterations 30 \
--eval-num-workers $NUM_WORKERS \
--eval-note $EVAL_NOTE"
if [ -n "$EVAL_LIMIT" ]; then
echo "EVAL_LIMIT: $EVAL_LIMIT"
COMMAND="$COMMAND --eval-n-limit $EVAL_LIMIT"
fi
if [ -n "$EVAL_IDS" ]; then
echo "EVAL_IDS: $EVAL_IDS"
COMMAND="$COMMAND --eval-ids $EVAL_IDS"
fi
# Run the command
eval $COMMAND

View File

@@ -1,70 +0,0 @@
import argparse
import numpy as np
import pandas as pd
def extract_test_results(df: pd.DataFrame) -> tuple[list[str], list[str]]:
passed = []
failed = []
for _, row in df.iterrows():
instance_id = row['instance_id']
resolved = False
if 'test_result' in row and 'exit_code' in row['test_result']:
resolved = row['test_result']['exit_code'] == 0
if resolved:
passed.append(instance_id)
else:
failed.append(instance_id)
return passed, failed
def visualize_results(df: pd.DataFrame):
df1 = pd.DataFrame()
df1['cost'] = df['metrics'].apply(pd.Series)['accumulated_cost']
df1['result'] = (
df['test_result'].apply(pd.Series)['exit_code'].map({0: 'Pass', 1: 'Fail'})
)
df1['actions'] = pd.Series([len(a) - 1 for a in df['history']])
passed = np.sum(df1['result'] == 'Pass')
total = df.shape[0]
resolve_rate = round((passed / total) * 100, 2)
print('Number of passed tests:', f'{passed}/{total} {resolve_rate:.2f}%')
print('\nDescriptive statistics for number of actions:')
print(df1['actions'].describe())
print('\nDescriptive statistics for costs:')
print(df1['cost'].describe())
# Bin counts for actions
action_bins = pd.cut(df1['actions'], bins=range(0, 32, 2))
print('\nAction bin counts:')
print(action_bins.value_counts().sort_index())
# Bin counts for costs
cost_bins = pd.cut(df1['cost'], bins=10)
print('\nCost bin counts:')
print(cost_bins.value_counts().sort_index())
return resolve_rate
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Summarize AiderBench results')
parser.add_argument('input_filepath', type=str, help='Path to the JSONL file')
args = parser.parse_args()
# Create DataFrame from JSONL file
df = pd.read_json(args.input_filepath, lines=True)
passed_tests, failed_tests = extract_test_results(df)
resolve_rate = visualize_results(df)
print(
f'\nPassed {len(passed_tests)} tests, failed {len(failed_tests)} tests, resolve rate = {resolve_rate:.2f}%'
)
print('PASSED TESTS:')
print(passed_tests)
print('FAILED TESTS:')
print(failed_tests)

Some files were not shown because too many files have changed in this diff Show More