Compare commits

...

47 Commits

Author SHA1 Message Date
ak684
a1025bbb2f fix: AttributeError when calling .get() on KeycloakUserInfo Pydantic model
Replace dict-style .get() method call with direct attribute access for
accessing identity_provider from KeycloakUserInfo Pydantic model.

The bug occurred because Pydantic models do not have a .get() method
like dictionaries. The fix changes:
  'idp': user_info.get('identity_provider', 'keycloak')
to:
  'idp': user_info.identity_provider or 'keycloak'

Also adds tests to verify the fix works correctly:
- test_keycloak_callback_new_user_analytics_event: Tests direct attribute access
- test_keycloak_callback_new_user_analytics_fallback_idp: Tests fallback to 'keycloak'

Fixes #13243

Co-authored-by: openhands <openhands@all-hands.dev>
2026-03-05 20:12:48 +00:00
amanape
7c8e0b1eec fix(q6): add missing await on OrgStore.get_org_by_id in oauth_device.py 2026-03-05 22:51:19 +04:00
chuckbutkus
1a5d024c47 Merge branch 'main' into feat/revise-posthog 2026-03-05 12:33:30 -05:00
Chuck Butkus
0738e75dcf Fix async call 2026-03-05 12:30:49 -05:00
openhands
54766b4aeb fix: update PostHog tests and add missing await in auth.py
- Update test_auth_routes.py to use get_analytics_service instead of posthog
- Add OrgStore.get_org_by_id mocks to all tests using analytics
- Fix missing await for OrgStore.get_org_by_id call in auth.py
- Update assertions from .set to .set_person_properties

Co-authored-by: openhands <openhands@all-hands.dev>
2026-03-05 14:27:41 +00:00
amanape
4f65eae750 Merge remote-tracking branch 'origin/main' into feat/revise-posthog
# Conflicts:
#	frontend/__tests__/components/context-menu/account-settings-context-menu.test.tsx
#	frontend/src/components/features/context-menu/account-settings-context-menu.tsx
2026-03-05 17:45:34 +04:00
amanape
fdb6369476 reset 9 more formatting-only files to origin/main 2026-03-05 17:42:25 +04:00
amanape
77d672c68d revert unrelated ruff formatting changes from 23 files 2026-03-05 17:39:27 +04:00
Tim O'Farrell
21ac2a77ff Merge branch 'main' into feat/revise-posthog 2026-03-04 11:35:00 -07:00
openhands
b1c61c1534 fix: apply ruff formatting to webhook_router.py
Co-authored-by: openhands <openhands@all-hands.dev>
2026-03-04 14:24:22 +00:00
amanape
f450c407b5 Merge branch 'main' into feat/revise-posthog 2026-03-04 17:51:55 +04:00
openhands
500ed84d01 fix: apply enterprise linting fixes
- Fix import order to follow conventions
- Reformat assert statements for better readability
- Remove unnecessary blank lines

Co-authored-by: openhands <openhands@all-hands.dev>
2026-03-03 20:54:43 +00:00
amanape
999c18e072 fix: run ruff format on all enterprise files and fix remaining test mocks 2026-03-04 00:38:00 +04:00
amanape
a2e16d4819 fix: resolve ruff lint/format errors and update test mocks for analytics service 2026-03-04 00:23:30 +04:00
sp.wack
2f5147836f Merge branch 'main' into feat/revise-posthog 2026-03-03 23:24:28 +04:00
amanape
ed0f104645 fix(quick-3): remove stale experiments package from enterprise/pyproject.toml 2026-03-03 23:22:59 +04:00
amanape
192cfd5d91 chore(quick-2): regenerate enterprise/poetry.lock for posthog ^7.0.0 and pillow ^12.1.1
- Sync lock file after pyproject.toml bumped posthog ^6.0.0 -> ^7.0.0 and pillow ^12.1.0 -> ^12.1.1
2026-03-03 22:22:54 +04:00
amanape
8c2d3d1b9d fix(quick-2): fix ruff lint and format errors in webhook_router.py and secrets.py
- Fix I001 import ordering in webhook_router.py (top-level block: analytics before app_server/config; inline block: stdlib uuid before datetime)
- Apply ruff-format to both files (line length, quote style, indentation)
2026-03-03 22:22:32 +04:00
amanape
d3a274bbfa Merge remote-tracking branch 'origin/main' into feat/revise-posthog
# Conflicts:
#	enterprise/poetry.lock
#	frontend/src/components/features/auth/login-content.tsx
2026-03-03 22:10:39 +04:00
amanape
b9107ea3ad chore: remove .planning/ from tracking and add to .gitignore 2026-03-03 18:03:24 +04:00
amanape
8d66a58943 docs(04-01): complete activation events plan summary and state update 2026-03-03 17:24:13 +04:00
amanape
7864e9a8e3 docs(04-02): complete onboarding submission endpoint plan 2026-03-03 17:22:33 +04:00
amanape
f0b7e36bab feat(04-01): add GIT_PROVIDER_CONNECTED event capture in store_provider_tokens 2026-03-03 17:21:58 +04:00
amanape
53e87a7c27 feat(04-01): add USER_ACTIVATED event capture for first finished conversation 2026-03-03 17:21:19 +04:00
amanape
926ebf6906 feat(04-02): wire frontend onboarding hook to real API endpoint 2026-03-03 17:20:33 +04:00
amanape
7f25e9cad8 feat(04-02): create backend onboarding endpoint with analytics capture 2026-03-03 17:20:16 +04:00
amanape
2689768c95 docs(04-activation-and-dashboards): create phase plan 2026-03-03 05:00:34 +04:00
amanape
2f467558ed docs(04): research phase 4 activation and dashboards 2026-03-03 04:54:12 +04:00
amanape
b42ab23e1f feat(03-02): delete posthog-tracking test and remove useTracking mocks from 5 test files 2026-03-03 04:00:50 +04:00
amanape
3c5c307930 feat(03-02): remove all direct posthog.capture() calls from source files 2026-03-03 03:57:52 +04:00
amanape
d62d32af74 feat(03-01): add PostHog tracing headers and remove useTracking hook 2026-03-03 03:55:52 +04:00
amanape
f6201dd0de feat(02-03): add V0 best-effort terminal state analytics in conversation_callback_utils 2026-03-03 03:32:42 +04:00
amanape
ed4e2efd50 feat(02-03): add V1 terminal state analytics in on_event webhook 2026-03-03 03:32:42 +04:00
amanape
9b00b66efd feat(02-02): capture 'conversation created' event in V0 conversation_service 2026-03-03 03:16:22 +04:00
amanape
d78d9c4d99 feat(02-01): capture 'credit purchased' event in success_callback 2026-03-03 03:16:14 +04:00
amanape
04577c6448 feat(02-02): capture 'conversation created' event in V1 webhook_router 2026-03-03 03:15:55 +04:00
amanape
f8a0533f91 feat(02-01): capture 'user signed up' event in keycloak_callback 2026-03-03 03:15:49 +04:00
amanape
893a0db754 feat(01-02): wire SaasAppLifespanService and PostHogSessionMiddleware into saas_server.py 2026-03-03 02:42:51 +04:00
amanape
1f2bef34e3 feat(01-03): add analytics identity to device auth and org switch flows 2026-03-03 02:42:51 +04:00
amanape
62ed9e47cf feat(01-03): replace posthog.set() with full analytics identity in keycloak_callback 2026-03-03 02:42:51 +04:00
amanape
7b87237d3e feat(01-02): implement SaasAppLifespanService and PostHogSessionMiddleware 2026-03-03 02:42:51 +04:00
amanape
af74146f80 test(01-02): add failing tests for SaasAppLifespanService and PostHogSessionMiddleware 2026-03-03 02:42:51 +04:00
amanape
7760aba8e7 feat(01-03): set user_consents_to_analytics=True for new SaaS users in UserStore.create_user() 2026-03-03 02:42:51 +04:00
amanape
8550c91d0d chore(01-04): remove enterprise/experiments/ directory 2026-03-03 02:42:51 +04:00
amanape
d8db62b85b chore(01-01): bump PostHog SDK constraint to ^7.0.0 in enterprise/pyproject.toml 2026-03-03 02:42:51 +04:00
amanape
677f9bdd81 feat(01-01): implement AnalyticsService, constants, and OSS install ID module 2026-03-03 02:42:51 +04:00
amanape
9b1ce6d330 test(01-01): add failing tests for AnalyticsService, OSS install ID, and constants 2026-03-03 02:42:51 +04:00
55 changed files with 2082 additions and 978 deletions

3
.gitignore vendored
View File

@@ -1,3 +1,6 @@
# Planning docs (local only)
.planning/
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]

46
enterprise/poetry.lock generated
View File

@@ -3378,40 +3378,6 @@ files = [
{file = "grpcio-1.76.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:980a846182ce88c4f2f7e2c22c56aefd515daeb36149d1c897f83cf57999e0b6"},
{file = "grpcio-1.76.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:f92f88e6c033db65a5ae3d97905c8fea9c725b63e28d5a75cb73b49bda5024d8"},
{file = "grpcio-1.76.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:4baf3cbe2f0be3289eb68ac8ae771156971848bb8aaff60bad42005539431980"},
{file = "grpcio-1.76.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:615ba64c208aaceb5ec83bfdce7728b80bfeb8be97562944836a7a0a9647d882"},
{file = "grpcio-1.76.0-cp312-cp312-win32.whl", hash = "sha256:45d59a649a82df5718fd9527ce775fd66d1af35e6d31abdcdc906a49c6822958"},
{file = "grpcio-1.76.0-cp312-cp312-win_amd64.whl", hash = "sha256:c088e7a90b6017307f423efbb9d1ba97a22aa2170876223f9709e9d1de0b5347"},
{file = "grpcio-1.76.0-cp313-cp313-linux_armv7l.whl", hash = "sha256:26ef06c73eb53267c2b319f43e6634c7556ea37672029241a056629af27c10e2"},
{file = "grpcio-1.76.0-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:45e0111e73f43f735d70786557dc38141185072d7ff8dc1829d6a77ac1471468"},
{file = "grpcio-1.76.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:83d57312a58dcfe2a3a0f9d1389b299438909a02db60e2f2ea2ae2d8034909d3"},
{file = "grpcio-1.76.0-cp313-cp313-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:3e2a27c89eb9ac3d81ec8835e12414d73536c6e620355d65102503064a4ed6eb"},
{file = "grpcio-1.76.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:61f69297cba3950a524f61c7c8ee12e55c486cb5f7db47ff9dcee33da6f0d3ae"},
{file = "grpcio-1.76.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:6a15c17af8839b6801d554263c546c69c4d7718ad4321e3166175b37eaacca77"},
{file = "grpcio-1.76.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:25a18e9810fbc7e7f03ec2516addc116a957f8cbb8cbc95ccc80faa072743d03"},
{file = "grpcio-1.76.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:931091142fd8cc14edccc0845a79248bc155425eee9a98b2db2ea4f00a235a42"},
{file = "grpcio-1.76.0-cp313-cp313-win32.whl", hash = "sha256:5e8571632780e08526f118f74170ad8d50fb0a48c23a746bef2a6ebade3abd6f"},
{file = "grpcio-1.76.0-cp313-cp313-win_amd64.whl", hash = "sha256:f9f7bd5faab55f47231ad8dba7787866b69f5e93bc306e3915606779bbfb4ba8"},
{file = "grpcio-1.76.0-cp314-cp314-linux_armv7l.whl", hash = "sha256:ff8a59ea85a1f2191a0ffcc61298c571bc566332f82e5f5be1b83c9d8e668a62"},
{file = "grpcio-1.76.0-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:06c3d6b076e7b593905d04fdba6a0525711b3466f43b3400266f04ff735de0cd"},
{file = "grpcio-1.76.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:fd5ef5932f6475c436c4a55e4336ebbe47bd3272be04964a03d316bbf4afbcbc"},
{file = "grpcio-1.76.0-cp314-cp314-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:b331680e46239e090f5b3cead313cc772f6caa7d0fc8de349337563125361a4a"},
{file = "grpcio-1.76.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:2229ae655ec4e8999599469559e97630185fdd53ae1e8997d147b7c9b2b72cba"},
{file = "grpcio-1.76.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:490fa6d203992c47c7b9e4a9d39003a0c2bcc1c9aa3c058730884bbbb0ee9f09"},
{file = "grpcio-1.76.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:479496325ce554792dba6548fae3df31a72cef7bad71ca2e12b0e58f9b336bfc"},
{file = "grpcio-1.76.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:1c9b93f79f48b03ada57ea24725d83a30284a012ec27eab2cf7e50a550cbbbcc"},
{file = "grpcio-1.76.0-cp314-cp314-win32.whl", hash = "sha256:747fa73efa9b8b1488a95d0ba1039c8e2dca0f741612d80415b1e1c560febf4e"},
{file = "grpcio-1.76.0-cp314-cp314-win_amd64.whl", hash = "sha256:922fa70ba549fce362d2e2871ab542082d66e2aaf0c19480ea453905b01f384e"},
{file = "grpcio-1.76.0-cp39-cp39-linux_armv7l.whl", hash = "sha256:8ebe63ee5f8fa4296b1b8cfc743f870d10e902ca18afc65c68cf46fd39bb0783"},
{file = "grpcio-1.76.0-cp39-cp39-macosx_11_0_universal2.whl", hash = "sha256:3bf0f392c0b806905ed174dcd8bdd5e418a40d5567a05615a030a5aeddea692d"},
{file = "grpcio-1.76.0-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:0b7604868b38c1bfd5cf72d768aedd7db41d78cb6a4a18585e33fb0f9f2363fd"},
{file = "grpcio-1.76.0-cp39-cp39-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:e6d1db20594d9daba22f90da738b1a0441a7427552cc6e2e3d1297aeddc00378"},
{file = "grpcio-1.76.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d099566accf23d21037f18a2a63d323075bebace807742e4b0ac210971d4dd70"},
{file = "grpcio-1.76.0-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:ebea5cc3aa8ea72e04df9913492f9a96d9348db876f9dda3ad729cfedf7ac416"},
{file = "grpcio-1.76.0-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:0c37db8606c258e2ee0c56b78c62fc9dee0e901b5dbdcf816c2dd4ad652b8b0c"},
{file = "grpcio-1.76.0-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:ebebf83299b0cb1721a8859ea98f3a77811e35dce7609c5c963b9ad90728f886"},
{file = "grpcio-1.76.0-cp39-cp39-win32.whl", hash = "sha256:0aaa82d0813fd4c8e589fac9b65d7dd88702555f702fb10417f96e2a2a6d4c0f"},
{file = "grpcio-1.76.0-cp39-cp39-win_amd64.whl", hash = "sha256:acab0277c40eff7143c2323190ea57b9ee5fd353d8190ee9652369fae735668a"},
{file = "grpcio-1.76.0.tar.gz", hash = "sha256:7be78388d6da1a25c0d5ec506523db58b18be22d9c37d8d3a32c08be4987bd73"},
]
[package.dependencies]
@@ -7108,14 +7074,14 @@ tests = ["pytest (>=5.4.1)", "pytest-cov (>=2.8.1)", "pytest-mypy (>=0.8.0)", "p
[[package]]
name = "posthog"
version = "6.9.3"
version = "7.9.6"
description = "Integrate PostHog into any python application."
optional = false
python-versions = ">=3.9"
python-versions = ">=3.10"
groups = ["main"]
files = [
{file = "posthog-6.9.3-py3-none-any.whl", hash = "sha256:c71e9cb7ac4ef13eb604f04c3161edd10b1d08a32499edd54437ba5eab591c58"},
{file = "posthog-6.9.3.tar.gz", hash = "sha256:7d201774ea9eba156f1de46d34313e30b2384d523900fe8e425accc92486cc34"},
{file = "posthog-7.9.6-py3-none-any.whl", hash = "sha256:b1ceda033c9a6660c5d21e2b1c0b4113aaa0969ff02914bf23942c99f602b0f7"},
{file = "posthog-7.9.6.tar.gz", hash = "sha256:4e0ecb63885ce522d6c7ad4593871771995931764ae83914c364db0ad5de2bbf"},
]
[package.dependencies]
@@ -7129,7 +7095,7 @@ typing-extensions = ">=4.2.0"
[package.extras]
dev = ["django-stubs", "lxml", "mypy", "mypy-baseline", "packaging", "pre-commit", "pydantic", "ruff", "setuptools", "tomli", "tomli_w", "twine", "types-mock", "types-python-dateutil", "types-requests", "types-setuptools", "types-six", "wheel"]
langchain = ["langchain (>=0.2.0)"]
test = ["anthropic", "coverage", "django", "freezegun (==1.5.1)", "google-genai", "langchain-anthropic (>=0.3.15)", "langchain-community (>=0.3.25)", "langchain-core (>=0.3.65)", "langchain-openai (>=0.3.22)", "langgraph (>=0.4.8)", "mock (>=2.0.0)", "openai", "parameterized (>=0.8.1)", "pydantic", "pytest", "pytest-asyncio", "pytest-timeout"]
test = ["anthropic (>=0.72)", "coverage", "django", "freezegun (==1.5.1)", "google-genai", "langchain-anthropic (>=1.0)", "langchain-community (>=0.4)", "langchain-core (>=1.0)", "langchain-openai (>=1.0)", "langgraph (>=1.0)", "mock (>=2.0.0)", "openai (>=2.0)", "parameterized (>=0.8.1)", "pydantic", "pytest", "pytest-asyncio", "pytest-timeout"]
[[package]]
name = "pre-commit"
@@ -14885,4 +14851,4 @@ cffi = ["cffi (>=1.17,<2.0) ; platform_python_implementation != \"PyPy\" and pyt
[metadata]
lock-version = "2.1"
python-versions = "^3.12,<3.14"
content-hash = "ef037f6d6085d26166d35c56ce266439f8f1a4fea90bc43ccf15cfeaf116cae5"
content-hash = "dd37c02db2f6940f625a931c01902daee227ad463efe20f1ac83cd761646b41e"

View File

@@ -36,7 +36,7 @@ resend = "^2.7.0"
tenacity = "^9.1.2"
slack-sdk = "^3.35.0"
ddtrace = "3.13.0" #pin to avoid yanked version 3.12.4
posthog = "^6.0.0"
posthog = "^7.0.0"
limits = "^5.2.0"
coredis = "^4.22.0"
httpx = "*"

View File

@@ -12,6 +12,9 @@ import socketio # noqa: E402
from fastapi import Request, status # noqa: E402
from fastapi.middleware.cors import CORSMiddleware # noqa: E402
from fastapi.responses import JSONResponse # noqa: E402
from server.app_lifespan.saas_app_lifespan_service import ( # noqa: E402
SaasAppLifespanService,
)
from server.auth.auth_error import ExpiredError, NoCredentialsError # noqa: E402
from server.auth.constants import ( # noqa: E402
ENABLE_JIRA,
@@ -22,7 +25,10 @@ from server.auth.constants import ( # noqa: E402
)
from server.constants import PERMITTED_CORS_ORIGINS # noqa: E402
from server.logger import logger # noqa: E402
from server.middleware import SetAuthCookieMiddleware # noqa: E402
from server.middleware import ( # noqa: E402
PostHogSessionMiddleware,
SetAuthCookieMiddleware,
)
from server.rate_limit import setup_rate_limit_handler # noqa: E402
from server.routes.api_keys import api_router as api_keys_router # noqa: E402
from server.routes.auth import api_router, oauth_router # noqa: E402
@@ -37,6 +43,7 @@ from server.routes.integration.linear import linear_integration_router # noqa:
from server.routes.integration.slack import slack_router # noqa: E402
from server.routes.mcp_patch import patch_mcp_server # noqa: E402
from server.routes.oauth_device import oauth_device_router # noqa: E402
from server.routes.onboarding import onboarding_router # noqa: E402
from server.routes.org_invitations import ( # noqa: E402
accept_router as invitation_accept_router,
)
@@ -60,6 +67,14 @@ from server.verified_models.verified_model_router import ( # noqa: E402
override_llm_models_dependency,
)
# Patch global config with SaaS lifespan BEFORE openhands.server.app is imported.
# app.py reads get_app_lifespan_service() at module level (line ~69), so this
# must execute first.
from openhands.app_server.config import get_global_config # noqa: E402
_config = get_global_config()
_config.lifespan = SaasAppLifespanService()
from openhands.server.app import app as base_app # noqa: E402
from openhands.server.listen_socket import sio # noqa: E402
from openhands.server.middleware import ( # noqa: E402
@@ -132,6 +147,7 @@ if ENABLE_LINEAR:
base_app.include_router(linear_integration_router)
base_app.include_router(email_router) # Add routes for email management
base_app.include_router(feedback_router) # Add routes for conversation feedback
base_app.include_router(onboarding_router) # Add route for onboarding submission
base_app.include_router(
event_webhook_router
) # Add routes for Events in nested runtimes
@@ -145,6 +161,7 @@ base_app.add_middleware(
allow_headers=['*'],
)
base_app.add_middleware(CacheControlMiddleware)
base_app.middleware('http')(PostHogSessionMiddleware())
base_app.middleware('http')(SetAuthCookieMiddleware())
base_app.mount('/', SPAStaticFiles(directory=directory, html=True), name='dist')

View File

@@ -0,0 +1,46 @@
"""SaaS-specific application lifespan service.
Initializes PostHog analytics on startup and flushes buffered events on
clean shutdown so no events are lost when the server exits gracefully.
"""
from __future__ import annotations
import os
from server.constants import IS_FEATURE_ENV
from openhands.analytics import get_analytics_service, init_analytics_service
from openhands.app_server.app_lifespan.app_lifespan_service import AppLifespanService
from openhands.core.logger import openhands_logger as logger
from openhands.server.types import AppMode
class SaasAppLifespanService(AppLifespanService):
"""Lifespan service for the SaaS server.
On enter: initialises the PostHog analytics singleton from environment vars.
On exit: calls ``analytics_service.shutdown()`` to flush any buffered events.
"""
async def __aenter__(self):
api_key = os.environ.get('POSTHOG_CLIENT_KEY', '')
host = os.environ.get('POSTHOG_HOST', 'https://us.i.posthog.com')
config_cls = os.environ.get('OPENHANDS_CONFIG_CLS', '')
app_mode = AppMode.SAAS if 'saas' in config_cls.lower() else AppMode.OPENHANDS
init_analytics_service(
api_key=api_key,
host=host,
app_mode=app_mode,
is_feature_env=IS_FEATURE_ENV,
)
return self
async def __aexit__(self, exc_type, exc_value, traceback):
try:
svc = get_analytics_service()
if svc is not None:
svc.shutdown()
except Exception:
logger.exception('Error shutting down analytics service')

View File

@@ -197,3 +197,19 @@ class SetAuthCookieMiddleware:
await token_manager.logout(user_auth.refresh_token.get_secret_value())
except Exception:
logger.debug('Error logging out')
class PostHogSessionMiddleware:
"""Extract the PostHog session ID from the incoming request header.
Stores the value on ``request.state.posthog_session_id`` so that
subsequent event-capture call sites can link server-side events to the
corresponding frontend session-replay recording.
When the ``X-POSTHOG-SESSION-ID`` header is absent the attribute is set
to ``None`` — never raises, never blocks.
"""
async def __call__(self, request: Request, call_next: Callable):
request.state.posthog_session_id = request.headers.get('X-POSTHOG-SESSION-ID')
return await call_next(request)

View File

@@ -7,7 +7,6 @@ from typing import Annotated, Literal, Optional, cast
from urllib.parse import quote
from uuid import UUID as parse_uuid
import posthog
from fastapi import APIRouter, Header, HTTPException, Request, Response, status
from fastapi.responses import JSONResponse, RedirectResponse
from pydantic import SecretStr
@@ -39,6 +38,7 @@ from storage.database import a_session_maker
from storage.user import User
from storage.user_store import UserStore
from openhands.analytics import analytics_constants, get_analytics_service
from openhands.core.logger import openhands_logger as logger
from openhands.integrations.provider import ProviderHandler
from openhands.integrations.service_types import ProviderType, TokenResponse
@@ -149,6 +149,35 @@ def _extract_recaptcha_state(state: str | None) -> tuple[str, str | None]:
return redirect_url, recaptcha_token
async def _get_user_orgs_with_data(user_id: str, org_member_ids: list) -> list:
"""Load Org objects for a user's org memberships.
Uses org_member.org_id list to batch-load Org objects, avoiding N+1
by loading all orgs a user belongs to in one query via OrgStore.
Args:
user_id: The user's ID string
org_member_ids: List of org_id UUIDs from user.org_members
Returns:
List of Org objects the user belongs to
"""
from storage.org_store import OrgStore
orgs = []
for org_id in org_member_ids:
try:
org = await OrgStore.get_org_by_id(org_id)
if org:
orgs.append(org)
except Exception:
logger.exception(
'auth:_get_user_orgs_with_data:failed',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
return orgs
@oauth_router.get('/keycloak/callback')
async def keycloak_callback(
request: Request,
@@ -204,9 +233,11 @@ async def keycloak_callback(
email = user_info.email
user_id = user_info.sub
user_info_dict = user_info.model_dump(exclude_none=True)
is_new_user = False
user = await UserStore.get_user_by_id(user_id)
if not user:
user = await UserStore.create_user(user_id, user_info_dict)
is_new_user = True
else:
# Existing user — gradually backfill contact_name if it still has a username-style value
await UserStore.backfill_contact_name(user_id, user_info_dict)
@@ -223,6 +254,37 @@ async def keycloak_callback(
logger.info(f'Logging in user {str(user.id)} in org {user.current_org_id}')
# Analytics: user signed up event (fires only for new users, once per user)
if is_new_user:
try:
analytics = get_analytics_service()
if analytics:
consented = (
user.user_consents_to_analytics is True
) # None = undecided = not consented
analytics.capture(
distinct_id=user_id,
event=analytics_constants.USER_SIGNED_UP,
properties={
'idp': user_info.identity_provider or 'keycloak',
'email_domain': email.split('@')[1]
if email and '@' in email
else None,
'invitation_source': 'invitation'
if invitation_token
else 'self_signup',
},
org_id=str(user.current_org_id) if user.current_org_id else None,
consented=consented,
)
analytics.set_person_properties(
distinct_id=user_id,
properties={'signed_up_at': datetime.now(timezone.utc).isoformat()},
consented=consented,
)
except Exception:
logger.exception('analytics:user_signed_up:failed')
# reCAPTCHA verification with Account Defender
if RECAPTCHA_SITE_KEY:
if not recaptcha_token:
@@ -370,36 +432,87 @@ async def keycloak_callback(
f'keycloakAccessToken: {keycloak_access_token}, keycloakUserId: {user_id}'
)
# adding in posthog tracking
# Server-side identity — full person and org group tracking via AnalyticsService
analytics = get_analytics_service()
if analytics:
consented = (
user.user_consents_to_analytics is True
) # None = undecided = not consented
# If this is a feature environment, add "FEATURE_" prefix to user_id for PostHog
posthog_user_id = f'FEATURE_{user_id}' if IS_FEATURE_ENV else user_id
# Load current org for person properties
from storage.org_store import OrgStore
try:
posthog.set(
distinct_id=posthog_user_id,
current_org = (
await OrgStore.get_org_by_id(user.current_org_id)
if user.current_org_id
else None
)
# Set person properties (SaaS only, consent-gated inside service)
analytics.set_person_properties(
distinct_id=user_id,
properties={
'user_id': posthog_user_id,
'original_user_id': user_id,
'is_feature_env': IS_FEATURE_ENV,
'email': email,
'org_id': str(user.current_org_id) if user.current_org_id else None,
'org_name': current_org.name if current_org else None,
'plan_tier': None, # plan_tier not yet on Org model — deferred to future phase
'created_at': str(user.accepted_tos)
if hasattr(user, 'accepted_tos') and user.accepted_tos
else None,
'idp': idp,
'last_login_at': datetime.now(timezone.utc).isoformat(),
},
consented=consented,
)
except Exception as e:
logger.error(
'auth:posthog_set:failed',
extra={
'user_id': user_id,
'error': str(e),
},
# Group identify for all orgs the user belongs to
# user.org_members is eagerly loaded via joinedload in UserStore.get_user_by_id
org_member_ids = (
[om.org_id for om in user.org_members] if user.org_members else []
)
user_orgs = await _get_user_orgs_with_data(user_id, org_member_ids)
from storage.org_member_store import OrgMemberStore
for org in user_orgs:
try:
member_count = await OrgMemberStore.get_org_members_count(org_id=org.id)
except Exception:
logger.exception(
'auth:group_identify:member_count_failed',
extra={'user_id': user_id, 'org_id': str(org.id)},
)
member_count = None
analytics.group_identify(
group_type='org',
group_key=str(org.id),
properties={
'org_name': org.name,
'plan_tier': None, # plan_tier not yet on Org model — deferred to future phase
'created_at': None, # created_at not yet on Org model — deferred to future phase
'member_count': member_count,
# credit_balance: deferred to Phase 2 (requires billing infrastructure)
},
distinct_id=user_id,
consented=consented,
)
# Capture login event
analytics.capture(
distinct_id=user_id,
event='user logged in',
properties={'idp': idp},
org_id=str(user.current_org_id) if user.current_org_id else None,
consented=consented,
)
# Continue execution as this is not critical
logger.info(
'user_logged_in',
extra={
'idp': idp,
'idp_type': idp_type,
'posthog_user_id': posthog_user_id,
'user_id': user_id,
'is_feature_env': IS_FEATURE_ENV,
},
)

View File

@@ -20,6 +20,7 @@ from storage.org import Org
from storage.subscription_access import SubscriptionAccess
from storage.user_store import UserStore
from openhands.analytics import analytics_constants, get_analytics_service
from openhands.app_server.config import get_global_config
from openhands.server.user_auth import get_user_id
@@ -28,9 +29,7 @@ billing_router = APIRouter(prefix='/api/billing', tags=['Billing'])
async def validate_billing_enabled() -> None:
"""
Validate that the billing feature flag is enabled
"""
"""Validate that the billing feature flag is enabled"""
config = get_global_config()
web_client_config = await config.web_client.get_web_client_config()
if not web_client_config.feature_flags.enable_billing:
@@ -299,6 +298,27 @@ async def success_callback(session_id: str, request: Request):
)
await session.commit()
# Analytics: credit purchased event (fires after commit so event only fires on success)
try:
analytics = get_analytics_service()
if analytics and user:
consented = (
user.user_consents_to_analytics is True
) # None = undecided = not consented
analytics.capture(
distinct_id=billing_session.user_id,
event=analytics_constants.CREDIT_PURCHASED,
properties={
'amount_usd': add_credits,
'credit_balance_before': max_budget,
'credit_balance_after': new_max_budget,
},
org_id=str(user.current_org_id) if user.current_org_id else None,
consented=consented,
)
except Exception:
logger.exception('analytics:credit_purchased:failed')
return RedirectResponse(
f'{_get_base_url(request)}settings/billing?checkout=success', status_code=302
)

View File

@@ -9,6 +9,7 @@ from pydantic import BaseModel
from storage.api_key_store import ApiKeyStore
from storage.device_code_store import DeviceCodeStore
from openhands.analytics import get_analytics_service
from openhands.core.logger import openhands_logger as logger
from openhands.server.user_auth import get_user_id
@@ -310,6 +311,54 @@ async def device_verification_authenticated(
'Device code authorized with API key successfully',
extra={'user_code': user_code, 'user_id': user_id},
)
# Server-side identity tracking for device auth flow
analytics = get_analytics_service()
if analytics:
try:
from storage.user_store import UserStore
user = await UserStore.get_user_by_id(user_id)
if user:
consented = user.user_consents_to_analytics is True
# Set person properties — same pattern as keycloak_callback
from storage.org_store import OrgStore
current_org = (
await OrgStore.get_org_by_id(user.current_org_id)
if user.current_org_id
else None
)
analytics.set_person_properties(
distinct_id=user_id,
properties={
'org_id': str(user.current_org_id)
if user.current_org_id
else None,
'org_name': current_org.name if current_org else None,
'idp': 'device_auth',
},
consented=consented,
)
# Capture login event for device auth
analytics.capture(
distinct_id=user_id,
event='user logged in',
properties={'idp': 'device_auth'},
org_id=str(user.current_org_id)
if user.current_org_id
else None,
consented=consented,
)
except Exception:
logger.exception(
'oauth_device:analytics:failed',
extra={'user_id': user_id},
)
return JSONResponse(
status_code=status.HTTP_200_OK,
content={'message': 'Device authorized successfully!'},

View File

@@ -0,0 +1,76 @@
"""Onboarding submission endpoint.
Receives user onboarding selections and fires analytics event.
"""
from datetime import datetime, timezone
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from openhands.server.user_auth import get_user_id
onboarding_router = APIRouter(prefix='/api', tags=['Onboarding'])
class OnboardingSubmission(BaseModel):
selections: dict[
str, str
] # step_id -> option_id (e.g., {"step1": "software_engineer", "step2": "solo", "step3": "new_features"})
class OnboardingResponse(BaseModel):
status: str
redirect_url: str
@onboarding_router.post('/onboarding', response_model=OnboardingResponse)
async def submit_onboarding(
body: OnboardingSubmission,
user_id: str | None = Depends(get_user_id),
) -> OnboardingResponse:
"""Submit onboarding form selections and fire analytics event."""
# ACTV-03: onboarding completed
try:
from openhands.analytics import analytics_constants, get_analytics_service
analytics = get_analytics_service()
if analytics and user_id:
from enterprise.storage.user_store import UserStore
user_obj = await UserStore.get_user_by_id(user_id)
if user_obj:
consented = user_obj.user_consents_to_analytics is True
org_id_str = (
str(user_obj.current_org_id) if user_obj.current_org_id else None
)
analytics.capture(
distinct_id=user_id,
event=analytics_constants.ONBOARDING_COMPLETED,
properties={
'role': body.selections.get('step1'),
'org_size': body.selections.get('step2'),
'use_case': body.selections.get('step3'),
},
org_id=org_id_str,
consented=consented,
)
# Associate onboarding timestamp with org group
if org_id_str:
analytics.group_identify(
group_type='org',
group_key=org_id_str,
properties={
'onboarding_completed_at': datetime.now(
timezone.utc
).isoformat(),
},
distinct_id=user_id,
consented=consented,
)
except Exception:
import logging
logging.getLogger(__name__).exception('analytics:onboarding_completed:failed')
return OnboardingResponse(status='ok', redirect_url='/')

View File

@@ -46,6 +46,7 @@ from server.services.org_member_service import OrgMemberService
from storage.org_service import OrgService
from storage.user_store import UserStore
from openhands.analytics import get_analytics_service
from openhands.core.logger import openhands_logger as logger
from openhands.server.user_auth import get_user_id
@@ -999,6 +1000,27 @@ async def switch_org(
org_id=org_id,
)
# Refresh person profile with new active org on org switch
analytics = get_analytics_service()
if analytics:
try:
user = await UserStore.get_user_by_id(user_id)
consented = user.user_consents_to_analytics is True if user else False
analytics.set_person_properties(
distinct_id=user_id,
properties={
'org_id': str(org_id),
'org_name': org.name,
'plan_tier': None, # plan_tier not yet on Org model — deferred to future phase
},
consented=consented,
)
except Exception:
logger.exception(
'orgs:switch_org:analytics:failed',
extra={'user_id': user_id, 'org_id': str(org_id)},
)
# Retrieve credits from LiteLLM for the new current org
credits = await OrgService.get_org_credits(user_id, org.id)

View File

@@ -14,6 +14,7 @@ from storage.conversation_work import ConversationWork
from storage.database import a_session_maker, session_maker
from storage.stored_conversation_metadata import StoredConversationMetadata
from openhands.analytics import analytics_constants, get_analytics_service
from openhands.core.config import load_openhands_config
from openhands.core.schema.agent import AgentState
from openhands.events.event_store import EventStore
@@ -31,6 +32,15 @@ from openhands.utils.async_utils import call_sync_from_async
config = load_openhands_config()
file_store = get_file_store(config.file_store, config.file_store_path)
# V0 terminal state sets for analytics
_TERMINAL_ERROR_STATES = {AgentState.ERROR}
_TERMINAL_FINISHED_STATES = {
AgentState.FINISHED,
AgentState.STOPPED,
AgentState.AWAITING_USER_INPUT,
}
_ALL_TERMINAL_STATES = _TERMINAL_ERROR_STATES | _TERMINAL_FINISHED_STATES
async def process_event(
user_id: str, conversation_id: str, subpath: str, content: dict
@@ -62,6 +72,140 @@ async def process_event(
# Load and invoke all active callbacks for this conversation
await invoke_conversation_callbacks(conversation_id, event)
# V0 best-effort analytics for terminal states
if event.agent_state in _ALL_TERMINAL_STATES:
try:
analytics = get_analytics_service()
if analytics and user_id:
from enterprise.storage.user_store import UserStore
user_obj = await UserStore.get_user_by_id(user_id)
if user_obj:
consented = user_obj.user_consents_to_analytics is True
org_id_str = (
str(user_obj.current_org_id)
if user_obj.current_org_id
else None
)
# Look up conversation metadata for cost/token data
with session_maker() as meta_session:
conv_meta = (
meta_session.query(StoredConversationMetadata)
.filter(
StoredConversationMetadata.conversation_id
== conversation_id
)
.first()
)
if event.agent_state in _TERMINAL_ERROR_STATES:
analytics.capture(
distinct_id=user_id,
event=analytics_constants.CONVERSATION_ERRORED,
properties={
'conversation_id': conversation_id,
'error_type': 'unknown', # V0: error classification not available from AgentState alone
'error_message': None,
'llm_model': conv_meta.llm_model
if conv_meta and hasattr(conv_meta, 'llm_model')
else None,
'turn_count': None,
'terminal_state': event.agent_state.value
if hasattr(event.agent_state, 'value')
else str(event.agent_state),
},
org_id=org_id_str,
consented=consented,
)
else:
analytics.capture(
distinct_id=user_id,
event=analytics_constants.CONVERSATION_FINISHED,
properties={
'conversation_id': conversation_id,
'terminal_state': event.agent_state.value
if hasattr(event.agent_state, 'value')
else str(event.agent_state),
'turn_count': None,
'accumulated_cost_usd': conv_meta.accumulated_cost
if conv_meta
else None,
'prompt_tokens': conv_meta.prompt_tokens
if conv_meta
else None,
'completion_tokens': conv_meta.completion_tokens
if conv_meta
else None,
'llm_model': conv_meta.llm_model
if conv_meta and hasattr(conv_meta, 'llm_model')
else None,
'trigger': None, # V0: trigger not available in callback context
},
org_id=org_id_str,
consented=consented,
)
# ACTV-01: user activated (first finished conversation only)
if event.agent_state == AgentState.FINISHED:
try:
import uuid as _uuid
from datetime import timezone
from sqlalchemy import func
from sqlalchemy import select as sa_select
from storage.stored_conversation_metadata_saas import (
StoredConversationMetadataSaas,
)
user_uuid = _uuid.UUID(user_id)
with session_maker() as act_session:
count_result = act_session.execute(
sa_select(func.count()).where(
StoredConversationMetadataSaas.user_id
== user_uuid,
StoredConversationMetadataSaas.conversation_id
!= conversation_id,
)
)
prior_count = count_result.scalar()
if prior_count == 0:
tos_ts = user_obj.accepted_tos
if tos_ts is not None:
if tos_ts.tzinfo is None:
tos_ts = tos_ts.replace(
tzinfo=timezone.utc
)
from datetime import datetime
time_to_activate_seconds = (
datetime.now(timezone.utc) - tos_ts
).total_seconds()
else:
time_to_activate_seconds = None
analytics.capture(
distinct_id=user_id,
event=analytics_constants.USER_ACTIVATED,
properties={
'conversation_id': conversation_id,
'time_to_activate_seconds': time_to_activate_seconds,
'llm_model': conv_meta.llm_model
if conv_meta
else None,
'trigger': None, # V0: trigger not available in callback context
},
org_id=org_id_str,
consented=consented,
)
except Exception:
logger.exception(
'analytics:user_activated:v0:failed'
)
except Exception:
logger.exception('analytics:v0_terminal_state:failed')
# Update active working seconds if agent state is not Running
if event.agent_state != AgentState.RUNNING:
event_store = EventStore(conversation_id, file_store, user_id)

View File

@@ -1,6 +1,7 @@
"""Store class for managing users."""
import asyncio
import os
import uuid
from typing import Optional
from uuid import UUID
@@ -81,6 +82,9 @@ class UserStore:
)
user.email = user_info.get('email')
user.email_verified = user_info.get('email_verified')
# SaaS consent is implicit via Terms of Service — new SaaS users default to consented
if 'saas' in (os.environ.get('OPENHANDS_CONFIG_CLS', '')).lower():
user.user_consents_to_analytics = True
session.add(user)
role = await RoleStore.get_role_by_name('owner')

View File

@@ -188,7 +188,10 @@ async def test_keycloak_callback_success_with_valid_offline_token(
patch('server.routes.auth.user_verifier') as mock_verifier,
patch('server.routes.auth.set_response_cookie') as mock_set_cookie,
patch('server.routes.auth.UserStore') as mock_user_store,
patch('server.routes.auth.posthog') as mock_posthog,
patch('server.routes.auth.get_analytics_service') as mock_posthog,
patch(
'storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock
) as mock_get_org,
):
# Mock user with accepted_tos
mock_user = MagicMock()
@@ -239,7 +242,7 @@ async def test_keycloak_callback_success_with_valid_offline_token(
secure=False,
accepted_tos=True,
)
mock_posthog.set.assert_called_once()
mock_posthog.return_value.set_person_properties.assert_called()
@pytest.mark.asyncio
@@ -359,7 +362,10 @@ async def test_keycloak_callback_success_without_offline_token(
patch('server.routes.auth.KEYCLOAK_REALM_NAME', 'test-realm'),
patch('server.routes.auth.KEYCLOAK_CLIENT_ID', 'test-client'),
patch('server.routes.auth.UserStore') as mock_user_store,
patch('server.routes.auth.posthog') as mock_posthog,
patch('server.routes.auth.get_analytics_service') as mock_posthog,
patch(
'storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock
) as mock_get_org,
):
# Mock user with accepted_tos
mock_user = MagicMock()
@@ -413,7 +419,7 @@ async def test_keycloak_callback_success_without_offline_token(
secure=False,
accepted_tos=True,
)
mock_posthog.set.assert_called_once()
mock_posthog.return_value.set_person_properties.assert_called()
@pytest.mark.asyncio
@@ -1218,7 +1224,8 @@ class TestKeycloakCallbackRecaptcha:
patch('server.routes.auth.a_session_maker') as mock_session_maker,
patch('server.routes.auth.domain_blocker') as mock_domain_blocker,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.posthog'),
patch('server.routes.auth.get_analytics_service'),
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
patch('server.routes.email.verify_email', new_callable=AsyncMock),
patch('server.routes.auth.UserStore') as mock_user_store,
):
@@ -1372,7 +1379,8 @@ class TestKeycloakCallbackRecaptcha:
patch('server.routes.auth.user_verifier') as mock_verifier,
patch('server.routes.auth.a_session_maker') as mock_session_maker,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.posthog'),
patch('server.routes.auth.get_analytics_service'),
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
patch('server.routes.email.verify_email', new_callable=AsyncMock),
patch('server.routes.auth.UserStore') as mock_user_store,
):
@@ -1461,7 +1469,8 @@ class TestKeycloakCallbackRecaptcha:
patch('server.routes.auth.user_verifier') as mock_verifier,
patch('server.routes.auth.a_session_maker') as mock_session_maker,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.posthog'),
patch('server.routes.auth.get_analytics_service'),
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
patch('server.routes.email.verify_email', new_callable=AsyncMock),
patch('server.routes.auth.UserStore') as mock_user_store,
):
@@ -1549,7 +1558,8 @@ class TestKeycloakCallbackRecaptcha:
patch('server.routes.auth.user_verifier') as mock_verifier,
patch('server.routes.auth.a_session_maker') as mock_session_maker,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.posthog'),
patch('server.routes.auth.get_analytics_service'),
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
patch('server.routes.email.verify_email', new_callable=AsyncMock),
patch('server.routes.auth.UserStore') as mock_user_store,
):
@@ -1634,7 +1644,8 @@ class TestKeycloakCallbackRecaptcha:
patch('server.routes.auth.user_verifier') as mock_verifier,
patch('server.routes.auth.a_session_maker') as mock_session_maker,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.posthog'),
patch('server.routes.auth.get_analytics_service'),
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
patch('server.routes.email.verify_email', new_callable=AsyncMock),
patch('server.routes.auth.UserStore') as mock_user_store,
):
@@ -1716,7 +1727,8 @@ class TestKeycloakCallbackRecaptcha:
patch('server.routes.auth.a_session_maker') as mock_session_maker,
patch('server.routes.auth.domain_blocker') as mock_domain_blocker,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.posthog'),
patch('server.routes.auth.get_analytics_service'),
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
patch('server.routes.email.verify_email', new_callable=AsyncMock),
patch('server.routes.auth.UserStore') as mock_user_store,
):
@@ -1786,7 +1798,8 @@ class TestKeycloakCallbackRecaptcha:
patch('server.routes.auth.a_session_maker') as mock_session_maker,
patch('server.routes.auth.domain_blocker') as mock_domain_blocker,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.posthog'),
patch('server.routes.auth.get_analytics_service'),
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
patch('server.routes.email.verify_email', new_callable=AsyncMock),
patch('server.routes.auth.UserStore') as mock_user_store,
):
@@ -1860,7 +1873,8 @@ class TestKeycloakCallbackRecaptcha:
patch('server.routes.auth.a_session_maker') as mock_session_maker,
patch('server.routes.auth.domain_blocker') as mock_domain_blocker,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.posthog'),
patch('server.routes.auth.get_analytics_service'),
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
patch('server.routes.auth.logger') as mock_logger,
patch('server.routes.auth.UserStore') as mock_user_store,
):
@@ -2013,7 +2027,8 @@ async def test_keycloak_callback_calls_backfill_user_email_for_existing_user(
patch('server.routes.auth.user_verifier') as mock_verifier,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.UserStore') as mock_user_store,
patch('server.routes.auth.posthog'),
patch('server.routes.auth.get_analytics_service'),
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
):
mock_user = MagicMock()
mock_user.id = 'test_user_id'
@@ -2094,3 +2109,165 @@ async def test_accept_tos_stores_timezone_naive_datetime(mock_request):
# The datetime assigned to user.accepted_tos must be timezone-naive
# (compatible with TIMESTAMP WITHOUT TIME ZONE database column)
assert mock_user.accepted_tos.tzinfo is None
@pytest.mark.asyncio
async def test_keycloak_callback_new_user_analytics_event(
mock_request, create_keycloak_user_info
):
"""Test that user signup analytics event correctly accesses KeycloakUserInfo attributes.
This test verifies the fix for the AttributeError that occurred when trying to call
.get() on a Pydantic model. The analytics code should use direct attribute access
(user_info.identity_provider) instead of dict-style .get() method.
Fixes: https://github.com/All-Hands-AI/OpenHands/issues/13243
"""
# Create a KeycloakUserInfo model (Pydantic) with identity_provider set
user_info = create_keycloak_user_info(
sub='test_new_user_id',
preferred_username='new_user',
email='newuser@example.com',
email_verified=True,
identity_provider='github',
)
with (
patch('server.routes.auth.token_manager') as mock_token_manager,
patch('server.routes.auth.user_verifier') as mock_verifier,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.UserStore') as mock_user_store,
patch('server.routes.auth.get_analytics_service') as mock_get_analytics,
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
):
# Mock analytics service
mock_analytics = MagicMock()
mock_get_analytics.return_value = mock_analytics
# Mock a new user (get_user_by_id returns None)
mock_user = MagicMock()
mock_user.id = 'test_new_user_id'
mock_user.current_org_id = 'test_org_id'
mock_user.accepted_tos = '2025-01-01'
mock_user.user_consents_to_analytics = True
mock_user_store.get_user_by_id = AsyncMock(return_value=None) # New user
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_user_store.backfill_user_email = AsyncMock()
mock_token_manager.get_keycloak_tokens = AsyncMock(
return_value=('test_access_token', 'test_refresh_token')
)
mock_token_manager.get_user_info = AsyncMock(return_value=user_info)
mock_token_manager.store_idp_tokens = AsyncMock()
mock_token_manager.validate_offline_token = AsyncMock(return_value=True)
mock_token_manager.check_duplicate_base_email = AsyncMock(return_value=False)
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
# Act - This would have raised AttributeError before the fix
result = await keycloak_callback(
code='test_code', state='test_state', request=mock_request
)
# Assert - Callback should succeed
assert isinstance(result, RedirectResponse)
assert result.status_code == 302
# Verify analytics.capture was called (may be called multiple times for signup + login)
assert mock_analytics.capture.call_count >= 1
# Find the 'user signed up' event call
signup_call = None
for call in mock_analytics.capture.call_args_list:
if call.kwargs.get('event') == 'user signed up':
signup_call = call
break
assert signup_call is not None, "Expected 'user signed up' analytics event"
# Check that identity_provider was correctly extracted from Pydantic model
properties = signup_call.kwargs['properties']
assert properties['idp'] == 'github'
assert properties['email_domain'] == 'example.com'
assert properties['invitation_source'] == 'self_signup'
@pytest.mark.asyncio
async def test_keycloak_callback_new_user_analytics_fallback_idp(
mock_request, create_keycloak_user_info
):
"""Test that analytics event uses 'keycloak' fallback when identity_provider is None.
This verifies the fallback behavior of 'user_info.identity_provider or 'keycloak''.
"""
# Create a KeycloakUserInfo model without identity_provider
user_info = create_keycloak_user_info(
sub='test_new_user_id',
preferred_username='new_user',
email='newuser@example.com',
email_verified=True,
identity_provider=None, # No identity provider
)
with (
patch('server.routes.auth.token_manager') as mock_token_manager,
patch('server.routes.auth.user_verifier') as mock_verifier,
patch('server.routes.auth.set_response_cookie'),
patch('server.routes.auth.UserStore') as mock_user_store,
patch('server.routes.auth.get_analytics_service') as mock_get_analytics,
patch('storage.org_store.OrgStore.get_org_by_id', new_callable=AsyncMock),
):
# Mock analytics service
mock_analytics = MagicMock()
mock_get_analytics.return_value = mock_analytics
# Mock a new user
mock_user = MagicMock()
mock_user.id = 'test_new_user_id'
mock_user.current_org_id = 'test_org_id'
mock_user.accepted_tos = '2025-01-01'
mock_user.user_consents_to_analytics = True
mock_user_store.get_user_by_id = AsyncMock(return_value=None) # New user
mock_user_store.create_user = AsyncMock(return_value=mock_user)
mock_user_store.backfill_contact_name = AsyncMock()
mock_user_store.backfill_user_email = AsyncMock()
mock_token_manager.get_keycloak_tokens = AsyncMock(
return_value=('test_access_token', 'test_refresh_token')
)
mock_token_manager.get_user_info = AsyncMock(return_value=user_info)
mock_token_manager.store_idp_tokens = AsyncMock()
mock_token_manager.validate_offline_token = AsyncMock(return_value=True)
mock_token_manager.check_duplicate_base_email = AsyncMock(return_value=False)
mock_verifier.is_active.return_value = True
mock_verifier.is_user_allowed.return_value = True
# Act
result = await keycloak_callback(
code='test_code', state='test_state', request=mock_request
)
# Assert
assert isinstance(result, RedirectResponse)
assert result.status_code == 302
# Verify analytics.capture was called (may be called multiple times)
assert mock_analytics.capture.call_count >= 1
# Find the 'user signed up' event call
signup_call = None
for call in mock_analytics.capture.call_args_list:
if call.kwargs.get('event') == 'user signed up':
signup_call = call
break
assert signup_call is not None, "Expected 'user signed up' analytics event"
# Check that fallback 'keycloak' was used when identity_provider is None
properties = signup_call.kwargs['properties']
assert properties['idp'] == 'keycloak'

View File

@@ -0,0 +1,97 @@
"""Tests for PostHogSessionMiddleware."""
from unittest.mock import AsyncMock, MagicMock
import pytest
from fastapi import Request, Response
@pytest.fixture
def mock_response():
return MagicMock(spec=Response)
def make_mock_request(headers: dict | None = None):
"""Create a mock FastAPI Request with a state object and headers dict."""
request = MagicMock(spec=Request)
request.headers = headers or {}
request.state = MagicMock()
return request
@pytest.mark.asyncio
async def test_middleware_sets_session_id_from_header(mock_response):
"""PostHogSessionMiddleware sets posthog_session_id from X-POSTHOG-SESSION-ID header."""
from server.middleware import PostHogSessionMiddleware
session_id = 'sess_abc123'
request = make_mock_request({'X-POSTHOG-SESSION-ID': session_id})
call_next = AsyncMock(return_value=mock_response)
middleware = PostHogSessionMiddleware()
result = await middleware(request, call_next)
assert request.state.posthog_session_id == session_id
call_next.assert_called_once_with(request)
assert result == mock_response
@pytest.mark.asyncio
async def test_middleware_sets_none_when_header_absent(mock_response):
"""PostHogSessionMiddleware sets posthog_session_id to None when header is absent."""
from server.middleware import PostHogSessionMiddleware
request = make_mock_request({}) # No X-POSTHOG-SESSION-ID header
call_next = AsyncMock(return_value=mock_response)
middleware = PostHogSessionMiddleware()
result = await middleware(request, call_next)
assert request.state.posthog_session_id is None
call_next.assert_called_once_with(request)
assert result == mock_response
@pytest.mark.asyncio
async def test_middleware_does_not_modify_response(mock_response):
"""PostHogSessionMiddleware returns the response unchanged."""
from server.middleware import PostHogSessionMiddleware
request = make_mock_request({'X-POSTHOG-SESSION-ID': 'sess_xyz'})
call_next = AsyncMock(return_value=mock_response)
middleware = PostHogSessionMiddleware()
result = await middleware(request, call_next)
assert result is mock_response
@pytest.mark.asyncio
async def test_middleware_does_not_block_request(mock_response):
"""PostHogSessionMiddleware always calls call_next (never blocks)."""
from server.middleware import PostHogSessionMiddleware
request = make_mock_request({})
call_next = AsyncMock(return_value=mock_response)
middleware = PostHogSessionMiddleware()
await middleware(request, call_next)
call_next.assert_called_once_with(request)
@pytest.mark.asyncio
async def test_middleware_handles_case_insensitive_header(mock_response):
"""PostHogSessionMiddleware uses .get() which handles header lookup."""
from server.middleware import PostHogSessionMiddleware
# FastAPI/Starlette Headers are case-insensitive, but we test with dict mock
# Test the exact header name used in the implementation
session_id = 'sess_case_test'
request = make_mock_request({'X-POSTHOG-SESSION-ID': session_id})
call_next = AsyncMock(return_value=mock_response)
middleware = PostHogSessionMiddleware()
await middleware(request, call_next)
assert request.state.posthog_session_id == session_id

View File

@@ -0,0 +1,108 @@
"""Tests for SaasAppLifespanService."""
from unittest.mock import MagicMock, patch
import pytest
@pytest.fixture
def mock_analytics_service():
svc = MagicMock()
svc.shutdown = MagicMock()
return svc
@pytest.mark.asyncio
async def test_aenter_calls_init_analytics_service():
"""SaasAppLifespanService.__aenter__ initializes the analytics service."""
from server.app_lifespan.saas_app_lifespan_service import SaasAppLifespanService
with patch(
'server.app_lifespan.saas_app_lifespan_service.init_analytics_service'
) as mock_init:
svc = SaasAppLifespanService()
await svc.__aenter__()
mock_init.assert_called_once()
@pytest.mark.asyncio
async def test_aenter_passes_env_vars_to_init():
"""SaasAppLifespanService reads config from env vars."""
from server.app_lifespan.saas_app_lifespan_service import SaasAppLifespanService
with (
patch(
'server.app_lifespan.saas_app_lifespan_service.init_analytics_service'
) as mock_init,
patch.dict(
'os.environ',
{
'POSTHOG_CLIENT_KEY': 'test-key',
'POSTHOG_HOST': 'https://test.posthog.com',
'OPENHANDS_CONFIG_CLS': 'enterprise.server.config.SaaSServerConfig',
},
),
):
svc = SaasAppLifespanService()
await svc.__aenter__()
call_kwargs = mock_init.call_args
assert call_kwargs.kwargs['api_key'] == 'test-key'
assert call_kwargs.kwargs['host'] == 'https://test.posthog.com'
@pytest.mark.asyncio
async def test_aexit_calls_shutdown_when_service_exists(mock_analytics_service):
"""SaasAppLifespanService.__aexit__ calls shutdown on the analytics service."""
from server.app_lifespan.saas_app_lifespan_service import SaasAppLifespanService
with (
patch('server.app_lifespan.saas_app_lifespan_service.init_analytics_service'),
patch(
'server.app_lifespan.saas_app_lifespan_service.get_analytics_service',
return_value=mock_analytics_service,
),
):
svc = SaasAppLifespanService()
await svc.__aenter__()
await svc.__aexit__(None, None, None)
mock_analytics_service.shutdown.assert_called_once()
@pytest.mark.asyncio
async def test_aexit_does_not_raise_when_service_is_none():
"""SaasAppLifespanService.__aexit__ does not raise if analytics service is None."""
from server.app_lifespan.saas_app_lifespan_service import SaasAppLifespanService
with (
patch('server.app_lifespan.saas_app_lifespan_service.init_analytics_service'),
patch(
'server.app_lifespan.saas_app_lifespan_service.get_analytics_service',
return_value=None,
),
):
svc = SaasAppLifespanService()
await svc.__aenter__()
# Must not raise
await svc.__aexit__(None, None, None)
@pytest.mark.asyncio
async def test_aexit_does_not_raise_on_shutdown_error(mock_analytics_service):
"""SaasAppLifespanService.__aexit__ swallows errors from shutdown."""
from server.app_lifespan.saas_app_lifespan_service import SaasAppLifespanService
mock_analytics_service.shutdown.side_effect = RuntimeError('connection closed')
with (
patch('server.app_lifespan.saas_app_lifespan_service.init_analytics_service'),
patch(
'server.app_lifespan.saas_app_lifespan_service.get_analytics_service',
return_value=mock_analytics_service,
),
):
svc = SaasAppLifespanService()
await svc.__aenter__()
# Must not raise even if shutdown errors
await svc.__aexit__(None, None, None)

View File

@@ -7,22 +7,6 @@ import { renderWithProviders } from "../../../test-utils";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { createMockWebClientConfig } from "../../helpers/mock-config";
const mockTrackAddTeamMembersButtonClick = vi.fn();
vi.mock("#/hooks/use-tracking", () => ({
useTracking: () => ({
trackAddTeamMembersButtonClick: mockTrackAddTeamMembersButtonClick,
}),
}));
// Mock posthog feature flag
vi.mock("posthog-js/react", () => ({
useFeatureFlagEnabled: vi.fn(),
}));
// Import the mocked module to get access to the mock
import * as posthog from "posthog-js/react";
describe("AccountSettingsContextMenu", () => {
const user = userEvent.setup();
const onClickAccountSettingsMock = vi.fn();
@@ -35,8 +19,6 @@ describe("AccountSettingsContextMenu", () => {
queryClient = new QueryClient({
defaultOptions: { queries: { retry: false } },
});
// Set default feature flag to false
vi.mocked(posthog.useFeatureFlagEnabled).mockReturnValue(false);
});
// Create a wrapper with MemoryRouter and renderWithProviders
@@ -67,8 +49,6 @@ describe("AccountSettingsContextMenu", () => {
onClickAccountSettingsMock.mockClear();
onLogoutMock.mockClear();
onCloseMock.mockClear();
mockTrackAddTeamMembersButtonClick.mockClear();
vi.mocked(posthog.useFeatureFlagEnabled).mockClear();
});
it("should always render the right options", () => {
@@ -142,73 +122,4 @@ describe("AccountSettingsContextMenu", () => {
expect(onCloseMock).toHaveBeenCalledOnce();
});
it("should show Add Team Members button in SaaS mode when feature flag is enabled", () => {
vi.mocked(posthog.useFeatureFlagEnabled).mockReturnValue(true);
renderWithSaasConfig(
<AccountSettingsContextMenu
onLogout={onLogoutMock}
onClose={onCloseMock}
/>,
);
expect(screen.getByTestId("add-team-members-button")).toBeInTheDocument();
expect(screen.getByText("SETTINGS$NAV_ADD_TEAM_MEMBERS")).toBeInTheDocument();
});
it("should not show Add Team Members button in SaaS mode when feature flag is disabled", () => {
vi.mocked(posthog.useFeatureFlagEnabled).mockReturnValue(false);
renderWithSaasConfig(
<AccountSettingsContextMenu
onLogout={onLogoutMock}
onClose={onCloseMock}
/>,
);
expect(screen.queryByTestId("add-team-members-button")).not.toBeInTheDocument();
expect(screen.queryByText("SETTINGS$NAV_ADD_TEAM_MEMBERS")).not.toBeInTheDocument();
});
it("should not show Add Team Members button in OSS mode even when feature flag is enabled", () => {
vi.mocked(posthog.useFeatureFlagEnabled).mockReturnValue(true);
renderWithOssConfig(
<AccountSettingsContextMenu
onLogout={onLogoutMock}
onClose={onCloseMock}
/>,
);
expect(screen.queryByTestId("add-team-members-button")).not.toBeInTheDocument();
expect(screen.queryByText("SETTINGS$NAV_ADD_TEAM_MEMBERS")).not.toBeInTheDocument();
});
it("should not show Add Team Members button when analytics consent is disabled", () => {
vi.mocked(posthog.useFeatureFlagEnabled).mockReturnValue(true);
renderWithSaasConfig(
<AccountSettingsContextMenu
onLogout={onLogoutMock}
onClose={onCloseMock}
/>,
{ analyticsConsent: false },
);
expect(screen.queryByTestId("add-team-members-button")).not.toBeInTheDocument();
expect(screen.queryByText("SETTINGS$NAV_ADD_TEAM_MEMBERS")).not.toBeInTheDocument();
});
it("should call tracking function and onClose when Add Team Members button is clicked", async () => {
vi.mocked(posthog.useFeatureFlagEnabled).mockReturnValue(true);
renderWithSaasConfig(
<AccountSettingsContextMenu
onLogout={onLogoutMock}
onClose={onCloseMock}
/>,
);
const addTeamMembersButton = screen.getByTestId("add-team-members-button");
await user.click(addTeamMembersButton);
expect(mockTrackAddTeamMembersButtonClick).toHaveBeenCalledOnce();
expect(onCloseMock).toHaveBeenCalledOnce();
});
});

View File

@@ -23,12 +23,6 @@ vi.mock("#/hooks/use-auth-url", () => ({
},
}));
vi.mock("#/hooks/use-tracking", () => ({
useTracking: () => ({
trackLoginButtonClick: vi.fn(),
}),
}));
vi.mock("#/hooks/query/use-config", () => ({
useConfig: () => ({
data: undefined,

View File

@@ -48,12 +48,6 @@ vi.mock("#/hooks/query/use-search-repositories", () => ({
useSearchRepositories: () => mockUseSearchRepositories(),
}));
vi.mock("#/hooks/use-tracking", () => ({
useTracking: () => ({
trackEvent: vi.fn(),
}),
}));
vi.mock("#/hooks/use-create-conversation-and-subscribe-multiple", () => ({
useCreateConversationAndSubscribeMultiple: () =>
mockUseCreateConversationAndSubscribeMultiple(),

View File

@@ -20,12 +20,6 @@ vi.mock("#/hooks/query/use-settings", async () => {
};
});
vi.mock("#/hooks/use-tracking", () => ({
useTracking: () => ({
trackConversationCreated: vi.fn(),
}),
}));
describe("useCreateConversation", () => {
it("passes suggested tasks to the V1 create conversation API", async () => {
const createConversationSpy = vi

View File

@@ -1,266 +0,0 @@
import {
describe,
it,
expect,
beforeAll,
afterAll,
afterEach,
vi,
} from "vitest";
import { screen, waitFor, render, cleanup } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import {
createMockAgentErrorEvent,
createMockConversationErrorEvent,
} from "#/mocks/mock-ws-helpers";
import { ConversationWebSocketProvider } from "#/contexts/conversation-websocket-context";
import { conversationWebSocketTestSetup } from "./helpers/msw-websocket-setup";
import { ConnectionStatusComponent } from "./helpers/websocket-test-components";
// Mock the tracking function
const mockTrackCreditLimitReached = vi.fn();
// Mock useTracking hook
vi.mock("#/hooks/use-tracking", () => ({
useTracking: () => ({
trackCreditLimitReached: mockTrackCreditLimitReached,
trackLoginButtonClick: vi.fn(),
trackConversationCreated: vi.fn(),
trackPushButtonClick: vi.fn(),
trackPullButtonClick: vi.fn(),
trackCreatePrButtonClick: vi.fn(),
trackGitProviderConnected: vi.fn(),
trackUserSignupCompleted: vi.fn(),
trackCreditsPurchased: vi.fn(),
}),
}));
// Mock useActiveConversation hook
vi.mock("#/hooks/query/use-active-conversation", () => ({
useActiveConversation: () => ({
data: null,
isLoading: false,
error: null,
}),
}));
// MSW WebSocket mock setup
const { wsLink, server: mswServer } = conversationWebSocketTestSetup();
beforeAll(() => {
// The global MSW server from vitest.setup.ts is already running
// We just need to start our WebSocket-specific server
mswServer.listen({ onUnhandledRequest: "bypass" });
});
afterEach(() => {
// Clear all mocks before each test
mockTrackCreditLimitReached.mockClear();
mswServer.resetHandlers();
// Clean up any React components
cleanup();
});
afterAll(async () => {
// Close the WebSocket MSW server
mswServer.close();
// Give time for any pending WebSocket connections to close. This is very important to prevent serious memory leaks
await new Promise((resolve) => {
setTimeout(resolve, 500);
});
});
// Helper function to render components with all necessary providers
function renderWithProviders(
children: React.ReactNode,
conversationId = "test-conversation-123",
conversationUrl = "http://localhost:3000/api/conversations/test-conversation-123",
) {
const queryClient = new QueryClient({
defaultOptions: {
queries: { retry: false },
mutations: { retry: false },
},
});
return render(
<QueryClientProvider client={queryClient}>
<ConversationWebSocketProvider
conversationId={conversationId}
conversationUrl={conversationUrl}
sessionApiKey={null}
>
{children}
</ConversationWebSocketProvider>
</QueryClientProvider>,
);
}
describe("PostHog Analytics Tracking", () => {
describe("Credit Limit Tracking", () => {
it("should track credit_limit_reached when AgentErrorEvent contains budget error", async () => {
// Create a mock AgentErrorEvent with budget-related error message
const mockBudgetErrorEvent = createMockAgentErrorEvent({
error: "ExceededBudget: Task exceeded maximum budget of $10.00",
});
// Set up MSW to send the budget error event when connection is established
mswServer.use(
wsLink.addEventListener("connection", ({ client, server }) => {
server.connect();
// Send the mock budget error event after connection
client.send(JSON.stringify(mockBudgetErrorEvent));
}),
);
// Render with all providers
renderWithProviders(<ConnectionStatusComponent />);
// Wait for connection to be established
await waitFor(() => {
expect(screen.getByTestId("connection-state")).toHaveTextContent(
"OPEN",
);
});
// Wait for the tracking event to be captured
await waitFor(() => {
expect(mockTrackCreditLimitReached).toHaveBeenCalledWith(
expect.objectContaining({
conversationId: "test-conversation-123",
}),
);
});
});
it("should track credit_limit_reached when AgentErrorEvent contains 'credit' keyword", async () => {
// Create error with "credit" keyword (case-insensitive)
const mockCreditErrorEvent = createMockAgentErrorEvent({
error: "Insufficient CREDIT to complete this operation",
});
mswServer.use(
wsLink.addEventListener("connection", ({ client, server }) => {
server.connect();
client.send(JSON.stringify(mockCreditErrorEvent));
}),
);
renderWithProviders(<ConnectionStatusComponent />);
await waitFor(() => {
expect(screen.getByTestId("connection-state")).toHaveTextContent(
"OPEN",
);
});
await waitFor(() => {
expect(mockTrackCreditLimitReached).toHaveBeenCalledWith(
expect.objectContaining({
conversationId: "test-conversation-123",
}),
);
});
});
it("should NOT track credit_limit_reached for non-budget errors", async () => {
// Create a regular error without budget/credit keywords
const mockRegularErrorEvent = createMockAgentErrorEvent({
error: "Failed to execute command: Permission denied",
});
mswServer.use(
wsLink.addEventListener("connection", ({ client, server }) => {
server.connect();
client.send(JSON.stringify(mockRegularErrorEvent));
}),
);
renderWithProviders(<ConnectionStatusComponent />);
// Wait for connection and error to be processed
await waitFor(() => {
expect(screen.getByTestId("connection-state")).toHaveTextContent(
"OPEN",
);
});
// Verify that credit_limit_reached was NOT tracked
expect(mockTrackCreditLimitReached).not.toHaveBeenCalled();
});
it("should only track credit_limit_reached once per error event", async () => {
const mockBudgetErrorEvent = createMockAgentErrorEvent({
error: "Budget exceeded: $10.00 limit reached",
});
mswServer.use(
wsLink.addEventListener("connection", ({ client, server }) => {
server.connect();
// Send the same error event twice
client.send(JSON.stringify(mockBudgetErrorEvent));
client.send(
JSON.stringify({ ...mockBudgetErrorEvent, id: "different-id" }),
);
}),
);
renderWithProviders(<ConnectionStatusComponent />);
await waitFor(() => {
expect(screen.getByTestId("connection-state")).toHaveTextContent(
"OPEN",
);
});
await waitFor(() => {
expect(mockTrackCreditLimitReached).toHaveBeenCalledTimes(2);
});
// Both calls should be for credit_limit_reached (once per event)
expect(mockTrackCreditLimitReached).toHaveBeenNthCalledWith(
1,
expect.objectContaining({
conversationId: "test-conversation-123",
}),
);
expect(mockTrackCreditLimitReached).toHaveBeenNthCalledWith(
2,
expect.objectContaining({
conversationId: "test-conversation-123",
}),
);
});
it("should track credit_limit_reached when ConversationErrorEvent contains budget error", async () => {
const mockBudgetConversationError = createMockConversationErrorEvent({
detail:
"Budget has been exceeded! Current cost: 18.51, Max budget: 18.24",
});
mswServer.use(
wsLink.addEventListener("connection", ({ client, server }) => {
server.connect();
client.send(JSON.stringify(mockBudgetConversationError));
}),
);
renderWithProviders(<ConnectionStatusComponent />);
await waitFor(() => {
expect(screen.getByTestId("connection-state")).toHaveTextContent(
"OPEN",
);
});
await waitFor(() => {
expect(mockTrackCreditLimitReached).toHaveBeenCalledWith(
expect.objectContaining({
conversationId: "test-conversation-123",
}),
);
});
});
});
});

View File

@@ -51,12 +51,6 @@ vi.mock("#/hooks/use-auth-url", () => ({
useAuthUrlMock(config),
}));
vi.mock("#/hooks/use-tracking", () => ({
useTracking: () => ({
trackLoginButtonClick: vi.fn(),
}),
}));
const { useInvitationMock, buildOAuthStateDataMock } = vi.hoisted(() => ({
useInvitationMock: vi.fn(() => ({
invitationToken: null as string | null,

View File

@@ -1,5 +1,4 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import posthog from "posthog-js";
import {
trackError,
showErrorToast,
@@ -8,12 +7,6 @@ import {
import * as Actions from "#/services/actions";
import * as CustomToast from "#/utils/custom-toast-handlers";
vi.mock("posthog-js", () => ({
default: {
captureException: vi.fn(),
},
}));
vi.mock("#/services/actions", () => ({
handleStatusMessage: vi.fn(),
}));
@@ -28,163 +21,48 @@ describe("Error Handler", () => {
});
describe("trackError", () => {
it("should send error to PostHog with basic info", () => {
const error = {
message: "Test error",
source: "test",
posthog,
};
trackError(error);
expect(posthog.captureException).toHaveBeenCalledWith(
new Error("Test error"),
{
error_source: "test",
},
);
it("should be a no-op (PostHog capture removed)", () => {
// trackError no longer does anything — error tracking is server-side
expect(() =>
trackError({ message: "Test error", source: "test" }),
).not.toThrow();
});
it("should include additional metadata in PostHog event", () => {
const error = {
message: "Test error",
source: "test",
metadata: {
extra: "info",
details: { foo: "bar" },
},
posthog,
};
trackError(error);
expect(posthog.captureException).toHaveBeenCalledWith(
new Error("Test error"),
{
error_source: "test",
extra: "info",
details: { foo: "bar" },
},
);
it("should accept ErrorDetails without throwing", () => {
expect(() =>
trackError({
message: "Test error",
source: "test",
metadata: { extra: "info" },
}),
).not.toThrow();
});
});
describe("showErrorToast", () => {
const errorToastSpy = vi.spyOn(CustomToast, "displayErrorToast");
it("should log error and show toast", () => {
const error = {
message: "Toast error",
source: "toast-test",
posthog,
};
showErrorToast(error);
it("should show toast with the error message", () => {
showErrorToast({ message: "Toast error", source: "toast-test" });
// Verify PostHog logging
expect(posthog.captureException).toHaveBeenCalledWith(
new Error("Toast error"),
{
error_source: "toast-test",
},
);
// Verify toast was shown
expect(errorToastSpy).toHaveBeenCalled();
expect(errorToastSpy).toHaveBeenCalledWith("Toast error");
});
it("should include metadata in PostHog event when showing toast", () => {
const error = {
message: "Toast error",
source: "toast-test",
metadata: { context: "testing" },
posthog,
};
it("should show toast even without source or metadata", () => {
showErrorToast({ message: "Simple error" });
showErrorToast(error);
expect(posthog.captureException).toHaveBeenCalledWith(
new Error("Toast error"),
{
error_source: "toast-test",
context: "testing",
},
);
});
it("should log errors from different sources with appropriate metadata", () => {
// Test agent status error
showErrorToast({
message: "Agent error",
source: "agent-status",
metadata: { id: "error.agent" },
posthog,
});
expect(posthog.captureException).toHaveBeenCalledWith(
new Error("Agent error"),
{
error_source: "agent-status",
id: "error.agent",
},
);
showErrorToast({
message: "Server error",
source: "server",
metadata: { error_code: 500, details: "Internal error" },
posthog,
});
expect(posthog.captureException).toHaveBeenCalledWith(
new Error("Server error"),
{
error_source: "server",
error_code: 500,
details: "Internal error",
},
);
});
it("should log feedback submission errors with conversation context", () => {
const error = new Error("Feedback submission failed");
showErrorToast({
message: error.message,
source: "feedback",
metadata: { conversationId: "123", error },
posthog,
});
expect(posthog.captureException).toHaveBeenCalledWith(
new Error("Feedback submission failed"),
{
error_source: "feedback",
conversationId: "123",
error,
},
);
expect(errorToastSpy).toHaveBeenCalledWith("Simple error");
});
});
describe("showChatError", () => {
it("should log error and show chat error message", () => {
const error = {
it("should show chat error message via handleStatusMessage", () => {
showChatError({
message: "Chat error",
source: "chat-test",
msgId: "123",
posthog,
};
});
showChatError(error);
// Verify PostHog logging
expect(posthog.captureException).toHaveBeenCalledWith(
new Error("Chat error"),
{
error_source: "chat-test",
},
);
// Verify error message was shown in chat
expect(Actions.handleStatusMessage).toHaveBeenCalledWith({
type: "error",
message: "Chat error",
@@ -192,5 +70,19 @@ describe("Error Handler", () => {
status_update: true,
});
});
it("should show chat error without msgId", () => {
showChatError({
message: "Chat error no id",
source: "chat-test",
});
expect(Actions.handleStatusMessage).toHaveBeenCalledWith({
type: "error",
message: "Chat error no id",
id: undefined,
status_update: true,
});
});
});
});

View File

@@ -7,7 +7,6 @@ import BitbucketLogo from "#/assets/branding/bitbucket-logo.svg?react";
import { useAuthUrl } from "#/hooks/use-auth-url";
import { WebClientConfig } from "#/api/option-service/option.types";
import { Provider } from "#/types/settings";
import { useTracking } from "#/hooks/use-tracking";
import { TermsAndPrivacyNotice } from "#/components/shared/terms-and-privacy-notice";
import { useRecaptcha } from "#/hooks/use-recaptcha";
import { useConfig } from "#/hooks/query/use-config";
@@ -39,7 +38,6 @@ export function LoginContent({
buildOAuthStateData,
}: LoginContentProps) {
const { t } = useTranslation();
const { trackLoginButtonClick } = useTracking();
const { data: config } = useConfig();
// reCAPTCHA - only need token generation, verification happens at backend callback
@@ -65,12 +63,7 @@ export function LoginContent({
authUrl,
});
const handleAuthRedirect = async (
redirectUrl: string,
provider: Provider,
) => {
trackLoginButtonClick({ provider });
const handleAuthRedirect = async (redirectUrl: string) => {
const url = new URL(redirectUrl);
const currentState =
url.searchParams.get("state") || window.location.origin;
@@ -105,25 +98,25 @@ export function LoginContent({
const handleGitHubAuth = () => {
if (githubAuthUrl) {
handleAuthRedirect(githubAuthUrl, "github");
handleAuthRedirect(githubAuthUrl);
}
};
const handleGitLabAuth = () => {
if (gitlabAuthUrl) {
handleAuthRedirect(gitlabAuthUrl, "gitlab");
handleAuthRedirect(gitlabAuthUrl);
}
};
const handleBitbucketAuth = () => {
if (bitbucketAuthUrl) {
handleAuthRedirect(bitbucketAuthUrl, "bitbucket");
handleAuthRedirect(bitbucketAuthUrl);
}
};
const handleBitbucketDataCenterAuth = () => {
if (bitbucketDataCenterAuthUrl) {
handleAuthRedirect(bitbucketDataCenterAuthUrl, "bitbucket_data_center");
handleAuthRedirect(bitbucketDataCenterAuthUrl);
}
};

View File

@@ -1,5 +1,4 @@
import React from "react";
import { usePostHog } from "posthog-js/react";
import { useParams } from "react-router";
import { useTranslation } from "react-i18next";
import { convertImageToBase64 } from "#/utils/convert-image-to-base-64";
@@ -15,7 +14,6 @@ import { useWsClient } from "#/context/ws-client-provider";
import { Messages as V0Messages } from "./messages";
import { ChatSuggestions } from "./chat-suggestions";
import { ScrollProvider } from "#/context/scroll-context";
import { useInitialQueryStore } from "#/stores/initial-query-store";
import { useSendMessage } from "#/hooks/use-send-message";
import { useAgentState } from "#/hooks/use-agent-state";
import { useHandleBuildPlanClick } from "#/hooks/use-handle-build-plan-click";
@@ -39,17 +37,7 @@ import { useConversationWebSocket } from "#/contexts/conversation-websocket-cont
import ChatStatusIndicator from "./chat-status-indicator";
import { getStatusColor, getStatusText } from "#/utils/utils";
function getEntryPoint(
hasRepository: boolean | null,
hasReplayJson: boolean | null,
): string {
if (hasRepository) return "github";
if (hasReplayJson) return "replay";
return "direct";
}
export function ChatInterface() {
const posthog = usePostHog();
const { setMessageToSend } = useConversationStore();
const { data: conversation } = useActiveConversation();
const { errorMessage, removeErrorMessage } = useErrorMessageStore();
@@ -118,7 +106,6 @@ export function ChatInterface() {
"positive" | "negative"
>("positive");
const [feedbackModalIsOpen, setFeedbackModalIsOpen] = React.useState(false);
const { selectedRepository, replayJson } = useInitialQueryStore();
const params = useParams();
const { mutateAsync: uploadFiles } = useUnifiedUploadFiles();
@@ -149,22 +136,6 @@ export function ChatInterface() {
// Create mutable copies of the arrays
const images = [...originalImages];
const files = [...originalFiles];
if (totalEvents === 0) {
posthog.capture("initial_query_submitted", {
entry_point: getEntryPoint(
selectedRepository !== null,
replayJson !== null,
),
query_character_length: content.length,
replay_json_size: replayJson?.length,
});
} else {
posthog.capture("user_message_sent", {
session_message_count: totalEvents,
current_message_length: content.length,
});
}
// Validate file sizes before any processing
const allFiles = [...images, ...files];
const validation = validateFiles(allFiles);

View File

@@ -4,7 +4,6 @@ import { cn, getCreatePRPrompt } from "#/utils/utils";
import { useUserProviders } from "#/hooks/use-user-providers";
import { I18nKey } from "#/i18n/declaration";
import { Provider } from "#/types/settings";
import { useTracking } from "#/hooks/use-tracking";
interface GitControlBarPrButtonProps {
onSuggestionsClick: (value: string) => void;
@@ -20,8 +19,6 @@ export function GitControlBarPrButton({
isConversationReady = true,
}: GitControlBarPrButtonProps) {
const { t } = useTranslation();
const { trackCreatePrButtonClick } = useTracking();
const { providers } = useUserProviders();
const providersAreSet = providers.length > 0;
@@ -29,7 +26,6 @@ export function GitControlBarPrButton({
providersAreSet && hasRepository && isConversationReady;
const handlePrClick = () => {
trackCreatePrButtonClick();
onSuggestionsClick(getCreatePRPrompt(currentGitProvider));
};

View File

@@ -4,7 +4,6 @@ import { cn, getGitPullPrompt } from "#/utils/utils";
import { useActiveConversation } from "#/hooks/query/use-active-conversation";
import { useUserProviders } from "#/hooks/use-user-providers";
import { I18nKey } from "#/i18n/declaration";
import { useTracking } from "#/hooks/use-tracking";
interface GitControlBarPullButtonProps {
onSuggestionsClick: (value: string) => void;
@@ -16,8 +15,6 @@ export function GitControlBarPullButton({
isConversationReady = true,
}: GitControlBarPullButtonProps) {
const { t } = useTranslation();
const { trackPullButtonClick } = useTracking();
const { data: conversation } = useActiveConversation();
const { providers } = useUserProviders();
@@ -27,7 +24,6 @@ export function GitControlBarPullButton({
providersAreSet && hasRepository && isConversationReady;
const handlePullClick = () => {
trackPullButtonClick();
onSuggestionsClick(getGitPullPrompt());
};

View File

@@ -4,7 +4,6 @@ import { cn, getGitPushPrompt } from "#/utils/utils";
import { useUserProviders } from "#/hooks/use-user-providers";
import { I18nKey } from "#/i18n/declaration";
import { Provider } from "#/types/settings";
import { useTracking } from "#/hooks/use-tracking";
interface GitControlBarPushButtonProps {
onSuggestionsClick: (value: string) => void;
@@ -20,8 +19,6 @@ export function GitControlBarPushButton({
isConversationReady = true,
}: GitControlBarPushButtonProps) {
const { t } = useTranslation();
const { trackPushButtonClick } = useTracking();
const { providers } = useUserProviders();
const providersAreSet = providers.length > 0;
@@ -29,7 +26,6 @@ export function GitControlBarPushButton({
providersAreSet && hasRepository && isConversationReady;
const handlePushClick = () => {
trackPushButtonClick();
onSuggestionsClick(getGitPushPrompt(currentGitProvider));
};

View File

@@ -1,7 +1,6 @@
import React from "react";
import { useTranslation } from "react-i18next";
import { Link } from "react-router";
import { useFeatureFlagEnabled } from "posthog-js/react";
import { ContextMenu } from "#/ui/context-menu";
import { ContextMenuListItem } from "./context-menu-list-item";
import { Divider } from "#/ui/divider";
@@ -9,11 +8,7 @@ import { useClickOutsideElement } from "#/hooks/use-click-outside-element";
import { I18nKey } from "#/i18n/declaration";
import LogOutIcon from "#/icons/log-out.svg?react";
import DocumentIcon from "#/icons/document.svg?react";
import PlusIcon from "#/icons/plus.svg?react";
import { useSettingsNavItems } from "#/hooks/use-settings-nav-items";
import { useConfig } from "#/hooks/query/use-config";
import { useSettings } from "#/hooks/query/use-settings";
import { useTracking } from "#/hooks/use-tracking";
interface AccountSettingsContextMenuProps {
onLogout: () => void;
@@ -26,20 +21,8 @@ export function AccountSettingsContextMenu({
}: AccountSettingsContextMenuProps) {
const ref = useClickOutsideElement<HTMLUListElement>(onClose);
const { t } = useTranslation();
const { trackAddTeamMembersButtonClick } = useTracking();
const { data: config } = useConfig();
const { data: settings } = useSettings();
const isAddTeamMemberEnabled = useFeatureFlagEnabled(
"exp_add_team_member_button",
);
// Get navigation items and filter out LLM settings if the feature flag is enabled
const items = useSettingsNavItems();
const isSaasMode = config?.app_mode === "saas";
const hasAnalyticsConsent = settings?.user_consents_to_analytics === true;
const showAddTeamMembers =
isSaasMode && isAddTeamMemberEnabled && hasAnalyticsConsent;
const navItems = items.map((item) => ({
...item,
icon: React.cloneElement(item.icon, {
@@ -49,11 +32,6 @@ export function AccountSettingsContextMenu({
}));
const handleNavigationClick = () => onClose();
const handleAddTeamMembers = () => {
trackAddTeamMembersButtonClick();
onClose();
};
return (
<ContextMenu
testId="account-settings-context-menu"
@@ -61,18 +39,6 @@ export function AccountSettingsContextMenu({
alignment="right"
className="mt-0 md:right-full md:left-full md:bottom-0 ml-0 w-fit z-[9999]"
>
{showAddTeamMembers && (
<ContextMenuListItem
testId="add-team-members-button"
onClick={handleAddTeamMembers}
className="flex items-center gap-2 p-2 hover:bg-[#5C5D62] rounded h-[30px]"
>
<PlusIcon width={16} height={16} />
<span className="text-white text-sm">
{t(I18nKey.SETTINGS$NAV_ADD_TEAM_MEMBERS)}
</span>
</ContextMenuListItem>
)}
{navItems.map(({ to, text, icon }) => (
<Link key={to} to={to} className="text-decoration-none">
<ContextMenuListItem

View File

@@ -1,5 +1,4 @@
import React from "react";
import { usePostHog } from "posthog-js/react";
import { cn } from "#/utils/utils";
import { transformVSCodeUrl } from "#/utils/vscode-url-helper";
import ConversationService from "#/api/conversation-service/conversation-service.api";
@@ -45,7 +44,6 @@ export function ConversationCard({
contextMenuOpen = false,
onContextMenuToggle,
}: ConversationCardProps) {
const posthog = usePostHog();
const [titleMode, setTitleMode] = React.useState<"view" | "edit">("view");
const { mutateAsync: downloadConversation } = useDownloadConversation();
@@ -82,7 +80,6 @@ export function ConversationCard({
) => {
event.preventDefault();
event.stopPropagation();
posthog.capture("download_via_vscode_button_clicked");
// Fetch the VS Code URL from the API
if (conversationId) {

View File

@@ -68,6 +68,7 @@ export function PostHogWrapper({ children }: { children: React.ReactNode }) {
api_host: "https://us.i.posthog.com",
person_profiles: "identified_only",
bootstrap: bootstrapIds,
__add_tracing_headers: [window.location.hostname],
}}
>
{children}

View File

@@ -1,7 +1,6 @@
import { useLocation } from "react-router";
import { useTranslation } from "react-i18next";
import React from "react";
import { usePostHog } from "posthog-js/react";
import { I18nKey } from "#/i18n/declaration";
import { organizeModelsAndProviders } from "#/utils/organize-models-and-providers";
import { DangerModal } from "../confirmation-modals/danger-modal";
@@ -22,7 +21,6 @@ interface SettingsFormProps {
}
export function SettingsForm({ settings, models, onClose }: SettingsFormProps) {
const posthog = usePostHog();
const { mutate: saveUserSettings } = useSaveSettings();
const location = useLocation();
@@ -39,14 +37,6 @@ export function SettingsForm({ settings, models, onClose }: SettingsFormProps) {
await saveUserSettings(newSettings, {
onSuccess: () => {
onClose();
posthog.capture("settings_saved", {
LLM_MODEL: newSettings.llm_model,
LLM_API_KEY_SET: newSettings.llm_api_key_set ? "SET" : "UNSET",
SEARCH_API_KEY_SET: newSettings.search_api_key ? "SET" : "UNSET",
REMOTE_RUNTIME_RESOURCE_FACTOR:
newSettings.remote_runtime_resource_factor,
});
},
});
};

View File

@@ -1,7 +1,6 @@
import React from "react";
import { io, Socket } from "socket.io-client";
import { useQueryClient } from "@tanstack/react-query";
import { usePostHog } from "posthog-js/react";
import EventLogger from "#/utils/event-logger";
import { handleAssistantMessage } from "#/services/actions";
import { showChatError, trackError } from "#/utils/error-handler";
@@ -101,10 +100,7 @@ interface ErrorArgData {
msg_id: string;
}
export function updateStatusWhenErrorMessagePresent(
data: ErrorArg | unknown,
posthog?: ReturnType<typeof usePostHog>,
) {
export function updateStatusWhenErrorMessagePresent(data: ErrorArg | unknown) {
const isObject = (val: unknown): val is object =>
!!val && typeof val === "object";
const isString = (val: unknown): val is string => typeof val === "string";
@@ -127,7 +123,6 @@ export function updateStatusWhenErrorMessagePresent(
source: "websocket",
metadata,
msgId,
posthog,
});
}
}
@@ -136,7 +131,6 @@ export function WsClientProvider({
conversationId,
children,
}: React.PropsWithChildren<WsClientProviderProps>) {
const posthog = usePostHog();
const { setErrorMessage, removeErrorMessage } = useErrorMessageStore();
const { removeOptimisticUserMessage } = useOptimisticUserMessageStore();
const { addEvent, clearEvents } = useEventStore();
@@ -205,7 +199,6 @@ export function WsClientProvider({
message: errorMessage,
source: "chat",
metadata: { msgId: event.id },
posthog,
});
setErrorMessage(errorMessage);
@@ -221,7 +214,6 @@ export function WsClientProvider({
message: event.message,
source: "chat",
metadata: { msgId: event.id },
posthog,
});
} else {
removeErrorMessage();
@@ -289,14 +281,14 @@ export function WsClientProvider({
sio.io.opts.query = sio.io.opts.query || {};
sio.io.opts.query.latest_event_id = lastEventRef.current?.id;
updateStatusWhenErrorMessagePresent(data, posthog);
updateStatusWhenErrorMessagePresent(data);
setErrorMessage(hasValidMessageProperty(data) ? data.message : "");
}
function handleError(data: unknown) {
// set status
setWebSocketStatus("DISCONNECTED");
updateStatusWhenErrorMessagePresent(data, posthog);
updateStatusWhenErrorMessagePresent(data);
setErrorMessage(
hasValidMessageProperty(data)

View File

@@ -8,7 +8,6 @@ import React, {
useRef,
} from "react";
import { useQueryClient } from "@tanstack/react-query";
import { usePostHog } from "posthog-js/react";
import { useWebSocket, WebSocketHookOptions } from "#/hooks/use-websocket";
import { useEventStore } from "#/stores/use-event-store";
import { useErrorMessageStore } from "#/stores/error-message-store";
@@ -42,7 +41,6 @@ import type {
import EventService from "#/api/event-service/event-service.api";
import { useConversationStore } from "#/stores/conversation-store";
import { isBudgetOrCreditError, trackError } from "#/utils/error-handler";
import { useTracking } from "#/hooks/use-tracking";
import { useReadConversationFile } from "#/hooks/mutation/use-read-conversation-file";
import useMetricsStore from "#/stores/metrics-store";
import { I18nKey } from "#/i18n/declaration";
@@ -93,14 +91,12 @@ export function ConversationWebSocketProvider({
const hasConnectedRefMain = React.useRef(false);
const hasConnectedRefPlanning = React.useRef(false);
const posthog = usePostHog();
const queryClient = useQueryClient();
const { addEvent } = useEventStore();
const { setErrorMessage, removeErrorMessage } = useErrorMessageStore();
const { removeOptimisticUserMessage } = useOptimisticUserMessageStore();
const { setExecutionStatus } = useV1ConversationStateStore();
const { appendInput, appendOutput } = useCommandStore();
const { trackCreditLimitReached } = useTracking();
// History loading state - separate per connection
const [isLoadingHistoryMain, setIsLoadingHistoryMain] = useState(true);
@@ -356,13 +352,9 @@ export function ConversationWebSocketProvider({
eventId: event.id,
errorCode: event.code,
},
posthog,
});
if (isBudgetOrCreditError(event.detail)) {
setErrorMessage(I18nKey.STATUS$ERROR_LLM_OUT_OF_CREDITS);
trackCreditLimitReached({
conversationId: conversationId || "unknown",
});
} else {
setErrorMessage(event.detail);
}
@@ -381,14 +373,10 @@ export function ConversationWebSocketProvider({
toolName: event.tool_name,
toolCallId: event.tool_call_id,
},
posthog,
});
// Use friendly i18n message for budget/credit errors instead of raw error
if (isBudgetOrCreditError(event.error)) {
setErrorMessage(I18nKey.STATUS$ERROR_LLM_OUT_OF_CREDITS);
trackCreditLimitReached({
conversationId: conversationId || "unknown",
});
} else {
setErrorMessage(event.error);
}
@@ -473,8 +461,6 @@ export function ConversationWebSocketProvider({
appendInput,
appendOutput,
updateMetricsFromStats,
trackCreditLimitReached,
posthog,
],
);
@@ -515,13 +501,9 @@ export function ConversationWebSocketProvider({
eventId: event.id,
errorCode: event.code,
},
posthog,
});
if (isBudgetOrCreditError(event.detail)) {
setErrorMessage(I18nKey.STATUS$ERROR_LLM_OUT_OF_CREDITS);
trackCreditLimitReached({
conversationId: conversationId || "unknown",
});
} else {
setErrorMessage(event.detail);
}
@@ -540,14 +522,10 @@ export function ConversationWebSocketProvider({
toolName: event.tool_name,
toolCallId: event.tool_call_id,
},
posthog,
});
// Use friendly i18n message for budget/credit errors instead of raw error
if (isBudgetOrCreditError(event.error)) {
setErrorMessage(I18nKey.STATUS$ERROR_LLM_OUT_OF_CREDITS);
trackCreditLimitReached({
conversationId: conversationId || "unknown",
});
} else {
setErrorMessage(event.error);
}
@@ -655,8 +633,6 @@ export function ConversationWebSocketProvider({
readConversationFile,
setPlanContent,
updateMetricsFromStats,
trackCreditLimitReached,
posthog,
],
);

View File

@@ -3,7 +3,6 @@ import { usePostHog } from "posthog-js/react";
import { useNavigate } from "react-router";
import { openHands } from "#/api/open-hands-axios";
import { handleCaptureConsent } from "#/utils/handle-capture-consent";
import { useTracking } from "#/hooks/use-tracking";
interface AcceptTosVariables {
redirectUrl: string;
@@ -16,7 +15,6 @@ interface AcceptTosResponse {
export const useAcceptTos = () => {
const posthog = usePostHog();
const navigate = useNavigate();
const { trackUserSignupCompleted } = useTracking();
return useMutation({
mutationFn: async ({ redirectUrl }: AcceptTosVariables) => {
@@ -29,9 +27,6 @@ export const useAcceptTos = () => {
});
},
onSuccess: (response, { redirectUrl }) => {
// Track user signup completion
trackUserSignupCompleted();
// Get the redirect URL from the response
const finalRedirectUrl = response.data.redirect_url || redirectUrl;

View File

@@ -1,11 +1,9 @@
import { useMutation, useQueryClient } from "@tanstack/react-query";
import { SecretsService } from "#/api/secrets-service";
import { Provider, ProviderToken } from "#/types/settings";
import { useTracking } from "#/hooks/use-tracking";
export const useAddGitProviders = () => {
const queryClient = useQueryClient();
const { trackGitProviderConnected } = useTracking();
return useMutation({
mutationFn: ({
@@ -13,18 +11,7 @@ export const useAddGitProviders = () => {
}: {
providers: Record<Provider, ProviderToken>;
}) => SecretsService.addGitProvider(providers),
onSuccess: async (_, { providers }) => {
// Track which providers were connected (filter out empty tokens)
const connectedProviders = Object.entries(providers)
.filter(([, value]) => value.token && value.token.trim() !== "")
.map(([key]) => key);
if (connectedProviders.length > 0) {
trackGitProviderConnected({
providers: connectedProviders,
});
}
onSuccess: async () => {
await queryClient.invalidateQueries({ queryKey: ["settings"] });
},
meta: {

View File

@@ -4,7 +4,6 @@ import V1ConversationService from "#/api/conversation-service/v1-conversation-se
import { SuggestedTask } from "#/utils/types";
import { Provider } from "#/types/settings";
import { CreateMicroagent, Conversation } from "#/api/open-hands.types";
import { useTracking } from "#/hooks/use-tracking";
import { useSettings } from "#/hooks/query/use-settings";
interface CreateConversationVariables {
@@ -33,7 +32,6 @@ interface CreateConversationResponse extends Partial<Conversation> {
export const useCreateConversation = () => {
const queryClient = useQueryClient();
const { trackConversationCreated } = useTracking();
const { data: settings } = useSettings();
return useMutation({
@@ -95,11 +93,7 @@ export const useCreateConversation = () => {
is_v1: false,
};
},
onSuccess: async (_, { repository }) => {
trackConversationCreated({
hasRepository: !!repository,
});
onSuccess: async () => {
queryClient.removeQueries({
queryKey: ["user", "conversations"],
});

View File

@@ -1,5 +1,4 @@
import { useMutation, useQueryClient } from "@tanstack/react-query";
import { usePostHog } from "posthog-js/react";
import { DEFAULT_SETTINGS } from "#/services/settings";
import SettingsService from "#/api/settings-service/settings-service.api";
import { Settings } from "#/types/settings";
@@ -27,7 +26,6 @@ const saveSettingsMutationFn = async (settings: Partial<Settings>) => {
};
export const useSaveSettings = () => {
const posthog = usePostHog();
const queryClient = useQueryClient();
const { data: currentSettings } = useSettings();
@@ -35,24 +33,6 @@ export const useSaveSettings = () => {
mutationFn: async (settings: Partial<Settings>) => {
const newSettings = { ...currentSettings, ...settings };
// Track MCP configuration changes
if (
settings.mcp_config &&
currentSettings?.mcp_config !== settings.mcp_config
) {
const hasMcpConfig = !!settings.mcp_config;
const sseServersCount = settings.mcp_config?.sse_servers?.length || 0;
const stdioServersCount =
settings.mcp_config?.stdio_servers?.length || 0;
// Track MCP configuration usage
posthog.capture("mcp_config_updated", {
has_mcp_config: hasMcpConfig,
sse_servers_count: sseServersCount,
stdio_servers_count: stdioServersCount,
});
}
await saveSettingsMutationFn(newSettings);
},
onSuccess: async () => {

View File

@@ -1,21 +1,30 @@
import { useMutation } from "@tanstack/react-query";
import { useNavigate } from "react-router";
import { openHands } from "#/api/open-hands-axios";
import { displayErrorToast } from "#/utils/custom-toast-handlers";
type SubmitOnboardingArgs = {
selections: Record<string, string>;
};
interface OnboardingResponse {
status: string;
redirect_url: string;
}
export const useSubmitOnboarding = () => {
const navigate = useNavigate();
return useMutation({
mutationFn: async ({ selections }: SubmitOnboardingArgs) =>
// TODO: mark onboarding as complete
// TODO: persist user responses
({ selections }),
onSuccess: () => {
const finalRedirectUrl = "/"; // TODO: use redirect url from api response
mutationFn: async ({ selections }: SubmitOnboardingArgs) => {
const { data } = await openHands.post<OnboardingResponse>(
"/api/onboarding",
{ selections },
);
return data;
},
onSuccess: (data) => {
const finalRedirectUrl = data.redirect_url || "/";
// Check if the redirect URL is an external URL (starts with http or https)
if (
finalRedirectUrl.startsWith("http://") ||

View File

@@ -1,6 +1,5 @@
import { useTranslation } from "react-i18next";
import React from "react";
import { usePostHog } from "posthog-js/react";
import { useParams, useNavigate } from "react-router";
import { transformVSCodeUrl } from "#/utils/vscode-url-helper";
import useMetricsStore from "#/stores/metrics-store";
@@ -38,7 +37,6 @@ export function useConversationNameContextMenu({
showOptions = false,
onContextMenuToggle,
}: UseConversationNameContextMenuProps) {
const posthog = usePostHog();
const { t } = useTranslation();
const { conversationId: currentConversationId } = useParams();
const navigate = useNavigate();
@@ -137,7 +135,6 @@ export function useConversationNameContextMenu({
) => {
event.preventDefault();
event.stopPropagation();
posthog.capture("download_via_vscode_button_clicked");
// Fetch the VS Code URL from the API
if (conversationId) {

View File

@@ -1,5 +1,4 @@
import { useMutation } from "@tanstack/react-query";
import { usePostHog } from "posthog-js/react";
import { useTranslation } from "react-i18next";
import V1ConversationService from "#/api/conversation-service/v1-conversation-service.api";
import { downloadBlob } from "#/utils/utils";
@@ -7,13 +6,11 @@ import { displayErrorToast } from "#/utils/custom-toast-handlers";
import { I18nKey } from "#/i18n/declaration";
export const useDownloadConversation = () => {
const posthog = usePostHog();
const { t } = useTranslation();
return useMutation({
mutationKey: ["conversations", "download"],
mutationFn: async (conversationId: string) => {
posthog.capture("download_trajectory_button_clicked");
const blob =
await V1ConversationService.downloadConversation(conversationId);
downloadBlob(blob, `conversation_${conversationId}.zip`);

View File

@@ -1,138 +0,0 @@
import { usePostHog } from "posthog-js/react";
import { useConfig } from "./query/use-config";
import { useSettings } from "./query/use-settings";
import { Provider } from "#/types/settings";
/**
* Hook that provides tracking functions with automatic data collection
* from available hooks (config, settings, etc.)
*/
export const useTracking = () => {
const posthog = usePostHog();
const { data: config } = useConfig();
const { data: settings } = useSettings();
// Common properties included in all tracking events
const commonProperties = {
app_surface: config?.app_mode || "unknown",
plan_tier: null,
current_url: window.location.href,
user_email: settings?.email || settings?.git_user_email || null,
};
const trackLoginButtonClick = ({ provider }: { provider: Provider }) => {
posthog.capture("login_button_clicked", {
provider,
...commonProperties,
});
};
const trackConversationCreated = ({
hasRepository,
}: {
hasRepository: boolean;
}) => {
posthog.capture("conversation_created", {
has_repository: hasRepository,
...commonProperties,
});
};
const trackPushButtonClick = () => {
posthog.capture("push_button_clicked", {
...commonProperties,
});
};
const trackPullButtonClick = () => {
posthog.capture("pull_button_clicked", {
...commonProperties,
});
};
const trackCreatePrButtonClick = () => {
posthog.capture("create_pr_button_clicked", {
...commonProperties,
});
};
const trackGitProviderConnected = ({
providers,
}: {
providers: string[];
}) => {
posthog.capture("git_provider_connected", {
providers,
...commonProperties,
});
};
const trackUserSignupCompleted = () => {
posthog.capture("user_signup_completed", {
signup_timestamp: new Date().toISOString(),
...commonProperties,
});
};
const trackCreditsPurchased = ({
amountUsd,
stripeSessionId,
}: {
amountUsd: number;
stripeSessionId: string;
}) => {
posthog.capture("credits_purchased", {
amount_usd: amountUsd,
stripe_session_id: stripeSessionId,
...commonProperties,
});
};
const trackCreditLimitReached = ({
conversationId,
}: {
conversationId: string;
}) => {
posthog.capture("credit_limit_reached", {
conversation_id: conversationId,
...commonProperties,
});
};
const trackAddTeamMembersButtonClick = () => {
posthog.capture("exp_add_team_members", {
...commonProperties,
});
};
const trackOnboardingCompleted = ({
role,
orgSize,
useCase,
}: {
role: string;
orgSize: string;
useCase: string;
}) => {
posthog.capture("onboarding_completed", {
role,
org_size: orgSize,
use_case: useCase,
...commonProperties,
});
};
return {
trackLoginButtonClick,
trackConversationCreated,
trackPushButtonClick,
trackPullButtonClick,
trackCreatePrButtonClick,
trackGitProviderConnected,
trackUserSignupCompleted,
trackCreditsPurchased,
trackCreditLimitReached,
trackAddTeamMembersButtonClick,
trackOnboardingCompleted,
};
};

View File

@@ -7,28 +7,14 @@ import {
displaySuccessToast,
} from "#/utils/custom-toast-handlers";
import { I18nKey } from "#/i18n/declaration";
import { useTracking } from "#/hooks/use-tracking";
function BillingSettingsScreen() {
const { t } = useTranslation();
const [searchParams, setSearchParams] = useSearchParams();
const { trackCreditsPurchased } = useTracking();
const checkoutStatus = searchParams.get("checkout");
React.useEffect(() => {
if (checkoutStatus === "success") {
// Get purchase details from URL params
const amount = searchParams.get("amount");
const sessionId = searchParams.get("session_id");
// Track credits purchased if we have the necessary data
if (amount && sessionId) {
trackCreditsPurchased({
amountUsd: parseFloat(amount),
stripeSessionId: sessionId,
});
}
displaySuccessToast(t(I18nKey.PAYMENT$SUCCESS));
setSearchParams({});
@@ -36,7 +22,7 @@ function BillingSettingsScreen() {
displayErrorToast(t(I18nKey.PAYMENT$CANCELLED));
setSearchParams({});
}
}, [checkoutStatus, searchParams, setSearchParams, t, trackCreditsPurchased]);
}, [checkoutStatus, searchParams, setSearchParams, t]);
return <PaymentForm />;
}

View File

@@ -9,7 +9,6 @@ import { BrandButton } from "#/components/features/settings/brand-button";
import { I18nKey } from "#/i18n/declaration";
import OpenHandsLogoWhite from "#/assets/branding/openhands-logo-white.svg?react";
import { useSubmitOnboarding } from "#/hooks/mutation/use-submit-onboarding";
import { useTracking } from "#/hooks/use-tracking";
import { ENABLE_ONBOARDING } from "#/utils/feature-flags";
import { cn } from "#/utils/utils";
import { ModalBackdrop } from "#/components/shared/modals/modal-backdrop";
@@ -136,7 +135,6 @@ function OnboardingForm() {
const { t } = useTranslation();
const navigate = useNavigate();
const { mutate: submitOnboarding } = useSubmitOnboarding();
const { trackOnboardingCompleted } = useTracking();
const [currentStepIndex, setCurrentStepIndex] = React.useState(0);
const [selections, setSelections] = React.useState<Record<string, string>>(
@@ -158,15 +156,6 @@ function OnboardingForm() {
const handleNext = () => {
if (isLastStep) {
submitOnboarding({ selections });
try {
trackOnboardingCompleted({
role: selections.step1,
orgSize: selections.step2,
useCase: selections.step3,
});
} catch (error) {
console.error("Failed to track onboarding:", error);
}
} else {
setCurrentStepIndex((prev) => prev + 1);
}

View File

@@ -72,7 +72,6 @@ export function handleStatusMessage(message: StatusMessage) {
message: message.message,
source: "chat",
metadata: { msgId: message.id },
posthog: undefined, // Service file - can't use hooks
});
}
}

View File

@@ -1,4 +1,3 @@
import type { PostHog } from "posthog-js";
import { handleStatusMessage } from "#/services/actions";
import { displayErrorToast } from "./custom-toast-handlers";
@@ -7,31 +6,19 @@ interface ErrorDetails {
source?: string;
metadata?: Record<string, unknown>;
msgId?: string;
posthog?: PostHog;
}
export function trackError({
message,
source,
metadata = {},
posthog,
}: ErrorDetails) {
if (!posthog) return;
const error = new Error(message);
posthog.captureException(error, {
error_source: source || "unknown",
...metadata,
});
}
// PostHog capture removed — error tracking is now handled server-side
export function trackError(
details: ErrorDetails, // eslint-disable-line @typescript-eslint/no-unused-vars
): void {}
export function showErrorToast({
message,
source,
metadata = {},
posthog,
}: ErrorDetails) {
trackError({ message, source, metadata, posthog });
trackError({ message, source, metadata });
displayErrorToast(message);
}
@@ -40,9 +27,8 @@ export function showChatError({
source,
metadata = {},
msgId,
posthog,
}: ErrorDetails) {
trackError({ message, source, metadata, posthog });
trackError({ message, source, metadata });
handleStatusMessage({
type: "error",
message,

View File

@@ -0,0 +1,53 @@
"""OpenHands analytics package.
Provides a module-level singleton pattern for the AnalyticsService.
Usage::
from openhands.analytics import init_analytics_service, get_analytics_service
# At application startup:
init_analytics_service(api_key=..., host=..., app_mode=..., is_feature_env=...)
# At call sites:
svc = get_analytics_service()
if svc:
svc.capture(...)
"""
from openhands.analytics.analytics_service import AnalyticsService
from openhands.server.types import AppMode
_analytics_service: AnalyticsService | None = None
def init_analytics_service(
api_key: str,
host: str,
app_mode: AppMode,
is_feature_env: bool,
) -> AnalyticsService:
"""Create and store the module-level AnalyticsService singleton.
Returns the newly created instance. Subsequent calls to
:func:`get_analytics_service` will return the same object.
"""
global _analytics_service
_analytics_service = AnalyticsService(
api_key=api_key,
host=host,
app_mode=app_mode,
is_feature_env=is_feature_env,
)
return _analytics_service
def get_analytics_service() -> AnalyticsService | None:
"""Return the module-level AnalyticsService singleton.
Returns ``None`` if :func:`init_analytics_service` has not been called yet.
"""
return _analytics_service
__all__ = ['AnalyticsService', 'get_analytics_service', 'init_analytics_service']

View File

@@ -0,0 +1,20 @@
"""Event name constants for PostHog analytics.
Naming convention: PostHog recommended object-action, lowercase with spaces.
"""
# Phase 1 events
USER_LOGGED_IN = 'user logged in'
# Phase 2 events
USER_SIGNED_UP = 'user signed up'
CONVERSATION_CREATED = 'conversation created'
CONVERSATION_FINISHED = 'conversation finished'
CONVERSATION_ERRORED = 'conversation errored'
CREDIT_PURCHASED = 'credit purchased'
CREDIT_LIMIT_REACHED = 'credit limit reached'
# Phase 4 events
USER_ACTIVATED = 'user activated'
GIT_PROVIDER_CONNECTED = 'git provider connected'
ONBOARDING_COMPLETED = 'onboarding completed'

View File

@@ -0,0 +1,181 @@
"""Core analytics service for OpenHands.
Provides a thin wrapper around the PostHog SDK with:
- Consent gate: all calls are no-ops when consented=False
- OSS/SaaS dual-mode: $process_person_profile is set to False in OSS mode;
set_person_properties and group_identify are SaaS-only
- Common properties: app_mode, is_feature_env added to every event
- Feature-env distinct_id prefix: FEATURE_ prefix for staging/feature envs
- SDK error isolation: all exceptions are caught and logged, never raised
This module must NOT import from enterprise/. It receives all configuration
via constructor args.
"""
from typing import Any
from posthog import Posthog
from openhands.core.logger import openhands_logger as logger
from openhands.server.types import AppMode
class AnalyticsService:
"""Server-side analytics service backed by PostHog.
Args:
api_key: PostHog project API key. Pass an empty string to disable.
host: PostHog ingest host URL.
app_mode: AppMode.OPENHANDS (OSS) or AppMode.SAAS.
is_feature_env: True when running in a feature/staging environment.
"""
def __init__(
self,
api_key: str,
host: str,
app_mode: AppMode,
is_feature_env: bool,
) -> None:
self._app_mode = app_mode
self._is_feature_env = is_feature_env
self._client: Posthog = Posthog(
project_api_key=api_key,
host=host,
disabled=not api_key,
)
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def capture(
self,
distinct_id: str,
event: str,
properties: dict[str, Any] | None = None,
org_id: str | None = None,
session_id: str | None = None,
consented: bool = True,
) -> None:
"""Capture a server-side event.
Consent gate: returns immediately when consented=False.
Common properties (app_mode, is_feature_env, and optionally org_id /
$session_id / $process_person_profile) are merged with caller-provided
properties before forwarding to PostHog.
"""
if not consented:
return
merged = self._common_properties(org_id=org_id, session_id=session_id)
if properties:
merged.update(properties)
try:
self._client.capture(
distinct_id=self._distinct_id(distinct_id),
event=event,
properties=merged,
)
except Exception:
logger.exception('AnalyticsService.capture failed')
def set_person_properties(
self,
distinct_id: str,
properties: dict[str, Any],
consented: bool = True,
) -> None:
"""Set person properties in PostHog (SaaS-only).
No-op in OSS mode or when consented=False.
"""
if not consented:
return
if self._app_mode != AppMode.SAAS:
return
try:
self._client.set(
distinct_id=self._distinct_id(distinct_id),
properties=properties,
)
except Exception:
logger.exception('AnalyticsService.set_person_properties failed')
def group_identify(
self,
group_type: str,
group_key: str,
properties: dict[str, Any],
distinct_id: str | None = None,
consented: bool = True,
) -> None:
"""Associate a group with properties (SaaS-only).
No-op in OSS mode or when consented=False.
"""
if not consented:
return
if self._app_mode != AppMode.SAAS:
return
try:
kwargs: dict[str, Any] = {
'group_type': group_type,
'group_key': group_key,
'properties': properties,
}
if distinct_id is not None:
kwargs['distinct_id'] = self._distinct_id(distinct_id)
self._client.group_identify(**kwargs)
except Exception:
logger.exception('AnalyticsService.group_identify failed')
def shutdown(self) -> None:
"""Flush and shut down the PostHog client.
Safe to call multiple times. SDK errors are logged, not raised.
"""
try:
self._client.shutdown()
except Exception:
logger.exception('AnalyticsService.shutdown failed')
# ------------------------------------------------------------------
# Private helpers
# ------------------------------------------------------------------
def _distinct_id(self, user_id: str) -> str:
"""Return the PostHog distinct_id for the given user.
In feature/staging environments, prefixes with 'FEATURE_' to keep
test traffic separate from production profiles.
"""
if self._is_feature_env:
return f'FEATURE_{user_id}'
return user_id
def _common_properties(
self,
org_id: str | None = None,
session_id: str | None = None,
) -> dict[str, Any]:
"""Build the base property dict included on every event."""
props: dict[str, Any] = {
'app_mode': self._app_mode.value,
'is_feature_env': self._is_feature_env,
}
if org_id is not None:
props['org_id'] = org_id
if session_id is not None:
props['$session_id'] = session_id
# PostHog person profiles are not useful in OSS mode (no user accounts)
if self._app_mode != AppMode.SAAS:
props['$process_person_profile'] = False
return props

View File

@@ -0,0 +1,39 @@
"""OSS install ID utility.
Provides a stable distinct_id for OSS installations by persisting a UUID
to a file in the persistence directory.
"""
import uuid
from pathlib import Path
def get_or_create_install_id(persistence_dir: Path) -> str:
"""Return a stable install UUID for an OSS installation.
On first call, generates a new UUID4, writes it to
``{persistence_dir}/analytics_id.txt``, and returns it.
On subsequent calls, reads and returns the stored UUID.
On any IOError (e.g., read-only filesystem), returns an ephemeral UUID
without crashing — the caller still gets a usable ID.
"""
id_file = persistence_dir / 'analytics_id.txt'
try:
if id_file.exists():
stored = id_file.read_text().strip()
if stored:
return stored
except OSError:
pass
new_id = str(uuid.uuid4())
try:
id_file.write_text(new_id)
except OSError:
# File system is not writable — return ephemeral UUID
pass
return new_id

View File

@@ -13,6 +13,7 @@ from pydantic import SecretStr
from openhands import tools # type: ignore[attr-defined]
from openhands.agent_server.models import ConversationInfo, Success
from openhands.analytics import analytics_constants, get_analytics_service
from openhands.app_server.app_conversation.app_conversation_info_service import (
AppConversationInfoService,
)
@@ -55,6 +56,29 @@ jwt_dependency = depends_jwt_service()
_logger = logging.getLogger(__name__)
def _classify_error_type(error_message: str | None) -> str:
"""Classify conversation error into broad categories for dashboard filtering.
Categories: budget_exceeded, model_error, runtime_error, timeout, user_cancelled, unknown.
Uses best-effort string matching per CONTEXT.md decision.
"""
if not error_message:
return 'unknown'
msg_lower = error_message.lower()
if 'budget' in msg_lower or 'budgetexceeded' in msg_lower:
return 'budget_exceeded'
if 'timeout' in msg_lower or 'timed out' in msg_lower:
return 'timeout'
if 'cancel' in msg_lower:
return 'user_cancelled'
if any(
kw in msg_lower
for kw in ('model', 'llm', 'api key', 'rate limit', 'authentication')
):
return 'model_error'
return 'runtime_error'
async def valid_sandbox(
user_context: UserContext = Depends(as_admin),
session_api_key: str = Depends(
@@ -132,6 +156,38 @@ async def on_conversation_update(
app_conversation_info
)
# Analytics: conversation created
try:
analytics = get_analytics_service()
if analytics and sandbox_info.created_by_user_id:
from enterprise.storage.user_store import UserStore
user_obj = await UserStore.get_user_by_id(sandbox_info.created_by_user_id)
if user_obj:
consented = user_obj.user_consents_to_analytics is True
org_id = (
str(user_obj.current_org_id) if user_obj.current_org_id else None
)
analytics.capture(
distinct_id=sandbox_info.created_by_user_id,
event=analytics_constants.CONVERSATION_CREATED,
properties={
'conversation_id': str(conversation_info.id),
'trigger': existing.trigger.value if existing.trigger else None,
'llm_model': (
conversation_info.agent.llm.model
if conversation_info.agent and conversation_info.agent.llm
else None
),
'agent_type': 'default',
'has_repository': existing.selected_repository is not None,
},
org_id=org_id,
consented=consented,
)
except Exception:
_logger.exception('analytics:conversation_created:failed')
return Success()
@@ -161,6 +217,187 @@ async def on_event(
event, conversation_id
)
# Analytics: conversation terminal state detection
for event in events:
if (
isinstance(event, ConversationStateUpdateEvent)
and event.key == 'execution_status'
):
try:
exec_status = ConversationExecutionStatus(event.value)
if exec_status.is_terminal():
analytics = get_analytics_service()
if analytics and sandbox_info.created_by_user_id:
from enterprise.storage.user_store import UserStore
user_obj = await UserStore.get_user_by_id(
sandbox_info.created_by_user_id
)
if user_obj:
consented = user_obj.user_consents_to_analytics is True
org_id = (
str(user_obj.current_org_id)
if user_obj.current_org_id
else None
)
# Extract metrics from stored conversation info (updated by process_stats_event above)
metrics = app_conversation_info.metrics
accumulated_cost = (
metrics.accumulated_cost if metrics else None
)
prompt_tokens = (
metrics.accumulated_token_usage.prompt_tokens
if metrics and metrics.accumulated_token_usage
else None
)
completion_tokens = (
metrics.accumulated_token_usage.completion_tokens
if metrics and metrics.accumulated_token_usage
else None
)
is_error = exec_status in (
ConversationExecutionStatus.ERROR,
ConversationExecutionStatus.STUCK,
)
if is_error:
# Look for the last error info in events batch
error_message = None
for ev in events:
if (
isinstance(ev, ConversationStateUpdateEvent)
and ev.key == 'last_error'
):
error_message = (
str(ev.value)[:500]
if ev.value
else None
)
error_type = _classify_error_type(error_message)
# BIZZ-06: conversation errored
analytics.capture(
distinct_id=sandbox_info.created_by_user_id,
event=analytics_constants.CONVERSATION_ERRORED,
properties={
'conversation_id': str(conversation_id),
'error_type': error_type,
'error_message': error_message,
'llm_model': app_conversation_info.llm_model,
'turn_count': None, # Not derivable from MetricsSnapshot alone
'terminal_state': exec_status.value,
},
org_id=org_id,
consented=consented,
)
# BIZZ-03: credit limit reached (fires alongside conversation errored)
if error_type == 'budget_exceeded':
analytics.capture(
distinct_id=sandbox_info.created_by_user_id,
event=analytics_constants.CREDIT_LIMIT_REACHED,
properties={
'conversation_id': str(conversation_id),
'credit_balance': None, # Not available in webhook context
'llm_model': app_conversation_info.llm_model,
},
org_id=org_id,
consented=consented,
)
else:
# BIZZ-05: conversation finished (includes FINISHED, STOPPED, etc.)
analytics.capture(
distinct_id=sandbox_info.created_by_user_id,
event=analytics_constants.CONVERSATION_FINISHED,
properties={
'conversation_id': str(conversation_id),
'terminal_state': exec_status.value,
'turn_count': None, # Not derivable from MetricsSnapshot alone
'accumulated_cost_usd': accumulated_cost,
'prompt_tokens': prompt_tokens,
'completion_tokens': completion_tokens,
'llm_model': app_conversation_info.llm_model,
'trigger': app_conversation_info.trigger.value
if app_conversation_info.trigger
else None,
},
org_id=org_id,
consented=consented,
)
# ACTV-01: user activated (first finished conversation only)
if (
exec_status
== ConversationExecutionStatus.FINISHED
):
try:
import uuid as _uuid
from datetime import datetime, timezone
from sqlalchemy import func
from sqlalchemy import select as sa_select
from enterprise.storage.database import (
a_session_maker,
)
from enterprise.storage.stored_conversation_metadata_saas import (
StoredConversationMetadataSaas,
)
user_uuid = _uuid.UUID(
sandbox_info.created_by_user_id
)
async with a_session_maker() as act_session:
count_result = await act_session.execute(
sa_select(func.count()).where(
StoredConversationMetadataSaas.user_id
== user_uuid,
StoredConversationMetadataSaas.conversation_id
!= str(conversation_id),
)
)
prior_count = count_result.scalar()
if prior_count == 0:
tos_ts = user_obj.accepted_tos
if tos_ts is not None:
if tos_ts.tzinfo is None:
tos_ts = tos_ts.replace(
tzinfo=timezone.utc
)
time_to_activate_seconds = (
datetime.now(timezone.utc)
- tos_ts
).total_seconds()
else:
time_to_activate_seconds = None
analytics.capture(
distinct_id=sandbox_info.created_by_user_id,
event=analytics_constants.USER_ACTIVATED,
properties={
'conversation_id': str(
conversation_id
),
'time_to_activate_seconds': time_to_activate_seconds,
'llm_model': app_conversation_info.llm_model,
'trigger': app_conversation_info.trigger.value
if app_conversation_info.trigger
else None,
},
org_id=org_id,
consented=consented,
)
except Exception:
_logger.exception(
'analytics:user_activated:failed'
)
except Exception:
_logger.exception('analytics:conversation_terminal:failed')
asyncio.create_task(
_run_callbacks_in_bg_and_close(
conversation_id, app_conversation_info.created_by_user_id, events

View File

@@ -24,6 +24,7 @@ from openhands.server.user_auth import (
get_provider_tokens,
get_secrets,
get_secrets_store,
get_user_id,
)
from openhands.storage.data_models.secrets import Secrets
from openhands.storage.data_models.settings import Settings
@@ -111,6 +112,7 @@ async def store_provider_tokens(
provider_info: POSTProviderModel,
secrets_store: SecretsStore = Depends(get_secrets_store),
provider_tokens: PROVIDER_TOKEN_TYPE | None = Depends(get_provider_tokens),
user_id: str | None = Depends(get_user_id),
) -> JSONResponse:
provider_err_msg = await check_provider_tokens(provider_info, provider_tokens)
if provider_err_msg:
@@ -147,6 +149,39 @@ async def store_provider_tokens(
)
await secrets_store.store(updated_secrets)
# ACTV-02: git provider connected
try:
from openhands.analytics import analytics_constants, get_analytics_service
analytics = get_analytics_service()
if analytics and user_id and provider_info.provider_tokens:
from enterprise.storage.user_store import UserStore
user_obj = await UserStore.get_user_by_id(user_id)
if user_obj:
consented = user_obj.user_consents_to_analytics is True
org_id_str = (
str(user_obj.current_org_id)
if user_obj.current_org_id
else None
)
for (
provider_type,
token_value,
) in provider_info.provider_tokens.items():
if token_value.token: # Only fire for providers with actual token, not host-only updates
analytics.capture(
distinct_id=user_id,
event=analytics_constants.GIT_PROVIDER_CONNECTED,
properties={
'provider_type': provider_type.value,
},
org_id=org_id_str,
consented=consented,
)
except Exception:
logger.exception('analytics:git_provider_connected:failed')
return JSONResponse(
status_code=status.HTTP_200_OK,
content={'message': 'Git providers stored'},

View File

@@ -10,6 +10,7 @@ import uuid
from types import MappingProxyType
from typing import Any
from openhands.analytics import analytics_constants, get_analytics_service
from openhands.core.config.mcp_config import MCPConfig
from openhands.core.logger import openhands_logger as logger
from openhands.events.action.message import MessageAction
@@ -71,6 +72,41 @@ async def initialize_conversation(
)
await conversation_store.save_metadata(conversation_metadata)
# Analytics: conversation created (V0 best-effort — llm_model not available at this point)
try:
analytics = get_analytics_service()
if analytics and user_id:
from enterprise.storage.user_store import UserStore
user_obj = await UserStore.get_user_by_id(user_id)
if user_obj:
consented = user_obj.user_consents_to_analytics is True
org_id = (
str(user_obj.current_org_id)
if user_obj.current_org_id
else None
)
analytics.capture(
distinct_id=user_id,
event=analytics_constants.CONVERSATION_CREATED,
properties={
'conversation_id': conversation_id,
'trigger': (
conversation_trigger.value
if conversation_trigger
else None
),
'llm_model': None, # V0: llm_model not available at conversation init time
'agent_type': 'default',
'has_repository': selected_repository is not None,
},
org_id=org_id,
consented=consented,
)
except Exception:
logger.exception('analytics:conversation_created:v0:failed')
return conversation_metadata
conversation_metadata = await conversation_store.get_metadata(conversation_id)

View File

@@ -0,0 +1,477 @@
"""Tests for the AnalyticsService and related utilities."""
import uuid
from unittest.mock import MagicMock, patch
import pytest
from openhands.analytics import (
AnalyticsService,
get_analytics_service,
init_analytics_service,
)
from openhands.analytics.analytics_constants import (
CONVERSATION_CREATED,
CONVERSATION_ERRORED,
CONVERSATION_FINISHED,
CREDIT_LIMIT_REACHED,
CREDIT_PURCHASED,
GIT_PROVIDER_CONNECTED,
ONBOARDING_COMPLETED,
USER_ACTIVATED,
USER_LOGGED_IN,
USER_SIGNED_UP,
)
from openhands.analytics.analytics_service import AnalyticsService as DirectService
from openhands.analytics.oss_install_id import get_or_create_install_id
from openhands.server.types import AppMode
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
def reset_singleton():
"""Reset the module-level singleton before each test."""
import openhands.analytics as analytics_module
original = analytics_module._analytics_service
analytics_module._analytics_service = None
yield
analytics_module._analytics_service = original
@pytest.fixture
def mock_posthog():
"""Patch posthog.Posthog and return the mock instance."""
with patch(
'openhands.analytics.analytics_service.Posthog', autospec=True
) as MockClass:
mock_client = MagicMock()
MockClass.return_value = mock_client
yield MockClass, mock_client
@pytest.fixture
def oss_service(mock_posthog):
"""AnalyticsService in OSS mode."""
MockClass, mock_client = mock_posthog
service = DirectService(
api_key='test-key',
host='https://posthog.example.com',
app_mode=AppMode.OPENHANDS,
is_feature_env=False,
)
return service, mock_client
@pytest.fixture
def saas_service(mock_posthog):
"""AnalyticsService in SaaS mode."""
MockClass, mock_client = mock_posthog
service = DirectService(
api_key='test-key',
host='https://posthog.example.com',
app_mode=AppMode.SAAS,
is_feature_env=False,
)
return service, mock_client
@pytest.fixture
def feature_env_service(mock_posthog):
"""AnalyticsService in feature-env mode (OSS)."""
MockClass, mock_client = mock_posthog
service = DirectService(
api_key='test-key',
host='https://posthog.example.com',
app_mode=AppMode.OPENHANDS,
is_feature_env=True,
)
return service, mock_client
# ---------------------------------------------------------------------------
# Module-level singleton tests
# ---------------------------------------------------------------------------
class TestSingleton:
def test_get_analytics_service_returns_none_before_init(self):
"""get_analytics_service() returns None before init_analytics_service() is called."""
assert get_analytics_service() is None
def test_init_analytics_service_returns_instance(self, mock_posthog):
"""init_analytics_service() returns an AnalyticsService instance."""
service = init_analytics_service(
api_key='key',
host='https://posthog.example.com',
app_mode=AppMode.SAAS,
is_feature_env=False,
)
assert isinstance(service, AnalyticsService)
def test_get_analytics_service_returns_instance_after_init(self, mock_posthog):
"""get_analytics_service() returns the initialized instance."""
init_analytics_service(
api_key='key',
host='https://posthog.example.com',
app_mode=AppMode.SAAS,
is_feature_env=False,
)
result = get_analytics_service()
assert isinstance(result, AnalyticsService)
def test_init_analytics_service_stores_singleton(self, mock_posthog):
"""Calling init twice returns the same object from get_analytics_service."""
svc1 = init_analytics_service(
api_key='key',
host='https://posthog.example.com',
app_mode=AppMode.SAAS,
is_feature_env=False,
)
svc2 = get_analytics_service()
assert svc1 is svc2
# ---------------------------------------------------------------------------
# Consent gate tests
# ---------------------------------------------------------------------------
class TestConsentGate:
def test_capture_with_consented_false_makes_zero_posthog_calls(self, oss_service):
"""capture() with consented=False produces zero PostHog client calls."""
service, mock_client = oss_service
service.capture(
distinct_id='user123',
event='test event',
consented=False,
)
mock_client.capture.assert_not_called()
def test_capture_with_consented_true_makes_posthog_call(self, oss_service):
"""capture() with consented=True calls the PostHog client."""
service, mock_client = oss_service
service.capture(
distinct_id='user123',
event='test event',
consented=True,
)
mock_client.capture.assert_called_once()
def test_set_person_properties_with_consented_false_is_noop(self, saas_service):
"""set_person_properties() with consented=False does not call SDK."""
service, mock_client = saas_service
service.set_person_properties(
distinct_id='user123',
properties={'name': 'Alice'},
consented=False,
)
mock_client.set.assert_not_called()
def test_group_identify_with_consented_false_is_noop(self, saas_service):
"""group_identify() with consented=False does not call SDK."""
service, mock_client = saas_service
service.group_identify(
group_type='org',
group_key='org123',
properties={'name': 'Acme'},
consented=False,
)
mock_client.group_identify.assert_not_called()
# ---------------------------------------------------------------------------
# OSS vs SaaS mode tests
# ---------------------------------------------------------------------------
class TestOssSaasMode:
def test_capture_oss_mode_includes_process_person_profile_false(self, oss_service):
"""In OSS mode, captured events include '$process_person_profile': False."""
service, mock_client = oss_service
service.capture(
distinct_id='user123',
event='test event',
consented=True,
)
_, kwargs = mock_client.capture.call_args
props = kwargs.get('properties', {})
assert props.get('$process_person_profile') is False
def test_capture_saas_mode_does_not_include_process_person_profile(
self, saas_service
):
"""In SaaS mode, captured events do NOT include '$process_person_profile'."""
service, mock_client = saas_service
service.capture(
distinct_id='user123',
event='test event',
consented=True,
)
_, kwargs = mock_client.capture.call_args
props = kwargs.get('properties', {})
assert '$process_person_profile' not in props
def test_set_person_properties_oss_mode_is_noop(self, oss_service):
"""set_person_properties() in OSS mode does not call SDK."""
service, mock_client = oss_service
service.set_person_properties(
distinct_id='user123',
properties={'name': 'Alice'},
consented=True,
)
mock_client.set.assert_not_called()
def test_set_person_properties_saas_mode_calls_sdk(self, saas_service):
"""set_person_properties() in SaaS mode calls SDK when consented."""
service, mock_client = saas_service
service.set_person_properties(
distinct_id='user123',
properties={'name': 'Alice'},
consented=True,
)
mock_client.set.assert_called_once()
def test_group_identify_oss_mode_is_noop(self, oss_service):
"""group_identify() in OSS mode does not call SDK."""
service, mock_client = oss_service
service.group_identify(
group_type='org',
group_key='org123',
properties={'name': 'Acme'},
consented=True,
)
mock_client.group_identify.assert_not_called()
def test_group_identify_saas_mode_calls_sdk(self, saas_service):
"""group_identify() in SaaS mode calls SDK when consented."""
service, mock_client = saas_service
service.group_identify(
group_type='org',
group_key='org123',
properties={'name': 'Acme'},
consented=True,
)
mock_client.group_identify.assert_called_once()
# ---------------------------------------------------------------------------
# Common properties tests
# ---------------------------------------------------------------------------
class TestCommonProperties:
def test_capture_always_includes_app_mode(self, saas_service):
"""Every captured event includes app_mode in properties."""
service, mock_client = saas_service
service.capture(distinct_id='user123', event='test event', consented=True)
_, kwargs = mock_client.capture.call_args
props = kwargs.get('properties', {})
assert 'app_mode' in props
assert props['app_mode'] == AppMode.SAAS.value
def test_capture_always_includes_is_feature_env(self, saas_service):
"""Every captured event includes is_feature_env in properties."""
service, mock_client = saas_service
service.capture(distinct_id='user123', event='test event', consented=True)
_, kwargs = mock_client.capture.call_args
props = kwargs.get('properties', {})
assert 'is_feature_env' in props
assert props['is_feature_env'] is False
def test_capture_includes_org_id_when_provided(self, saas_service):
"""capture() includes org_id when provided."""
service, mock_client = saas_service
service.capture(
distinct_id='user123',
event='test event',
org_id='org-abc',
consented=True,
)
_, kwargs = mock_client.capture.call_args
props = kwargs.get('properties', {})
assert props.get('org_id') == 'org-abc'
def test_capture_omits_org_id_when_not_provided(self, saas_service):
"""capture() omits org_id when not provided."""
service, mock_client = saas_service
service.capture(distinct_id='user123', event='test event', consented=True)
_, kwargs = mock_client.capture.call_args
props = kwargs.get('properties', {})
assert 'org_id' not in props
def test_capture_includes_session_id_when_provided(self, saas_service):
"""capture() includes $session_id when session_id provided."""
service, mock_client = saas_service
service.capture(
distinct_id='user123',
event='test event',
session_id='sess-xyz',
consented=True,
)
_, kwargs = mock_client.capture.call_args
props = kwargs.get('properties', {})
assert props.get('$session_id') == 'sess-xyz'
def test_capture_omits_session_id_when_not_provided(self, saas_service):
"""capture() omits $session_id when not provided."""
service, mock_client = saas_service
service.capture(distinct_id='user123', event='test event', consented=True)
_, kwargs = mock_client.capture.call_args
props = kwargs.get('properties', {})
assert '$session_id' not in props
def test_capture_merges_caller_properties(self, saas_service):
"""capture() merges caller-provided properties with common ones."""
service, mock_client = saas_service
service.capture(
distinct_id='user123',
event='test event',
properties={'custom_prop': 'custom_val'},
consented=True,
)
_, kwargs = mock_client.capture.call_args
props = kwargs.get('properties', {})
assert props.get('custom_prop') == 'custom_val'
assert 'app_mode' in props # common props still present
# ---------------------------------------------------------------------------
# distinct_id / feature-env tests
# ---------------------------------------------------------------------------
class TestDistinctId:
def test_distinct_id_normal_mode_returns_raw_user_id(self, saas_service):
"""_distinct_id() returns raw user_id when is_feature_env=False."""
service, _ = saas_service
assert service._distinct_id('user123') == 'user123'
def test_distinct_id_feature_env_returns_prefixed_user_id(
self, feature_env_service
):
"""_distinct_id() returns 'FEATURE_{user_id}' when is_feature_env=True."""
service, _ = feature_env_service
assert service._distinct_id('user123') == 'FEATURE_user123'
def test_capture_uses_distinct_id_helper(self, feature_env_service):
"""capture() passes the prefixed distinct_id to the PostHog client."""
service, mock_client = feature_env_service
service.capture(
distinct_id='user123',
event='test event',
consented=True,
)
_, kwargs = mock_client.capture.call_args
assert kwargs.get('distinct_id') == 'FEATURE_user123'
# ---------------------------------------------------------------------------
# Shutdown tests
# ---------------------------------------------------------------------------
class TestShutdown:
def test_shutdown_calls_client_shutdown(self, saas_service):
"""shutdown() calls client.shutdown() without raising."""
service, mock_client = saas_service
service.shutdown()
mock_client.shutdown.assert_called_once()
def test_shutdown_logs_errors_without_raising(self, saas_service):
"""shutdown() catches SDK errors and does not propagate them."""
service, mock_client = saas_service
mock_client.shutdown.side_effect = RuntimeError('SDK error')
# Should not raise
service.shutdown()
# ---------------------------------------------------------------------------
# SDK error handling tests
# ---------------------------------------------------------------------------
class TestErrorHandling:
def test_capture_logs_sdk_errors_without_raising(self, saas_service):
"""capture() catches SDK errors and does not raise to caller."""
service, mock_client = saas_service
mock_client.capture.side_effect = RuntimeError('Network error')
# Should not raise
service.capture(distinct_id='user123', event='test event', consented=True)
# ---------------------------------------------------------------------------
# Event constants tests
# ---------------------------------------------------------------------------
class TestEventConstants:
def test_event_constants_are_lowercase_strings_with_spaces(self):
"""Event constants follow PostHog naming convention (lowercase, spaces)."""
for const in [
USER_LOGGED_IN,
USER_SIGNED_UP,
CONVERSATION_CREATED,
CONVERSATION_FINISHED,
CONVERSATION_ERRORED,
CREDIT_PURCHASED,
CREDIT_LIMIT_REACHED,
USER_ACTIVATED,
GIT_PROVIDER_CONNECTED,
ONBOARDING_COMPLETED,
]:
assert isinstance(const, str)
assert const == const.lower(), f'{const!r} is not lowercase'
assert ' ' in const, f'{const!r} does not contain spaces'
def test_user_logged_in_constant_value(self):
"""USER_LOGGED_IN has the correct value."""
assert USER_LOGGED_IN == 'user logged in'
def test_user_signed_up_constant_value(self):
"""USER_SIGNED_UP has the correct value."""
assert USER_SIGNED_UP == 'user signed up'
# ---------------------------------------------------------------------------
# OSS install ID tests
# ---------------------------------------------------------------------------
class TestOssInstallId:
def test_get_or_create_install_id_creates_file_on_first_call(self, tmp_path):
"""get_or_create_install_id() creates the analytics_id.txt file on first call."""
result = get_or_create_install_id(tmp_path)
assert (tmp_path / 'analytics_id.txt').exists()
assert result is not None
def test_get_or_create_install_id_returns_valid_uuid(self, tmp_path):
"""get_or_create_install_id() returns a valid UUID string."""
result = get_or_create_install_id(tmp_path)
parsed = uuid.UUID(result) # raises if invalid
assert str(parsed) == result
def test_get_or_create_install_id_returns_same_uuid_on_second_call(self, tmp_path):
"""get_or_create_install_id() returns the same UUID on subsequent calls."""
first = get_or_create_install_id(tmp_path)
second = get_or_create_install_id(tmp_path)
assert first == second
def test_get_or_create_install_id_returns_uuid_on_file_write_failure(
self, tmp_path
):
"""get_or_create_install_id() returns an ephemeral UUID when file write fails."""
read_only_dir = tmp_path / 'readonly'
read_only_dir.mkdir()
read_only_dir.chmod(0o444) # read-only
try:
result = get_or_create_install_id(read_only_dir)
# Should still return a valid UUID, not crash
parsed = uuid.UUID(result)
assert str(parsed) == result
finally:
read_only_dir.chmod(0o755) # restore for cleanup