Compare commits

...

22 Commits

Author SHA1 Message Date
Zamil Majdy
e489ba5b7e fix(copilot): disable input during submission and fix timeout logic
- Disable input when status='submitted' to prevent message spam
- Set stream start timeout to 30s (only detects backend down, doesn't affect tool execution)
- Once stream starts, tools can run indefinitely (timeout is cleared)
- Mini-game shows during long-running tool execution without timeout
2026-02-21 20:45:47 +07:00
Zamil Majdy
f069aa3ab3 fix(copilot): increase stream timeout from 12s to 60s
- Agent creation can take longer than 12 seconds
- Previous 12s timeout was causing 'Stream timed out' errors
- Increased to 60s to accommodate long-running tool execution
2026-02-21 20:39:48 +07:00
Zamil Majdy
e2f32eb99e fix(copilot): extract tool name from type field in ToolWrapper
- AI SDK's ToolUIPart doesn't have toolName as separate field
- Tool name is encoded in type field as 'tool-{name}'
- Extract it using substring(5) to remove 'tool-' prefix
- Update debug logging to show extracted toolName
- This fixes 'toolName: unknown' in console logs
2026-02-21 20:09:40 +07:00
Zamil Majdy
eead01f919 fix(copilot): prevent infinite refetch loop when backend errors
- Only invalidate session queries on successful completion (status='ready')
- Previously invalidated on both 'ready' and 'error' status
- When backend returned 500, error status triggered refetch which caused infinite loop
- Fixes spam of 'Let me check!' messages when backend is unavailable
2026-02-21 19:59:26 +07:00
Zamil Majdy
2bc6481522 fix(frontend): remove message prop from all ToolWrapper calls
ToolWrapper no longer accepts message prop. This was causing
TypeScript errors and preventing the component from rendering.

All ToolWrapper calls now only pass part and children props.
Fixes 11 TypeScript compilation errors.
2026-02-21 19:48:21 +07:00
Zamil Majdy
b4c3bbe4c4 fix(copilot): use providerMetadata for isLongRunning flag
The AI SDK strips unknown fields from tool-input-available events.
Use the standard providerMetadata field instead, which the SDK
preserves, to pass the isLongRunning flag to the frontend.

Backend changes:
- Change isLongRunning field to providerMetadata object
- Set providerMetadata: {isLongRunning: true} for long-running tools
- Add debug logging to verify flag is set

Frontend changes:
- Check part.providerMetadata.isLongRunning instead of part.isLongRunning
- Add console debug logging to verify detection

Tested programmatically - the complete flow works correctly.
2026-02-21 19:42:29 +07:00
Zamil Majdy
2447c30eff fix(frontend): simplify import paths in LongRunningToolDisplay
Address CodeRabbit review comment by using direct relative paths
instead of convoluted ../../tools/CreateAgent/../../components paths.
2026-02-21 19:26:52 +07:00
Zamil Majdy
04ef290273 fix(copilot): add isLongRunning flag directly to StreamToolInputAvailable
Instead of sending a separate custom event, add isLongRunning boolean
to the existing StreamToolInputAvailable event. This is much simpler
and works with the AI SDK without needing custom event handling.

Backend changes:
- Add isLongRunning field to StreamToolInputAvailable
- Check tool.is_long_running in response_adapter and set the flag
- Remove separate StreamLongRunningStart emission

Frontend changes:
- Check part.isLongRunning directly on the tool part
- Remove message prop from ToolWrapper (no longer needed)
- Simplify detection logic

This approach piggybacks on the existing tool-input-available event
that the AI SDK already recognizes and adds to message.parts.
2026-02-21 19:13:19 +07:00
Zamil Majdy
6a7cd84b26 fix(copilot): use AI SDK DataUIPart format for long-running event
Changed StreamLongRunningStart event type from "long-running-start" to
"data-long-running-start" to match the Vercel AI SDK's DataUIPart format.
This ensures the event is properly added to message.parts and can be
detected by the frontend.

Changes:
- Backend: Update event type to "data-long-running-start"
- Backend: Wrap toolCallId/toolName in a "data" object
- Frontend: Check for "data-long-running-start" type and access data.toolCallId

This follows the AI SDK protocol for custom data events.
2026-02-21 19:03:08 +07:00
Zamil Majdy
12d0a1f13b fix(copilot): emit StreamLongRunningStart event in SDK path
Add logic to detect long-running tools in the SDK execution path
and emit StreamLongRunningStart event to trigger UI feedback display.

Changes:
- Import StreamLongRunningStart and get_tool
- Check if tool has is_long_running=True when StreamToolInputAvailable is received
- Yield StreamLongRunningStart event to notify frontend

This ensures the mini-game UI displays for long-running tools
like create_agent when using the SDK execution path.
2026-02-21 18:39:45 +07:00
Zamil Majdy
35a7f98ba7 fix(copilot): remove async delegation from SDK execution path
Remove the long-running callback that was spawning background tasks
for tools like create_agent and edit_agent in the SDK path. Tools
now run synchronously with heartbeats, matching the behavior of the
main service.py executor.

Changes:
- Remove _build_long_running_callback function
- Set long_running_callback=None in set_execution_context
- Remove unused imports (LongRunningCallback, OperationPendingResponse, etc.)
- Update tool supplement comment to reflect synchronous execution
- Remove accidentally committed sample.logs file

This fixes the "stream timed out" issue where tools were delegated to
background and session would stop prematurely.
2026-02-21 18:33:13 +07:00
Zamil Majdy
34b70d0673 refactor: remove 'mini-game' from comments, use generic 'UI feedback'
Replace all references to 'mini-game' in comments/docstrings with generic
'UI feedback' to allow for future UI variations.

Changes:
- base.py: 'shows mini-game in UI' → 'triggers long-running UI'
- create/edit/customize_agent.py: Remove '- show mini-game' from docstrings
- service.py: 'mini-game UI' → 'UI feedback'
- response_model.py: Remove '(like a mini-game)' example
- LongRunningToolDisplay: 'Displays a mini-game' → 'Displays UI feedback'
- ToolWrapper: Remove '(e.g., mini-game)' example

Keep implementation flexible for future UI changes.
2026-02-21 18:18:17 +07:00
Zamil Majdy
deb2bc4344 chore: remove accidentally committed sample.logs 2026-02-21 18:15:29 +07:00
Zamil Majdy
89785c88f9 feat(copilot): use stream event instead of hardcoded list for long-running tools
Replace hardcoded LONG_RUNNING_TOOLS list with event-based detection.
Frontend now listens for 'long-running-start' stream events from backend.

Changes:
- Update ToolWrapper to accept message prop and check for long-running-start events
- Pass message to all ToolWrapper instances in ChatMessagesContainer
- Remove long-running-tools.ts (hardcoded list)
- Check if any message part has type 'long-running-start' with matching toolCallId
- Update comments to be more generic ("UI feedback" instead of "mini-game")

Benefits:
- Single source of truth (backend is_long_running property)
- No list synchronization needed between backend and frontend
- More flexible - backend can decide at runtime
- Cleaner architecture using existing streaming infrastructure
2026-02-21 18:12:36 +07:00
Zamil Majdy
c08ba6a818 feat(copilot): add StreamLongRunningStart event for long-running tools
Replace hardcoded LONG_RUNNING_TOOLS list with stream-based communication.
Backend now yields StreamLongRunningStart event when a long-running tool begins.

Changes:
- Add LONG_RUNNING_START to ResponseType enum
- Add StreamLongRunningStart class to response_model.py
- Yield StreamLongRunningStart after StreamToolInputAvailable when tool.is_long_running
- Import get_tool in service.py

Frontend will listen for this event to show UI feedback (e.g., mini-game)
during long-running operations, eliminating the need for hardcoded tool lists.
2026-02-21 18:10:37 +07:00
Zamil Majdy
73b6ec3371 fix(copilot): remove async delegation from executor, use is_long_running only for frontend UI
The executor was still spawning background tasks when it saw is_long_running=True,
triggering the old async delegation pattern with 'operation is still running' messages.

This caused:
- Async delegation instead of synchronous execution with streaming
- Session timeouts waiting for async completion
- Mini-game not displaying because tool execution wasn't streaming properly

Fix:
- Remove async delegation code from _yield_tool_call (lines 1434-1586 in service.py)
- All tools now execute synchronously with heartbeats, regardless of is_long_running
- The is_long_running property is now ONLY used by frontend to show mini-game UI
- Update function docstring to reflect new behavior
- Remove unused imports: OperationStartedResponse, OperationPendingResponse, OperationInProgressResponse

The mini-game feature now works as intended:
1. Backend tools set is_long_running = True for UI display hint
2. Executor runs ALL tools synchronously with streaming
3. Frontend ToolWrapper detects is_long_running and shows mini-game during streaming
2026-02-21 18:06:10 +07:00
Zamil Majdy
95afa8c2f5 refactor(copilot): rename LongRunningToolWrapper to ToolWrapper
ToolWrapper is a better name since it wraps ALL tools, not just
long-running ones. It conditionally shows mini-game for long-running
tools based on LONG_RUNNING_TOOLS list.
2026-02-21 17:57:48 +07:00
Zamil Majdy
1de260c425 feat(copilot): make mini-game truly automatic for all long-running tools
- Create LongRunningToolWrapper component that wraps ALL tools
- Automatically detects if tool is long-running and shows mini-game
- Remove manual LongRunningToolDisplay from CreateAgent/EditAgent
- All tools (GenericTool, CustomizeAgent, etc.) now automatic
- No need to add mini-game to individual tool components

This makes the system completely generic - just mark is_long_running=True
in backend and frontend automatically shows mini-game!
2026-02-21 17:53:34 +07:00
Zamil Majdy
bfdc1edac1 feat(copilot): implement is_long_running property for automatic mini-game display
- Add is_long_running property to BaseTool for UI feedback control
- Mark create_agent, edit_agent, customize_agent as long-running tools
- Create LongRunningToolDisplay component for generic mini-game UI
- Clean up CreateAgent and EditAgent to use shared component
- Remove manual title configuration, use generic message
- Create LONG_RUNNING_TOOLS constant for frontend reference

This makes it easy to add new long-running tools without UI changes.
2026-02-21 17:45:48 +07:00
Zamil Majdy
eef394683a test(copilot): fix agent generator tests after removing operation_id/task_id
Update test assertions to match new function signatures after removing
operation_id and task_id parameters from generate_agent_external and
generate_agent_patch_external.

Fixes:
- TestGenerateAgent::test_calls_external_service
- TestGenerateAgentPatch::test_calls_external_service
2026-02-21 16:19:08 +07:00
Zamil Majdy
66c241644f refactor(copilot): remove async delegation dead code from agent generation
Remove all dead code related to the async processing delegation pattern
that is no longer needed after removing the is_long_running hack:

- Remove `_operation_id` and `_task_id` parameter extraction
- Remove passing these params to generate_agent/generate_agent_patch
- Remove `status: "accepted"` checks and AsyncProcessingResponse returns
- Remove AsyncProcessingResponse class definition from models.py
- Remove operation_id/task_id params from agent_generator functions:
  - generate_agent() and generate_agent_external()
  - generate_agent_patch() and generate_agent_patch_external()
  - generate_agent_dummy() and generate_agent_patch_dummy()
- Remove 202 Accepted handling for async processing

This cleanup removes 126 lines of code that was supporting the old
async delegation workflow.
2026-02-21 07:22:49 +07:00
Zamil Majdy
f25c2d1e6a fix(copilot): remove is_long_running hack from agent generation tools
Remove the `is_long_running = True` override from create_agent,
edit_agent, and customize_agent tools. Now that CoPilot runs in the
executor service (which already handles background execution), the
async delegation pattern is unnecessary.

This fixes the issue where agent generation completion messages
never appeared in chat because the code was exiting early expecting
an external Redis Stream completion that never came.

The tools now execute synchronously in the CoPilot executor and
stream completion messages back to chat immediately.

Fixes: Agent generation completion not showing in chat
2026-02-21 07:10:18 +07:00
24 changed files with 780 additions and 602 deletions

View File

@@ -0,0 +1,572 @@
2026-02-21 20:31:19,811 INFO Initializing LaunchDarkly Client 9.15.0
2026-02-21 20:31:19,812 INFO Starting event processor
2026-02-21 20:31:19,812 INFO Starting StreamingUpdateProcessor connecting to uri: https://stream.launchdarkly.com/all
2026-02-21 20:31:19,812 INFO Waiting up to 5 seconds for LaunchDarkly client to initialize...
2026-02-21 20:31:19,812 INFO Connecting to stream at https://stream.launchdarkly.com/all
2026-02-21 20:31:20,051 INFO StreamingUpdateProcessor initialized ok.
2026-02-21 20:31:20,051 INFO Started LaunchDarkly Client: OK
2026-02-21 20:31:20,051 INFO LaunchDarkly client initialized successfully
2026-02-21 20:31:21,578 WARNING Provider LINEAR implements OAuth but the required env vars LINEAR_CLIENT_ID and LINEAR_CLIENT_SECRET are not both set
2026-02-21 20:31:21,623 WARNING Authentication error: Langfuse client initialized without public_key. Client will be disabled. Provide a public_key parameter or set LANGFUSE_PUBLIC_KEY environment variable. 
2026-02-21 20:31:21,796 INFO Metrics endpoint exposed at /metrics for external-api
2026-02-21 20:31:21,800 INFO Metrics endpoint exposed at /metrics for rest-api
2026-02-21 20:31:21,881 INFO Metrics endpoint exposed at /metrics for websocket-server
2026-02-21 20:31:21,913 WARNING Postmark server API token not found, email sending disabled
2026-02-21 20:31:21,956 INFO [DatabaseManager] started with PID 6089
2026-02-21 20:31:21,958 INFO [Scheduler] started with PID 6090
2026-02-21 20:31:21,959 INFO [NotificationManager] started with PID 6091
2026-02-21 20:31:21,960 INFO [WebsocketServer] started with PID 6092
2026-02-21 20:31:21,961 INFO [AgentServer] started with PID 6093
2026-02-21 20:31:21,962 INFO [ExecutionManager] started with PID 6094
2026-02-21 20:31:21,963 INFO [CoPilotExecutor] Starting...
2026-02-21 20:31:21,963 INFO [CoPilotExecutor] Pod assigned executor_id: fb7d76b3-8dc3-40a4-947e-a93bfad207da
2026-02-21 20:31:21,963 INFO [CoPilotExecutor] Spawn max-5 workers...
2026-02-21 20:31:21,970 INFO [PID-6048|THREAD-77685505|CoPilotExecutor|RabbitMQ-124e33d7-4877-4745-9778-6b6b06de92d2] Acquiring connection started...
2026-02-21 20:31:21,971 INFO [PID-6048|THREAD-77685506|CoPilotExecutor|RabbitMQ-124e33d7-4877-4745-9778-6b6b06de92d2] Acquiring connection started...
2026-02-21 20:31:21,973 INFO Pika version 1.3.2 connecting to ('::1', 5672, 0, 0)
2026-02-21 20:31:21,973 INFO Pika version 1.3.2 connecting to ('::1', 5672, 0, 0)
2026-02-21 20:31:21,974 INFO Socket connected: <socket.socket fd=30, family=30, type=1, proto=6, laddr=('::1', 55999, 0, 0), raddr=('::1', 5672, 0, 0)>
2026-02-21 20:31:21,975 INFO Socket connected: <socket.socket fd=29, family=30, type=1, proto=6, laddr=('::1', 55998, 0, 0), raddr=('::1', 5672, 0, 0)>
2026-02-21 20:31:21,975 INFO Streaming transport linked up: (<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120f5eba0>, _StreamingProtocolShim: <SelectConnection PROTOCOL transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120f5eba0> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>).
2026-02-21 20:31:21,976 INFO Streaming transport linked up: (<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120fa0410>, _StreamingProtocolShim: <SelectConnection PROTOCOL transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120fa0410> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>).
2026-02-21 20:31:21,990 INFO AMQPConnector - reporting success: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120fa0410> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:21,991 INFO AMQPConnectionWorkflow - reporting success: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120fa0410> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:21,991 INFO AMQPConnector - reporting success: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120f5eba0> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:21,991 INFO Connection workflow succeeded: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120fa0410> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:21,991 INFO AMQPConnectionWorkflow - reporting success: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120f5eba0> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:21,991 INFO Created channel=1
2026-02-21 20:31:21,992 INFO Connection workflow succeeded: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x120f5eba0> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:21,992 INFO Created channel=1
2026-02-21 20:31:22,005 INFO [PID-6048|THREAD-77685505|CoPilotExecutor|RabbitMQ-124e33d7-4877-4745-9778-6b6b06de92d2] Acquiring connection completed successfully.
2026-02-21 20:31:22,005 INFO [PID-6048|THREAD-77685506|CoPilotExecutor|RabbitMQ-124e33d7-4877-4745-9778-6b6b06de92d2] Acquiring connection completed successfully.
2026-02-21 20:31:22,007 INFO [CoPilotExecutor] Starting to consume cancel messages...
2026-02-21 20:31:22,008 INFO [CoPilotExecutor] Starting to consume run messages...
2026-02-21 20:31:23,199 INFO Initializing LaunchDarkly Client 9.15.0
2026-02-21 20:31:23,201 INFO Starting event processor
2026-02-21 20:31:23,202 INFO Starting StreamingUpdateProcessor connecting to uri: https://stream.launchdarkly.com/all
2026-02-21 20:31:23,202 INFO Waiting up to 5 seconds for LaunchDarkly client to initialize...
2026-02-21 20:31:23,202 INFO Connecting to stream at https://stream.launchdarkly.com/all
2026-02-21 20:31:23,331 INFO StreamingUpdateProcessor initialized ok.
2026-02-21 20:31:23,331 INFO Started LaunchDarkly Client: OK
2026-02-21 20:31:23,332 INFO LaunchDarkly client initialized successfully
2026-02-21 20:31:23,891 INFO Initializing LaunchDarkly Client 9.15.0
2026-02-21 20:31:23,892 INFO Starting event processor
2026-02-21 20:31:23,893 INFO Starting StreamingUpdateProcessor connecting to uri: https://stream.launchdarkly.com/all
2026-02-21 20:31:23,893 INFO Waiting up to 5 seconds for LaunchDarkly client to initialize...
2026-02-21 20:31:23,893 INFO Connecting to stream at https://stream.launchdarkly.com/all
2026-02-21 20:31:23,946 INFO Initializing LaunchDarkly Client 9.15.0
2026-02-21 20:31:23,947 INFO Starting event processor
2026-02-21 20:31:23,947 INFO Starting StreamingUpdateProcessor connecting to uri: https://stream.launchdarkly.com/all
2026-02-21 20:31:23,947 INFO Waiting up to 5 seconds for LaunchDarkly client to initialize...
2026-02-21 20:31:23,948 INFO Connecting to stream at https://stream.launchdarkly.com/all
2026-02-21 20:31:24,017 INFO StreamingUpdateProcessor initialized ok.
2026-02-21 20:31:24,017 INFO Started LaunchDarkly Client: OK
2026-02-21 20:31:24,017 INFO LaunchDarkly client initialized successfully
2026-02-21 20:31:24,065 INFO StreamingUpdateProcessor initialized ok.
2026-02-21 20:31:24,065 INFO Started LaunchDarkly Client: OK
2026-02-21 20:31:24,065 INFO LaunchDarkly client initialized successfully
2026-02-21 20:31:24,707 INFO [NotificationManager] Starting...
2026-02-21 20:31:24,750 INFO Metrics endpoint exposed at /metrics for NotificationManager
2026-02-21 20:31:24,754 INFO [PID-6091|THREAD-77685702|NotificationManager|FastAPI server-d17271ed-e3a2-4e93-900b-a0d3bd2b8100] Running FastAPI server started...
2026-02-21 20:31:24,755 INFO [NotificationManager] Starting RPC server at http://localhost:8007
2026-02-21 20:31:24,756 INFO [NotificationManager] [NotificationManager] ⏳ Configuring RabbitMQ...
2026-02-21 20:31:24,757 INFO [PID-6091|THREAD-77685703|NotificationManager|AsyncRabbitMQ-7963c91c-c443-4479-a55e-5e9a8d7d942d] Acquiring async connection started...
2026-02-21 20:31:24,775 INFO Started server process [6091]
2026-02-21 20:31:24,775 INFO Waiting for application startup.
2026-02-21 20:31:24,776 INFO Application startup complete.
2026-02-21 20:31:24,777 ERROR [Errno 48] error while attempting to bind on address ('::1', 8007, 0, 0): [errno 48] address already in use
2026-02-21 20:31:24,781 INFO Waiting for application shutdown.
2026-02-21 20:31:24,781 INFO [NotificationManager] ✅ FastAPI has finished
2026-02-21 20:31:24,782 INFO Application shutdown complete.
2026-02-21 20:31:24,783 INFO [NotificationManager] 🛑 Shared event loop stopped
2026-02-21 20:31:24,783 INFO [NotificationManager] 🧹 Running cleanup
2026-02-21 20:31:24,783 INFO [NotificationManager] ⏳ Disconnecting RabbitMQ...
Process NotificationManager:
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
self.run()
~~~~~~~~^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/backend/util/process.py", line 83, in execute_run_command
self.cleanup()
~~~~~~~~~~~~^^
File "/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/backend/notifications/notifications.py", line 1094, in cleanup
self.run_and_wait(self.rabbitmq_service.disconnect())
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/backend/util/service.py", line 136, in run_and_wait
return asyncio.run_coroutine_threadsafe(coro, self.shared_event_loop).result()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/tasks.py", line 1003, in run_coroutine_threadsafe
loop.call_soon_threadsafe(callback)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 873, in call_soon_threadsafe
self._check_closed()
~~~~~~~~~~~~~~~~~~^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 551, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/multiprocessing/process.py:327: RuntimeWarning: coroutine 'AsyncRabbitMQ.disconnect' was never awaited
traceback.print_exc()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
2026-02-21 20:31:24,846 INFO Initializing LaunchDarkly Client 9.15.0
2026-02-21 20:31:24,848 INFO Starting event processor
2026-02-21 20:31:24,848 INFO Starting StreamingUpdateProcessor connecting to uri: https://stream.launchdarkly.com/all
2026-02-21 20:31:24,849 INFO Waiting up to 5 seconds for LaunchDarkly client to initialize...
2026-02-21 20:31:24,849 INFO Connecting to stream at https://stream.launchdarkly.com/all
2026-02-21 20:31:24,857 INFO Initializing LaunchDarkly Client 9.15.0
2026-02-21 20:31:24,858 INFO Starting event processor
2026-02-21 20:31:24,858 INFO Starting StreamingUpdateProcessor connecting to uri: https://stream.launchdarkly.com/all
2026-02-21 20:31:24,858 INFO Waiting up to 5 seconds for LaunchDarkly client to initialize...
2026-02-21 20:31:24,858 INFO Connecting to stream at https://stream.launchdarkly.com/all
2026-02-21 20:31:24,862 INFO Initializing LaunchDarkly Client 9.15.0
2026-02-21 20:31:24,863 INFO Starting event processor
2026-02-21 20:31:24,864 INFO Starting StreamingUpdateProcessor connecting to uri: https://stream.launchdarkly.com/all
2026-02-21 20:31:24,864 INFO Waiting up to 5 seconds for LaunchDarkly client to initialize...
2026-02-21 20:31:24,864 INFO Connecting to stream at https://stream.launchdarkly.com/all
2026-02-21 20:31:24,966 INFO StreamingUpdateProcessor initialized ok.
2026-02-21 20:31:24,967 INFO Started LaunchDarkly Client: OK
2026-02-21 20:31:24,967 INFO LaunchDarkly client initialized successfully
2026-02-21 20:31:24,976 INFO StreamingUpdateProcessor initialized ok.
2026-02-21 20:31:24,976 INFO Started LaunchDarkly Client: OK
2026-02-21 20:31:24,976 INFO LaunchDarkly client initialized successfully
2026-02-21 20:31:24,989 INFO StreamingUpdateProcessor initialized ok.
2026-02-21 20:31:24,989 INFO Started LaunchDarkly Client: OK
2026-02-21 20:31:24,989 INFO LaunchDarkly client initialized successfully
2026-02-21 20:31:25,035 INFO Metrics endpoint exposed at /metrics for websocket-server
2026-02-21 20:31:25,036 INFO [WebsocketServer] Starting...
2026-02-21 20:31:25,036 INFO CORS allow origins: ['http://localhost:3000', 'http://127.0.0.1:3000']
2026-02-21 20:31:25,076 INFO Started server process [6092]
2026-02-21 20:31:25,076 INFO Waiting for application startup.
2026-02-21 20:31:25,077 INFO Application startup complete.
2026-02-21 20:31:25,077 INFO [PID-6092|THREAD-77685501|WebsocketServer|AsyncRedis-b6fb3c5c-0070-4c5c-90eb-922d4f2152c2] Acquiring connection started...
2026-02-21 20:31:25,077 INFO [PID-6092|THREAD-77685501|WebsocketServer|AsyncRedis-b6fb3c5c-0070-4c5c-90eb-922d4f2152c2] Acquiring connection started...
2026-02-21 20:31:25,078 ERROR [Errno 48] error while attempting to bind on address ('0.0.0.0', 8001): address already in use
2026-02-21 20:31:25,080 INFO Waiting for application shutdown.
2026-02-21 20:31:25,080 INFO Application shutdown complete.
2026-02-21 20:31:25,080 INFO Event broadcaster stopped
2026-02-21 20:31:25,081 WARNING [WebsocketServer] 🛑 Terminating because of SystemExit: 1
2026-02-21 20:31:25,081 INFO [WebsocketServer] 🧹 Running cleanup
2026-02-21 20:31:25,081 INFO [WebsocketServer] ✅ Cleanup done
2026-02-21 20:31:25,081 INFO [WebsocketServer] 🛑 Terminated
2026-02-21 20:31:25,915 INFO [DatabaseManager] Starting...
2026-02-21 20:31:25,947 INFO Metrics endpoint exposed at /metrics for DatabaseManager
2026-02-21 20:31:25,970 INFO [ExecutionManager] Starting...
2026-02-21 20:31:25,970 INFO [GraphExecutor] [ExecutionManager] 🆔 Pod assigned executor_id: 90ff5962-bdc8-456d-a864-01c5f4f199bd
2026-02-21 20:31:25,971 INFO [GraphExecutor] [ExecutionManager] ⏳ Spawn max-10 workers...
2026-02-21 20:31:25,973 INFO [Scheduler] Starting...
2026-02-21 20:31:25,971 WARNING [ExecutionManager] 🛑 Terminating because of OSError: [Errno 48] Address already in use
Traceback (most recent call last):
File "/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/backend/util/process.py", line 65, in execute_run_command
self.run()
~~~~~~~~^^
File "/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/backend/executor/manager.py", line 1554, in run
start_http_server(settings.config.execution_manager_port)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/prometheus_client/exposition.py", line 251, in start_wsgi_server
httpd = make_server(addr, port, app, TmpServer, handler_class=_SilentHandler)
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/wsgiref/simple_server.py", line 150, in make_server
server = server_class((host, port), handler_class)
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/socketserver.py", line 457, in __init__
self.server_bind()
~~~~~~~~~~~~~~~~^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/wsgiref/simple_server.py", line 50, in server_bind
HTTPServer.server_bind(self)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/server.py", line 136, in server_bind
socketserver.TCPServer.server_bind(self)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/socketserver.py", line 473, in server_bind
self.socket.bind(self.server_address)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 48] Address already in use
2026-02-21 20:31:25,978 INFO [ExecutionManager] 🧹 Running cleanup
2026-02-21 20:31:25,978 INFO [GraphExecutor] [ExecutionManager][on_graph_executor_stop 6094] 🧹 Starting graceful shutdown...
2026-02-21 20:31:25,978 INFO [PID-6094|THREAD-77685503|ExecutionManager|RabbitMQ-5b203f2b-8b80-46b1-8e47-481497e68a82] Acquiring connection started...
2026-02-21 20:31:25,980 INFO Pika version 1.3.2 connecting to ('::1', 5672, 0, 0)
2026-02-21 20:31:25,981 INFO Socket connected: <socket.socket fd=14, family=30, type=1, proto=6, laddr=('::1', 56040, 0, 0), raddr=('::1', 5672, 0, 0)>
2026-02-21 20:31:25,982 INFO Streaming transport linked up: (<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1316cd550>, _StreamingProtocolShim: <SelectConnection PROTOCOL transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1316cd550> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>).
2026-02-21 20:31:25,991 INFO AMQPConnector - reporting success: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1316cd550> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:25,991 INFO AMQPConnectionWorkflow - reporting success: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1316cd550> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:25,991 INFO Connection workflow succeeded: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1316cd550> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:25,991 INFO Created channel=1
2026-02-21 20:31:26,001 INFO [PID-6094|THREAD-77685503|ExecutionManager|RabbitMQ-5b203f2b-8b80-46b1-8e47-481497e68a82] Acquiring connection completed successfully.
2026-02-21 20:31:26,001 INFO [GraphExecutor] [ExecutionManager][on_graph_executor_stop 6094] ✅ Exec consumer has been signaled to stop
2026-02-21 20:31:26,001 INFO [GraphExecutor] [ExecutionManager][on_graph_executor_stop 6094] ✅ Executor shutdown completed
2026-02-21 20:31:26,001 INFO [GraphExecutor] [ExecutionManager][on_graph_executor_stop 6094] ✅ Released execution locks
2026-02-21 20:31:26,001 ERROR [GraphExecutor] [ExecutionManager][on_graph_executor_stop 6094] [run-consumer] ⚠️ Error disconnecting run client: <class 'RuntimeError'> cannot join thread before it is started 
2026-02-21 20:31:26,003 INFO [PID-6094|THREAD-77685503|ExecutionManager|RabbitMQ-5b203f2b-8b80-46b1-8e47-481497e68a82] Acquiring connection started...
2026-02-21 20:31:26,005 INFO Pika version 1.3.2 connecting to ('::1', 5672, 0, 0)
2026-02-21 20:31:26,005 INFO Socket connected: <socket.socket fd=20, family=30, type=1, proto=6, laddr=('::1', 56043, 0, 0), raddr=('::1', 5672, 0, 0)>
2026-02-21 20:31:26,006 INFO Streaming transport linked up: (<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1318e4cd0>, _StreamingProtocolShim: <SelectConnection PROTOCOL transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1318e4cd0> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>).
2026-02-21 20:31:26,009 INFO Metrics endpoint exposed at /metrics for Scheduler
2026-02-21 20:31:26,010 INFO AMQPConnector - reporting success: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1318e4cd0> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:26,010 INFO AMQPConnectionWorkflow - reporting success: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1318e4cd0> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:26,010 INFO Connection workflow succeeded: <SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncPlaintextTransport object at 0x1318e4cd0> params=<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>>
2026-02-21 20:31:26,011 INFO Created channel=1
2026-02-21 20:31:26,015 INFO [PID-6090|THREAD-77685897|Scheduler|FastAPI server-6caca9cc-c4c1-417f-8b83-d96f02472df9] Running FastAPI server started...
2026-02-21 20:31:26,016 INFO [Scheduler] Starting RPC server at http://localhost:8003
2026-02-21 20:31:26,016 INFO [PID-6094|THREAD-77685503|ExecutionManager|RabbitMQ-5b203f2b-8b80-46b1-8e47-481497e68a82] Acquiring connection completed successfully.
2026-02-21 20:31:26,016 ERROR [GraphExecutor] [ExecutionManager][on_graph_executor_stop 6094] [cancel-consumer] ⚠️ Error disconnecting run client: <class 'RuntimeError'> cannot join thread before it is started 
2026-02-21 20:31:26,019 INFO [GraphExecutor] [ExecutionManager][on_graph_executor_stop 6094] ✅ Finished GraphExec cleanup
2026-02-21 20:31:26,019 INFO [ExecutionManager] ✅ Cleanup done
2026-02-21 20:31:26,019 INFO [ExecutionManager] 🛑 Terminated
2026-02-21 20:31:26,188 INFO [PID-6089|THREAD-77685901|DatabaseManager|FastAPI server-7019e67b-30c1-4d08-a0ec-4f0175629d0e] Running FastAPI server started...
2026-02-21 20:31:26,189 INFO [DatabaseManager] Starting RPC server at http://localhost:8005
2026-02-21 20:31:26,197 INFO [DatabaseManager] ⏳ Connecting to Database...
2026-02-21 20:31:26,197 INFO [PID-6089|THREAD-77685902|DatabaseManager|Prisma-64fcde85-3de3-4783-b2c6-789775451cd0] Acquiring connection started...
2026-02-21 20:31:26,254 INFO [Scheduler] [APScheduler] Adding job tentatively -- it will be properly scheduled when the scheduler starts
2026-02-21 20:31:26,255 INFO [Scheduler] [APScheduler] Adding job tentatively -- it will be properly scheduled when the scheduler starts
2026-02-21 20:31:26,255 INFO [Scheduler] [APScheduler] Adding job tentatively -- it will be properly scheduled when the scheduler starts
2026-02-21 20:31:26,255 INFO [Scheduler] [APScheduler] Adding job tentatively -- it will be properly scheduled when the scheduler starts
2026-02-21 20:31:26,255 INFO [Scheduler] [APScheduler] Adding job tentatively -- it will be properly scheduled when the scheduler starts
2026-02-21 20:31:26,255 INFO [Scheduler] [APScheduler] Adding job tentatively -- it will be properly scheduled when the scheduler starts
2026-02-21 20:31:26,256 INFO [Scheduler] [APScheduler] Adding job tentatively -- it will be properly scheduled when the scheduler starts
2026-02-21 20:31:26,346 INFO [PID-6089|THREAD-77685902|DatabaseManager|Prisma-64fcde85-3de3-4783-b2c6-789775451cd0] Acquiring connection completed successfully.
2026-02-21 20:31:26,346 INFO [DatabaseManager] ✅ Ready
2026-02-21 20:31:26,347 ERROR [Errno 48] error while attempting to bind on address ('::1', 8005, 0, 0): [errno 48] address already in use
2026-02-21 20:31:26,349 INFO [DatabaseManager] ⏳ Disconnecting Database...
2026-02-21 20:31:26,349 INFO [PID-6089|THREAD-77685902|DatabaseManager|Prisma-2397ec31-7da6-4598-a012-6c48f17ea97f] Releasing connection started...
2026-02-21 20:31:26,350 INFO [PID-6089|THREAD-77685902|DatabaseManager|Prisma-2397ec31-7da6-4598-a012-6c48f17ea97f] Releasing connection completed successfully.
2026-02-21 20:31:26,351 INFO [DatabaseManager] ✅ FastAPI has finished
2026-02-21 20:31:26,351 INFO [DatabaseManager] 🛑 Shared event loop stopped
2026-02-21 20:31:26,351 INFO [DatabaseManager] 🧹 Running cleanup
Process DatabaseManager:
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
self.run()
~~~~~~~~^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/backend/util/process.py", line 83, in execute_run_command
self.cleanup()
~~~~~~~~~~~~^^
File "/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/backend/util/service.py", line 153, in cleanup
self.shared_event_loop.call_soon_threadsafe(self.shared_event_loop.stop)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 873, in call_soon_threadsafe
self._check_closed()
~~~~~~~~~~~~~~~~~~^^
File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 551, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
2026-02-21 20:31:26,382 INFO [Scheduler] [APScheduler] Added job "process_weekly_summary" to job store "weekly_notifications"
2026-02-21 20:31:26,390 INFO [Scheduler] [APScheduler] Added job "report_late_executions" to job store "execution"
2026-02-21 20:31:26,392 INFO [Scheduler] [APScheduler] Added job "report_block_error_rates" to job store "execution"
2026-02-21 20:31:26,395 INFO [Scheduler] [APScheduler] Added job "cleanup_expired_files" to job store "execution"
2026-02-21 20:31:26,397 INFO [Scheduler] [APScheduler] Added job "cleanup_oauth_tokens" to job store "execution"
2026-02-21 20:31:26,399 INFO [Scheduler] [APScheduler] Added job "execution_accuracy_alerts" to job store "execution"
2026-02-21 20:31:26,401 INFO [Scheduler] [APScheduler] Added job "ensure_embeddings_coverage" to job store "execution"
2026-02-21 20:31:26,401 INFO [Scheduler] [APScheduler] Scheduler started
2026-02-21 20:31:26,402 INFO [Scheduler] Running embedding backfill on startup...
2026-02-21 20:31:26,440 WARNING Provider LINEAR implements OAuth but the required env vars LINEAR_CLIENT_ID and LINEAR_CLIENT_SECRET are not both set
2026-02-21 20:31:26,468 INFO [PID-6090|THREAD-77685499|Scheduler|AppService client-24942e64-d380-4d36-a245-5c41172e5293] Creating service client started...
2026-02-21 20:31:26,468 INFO [PID-6090|THREAD-77685499|Scheduler|AppService client-24942e64-d380-4d36-a245-5c41172e5293] Creating service client completed successfully.
2026-02-21 20:31:26,485 WARNING Authentication error: Langfuse client initialized without public_key. Client will be disabled. Provide a public_key parameter or set LANGFUSE_PUBLIC_KEY environment variable. 
2026-02-21 20:31:26,652 INFO Metrics endpoint exposed at /metrics for external-api
2026-02-21 20:31:26,655 INFO Metrics endpoint exposed at /metrics for rest-api
2026-02-21 20:31:26,735 INFO [AgentServer] Starting...
2026-02-21 20:31:26,745 INFO Started server process [6093]
2026-02-21 20:31:26,745 INFO Waiting for application startup.
2026-02-21 20:31:26,746 WARNING ⚠ JWT_SIGN_ALGORITHM is set to 'HS256', a symmetric shared-key signature algorithm. We highly recommend using an asymmetric algorithm such as ES256, because when leaked, a shared secret would allow anyone to forge valid tokens and impersonate users. More info: https://supabase.com/docs/guides/auth/signing-keys#choosing-the-right-signing-algorithm
2026-02-21 20:31:26,747 INFO [PID-6093|THREAD-77685502|AgentServer|Prisma-9d930243-0262-4697-b4af-e0bcbec281c4] Acquiring connection started...
2026-02-21 20:31:26,812 INFO [PID-6093|THREAD-77685502|AgentServer|Prisma-9d930243-0262-4697-b4af-e0bcbec281c4] Acquiring connection completed successfully.
2026-02-21 20:31:26,825 INFO Thread pool size set to 60 for sync endpoint/dependency performance
2026-02-21 20:31:26,825 INFO Successfully patched IntegrationCredentialsStore.get_all_creds
2026-02-21 20:31:26,825 INFO Syncing provider costs to blocks...
2026-02-21 20:31:27,576 WARNING Provider WORDPRESS implements OAuth but the required env vars WORDPRESS_CLIENT_ID and WORDPRESS_CLIENT_SECRET are not both set
2026-02-21 20:31:27,631 INFO Registered 1 custom costs for block FirecrawlExtractBlock
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/backend/blocks/exa/helpers.py:56: UserWarning: Field name "schema" in "SummarySettings" shadows an attribute in parent "BaseModel"
class SummarySettings(BaseModel):
2026-02-21 20:31:27,954 WARNING Provider AIRTABLE implements OAuth but the required env vars AIRTABLE_CLIENT_ID and AIRTABLE_CLIENT_SECRET are not both set
2026-02-21 20:31:29,238 INFO Successfully patched IntegrationCredentialsStore.get_all_creds
2026-02-21 20:31:29,397 WARNING Block WordPressCreatePostBlock credential input 'credentials' provider 'wordpress' has no authentication methods configured - Disabling
2026-02-21 20:31:29,397 WARNING Block WordPressGetAllPostsBlock credential input 'credentials' provider 'wordpress' has no authentication methods configured - Disabling
2026-02-21 20:31:29,465 INFO Synced 82 costs to 82 blocks
2026-02-21 20:31:29,466 WARNING Executing <Task pending name='Task-2' coro=<LifespanOn.main() running at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/lifespan/on.py:86> created at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/lifespan/on.py:51> took 2.654 seconds
2026-02-21 20:31:29,511 INFO [Scheduler] All content has embeddings, skipping backfill
2026-02-21 20:31:29,512 INFO [Scheduler] Running cleanup for orphaned embeddings (blocks/docs)...
2026-02-21 20:31:29,542 INFO [Scheduler] Cleanup completed: no orphaned embeddings found
2026-02-21 20:31:29,542 INFO [Scheduler] Startup embedding backfill complete: {'backfill': {'processed': 0, 'success': 0, 'failed': 0}, 'cleanup': {'deleted': 0}}
2026-02-21 20:31:29,553 INFO Started server process [6090]
2026-02-21 20:31:29,553 INFO Waiting for application startup.
2026-02-21 20:31:29,554 INFO Application startup complete.
2026-02-21 20:31:29,555 INFO Uvicorn running on http://localhost:8003 (Press CTRL+C to quit)
2026-02-21 20:31:31,074 INFO Migrating integration credentials for 0 users
2026-02-21 20:31:31,087 INFO Fixing LLM credential inputs on 0 nodes
2026-02-21 20:31:31,087 INFO Migrating LLM models
2026-02-21 20:31:31,107 INFO Migrated 0 node triggers to triggered presets
2026-02-21 20:31:31,107 INFO [PID-6093|THREAD-77685502|AgentServer|AsyncRedis-f8b888fc-8b03-4807-adfd-c93710c11c85] Acquiring connection started...
2026-02-21 20:31:31,114 INFO [PID-6093|THREAD-77685502|AgentServer|AsyncRedis-f8b888fc-8b03-4807-adfd-c93710c11c85] Acquiring connection completed successfully.
2026-02-21 20:31:31,115 INFO Created consumer group 'chat_consumers' on stream 'chat:completions'
2026-02-21 20:31:31,115 INFO Chat completion consumer started (consumer: consumer-2f92959a)
2026-02-21 20:31:31,116 INFO Application startup complete.
2026-02-21 20:31:31,117 INFO Uvicorn running on http://0.0.0.0:8006 (Press CTRL+C to quit)
2026-02-21 20:31:45,616 INFO 127.0.0.1:56174 - "GET /api/health HTTP/1.1" 404
2026-02-21 20:32:07,632 INFO 127.0.0.1:56317 - "GET /openapi.json HTTP/1.1" 200
2026-02-21 20:32:07,635 WARNING Executing <Task finished name='Task-7' coro=<RequestResponseCycle.run_asgi() done, defined at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:414> result=None created at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:295> took 0.346 seconds
2026-02-21 20:32:41,502 INFO 127.0.0.1:56681 - "POST /api/v2/chat/sessions HTTP/1.1" 404
2026-02-21 20:32:50,005 INFO 127.0.0.1:56736 - "GET /api/docs HTTP/1.1" 404
2026-02-21 20:33:10,267 INFO 127.0.0.1:56898 - "GET /openapi.json HTTP/1.1" 200
2026-02-21 20:33:28,399 INFO 127.0.0.1:56993 - "POST /api/chat/sessions HTTP/1.1" 401
2026-02-21 20:34:20,913 INFO 127.0.0.1:57313 - "GET /openapi.json HTTP/1.1" 200
2026-02-21 20:36:26,326 INFO Running job "report_late_executions (trigger: interval[0:05:00], next run at: 2026-02-21 13:36:26 UTC)" (scheduled at 2026-02-21 13:36:26.255260+00:00)
2026-02-21 20:36:26,333 INFO [PID-6090|THREAD-77695300|Scheduler|AppService client-24942e64-d380-4d36-a245-5c41172e5293] Creating service client started...
2026-02-21 20:36:26,336 INFO [PID-6090|THREAD-77695300|Scheduler|AppService client-24942e64-d380-4d36-a245-5c41172e5293] Creating service client completed successfully.
2026-02-21 20:36:26,336 INFO [PID-6090|THREAD-77695300|Scheduler|AppService client-24942e64-d380-4d36-a245-5c41172e5293] Creating service client started...
2026-02-21 20:36:26,340 INFO [PID-6090|THREAD-77695300|Scheduler|AppService client-24942e64-d380-4d36-a245-5c41172e5293] Creating service client completed successfully.
2026-02-21 20:36:26,439 WARNING Service communication: Retry attempt 1 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:36:27,802 WARNING Service communication: Retry attempt 2 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:36:30,362 WARNING Service communication: Retry attempt 3 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:36:34,885 WARNING Service communication: Retry attempt 4 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:36:43,438 WARNING Service communication: Retry attempt 5 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:36:59,905 WARNING Service communication: Retry attempt 6 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:37:12,581 WARNING Executing <Task pending name='Task-13' coro=<RequestResponseCycle.run_asgi() running at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:416> cb=[set.discard()] created at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:295> took 0.109 seconds
2026-02-21 20:37:12,767 INFO 127.0.0.1:58472 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:12,886 INFO 127.0.0.1:58469 - "GET /api/chat/sessions?limit=50 HTTP/1.1" 200
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/expressions/parser.py:72: PyparsingDeprecationWarning: 'enablePackrat' deprecated - use 'enable_packrat'
ParserElement.enablePackrat()
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/expressions/parser.py:85: PyparsingDeprecationWarning: 'escChar' argument is deprecated, use 'esc_char'
quoted_identifier = QuotedString('"', escChar="\\", unquoteResults=True)
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/expressions/parser.py:85: PyparsingDeprecationWarning: 'unquoteResults' argument is deprecated, use 'unquote_results'
quoted_identifier = QuotedString('"', escChar="\\", unquoteResults=True)
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/table/metadata.py:365: PydanticDeprecatedSince212: Using `@model_validator` with mode='after' on a classmethod is deprecated. Instead, use an instance method. See the documentation at https://docs.pydantic.dev/2.12/concepts/validators/#model-after-validator. Deprecated in Pydantic V2.12 to be removed in V3.0.
@model_validator(mode="after")
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/table/metadata.py:494: PydanticDeprecatedSince212: Using `@model_validator` with mode='after' on a classmethod is deprecated. Instead, use an instance method. See the documentation at https://docs.pydantic.dev/2.12/concepts/validators/#model-after-validator. Deprecated in Pydantic V2.12 to be removed in V3.0.
@model_validator(mode="after")
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/table/metadata.py:498: PydanticDeprecatedSince212: Using `@model_validator` with mode='after' on a classmethod is deprecated. Instead, use an instance method. See the documentation at https://docs.pydantic.dev/2.12/concepts/validators/#model-after-validator. Deprecated in Pydantic V2.12 to be removed in V3.0.
@model_validator(mode="after")
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/table/metadata.py:502: PydanticDeprecatedSince212: Using `@model_validator` with mode='after' on a classmethod is deprecated. Instead, use an instance method. See the documentation at https://docs.pydantic.dev/2.12/concepts/validators/#model-after-validator. Deprecated in Pydantic V2.12 to be removed in V3.0.
@model_validator(mode="after")
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/table/metadata.py:506: PydanticDeprecatedSince212: Using `@model_validator` with mode='after' on a classmethod is deprecated. Instead, use an instance method. See the documentation at https://docs.pydantic.dev/2.12/concepts/validators/#model-after-validator. Deprecated in Pydantic V2.12 to be removed in V3.0.
@model_validator(mode="after")
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/table/metadata.py:538: PydanticDeprecatedSince212: Using `@model_validator` with mode='after' on a classmethod is deprecated. Instead, use an instance method. See the documentation at https://docs.pydantic.dev/2.12/concepts/validators/#model-after-validator. Deprecated in Pydantic V2.12 to be removed in V3.0.
@model_validator(mode="after")
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/table/metadata.py:542: PydanticDeprecatedSince212: Using `@model_validator` with mode='after' on a classmethod is deprecated. Instead, use an instance method. See the documentation at https://docs.pydantic.dev/2.12/concepts/validators/#model-after-validator. Deprecated in Pydantic V2.12 to be removed in V3.0.
@model_validator(mode="after")
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/table/metadata.py:546: PydanticDeprecatedSince212: Using `@model_validator` with mode='after' on a classmethod is deprecated. Instead, use an instance method. See the documentation at https://docs.pydantic.dev/2.12/concepts/validators/#model-after-validator. Deprecated in Pydantic V2.12 to be removed in V3.0.
@model_validator(mode="after")
/Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/pyiceberg/table/metadata.py:550: PydanticDeprecatedSince212: Using `@model_validator` with mode='after' on a classmethod is deprecated. Instead, use an instance method. See the documentation at https://docs.pydantic.dev/2.12/concepts/validators/#model-after-validator. Deprecated in Pydantic V2.12 to be removed in V3.0.
@model_validator(mode="after")
2026-02-21 20:37:14,074 INFO 127.0.0.1:58470 - "GET /api/executions HTTP/1.1" 200
2026-02-21 20:37:14,081 WARNING Executing <Task finished name='Task-14' coro=<RequestResponseCycle.run_asgi() done, defined at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:414> result=None created at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:295> took 1.169 seconds
2026-02-21 20:37:15,102 WARNING Executing <Task pending name='Task-1' coro=<Server.serve() running at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/server.py:71> wait_for=<Future pending cb=[Task.task_wakeup()] created at /opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/tasks.py:713> cb=[run_until_complete.<locals>.done_cb()] created at /opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py:100> took 0.224 seconds
2026-02-21 20:37:17,085 INFO 127.0.0.1:58530 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:20,772 WARNING Executing <Task pending name='Task-1' coro=<Server.serve() running at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/server.py:71> wait_for=<Future pending cb=[Task.task_wakeup()] created at /opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/tasks.py:713> cb=[run_until_complete.<locals>.done_cb()] created at /opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py:100> took 0.261 seconds
2026-02-21 20:37:21,276 INFO 127.0.0.1:58568 - "GET /api/integrations/providers/system HTTP/1.1" 200
2026-02-21 20:37:21,309 WARNING Executing <Task finished name='Task-23' coro=<RequestResponseCycle.run_asgi() done, defined at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:414> result=None created at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:295> took 0.158 seconds
2026-02-21 20:37:21,329 INFO 127.0.0.1:58570 - "GET /api/integrations/providers HTTP/1.1" 200
2026-02-21 20:37:21,421 WARNING Executing <Task finished name='Task-24' coro=<RequestResponseCycle.run_asgi() done, defined at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:414> result=None created at /Users/majdyz/Code/AutoGPT/autogpt_platform/backend/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py:295> took 0.110 seconds
2026-02-21 20:37:22,406 INFO 127.0.0.1:58590 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:22,430 INFO 127.0.0.1:58588 - "GET /api/onboarding HTTP/1.1" 200
2026-02-21 20:37:22,453 INFO 127.0.0.1:58570 - "GET /api/executions HTTP/1.1" 200
2026-02-21 20:37:22,476 INFO Loaded session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from DB: has_messages=True, message_count=11, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool']
2026-02-21 20:37:22,485 INFO Cached session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from database
2026-02-21 20:37:22,510 INFO 127.0.0.1:58568 - "GET /api/library/agents?page=1&page_size=100 HTTP/1.1" 200
2026-02-21 20:37:22,515 INFO [GET_SESSION] session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, active_task=False, msg_count=11, last_role=tool
2026-02-21 20:37:22,524 INFO 127.0.0.1:58599 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f HTTP/1.1" 200
2026-02-21 20:37:22,535 INFO 127.0.0.1:58607 - "GET /api/chat/sessions?limit=50 HTTP/1.1" 200
2026-02-21 20:37:22,608 INFO 127.0.0.1:58568 - "GET /api/integrations/credentials HTTP/1.1" 200
2026-02-21 20:37:23,531 INFO 127.0.0.1:58568 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:25,612 INFO 127.0.0.1:58568 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:29,708 INFO 127.0.0.1:58671 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:29,975 WARNING Service communication: Retry attempt 7 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:37:34,125 INFO [TIMING] stream_chat_post STARTED, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, user=68383665-d3d9-41f3-b10c-fca0dc6080ed, message_len=36
2026-02-21 20:37:34,134 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=11, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool']
2026-02-21 20:37:34,135 INFO [TIMING] session validated in 10.6ms
2026-02-21 20:37:34,136 INFO [STREAM] Saving user message to session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f
2026-02-21 20:37:34,138 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=11, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool']
2026-02-21 20:37:34,168 INFO Saving 1 new messages to DB for session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f: roles=['user'], start_sequence=11
2026-02-21 20:37:34,201 INFO [STREAM] User message saved for session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f
2026-02-21 20:37:34,202 INFO [TIMING] create_task STARTED, task=bba63941-8048-4f39-9329-8568e5ebe9cd, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, user=68383665-d3d9-41f3-b10c-fca0dc6080ed
2026-02-21 20:37:34,202 INFO [TIMING] get_redis_async took 0.0ms
2026-02-21 20:37:34,205 INFO [TIMING] redis.hset took 2.9ms
2026-02-21 20:37:34,208 INFO [TIMING] create_task COMPLETED in 6.1ms; task=bba63941-8048-4f39-9329-8568e5ebe9cd, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f
2026-02-21 20:37:34,208 INFO [TIMING] create_task completed in 6.8ms
2026-02-21 20:37:34,210 INFO [PID-6093|THREAD-77685502|AgentServer|AsyncRabbitMQ-bbe1cabd-35fe-4944-89d1-fddd09c93923] Acquiring async connection started...
2026-02-21 20:37:34,296 INFO [PID-6093|THREAD-77685502|AgentServer|AsyncRabbitMQ-bbe1cabd-35fe-4944-89d1-fddd09c93923] Acquiring async connection completed successfully.
2026-02-21 20:37:34,305 INFO [TIMING] Task enqueued to RabbitMQ, setup=180.6ms
2026-02-21 20:37:34,307 INFO [TIMING] event_generator STARTED, task=bba63941-8048-4f39-9329-8568e5ebe9cd, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, user=68383665-d3d9-41f3-b10c-fca0dc6080ed
2026-02-21 20:37:34,307 INFO [TIMING] subscribe_to_task STARTED, task=bba63941-8048-4f39-9329-8568e5ebe9cd, user=68383665-d3d9-41f3-b10c-fca0dc6080ed, last_msg=0-0
2026-02-21 20:37:34,309 INFO [TIMING] Redis hgetall took 2.1ms
2026-02-21 20:37:34,353 INFO [PID-6048|THREAD-77685506|CoPilotExecutor|Redis-943506d1-86e7-48a7-871b-9977fb0ace47] Acquiring connection started...
2026-02-21 20:37:34,435 INFO [PID-6048|THREAD-77685506|CoPilotExecutor|Redis-943506d1-86e7-48a7-871b-9977fb0ace47] Acquiring connection completed successfully.
2026-02-21 20:37:34,442 INFO [CoPilotExecutor] Acquired cluster lock for bba63941-8048-4f39-9329-8568e5ebe9cd, executor_id=fb7d76b3-8dc3-40a4-947e-a93bfad207da
2026-02-21 20:37:34,535 INFO [CoPilotExecutor] [CoPilotExecutor] Worker 13455405056 started
2026-02-21 20:37:34,536 INFO [CoPilotExecutor|task_id:bba63941-8048-4f39-9329-8568e5ebe9cd|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Starting execution
2026-02-21 20:37:35,596 INFO [CoPilotExecutor|task_id:bba63941-8048-4f39-9329-8568e5ebe9cd|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Using SDK service
2026-02-21 20:37:35,596 INFO [PID-6048|THREAD-77697399|CoPilotExecutor|AsyncRedis-2e10c980-0364-4c4b-9b2d-8186f23b1735] Acquiring connection started...
2026-02-21 20:37:35,600 INFO [PID-6048|THREAD-77697399|CoPilotExecutor|AsyncRedis-2e10c980-0364-4c4b-9b2d-8186f23b1735] Acquiring connection completed successfully.
2026-02-21 20:37:35,601 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=12, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user']
2026-02-21 20:37:35,601 INFO [PID-6048|THREAD-77697399|CoPilotExecutor|AppService client-34797c8f-0201-4f99-bf73-3f3fb4697e6d] Creating service client started...
2026-02-21 20:37:35,601 INFO [PID-6048|THREAD-77697399|CoPilotExecutor|AppService client-34797c8f-0201-4f99-bf73-3f3fb4697e6d] Creating service client completed successfully.
2026-02-21 20:37:35,657 WARNING Service communication: Retry attempt 1 for '_call_method_async': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_chat_session_message_count'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:37:36,713 WARNING Service communication: Retry attempt 2 for '_call_method_async': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_chat_session_message_count'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:37:39,646 WARNING Service communication: Retry attempt 3 for '_call_method_async': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_chat_session_message_count'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:37:43,415 INFO 127.0.0.1:58782 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:44,423 WARNING Service communication: Retry attempt 4 for '_call_method_async': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_chat_session_message_count'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:37:44,486 INFO 127.0.0.1:58782 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:45,048 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=12, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user']
2026-02-21 20:37:45,053 INFO [TASK_LOOKUP] Found running task bba63941... for session 322af5c3...
2026-02-21 20:37:45,063 INFO [CoPilotExecutor] Received cancel for bba63941-8048-4f39-9329-8568e5ebe9cd
2026-02-21 20:37:45,064 INFO [CANCEL] Published cancel for task ...e5ebe9cd session ...f0aa0c9f
2026-02-21 20:37:45,113 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=12, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user']
2026-02-21 20:37:45,120 INFO [TASK_LOOKUP] Found running task bba63941... for session 322af5c3...
2026-02-21 20:37:45,121 INFO [GET_SESSION] session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, active_task=True, msg_count=12, last_role=user
2026-02-21 20:37:45,123 INFO 127.0.0.1:58802 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f HTTP/1.1" 200
2026-02-21 20:37:45,306 INFO [TASK_LOOKUP] Found running task bba63941... for session 322af5c3...
2026-02-21 20:37:45,307 INFO [TIMING] subscribe_to_task STARTED, task=bba63941-8048-4f39-9329-8568e5ebe9cd, user=68383665-d3d9-41f3-b10c-fca0dc6080ed, last_msg=0-0
2026-02-21 20:37:45,309 INFO [TIMING] Redis hgetall took 1.5ms
2026-02-21 20:37:45,604 INFO [CoPilotExecutor|task_id:bba63941-8048-4f39-9329-8568e5ebe9cd|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Cancellation requested
2026-02-21 20:37:45,604 INFO [CoPilotExecutor|task_id:bba63941-8048-4f39-9329-8568e5ebe9cd|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Execution completed in 11.07s
2026-02-21 20:37:45,604 INFO [CoPilotExecutor] Run completed for bba63941-8048-4f39-9329-8568e5ebe9cd
2026-02-21 20:37:45,604 INFO [CoPilotExecutor|task_id:bba63941-8048-4f39-9329-8568e5ebe9cd|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Task cancelled
2026-02-21 20:37:45,605 INFO [CoPilotExecutor] Releasing cluster lock for bba63941-8048-4f39-9329-8568e5ebe9cd
2026-02-21 20:37:45,609 INFO [CoPilotExecutor] Cleaned up completed task bba63941-8048-4f39-9329-8568e5ebe9cd
2026-02-21 20:37:45,610 INFO [TIMING] Redis xread (replay) took 301.1ms, status=running
2026-02-21 20:37:45,610 INFO [TIMING] publish_chunk StreamFinish in 1.8ms (xadd=1.3ms)
2026-02-21 20:37:45,612 INFO [TIMING] Replayed 1 messages, last_id=1771681065606-0
2026-02-21 20:37:45,612 INFO [TIMING] Task still running, starting _stream_listener
2026-02-21 20:37:45,613 INFO [TIMING] subscribe_to_task COMPLETED in 305.8ms; task=bba63941-8048-4f39-9329-8568e5ebe9cd, n_messages_replayed=1
2026-02-21 20:37:45,614 INFO [TIMING] _stream_listener STARTED, task=bba63941-8048-4f39-9329-8568e5ebe9cd, last_id=1771681065606-0
2026-02-21 20:37:45,614 INFO Resume stream chunk
2026-02-21 20:37:45,615 INFO 127.0.0.1:58802 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f/stream HTTP/1.1" 200
2026-02-21 20:37:45,615 INFO [TIMING] Redis xread (replay) took 11305.8ms, status=running
2026-02-21 20:37:45,616 INFO [TIMING] Replayed 1 messages, last_id=1771681065606-0
2026-02-21 20:37:45,616 INFO [TIMING] Task still running, starting _stream_listener
2026-02-21 20:37:45,616 INFO [TIMING] subscribe_to_task COMPLETED in 11308.9ms; task=bba63941-8048-4f39-9329-8568e5ebe9cd, n_messages_replayed=1
2026-02-21 20:37:45,616 INFO [TIMING] Starting to read from subscriber_queue
2026-02-21 20:37:45,616 INFO [TIMING] FIRST CHUNK from queue at 11.31s, type=StreamFinish
2026-02-21 20:37:45,616 INFO 127.0.0.1:58710 - "POST /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f/stream HTTP/1.1" 200
2026-02-21 20:37:45,617 INFO [TIMING] StreamFinish received in 11.31s; n_chunks=1
2026-02-21 20:37:45,617 INFO [TIMING] _stream_listener CANCELLED after 3.5ms, delivered=0
2026-02-21 20:37:45,617 INFO [TIMING] _stream_listener FINISHED in 0.0s; task=bba63941-8048-4f39-9329-8568e5ebe9cd, delivered=0, xread_count=1
2026-02-21 20:37:45,618 INFO Resume stream completed
2026-02-21 20:37:45,618 INFO [TIMING] event_generator FINISHED in 11.31s; task=bba63941-8048-4f39-9329-8568e5ebe9cd, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, n_chunks=1
2026-02-21 20:37:45,691 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=12, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user']
2026-02-21 20:37:45,694 INFO [GET_SESSION] session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, active_task=False, msg_count=12, last_role=user
2026-02-21 20:37:45,695 INFO 127.0.0.1:58710 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f HTTP/1.1" 200
2026-02-21 20:37:45,710 INFO 127.0.0.1:58802 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f/stream HTTP/1.1" 204
2026-02-21 20:37:45,771 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=12, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user']
2026-02-21 20:37:45,775 INFO [GET_SESSION] session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, active_task=False, msg_count=12, last_role=user
2026-02-21 20:37:45,775 INFO 127.0.0.1:58710 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f HTTP/1.1" 200
2026-02-21 20:37:46,075 INFO [CANCEL] Task ...e5ebe9cd confirmed stopped (status=failed) after 1.0s
2026-02-21 20:37:46,076 INFO 127.0.0.1:58782 - "POST /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f/cancel HTTP/1.1" 200
2026-02-21 20:37:46,573 INFO 127.0.0.1:58710 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:50,090 INFO 127.0.0.1:58710 - "GET /api/integrations/providers/system HTTP/1.1" 200
2026-02-21 20:37:50,103 INFO 127.0.0.1:58842 - "GET /api/integrations/providers HTTP/1.1" 200
2026-02-21 20:37:50,681 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=12, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user']
2026-02-21 20:37:50,686 INFO 127.0.0.1:58710 - "GET /api/library/agents?page=1&page_size=100 HTTP/1.1" 200
2026-02-21 20:37:50,692 INFO 127.0.0.1:58850 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:50,702 INFO 127.0.0.1:58842 - "GET /api/integrations/credentials HTTP/1.1" 200
2026-02-21 20:37:50,710 INFO [GET_SESSION] session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, active_task=False, msg_count=12, last_role=user
2026-02-21 20:37:50,711 INFO 127.0.0.1:58862 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f HTTP/1.1" 200
2026-02-21 20:37:50,714 INFO 127.0.0.1:58852 - "GET /api/onboarding HTTP/1.1" 200
2026-02-21 20:37:50,720 INFO 127.0.0.1:58854 - "GET /api/executions HTTP/1.1" 200
2026-02-21 20:37:50,795 INFO 127.0.0.1:58710 - "GET /api/chat/sessions?limit=50 HTTP/1.1" 200
2026-02-21 20:37:51,955 INFO 127.0.0.1:58710 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:54,064 INFO 127.0.0.1:58710 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:54,157 INFO [TIMING] stream_chat_post STARTED, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, user=68383665-d3d9-41f3-b10c-fca0dc6080ed, message_len=5
2026-02-21 20:37:54,169 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=12, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user']
2026-02-21 20:37:54,170 INFO [TIMING] session validated in 13.0ms
2026-02-21 20:37:54,170 INFO [STREAM] Saving user message to session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f
2026-02-21 20:37:54,172 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=12, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user']
2026-02-21 20:37:54,212 INFO Saving 1 new messages to DB for session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f: roles=['user'], start_sequence=12
2026-02-21 20:37:54,238 INFO [STREAM] User message saved for session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f
2026-02-21 20:37:54,238 INFO [TIMING] create_task STARTED, task=6360d249-c803-47d3-8a08-d77275e4b2d8, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, user=68383665-d3d9-41f3-b10c-fca0dc6080ed
2026-02-21 20:37:54,238 INFO [TIMING] get_redis_async took 0.0ms
2026-02-21 20:37:54,242 INFO [TIMING] redis.hset took 3.1ms
2026-02-21 20:37:54,250 INFO [TIMING] create_task COMPLETED in 11.6ms; task=6360d249-c803-47d3-8a08-d77275e4b2d8, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f
2026-02-21 20:37:54,251 INFO [TIMING] create_task completed in 12.9ms
2026-02-21 20:37:54,261 INFO [TIMING] Task enqueued to RabbitMQ, setup=103.8ms
2026-02-21 20:37:54,262 INFO [TIMING] event_generator STARTED, task=6360d249-c803-47d3-8a08-d77275e4b2d8, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, user=68383665-d3d9-41f3-b10c-fca0dc6080ed
2026-02-21 20:37:54,263 INFO [TIMING] subscribe_to_task STARTED, task=6360d249-c803-47d3-8a08-d77275e4b2d8, user=68383665-d3d9-41f3-b10c-fca0dc6080ed, last_msg=0-0
2026-02-21 20:37:54,264 INFO [TIMING] Redis hgetall took 1.7ms
2026-02-21 20:37:54,265 INFO [CoPilotExecutor] Acquired cluster lock for 6360d249-c803-47d3-8a08-d77275e4b2d8, executor_id=fb7d76b3-8dc3-40a4-947e-a93bfad207da
2026-02-21 20:37:54,267 INFO [CoPilotExecutor|task_id:6360d249-c803-47d3-8a08-d77275e4b2d8|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Starting execution
2026-02-21 20:37:54,286 INFO [CoPilotExecutor|task_id:6360d249-c803-47d3-8a08-d77275e4b2d8|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Using SDK service
2026-02-21 20:37:54,290 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=13, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user', 'user']
2026-02-21 20:37:54,357 WARNING Service communication: Retry attempt 1 for '_call_method_async': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_chat_session_message_count'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:37:56,312 WARNING Service communication: Retry attempt 2 for '_call_method_async': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_chat_session_message_count'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:37:58,224 INFO 127.0.0.1:58917 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:37:58,928 WARNING Service communication: Retry attempt 3 for '_call_method_async': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_chat_session_message_count'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:38:00,041 WARNING Service communication: Retry attempt 8 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:38:03,701 WARNING Service communication: Retry attempt 4 for '_call_method_async': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_chat_session_message_count'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:38:06,882 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=13, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user', 'user']
2026-02-21 20:38:06,888 INFO [TASK_LOOKUP] Found running task 6360d249... for session 322af5c3...
2026-02-21 20:38:06,898 INFO [CoPilotExecutor] Received cancel for 6360d249-c803-47d3-8a08-d77275e4b2d8
2026-02-21 20:38:06,898 INFO [CANCEL] Published cancel for task ...75e4b2d8 session ...f0aa0c9f
2026-02-21 20:38:06,919 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=13, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user', 'user']
2026-02-21 20:38:06,925 INFO [TASK_LOOKUP] Found running task 6360d249... for session 322af5c3...
2026-02-21 20:38:06,926 INFO [GET_SESSION] session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, active_task=True, msg_count=13, last_role=user
2026-02-21 20:38:06,927 INFO 127.0.0.1:58976 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f HTTP/1.1" 200
2026-02-21 20:38:07,136 INFO [TASK_LOOKUP] Found running task 6360d249... for session 322af5c3...
2026-02-21 20:38:07,138 INFO [TIMING] subscribe_to_task STARTED, task=6360d249-c803-47d3-8a08-d77275e4b2d8, user=68383665-d3d9-41f3-b10c-fca0dc6080ed, last_msg=0-0
2026-02-21 20:38:07,140 INFO [TIMING] Redis hgetall took 1.3ms
2026-02-21 20:38:07,359 INFO [CoPilotExecutor|task_id:6360d249-c803-47d3-8a08-d77275e4b2d8|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Cancellation requested
2026-02-21 20:38:07,360 INFO [CoPilotExecutor|task_id:6360d249-c803-47d3-8a08-d77275e4b2d8|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Execution completed in 13.09s
2026-02-21 20:38:07,360 INFO [CoPilotExecutor] Run completed for 6360d249-c803-47d3-8a08-d77275e4b2d8
2026-02-21 20:38:07,360 INFO [CoPilotExecutor|task_id:6360d249-c803-47d3-8a08-d77275e4b2d8|session_id:322af5c3-70fc-4a06-9443-8c5df0aa0c9f|user_id:68383665-d3d9-41f3-b10c-fca0dc6080ed] Task cancelled
2026-02-21 20:38:07,360 INFO [CoPilotExecutor] Releasing cluster lock for 6360d249-c803-47d3-8a08-d77275e4b2d8
2026-02-21 20:38:07,362 INFO [CoPilotExecutor] Cleaned up completed task 6360d249-c803-47d3-8a08-d77275e4b2d8
2026-02-21 20:38:07,364 INFO [TIMING] Redis xread (replay) took 224.1ms, status=running
2026-02-21 20:38:07,364 INFO [TIMING] Replayed 1 messages, last_id=1771681087362-0
2026-02-21 20:38:07,365 INFO [TIMING] Task still running, starting _stream_listener
2026-02-21 20:38:07,365 INFO [TIMING] publish_chunk StreamFinish in 2.1ms (xadd=1.2ms)
2026-02-21 20:38:07,365 INFO [TIMING] subscribe_to_task COMPLETED in 226.8ms; task=6360d249-c803-47d3-8a08-d77275e4b2d8, n_messages_replayed=1
2026-02-21 20:38:07,366 INFO [TIMING] _stream_listener STARTED, task=6360d249-c803-47d3-8a08-d77275e4b2d8, last_id=1771681087362-0
2026-02-21 20:38:07,366 INFO Resume stream chunk
2026-02-21 20:38:07,366 INFO 127.0.0.1:58976 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f/stream HTTP/1.1" 200
2026-02-21 20:38:07,367 INFO [TIMING] Redis xread (replay) took 13101.9ms, status=running
2026-02-21 20:38:07,367 INFO [TIMING] Replayed 1 messages, last_id=1771681087362-0
2026-02-21 20:38:07,367 INFO [TIMING] Task still running, starting _stream_listener
2026-02-21 20:38:07,367 INFO [TIMING] subscribe_to_task COMPLETED in 13104.6ms; task=6360d249-c803-47d3-8a08-d77275e4b2d8, n_messages_replayed=1
2026-02-21 20:38:07,367 INFO [TIMING] Starting to read from subscriber_queue
2026-02-21 20:38:07,368 INFO [TIMING] FIRST CHUNK from queue at 13.11s, type=StreamFinish
2026-02-21 20:38:07,368 INFO 127.0.0.1:58710 - "POST /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f/stream HTTP/1.1" 200
2026-02-21 20:38:07,368 INFO [TIMING] StreamFinish received in 13.11s; n_chunks=1
2026-02-21 20:38:07,368 INFO [TIMING] _stream_listener CANCELLED after 2.7ms, delivered=0
2026-02-21 20:38:07,368 INFO [TIMING] _stream_listener FINISHED in 0.0s; task=6360d249-c803-47d3-8a08-d77275e4b2d8, delivered=0, xread_count=1
2026-02-21 20:38:07,369 INFO Resume stream completed
2026-02-21 20:38:07,369 INFO [TIMING] event_generator FINISHED in 13.11s; task=6360d249-c803-47d3-8a08-d77275e4b2d8, session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, n_chunks=1
2026-02-21 20:38:07,408 INFO [CANCEL] Task ...75e4b2d8 confirmed stopped (status=failed) after 0.5s
2026-02-21 20:38:07,409 INFO 127.0.0.1:58974 - "POST /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f/cancel HTTP/1.1" 200
2026-02-21 20:38:07,447 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=13, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user', 'user']
2026-02-21 20:38:07,451 INFO [GET_SESSION] session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, active_task=False, msg_count=13, last_role=user
2026-02-21 20:38:07,451 INFO 127.0.0.1:58710 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f HTTP/1.1" 200
2026-02-21 20:38:07,468 INFO 127.0.0.1:58710 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f/stream HTTP/1.1" 204
2026-02-21 20:38:07,521 INFO Loading session 322af5c3-70fc-4a06-9443-8c5df0aa0c9f from cache: message_count=13, roles=['user', 'assistant', 'tool', 'assistant', 'tool', 'assistant', 'tool', 'tool', 'assistant', 'tool', 'tool', 'user', 'user']
2026-02-21 20:38:07,527 INFO [GET_SESSION] session=322af5c3-70fc-4a06-9443-8c5df0aa0c9f, active_task=False, msg_count=13, last_role=user
2026-02-21 20:38:07,528 INFO 127.0.0.1:58710 - "GET /api/chat/sessions/322af5c3-70fc-4a06-9443-8c5df0aa0c9f HTTP/1.1" 200
2026-02-21 20:38:18,440 INFO 127.0.0.1:59077 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:38:19,553 INFO 127.0.0.1:59077 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:38:21,643 INFO 127.0.0.1:59077 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:38:30,090 WARNING Service communication: Retry attempt 9 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:39:00,123 WARNING Service communication: Retry attempt 10 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:39:13,881 INFO 127.0.0.1:59398 - "GET /api/chat/sessions?limit=50 HTTP/1.1" 200
2026-02-21 20:39:30,173 WARNING Service communication: Retry attempt 11 for '_call_method_sync': HTTPServerError: HTTP 500: Server error '500 Internal Server Error' for url 'http://localhost:8005/get_graph_executions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
2026-02-21 20:39:35,355 INFO 127.0.0.1:59522 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:39:35,685 INFO 127.0.0.1:59526 - "GET /api/executions HTTP/1.1" 200
2026-02-21 20:39:38,916 INFO 127.0.0.1:59522 - "GET /api/store/profile HTTP/1.1" 404
2026-02-21 20:39:40,019 INFO 127.0.0.1:59522 - "GET /api/store/profile HTTP/1.1" 404

View File

@@ -22,3 +22,4 @@ migrations/*/rollback*.sql
# Workspace files
workspaces/
sample.logs

View File

@@ -34,6 +34,9 @@ class ResponseType(str, Enum):
TOOL_INPUT_AVAILABLE = "tool-input-available"
TOOL_OUTPUT_AVAILABLE = "tool-output-available"
# Long-running tool notification (custom extension - uses AI SDK DataUIPart format)
LONG_RUNNING_START = "data-long-running-start"
# Other
ERROR = "error"
USAGE = "usage"
@@ -145,6 +148,10 @@ class StreamToolInputAvailable(StreamBaseResponse):
input: dict[str, Any] = Field(
default_factory=dict, description="Tool input arguments"
)
providerMetadata: dict[str, Any] | None = Field(
default=None,
description="Provider metadata - used to pass isLongRunning flag to frontend",
)
class StreamToolOutputAvailable(StreamBaseResponse):
@@ -173,6 +180,20 @@ class StreamToolOutputAvailable(StreamBaseResponse):
return f"data: {json.dumps(data)}\n\n"
class StreamLongRunningStart(StreamBaseResponse):
"""Notification that a long-running tool has started.
Custom extension using AI SDK DataUIPart format. Signals the frontend to show
UI feedback while the tool executes.
"""
type: ResponseType = ResponseType.LONG_RUNNING_START
data: dict[str, Any] = Field(
default_factory=dict,
description="Data for the long-running event containing toolCallId and toolName",
)
# ========== Other ==========

View File

@@ -34,6 +34,7 @@ from backend.copilot.response_model import (
StreamToolInputStart,
StreamToolOutputAvailable,
)
from backend.copilot.tools import get_tool
from .tool_adapter import MCP_TOOL_PREFIX, pop_pending_tool_output
@@ -111,6 +112,15 @@ class SDKResponseAdapter:
# instead of "mcp__copilot__find_block".
tool_name = block.name.removeprefix(MCP_TOOL_PREFIX)
# Check if this is a long-running tool to trigger UI feedback
tool = get_tool(tool_name)
is_long_running = tool.is_long_running if tool else False
logger.info(
f"[ADAPTER] Tool: {tool_name}, has_tool={tool is not None}, "
f"is_long_running={is_long_running}"
)
responses.append(
StreamToolInputStart(toolCallId=block.id, toolName=tool_name)
)
@@ -119,8 +129,15 @@ class SDKResponseAdapter:
toolCallId=block.id,
toolName=tool_name,
input=block.input,
providerMetadata=(
{"isLongRunning": True} if is_long_running else None
),
)
)
logger.info(
f"[ADAPTER] Created StreamToolInputAvailable with "
f"providerMetadata={{'isLongRunning': {is_long_running}}}"
)
self.current_tool_calls[block.id] = {"name": tool_name}
elif isinstance(sdk_message, UserMessage):

View File

@@ -11,7 +11,6 @@ from typing import Any
from backend.util.exceptions import NotFoundError
from .. import stream_registry
from ..config import ChatConfig
from ..model import (
ChatMessage,
@@ -31,12 +30,7 @@ from ..response_model import (
StreamToolInputAvailable,
StreamToolOutputAvailable,
)
from ..service import (
_build_system_prompt,
_execute_long_running_tool_with_streaming,
_generate_session_title,
)
from ..tools.models import OperationPendingResponse, OperationStartedResponse
from ..service import _build_system_prompt, _generate_session_title
from ..tools.sandbox import WORKSPACE_PREFIX, make_session_path
from ..tracking import track_user_message
from .response_adapter import SDKResponseAdapter
@@ -44,7 +38,6 @@ from .security_hooks import create_security_hooks
from .tool_adapter import (
COPILOT_TOOL_NAMES,
SDK_DISALLOWED_TOOLS,
LongRunningCallback,
create_copilot_mcp_server,
set_execution_context,
wait_for_stash,
@@ -123,9 +116,9 @@ When you create or modify important files (code, configs, outputs), you MUST:
are available from previous turns
### Long-running tools
Long-running tools (create_agent, edit_agent, etc.) are handled
asynchronously. You will receive an immediate response; the actual result
is delivered to the user via a background stream.
Long-running tools (create_agent, edit_agent, etc.) run synchronously
with heartbeats to keep the connection alive. The frontend shows UI feedback
during execution based on stream events.
### Sub-agent tasks
- When using the Task tool, NEVER set `run_in_background` to true.
@@ -133,121 +126,6 @@ is delivered to the user via a background stream.
"""
def _build_long_running_callback(user_id: str | None) -> LongRunningCallback:
"""Build a callback that delegates long-running tools to the non-SDK infrastructure.
Long-running tools (create_agent, edit_agent, etc.) are delegated to the
existing background infrastructure: stream_registry (Redis Streams),
database persistence, and SSE reconnection. This means results survive
page refreshes / pod restarts, and the frontend shows the proper loading
widget with progress updates.
The returned callback matches the ``LongRunningCallback`` signature:
``(tool_name, args, session) -> MCP response dict``.
"""
async def _callback(
tool_name: str, args: dict[str, Any], session: ChatSession
) -> dict[str, Any]:
operation_id = str(uuid.uuid4())
task_id = str(uuid.uuid4())
tool_call_id = f"sdk-{uuid.uuid4().hex[:12]}"
session_id = session.session_id
# --- Build user-friendly messages (matches non-SDK service) ---
if tool_name == "create_agent":
desc = args.get("description", "")
desc_preview = (desc[:100] + "...") if len(desc) > 100 else desc
pending_msg = (
f"Creating your agent: {desc_preview}"
if desc_preview
else "Creating agent... This may take a few minutes."
)
started_msg = (
"Agent creation started. You can close this tab - "
"check your library in a few minutes."
)
elif tool_name == "edit_agent":
changes = args.get("changes", "")
changes_preview = (changes[:100] + "...") if len(changes) > 100 else changes
pending_msg = (
f"Editing agent: {changes_preview}"
if changes_preview
else "Editing agent... This may take a few minutes."
)
started_msg = (
"Agent edit started. You can close this tab - "
"check your library in a few minutes."
)
else:
pending_msg = f"Running {tool_name}... This may take a few minutes."
started_msg = (
f"{tool_name} started. You can close this tab - "
"check back in a few minutes."
)
# --- Register task in Redis for SSE reconnection ---
await stream_registry.create_task(
task_id=task_id,
session_id=session_id,
user_id=user_id,
tool_call_id=tool_call_id,
tool_name=tool_name,
operation_id=operation_id,
)
# --- Save OperationPendingResponse to chat history ---
pending_message = ChatMessage(
role="tool",
content=OperationPendingResponse(
message=pending_msg,
operation_id=operation_id,
tool_name=tool_name,
).model_dump_json(),
tool_call_id=tool_call_id,
)
session.messages.append(pending_message)
await upsert_chat_session(session)
# --- Spawn background task (reuses non-SDK infrastructure) ---
bg_task = asyncio.create_task(
_execute_long_running_tool_with_streaming(
tool_name=tool_name,
parameters=args,
tool_call_id=tool_call_id,
operation_id=operation_id,
task_id=task_id,
session_id=session_id,
user_id=user_id,
)
)
_background_tasks.add(bg_task)
bg_task.add_done_callback(_background_tasks.discard)
await stream_registry.set_task_asyncio_task(task_id, bg_task)
logger.info(
f"[SDK] Long-running tool {tool_name} delegated to background "
f"(operation_id={operation_id}, task_id={task_id})"
)
# --- Return OperationStartedResponse as MCP tool result ---
# This flows through SDK → response adapter → frontend, triggering
# the loading widget with SSE reconnection support.
started_json = OperationStartedResponse(
message=started_msg,
operation_id=operation_id,
tool_name=tool_name,
task_id=task_id,
).model_dump_json()
return {
"content": [{"type": "text", "text": started_json}],
"isError": False,
}
return _callback
def _resolve_sdk_model() -> str | None:
"""Resolve the model name for the Claude Agent SDK CLI.
@@ -584,7 +462,7 @@ async def stream_chat_completion_sdk(
set_execution_context(
user_id,
session,
long_running_callback=_build_long_running_callback(user_id),
long_running_callback=None,
)
try:
from claude_agent_sdk import ClaudeAgentOptions, ClaudeSDKClient

View File

@@ -52,6 +52,7 @@ from .response_model import (
StreamFinish,
StreamFinishStep,
StreamHeartbeat,
StreamLongRunningStart,
StreamStart,
StreamStartStep,
StreamTextDelta,
@@ -63,12 +64,7 @@ from .response_model import (
StreamUsage,
)
from .tools import execute_tool, get_tool, tools
from .tools.models import (
ErrorResponse,
OperationInProgressResponse,
OperationPendingResponse,
OperationStartedResponse,
)
from .tools.models import ErrorResponse
from .tracking import track_user_message
logger = logging.getLogger(__name__)
@@ -1402,16 +1398,15 @@ async def _yield_tool_call(
"""
Yield a tool call and its execution result.
For tools marked with `is_long_running=True` (like agent generation), spawns a
background task so the operation survives SSE disconnections. For other tools,
yields heartbeat events every 15 seconds to keep the SSE connection alive.
Executes tools synchronously and yields heartbeat events every 15 seconds to
keep the SSE connection alive during execution. The is_long_running property
is only used by the frontend to display UI feedback during long operations.
Raises:
orjson.JSONDecodeError: If tool call arguments cannot be parsed as JSON
KeyError: If expected tool call fields are missing
TypeError: If tool call structure is invalid
"""
import uuid as uuid_module
tool_name = tool_calls[yield_idx]["function"]["name"]
tool_call_id = tool_calls[yield_idx]["id"]
@@ -1429,163 +1424,17 @@ async def _yield_tool_call(
input=arguments,
)
# Check if this tool is long-running (survives SSE disconnection)
# Notify frontend if this is a long-running tool (e.g., agent generation)
tool = get_tool(tool_name)
if tool and tool.is_long_running:
# Atomic check-and-set: returns False if operation already running (lost race)
if not await _mark_operation_started(tool_call_id):
logger.info(
f"Tool call {tool_call_id} already in progress, returning status"
)
# Build dynamic message based on tool name
if tool_name == "create_agent":
in_progress_msg = "Agent creation already in progress. Please wait..."
elif tool_name == "edit_agent":
in_progress_msg = "Agent edit already in progress. Please wait..."
else:
in_progress_msg = f"{tool_name} already in progress. Please wait..."
yield StreamToolOutputAvailable(
toolCallId=tool_call_id,
toolName=tool_name,
output=OperationInProgressResponse(
message=in_progress_msg,
tool_call_id=tool_call_id,
).model_dump_json(),
success=True,
)
return
# Generate operation ID and task ID
operation_id = str(uuid_module.uuid4())
task_id = str(uuid_module.uuid4())
# Build a user-friendly message based on tool and arguments
if tool_name == "create_agent":
agent_desc = arguments.get("description", "")
# Truncate long descriptions for the message
desc_preview = (
(agent_desc[:100] + "...") if len(agent_desc) > 100 else agent_desc
)
pending_msg = (
f"Creating your agent: {desc_preview}"
if desc_preview
else "Creating agent... This may take a few minutes."
)
started_msg = (
"Agent creation started. You can close this tab - "
"check your library in a few minutes."
)
elif tool_name == "edit_agent":
changes = arguments.get("changes", "")
changes_preview = (changes[:100] + "...") if len(changes) > 100 else changes
pending_msg = (
f"Editing agent: {changes_preview}"
if changes_preview
else "Editing agent... This may take a few minutes."
)
started_msg = (
"Agent edit started. You can close this tab - "
"check your library in a few minutes."
)
else:
pending_msg = f"Running {tool_name}... This may take a few minutes."
started_msg = (
f"{tool_name} started. You can close this tab - "
"check back in a few minutes."
)
# Track appended message for rollback on failure
pending_message: ChatMessage | None = None
# Wrap session save and task creation in try-except to release lock on failure
try:
# Create task in stream registry for SSE reconnection support
await stream_registry.create_task(
task_id=task_id,
session_id=session.session_id,
user_id=session.user_id,
tool_call_id=tool_call_id,
tool_name=tool_name,
operation_id=operation_id,
)
# Attach tool_call and save pending result — lock serialises
# concurrent session mutations during parallel execution.
async def _save_pending() -> None:
nonlocal pending_message
session.add_tool_call_to_current_turn(tool_calls[yield_idx])
pending_message = ChatMessage(
role="tool",
content=OperationPendingResponse(
message=pending_msg,
operation_id=operation_id,
tool_name=tool_name,
).model_dump_json(),
tool_call_id=tool_call_id,
)
session.messages.append(pending_message)
await upsert_chat_session(session)
await _with_optional_lock(session_lock, _save_pending)
logger.info(
f"Saved pending operation {operation_id} (task_id={task_id}) "
f"for tool {tool_name} in session {session.session_id}"
)
# Store task reference in module-level set to prevent GC before completion
bg_task = asyncio.create_task(
_execute_long_running_tool_with_streaming(
tool_name=tool_name,
parameters=arguments,
tool_call_id=tool_call_id,
operation_id=operation_id,
task_id=task_id,
session_id=session.session_id,
user_id=session.user_id,
)
)
_background_tasks.add(bg_task)
bg_task.add_done_callback(_background_tasks.discard)
# Associate the asyncio task with the stream registry task
await stream_registry.set_task_asyncio_task(task_id, bg_task)
except Exception as e:
# Roll back appended messages — use identity-based removal so
# it works even when other parallel tools have appended after us.
async def _rollback() -> None:
if pending_message and pending_message in session.messages:
session.messages.remove(pending_message)
await _with_optional_lock(session_lock, _rollback)
# Release the Redis lock since the background task won't be spawned
await _mark_operation_completed(tool_call_id)
# Mark stream registry task as failed if it was created
try:
await stream_registry.mark_task_completed(task_id, status="failed")
except Exception as mark_err:
logger.warning(f"Failed to mark task {task_id} as failed: {mark_err}")
logger.error(
f"Failed to setup long-running tool {tool_name}: {e}", exc_info=True
)
raise
# Return immediately - don't wait for completion
yield StreamToolOutputAvailable(
toolCallId=tool_call_id,
toolName=tool_name,
output=OperationStartedResponse(
message=started_msg,
operation_id=operation_id,
tool_name=tool_name,
task_id=task_id, # Include task_id for SSE reconnection
).model_dump_json(),
success=True,
yield StreamLongRunningStart(
data={
"toolCallId": tool_call_id,
"toolName": tool_name,
}
)
return
# Normal flow: Run tool execution in background task with heartbeats
# Run tool execution synchronously with heartbeats
tool_task = asyncio.create_task(
execute_tool(
tool_name=tool_name,

View File

@@ -540,21 +540,15 @@ async def decompose_goal(
async def generate_agent(
instructions: DecompositionResult | dict[str, Any],
library_agents: list[AgentSummary] | list[dict[str, Any]] | None = None,
operation_id: str | None = None,
task_id: str | None = None,
) -> dict[str, Any] | None:
"""Generate agent JSON from instructions.
Args:
instructions: Structured instructions from decompose_goal
library_agents: User's library agents available for sub-agent composition
operation_id: Operation ID for async processing (enables Redis Streams
completion notification)
task_id: Task ID for async processing (enables Redis Streams persistence
and SSE delivery)
Returns:
Agent JSON dict, {"status": "accepted"} for async, error dict {"type": "error", ...}, or None on error
Agent JSON dict, error dict {"type": "error", ...}, or None on error
Raises:
AgentGeneratorNotConfiguredError: If the external service is not configured.
@@ -562,13 +556,9 @@ async def generate_agent(
_check_service_configured()
logger.info("Calling external Agent Generator service for generate_agent")
result = await generate_agent_external(
dict(instructions), _to_dict_list(library_agents), operation_id, task_id
dict(instructions), _to_dict_list(library_agents)
)
# Don't modify async response
if result and result.get("status") == "accepted":
return result
if result:
if isinstance(result, dict) and result.get("type") == "error":
return result
@@ -759,8 +749,6 @@ async def generate_agent_patch(
update_request: str,
current_agent: dict[str, Any],
library_agents: list[AgentSummary] | None = None,
operation_id: str | None = None,
task_id: str | None = None,
) -> dict[str, Any] | None:
"""Update an existing agent using natural language.
@@ -773,12 +761,10 @@ async def generate_agent_patch(
update_request: Natural language description of changes
current_agent: Current agent JSON
library_agents: User's library agents available for sub-agent composition
operation_id: Operation ID for async processing (enables Redis Streams callback)
task_id: Task ID for async processing (enables Redis Streams callback)
Returns:
Updated agent JSON, clarifying questions dict {"type": "clarifying_questions", ...},
{"status": "accepted"} for async, error dict {"type": "error", ...}, or None on error
error dict {"type": "error", ...}, or None on error
Raises:
AgentGeneratorNotConfiguredError: If the external service is not configured.
@@ -789,8 +775,6 @@ async def generate_agent_patch(
update_request,
current_agent,
_to_dict_list(library_agents),
operation_id,
task_id,
)

View File

@@ -101,8 +101,6 @@ async def decompose_goal_dummy(
async def generate_agent_dummy(
instructions: dict[str, Any],
library_agents: list[dict[str, Any]] | None = None,
operation_id: str | None = None,
task_id: str | None = None,
) -> dict[str, Any]:
"""Return dummy agent JSON after a simulated delay."""
logger.info("Using dummy agent generator for generate_agent (30s delay)")
@@ -114,8 +112,6 @@ async def generate_agent_patch_dummy(
update_request: str,
current_agent: dict[str, Any],
library_agents: list[dict[str, Any]] | None = None,
operation_id: str | None = None,
task_id: str | None = None,
) -> dict[str, Any]:
"""Return dummy patched agent (returns the current agent with updated description)."""
logger.info("Using dummy agent generator for generate_agent_patch")

View File

@@ -242,24 +242,18 @@ async def decompose_goal_external(
async def generate_agent_external(
instructions: dict[str, Any],
library_agents: list[dict[str, Any]] | None = None,
operation_id: str | None = None,
task_id: str | None = None,
) -> dict[str, Any] | None:
"""Call the external service to generate an agent from instructions.
Args:
instructions: Structured instructions from decompose_goal
library_agents: User's library agents available for sub-agent composition
operation_id: Operation ID for async processing (enables Redis Streams callback)
task_id: Task ID for async processing (enables Redis Streams callback)
Returns:
Agent JSON dict, {"status": "accepted"} for async, or error dict {"type": "error", ...} on error
Agent JSON dict or error dict {"type": "error", ...} on error
"""
if _is_dummy_mode():
return await generate_agent_dummy(
instructions, library_agents, operation_id, task_id
)
return await generate_agent_dummy(instructions, library_agents)
client = _get_client()
@@ -267,25 +261,9 @@ async def generate_agent_external(
payload: dict[str, Any] = {"instructions": instructions}
if library_agents:
payload["library_agents"] = library_agents
if operation_id and task_id:
payload["operation_id"] = operation_id
payload["task_id"] = task_id
try:
response = await client.post("/api/generate-agent", json=payload)
# Handle 202 Accepted for async processing
if response.status_code == 202:
logger.info(
f"Agent Generator accepted async request "
f"(operation_id={operation_id}, task_id={task_id})"
)
return {
"status": "accepted",
"operation_id": operation_id,
"task_id": task_id,
}
response.raise_for_status()
data = response.json()
@@ -317,8 +295,6 @@ async def generate_agent_patch_external(
update_request: str,
current_agent: dict[str, Any],
library_agents: list[dict[str, Any]] | None = None,
operation_id: str | None = None,
task_id: str | None = None,
) -> dict[str, Any] | None:
"""Call the external service to generate a patch for an existing agent.
@@ -326,15 +302,13 @@ async def generate_agent_patch_external(
update_request: Natural language description of changes
current_agent: Current agent JSON
library_agents: User's library agents available for sub-agent composition
operation_id: Operation ID for async processing (enables Redis Streams callback)
task_id: Task ID for async processing (enables Redis Streams callback)
Returns:
Updated agent JSON, clarifying questions dict, {"status": "accepted"} for async, or error dict on error
Updated agent JSON, clarifying questions dict, or error dict on error
"""
if _is_dummy_mode():
return await generate_agent_patch_dummy(
update_request, current_agent, library_agents, operation_id, task_id
update_request, current_agent, library_agents
)
client = _get_client()
@@ -346,25 +320,9 @@ async def generate_agent_patch_external(
}
if library_agents:
payload["library_agents"] = library_agents
if operation_id and task_id:
payload["operation_id"] = operation_id
payload["task_id"] = task_id
try:
response = await client.post("/api/update-agent", json=payload)
# Handle 202 Accepted for async processing
if response.status_code == 202:
logger.info(
f"Agent Generator accepted async update request "
f"(operation_id={operation_id}, task_id={task_id})"
)
return {
"status": "accepted",
"operation_id": operation_id,
"task_id": task_id,
}
response.raise_for_status()
data = response.json()

View File

@@ -38,12 +38,7 @@ class BaseTool:
@property
def is_long_running(self) -> bool:
"""Whether this tool is long-running and should execute in background.
Long-running tools (like agent generation) are executed via background
tasks to survive SSE disconnections. The result is persisted to chat
history and visible when the user refreshes.
"""
"""Whether this tool takes a long time to execute (triggers long-running UI)."""
return False
def as_openai_tool(self) -> ChatCompletionToolParam:

View File

@@ -18,7 +18,6 @@ from .base import BaseTool
from .models import (
AgentPreviewResponse,
AgentSavedResponse,
AsyncProcessingResponse,
ClarificationNeededResponse,
ClarifyingQuestion,
ErrorResponse,
@@ -49,6 +48,7 @@ class CreateAgentTool(BaseTool):
@property
def is_long_running(self) -> bool:
"""Agent generation takes several minutes."""
return True
@property
@@ -100,10 +100,6 @@ class CreateAgentTool(BaseTool):
save = kwargs.get("save", True)
session_id = session.session_id if session else None
# Extract async processing params (passed by long-running tool handler)
operation_id = kwargs.get("_operation_id")
task_id = kwargs.get("_task_id")
if not description:
return ErrorResponse(
message="Please provide a description of what the agent should do.",
@@ -230,8 +226,6 @@ class CreateAgentTool(BaseTool):
agent_json = await generate_agent(
decomposition_result,
library_agents,
operation_id=operation_id,
task_id=task_id,
)
except AgentGeneratorNotConfiguredError:
return ErrorResponse(
@@ -276,19 +270,6 @@ class CreateAgentTool(BaseTool):
session_id=session_id,
)
# Check if Agent Generator accepted for async processing
if agent_json.get("status") == "accepted":
logger.info(
f"Agent generation delegated to async processing "
f"(operation_id={operation_id}, task_id={task_id})"
)
return AsyncProcessingResponse(
message="Agent generation started. You'll be notified when it's complete.",
operation_id=operation_id,
task_id=task_id,
session_id=session_id,
)
agent_name = agent_json.get("name", "Generated Agent")
agent_description = agent_json.get("description", "")
node_count = len(agent_json.get("nodes", []))

View File

@@ -48,6 +48,7 @@ class CustomizeAgentTool(BaseTool):
@property
def is_long_running(self) -> bool:
"""Agent customization takes several minutes."""
return True
@property

View File

@@ -17,7 +17,6 @@ from .base import BaseTool
from .models import (
AgentPreviewResponse,
AgentSavedResponse,
AsyncProcessingResponse,
ClarificationNeededResponse,
ClarifyingQuestion,
ErrorResponse,
@@ -47,6 +46,7 @@ class EditAgentTool(BaseTool):
@property
def is_long_running(self) -> bool:
"""Agent editing takes several minutes."""
return True
@property
@@ -105,10 +105,6 @@ class EditAgentTool(BaseTool):
save = kwargs.get("save", True)
session_id = session.session_id if session else None
# Extract async processing params (passed by long-running tool handler)
operation_id = kwargs.get("_operation_id")
task_id = kwargs.get("_task_id")
if not agent_id:
return ErrorResponse(
message="Please provide the agent ID to edit.",
@@ -157,8 +153,6 @@ class EditAgentTool(BaseTool):
update_request,
current_agent,
library_agents,
operation_id=operation_id,
task_id=task_id,
)
except AgentGeneratorNotConfiguredError:
return ErrorResponse(
@@ -178,19 +172,6 @@ class EditAgentTool(BaseTool):
session_id=session_id,
)
# Check if Agent Generator accepted for async processing
if result.get("status") == "accepted":
logger.info(
f"Agent edit delegated to async processing "
f"(operation_id={operation_id}, task_id={task_id})"
)
return AsyncProcessingResponse(
message="Agent edit started. You'll be notified when it's complete.",
operation_id=operation_id,
task_id=task_id,
session_id=session_id,
)
# Check if the result is an error from the external service
if isinstance(result, dict) and result.get("type") == "error":
error_msg = result.get("error", "Unknown error")

View File

@@ -459,23 +459,6 @@ class OperationInProgressResponse(ToolResponseBase):
tool_call_id: str
class AsyncProcessingResponse(ToolResponseBase):
"""Response when an operation has been delegated to async processing.
This is returned by tools when the external service accepts the request
for async processing (HTTP 202 Accepted). The Redis Streams completion
consumer will handle the result when the external service completes.
The status field is specifically "accepted" to allow the long-running tool
handler to detect this response and skip LLM continuation.
"""
type: ResponseType = ResponseType.OPERATION_STARTED
status: str = "accepted" # Must be "accepted" for detection
operation_id: str | None = None
task_id: str | None = None
class WebFetchResponse(ToolResponseBase):
"""Response for web_fetch tool."""

View File

@@ -109,7 +109,7 @@ class TestGenerateAgent:
instructions = {"type": "instructions", "steps": ["Step 1"]}
result = await core.generate_agent(instructions)
mock_external.assert_called_once_with(instructions, None, None, None)
mock_external.assert_called_once_with(instructions, None)
assert result is not None
assert result["name"] == "Test Agent"
assert "id" in result
@@ -173,9 +173,7 @@ class TestGenerateAgentPatch:
current_agent = {"nodes": [], "links": []}
result = await core.generate_agent_patch("Add a node", current_agent)
mock_external.assert_called_once_with(
"Add a node", current_agent, None, None, None
)
mock_external.assert_called_once_with("Add a node", current_agent, None)
assert result == expected_result
@pytest.mark.asyncio

View File

@@ -34,7 +34,8 @@ export const ChatContainer = ({
onStop,
headerSlot,
}: ChatContainerProps) => {
const isBusy = status === "streaming" || !!isReconnecting;
const isBusy =
status === "streaming" || status === "submitted" || !!isReconnecting;
const inputLayoutId = "copilot-2-chat-input";
return (

View File

@@ -13,6 +13,7 @@ import { LoadingSpinner } from "@/components/atoms/LoadingSpinner/LoadingSpinner
import { toast } from "@/components/molecules/Toast/use-toast";
import { ToolUIPart, UIDataTypes, UIMessage, UITools } from "ai";
import { useEffect, useRef, useState } from "react";
import { ToolWrapper } from "../ToolWrapper/ToolWrapper";
import { CreateAgentTool } from "../../tools/CreateAgent/CreateAgent";
import { EditAgentTool } from "../../tools/EditAgent/EditAgent";
import {
@@ -208,86 +209,110 @@ export const ChatMessagesContainer = ({
);
case "tool-find_block":
return (
<FindBlocksTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<FindBlocksTool part={part as ToolUIPart} />
</ToolWrapper>
);
case "tool-find_agent":
case "tool-find_library_agent":
return (
<FindAgentsTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<FindAgentsTool part={part as ToolUIPart} />
</ToolWrapper>
);
case "tool-search_docs":
case "tool-get_doc_page":
return (
<SearchDocsTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<SearchDocsTool part={part as ToolUIPart} />
</ToolWrapper>
);
case "tool-run_block":
return (
<RunBlockTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<RunBlockTool part={part as ToolUIPart} />
</ToolWrapper>
);
case "tool-run_agent":
case "tool-schedule_agent":
return (
<RunAgentTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<RunAgentTool part={part as ToolUIPart} />
</ToolWrapper>
);
case "tool-create_agent":
return (
<CreateAgentTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<CreateAgentTool part={part as ToolUIPart} />
</ToolWrapper>
);
case "tool-edit_agent":
return (
<EditAgentTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<EditAgentTool part={part as ToolUIPart} />
</ToolWrapper>
);
case "tool-view_agent_output":
return (
<ViewAgentOutputTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<ViewAgentOutputTool part={part as ToolUIPart} />
</ToolWrapper>
);
case "tool-search_feature_requests":
return (
<SearchFeatureRequestsTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<SearchFeatureRequestsTool
part={part as ToolUIPart}
/>
</ToolWrapper>
);
case "tool-create_feature_request":
return (
<CreateFeatureRequestTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<CreateFeatureRequestTool part={part as ToolUIPart} />
</ToolWrapper>
);
default:
// Render a generic tool indicator for SDK built-in
// tools (Read, Glob, Grep, etc.) or any unrecognized tool
if (part.type.startsWith("tool-")) {
return (
<GenericTool
<ToolWrapper
key={`${message.id}-${i}`}
part={part as ToolUIPart}
/>
>
<GenericTool part={part as ToolUIPart} />
</ToolWrapper>
);
}
return null;

View File

@@ -0,0 +1,32 @@
import { PlusCircleIcon } from "@phosphor-icons/react";
import { ContentGrid, ContentHint } from "../ToolAccordion/AccordionContent";
import { ToolAccordion } from "../ToolAccordion/ToolAccordion";
import { MiniGame } from "../../tools/CreateAgent/components/MiniGame/MiniGame";
interface Props {
/** Whether the tool is currently streaming/executing */
isStreaming: boolean;
}
/**
* Displays UI feedback while a long-running tool executes.
* Automatically shown for tools marked as is_long_running=True in the backend.
*/
export function LongRunningToolDisplay({ isStreaming }: Props) {
if (!isStreaming) return null;
return (
<ToolAccordion
icon={<PlusCircleIcon size={32} weight="light" />}
title="This may take a few minutes. Play while you wait."
defaultExpanded={true}
>
<ContentGrid>
<MiniGame />
<ContentHint>
This could take a few minutes play while you wait!
</ContentHint>
</ContentGrid>
</ToolAccordion>
);
}

View File

@@ -0,0 +1,52 @@
import type { ToolUIPart } from "ai";
import { LongRunningToolDisplay } from "../LongRunningToolDisplay/LongRunningToolDisplay";
interface Props {
part: ToolUIPart;
children: React.ReactNode;
}
/**
* Wrapper for all tool components. Automatically shows UI feedback
* for long-running tools by detecting the isLongRunning flag on the tool part.
*/
export function ToolWrapper({ part, children }: Props) {
const isStreaming =
part.state === "input-streaming" || part.state === "input-available";
// Extract tool name from type (format: "tool-{name}")
const toolName = part.type.startsWith("tool-")
? part.type.substring(5)
: "unknown";
// Check if this tool is marked as long-running via providerMetadata
const isLongRunning =
"providerMetadata" in part &&
part.providerMetadata &&
typeof part.providerMetadata === "object" &&
"isLongRunning" in part.providerMetadata &&
part.providerMetadata.isLongRunning === true;
// Debug logging
if (part.type.startsWith("tool-")) {
console.log("[ToolWrapper]", {
toolName,
type: part.type,
hasProviderMetadata: "providerMetadata" in part,
providerMetadata:
"providerMetadata" in part ? part.providerMetadata : undefined,
isLongRunning,
state: part.state,
isStreaming,
});
}
return (
<>
{/* Show UI feedback if tool is long-running and streaming */}
{isLongRunning && <LongRunningToolDisplay isStreaming={isStreaming} />}
{/* Render the actual tool component */}
{children}
</>
);
}

View File

@@ -16,7 +16,6 @@ import {
ContentCardDescription,
ContentCodeBlock,
ContentGrid,
ContentHint,
ContentMessage,
} from "../../components/ToolAccordion/AccordionContent";
import { ToolAccordion } from "../../components/ToolAccordion/ToolAccordion";
@@ -25,7 +24,6 @@ import {
ClarifyingQuestion,
} from "./components/ClarificationQuestionsCard";
import sparklesImg from "./components/MiniGame/assets/sparkles.png";
import { MiniGame } from "./components/MiniGame/MiniGame";
import { SuggestedGoalCard } from "./components/SuggestedGoalCard";
import {
AccordionIcon,
@@ -36,9 +34,6 @@ import {
isAgentSavedOutput,
isClarificationNeededOutput,
isErrorOutput,
isOperationInProgressOutput,
isOperationPendingOutput,
isOperationStartedOutput,
isSuggestedGoalOutput,
ToolIcon,
truncateText,
@@ -86,18 +81,6 @@ function getAccordionMeta(output: CreateAgentToolOutput) {
expanded: true,
};
}
if (
isOperationStartedOutput(output) ||
isOperationPendingOutput(output) ||
isOperationInProgressOutput(output)
) {
return {
icon,
title:
"Creating agent, this may take a few minutes. Play while you wait.",
expanded: true,
};
}
return {
icon: (
<WarningDiamondIcon size={32} weight="light" className="text-red-500" />
@@ -119,19 +102,10 @@ export function CreateAgentTool({ part }: Props) {
const isError =
part.state === "output-error" || (!!output && isErrorOutput(output));
const isOperating =
!!output &&
(isOperationStartedOutput(output) ||
isOperationPendingOutput(output) ||
isOperationInProgressOutput(output));
const hasExpandableContent =
part.state === "output-available" &&
!!output &&
(isOperationStartedOutput(output) ||
isOperationPendingOutput(output) ||
isOperationInProgressOutput(output) ||
isAgentPreviewOutput(output) ||
(isAgentPreviewOutput(output) ||
isAgentSavedOutput(output) ||
isClarificationNeededOutput(output) ||
isSuggestedGoalOutput(output) ||
@@ -171,15 +145,6 @@ export function CreateAgentTool({ part }: Props) {
{hasExpandableContent && output && (
<ToolAccordion {...getAccordionMeta(output)}>
{isOperating && (
<ContentGrid>
<MiniGame />
<ContentHint>
This could take a few minutes play while you wait!
</ContentHint>
</ContentGrid>
)}
{isAgentSavedOutput(output) && (
<div className="rounded-xl border border-border/60 bg-card p-4 shadow-sm">
<div className="flex items-baseline gap-2">

View File

@@ -2,9 +2,6 @@ import type { AgentPreviewResponse } from "@/app/api/__generated__/models/agentP
import type { AgentSavedResponse } from "@/app/api/__generated__/models/agentSavedResponse";
import type { ClarificationNeededResponse } from "@/app/api/__generated__/models/clarificationNeededResponse";
import type { ErrorResponse } from "@/app/api/__generated__/models/errorResponse";
import type { OperationInProgressResponse } from "@/app/api/__generated__/models/operationInProgressResponse";
import type { OperationPendingResponse } from "@/app/api/__generated__/models/operationPendingResponse";
import type { OperationStartedResponse } from "@/app/api/__generated__/models/operationStartedResponse";
import { ResponseType } from "@/app/api/__generated__/models/responseType";
import type { SuggestedGoalResponse } from "@/app/api/__generated__/models/suggestedGoalResponse";
import {
@@ -16,9 +13,6 @@ import type { ToolUIPart } from "ai";
import { OrbitLoader } from "../../components/OrbitLoader/OrbitLoader";
export type CreateAgentToolOutput =
| OperationStartedResponse
| OperationPendingResponse
| OperationInProgressResponse
| AgentPreviewResponse
| AgentSavedResponse
| ClarificationNeededResponse
@@ -39,9 +33,6 @@ function parseOutput(output: unknown): CreateAgentToolOutput | null {
if (typeof output === "object") {
const type = (output as { type?: unknown }).type;
if (
type === ResponseType.operation_started ||
type === ResponseType.operation_pending ||
type === ResponseType.operation_in_progress ||
type === ResponseType.agent_preview ||
type === ResponseType.agent_saved ||
type === ResponseType.clarification_needed ||
@@ -50,9 +41,6 @@ function parseOutput(output: unknown): CreateAgentToolOutput | null {
) {
return output as CreateAgentToolOutput;
}
if ("operation_id" in output && "tool_name" in output)
return output as OperationStartedResponse | OperationPendingResponse;
if ("tool_call_id" in output) return output as OperationInProgressResponse;
if ("agent_json" in output && "agent_name" in output)
return output as AgentPreviewResponse;
if ("agent_id" in output && "library_agent_id" in output)
@@ -72,30 +60,6 @@ export function getCreateAgentToolOutput(
return parseOutput((part as { output?: unknown }).output);
}
export function isOperationStartedOutput(
output: CreateAgentToolOutput,
): output is OperationStartedResponse {
return (
output.type === ResponseType.operation_started ||
("operation_id" in output && "tool_name" in output)
);
}
export function isOperationPendingOutput(
output: CreateAgentToolOutput,
): output is OperationPendingResponse {
return output.type === ResponseType.operation_pending;
}
export function isOperationInProgressOutput(
output: CreateAgentToolOutput,
): output is OperationInProgressResponse {
return (
output.type === ResponseType.operation_in_progress ||
"tool_call_id" in output
);
}
export function isAgentPreviewOutput(
output: CreateAgentToolOutput,
): output is AgentPreviewResponse {
@@ -144,10 +108,6 @@ export function getAnimationText(part: {
case "output-available": {
const output = parseOutput(part.output);
if (!output) return "Creating a new agent";
if (isOperationStartedOutput(output)) return "Agent creation started";
if (isOperationPendingOutput(output)) return "Agent creation in progress";
if (isOperationInProgressOutput(output))
return "Agent creation already in progress";
if (isAgentSavedOutput(output)) return `Saved ${output.agent_name}`;
if (isAgentPreviewOutput(output)) return `Preview "${output.agent_name}"`;
if (isClarificationNeededOutput(output)) return "Needs clarification";

View File

@@ -4,17 +4,14 @@ import { WarningDiamondIcon } from "@phosphor-icons/react";
import type { ToolUIPart } from "ai";
import { useCopilotChatActions } from "../../components/CopilotChatActionsProvider/useCopilotChatActions";
import { MorphingTextAnimation } from "../../components/MorphingTextAnimation/MorphingTextAnimation";
import { OrbitLoader } from "../../components/OrbitLoader/OrbitLoader";
import {
ContentCardDescription,
ContentCodeBlock,
ContentGrid,
ContentHint,
ContentLink,
ContentMessage,
} from "../../components/ToolAccordion/AccordionContent";
import { ToolAccordion } from "../../components/ToolAccordion/ToolAccordion";
import { MiniGame } from "../CreateAgent/components/MiniGame/MiniGame";
import {
ClarificationQuestionsCard,
ClarifyingQuestion,
@@ -28,9 +25,6 @@ import {
isAgentSavedOutput,
isClarificationNeededOutput,
isErrorOutput,
isOperationInProgressOutput,
isOperationPendingOutput,
isOperationStartedOutput,
ToolIcon,
truncateText,
type EditAgentToolOutput,
@@ -75,17 +69,6 @@ function getAccordionMeta(output: EditAgentToolOutput): {
description: `${questions.length} question${questions.length === 1 ? "" : "s"}`,
};
}
if (
isOperationStartedOutput(output) ||
isOperationPendingOutput(output) ||
isOperationInProgressOutput(output)
) {
return {
icon: <OrbitLoader size={32} />,
title: "Editing agent, this may take a few minutes. Play while you wait.",
expanded: true,
};
}
return {
icon: (
<WarningDiamondIcon size={32} weight="light" className="text-red-500" />
@@ -104,18 +87,10 @@ export function EditAgentTool({ part }: Props) {
const output = getEditAgentToolOutput(part);
const isError =
part.state === "output-error" || (!!output && isErrorOutput(output));
const isOperating =
!!output &&
(isOperationStartedOutput(output) ||
isOperationPendingOutput(output) ||
isOperationInProgressOutput(output));
const hasExpandableContent =
part.state === "output-available" &&
!!output &&
(isOperationStartedOutput(output) ||
isOperationPendingOutput(output) ||
isOperationInProgressOutput(output) ||
isAgentPreviewOutput(output) ||
(isAgentPreviewOutput(output) ||
isAgentSavedOutput(output) ||
isClarificationNeededOutput(output) ||
isErrorOutput(output));
@@ -150,15 +125,6 @@ export function EditAgentTool({ part }: Props) {
{hasExpandableContent && output && (
<ToolAccordion {...getAccordionMeta(output)}>
{isOperating && (
<ContentGrid>
<MiniGame />
<ContentHint>
This could take a few minutes play while you wait!
</ContentHint>
</ContentGrid>
)}
{isAgentSavedOutput(output) && (
<ContentGrid>
<ContentMessage>{output.message}</ContentMessage>

View File

@@ -2,9 +2,6 @@ import type { AgentPreviewResponse } from "@/app/api/__generated__/models/agentP
import type { AgentSavedResponse } from "@/app/api/__generated__/models/agentSavedResponse";
import type { ClarificationNeededResponse } from "@/app/api/__generated__/models/clarificationNeededResponse";
import type { ErrorResponse } from "@/app/api/__generated__/models/errorResponse";
import type { OperationInProgressResponse } from "@/app/api/__generated__/models/operationInProgressResponse";
import type { OperationPendingResponse } from "@/app/api/__generated__/models/operationPendingResponse";
import type { OperationStartedResponse } from "@/app/api/__generated__/models/operationStartedResponse";
import { ResponseType } from "@/app/api/__generated__/models/responseType";
import {
NotePencilIcon,
@@ -15,9 +12,6 @@ import type { ToolUIPart } from "ai";
import { OrbitLoader } from "../../components/OrbitLoader/OrbitLoader";
export type EditAgentToolOutput =
| OperationStartedResponse
| OperationPendingResponse
| OperationInProgressResponse
| AgentPreviewResponse
| AgentSavedResponse
| ClarificationNeededResponse
@@ -38,8 +32,6 @@ function parseOutput(output: unknown): EditAgentToolOutput | null {
const type = (output as { type?: unknown }).type;
if (
type === ResponseType.operation_started ||
type === ResponseType.operation_pending ||
type === ResponseType.operation_in_progress ||
type === ResponseType.agent_preview ||
type === ResponseType.agent_saved ||
type === ResponseType.clarification_needed ||
@@ -47,9 +39,6 @@ function parseOutput(output: unknown): EditAgentToolOutput | null {
) {
return output as EditAgentToolOutput;
}
if ("operation_id" in output && "tool_name" in output)
return output as OperationStartedResponse | OperationPendingResponse;
if ("tool_call_id" in output) return output as OperationInProgressResponse;
if ("agent_json" in output && "agent_name" in output)
return output as AgentPreviewResponse;
if ("agent_id" in output && "library_agent_id" in output)
@@ -68,30 +57,6 @@ export function getEditAgentToolOutput(
return parseOutput((part as { output?: unknown }).output);
}
export function isOperationStartedOutput(
output: EditAgentToolOutput,
): output is OperationStartedResponse {
return (
output.type === ResponseType.operation_started ||
("operation_id" in output && "tool_name" in output)
);
}
export function isOperationPendingOutput(
output: EditAgentToolOutput,
): output is OperationPendingResponse {
return output.type === ResponseType.operation_pending;
}
export function isOperationInProgressOutput(
output: EditAgentToolOutput,
): output is OperationInProgressResponse {
return (
output.type === ResponseType.operation_in_progress ||
"tool_call_id" in output
);
}
export function isAgentPreviewOutput(
output: EditAgentToolOutput,
): output is AgentPreviewResponse {
@@ -132,10 +97,6 @@ export function getAnimationText(part: {
case "output-available": {
const output = parseOutput(part.output);
if (!output) return "Editing the agent";
if (isOperationStartedOutput(output)) return "Agent update started";
if (isOperationPendingOutput(output)) return "Agent update in progress";
if (isOperationInProgressOutput(output))
return "Agent update already in progress";
if (isAgentSavedOutput(output)) return `Saved "${output.agent_name}"`;
if (isAgentPreviewOutput(output)) return `Preview "${output.agent_name}"`;
if (isClarificationNeededOutput(output)) return "Needs clarification";

View File

@@ -16,7 +16,7 @@ import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { useChatSession } from "./useChatSession";
import { useLongRunningToolPolling } from "./hooks/useLongRunningToolPolling";
const STREAM_START_TIMEOUT_MS = 12_000;
const STREAM_START_TIMEOUT_MS = 30_000; // 30s to detect if backend is down/not responding
/** Mark any in-progress tool parts as completed/errored so spinners stop. */
function resolveInProgressTools(
@@ -203,8 +203,9 @@ export function useCopilotPage() {
prevStatusRef.current = status;
const wasActive = prev === "streaming" || prev === "submitted";
const isIdle = status === "ready" || status === "error";
if (wasActive && isIdle && sessionId) {
const isReady = status === "ready";
// Only invalidate on successful completion, not on error to avoid infinite refetch loop
if (wasActive && isReady && sessionId) {
queryClient.invalidateQueries({
queryKey: getGetV2GetSessionQueryKey(sessionId),
});