Compare commits

...

1 Commits

Author SHA1 Message Date
Zamil Majdy
dc74d8d89c fix(backend/executor): Fix RabbitMQ channel retry logic in executor
The executor was stuck in a retry loop when RabbitMQ channels were closed,
repeatedly failing with "Channel is closed" errors. The continuous_retry
decorator was attempting to reuse closed channels instead of creating new ones.

This fix ensures that when a channel is closed (detected via is_ready check),
the client disconnects and establishes a fresh connection before attempting
to consume messages. This prevents the infinite retry loop on closed channels.

Changes:
- Added channel state check before connecting in _consume_execution_run
- Added channel state check before connecting in _consume_execution_cancel
- Forces reconnection when channel is not ready to ensure fresh channel

Fixes the issue where executors accumulate 7000+ retry attempts and fail
to process messages from the queue.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-16 20:55:41 +00:00

View File

@@ -1208,6 +1208,9 @@ class ExecutionManager(AppProcess):
)
return
# Check if channel is closed and force reconnection if needed
if not self.cancel_client.is_ready:
self.cancel_client.disconnect()
self.cancel_client.connect()
cancel_channel = self.cancel_client.get_channel()
cancel_channel.basic_consume(
@@ -1237,6 +1240,9 @@ class ExecutionManager(AppProcess):
)
return
# Check if channel is closed and force reconnection if needed
if not self.run_client.is_ready:
self.run_client.disconnect()
self.run_client.connect()
run_channel = self.run_client.get_channel()
run_channel.basic_qos(prefetch_count=self.pool_size)