Zamil Majdy 91dd9364bb fix(backend): implement retry mechanism for SmartDecisionMaker tool call validation (#11015) ...

<!-- Clearly explain the need for these changes: -->

This PR fixes a critical production issue where SmartDecisionMakerBlock
was silently accepting tool calls with typo'd parameter names (e.g.,
'maximum_keyword_difficulty' instead of 'max_keyword_difficulty'),
causing downstream blocks to receive null values and execution failures.

The solution implements comprehensive parameter validation with
automatic retry when the LLM provides malformed tool calls, giving the
LLM specific feedback to correct the errors.

### Changes 🏗️

<!-- Concisely describe all of the changes made in this pull request:
-->

**Core Validation & Retry Logic
(`backend/blocks/smart_decision_maker.py`)**
- Add tool call parameter validation against function schema
- Implement retry mechanism using existing `create_retry_decorator` from
`backend.util.retry`
- Validate provided parameters against expected schema properties and
required fields
- Generate specific error messages for unknown parameters (typos) and
missing required parameters
- Add error feedback to conversation history for LLM learning on retry
attempts
- Use `input_data.retry` field to configure number of retry attempts

**Comprehensive Test Coverage
(`backend/blocks/test/test_smart_decision_maker.py`)**
- Add `test_smart_decision_maker_parameter_validation` with 4
comprehensive test scenarios:
1. Tool call with typo'd parameter (should retry and eventually fail
with clear error)
2. Tool call missing required parameter (should fail immediately with
clear error)
  3. Valid tool call with optional parameter missing (should succeed)
  4. Valid tool call with all parameters provided (should succeed)
- Verify retry mechanism works correctly and respects retry count
- Mock LLM responses for controlled testing of validation logic

**Load Tests Documentation Update (`load-tests/README.md`)**
- Update documentation to reflect current orchestrator-based
architecture
- Remove references to deprecated `run-tests.js` and
`comprehensive-orchestrator.js`
- Streamline documentation to focus on working
`orchestrator/orchestrator.js`
- Update NPM scripts and command examples for current workflow
- Clean up outdated file references to match actual infrastructure

**Production Impact**
- **Prevents silent failures**: Tool call parameter typos now cause
retries instead of null downstream values
- **Maintains compatibility**: No breaking changes to existing
SmartDecisionMaker functionality
- **Improves reliability**: LLM receives feedback to correct parameter
errors
- **Configurable retries**: Uses existing `retry` field for user control
- **Accurate documentation**: Load-tests docs now match actual working
infrastructure

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
- [x] Run existing SmartDecisionMaker tests to ensure no regressions:
`poetry run pytest backend/blocks/test/test_smart_decision_maker.py
-xvs` ✅ All 4 tests passed
- [x] Run new parameter validation test specifically: `poetry run pytest
backend/blocks/test/test_smart_decision_maker.py::test_smart_decision_maker_parameter_validation
-xvs` ✅ Passed with retry behavior confirmed
- [x] Verify retry mechanism works by checking log output for retry
attempts ✅ Confirmed in test logs
- [x] Test tool call validation with different scenarios (typos, missing
params, valid calls) ✅ All scenarios covered and working
- [x] Run code formatting and linting: `poetry run format` ✅ All
formatters passed
- [x] Verify no breaking changes to existing SmartDecisionMaker
functionality ✅ All existing tests pass
- [x] Verify load-tests documentation accuracy ✅ README now matches
actual orchestrator infrastructure

#### For configuration changes:

- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

**Note**: No configuration changes were needed as this uses existing
retry infrastructure and block schema validation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

2025-09-30 16:18:05 +00:00

..

orchestrator.js

fix(backend): implement retry mechanism for SmartDecisionMaker tool call validation (#11015)

2025-09-30 16:18:05 +00:00