Nicholas Tindle b849eafb7f feat(direct_benchmark): enable shell command execution with safety denylist ...

Enable agents to execute shell commands during benchmarks by setting
execute_local_commands=True and using denylist mode to block dangerous
commands (rm, sudo, chmod, kill, etc.) while allowing safe operations.

Also adds ExecutePython challenge to test code execution capability.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-20 00:52:06 -06:00

..

code

feat(direct_benchmark): enable shell command execution with safety denylist

2026-01-20 00:52:06 -06:00

data

refactor(classic): migrate from agbenchmark to direct_benchmark harness

2026-01-19 22:29:51 -06:00

scrape

refactor(classic): migrate from agbenchmark to direct_benchmark harness

2026-01-19 22:29:51 -06:00

synthesize/1_basic_content_gen

refactor(classic): migrate from agbenchmark to direct_benchmark harness

2026-01-19 22:29:51 -06:00