5 Commits

Author	SHA1	Message	Date
Graham Neubig	cab7a288ca	Add NUM_WORKERS variable to run_infer.sh scripts for configurable woker settings (#2597) ... * Add NUM_WORKERS variable to run_infer.sh scripts for configurable worker settings * Update evaluation/webarena/scripts/run_infer.sh --------- Co-authored-by: OpenDevin <opendevin@all-hands.dev>	2024-06-23 03:43:43 +00:00
Boxuan Li	feabc97aba	Evaluation time travel: build sandbox on the fly (#2491)	2024-06-20 20:22:02 -06:00
Boxuan Li	6f235937cf	Evaluation time travel: allow evaluation on a specific version (#2356) ... * Time travel for evaluation * Fix source script path * Exit script if given version doesn't exist * Exit on failure * Update README * Change scripts of all other benchmarks * Modify README files * Fix logic_reasoning README	2024-06-16 10:25:14 -04:00
Yizhe Zhang	8d79c3edbc	modify the exiting logic and reward calculation, delete unused function (#2198)	2024-06-02 06:38:09 +00:00
Yizhe Zhang	0c829cd067	Support Entity-Deduction-Arena (EDA) Benchmark (#1931) ... * adding draft evaluation code for EDA, using chatgpt as the temporal agent for now * Update README.md * Delete frontend/package.json * reverse the irrelevant changes * reverse package.json * use chatgpt as the codeactagent * integrate with opendevin * Update evaluation/EDA/README.md * Update evaluation/EDA/README.md * Use poetry to manage packages * integrate with opendevin * minor update * minor update * update poetry * update README * clean-up infer scripts * add run_infer script and improve readme * log final success and final message & ground truth --------- Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xingyao Wang <xingyao6@illinois.edu> Co-authored-by: yufansong <yufan@risingwave-labs.com> Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>	2024-05-25 23:17:04 +08:00