Commit Graph

13 Commits

Author SHA1 Message Date
Graham Neubig
3a21198424 Remove monologue agent (#3036)
* Remove monologue agent

* Fixes
2024-07-19 19:25:05 +00:00
jigsawlabs-student
fa6c12473e #2220, integrated aider style linting, currently passes related o… (#2489)
* WIP for integrate aider linter, see OpenDevin#2220

Updated aider linter to:
    * Always return text and line numbers
    * Moved extract line number more consistently
    * Changed pylint to stop after first linter detects errors
Updated agentskills
    * To get back a LintResult object and then use lines and text for error message and related line number
    * Moved code for extracting line number to aider linter
Tests:
* Added additional unit tests for aider to test for
* Return values from lint failures
* Confirm linter works for non-configured languages like Ruby

* move to agent_skills, fixes not seeing skills error

* format/lint to new code, fix failing tests, remove unused code from aider linter

* small changes (remove litellm, fix readme typo)

* fix failing sandbox test

* keep, change dumping of metadata

* WIP for integrate aider linter, see OpenDevin#2220

Updated aider linter to:
    * Always return text and line numbers
    * Moved extract line number more consistently
    * Changed pylint to stop after first linter detects errors
Updated agentskills
    * To get back a LintResult object and then use lines and text for error message and related line number
    * Moved code for extracting line number to aider linter
Tests:
* Added additional unit tests for aider to test for
* Return values from lint failures
* Confirm linter works for non-configured languages like Ruby

* move to agent_skills, fixes not seeing skills error

* format/lint to new code, fix failing tests, remove unused code from aider linter

* remove duplication of tree-sitter, grep-ast and update poetry.lock

* revert to main branch poetry.lock version

* only update necessary package

* fix jupyter kernel wrong interpreter issue (only for swebench)

* fix failing lint tests

* update syntax error checks for flake

* update poetry lock file

* update poetry.lock file, which update content-hash

* add grep ast

* remove extra stuff caused by merge

* update pyproject

* remove extra pytest fixture, ruff styling fixes

* lint files

* update poetry.lock file

---------

Co-authored-by: Jeff Katzy <jeffreyerickatz@gmail.com>
Co-authored-by: yufansong <yufan@risingwave-labs.com>
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: tobitege <tobitege@gmx.de>
2024-07-19 21:58:54 +08:00
Xingyao Wang
cf910dfa9d fix eval api_key leak in metadata; fix llm config in run infer (#2998) 2024-07-18 15:46:59 +00:00
Engel Nyst
2df1d67007 History clean up (#2849)
* clean up add_history

* refactor last agent message
2024-07-08 05:10:21 +02:00
Engel Nyst
d37b2973b2 Refactoring: event stream based agent history (#2709)
* add to event stream sync

* remove async from tests

* small logging spam fix

* remove swe agent

* arch refactoring: use history from the event stream

* refactor agents

* monologue agent

* ruff

* planner agent

* micro-agents

* refactor history in evaluations

* evals history refactoring

* adapt evals and tests

* unit testing stuck

* testing micro agents, event stream

* fix planner agent

* fix tests

* fix stuck after rename

* fix test

* small clean up

* fix merge

* fix merge issue

* fix integration tests

* Update agenthub/dummy_agent/agent.py

* fix tests

* rename more clearly; add todo; clean up
2024-07-07 21:04:23 +00:00
Graham Neubig
d0384cafdd Two fixes to swe bench eval (#2831)
* Two fixes to swe bench eval

* Add error message

* Change dumping of metadata
2024-07-07 07:21:50 +00:00
Xingyao Wang
f6dc89b41a [Evaluation] Simplify eval & and multi-processing related fixes (#2810)
* initialize agent inside process_instance_fn;

* remove dependency on `config.max_iterations`

* switch back to only include llm config to metadata
2024-07-06 07:18:46 +08:00
Graham Neubig
a081935fd8 Simplify eval code (#2775)
* Start simplifying eval code

* Update

* Add EDA

* Updated GAIA

* Update gpqa

* Add humanevalfix

* Fix logic_reasoning

* Add miniwob

* Add mint and ml_bench

* toolqa

* Added swe-bench

* Fixed webarena

* Refactor parameters
2024-07-05 19:33:08 +09:00
Graham Neubig
ffd3c7144c Remove global args (#2760)
* Remove global args

* Remove global args

* Update files

* Update main

* Bug fixes

* Fix logging
2024-07-03 20:07:52 +09:00
Engel Nyst
2d9bb56763 Add ability to restore the cli session (optional) (#2699)
* add ability to restore the main session

* add quick log

* rename to cli session
2024-06-30 06:56:55 +00:00
Engel Nyst
874b4c9075 CLI concurrency (#2695)
* add session id in cli, evals

* fix main sid
2024-06-30 04:04:30 +02:00
RainRat
745ae42a72 fix typos (#2352) 2024-06-09 12:57:58 -07:00
yueqis
82d4d25b09 feat: support ToolQA benchmark (#2263)
* Add files via upload

* Update README.md

* Update run_infer.py

* Update utils.py

* make lint

* Update evaluation/toolqa/run_infer.py

---------

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: yufansong <yufan@risingwave-labs.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-06-08 07:54:01 -04:00