HumanEvalFix integration (#1908)

* Preliminary HumanEvalFix integration

* Clean paths

* fix: set workspace path correctly for config
fix: task in that contains /

* add missing run_infer.sh

* update run_infer w/o hard coded agent

* fix typo

* change `instance_id` to `task_id`

* add the warning and env var setting to run_infer.sh

* reset back workspace mount at the end of each instance

* 10 max iter is probably enough for humanevalfix

* Remove unneeded section

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>

* Fix link

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Use logger

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>

* Update run_infer.py

fix a bug:

ERROR:concurrent.futures:exception calling callback for <Future at 0x309cbc470 state=finished raised NameError>
concurrent.futures.process._RemoteTraceback:

* Update README.md

* Update README.md

* Update README.md

* Update README.md

added an example

* Update README.md

added: enable_auto_lint = true

* Update pyproject.toml

add: evaluate package

* Delete poetry.lock

update poetry.lock

* update poetry.lock

update poetry.lock

* Update README.md

* Update README.md

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
Co-authored-by: Robert <871607149@qq.com>
This commit is contained in:
Niklas Muennighoff
2024-05-23 06:09:40 -07:00
committed by GitHub
parent f53a91b17c
commit ef6cdb7532
7 changed files with 664 additions and 2 deletions

4
poetry.lock generated
View File

@@ -1,4 +1,4 @@
# This file is automatically @generated by Poetry 1.8.2 and should not be changed by hand.
# This file is automatically @generated by Poetry 1.8.3 and should not be changed by hand.
[[package]]
name = "aenum"
@@ -7327,4 +7327,4 @@ testing = ["coverage (>=5.0.3)", "zope.event", "zope.testing"]
[metadata]
lock-version = "2.0"
python-versions = "^3.11"
content-hash = "56eda3b68982ff0d01c77ef11e591fb12c4d0b786357638ca65d5008f9b6d65a"
content-hash = "797df3be428ae7c43c8b858b222b0c32e386345df9bac9711c261ec8ad481b94"