From ca5abff93fb5e170788415f23d2bf098a26dc4f6 Mon Sep 17 00:00:00 2001 From: merwanehamadi Date: Sat, 6 May 2023 09:48:08 -0700 Subject: [PATCH] add information retrieval challenge to the wiki (#3876) --- .../information_retrieval/challenge_a.md | 20 +++++++++++++++++++ .../information_retrieval/introduction.md | 3 +++ mkdocs.yml | 3 +++ 3 files changed, 26 insertions(+) create mode 100644 docs/challenges/information_retrieval/challenge_a.md create mode 100644 docs/challenges/information_retrieval/introduction.md diff --git a/docs/challenges/information_retrieval/challenge_a.md b/docs/challenges/information_retrieval/challenge_a.md new file mode 100644 index 0000000000..51762fc421 --- /dev/null +++ b/docs/challenges/information_retrieval/challenge_a.md @@ -0,0 +1,20 @@ +# Information Retrieval Challenge A + +**Status**: Current level to beat: level 1 + +**Command to try**: + +``` +pytest -s tests/integration/challenges/information_retrieval/test_information_retrieval_challenge_a.py +``` + +## Description + +The agent's goal is to find the revenue of Tesla in 2022. + +It should write the result in a file called output.txt. + +The agent should be able to beat this test consistently (this is the hardest part). +## Objective + +The objective of this challenge is to test the agent's ability to retrieve information in a consistent way. diff --git a/docs/challenges/information_retrieval/introduction.md b/docs/challenges/information_retrieval/introduction.md new file mode 100644 index 0000000000..2e997d7a7f --- /dev/null +++ b/docs/challenges/information_retrieval/introduction.md @@ -0,0 +1,3 @@ +# Information Retrieval + +Information retrieval challenges are designed to evaluate the proficiency of an AI agent, such as Auto-GPT, in searching, extracting, and presenting relevant information from a vast array of sources. These challenges often encompass tasks such as interpreting user queries, browsing the web, and filtering through unstructured data. diff --git a/mkdocs.yml b/mkdocs.yml index 6a7d0464fa..13b86eb75f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -24,6 +24,9 @@ nav: - Introduction: challenges/memory/introduction.md - Memory Challenge A: challenges/memory/challenge_a.md - Memory Challenge B: challenges/memory/challenge_b.md + - Information retrieval: + - Introduction: challenges/information_retrieval/introduction.md + - Information Retrieval Challenge A: challenges/information_retrieval/challenge_a.md - Submit a Challenge: challenges/submit.md - Beat a Challenge: challenges/beat.md