Commit Graph

29 Commits

Author SHA1 Message Date
Wael Karkoub
a9171211c7 Streamline Testing with pytest-cov and pytest Defaults (#2490)
* done

* update docs

* try fix

* update workflows

* undo minor fix

* resolve comments

* adds back pytest-asyncio

* minor fix

* add branch coverage

* restore pip install e.

* test with coverage

* fix mypy

* fix coverage + docker + windows combo

* fix bash command

* formatter

* formatter

* one last fix

* I lied, last fix

* fix

* fix retrieve chat test

* fix windows paths

* change cache seed

* down grade openai version

* fix openai mypy

* better error type

* fix image gen cache test

* fix

* experimenting

* fix lmm

* skip cosmos test

* remove cosmos db

* unused imports

* handle more cosmosdb skips

* fix flaky test
2024-04-29 00:27:34 +00:00
Audel Rouhi
1b8d65df0a 2447 fix pgvector tests and notebook (#2455)
* Re-added missing notebook

* Test installing postgres

* Error handle the connection.

* Fixed import.

* Fixed import.

* Fixed creation of collection without client.

* PGVector portion working. OpenAI untested.

* Fixed prints.

* Added output.

* Fixed pre-commits.

* Run pgvector notebook

* Improve efficiency of get_collection

* Fix delete_collection

* Fixed issues with pytests and validated functions.

* Validated pytests.

* Fixed pre-commits

* Separated extra_requires to allow more logic. Retrieve_chat base dependencies included on pgvector and qdrant.

* Fixed extra newline.

* Added username and password fields.

* URL Encode the connection string parameters to support symbols like %

* Fixed pre-commits.

* Added pgvector service

* pgvector doesn't have health intervals.

* Switched to colon based key values.

* Run on Ubuntu only. Linux is only option with container service support.

* Using default credentials instead.

* Fix postgres setup

* Fix postgres setup

* Don't skip tests on win and mac

* Fix command error

* Try apt install postgresql

* Assert table does not exist when deleted.

* Raise value error on a empty list or None value provided for IDs

* pre-commit

* Add install pgvector

* Add install pgvector

* Reorg test files, create a separate job for test pgvector

* Fix format

* Fix env format

* Simplify job name, enable test_retrieve_config

* Fix test_retrieve_config

* Corrected behavior for get_docs_by_ids with no ids returning all docs.

* Corrected behavior for get_docs_by_ids with no ids returning all docs.

* Fixed pre-commits.

* Added return values for all functions.

* Validated distance search is implemented correctly.

* Validated all pytests

* Removed print.

* Added default clause.

* Make ids optional

* Fix test, make it more robust

* Bump version of openai for the vector_store support

* Added support for choosing the sentence transformer model.

* Added error handling for model name entered.

* Updated model info.

* Added model_name db_config param.

* pre-commit fixes and last link fix.

* Use secrets password.

* fix: link fixed

* updated tests

* Updated config_list.

* pre-commit fix.

* Added chat_result to all output.
Unable to re-run notebooks.

* Pre-commit fix detected this requirement.

* Fix python 3.8 and 3.9 not supported for macos

* Fix python 3.8 and 3.9 not supported for macos

* Fix format

* Reran notebook with MetaLlama3Instruct7BQ4_k_M

* added gpt model.

* Reran notebook

---------

Co-authored-by: Li Jiang <bnujli@gmail.com>
Co-authored-by: Hk669 <hrushi669@gmail.com>
2024-04-28 13:43:02 +00:00
Li Jiang
2a4ccd0622 Fix python not supported in macos (#2503)
* Fix python 3.8 and 3.9 not supported for macos

* Fix python 3.8 and 3.9 not supported for macos

* Fix format
2024-04-25 14:33:36 +00:00
Audel Rouhi
ded2d612c3 [Re-Opened] Support for PGVector Database in Autogen (#2439)
* PGVector Contrib Initial Commit - KnucklesTeam:autogen:pgvector_contrib fork

* Update website/docs/ecosystem/pgvector.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Updated qdrant installation instructions.

* Fixed openai version.

* Added dependencies to install for qdrant and pgvector in contrib tests.

* Added dependencies to install for qdrant and pgvector in contrib tests.

* Cleaned up dependencies.

* Removed flaml out of setup.py. Used only for notebook example.

* Added PGVector notebook link

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-04-18 20:00:03 +00:00
Beibin Li
0aaf30a8da Merge "Gemini" feature into the main branch (#2360)
* Start Gemini integration: works ok with Text now

* Gemini notebook lint

* try catch "import" for Gemini

* Debug: id issue for chat completion in Gemini

* Add RAG example

* Update docs for RAG

* Fix missing pydash

* Remove temp folder

* Fix test error in runs/7206014032/job/19630042864

* Fix tqdm warning

* Fix notebook output

* Gemini's vision model is supported now

* Install instructions for the Gemini branch

* Catch and retry when see Interval Server Error 500

* Allow gemini to take more flexible messages
i.e., it can take messages where "user" is not the last role.

* Use int time for Gemini client

* Handle other exceptions in gemini call

* rename to "create" function for gemini

* GeminiClient compatible with ModelClient now

* Lint

* Update instructions in Gemini notebook

* Lint

* Remove empty blocks from Gemini notebook

* Add gemini into example page

* self.create instead of call

* Add py and Py into python execution

* Remove error code from merging

* Remove pydash dependency for gemini

* Add cloud-gemini doc

* Remove temp file

* cache import update

* Add test case for summary with mm input

* Lint: warnings instead of print

* Add test cases for gemini

* Gemini test config

* Disable default model for gemini

* Typo fix in gemini workflow

* Correct grammar in example notebook

* Raise if "model" is not provided in create(...)

* Move TODOs into a roadmap

* Update .github/workflows/contrib-tests.yml

Co-authored-by: Davor Runje <davor@airt.ai>

* Gemini test config update

* Update setup.py

Co-authored-by: Davor Runje <davor@airt.ai>

* Update test/oai/test_gemini.py

Co-authored-by: Davor Runje <davor@airt.ai>

* Update test/oai/test_gemini.py

Co-authored-by: Davor Runje <davor@airt.ai>

* Remove python 3.8 from gemini
No google's generativeai for Windows with Python 3.8

* Update import error handling for gemini

* Count tokens and cost for gemini

---------

Co-authored-by: Li Jiang <bnujli@gmail.com>
Co-authored-by: Davor Runje <davor@airt.ai>
2024-04-17 00:24:07 +00:00
Li Jiang
adb8fd849c Add a vectordb module (#2263)
* Added vectordb base and chromadb

* Remove timer and unused functions

* Added filter by distance

* Added test utils

* Fix format

* Fix type hint of dict

* Rename test

* Add test chromadb

* Fix test no chromadb

* Add coverage

* Don't skip test vectordb utils

* Add types

* Fix tests

* Fix docs build error

* Add types to base

* Update base

* Update utils

* Update chromadb

* Add get_docs_by_ids

* Improve docstring

* Add get all docs

* Move chroma_results_to_query_results to utils

* Improve type hints

* Update logger

* Update init, add embedding func

* Improve docstring of vectordb, add two attributes

* Improve test workflow
2024-04-10 17:43:08 +00:00
Li Jiang
0d99d45b0f Fix FileNotFoundError english-words.txt (#2284) 2024-04-05 04:59:57 +00:00
Eric Zhu
d4aa27a21b Fix LMM test in CI by installing lfs (#2282) 2024-04-05 02:39:00 +00:00
Li Jiang
eec854279a Fix unstructured deps installation error (#2248) 2024-04-02 05:17:13 +00:00
Wael Karkoub
8cb5c14c82 Transform Messages Capability (#1923)
* wip

* Adds docstrings

* fixed spellings

* wip

* fixed errors

* better class names

* adds tests

* added tests to workflow

* improved token counting

* improved notebook

* improved token counting in test

* improved docstrings

* fix inconsistencies

* changed by mistake

* fixed docstring

* fixed details

* improves tests + adds openai contrib test

* fix spelling oai contrib test

* clearer docstrings

* remove repeated docstr

* improved notebook

* adds metadata to notebook

* Improve outline and description (#2125)

* better dir structure

* clip max tokens to allowed tokens

* more accurate comments/docstrs

* add deperecation warning

* fix front matter desc

* add deperecation warning notebook

* undo local notebook settings changes

* format notebook

* format workflow

---------

Co-authored-by: gagb <gagb@users.noreply.github.com>
2024-03-28 16:16:17 +00:00
Beibin Li
2f109f5f94 Add vision capability (#2025)
* Add vision capability

* Configurate: description_prompt

* Print warning instead of raising issues for type

* Skip vision capability test if dependencies not installed

* Append "vision" to agent's system message when enabled VisionCapability

* GPT-4V notebook update with ConversableAgent

* Clean GPT-4V notebook

* Add vision capability test to workflow

* Lint import

* Update system message for vision capability

* Add a `custom_caption_func` to VisionCapability

* Add custom function example for vision capability

* Skip test Vision capability custom func

* GPT-4V notebook metadata to website

* Remove redundant files

* The custom caption function takes more inputs now

* Add a more complex example of custom caption func

* Remove trailing space

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-03-24 19:46:55 +00:00
Davor Runje
4af0440730 Update GitHub actions (#2078)
* Add mypy check in pre-commit

* Add mypy check in pre-commit

* bug fix

* CI fix

* add python version matrix for checking types in CI

* update github actions

* revert some accidental changes

* revert some accidental changes

* revert some accidental changes

* revert some accidental changes

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-03-20 17:18:11 +00:00
Wael Karkoub
c5536ee92b [Feature] Adds Image Generation Capability 2.0 (#1907)
* adds image generation capability

* add todo

* readded cache

* wip

* fix content str bugs

* removed todo: delete imshow

* wip

* fix circular imports

* add notebook

* improve prompt

* improved text analyzer + notebook

* notebook update

* improve notebook

* smaller notebook size

* made changes to the wrong branch :(

* resolve comments + 1

* adds doc strings

* adds cache doc string

* adds doc string to add_to_agent

* adds doc string to ImageGeneration

* instructions are not configurable

* removed unnecessary imports

* changed doc string location

* more doc strings

* improves testability

* adds tests

* adds cache test

* added test to github workflow

* compatible llm config format

* configurable reply function position

* skip_openai + better comments

* fix test

* fix test?

* please fix test?

* last fix test?

* remove type hint

* skip cache test

* adds mock api key

* dalle-2 test

* fix dalle config

* use apu key function

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-03-15 22:11:53 +00:00
Chi Wang
cd3b5c6bbd add doc about effects for capabilities (#1842)
* add doc about effects for capabilities

* remove unnecessary imports

* improve doc

* test

* test location

* polish

* improve import of colored

* termcolor

* termcolor

* use pull request for openai test
2024-03-03 23:15:52 +00:00
gagb
47d6c7567e Proxy PR for Long Context Capability 1513 (#1591)
* Add new capability to handle long context

* Make print conditional

* Remove superfluous comment

* Fix msg order

* Allow user to specify max_tokens

* Add ability to specify max_tokens per message; improve name

* Improve doc and readability

* Add tests

* Improve documentation and add tests per Erik and Chi's feedback

* Update notebook

* Update doc string of add to agents

* Improve doc string

* improve notebook

* Update github workflows for context handling

* Update docstring

* update notebook to use raw config list.

* Update contrib-openai.yml remove _target

* Fix code formatting

* Fix workflow file

* Update .github/workflows/contrib-openai.yml

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-02-08 18:26:00 +00:00
Chi Wang
fd0094ce51 bump version to 0.2.9 (#1430)
* Fixed rounding bug.

* bump version to 0.2.9

* fix format error

* simplify contrib tests

---------

Co-authored-by: Adam Fourney <adamfo@microsoft.com>
2024-01-27 18:49:24 +00:00
Olaoluwa Ademola Salami
bcfd770a34 test permissions (#1264)
* test permissions

* test least permissions

* remove contents permissions

* testing permisions

* Update deploy-website.yml permission block

---------

Co-authored-by: Davor Runje <davor@airt.ai>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-01-24 21:50:37 +00:00
afourney
708eb4d884 Add a web surfer agent that can search and browse the web. (#1093)
* Initial commit of WebSurfer. Adds the browser_utils, and related tests. WebSurfer will be added in a subsequent commit.

* Added the web surfer agent, and related tests.

* Added a notebook to show how WebSurferAgent works.

* Fixed a typo.

* Updated test_web_surfer for compatibility with #1110.

* Updated skip_oai logic.

* Fixed code formatting.

* More pre-commit fixes.

* Added block to contrib-openai.yml

* Added block to contrib-openai.yml

* Added hook for BING_API_KEY

* Temporarily commented out other tests, per request.

* Fixed indentation (maybe?)

* Restoring contrib-openai.yml
2024-01-22 03:43:15 +00:00
olgavrou
a911d1c2ec set use_docker to default to True (#1147)
* set use_docker to default to true

* black formatting

* centralize checking and add env variable option

* set docker env flag for contrib tests

* set docker env flag for contrib tests

* better error message and cleanup

* disable explicit docker tests

* docker is installed so can't check for that in test

* pr comments and fix test

* rename and fix function descriptions

* documentation

* update notebooks so that they can be run with change in default

* add unit tests for new code

* cache and restore env var

* skip on windows because docker is running in the CI but there are problems connecting the volume

* update documentation

* move header

* update contrib tests
2024-01-18 17:03:49 +00:00
Eric Zhu
22e36cbb10 Support for Python 3.12 (#1317)
* support for Python 3.12

* add python 3.12 to workflows

* version string fix

* retrieval chat

* teachability

* workflow

* redistribute notebook tests

* fix incorrect notebook introduction

* update banner; remove unused imports; update openai workflow

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-01-18 07:15:06 +00:00
Maxim Saplin
083ba32a73 Change contrib-tests.yml tests to use --skip-openai (#1132)
* Fixing RetrieveChat part

* All 5 test groups are updated to --skip-openai

* sys.path.append() fix
2024-01-10 01:42:07 +00:00
Ricky Loynd
3680197f70 Teachability for any agent (#1091)
* Partial implementation

* Partial implementation

* Fixes

* update tests

* cleanup

* update tests

* comments

* logging

* wording

* underscore

* Extend notebook for teachable GPTAssistantAgent

* Notebook for teachable GPTAssistantAgents

* Update notebook

* Update notebook

* Update notebook

* Update notebook

* revert file

* Update blog post and other documentation.

* pre-commit

* Address reviewer feedback.

* Add new nb link to examples page.

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-01-07 19:59:42 +00:00
Ian
70f4c1c52d Enhanced Independent Coverage Reporting (#886)
* Independent Test Execution and Coverage Reporting

* format

* Apply suggestions from code review

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* add comments

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2023-12-09 23:35:15 +00:00
Beibin Li
c19f234149 Message "content" now supports both str and List in Agents (#713)
* Change "content" type in Conversable Agent

* content and system_message support str and List
Update for all other agents

* Content_str now also takes None as input

* Group Chat now works with LMM too

* Style: newline for import in Conversable Agentt

* Add test for gourpchat + lmm

* Resolve comments
1. Undo AssistantAgent changes
2. Modify the asserts and raises in `content_str` function and update
test accordingly.

* Undo AssistantAgent

* Update comments and add assertion for LMM

* Typo fix in docstring for content_str

* Remove “None” out conversable_agent.py

* Lint message to dict in multimodal_conversable_agent.py

* Address lint issues

* linting

* Move lmm test into contrib test

* Resolve 2 comments

* Move img_utils into contrib folder

* Resolve img_utils path issues
2023-12-03 01:40:50 +00:00
Ricky Loynd
0468f53210 Restore TeachableAgent tests (#761)
* Update chat_with_teachable_agent.py to v2.

* Update agentchat_teachability.ipynb to v2.

* Add test of teachability accuracy.

* Update installation instructions.

* Add to contrib tests.

* pre-commit fixes

* Apply reviewer suggestions to test workflows.
2023-11-27 02:10:02 +00:00
Li Jiang
07646d448c Support custom text formats and recursive (#496)
* Add custom text types and recursive

* Add custom text types and recursive

* Fix format

* Update qdrant, Add pdf to unstructured

* Use unstructed as the default text extractor if installed

* Add tests for unstructured

* Update tests env for unstructured

* Fix error if last message is a function call, issue #569

* Remove csv, md and tsv from UNSTRUCTURED_FORMATS

* Update docstring of docs_path

* Update test for get_files_from_dir

* Update docstring of custom_text_types

* Fix missing search_string in update_context

* Add custom_text_types to notebook example
2023-11-21 03:53:50 +00:00
gagb
df2cd36dee Refactor GPTAssistantAgent (#632)
* Refactor GPTAssistantAgent constructor to handle
instructions and overwrite_instructions flag

- Ensure that `system_message` is always consistent with `instructions`
- Ensure provided instructions are always used
- Add option to permanently modify the instructions of the assistant

* Improve default behavior

* Add a test; add method to delete assistant

* Add a new test for overwriting instructions

* Add test case for when no instructions are given for existing assistant

* Add pytest markers to test_gpt_assistant.py

* add test in workflow

* update

* fix test_client_stream

* comment out test_hierarchy_

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: kevin666aa <yrwu000627@gmail.com>
2023-11-12 06:03:51 +00:00
Yiran Wu
19ea8b8b7a Add CompressibleAgent (#443)
* api_base -> base_url (#383)

* InvalidRequestError -> BadRequestError (#389)

* remove api_key_path; close #388

* close #402 (#403)

* openai client (#419)

* openai client

* client test

* _client -> client

* _client -> client

* extra kwargs

* Completion -> client (#426)

* Completion -> client

* Completion -> client

* Completion -> client

* Completion -> client

* support aoai

* fix test error

* remove commented code

* support aoai

* annotations

* import

* reduce test

* skip test

* skip test

* skip test

* debug test

* rename test

* update workflow

* update workflow

* env

* py version

* doc improvement

* docstr update

* openai<1

* add compressibleagent

* revise doc, add tests, add example

* fix link

* fix link

* fix link

* remove test

* update doc

* update doc

* add tiktoken to dependency

* filter_func

* async test

* dependency

* revision

* migration guide (#477)

* migration guide

* change in kwargs

* simplify header

* update optigude description

* update for dev

* revision

* revision

* allow not compressing last n msgs

* update

* correct merge

* update test workflow

* check test

* update for test

* update

* update notebook

* update

* fix bug

* update

* update

* update

* check to "pull_request_target" in contrib-openai

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2023-11-10 04:11:28 +00:00
Chi Wang
c4f8b1c761 Dev/v0.2 (#393)
* api_base -> base_url (#383)

* InvalidRequestError -> BadRequestError (#389)

* remove api_key_path; close #388

* close #402 (#403)

* openai client (#419)

* openai client

* client test

* _client -> client

* _client -> client

* extra kwargs

* Completion -> client (#426)

* Completion -> client

* Completion -> client

* Completion -> client

* Completion -> client

* support aoai

* fix test error

* remove commented code

* support aoai

* annotations

* import

* reduce test

* skip test

* skip test

* skip test

* debug test

* rename test

* update workflow

* update workflow

* env

* py version

* doc improvement

* docstr update

* openai<1

* add tiktoken to dependency

* filter_func

* async test

* dependency

* migration guide (#477)

* migration guide

* change in kwargs

* simplify header

* update optigude description

* deal with azure gpt-3.5

* add back test_eval_math_responses

* timeout

* Add back tests for RetrieveChat (#480)

* Add back tests for RetrieveChat

* Fix format

* Update dependencies order

* Fix path

* Fix path

* Fix path

* Fix tests

* Add not run openai on MacOS or Win

* Update skip openai tests

* Remove unnecessary dependencies, improve format

* Add py3.8 for testing qdrant

* Fix multiline error of windows

* Add openai tests

* Add dependency mathchat, remove unused envs

* retrieve chat is tested

* bump version to 0.2.0b1

---------

Co-authored-by: Li Jiang <bnujli@gmail.com>
2023-11-04 04:01:49 +00:00