150 Commits

Author SHA1 Message Date
Audel Rouhi
ded2d612c3 [Re-Opened] Support for PGVector Database in Autogen (#2439)
* PGVector Contrib Initial Commit - KnucklesTeam:autogen:pgvector_contrib fork

* Update website/docs/ecosystem/pgvector.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Updated qdrant installation instructions.

* Fixed openai version.

* Added dependencies to install for qdrant and pgvector in contrib tests.

* Added dependencies to install for qdrant and pgvector in contrib tests.

* Cleaned up dependencies.

* Removed flaml out of setup.py. Used only for notebook example.

* Added PGVector notebook link

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-04-18 20:00:03 +00:00
Eric Zhu
9f281dad75 Add Git LFS check action, fix broken lfs files (#2436)
* Add images to lfs.

* Add github action for lfs check.
2024-04-18 16:50:15 +00:00
Beibin Li
0aaf30a8da Merge "Gemini" feature into the main branch (#2360)
* Start Gemini integration: works ok with Text now

* Gemini notebook lint

* try catch "import" for Gemini

* Debug: id issue for chat completion in Gemini

* Add RAG example

* Update docs for RAG

* Fix missing pydash

* Remove temp folder

* Fix test error in runs/7206014032/job/19630042864

* Fix tqdm warning

* Fix notebook output

* Gemini's vision model is supported now

* Install instructions for the Gemini branch

* Catch and retry when see Interval Server Error 500

* Allow gemini to take more flexible messages
i.e., it can take messages where "user" is not the last role.

* Use int time for Gemini client

* Handle other exceptions in gemini call

* rename to "create" function for gemini

* GeminiClient compatible with ModelClient now

* Lint

* Update instructions in Gemini notebook

* Lint

* Remove empty blocks from Gemini notebook

* Add gemini into example page

* self.create instead of call

* Add py and Py into python execution

* Remove error code from merging

* Remove pydash dependency for gemini

* Add cloud-gemini doc

* Remove temp file

* cache import update

* Add test case for summary with mm input

* Lint: warnings instead of print

* Add test cases for gemini

* Gemini test config

* Disable default model for gemini

* Typo fix in gemini workflow

* Correct grammar in example notebook

* Raise if "model" is not provided in create(...)

* Move TODOs into a roadmap

* Update .github/workflows/contrib-tests.yml

Co-authored-by: Davor Runje <davor@airt.ai>

* Gemini test config update

* Update setup.py

Co-authored-by: Davor Runje <davor@airt.ai>

* Update test/oai/test_gemini.py

Co-authored-by: Davor Runje <davor@airt.ai>

* Update test/oai/test_gemini.py

Co-authored-by: Davor Runje <davor@airt.ai>

* Remove python 3.8 from gemini
No google's generativeai for Windows with Python 3.8

* Update import error handling for gemini

* Count tokens and cost for gemini

---------

Co-authored-by: Li Jiang <bnujli@gmail.com>
Co-authored-by: Davor Runje <davor@airt.ai>
2024-04-17 00:24:07 +00:00
Shaokun Zhang
0c5aed0489 add AgentOptimizer test in CI (#2380)
* add test

* fix test

* fix

---------

Co-authored-by: AnonymousRepoSub <“shaokunzhang529@outlook.com” >
2024-04-16 05:07:58 +00:00
Chi Wang
fbe8476726 fix local test failures (#2386)
* fix local test failures

* set skip
2024-04-15 12:34:26 +00:00
Li Jiang
adb8fd849c Add a vectordb module (#2263)
* Added vectordb base and chromadb

* Remove timer and unused functions

* Added filter by distance

* Added test utils

* Fix format

* Fix type hint of dict

* Rename test

* Add test chromadb

* Fix test no chromadb

* Add coverage

* Don't skip test vectordb utils

* Add types

* Fix tests

* Fix docs build error

* Add types to base

* Update base

* Update utils

* Update chromadb

* Add get_docs_by_ids

* Improve docstring

* Add get all docs

* Move chroma_results_to_query_results to utils

* Improve type hints

* Update logger

* Update init, add embedding func

* Improve docstring of vectordb, add two attributes

* Improve test workflow
2024-04-10 17:43:08 +00:00
Li Jiang
0d99d45b0f Fix FileNotFoundError english-words.txt (#2284) 2024-04-05 04:59:57 +00:00
Eric Zhu
d4aa27a21b Fix LMM test in CI by installing lfs (#2282) 2024-04-05 02:39:00 +00:00
Li Jiang
eec854279a Fix unstructured deps installation error (#2248) 2024-04-02 05:17:13 +00:00
Wael Karkoub
8cb5c14c82 Transform Messages Capability (#1923)
* wip

* Adds docstrings

* fixed spellings

* wip

* fixed errors

* better class names

* adds tests

* added tests to workflow

* improved token counting

* improved notebook

* improved token counting in test

* improved docstrings

* fix inconsistencies

* changed by mistake

* fixed docstring

* fixed details

* improves tests + adds openai contrib test

* fix spelling oai contrib test

* clearer docstrings

* remove repeated docstr

* improved notebook

* adds metadata to notebook

* Improve outline and description (#2125)

* better dir structure

* clip max tokens to allowed tokens

* more accurate comments/docstrs

* add deperecation warning

* fix front matter desc

* add deperecation warning notebook

* undo local notebook settings changes

* format notebook

* format workflow

---------

Co-authored-by: gagb <gagb@users.noreply.github.com>
2024-03-28 16:16:17 +00:00
Jack Gerrits
5ef2dfc104 Implement User Defined Functions for Local CLI Executor (#2102)
* Implement user defined functions feature for local cli exec, add docs

* add tests, update docs

* fixes

* fix test

* add pandas test dep

* install test

* provide template as func

* formatting

* undo change

* address comments

* add test deps

* formatting

* test only in 1 env

* formatting

* remove test for local only

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-03-27 23:45:17 +00:00
Davor Runje
78aa0eb220 Introducing IOStream protocol and adding support for websockets (#1551)
* Introducing IOStream

* bug fixing

* polishing

* refactoring

* refactoring

* refactoring

* wip: async tests

* websockets added

* wip

* merge with main

* notebook added

* FastAPI example added

* wip

* merge

* getter/setter to iostream added

* website/blog/2024-03-03-AutoGen-Update/img/dalle_gpt4v.png: convert to Git LFS

* website/blog/2024-03-03-AutoGen-Update/img/gaia.png: convert to Git LFS

* website/blog/2024-03-03-AutoGen-Update/img/teach.png: convert to Git LFS

* add SSL support

* wip

* wip

* exception handling added to on_connect()

* refactoring: default iostream is being set in a context manager

* test fix

* polishing

* polishing

* polishing

* fixed bug with new thread

* polishing

* a bit of refactoring and docs added

* notebook added to docs

* type checking added to CI

* CI fix

* CI fix

* CI fix

* polishing

* obsolete todo comment removed

* fixed precommit error

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-03-26 22:39:55 +00:00
Beibin Li
2f109f5f94 Add vision capability (#2025)
* Add vision capability

* Configurate: description_prompt

* Print warning instead of raising issues for type

* Skip vision capability test if dependencies not installed

* Append "vision" to agent's system message when enabled VisionCapability

* GPT-4V notebook update with ConversableAgent

* Clean GPT-4V notebook

* Add vision capability test to workflow

* Lint import

* Update system message for vision capability

* Add a `custom_caption_func` to VisionCapability

* Add custom function example for vision capability

* Skip test Vision capability custom func

* GPT-4V notebook metadata to website

* Remove redundant files

* The custom caption function takes more inputs now

* Add a more complex example of custom caption func

* Remove trailing space

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-03-24 19:46:55 +00:00
Jack Gerrits
2ad016d18d Print slow tests in CI (#2082)
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-03-20 22:27:35 +00:00
Davor Runje
4af0440730 Update GitHub actions (#2078)
* Add mypy check in pre-commit

* Add mypy check in pre-commit

* bug fix

* CI fix

* add python version matrix for checking types in CI

* update github actions

* revert some accidental changes

* revert some accidental changes

* revert some accidental changes

* revert some accidental changes

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-03-20 17:18:11 +00:00
Eric Zhu
5b5727172c Pin databind package versions in gh-release CI (#2094)
* Pin databind version

* Pin databind version in gh-release CI
2024-03-20 06:01:18 +00:00
Davor Runje
99e48dc024 Add mypy to pre-commit (#2068)
* Add mypy check in pre-commit

* Add mypy check in pre-commit

* bug fix

* CI fix

* add python version matrix for checking types in CI

* added 3.12 to CI

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-03-20 02:01:16 +00:00
Eric Zhu
6cab643906 Pin databind version (#2091) 2024-03-20 01:16:10 +00:00
Jack Gerrits
38b64b6ade Fix type issues in openai_utils.py (#2062)
* Fix type issues in openai_utils.py

* fix incorrect impl

* address comment

* add to CI
2024-03-19 19:35:19 +00:00
Jack Gerrits
9d33dc6f9a Resolve types issues in coding (#2055)
* Improve conversable agent interface, fix some typing

* formatting

* import

* rearrange to fix circular import

* type fixes

* undo system message change

* add coding to check, lint

* Update conversable_agent.py

* Apply suggestions from code review

Co-authored-by: Davor Runje <davor@airt.ai>

* put back # type: ignore

* formatting

* revert conversable agent

* ci fix

* Update type-check.yml

* fix double install, and redudant args, address type ignores

---------

Co-authored-by: Davor Runje <davor@airt.ai>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-03-19 16:49:37 +00:00
Wael Karkoub
c5536ee92b [Feature] Adds Image Generation Capability 2.0 (#1907)
* adds image generation capability

* add todo

* readded cache

* wip

* fix content str bugs

* removed todo: delete imshow

* wip

* fix circular imports

* add notebook

* improve prompt

* improved text analyzer + notebook

* notebook update

* improve notebook

* smaller notebook size

* made changes to the wrong branch :(

* resolve comments + 1

* adds doc strings

* adds cache doc string

* adds doc string to add_to_agent

* adds doc string to ImageGeneration

* instructions are not configurable

* removed unnecessary imports

* changed doc string location

* more doc strings

* improves testability

* adds tests

* adds cache test

* added test to github workflow

* compatible llm config format

* configurable reply function position

* skip_openai + better comments

* fix test

* fix test?

* please fix test?

* last fix test?

* remove type hint

* skip cache test

* adds mock api key

* dalle-2 test

* fix dalle config

* use apu key function

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-03-15 22:11:53 +00:00
Jack Gerrits
24418bd5d2 Fix type issues in exception_utils.py (#1972)
* Fix type issues in exception_utils.py

* fix yaml
2024-03-13 20:24:21 +00:00
Xiaoyun Zhang
ff1529c976 add release pipeline (#1989) 2024-03-13 18:14:19 +00:00
olgavrou
ce71d85e77 Ability to fine tune custom model on conversable agents (#1787)
* uAbility to update_model on conversable agents

* formatting

* formatting

* move code from conversable agent into samples/tools and add testing and README

* forgot install step

* fix

* leave core lib unchanged and move everything to samples/tools

* remove skip openai

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-03-11 21:26:53 +00:00
Jack Gerrits
0e1a4b9fdd Add initial type check CI (#1882)
* Add type check CI

* newline

* dont install editable

* add --non-interactive

* Update type-check.yml
2024-03-09 03:07:51 +00:00
Jack Gerrits
d711bd8e5d Update deploy-website.yml (#1894) 2024-03-07 00:17:59 +00:00
Jack Gerrits
de757707bc Upgrade quarto and include mitigation for title issue, fix links (#1851)
* Upgrade quarto and include mitigation for title issue, fix links

* repo_root

* dont run quarto independently

* test paths

* test

* resolve intermediate parents

* remove print
2024-03-04 14:44:41 +00:00
Chi Wang
cd3b5c6bbd add doc about effects for capabilities (#1842)
* add doc about effects for capabilities

* remove unnecessary imports

* improve doc

* test

* test location

* polish

* improve import of colored

* termcolor

* termcolor

* use pull request for openai test
2024-03-03 23:15:52 +00:00
Jack Gerrits
2912f554a3 revert quarto upgrade (#1840)
* revert quarto upgrade

* fix ruff
2024-03-02 19:45:28 +00:00
Jack Gerrits
d60464374d Upgrade Quarto and use notebook metadata for frontmatter (#1836)
* Update process_notebook to use metadata instead of a yaml comment

* upgrade quarto and version check in tool

* formatting

* address comments
2024-03-02 14:27:11 +00:00
Chi Wang
0a49f2a256 bug fix and cleanup for 0.2.16 (#1827) 2024-03-01 04:31:49 +00:00
Jack Gerrits
f6c9b13ac4 Extend process_notebooks for testing (#1789)
* Extend process_notebooks for testing

* add command

* spelling and lint

* update docs

* Update contributing.md

* add shebang

* Update contributing.md

* lint
2024-02-29 20:47:30 +00:00
Jack Gerrits
76ef0789c0 Implement docker based jupyter executor (#1794)
* Implement docker based jupyter client

* formatting

* skip docker tests when asked

* feedback

* add log

* update build

* formatting

* structural changes

* update setup.py

* update tests

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-02-29 14:54:11 +00:00
Joshua Kim
d797d267e4 Blog regarding finite-state machine (#1622)
* Initialization of fsm blog

* Pre-commit pass

* Pinyin name correction

* Updated title

* Updated title

* Added summary of notebook and note about DAG

* Update index.mdx

* Update website/blog/2024-02-11-FSM-GroupChat/index.mdx

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/blog/2024-02-11-FSM-GroupChat/index.mdx

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Removed outdated filename and followed SoM notebook format

* Pre-commit pass

* Update notebook link

* Corrected link to rendered notebook

* Removed agentchat_hierarchy_flow_using_select_speaker and modified agentchat_groupchat_finite_state_machine

* Add notebook/agentchat_groupchat_finite_state_machine.ipynb to paths

---------

Co-authored-by: freedeaths <register917@gmail.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-02-26 18:10:00 +00:00
Jack Gerrits
fbc2f6e911 Use jupyer-kernel-gateway for ipython executor (#1748)
* checkpoint async based

* Implement jupyter client and use jupyer gateway

* update deps

* address comments

* add missing parenthesis

* Update build.yml

* CI fixes

* change requirement name

* debug

* print stderr

* dont seek

* show token

* mitigaton for windows bug

* use hex token to avoid - in token

* formatting

* put back in place original while the windows bug exists

* lint

* Update autogen/coding/jupyter_code_executor.py

* Update jupyter_code_executor.py

* Update test_embedded_ipython_code_executor.py

* Update setup.py

* Update build.yml

* fix nameerror

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-02-23 19:09:59 +00:00
Xiaoyun Zhang
b05b148cec update dotnet workflow (#1669)
* cherry-pick change in dotnet workflow

* delete openai test workflow as it's part of dotnet-build already
2024-02-13 21:28:51 +00:00
Eric Zhu
609ba7c649 Code executors (#1405)
* code executor

* test

* revert to main conversable agent

* prepare for pr

* kernel

* run open ai tests only when it's out of draft status

* update workflow file

* revert workflow changes

* ipython executor

* check kernel installed; fix tests

* fix tests

* fix tests

* update system prompt

* Update notebook, more tests

* notebook

* raise instead of return None

* allow user provided code executor.

* fixing types

* wip

* refactoring

* polishing

* fixed failing tests

* resolved merge conflict

* fixing failing test

* wip

* local command line executor and embedded ipython executor

* revert notebook

* fix format

* fix merged error

* fix lmm test

* fix lmm test

* move warning

* name and description should be part of the agent protocol, reset is not as it is only used for ConversableAgent; removing accidentally commited file

* version for dependency

* Update autogen/agentchat/conversable_agent.py

Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>

* ordering of protocol

* description

* fix tests

* make ipython executor dependency optional

* update document optional dependencies

* Remove exclude from Agent protocol

* Make ConversableAgent consistent with Agent

* fix tests

* add doc string

* add doc string

* fix notebook

* fix interface

* merge and update agents

* disable config usage in reply function

* description field setter

* customize system message update

* update doc

---------

Co-authored-by: Davor Runje <davor@airt.ai>
Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>
Co-authored-by: Aaron <aaronlaptop12@hotmail.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-02-10 04:52:16 +00:00
gagb
47d6c7567e Proxy PR for Long Context Capability 1513 (#1591)
* Add new capability to handle long context

* Make print conditional

* Remove superfluous comment

* Fix msg order

* Allow user to specify max_tokens

* Add ability to specify max_tokens per message; improve name

* Improve doc and readability

* Add tests

* Improve documentation and add tests per Erik and Chi's feedback

* Update notebook

* Update doc string of add to agents

* Improve doc string

* improve notebook

* Update github workflows for context handling

* Update docstring

* update notebook to use raw config list.

* Update contrib-openai.yml remove _target

* Fix code formatting

* Fix workflow file

* Update .github/workflows/contrib-openai.yml

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-02-08 18:26:00 +00:00
Eric Zhu
1abbcf3c44 Update tests for code_utils and use ThreadPoolExecutor for code execution in windows (#1472)
* threadpoolexecutor

* update tests

* update tests

* fix tests on windows

* fix test

* update tests

* update tests

* update tests

* update tests

* update tests

* update tests

* fix build.yml

* build yaml

* test

* use skip-docker to explicitly skipping docker tests

* update tests

* contribution doc update

* Update website/docs/Contribute.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* update doc

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-02-04 15:08:14 +00:00
Jack Gerrits
38d03b0e63 Add notebooks section on website (#1495)
* Initial infrasctructure for notebooks page

* migrate two notebooks

* add readme notification for notebook dir

* override 'text' prism language to add basic syntactical structure to autogens output

* Rework to retain existing directory and not expose front matter to consumers of the notebook

* improve error handling of process notebooks

* format, ruff and type fixes

* undo changes to navbar

* update readme, CI

* whitespace

* spelling mistakes

* spelling

* Add contributing guide for notebooks

* update notebook

* formatting
2024-02-03 17:01:00 +00:00
Jack Gerrits
276d5b7d31 docs: initial Jupyter support for website docs, move config notebook (#1448)
* docs: jupyter support for website docs, move config docs

* update devcontainer dockerfile, fix CI issues

* update bsed on working directory

* update TLDR

* install into temp

* remove unrelated change from diff

* fix spelling issue

* Update sidebars.js
2024-01-30 22:54:44 +00:00
Chi Wang
fd0094ce51 bump version to 0.2.9 (#1430)
* Fixed rounding bug.

* bump version to 0.2.9

* fix format error

* simplify contrib tests

---------

Co-authored-by: Adam Fourney <adamfo@microsoft.com>
2024-01-27 18:49:24 +00:00
Eric Zhu
6bf62572ee Fix permission block for deploy-website.yml (#1397)
Based on the instruction here: https://github.com/peaceiris/actions-gh-pages?tab=readme-ov-file#%EF%B8%8F-first-deployment-with-github_token

Write permission is needed to publish GitHub page content to the gh-pages branch.
2024-01-25 16:54:21 +00:00
Olaoluwa Ademola Salami
bcfd770a34 test permissions (#1264)
* test permissions

* test least permissions

* remove contents permissions

* testing permisions

* Update deploy-website.yml permission block

---------

Co-authored-by: Davor Runje <davor@airt.ai>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-01-24 21:50:37 +00:00
Chi Wang
05923a7301 bump version to 0.2.8 (#1349)
* bump version to 0.2.8

* make import easier

* redis
2024-01-22 14:17:59 +00:00
afourney
708eb4d884 Add a web surfer agent that can search and browse the web. (#1093)
* Initial commit of WebSurfer. Adds the browser_utils, and related tests. WebSurfer will be added in a subsequent commit.

* Added the web surfer agent, and related tests.

* Added a notebook to show how WebSurferAgent works.

* Fixed a typo.

* Updated test_web_surfer for compatibility with #1110.

* Updated skip_oai logic.

* Fixed code formatting.

* More pre-commit fixes.

* Added block to contrib-openai.yml

* Added block to contrib-openai.yml

* Added hook for BING_API_KEY

* Temporarily commented out other tests, per request.

* Fixed indentation (maybe?)

* Restoring contrib-openai.yml
2024-01-22 03:43:15 +00:00
Vijay Ramesh
ee6ad8d519 [Core] implement redis cache mode (#1222)
* implement redis cache mode, if redis_url is set in the llm_config then
it will try to use this.  also adds a test to validate both the existing
and the redis cache behavior.

* PR feedback, add unit tests

* more PR feedback, move the new style cache to a context manager

* Update agent_chat.md

* more PR feedback, remove tests from contrib and have them run with the normal jobs

* doc

* updated

* Update website/docs/Use-Cases/agent_chat.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* update docs

* update docs; let openaiwrapper to use cache object

* typo

* Update website/docs/Use-Cases/enhanced_inference.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* save previous client cache and reset it after send/a_send

* a_run_chat

---------

Co-authored-by: Vijay Ramesh <vijay@regrello.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-01-20 17:06:29 +00:00
olgavrou
a911d1c2ec set use_docker to default to True (#1147)
* set use_docker to default to true

* black formatting

* centralize checking and add env variable option

* set docker env flag for contrib tests

* set docker env flag for contrib tests

* better error message and cleanup

* disable explicit docker tests

* docker is installed so can't check for that in test

* pr comments and fix test

* rename and fix function descriptions

* documentation

* update notebooks so that they can be run with change in default

* add unit tests for new code

* cache and restore env var

* skip on windows because docker is running in the CI but there are problems connecting the volume

* update documentation

* move header

* update contrib tests
2024-01-18 17:03:49 +00:00
Eric Zhu
22e36cbb10 Support for Python 3.12 (#1317)
* support for Python 3.12

* add python 3.12 to workflows

* version string fix

* retrieval chat

* teachability

* workflow

* redistribute notebook tests

* fix incorrect notebook introduction

* update banner; remove unused imports; update openai workflow

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-01-18 07:15:06 +00:00
Chi Wang
f9c16d26a3 Fix bug in async group chat with func/tool call (#1243)
* test func/tool call in group chat w/ >2 agents

* filter_dict -> exclude

* modify comment

* add last message to groupchat

* typecheck allow_repeat_speaker
2024-01-14 18:24:05 +00:00