mirror of
https://github.com/microsoft/autogen.git
synced 2026-04-20 03:02:16 -04:00
85ad929f34
* add agentops req * track conversable agents with agentops * track tool usage * track message sending * remove record from parent * remove record * simple example * notebook example * remove spacing change * optional dependency * documentation * remove extra import * optional import * record if agentops * if agentops * wrap function auto name * install agentops before notebook test * documentation fixes * notebook metadata * notebook metadata * pre-commit hook changes * doc link fixes * git lfs * autogen tag * bump agentops version * log tool events * notebook fixes * docs * formatting * Updated ecosystem manual * Update notebook for clarity * cleaned up notebook * updated precommit recommendations * Fixed links to screenshots and examples * removed unused files * changed notebook hyperlink * update docusaurus link path * reverted setup.py * change setup again * undo changes * revert conversable agent * removed file not in branch * Updated notebook to look nicer * change letter * revert setup * revert setup again * change ref link * change reflink * remove optional dependency * removed duplicated section * Addressed clarity commetns from howard * minor updates to wording * formatting and pr fixes * added info markdown cell * better docs * notebook * observability docs * pre-commit fixes * example images in notebook * example images in docs * example images in docs * delete agentops ong * doc updates * docs updates * docs updates * use agent as extra_kwarg * add logging tests * pass function properly * create table * dummy function name * log chat completion source name * safe serialize * test fixes * formatting * type checks --------- Co-authored-by: reibs <areibman@gmail.com> Co-authored-by: Chi Wang <wang.chi@microsoft.com> Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com> Co-authored-by: Howard Gil <howardbgil@gmail.com> Co-authored-by: Alex Reibman <meta.alex.r@gmail.com>
2.1 KiB
2.1 KiB
LLM Observability
AutoGen supports advanced LLM observability and monitoring through built-in logging and partner providers.
What is LLM Observability
AI agent observability is the ability to monitor, measure, and understand the internal states and behaviors of AI agent systems. Observability is crucial for ensuring transparency, reliability, and accountability in your agent systems.
Development
Agent Development in Terminal is Limited
- Lose track of what your agents did in between executions
- Parsing through terminal output searching for LLM completions
- Printing “tool called”
Agent Development Dashboards Enable More
- Visual dashboard so you can see what your agents did in human-readable format
- LLM calls are magically recorded - prompt, completion, timestamps for each - with one line of code
- Agents and their events (including tool calls) are recorded with one more line of code
- Errors are magically associated to its causal event
- Record any other events to your session with two more lines of code
- Tons of other useful data if you’re developing with supported agent frameworks: SDK version
Compliance
Observability and monitoring is critical to ensure AI agent systems adhere to laws and regulations in industries like finance and healthcare, preventing violations such as data breaches and privacy issues.
- Insights into AI decision-making, allowing organizations to explain outcomes and build trust with stakeholders.
- Helps detect anomalies and unintended behaviors early, mitigating operational, financial, and reputational risks.
- Ensures compliance with data privacy regulations, preventing unauthorized access and misuse of sensitive information.
- Quick identification and response to compliance violations, supporting incident analysis and prevention.
Available Observability Integrations
Logging
- Autogen SQLite and File Logger - Tutorial
Full-Service Partners
Autogen is currently partnered with AgentOps for seamless observability integration.