mirror of
https://github.com/microsoft/autogen.git
synced 2026-02-08 00:55:23 -05:00
85ad929f34
* add agentops req * track conversable agents with agentops * track tool usage * track message sending * remove record from parent * remove record * simple example * notebook example * remove spacing change * optional dependency * documentation * remove extra import * optional import * record if agentops * if agentops * wrap function auto name * install agentops before notebook test * documentation fixes * notebook metadata * notebook metadata * pre-commit hook changes * doc link fixes * git lfs * autogen tag * bump agentops version * log tool events * notebook fixes * docs * formatting * Updated ecosystem manual * Update notebook for clarity * cleaned up notebook * updated precommit recommendations * Fixed links to screenshots and examples * removed unused files * changed notebook hyperlink * update docusaurus link path * reverted setup.py * change setup again * undo changes * revert conversable agent * removed file not in branch * Updated notebook to look nicer * change letter * revert setup * revert setup again * change ref link * change reflink * remove optional dependency * removed duplicated section * Addressed clarity commetns from howard * minor updates to wording * formatting and pr fixes * added info markdown cell * better docs * notebook * observability docs * pre-commit fixes * example images in notebook * example images in docs * example images in docs * delete agentops ong * doc updates * docs updates * docs updates * use agent as extra_kwarg * add logging tests * pass function properly * create table * dummy function name * log chat completion source name * safe serialize * test fixes * formatting * type checks --------- Co-authored-by: reibs <areibman@gmail.com> Co-authored-by: Chi Wang <wang.chi@microsoft.com> Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com> Co-authored-by: Howard Gil <howardbgil@gmail.com> Co-authored-by: Alex Reibman <meta.alex.r@gmail.com>
3.2 KiB
3.2 KiB
AgentOps 🖇️
AgentOps provides session replays, metrics, and monitoring for agents.
At a high level, AgentOps gives you the ability to monitor LLM calls, costs, latency, agent failures, multi-agent interactions, tool usage, session-wide statistics, and more. For more info, check out the AgentOps Repo.
Agent Dashboard
Session Analytics
Session Replays
Installation
AgentOps works seamlessly with applications built using Autogen.
- Install AgentOps
pip install agentops
-
Create an API Key: Create a user API key here: Create API Key
-
Configure Your Environment: Add your API key to your environment variables
AGENTOPS_API_KEY=<YOUR_AGENTOPS_API_KEY>
- Initialize AgentOps
To start tracking all available data on Autogen runs, simply add two lines of code before implementing Autogen.
import agentops
agentops.init() # Or: agentops.init(api_key="your-api-key-here")
After initializing AgentOps, Autogen will now start automatically tracking your agent runs.
Features
- LLM Costs: Track spend with foundation model providers
- Replay Analytics: Watch step-by-step agent execution graphs
- Recursive Thought Detection: Identify when agents fall into infinite loops
- Custom Reporting: Create custom analytics on agent performance
- Analytics Dashboard: Monitor high level statistics about agents in development and production
- Public Model Testing: Test your agents against benchmarks and leaderboards
- Custom Tests: Run your agents against domain specific tests
- Time Travel Debugging: Save snapshots of session states to rewind and replay agent runs from chosen checkpoints.
- Compliance and Security: Create audit logs and detect potential threats such as profanity and PII leaks
- Prompt Injection Detection: Identify potential code injection and secret leaks