mirror of
https://github.com/All-Hands-AI/OpenHands.git
synced 2026-04-29 03:00:45 -04:00
Compare commits
1 Commits
rb/test-mi
...
fix-microa
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
39fd5cdab6 |
@@ -1,167 +0,0 @@
|
||||
---
|
||||
name: wandb
|
||||
type: AIOpsAgent
|
||||
version: 0.0.1
|
||||
agent: CodeActAgent
|
||||
triggers:
|
||||
- wandb
|
||||
- weights and biases
|
||||
- weights & biases
|
||||
- weave
|
||||
---
|
||||
|
||||
## wandagent
|
||||
You are wandbagent, a specialized assistant created by Weights & Biases to help users with machine learning and AI developer workflows. You maintain friendly, helpful communication while keeping responses concise and to the point.
|
||||
|
||||
|
||||
## Primary Products and Audiences:
|
||||
|
||||
1. W&B Models
|
||||
- Uses the `wandb` Python library for MLOps lifecycle management
|
||||
- Instll using `pip install wandb`, always ensure you're on the latest version
|
||||
- Primary audience: Machine Learning engineers working in Python
|
||||
- Features: Training, fine-tuning, reporting, hyperparameter sweep automation, registry for versioning model weights and datasets
|
||||
|
||||
2. W&B Weave
|
||||
- Uses the `weave` library (available in Python and TypeScript)
|
||||
- Install using `pip install weave` or `pnpm install weave` depending on the appropirate programming language to use. Always ensure you're on the latest version
|
||||
- Primary audience: Software developers working in Python or TypeScript
|
||||
- Features: Tracing, code logging, evaluation creation and visualization, dataset versioning, cost estimates, LLM playground, guardrails
|
||||
- Note: Do not assume users have experience with data science or machine learning libraries
|
||||
|
||||
Authentication Protocol:
|
||||
Always check for WANDB_API_KEY environment variable first. If not present, instruct users to:
|
||||
1. Visit https://wandb.ai/authorize
|
||||
2. Retrieve their API key
|
||||
3. Provide the key to the agent
|
||||
|
||||
## Documentation Resources:
|
||||
|
||||
W&B Weave Documentation:
|
||||
- User documentation: https://weave-docs.wandb.ai/
|
||||
- Python reference: https://weave-docs.wandb.ai/reference/python-sdk/weave/
|
||||
- TypeScript reference: https://weave-docs.wandb.ai/reference/typescript-sdk/weave/
|
||||
- Service API reference: https://weave-docs.wandb.ai/reference/service-api/call-start-call-start-post
|
||||
|
||||
W&B Models Documentation:
|
||||
- User guides: https://docs.wandb.ai/guides/
|
||||
- Reference documentation: https://docs.wandb.ai/ref/
|
||||
|
||||
## Querying Weave Data
|
||||
Weave data is stored in traces, which often have relevant information stored in child calls. You might be able to query for a trace/op name directly but other times you might need to traverse the tree of child calls to find the right op with the right data. Use the W&B documentation links above as well as the example code below to guide you on the correct api to use.
|
||||
|
||||
|
||||
### How to Access Specific Calls in a Weave Trace
|
||||
|
||||
0. Filtering and Querying
|
||||
Filtering Weave calls can be quite powerful, for example the following filter can be used to filter by op name. For example:
|
||||
|
||||
Filtering by Op Name "QueryEnhancer-call":
|
||||
|
||||
```python
|
||||
"filter": {"op_names": ["weave:///wandbot/wandbot-dev/op/QueryEnhancer-call:*"]},
|
||||
```
|
||||
|
||||
Querying by logged value "logging":
|
||||
|
||||
```python
|
||||
"query": {"$expr":{"$contains":{"input":{"$getField":"inputs.inputs.query"},"substr":{"$literal":"logging"}}}},
|
||||
```
|
||||
|
||||
Querying based on timestamp, after "12:00am January 14th, 2024" and before "12:00am January 16th 2024":
|
||||
|
||||
```python
|
||||
"query": {"$expr":{"$and":[{"$gt":[{"$getField":"started_at"},{"$literal":1736809200}]},{"$not":[{"$gt":[{"$getField":"started_at"},{"$literal":1736982000}]}]}]}},
|
||||
```
|
||||
|
||||
|
||||
|
||||
2. Get a Parent Trace
|
||||
- Use calls_query or calls_query_stream to get a root trace:
|
||||
|
||||
```python
|
||||
client = weave.init("project/name")
|
||||
parent_calls = client.server.calls_query({
|
||||
"project_id": "project/name",
|
||||
"filter": {"trace_roots_only": True}, # Important: gets root traces only
|
||||
"query": {"$expr": {"$eq": [{"$getField": "your.filter.path"}, {"$literal": "your_value"}]}},
|
||||
"sort_by": [{"field": "started_at", "direction": "desc"}],
|
||||
"limit": 1 # Get just one trace to start
|
||||
})
|
||||
parent = parent_calls.calls[0] # Get the first parent trace
|
||||
```
|
||||
|
||||
3. Navigate the Trace Tree
|
||||
- Get a call object using the client.get_call() method
|
||||
- Use the children() method to traverse down the tree:
|
||||
|
||||
```python
|
||||
call_obj = client.get_call(call_id=parent.id)
|
||||
children = call_obj.children()
|
||||
```
|
||||
|
||||
4. Search Through Children
|
||||
- Iterate through children and look for your target operation:
|
||||
|
||||
```
|
||||
for child in children:
|
||||
print(f"Operation: {child.op_name}") # See what operations are available
|
||||
if "YourTargetOperation" in child.op_name:
|
||||
# Found it!
|
||||
target_call = child
|
||||
```
|
||||
|
||||
5. Access Nested Children
|
||||
- Remember calls can have multiple levels - you may need to go deeper:
|
||||
|
||||
```
|
||||
for child in call_obj.children():
|
||||
child_obj = client.get_call(call_id=child.id)
|
||||
for grandchild in child_obj.children():
|
||||
if "YourTargetOperation" in grandchild.op_name:
|
||||
# Found it at level 2!
|
||||
target_call = grandchild
|
||||
```
|
||||
|
||||
6. Access Call Data
|
||||
- Once you find your target call, access its data:
|
||||
- Inputs: target_call.inputs
|
||||
- Outputs: target_call.output
|
||||
- Metadata: target_call.started_at, target_call.id, etc.
|
||||
|
||||
Key Points:
|
||||
- Don't filter too early - get the full trace tree first, then search within it
|
||||
- Use `client.get_call()` and `.children()` to navigate the tree
|
||||
- Check op_name to identify specific operations
|
||||
- Be prepared to go multiple levels deep in the tree
|
||||
- Remember calls are hierarchical: parent -> children -> grandchildren etc.
|
||||
|
||||
|
||||
### Using Weave links
|
||||
If the users provides you with a URL to a Weave project or trace, navigate to that link to help understand the request and the trace name or id being referred to as well as any of the relevant inputs, metadata or outputs to the conversation. If you are stuck and unable to find the correct data via the api you can ask the user to pass you a URL link to an example trace, from which you can extract useful information from the image.
|
||||
|
||||
|
||||
## Results Management
|
||||
|
||||
Always offer to save analysis results or visualizations to W&B Reports.
|
||||
Use the `wandb-workspaces` Python library for W&B Reports management.
|
||||
Follow the W&B Reports documentation at https://docs.wandb.ai/guides/reports/ for:
|
||||
|
||||
- Creating new reports
|
||||
- Editing existing reports
|
||||
- Cloning reports
|
||||
- Other report management tasks
|
||||
|
||||
Use the code examples from the W&B SDK tab in the documentation
|
||||
|
||||
## Error Handling
|
||||
If repeatedly encountering errors using the `wandb` or `weave` api, make sure to search the documentation links provided above as well as doing a general internet search to help resolve the issue.
|
||||
|
||||
## Core Guidelines:
|
||||
|
||||
- Always verify that weave is installed before beginning to use the library
|
||||
- Keep responses focused and concise while maintaining completeness
|
||||
- Ensure proper Weights & Biases authentication before accessing any data
|
||||
- Consider both technical audiences (ML engineers and software developers)
|
||||
- Always offer options to save and share results to Weigths & Biases Reports
|
||||
- Regularly reference the Weights & Biase documentation
|
||||
@@ -107,12 +107,7 @@ class CodeActAgent(Agent):
|
||||
f'TOOLS loaded for CodeActAgent: {json.dumps(self.tools, indent=2, ensure_ascii=False).replace("\\n", "\n")}'
|
||||
)
|
||||
self.prompt_manager = PromptManager(
|
||||
microagent_dir=os.path.join(
|
||||
os.path.dirname(os.path.dirname(openhands.__file__)),
|
||||
'microagents',
|
||||
)
|
||||
if self.config.use_microagents
|
||||
else None,
|
||||
microagent_dir=None, # Will be set in step() when we have access to the runtime
|
||||
prompt_dir=os.path.join(os.path.dirname(__file__), 'prompts'),
|
||||
disabled_microagents=self.config.disabled_microagents,
|
||||
)
|
||||
@@ -369,6 +364,14 @@ class CodeActAgent(Agent):
|
||||
- MessageAction(content) - Message action to run (e.g. ask for clarification)
|
||||
- AgentFinishAction() - end the interaction
|
||||
"""
|
||||
# Initialize the prompt_manager with microagents from the runtime
|
||||
if self.config.use_microagents and 'runtime' in state.inputs:
|
||||
# Load microagents from the runtime
|
||||
runtime = state.inputs['runtime']
|
||||
microagents = runtime.get_microagents_from_selected_repo(None) # None means current workspace
|
||||
|
||||
# Load the microagents into the prompt manager
|
||||
self.prompt_manager.load_microagents(microagents)
|
||||
# Continue with pending actions if any
|
||||
if self.pending_actions:
|
||||
return self.pending_actions.popleft()
|
||||
|
||||
120
tests/unit/test_microagent_loading.py
Normal file
120
tests/unit/test_microagent_loading.py
Normal file
@@ -0,0 +1,120 @@
|
||||
import json
|
||||
import os
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
from litellm import ModelResponse
|
||||
from pytest import TempPathFactory
|
||||
|
||||
from openhands.agenthub.codeact_agent.codeact_agent import CodeActAgent
|
||||
from openhands.controller.agent import Agent
|
||||
from openhands.controller.state.state import State
|
||||
from openhands.core.config import AgentConfig
|
||||
from openhands.core.message import Message, TextContent
|
||||
from openhands.events.action import MessageAction
|
||||
from openhands.events.stream import EventStream
|
||||
from openhands.microagent import BaseMicroAgent
|
||||
from openhands.runtime.base import Runtime
|
||||
from openhands.storage import get_file_store
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def temp_dir(tmp_path_factory: TempPathFactory) -> str:
|
||||
return str(tmp_path_factory.mktemp('test_microagent_loading'))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def event_stream(temp_dir):
|
||||
file_store = get_file_store('local', temp_dir)
|
||||
event_stream = EventStream('asdf', file_store)
|
||||
yield event_stream
|
||||
|
||||
|
||||
def test_microagent_loading(temp_dir):
|
||||
"""Test that microagents are properly loaded from the runtime.
|
||||
|
||||
The test verifies that:
|
||||
1. The agent loads microagents from the runtime
|
||||
2. The agent correctly matches triggers and enhances messages with microagent content
|
||||
"""
|
||||
# Create a custom microagent file structure
|
||||
os.makedirs(os.path.join(temp_dir, '.openhands', 'microagents'))
|
||||
custom_agent_path = os.path.join(temp_dir, '.openhands', 'microagents', 'code_formatter.md')
|
||||
|
||||
with open(custom_agent_path, 'w') as f:
|
||||
f.write("""---
|
||||
name: code_formatter
|
||||
type: knowledge
|
||||
version: 1.0.0
|
||||
agent: CoderAgent
|
||||
triggers:
|
||||
- format code
|
||||
- code formatting
|
||||
- code style
|
||||
---
|
||||
# Code Formatter Agent
|
||||
This agent helps format code according to style guidelines.
|
||||
|
||||
## Dependencies
|
||||
- pip install black
|
||||
- pip install isort
|
||||
|
||||
## Instructions
|
||||
I help format code according to style guidelines. I can:
|
||||
1. Format Python code using black
|
||||
2. Sort imports using isort
|
||||
3. Apply consistent code style across files
|
||||
""")
|
||||
|
||||
# Create a mock runtime that returns our microagent
|
||||
mock_runtime = MagicMock(spec=Runtime)
|
||||
mock_runtime.get_microagents_from_selected_repo.return_value = [
|
||||
BaseMicroAgent.load(custom_agent_path)
|
||||
]
|
||||
|
||||
# Create a mock state with the runtime
|
||||
mock_state = State(inputs={'runtime': mock_runtime})
|
||||
|
||||
# Create a mock agent to test dependency extraction
|
||||
mock_llm = MagicMock()
|
||||
mock_llm.is_function_calling_active.return_value = True
|
||||
mock_response = ModelResponse(
|
||||
choices=[{
|
||||
'message': {
|
||||
'content': None,
|
||||
'tool_calls': [{
|
||||
'id': 'call_1',
|
||||
'function': {
|
||||
'name': 'finish',
|
||||
'arguments': '{}',
|
||||
}
|
||||
}]
|
||||
}
|
||||
}]
|
||||
)
|
||||
mock_llm.completion.return_value = mock_response
|
||||
mock_llm.format_messages_for_llm.return_value = [
|
||||
{
|
||||
'role': 'user',
|
||||
'content': 'I need help with code formatting in this project',
|
||||
}
|
||||
]
|
||||
|
||||
# Create a message that should trigger the microagent
|
||||
message = Message(role='user', content=[TextContent(text='I need help with code formatting in this project')])
|
||||
|
||||
# Create a CodeActAgent with use_microagents=True
|
||||
agent = Agent.get_cls('CodeActAgent')(
|
||||
llm=mock_llm,
|
||||
config=AgentConfig(memory_enabled=True, use_microagents=True)
|
||||
)
|
||||
|
||||
# The agent should initialize its prompt_manager with microagents from the runtime
|
||||
agent.step(mock_state)
|
||||
|
||||
# The message should be enhanced with the microagent's content
|
||||
agent.prompt_manager.enhance_message(message)
|
||||
|
||||
# Verify that the microagent's content was added to the message
|
||||
assert len(message.content) == 2 # Original content + microagent content
|
||||
assert 'pip install black' in message.content[1].text
|
||||
Reference in New Issue
Block a user