mirror of
https://github.com/googleapis/genai-toolbox.git
synced 2026-05-02 03:00:36 -04:00
396 lines
13 KiB
Markdown
396 lines
13 KiB
Markdown
---
|
|
title: "Quickstart"
|
|
type: docs
|
|
weight: 2
|
|
description: >
|
|
How to get started running Toolbox locally with Python, PostgreSQL, and
|
|
LangGraph or LlamaIndex.
|
|
---
|
|
|
|
[](https://colab.research.google.com/github/googleapis/genai-toolbox/blob/main/docs/en/getting-started/colab_quickstart.ipynb)
|
|
|
|
## Before you begin
|
|
|
|
This guide assumes you have already done the following:
|
|
|
|
1. Installed [Python 3.9+][install-python] (including [pip][install-pip] and
|
|
your preferred virtual environment tool for managing dependencies e.g. [venv][install-venv])
|
|
1. Installed [PostgreSQL 16+ and the `psql` client][install-postgres]
|
|
1. Completed setup for usage with an LLM model such as
|
|
{{< tabpane text=true persist=header >}}
|
|
{{% tab header="LangChain" lang="en" %}}
|
|
- [langchain-vertexai](https://python.langchain.com/docs/integrations/llms/google_vertex_ai_palm/#setup) package.
|
|
|
|
- [langchain-google-genai](https://python.langchain.com/docs/integrations/chat/google_generative_ai/#setup) package.
|
|
|
|
- [langchain-anthropic](https://python.langchain.com/docs/integrations/chat/anthropic/#setup) package.
|
|
{{% /tab %}}
|
|
{{% tab header="LlamaIndex" lang="en" %}}
|
|
- [llama-index-llms-google-genai](https://pypi.org/project/llama-index-llms-google-genai/) package.
|
|
|
|
- [llama-index-llms-anthropic](https://docs.llamaindex.ai/en/stable/examples/llm/anthropic) package.
|
|
{{% /tab %}}
|
|
{{< /tabpane >}}
|
|
|
|
[install-python]: https://wiki.python.org/moin/BeginnersGuide/Download
|
|
[install-pip]: https://pip.pypa.io/en/stable/installation/
|
|
[install-venv]: https://packaging.python.org/en/latest/tutorials/installing-packages/#creating-virtual-environments
|
|
[install-postgres]: https://www.postgresql.org/download/
|
|
|
|
## Step 1: Set up your database
|
|
|
|
In this section, we will create a database, insert some data that needs to be
|
|
access by our agent, and create a database user for Toolbox to connect with.
|
|
|
|
1. Connect to postgres using the `psql` command:
|
|
|
|
```bash
|
|
psql -h 127.0.0.1 -U postgres
|
|
```
|
|
|
|
Here, `postgres` denotes the default postgres superuser.
|
|
|
|
1. Create a new database and a new user:
|
|
|
|
{{< notice tip >}}
|
|
For a real application, it's best to follow the principle of least permission
|
|
and only grant the privileges your application needs.
|
|
{{< /notice >}}
|
|
|
|
```sql
|
|
CREATE USER toolbox_user WITH PASSWORD 'my-password';
|
|
|
|
CREATE DATABASE toolbox_db;
|
|
GRANT ALL PRIVILEGES ON DATABASE toolbox_db TO toolbox_user;
|
|
|
|
ALTER DATABASE toolbox_db OWNER TO toolbox_user;
|
|
```
|
|
|
|
|
|
|
|
1. End the database session:
|
|
|
|
```bash
|
|
\q
|
|
```
|
|
|
|
1. Connect to your database with your new user:
|
|
|
|
```bash
|
|
psql -h 127.0.0.1 -U toolbox_user -d toolbox_db
|
|
```
|
|
|
|
1. Create a table using the following command:
|
|
|
|
```sql
|
|
CREATE TABLE hotels(
|
|
id INTEGER NOT NULL PRIMARY KEY,
|
|
name VARCHAR NOT NULL,
|
|
location VARCHAR NOT NULL,
|
|
price_tier VARCHAR NOT NULL,
|
|
checkin_date DATE NOT NULL,
|
|
checkout_date DATE NOT NULL,
|
|
booked BIT NOT NULL
|
|
);
|
|
```
|
|
|
|
1. Insert data into the table.
|
|
|
|
```sql
|
|
INSERT INTO hotels(id, name, location, price_tier, checkin_date, checkout_date, booked)
|
|
VALUES
|
|
(1, 'Hilton Basel', 'Basel', 'Luxury', '2024-04-22', '2024-04-20', B'0'),
|
|
(2, 'Marriott Zurich', 'Zurich', 'Upscale', '2024-04-14', '2024-04-21', B'0'),
|
|
(3, 'Hyatt Regency Basel', 'Basel', 'Upper Upscale', '2024-04-02', '2024-04-20', B'0'),
|
|
(4, 'Radisson Blu Lucerne', 'Lucerne', 'Midscale', '2024-04-24', '2024-04-05', B'0'),
|
|
(5, 'Best Western Bern', 'Bern', 'Upper Midscale', '2024-04-23', '2024-04-01', B'0'),
|
|
(6, 'InterContinental Geneva', 'Geneva', 'Luxury', '2024-04-23', '2024-04-28', B'0'),
|
|
(7, 'Sheraton Zurich', 'Zurich', 'Upper Upscale', '2024-04-27', '2024-04-02', B'0'),
|
|
(8, 'Holiday Inn Basel', 'Basel', 'Upper Midscale', '2024-04-24', '2024-04-09', B'0'),
|
|
(9, 'Courtyard Zurich', 'Zurich', 'Upscale', '2024-04-03', '2024-04-13', B'0'),
|
|
(10, 'Comfort Inn Bern', 'Bern', 'Midscale', '2024-04-04', '2024-04-16', B'0');
|
|
```
|
|
|
|
1. End the database session:
|
|
|
|
```bash
|
|
\q
|
|
```
|
|
|
|
|
|
## Step 2: Install and configure Toolbox
|
|
|
|
In this section, we will download Toolbox, configure our tools in a
|
|
`tools.yaml`, and then run the Toolbox server.
|
|
|
|
1. Download the latest version of Toolbox as a binary:
|
|
|
|
{{< notice tip >}}
|
|
Select the
|
|
[correct binary](https://github.com/googleapis/genai-toolbox/releases)
|
|
corresponding to your OS and CPU architecture.
|
|
{{< /notice >}}
|
|
<!-- {x-release-please-start-version} -->
|
|
```bash
|
|
export OS="linux/amd64" # one of linux/amd64, darwin/arm64, darwin/amd64, or windows/amd64
|
|
curl -O https://storage.googleapis.com/genai-toolbox/v0.2.1/$OS/toolbox
|
|
```
|
|
<!-- {x-release-please-end} -->
|
|
|
|
1. Make the binary executable:
|
|
|
|
```bash
|
|
chmod +x toolbox
|
|
```
|
|
|
|
1. Write the following into a `tools.yaml` file. Be sure to update any fields
|
|
such as `user`, `password`, or `database` that you may have customized in the
|
|
previous step.
|
|
|
|
```yaml
|
|
sources:
|
|
my-pg-source:
|
|
kind: postgres
|
|
host: 127.0.0.1
|
|
port: 5432
|
|
database: toolbox_db
|
|
user: toolbox_user
|
|
password: my-password
|
|
tools:
|
|
search-hotels-by-name:
|
|
kind: postgres-sql
|
|
source: my-pg-source
|
|
description: Search for hotels based on name.
|
|
parameters:
|
|
- name: name
|
|
type: string
|
|
description: The name of the hotel.
|
|
statement: SELECT * FROM hotels WHERE name ILIKE '%' || $1 || '%';
|
|
search-hotels-by-location:
|
|
kind: postgres-sql
|
|
source: my-pg-source
|
|
description: Search for hotels based on location.
|
|
parameters:
|
|
- name: location
|
|
type: string
|
|
description: The location of the hotel.
|
|
statement: SELECT * FROM hotels WHERE location ILIKE '%' || $1 || '%';
|
|
book-hotel:
|
|
kind: postgres-sql
|
|
source: my-pg-source
|
|
description: >-
|
|
Book a hotel by its ID. If the hotel is successfully booked, returns a NULL, raises an error if not.
|
|
parameters:
|
|
- name: hotel_id
|
|
type: string
|
|
description: The ID of the hotel to book.
|
|
statement: UPDATE hotels SET booked = B'1' WHERE id = $1;
|
|
update-hotel:
|
|
kind: postgres-sql
|
|
source: my-pg-source
|
|
description: >-
|
|
Update a hotel's check-in and check-out dates by its ID. Returns a message
|
|
indicating whether the hotel was successfully updated or not.
|
|
parameters:
|
|
- name: hotel_id
|
|
type: string
|
|
description: The ID of the hotel to update.
|
|
- name: checkin_date
|
|
type: string
|
|
description: The new check-in date of the hotel.
|
|
- name: checkout_date
|
|
type: string
|
|
description: The new check-out date of the hotel.
|
|
statement: >-
|
|
UPDATE hotels SET checkin_date = CAST($2 as date), checkout_date = CAST($3
|
|
as date) WHERE id = $1;
|
|
cancel-hotel:
|
|
kind: postgres-sql
|
|
source: my-pg-source
|
|
description: Cancel a hotel by its ID.
|
|
parameters:
|
|
- name: hotel_id
|
|
type: string
|
|
description: The ID of the hotel to cancel.
|
|
statement: UPDATE hotels SET booked = B'0' WHERE id = $1;
|
|
```
|
|
For more info on tools, check out the `Resources` section of the docs.
|
|
|
|
1. Run the Toolbox server, pointing to the `tools.yaml` file created earlier:
|
|
|
|
```bash
|
|
./toolbox --tools_file "tools.yaml"
|
|
```
|
|
|
|
## Step 3: Connect your agent to Toolbox
|
|
|
|
In this section, we will write and run an agent that will load the Tools
|
|
from Toolbox.
|
|
|
|
{{< notice tip>}} If you prefer to experiment within a Google Colab environment,
|
|
you can connect to a
|
|
[local runtime](https://research.google.com/colaboratory/local-runtimes.html).
|
|
{{< /notice >}}
|
|
|
|
|
|
1. In a new terminal, install the SDK package.
|
|
|
|
{{< tabpane persist=header >}}
|
|
{{< tab header="Langchain" lang="bash" >}}
|
|
|
|
pip install toolbox-langchain
|
|
{{< /tab >}}
|
|
{{< tab header="LlamaIndex" lang="bash" >}}
|
|
|
|
pip install toolbox-llamaindex
|
|
{{< /tab >}}
|
|
{{< /tabpane >}}
|
|
|
|
1. Install other required dependencies:
|
|
|
|
{{< tabpane persist=header >}}
|
|
{{< tab header="Langchain" lang="bash" >}}
|
|
|
|
# TODO(developer): replace with correct package if needed
|
|
pip install langgraph langchain-google-vertexai
|
|
# pip install langchain-google-genai
|
|
# pip install langchain-anthropic
|
|
{{< /tab >}}
|
|
{{< tab header="LlamaIndex" lang="bash" >}}
|
|
|
|
# TODO(developer): replace with correct package if needed
|
|
pip install llama-index-llms-google-genai
|
|
# pip install llama-index-llms-anthropic
|
|
{{< /tab >}}
|
|
{{< /tabpane >}}
|
|
|
|
1. Create a new file named `hotel_agent.py` and copy the following
|
|
code to create an agent:
|
|
{{< tabpane persist=header >}}
|
|
{{< tab header="LangChain" lang="python" >}}
|
|
|
|
from langgraph.prebuilt import create_react_agent
|
|
# TODO(developer): replace this with another import if needed
|
|
from langchain_google_vertexai import ChatVertexAI
|
|
# from langchain_google_genai import ChatGoogleGenerativeAI
|
|
# from langchain_anthropic import ChatAnthropic
|
|
from langgraph.checkpoint.memory import MemorySaver
|
|
|
|
from toolbox_langchain import ToolboxClient
|
|
|
|
prompt = """
|
|
You're a helpful hotel assistant. You handle hotel searching, booking and
|
|
cancellations. When the user searches for a hotel, mention it's name, id,
|
|
location and price tier. Always mention hotel ids while performing any
|
|
searches. This is very important for any operations. For any bookings or
|
|
cancellations, please provide the appropriate confirmation. Be sure to
|
|
update checkin or checkout dates if mentioned by the user.
|
|
Don't ask for confirmations from the user.
|
|
"""
|
|
|
|
queries = [
|
|
"Find hotels in Basel with Basel in it's name.",
|
|
"Can you book the Hilton Basel for me?",
|
|
"Oh wait, this is too expensive. Please cancel it and book the Hyatt Regency instead.",
|
|
"My check in dates would be from April 10, 2024 to April 19, 2024.",
|
|
]
|
|
|
|
def main():
|
|
# TODO(developer): replace this with another model if needed
|
|
model = ChatVertexAI(model_name="gemini-1.5-pro")
|
|
# model = ChatGoogleGenerativeAI(model="gemini-1.5-pro")
|
|
# model = ChatAnthropic(model="claude-3-5-sonnet-20240620")
|
|
|
|
# Load the tools from the Toolbox server
|
|
client = ToolboxClient("http://127.0.0.1:5000")
|
|
tools = client.load_toolset()
|
|
|
|
agent = create_react_agent(model, tools, checkpointer=MemorySaver())
|
|
|
|
config = {"configurable": {"thread_id": "thread-1"}}
|
|
for query in queries:
|
|
inputs = {"messages": [("user", prompt + query)]}
|
|
response = agent.invoke(inputs, stream_mode="values", config=config)
|
|
print(response["messages"][-1].content)
|
|
|
|
main()
|
|
{{< /tab >}}
|
|
{{< tab header="LlamaIndex" lang="python" >}}
|
|
import asyncio
|
|
import os
|
|
|
|
from llama_index.core.agent.workflow import AgentWorkflow
|
|
|
|
from llama_index.core.workflow import Context
|
|
|
|
# TODO(developer): replace this with another import if needed
|
|
from llama_index.llms.google_genai import GoogleGenAI
|
|
# from llama_index.llms.anthropic import Anthropic
|
|
|
|
from toolbox_llamaindex import ToolboxClient
|
|
|
|
prompt = """
|
|
You're a helpful hotel assistant. You handle hotel searching, booking and
|
|
cancellations. When the user searches for a hotel, mention it's name, id,
|
|
location and price tier. Always mention hotel ids while performing any
|
|
searches. This is very important for any operations. For any bookings or
|
|
cancellations, please provide the appropriate confirmation. Be sure to
|
|
update checkin or checkout dates if mentioned by the user.
|
|
Don't ask for confirmations from the user.
|
|
"""
|
|
|
|
queries = [
|
|
"Find hotels in Basel with Basel in it's name.",
|
|
"Can you book the Hilton Basel for me?",
|
|
"Oh wait, this is too expensive. Please cancel it and book the Hyatt Regency instead.",
|
|
"My check in dates would be from April 10, 2024 to April 19, 2024.",
|
|
]
|
|
|
|
async def main():
|
|
# TODO(developer): replace this with another model if needed
|
|
llm = GoogleGenAI(
|
|
model="gemini-1.5-pro",
|
|
vertexai_config={"project": "project-id", "location": "us-central1"},
|
|
)
|
|
# llm = GoogleGenAI(
|
|
# api_key=os.getenv("GOOGLE_API_KEY"),
|
|
# model="gemini-1.5-pro",
|
|
# )
|
|
# llm = Anthropic(
|
|
# model="claude-3-7-sonnet-latest",
|
|
# api_key=os.getenv("ANTHROPIC_API_KEY")
|
|
# )
|
|
|
|
# Load the tools from the Toolbox server
|
|
client = ToolboxClient("http://127.0.0.1:5000")
|
|
tools = client.load_toolset()
|
|
|
|
agent = AgentWorkflow.from_tools_or_functions(
|
|
tools,
|
|
llm=llm,
|
|
system_prompt=prompt,
|
|
)
|
|
ctx = Context(agent)
|
|
for query in queries:
|
|
response = await agent.run(user_msg=query, ctx=ctx)
|
|
print(f"---- {query} ----")
|
|
print(str(response))
|
|
|
|
asyncio.run(main())
|
|
{{< /tab >}}
|
|
{{< /tabpane >}}
|
|
|
|
{{< tabpane text=true persist=header >}}
|
|
{{% tab header="Langchain" lang="en" %}}
|
|
To learn more about Agents in LangChain, check out the [LangGraph Agent documentation.](https://langchain-ai.github.io/langgraph/reference/prebuilt/#langgraph.prebuilt.chat_agent_executor.create_react_agent)
|
|
{{% /tab %}}
|
|
{{% tab header="LlamaIndex" lang="en" %}}
|
|
To learn more about Agents in LlamaIndex, check out the [LlamaIndex AgentWorkflow documentation.](https://docs.llamaindex.ai/en/stable/examples/agent/agent_workflow_basic/)
|
|
{{% /tab %}}
|
|
{{< /tabpane >}}
|
|
1. Run your agent, and observe the results:
|
|
|
|
```sh
|
|
python hotel_agent.py
|
|
```
|