This PR introduces a significant update to the Toolbox configuration
file format, which is one of the primary **breaking changes** required
for the implementation of the Advanced Control Plane.
# Summary of Changes
The configuration schema has been updated to enforce resource isolation
and facilitate atomic, incremental updates.
* Resource Isolation: Resource definitions are now separated into
individual blocks, using a distinct structure for each resource type
(Source, Tool, Toolset, etc.). This improves readability, management,
and auditing of configuration files.
* Field Name Modification: Internal field names have been modified to
align with declarative methodologies. Specifically, the configuration
now separates kind (general resource type, e.g., Source) from type
(specific implementation, e.g., Postgres).
# User Impact
Existing tools.yaml configuration files are now in an outdated format.
Users must eventually update their files to the new YAML format.
# Mitigation & Compatibility
Backward compatibility is maintained during this transition to ensure no
immediate user action is required for existing files.
* Immediate Backward Compatibility: The source code includes a
pre-processing layer that automatically detects outdated configuration
files (v1 format) and converts them to the new v2 format under the hood.
* [COMING SOON] Migration Support: The new toolbox migrate subcommand
will be introduced to allow users to automatically convert their old
configuration files to the latest format.
# Example
Example for config file v2:
```
kind: sources
name: my-pg-instance
type: cloud-sql-postgres
project: my-project
region: my-region
instance: my-instance
database: my_db
user: my_user
password: my_pass
---
kind: authServices
name: my-google-auth
type: google
clientId: testing-id
---
kind: tools
name: example_tool
type: postgres-sql
source: my-pg-instance
description: some description
statement: SELECT * FROM SQL_STATEMENT;
parameters:
- name: country
type: string
description: some description
---
kind: tools
name: example_tool_2
type: postgres-sql
source: my-pg-instance
description: returning the number one
statement: SELECT 1;
---
kind: toolsets
name: example_toolset
tools:
- example_tool
```
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
Add parameter `embeddedBy` field to support vector embedding & semantic
search.
Major change in `internal/util/parameters/parameters.go`
This PR only adds vector formatter for the postgressql tool. Other tools
requiring vector formatting may not work with embeddedBy.
Second part of the Semantic Search support. First part:
https://github.com/googleapis/genai-toolbox/pull/2121
Move source-related queries from `Invoke()` function into Source.
The following sources are updated in this PR:
* spanner
* sqlite
* tidb
* trino
* valkey
* yugabytedb
This is an effort to generalizing tools to work with any Source that
implements a specific interface. This will provide a better segregation
of the roles for Tools vs Source.
Tool's role will be limited to the following:
* Resolve any pre-implementation steps or parameters (e.g. template
parameters)
* Retrieving Source
* Calling the source's implementation
This PR update the linking mechanism between Source and Tool.
Tools are directly linked to their Source, either by pointing to the
Source's functions or by assigning values from the source during Tool's
initialization. However, the existing approach means that any
modification to the Source after Tool's initialization might not be
reflected. To address this limitation, each tool should only store a
name reference to the Source, rather than direct link or assigned
values.
Tools will provide interface for `compatibleSource`. This will be used
to determine if a Source is compatible with the Tool.
```
type compatibleSource interface{
Client() http.Client
ProjectID() string
}
```
During `Invoke()`, the tool will run the following operations:
* retrieve Source from the `resourceManager` with source's named defined
in Tool's config
* validate Source via `compatibleSource interface{}`
* run the remaining `Invoke()` function. Fields that are needed is
retrieved directly from the source.
With this update, resource manager is also added as input to other
Tool's function that require access to source (e.g.
`RequiresClientAuthorization()`).
## Description
Tool `invoke()` and `RequiresClientAuthorization()` takes a new input
argument -- Resource Manager. Resource manager will be used to retrieve
Source in the next step.
In order to achieve the goal, this PR implements the follows:
* move resource manager from the server package to a new package to
prevent import cycles (between server and mcp)
* added a new interface in `tools.go` to prevent import cycle (between
resources and tools package)
* add new input argument in all tools
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change
## Description
The MCP spec supports tool annotations like the below structure in the
2025-06-18 version of the spec.
https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations
```
{
destructiveHint?: boolean;
idempotentHint?: boolean;
openWorldHint?: boolean;
readOnlyHint?: boolean;
}
```
Added a ToolAnnotations structure, an Annotations member to the
McpManifest structure, and a nil initializer for the Annotations member
to all calls to GetMcpManifest.
The ToolAnnotations structure and the member annotations are all defined
as pointers so that they are omited when not set. There are times when
the zero value is meaningful so this was the only way to make sure that
we distinguish between not setting the annotation and setting it with a
zero value.
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change
🛠️Fixes#927
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
## Description
This commit allows a tool to pull an alternate authorization
token from the header of the http request.
This is initially being built for the Looker integration. Looker
uses its own OAuth token. When deploying MCP Toolbox to Cloud
Run, the default token in the "Authorization" header is for
authentication with Cloud Run. An alternate token can be put into
another header by a client such as ADK or any other client that
can programatically set http headers. This token will be used
to authenticate with Looker.
If needed, other sources can use this by setting the header name
in the source config, passing it into the tool config, and returning
the header name in the Tool GetAuthTokenHeaderName() function.
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change
🛠️Fixes#1540
To keep a persistent backend storage for configuration, we will have to
keep a single source of truth. This involves supporting bi-directional
conversion between Config and Tool.
This PR make the following changes:
* Embed Config in Tool
* Add `ToConfig()` to extract Config from Tool.
Jules PR
---
*PR created automatically by Jules for task
[11947649751737965380](https://jules.google.com/task/11947649751737965380)*
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
To facilitate the transition of moving invocation implementation to
Source, we will have to move parameter to `internal/util`. This approach
is crucial because certain parameters may not be fully resolvable
pre-implementation. Since both `internal/sources` and `internal/tools`
will need access to `parameters`, it will be more relevant to move
parameters implementation to utils.
This commit introduces a new `orderedmap` package to preserve the column
order of SQL query results when they are marshaled to JSON.
The default Go `json.Marshal` function sorts map keys, which was causing
the column order to be lost in the output of the database tools.
This commit updates the following tools to use the new `orderedmap`
package:
- `mysqlexecutesql`
- `mssqlexecutesql`
- `postgresexecutesql`
- `spannerexecutesql`
- `sqliteexecutesql`
- `bigqueryexecutesql`
A new test has been added to the `mysqlexecutesql` tool to verify that
the column order is preserved.
## Description
> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [ ] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change
🛠️Fixes#1492
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
## Description
The debug context logger does not take in value placeholders. The
statements must be first converted to a string.
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [ ] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change
🛠️ Fixes #<issue_number_goes_here>
Add `_meta` for `tools/list` method in MCP Toolbox.
If there are authorized invocation, the following will be return in
`_meta`:
```
{
"name":"my-tool-name",
"description":"my tool description",
"inputSchema":{
"type":"object",
"properties":{
"user_id":{"type":"string","description":"user's name from google login"}
},
"required":["user_id"]
},
"_meta":{
"toolbox/authParam":{"user_id":["my_auth"]}
}
}
```
If there are authenticated parameter, the following will be return in
`_meta`:
```
{
"name":"my-tool-name",
"description":"my tool description",
"inputSchema":{
"type":"object",
"properties":{
"sql":{"type":"string","description":"The sql to execute."}
},
"required":["sql"]
},
"_meta":{
"toolbox/authInvoke":["my_auth"]
}
}
```
If there are no authorized invocation or authenticated prameter, the
`_meta` field will be omitted.
With this feature, the following were updated in the source code:
* In each `func(p CommonParameter) McpManifest()`, we will return a
`[]string` for the list of authenticated parameters. This is similar to
how Manifest() return the list of authNames in non-MCP Toolbox's
manifest.
* The `func(ps Parameters) McpManifest()` will return a
`map[string][]string` that with key as param's name, and value as the
param's auth.
* Added a new function `GetMcpManifest()` in `tools.go`. This function
will consctruct the McpManifest, and add the `Metadata` field.
* Associated tests were added or updated.
## Description
Prebuilt tools for the sqlite
- [x] `list_table` with simple and detailed(trigger,index,column) for
each table
- [x] `execute-sql` for executing any SQL statement for sqlite.
- [x] added tests and done required changes in config.
- [x] **Documentation update**:Done
## PR Checklist
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/langchain-google-alloydb-pg-python/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involves a breaking change
🛠️ Fixes: https://github.com/googleapis/genai-toolbox/issues/1226
---------
Co-authored-by: Averi Kitsch <akitsch@google.com>
