The `ParseParams` Tool interface is only passing the tool's parameter
into a generic `parameters.ParseParams()` helper. Instead of keeping it
as a tool interface, we add a `GetParameters()` method
(https://github.com/googleapis/genai-toolbox/pull/2374) to the tool
interface and call it directly from the API handlers. This way we keep
the parameter parsing logic independent from the tools.
This PR introduces a significant update to the Toolbox configuration
file format, which is one of the primary **breaking changes** required
for the implementation of the Advanced Control Plane.
# Summary of Changes
The configuration schema has been updated to enforce resource isolation
and facilitate atomic, incremental updates.
* Resource Isolation: Resource definitions are now separated into
individual blocks, using a distinct structure for each resource type
(Source, Tool, Toolset, etc.). This improves readability, management,
and auditing of configuration files.
* Field Name Modification: Internal field names have been modified to
align with declarative methodologies. Specifically, the configuration
now separates kind (general resource type, e.g., Source) from type
(specific implementation, e.g., Postgres).
# User Impact
Existing tools.yaml configuration files are now in an outdated format.
Users must eventually update their files to the new YAML format.
# Mitigation & Compatibility
Backward compatibility is maintained during this transition to ensure no
immediate user action is required for existing files.
* Immediate Backward Compatibility: The source code includes a
pre-processing layer that automatically detects outdated configuration
files (v1 format) and converts them to the new v2 format under the hood.
* [COMING SOON] Migration Support: The new toolbox migrate subcommand
will be introduced to allow users to automatically convert their old
configuration files to the latest format.
# Example
Example for config file v2:
```
kind: sources
name: my-pg-instance
type: cloud-sql-postgres
project: my-project
region: my-region
instance: my-instance
database: my_db
user: my_user
password: my_pass
---
kind: authServices
name: my-google-auth
type: google
clientId: testing-id
---
kind: tools
name: example_tool
type: postgres-sql
source: my-pg-instance
description: some description
statement: SELECT * FROM SQL_STATEMENT;
parameters:
- name: country
type: string
description: some description
---
kind: tools
name: example_tool_2
type: postgres-sql
source: my-pg-instance
description: returning the number one
statement: SELECT 1;
---
kind: toolsets
name: example_toolset
tools:
- example_tool
```
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
Add parameter `embeddedBy` field to support vector embedding & semantic
search.
Major change in `internal/util/parameters/parameters.go`
This PR only adds vector formatter for the postgressql tool. Other tools
requiring vector formatting may not work with embeddedBy.
Second part of the Semantic Search support. First part:
https://github.com/googleapis/genai-toolbox/pull/2121
Move source-related queries from `Invoke()` function into Source.
This is an effort to generalizing tools to work with any Source that
implements a specific interface. This will provide a better segregation
of the roles for Tools vs Source.
Tool's role will be limited to the following:
* Resolve any pre-implementation steps or parameters (e.g. template
parameters)
* Retrieving Source
* Calling the source's implementation
This PR update the linking mechanism between Source and Tool.
Tools are directly linked to their Source, either by pointing to the
Source's functions or by assigning values from the source during Tool's
initialization. However, the existing approach means that any
modification to the Source after Tool's initialization might not be
reflected. To address this limitation, each tool should only store a
name reference to the Source, rather than direct link or assigned
values.
Tools will provide interface for `compatibleSource`. This will be used
to determine if a Source is compatible with the Tool.
```
type compatibleSource interface{
Client() http.Client
ProjectID() string
}
```
During `Invoke()`, the tool will run the following operations:
* retrieve Source from the `resourceManager` with source's named defined
in Tool's config
* validate Source via `compatibleSource interface{}`
* run the remaining `Invoke()` function. Fields that are needed is
retrieved directly from the source.
With this update, resource manager is also added as input to other
Tool's function that require access to source (e.g.
`RequiresClientAuthorization()`).
## Description
Tool `invoke()` and `RequiresClientAuthorization()` takes a new input
argument -- Resource Manager. Resource manager will be used to retrieve
Source in the next step.
In order to achieve the goal, this PR implements the follows:
* move resource manager from the server package to a new package to
prevent import cycles (between server and mcp)
* added a new interface in `tools.go` to prevent import cycle (between
resources and tools package)
* add new input argument in all tools
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change
## Description
The MCP spec supports tool annotations like the below structure in the
2025-06-18 version of the spec.
https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations
```
{
destructiveHint?: boolean;
idempotentHint?: boolean;
openWorldHint?: boolean;
readOnlyHint?: boolean;
}
```
Added a ToolAnnotations structure, an Annotations member to the
McpManifest structure, and a nil initializer for the Annotations member
to all calls to GetMcpManifest.
The ToolAnnotations structure and the member annotations are all defined
as pointers so that they are omited when not set. There are times when
the zero value is meaningful so this was the only way to make sure that
we distinguish between not setting the annotation and setting it with a
zero value.
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change
🛠️Fixes#927
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
## Description
This commit allows a tool to pull an alternate authorization
token from the header of the http request.
This is initially being built for the Looker integration. Looker
uses its own OAuth token. When deploying MCP Toolbox to Cloud
Run, the default token in the "Authorization" header is for
authentication with Cloud Run. An alternate token can be put into
another header by a client such as ADK or any other client that
can programatically set http headers. This token will be used
to authenticate with Looker.
If needed, other sources can use this by setting the header name
in the source config, passing it into the tool config, and returning
the header name in the Tool GetAuthTokenHeaderName() function.
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change
🛠️Fixes#1540
To keep a persistent backend storage for configuration, we will have to
keep a single source of truth. This involves supporting bi-directional
conversion between Config and Tool.
This PR make the following changes:
* Embed Config in Tool
* Add `ToConfig()` to extract Config from Tool.
Jules PR
---
*PR created automatically by Jules for task
[11947649751737965380](https://jules.google.com/task/11947649751737965380)*
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
To facilitate the transition of moving invocation implementation to
Source, we will have to move parameter to `internal/util`. This approach
is crucial because certain parameters may not be fully resolvable
pre-implementation. Since both `internal/sources` and `internal/tools`
will need access to `parameters`, it will be more relevant to move
parameters implementation to utils.
This pull request adds support for a new `dry_run` mode to the Neo4j
Cypher execution tool, allowing users to validate queries and view
execution plans without running them. It also sets a custom user agent
for Neo4j connections and improves error handling and documentation. The
most important changes are grouped below.
### New dry run feature for Cypher execution
* Added an optional `dry_run` boolean parameter to the
`neo4j-execute-cypher` tool, allowing users to validate Cypher queries
and receive execution plan details without running the query. The tool
now prepends `EXPLAIN` to the query when `dry_run` is true and returns a
structured summary of the execution plan.
[[1]](diffhunk://#diff-de7fdd7e68c95ea9813c704a89fffb8fd6de34e81b43a484623fdff7683e18f3L87-R93)
[[2]](diffhunk://#diff-de7fdd7e68c95ea9813c704a89fffb8fd6de34e81b43a484623fdff7683e18f3R155-R188)
[[3]](diffhunk://#diff-de7fdd7e68c95ea9813c704a89fffb8fd6de34e81b43a484623fdff7683e18f3R219-R236)
[[4]](diffhunk://#diff-1dca93fc9450e9b9ea64bc1ae02774c3198ea6f8310b2437815bd1a5eae11e79L30-R32)
* Updated integration tests to cover the new `dry_run` functionality,
including successful dry runs, error handling for invalid syntax, and
enforcement of read-only mode.
[[1]](diffhunk://#diff-b07de4a304bc72964b5de9481cbc6aec6cf9bb9dabd903a837eb8974e7100a90R163-R169)
[[2]](diffhunk://#diff-b07de4a304bc72964b5de9481cbc6aec6cf9bb9dabd903a837eb8974e7100a90R250-R291)
### Improved error handling
* Enhanced error messages for parameter casting in the tool's `Invoke`
method to clarify issues with input parameters.
### Neo4j driver configuration
* Set a custom user agent (`genai-toolbox/neo4j-source`) for Neo4j
driver connections to help identify requests from this tool.
[[1]](diffhunk://#diff-3f0444add0913f1722d678118ffedc70039cca3603f31c9927c06be5e00ffb29R24-R29)
[[2]](diffhunk://#diff-3f0444add0913f1722d678118ffedc70039cca3603f31c9927c06be5e00ffb29L109-R113)
### Documentation updates
* Updated the documentation to describe the new `dry_run` parameter and
its usage for query validation.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Add `_meta` for `tools/list` method in MCP Toolbox.
If there are authorized invocation, the following will be return in
`_meta`:
```
{
"name":"my-tool-name",
"description":"my tool description",
"inputSchema":{
"type":"object",
"properties":{
"user_id":{"type":"string","description":"user's name from google login"}
},
"required":["user_id"]
},
"_meta":{
"toolbox/authParam":{"user_id":["my_auth"]}
}
}
```
If there are authenticated parameter, the following will be return in
`_meta`:
```
{
"name":"my-tool-name",
"description":"my tool description",
"inputSchema":{
"type":"object",
"properties":{
"sql":{"type":"string","description":"The sql to execute."}
},
"required":["sql"]
},
"_meta":{
"toolbox/authInvoke":["my_auth"]
}
}
```
If there are no authorized invocation or authenticated prameter, the
`_meta` field will be omitted.
With this feature, the following were updated in the source code:
* In each `func(p CommonParameter) McpManifest()`, we will return a
`[]string` for the list of authenticated parameters. This is similar to
how Manifest() return the list of authNames in non-MCP Toolbox's
manifest.
* The `func(ps Parameters) McpManifest()` will return a
`map[string][]string` that with key as param's name, and value as the
param's auth.
* Added a new function `GetMcpManifest()` in `tools.go`. This function
will consctruct the McpManifest, and add the `Metadata` field.
* Associated tests were added or updated.
This pull request introduces a utility function to standardize the
conversion of Neo4j driver values into JSON-compatible types. The new
`ConvertValue` function is added to the `helpers` package, and is now
used in both the `neo4jcypher` and `neo4jexecutecypher` tools to ensure
consistent output formatting. Comprehensive unit tests for this function
are also included. Additionally, a new `ValueType` interface is defined
to generalize Neo4j value stringification.
**Helpers and Value Conversion:**
* Added the `ConvertValue` function to
`internal/tools/neo4j/neo4jschema/helpers/helpers.go` to recursively
convert Neo4j driver types (including nodes, relationships, paths,
points, and temporal types) into JSON-compatible Go values. This ensures
proper serialization of complex Neo4j types.
* Defined a `ValueType` interface in
`internal/tools/neo4j/neo4jschema/types/types.go` to generalize the
stringification of Neo4j value types.
**Integration with Tools:**
* Updated the `Invoke` methods in both `neo4jcypher` and
`neo4jexecutecypher` tools to use `helpers.ConvertValue` when processing
Neo4j query results, ensuring consistent and correct output formatting.
[[1]](diffhunk://#diff-b3e792b742cb92c92d1f5136b444c8fe0a7ec0376920868182dc88f13002e8eeL138-R139)
[[2]](diffhunk://#diff-de7fdd7e68c95ea9813c704a89fffb8fd6de34e81b43a484623fdff7683e18f3L160-R161)
* Added the necessary imports for the helpers package in the affected
files.
[[1]](diffhunk://#diff-b3e792b742cb92c92d1f5136b444c8fe0a7ec0376920868182dc88f13002e8eeR23)
[[2]](diffhunk://#diff-de7fdd7e68c95ea9813c704a89fffb8fd6de34e81b43a484623fdff7683e18f3R26)
[[3]](diffhunk://#diff-97a47eb63017102fc7084aa79689e22ef8cb6d1952945177058f118d57be306fR26)
**Testing:**
* Added extensive tests for `ConvertValue` in `helpers_test.go`,
covering all supported Neo4j types, primitives, slices, maps, and
unhandled types.
* Included required imports in the test file for Neo4j driver and time
handling.
**Relates to:** https://github.com/googleapis/genai-toolbox/issues/1344
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
This pull request introduces a new tool, `neo4j-schema`, for extracting
and processing comprehensive schema information from Neo4j databases. It
includes updates to the documentation, implementation of caching
mechanisms, helper utilities for schema transformation, and
corresponding unit tests. The most important changes are grouped by
theme below:
### Tool Integration:
- **`cmd/root.go`**: Added import for the new `neo4j-schema` tool to
integrate it into the application.
### Documentation:
- **`docs/en/resources/tools/neo4j/neo4j-schema.md`**: Added detailed
documentation for the `neo4j-schema` tool, describing its functionality,
caching behavior, and usage examples.
### Caching Implementation:
- **`internal/tools/neo4j/neo4jschema/cache/cache.go`**: Implemented a
thread-safe, in-memory cache with expiration and optional janitor for
cleaning expired items.
### Unit Tests:
- **`internal/tools/neo4j/neo4jschema/cache/cache_test.go`**: Added
comprehensive tests for the caching system, including functionality for
setting, retrieving, expiration, janitor cleanup, and concurrent access.
### Helper Utilities:
- **`internal/tools/neo4j/neo4jschema/helpers/helpers.go`**: Added
utility functions for processing schema data, including support for APOC
and native Cypher queries, and converting raw query results into
structured formats.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
This pull request introduces a new tool for executing arbitrary Cypher
queries against a Neo4j database (`neo4j-execute-cypher`) and implements
robust query classification functionality to distinguish between read
and write operations. The changes include updates to documentation, the
addition of a query classifier, and comprehensive test coverage for the
classifier.
### Addition of `neo4j-execute-cypher` tool:
- **Documentation**: Added a new markdown file `neo4j-execute-cypher.md`
that explains the tool's functionality, usage, and configuration
options, including the ability to enforce read-only mode for security.
- **Import statement**: Registered the new tool in the `cmd/root.go`
file to make it available in the toolbox.
### Query classification functionality:
- **Query classifier implementation**: Added `QueryClassifier` in
`classifier.go`, which classifies Cypher queries into read or write
operations based on keywords, procedures, and subquery analysis. It
supports handling edge cases like nested subqueries, multi-word
keywords, and invalid syntax.
- **Test coverage**: Created extensive tests in `classifier_test.go` to
validate the classifier's behavior across various query types, including
abuse cases, subqueries, and procedure calls. Tests ensure the
classifier is robust and does not panic on malformed queries.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
**Package Renaming**: The `neo4j` package has been renamed to
`neo4jcypher` to better reflect its functionality. This change affects
the tool's implementation
(`internal/tools/neo4j/neo4jcypher/neo4jcypher.go`) and its associated
tests (`internal/tools/neo4j/neo4jcypher/neo4jcypher_test.go`).
Update `tool.Invoke()` to return type `any` instead of `[]any`.
Toolbox return a map with the `results` key, and the SDK reads the
string from the key. So this won't break existing SDK implementation.
Fixes#870
This PR refactors the tool configuration and loading mechanism to use a
dynamic registration pattern. Each tool package now registers itself
with a central registry, and the server configuration code uses this
registry to decode and initialize tools.
Key changes:
- Introduced tools.Register and tools.DecodeToolConfig for dynamic tool
handling.
- Removed direct imports of specific tool packages from
internal/server/config.go.
- Updated individual tool packages to include init() functions for
self-registration.
- Modified ToolKind constants to be local kind constants within each
tool package.
- Adjusted test files to reflect the changes in tool kind identifiers.
This change simplifies adding new tools and decouples the server
configuration from specific tool implementations.
---------
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
Rename existing `authSource` to `authService` through deprecation.
`AuthService` more clearly distinguishes it from `Sources` objects.
`authSources` will be converted into `authServices` after the
unmarshalling process. A warning log is shown if `authSources` are used
(for both within tools parameters and defining auth services):
```
2025-02-20T13:57:51.156025-08:00 WARN "`authSources` is deprecated, use `authServices` for parameters instead"
2025-02-20T13:57:51.156569-08:00 WARN "`authSources` is deprecated, use `authServices` instead"
2025-02-20T13:57:52.047584-08:00 INFO "Initialized 1 sources."
...
```
The manifest generated will continue to use `authSources` to keep
compatibility with the sdks:
```
{
"serverVersion":"0.1.0",
"tools":{
"test_tool2":{
"description":"Use this tool to test\n",
"parameters":[{
"name":"user_id",
"type":"string",
"description":"Auto-populated from Google login",
"authSources":["my-google-auth"]
}]
}
}
}
```
Test cases with `authSources` are kept for compatibility. Will be
removed when `authSources` are no longer supported.
This only checks within `SourceConfig`, `ToolConfig`, and
`AuthSourceConfig`.
Error when an unknown field is provided:
`2025-01-27T22:43:46.988401-08:00 ERROR "unable to parse tool file at
\"tools.yaml\": unable to parse as \"cloud-sql-postgres\": [2:1] unknown
field \"extra\"\n 1 | database: test_database\n> 2 | extra: here\n ^\n 3
| instance: toolbox-cloudsql\n 4 | kind: cloud-sql-postgres\n 5 |
password: postgres\n 6 | "`
Error when a required field is not provided:
`2025-01-27T17:49:47.584846-08:00 ERROR "unable to parse tool file at
\"tools.yaml\": validation failed: Key: 'Config.Region' Error:Field
validation for 'Region' failed on the 'required' tag"`
---------
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
- configure neo4j source with url, username, password, database
- configure neo4j tools with cypher statement and paramters
- tests based on the postgres tests
- neo4j.yaml for integration tests
---------
Co-authored-by: duwenxin <duwenxin@google.com>
