This pull request introduces a new tool, `neo4j-schema`, for extracting
and processing comprehensive schema information from Neo4j databases. It
includes updates to the documentation, implementation of caching
mechanisms, helper utilities for schema transformation, and
corresponding unit tests. The most important changes are grouped by
theme below:
### Tool Integration:
- **`cmd/root.go`**: Added import for the new `neo4j-schema` tool to
integrate it into the application.
### Documentation:
- **`docs/en/resources/tools/neo4j/neo4j-schema.md`**: Added detailed
documentation for the `neo4j-schema` tool, describing its functionality,
caching behavior, and usage examples.
### Caching Implementation:
- **`internal/tools/neo4j/neo4jschema/cache/cache.go`**: Implemented a
thread-safe, in-memory cache with expiration and optional janitor for
cleaning expired items.
### Unit Tests:
- **`internal/tools/neo4j/neo4jschema/cache/cache_test.go`**: Added
comprehensive tests for the caching system, including functionality for
setting, retrieving, expiration, janitor cleanup, and concurrent access.
### Helper Utilities:
- **`internal/tools/neo4j/neo4jschema/helpers/helpers.go`**: Added
utility functions for processing schema data, including support for APOC
and native Cypher queries, and converting raw query results into
structured formats.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
- Users have the preference to choose their clients. Below example is
using Gemini CLI.
- Users can use the pre-built Dataplex tools by creating a settings.json
file under .gemini directory. The contents of settings.json would be as
follows:-
```
{
"mcpServers": {
"dataplex": {
"command": "./toolbox",
"args": ["--prebuilt","dataplex","--stdio"],
"env": {
"DATAPLEX_PROJECT": "test-project"
}
}
}
}
```
Fixes#831
---------
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
Co-authored-by: Mateusz Nowak <matnow@google.com>
Co-authored-by: Mateusz Nowak <kontakt@mateusznowak.pl>
## 1. Why do we need to support the `encrypt` parameter?
MSSQL databases that `genai-toolbox` attempts to connect to may have
their encryption levels set differently. For example, a testing/demo
purpose MSSQL database may not require encryption at all. However,
`genai-toolbox` currently uses the default encryption parameter
(`encrypt=false`) to connect to this type of database and will throw an
error:
```
ERROR "toolbox failed to initialize: unable to initialize configs: unable to initialize source "my-mssql-source": unable to connect successfully: TLS Handshake failed: cannot read handshake packet: EOF"
```
> In this case, the encryption parameter should be set to
`encrypt=disable`.
## 2. Is this a necessary feature?
`genai-toolbox` uses the `github.com/microsoft/go-mssqldb` package as a
dependency to connect to MSSQL databases. According to the
[README](https://github.com/microsoft/go-mssqldb?tab=readme-ov-file#common-parameters)
of the `github.com/microsoft/go-mssqldb` package, `encrypt` is one of
the common parameters. Therefore, I believe supporting the `encrypt`
parameter in `genai-toolbox` is necessary.
## 3. How to replicate the error mentioned above?
### 3.1 Use this `docker-compose.yaml` file to start a demo MSSQL
instance
```
services:
demo-mssql-database:
image: mcr.microsoft.com/mssql/server:2017-CU1-ubuntu
ports:
- "20256:1433"
environment:
ACCEPT_EULA: "Y"
MSSQL_SA_PASSWORD: "hellopassword!"
restart: unless-stopped
healthcheck:
test: ["CMD", "/opt/mssql-tools/bin/sqlcmd", "-S", "localhost", "-U", "sa", "-P", "hellopassword!", "-Q", "SELECT 1"]
interval: 5s
retries: 10
demo-mssql-database-init:
image: mcr.microsoft.com/mssql/server:2017-CU1-ubuntu
network_mode: service:demo-mssql-database
command: >
/bin/bash -c "/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P hellopassword! -d master -Q 'CREATE DATABASE DemoDatabase;'"
depends_on:
demo-mssql-database:
condition: service_healthy
```
### 3.2 Use `genai-toolbox` to connect to the above demo MSSQL database
with this `tools.yaml` configuration file:
```
sources:
my-mssql-source:
kind: mssql
host: localhost
port: 20256
database: master
user: sa
password: 'hellopassword!'
```
### 3.3 We shall see the error:
```
ERROR "toolbox failed to initialize: unable to initialize configs: unable to initialize source "my-mssql-source": unable to connect successfully: TLS Handshake failed: cannot read handshake packet: EOF"
```
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
According to the json schema spec:There are two numeric types in JSON
Schema: integer and number So 'float' is not with mcpcurl and mcphost
Fixes#984, #797
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
mysql driver return []uint8 types for "JSON", hence we will need to cast
it.
Toolbox will unmarshal JSON data retrieved from database during
invocation. If directly convert results into string, it will be
marshaled again when returning results to the user (causing double
marshaling).
Fixes#840
Example:
```
alloydb-operations-get:
kind: wait-for-operation
source: alloydb-api-source
method: GET
path: /v1/projects/{{.projectId}}/locations/{{.locationId}}/operations/{{.operationId}}
description: "Makes API call to check whether operation is done or not. This tool is run first then wait for tool. if its still in create phase trigger it after 3 minutes. Print a message saying still not done. We will notify once its done."
pathParams:
- name: projectId
type: string
description: The dynamic path parameter
- name: locationId
type: string
description: The dynamic path parameter
default: us-central1
- name: operationId
type: string
description: Operation status check for previous task
```
This pull request introduces a new tool for executing arbitrary Cypher
queries against a Neo4j database (`neo4j-execute-cypher`) and implements
robust query classification functionality to distinguish between read
and write operations. The changes include updates to documentation, the
addition of a query classifier, and comprehensive test coverage for the
classifier.
### Addition of `neo4j-execute-cypher` tool:
- **Documentation**: Added a new markdown file `neo4j-execute-cypher.md`
that explains the tool's functionality, usage, and configuration
options, including the ability to enforce read-only mode for security.
- **Import statement**: Registered the new tool in the `cmd/root.go`
file to make it available in the toolbox.
### Query classification functionality:
- **Query classifier implementation**: Added `QueryClassifier` in
`classifier.go`, which classifies Cypher queries into read or write
operations based on keywords, procedures, and subquery analysis. It
supports handling edge cases like nested subqueries, multi-word
keywords, and invalid syntax.
- **Test coverage**: Created extensive tests in `classifier_test.go` to
validate the classifier's behavior across various query types, including
abuse cases, subqueries, and procedure calls. Tests ensure the
classifier is robust and does not panic on malformed queries.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
### What this PR does:
This Pull Request resolves a critical configuration error in the
`cloud-sql-mssql.yaml` prebuilt tool definition, which was preventing
the `execute_sql` tool from correctly initializing and connecting to a
Cloud SQL for SQL Server instance.
### Problem:
Previously, the `source` field in
`internal/prebuiltconfigs/tools/cloud-sql-mssql.yaml` was incorrectly
configured as `cloud-sql-mssql-source`. This led to the `genai-toolbox`
server failing to find the corresponding data source, resulting in
errors like:
**Package Renaming**: The `neo4j` package has been renamed to
`neo4jcypher` to better reflect its functionality. This change affects
the tool's implementation
(`internal/tools/neo4j/neo4jcypher/neo4jcypher.go`) and its associated
tests (`internal/tools/neo4j/neo4jcypher/neo4jcypher_test.go`).
support both generic and typed map. Config example:
```
parameters:
- name: user_scores
type: map
description: A map of user IDs to their scores. All scores must be integers.
valueType: integer # This enforces the value type for all entries. Leave it blank for generic map
```
Represented as `Object` with `additionalProperties` in manifests.
Added a util function to convert json.Number (string type) to int/float
types to address the problem where int/float values are converted to
strings for the generic map.
- Added a dry run step to identify the query type (e.g., SELECT, DML),
which allows the tool to correctly handle the query's output.
- The recommended high-level client, cloud.google.com/go/bigquery, does
not expose the statement type from a dry run. To circumvent this
limitation, the low-level BigQuery REST API client
(google.golang.org/api/bigquery/v2) was added to gain access to these
necessary details.
fixes: #915
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Firestore is a NoSQL document database built for automatic scaling, high
performance, and ease of application development. It's a fully managed,
serverless database that supports mobile, web, and server development.
This change adds Firestore as a source in toolbox
This PR consist of two fixes:
1. Added `default` to table_names parameter since the default value is
an empty string.
2. Update the sql statement for reusing parameter within the statement.
Fixes partial #877
Update `tool.Invoke()` to return type `any` instead of `[]any`.
Toolbox return a map with the `results` key, and the SDK reads the
string from the key. So this won't break existing SDK implementation.
Fixes#870
`
tools:
wait_for_tool:
kind: wait-for
description: "Tool puts chat to sleep for given timeout example 3m,10s,
5m etc. For tasks such as cluster creation no need to wait. It takes
longer time such as 8m for instance creation. Use timeout value as
default if nothing is provided"
timeout: 10s
`
Output:
`
prernakakkar@prernakakkar:~/senseai/genai-toolbox$ curl -X POST -H
"Content-Type: application/json" -d '{"duration": "5m"}'
http://127.0.0.1:5000/api/tool/wa
it_for_tool/invoke
{"result":"[\"Wait for 5m0s completed successfully.\"]"}
`
This PR add supports to MCP version 2025-06-18 defined
[here](https://modelcontextprotocol.io/specification/2025-06-18).
The main updates includes:
* Retrieving protocol version from header via `MCP-Protocol-Version`.
* Throwing `400 Bad Request` when an invalid version is received.
Added an option for user to indicate if the parameter is required.
Example:
```
parameters:
- name: foo
type: string
description: foo
required: false
```
If the `required` field is not provided, it will be defaulted to `true`.
If a `default` value is provided, `required = false` regardless if the
field is indicated.
```
parameters:
- name: foo
type: string
description: foo
default: hello world
```
Fixes#736
## Summary
- Added configurable query timeout to MySQL source configuration
- Updated connection DSN to include readTimeout parameter
- Added documentation and example usage
- Added test coverage for queryTimeout configuration
## Test plan
- [x] Added unit test for queryTimeout configuration parsing
- [x] Updated documentation with queryTimeout field description
- [x] Verified timeout parameter is correctly added to DSN when
specified
## Caveat
When queryTimeout is exceeded, we get an obscure error message
([screenshot](https://github.com/user-attachments/assets/fd292f91-328d-4ebc-9a87-2d92e9887300)):
```
unable to execute query: invalid connection
```
This seems to be a problem with the mysql-go library:
https://stackoverflow.com/q/65253798/10720618
I tried to use
[MAX_EXECUTION_TIME](https://dev.mysql.com/doc/refman/8.0/en/optimizer-hints.html#optimizer-hints-execution-time)
but it didn't work as expected (my `sleep(MAX_EXECUTION_TIME+3)` query
finished successfully after MAX_EXECUTION_TIME milliseconds)
Any ideas on what can be done here? The error message is very
misleading. My goal with adding timeouts is to communicate to the LLM
when it has issued a slow query and force it to adjust (e.g. query
indexes and write a more optimized query) but this defeats the purpose.
---------
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
- Spanner: convert array to typed before querying, as Spanner does not
accept non-typed array []any
- BigQuery: fix https://github.com/googleapis/genai-toolbox/issues/793
- Bigtable: add the required `ElemType` for array-type params
- Redis/Valkey: change indexing to append to avoid extra spaces
Add integration tests for array parameters, skipped for the sources not
supporting arrays:
- SQLite
- Cloud SQL MSSQL
- Cloud SQL MySQL
- MSSQL
- MySQL
Allow Toolbox server to automatically update when users modify their
tool configuration file(s), instead of requiring a restart.
This feature is automatically enabled, but can be turned off with the
flag `--disable-reload`.
Optional projectID parameter enables dynamic, cross-project resource
access in BigQuery tools.
This allows a single tool configuration to target different projects at
runtime, rather than being fixed to the project in its source
configuration.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
When go-yaml decode into CommonParameter with Default being an any type,
int will be converted into []uint64.
It will fail the Parse() when the value is being used since it does not
belong to either of the int types.
Unmarshal `default` value into each field's type directly.
Fixes#771
Copy fix from #757.
The Invoke function had two bugs:
A panic would occur when scanning a row containing a NULL value in a
TEXT or VARCHAR column. The code did not check for nil before attempting
a type assertion on the scanned value.
The *sql.Rows result was not being closed on all code paths, leading to
connection leaks that could exhaust the database connection pool.
This change corrects both issues. A guard clause now checks for nil
values before processing, and rows.Close() is deferred to guarantee the
connection is released.
The Invoke function had two bugs:
1. A panic would occur when scanning a row containing a NULL value in a
TEXT or VARCHAR column. The code did not check for nil before attempting
a type assertion on the scanned value.
2. The *sql.Rows result was not being closed on all code paths, leading
to connection leaks that could exhaust the database connection pool.
This change corrects both issues. A guard clause now checks for nil
values before processing, and rows.Close() is deferred to guarantee the
connection is released.
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
This feature includes the following:
* Implement initialize lifecycle (including version negotiation)
* Add the v20250326 schema
* Supporting the `DELETE` and `GET` endpoint for MCP.
* Supporting streamable HTTP (without SSE).
* Terminating sessions after timeout (default = 10 minutes from last
active).
* Toolbox do not support batch request. Will response with `Invalid
requests` if batch requests is received.
Add a `default` field to parameters, that enables users to specify a
default value.
e.g.
```
parameters:
- name: name
type: string
default: "some-default-value"
description: The name of the hotel.
```
if this parameter is invoked without specifying `name`, the parameter
would default to "some-default-value"
For parameter manifest, there will be an additional `Required` field.
The default `Required` field is true. If a `default` value is presented,
`Required: false`. Array parameter's item's `Required` field will
inherit the array's `Required` field.
Fixes#475
