This pull request introduces a new tool, `neo4j-schema`, for extracting
and processing comprehensive schema information from Neo4j databases. It
includes updates to the documentation, implementation of caching
mechanisms, helper utilities for schema transformation, and
corresponding unit tests. The most important changes are grouped by
theme below:
### Tool Integration:
- **`cmd/root.go`**: Added import for the new `neo4j-schema` tool to
integrate it into the application.
### Documentation:
- **`docs/en/resources/tools/neo4j/neo4j-schema.md`**: Added detailed
documentation for the `neo4j-schema` tool, describing its functionality,
caching behavior, and usage examples.
### Caching Implementation:
- **`internal/tools/neo4j/neo4jschema/cache/cache.go`**: Implemented a
thread-safe, in-memory cache with expiration and optional janitor for
cleaning expired items.
### Unit Tests:
- **`internal/tools/neo4j/neo4jschema/cache/cache_test.go`**: Added
comprehensive tests for the caching system, including functionality for
setting, retrieving, expiration, janitor cleanup, and concurrent access.
### Helper Utilities:
- **`internal/tools/neo4j/neo4jschema/helpers/helpers.go`**: Added
utility functions for processing schema data, including support for APOC
and native Cypher queries, and converting raw query results into
structured formats.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
- Users have the preference to choose their clients. Below example is
using Gemini CLI.
- Users can use the pre-built Dataplex tools by creating a settings.json
file under .gemini directory. The contents of settings.json would be as
follows:-
```
{
"mcpServers": {
"dataplex": {
"command": "./toolbox",
"args": ["--prebuilt","dataplex","--stdio"],
"env": {
"DATAPLEX_PROJECT": "test-project"
}
}
}
}
```
Fixes#831
---------
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
Co-authored-by: Mateusz Nowak <matnow@google.com>
Co-authored-by: Mateusz Nowak <kontakt@mateusznowak.pl>
According to the json schema spec:There are two numeric types in JSON
Schema: integer and number So 'float' is not with mcpcurl and mcphost
Fixes#984, #797
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
mysql driver return []uint8 types for "JSON", hence we will need to cast
it.
Toolbox will unmarshal JSON data retrieved from database during
invocation. If directly convert results into string, it will be
marshaled again when returning results to the user (causing double
marshaling).
Fixes#840
Example:
```
alloydb-operations-get:
kind: wait-for-operation
source: alloydb-api-source
method: GET
path: /v1/projects/{{.projectId}}/locations/{{.locationId}}/operations/{{.operationId}}
description: "Makes API call to check whether operation is done or not. This tool is run first then wait for tool. if its still in create phase trigger it after 3 minutes. Print a message saying still not done. We will notify once its done."
pathParams:
- name: projectId
type: string
description: The dynamic path parameter
- name: locationId
type: string
description: The dynamic path parameter
default: us-central1
- name: operationId
type: string
description: Operation status check for previous task
```
This pull request introduces a new tool for executing arbitrary Cypher
queries against a Neo4j database (`neo4j-execute-cypher`) and implements
robust query classification functionality to distinguish between read
and write operations. The changes include updates to documentation, the
addition of a query classifier, and comprehensive test coverage for the
classifier.
### Addition of `neo4j-execute-cypher` tool:
- **Documentation**: Added a new markdown file `neo4j-execute-cypher.md`
that explains the tool's functionality, usage, and configuration
options, including the ability to enforce read-only mode for security.
- **Import statement**: Registered the new tool in the `cmd/root.go`
file to make it available in the toolbox.
### Query classification functionality:
- **Query classifier implementation**: Added `QueryClassifier` in
`classifier.go`, which classifies Cypher queries into read or write
operations based on keywords, procedures, and subquery analysis. It
supports handling edge cases like nested subqueries, multi-word
keywords, and invalid syntax.
- **Test coverage**: Created extensive tests in `classifier_test.go` to
validate the classifier's behavior across various query types, including
abuse cases, subqueries, and procedure calls. Tests ensure the
classifier is robust and does not panic on malformed queries.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
**Package Renaming**: The `neo4j` package has been renamed to
`neo4jcypher` to better reflect its functionality. This change affects
the tool's implementation
(`internal/tools/neo4j/neo4jcypher/neo4jcypher.go`) and its associated
tests (`internal/tools/neo4j/neo4jcypher/neo4jcypher_test.go`).
support both generic and typed map. Config example:
```
parameters:
- name: user_scores
type: map
description: A map of user IDs to their scores. All scores must be integers.
valueType: integer # This enforces the value type for all entries. Leave it blank for generic map
```
Represented as `Object` with `additionalProperties` in manifests.
Added a util function to convert json.Number (string type) to int/float
types to address the problem where int/float values are converted to
strings for the generic map.
- Added a dry run step to identify the query type (e.g., SELECT, DML),
which allows the tool to correctly handle the query's output.
- The recommended high-level client, cloud.google.com/go/bigquery, does
not expose the statement type from a dry run. To circumvent this
limitation, the low-level BigQuery REST API client
(google.golang.org/api/bigquery/v2) was added to gain access to these
necessary details.
fixes: #915
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Firestore is a NoSQL document database built for automatic scaling, high
performance, and ease of application development. It's a fully managed,
serverless database that supports mobile, web, and server development.
This change adds Firestore as a source in toolbox
Update `tool.Invoke()` to return type `any` instead of `[]any`.
Toolbox return a map with the `results` key, and the SDK reads the
string from the key. So this won't break existing SDK implementation.
Fixes#870
`
tools:
wait_for_tool:
kind: wait-for
description: "Tool puts chat to sleep for given timeout example 3m,10s,
5m etc. For tasks such as cluster creation no need to wait. It takes
longer time such as 8m for instance creation. Use timeout value as
default if nothing is provided"
timeout: 10s
`
Output:
`
prernakakkar@prernakakkar:~/senseai/genai-toolbox$ curl -X POST -H
"Content-Type: application/json" -d '{"duration": "5m"}'
http://127.0.0.1:5000/api/tool/wa
it_for_tool/invoke
{"result":"[\"Wait for 5m0s completed successfully.\"]"}
`
Added an option for user to indicate if the parameter is required.
Example:
```
parameters:
- name: foo
type: string
description: foo
required: false
```
If the `required` field is not provided, it will be defaulted to `true`.
If a `default` value is provided, `required = false` regardless if the
field is indicated.
```
parameters:
- name: foo
type: string
description: foo
default: hello world
```
Fixes#736
- Spanner: convert array to typed before querying, as Spanner does not
accept non-typed array []any
- BigQuery: fix https://github.com/googleapis/genai-toolbox/issues/793
- Bigtable: add the required `ElemType` for array-type params
- Redis/Valkey: change indexing to append to avoid extra spaces
Add integration tests for array parameters, skipped for the sources not
supporting arrays:
- SQLite
- Cloud SQL MSSQL
- Cloud SQL MySQL
- MSSQL
- MySQL
Optional projectID parameter enables dynamic, cross-project resource
access in BigQuery tools.
This allows a single tool configuration to target different projects at
runtime, rather than being fixed to the project in its source
configuration.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
When go-yaml decode into CommonParameter with Default being an any type,
int will be converted into []uint64.
It will fail the Parse() when the value is being used since it does not
belong to either of the int types.
Unmarshal `default` value into each field's type directly.
Fixes#771
Copy fix from #757.
The Invoke function had two bugs:
A panic would occur when scanning a row containing a NULL value in a
TEXT or VARCHAR column. The code did not check for nil before attempting
a type assertion on the scanned value.
The *sql.Rows result was not being closed on all code paths, leading to
connection leaks that could exhaust the database connection pool.
This change corrects both issues. A guard clause now checks for nil
values before processing, and rows.Close() is deferred to guarantee the
connection is released.
The Invoke function had two bugs:
1. A panic would occur when scanning a row containing a NULL value in a
TEXT or VARCHAR column. The code did not check for nil before attempting
a type assertion on the scanned value.
2. The *sql.Rows result was not being closed on all code paths, leading
to connection leaks that could exhaust the database connection pool.
This change corrects both issues. A guard clause now checks for nil
values before processing, and rows.Close() is deferred to guarantee the
connection is released.
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
Add a `default` field to parameters, that enables users to specify a
default value.
e.g.
```
parameters:
- name: name
type: string
default: "some-default-value"
description: The name of the hotel.
```
if this parameter is invoked without specifying `name`, the parameter
would default to "some-default-value"
For parameter manifest, there will be an additional `Required` field.
The default `Required` field is true. If a `default` value is presented,
`Required: false`. Array parameter's item's `Required` field will
inherit the array's `Required` field.
Fixes#475
Add templateParameters to support non-filter parameters and DDL
statements.
Added a new argument `ignoreInsert` at integration test. Bigtable only
allow `SELECT` statement. This is used to filter insert statement for
bigtable.
Part of #535
Add `templateParameters` to support non-filter parameters and DDL
statements.
Added a new argument `ignoreDdl` at integration test. Admin client is
needed to execute ddl statement in spanner. Toolbox does not use admin
client.
Part of #535
Add new tool field, templateParameters, to support non-filter parameters
and DDL statements
Fix#535 for postgressql tool.
---------
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
This PR refactors the tool configuration and loading mechanism to use a
dynamic registration pattern. Each tool package now registers itself
with a central registry, and the server configuration code uses this
registry to decode and initialize tools.
Key changes:
- Introduced tools.Register and tools.DecodeToolConfig for dynamic tool
handling.
- Removed direct imports of specific tool packages from
internal/server/config.go.
- Updated individual tool packages to include init() functions for
self-registration.
- Modified ToolKind constants to be local kind constants within each
tool package.
- Adjusted test files to reflect the changes in tool kind identifiers.
This change simplifies adding new tools and decouples the server
configuration from specific tool implementations.
---------
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
@Yuan325 please review and merge this if all looks ok.
In existing code, for the character field with null values it will panic
with error:
"panic: interface conversion: interface {} is nil, not []uint8"
gosimple had been deprecated in favor of staticcheck:
https://github.com/golangci/golangci-lint/issues/357
Other requirements are all migrated.
`std-error-handling` exclusions is included because without that, it
will ask to check all error returns from (`Close()`, or `os.Setenv`s, or
`fmt.Fprint`s...
