This pull request introduces a new tool, `neo4j-schema`, for extracting
and processing comprehensive schema information from Neo4j databases. It
includes updates to the documentation, implementation of caching
mechanisms, helper utilities for schema transformation, and
corresponding unit tests. The most important changes are grouped by
theme below:
### Tool Integration:
- **`cmd/root.go`**: Added import for the new `neo4j-schema` tool to
integrate it into the application.
### Documentation:
- **`docs/en/resources/tools/neo4j/neo4j-schema.md`**: Added detailed
documentation for the `neo4j-schema` tool, describing its functionality,
caching behavior, and usage examples.
### Caching Implementation:
- **`internal/tools/neo4j/neo4jschema/cache/cache.go`**: Implemented a
thread-safe, in-memory cache with expiration and optional janitor for
cleaning expired items.
### Unit Tests:
- **`internal/tools/neo4j/neo4jschema/cache/cache_test.go`**: Added
comprehensive tests for the caching system, including functionality for
setting, retrieving, expiration, janitor cleanup, and concurrent access.
### Helper Utilities:
- **`internal/tools/neo4j/neo4jschema/helpers/helpers.go`**: Added
utility functions for processing schema data, including support for APOC
and native Cypher queries, and converting raw query results into
structured formats.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
- Users have the preference to choose their clients. Below example is
using Gemini CLI.
- Users can use the pre-built Dataplex tools by creating a settings.json
file under .gemini directory. The contents of settings.json would be as
follows:-
```
{
"mcpServers": {
"dataplex": {
"command": "./toolbox",
"args": ["--prebuilt","dataplex","--stdio"],
"env": {
"DATAPLEX_PROJECT": "test-project"
}
}
}
}
```
Fixes#831
---------
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
Co-authored-by: Mateusz Nowak <matnow@google.com>
Co-authored-by: Mateusz Nowak <kontakt@mateusznowak.pl>
Example:
```
alloydb-operations-get:
kind: wait-for-operation
source: alloydb-api-source
method: GET
path: /v1/projects/{{.projectId}}/locations/{{.locationId}}/operations/{{.operationId}}
description: "Makes API call to check whether operation is done or not. This tool is run first then wait for tool. if its still in create phase trigger it after 3 minutes. Print a message saying still not done. We will notify once its done."
pathParams:
- name: projectId
type: string
description: The dynamic path parameter
- name: locationId
type: string
description: The dynamic path parameter
default: us-central1
- name: operationId
type: string
description: Operation status check for previous task
```
This pull request introduces a new tool for executing arbitrary Cypher
queries against a Neo4j database (`neo4j-execute-cypher`) and implements
robust query classification functionality to distinguish between read
and write operations. The changes include updates to documentation, the
addition of a query classifier, and comprehensive test coverage for the
classifier.
### Addition of `neo4j-execute-cypher` tool:
- **Documentation**: Added a new markdown file `neo4j-execute-cypher.md`
that explains the tool's functionality, usage, and configuration
options, including the ability to enforce read-only mode for security.
- **Import statement**: Registered the new tool in the `cmd/root.go`
file to make it available in the toolbox.
### Query classification functionality:
- **Query classifier implementation**: Added `QueryClassifier` in
`classifier.go`, which classifies Cypher queries into read or write
operations based on keywords, procedures, and subquery analysis. It
supports handling edge cases like nested subqueries, multi-word
keywords, and invalid syntax.
- **Test coverage**: Created extensive tests in `classifier_test.go` to
validate the classifier's behavior across various query types, including
abuse cases, subqueries, and procedure calls. Tests ensure the
classifier is robust and does not panic on malformed queries.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
support both generic and typed map. Config example:
```
parameters:
- name: user_scores
type: map
description: A map of user IDs to their scores. All scores must be integers.
valueType: integer # This enforces the value type for all entries. Leave it blank for generic map
```
Represented as `Object` with `additionalProperties` in manifests.
Added a util function to convert json.Number (string type) to int/float
types to address the problem where int/float values are converted to
strings for the generic map.
Firestore is a NoSQL document database built for automatic scaling, high
performance, and ease of application development. It's a fully managed,
serverless database that supports mobile, web, and server development.
This change adds Firestore as a source in toolbox
`
tools:
wait_for_tool:
kind: wait-for
description: "Tool puts chat to sleep for given timeout example 3m,10s,
5m etc. For tasks such as cluster creation no need to wait. It takes
longer time such as 8m for instance creation. Use timeout value as
default if nothing is provided"
timeout: 10s
`
Output:
`
prernakakkar@prernakakkar:~/senseai/genai-toolbox$ curl -X POST -H
"Content-Type: application/json" -d '{"duration": "5m"}'
http://127.0.0.1:5000/api/tool/wa
it_for_tool/invoke
{"result":"[\"Wait for 5m0s completed successfully.\"]"}
`
feat(docs): Enhance Bigtable GoogleSQL DML clarity
**Description:**
This PR updates the `bigtable-sql` tool documentation to provide a
clearer understanding of GoogleSQL's DML capabilities within Bigtable.
**Changes Made:**
- Added a note indicating that Bigtable's GoogleSQL dialect only
supports `SELECT` DML operations.
- Explicitly states that `INSERT`, `UPDATE`, and `DELETE` DML statements
are not supported.
- Updated the external link to directly reference the "Use cases"
section in the official Google Cloud Bigtable GoogleSQL overview, where
this limitation is detailed.
**Reasoning:**
The previous documentation, while linking to the overview, did not
explicitly highlight the DML limitations. This clarification will
prevent potential confusion for users expecting full DML support,
ensuring the documentation accurately reflects Bigtable's capabilities
with GoogleSQL.
---------
Co-authored-by: Twisha Bansal <58483338+twishabansal@users.noreply.github.com>
Optional projectID parameter enables dynamic, cross-project resource
access in BigQuery tools.
This allows a single tool configuration to target different projects at
runtime, rather than being fixed to the project in its source
configuration.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
## Problem
Users attempting to filter results in a SQL query based on an array
parameter from a tool may intuitively write a `statement` using the `IN`
clause, like so:
```sql
SELECT * FROM flights WHERE preferred_airlines IN ($1);
```
When this query is executed with an array argument (e.g., `["Delta",
"United"]`), it fails with a cryptic error message from the database
driver:
```
Exception: error while invoking tool: unable to execute query: failed to encode args[0]: unable to encode []interface {}{"Delta", "United"} into text format for text (OID 25): cannot find encode plan
```
This error occurs because the driver does not automatically expand the
single `$1` placeholder into a list of values `('Delta', 'United')`.
Instead, it tries to encode the entire Go slice `[]interface{}` as a
single text value, which fails. This creates a point of friction, as the
correct syntax is not immediately obvious and can lead to user
frustration and debugging time.
## Solution
This PR updates our documentation and example usage to demonstrate the
correct SQL syntax for handling array parameters. The proper way to
check for a value's existence in an array parameter is by using
PostgreSQL's `ANY()` operator:
```sql
SELECT * FROM flights WHERE preferred_airlines = ANY($1);
```
When this syntax is used, the database driver correctly interprets the
Go slice passed as `$1` as a PostgreSQL array, and the query executes as
intended.
## Impact
Saves developers time they would otherwise spend troubleshooting a
non-obvious database driver behavior.
Add a `default` field to parameters, that enables users to specify a
default value.
e.g.
```
parameters:
- name: name
type: string
default: "some-default-value"
description: The name of the hotel.
```
if this parameter is invoked without specifying `name`, the parameter
would default to "some-default-value"
For parameter manifest, there will be an additional `Required` field.
The default `Required` field is true. If a `default` value is presented,
`Required: false`. Array parameter's item's `Required` field will
inherit the array's `Required` field.
Fixes#475
This tool can be used across spanner sources.
`spanner-execute-sql` config is as below:
```
tools:
spanner_execute_sql_tool:
kind: "spanner-execute-sql"
source: my-spanner-source
description: Use this tool to execute sql.
```
The `spanner-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/spanner_execute_sql_tool/invoke
```
---------
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
This tool can be used across mysql sources.
`mysql-execute-sql` config is as below:
```
tools:
mysql_execute_sql_tool:
kind: "mysql-execute-sql"
source: my-mysql-source
description: Use this tool to execute sql.
```
The `mysql-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/mysql_execute_sql_tool/invoke
```
Allowing user to add `readOnly` field in spanner tools.
The existing tool doesn't work for reading schema tables since schema
tables can only be accessed through read-only transaction.
This PR also resolve#435 for Spanner tool.
---------
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
The wording for `Authorized Invocations` reads a bit funny and has a
typo.
Updating it to make a bit more sense and fixing a couple other typos
around auth.
This tool can be used across all postgres sources.
`postgres-execute-sql` config is as below:
```
tools:
postgres_execute_sql_tool:
kind: "postgres-execute-sql"
source: my-alloydb-source // or any other sources that is compatible with this tool
description: Use this tool to execute sql.
```
The `postgres-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/postgres_execute_sql_tool/invoke
```
A `BigQuery` source can be added as the following example:
```yaml
sources:
my-bigquery-source:
kind: bigquery
project: bigframes-dev
location: us # This field is optional
```
A `BigQuery` tool can be added as below:
```yaml
tools:
search-hotels-by-name:
kind: bigquery-sql
source: my-bigquery-source
description: Search for hotels based on name.
parameters:
- name: name
type: string
description: The name of the hotel.
```
---------
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
When previewing the site, warnings are printed out:
```
WARN Shortcode "tabpane": duplicate tab-persistence key "yaml" detected, disabling persistence to avoid multiple tab display.
```
This PR fixes these issues.
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
# Add Bigtable support
A `bigtable` source can be added as the following example
```
sources:
test-bigtable-source:
kind: "bigtable"
project: "sample-project"
instance: "sample-instance"
```
A `bigtable` tool can be added as below
```
tools:
get-test-tool-data:
kind: bigtable-sql
source: test-bigtable-source
description: Some description
statement: SELECT * FROM `test-table` WHERE address['state'] = @state;
parameters:
- name: state
type: string
description: Filter by state
```
---------
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
