1. Add ask-data-insights tool based on conversational analytic API.
2. Add tokenSource for ask-data-insights tool, it uses access token
instead of client or restService.
3. Add a max row count to source, currently fixed to 50 and used only
for ask-data-insights tool. Later we may make it available for user to
make change and apply to bigquery-execute-sql and bigquery-sql to avoid
return too many data by accident.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
## Description
This Pull Request introduces support for the Firebird database as a new
`source` and adds the `firebird-sql` and `firebird-execute-sql` tools.
The implementation follows the architectural pattern established by the
existing PostgreSQL and MySQL tools.
Comprehensive integration tests have been added in the `tests/firebird`
directory, covering table creation, data insertion, and the invocation
of all related tools. Corresponding user and developer documentation has
also been included.
## Test Results
The core functionality is working as demonstrated by the vast majority
of tests passing. However, a few tests fail due to limitations and
inflexibilities in the generic test harness (`tests/tool.go`), which is
not fully compatible with Firebird's specific syntax and behavior.
### Known and Justified Failures:
* **`invoke_my-tool` / `invoke_my-array-tool`:** The `RunToolInvokeTest`
function reuses the same `want` variable for both sub-tests, which have
different inputs and expected outputs. The test is configured for
`my-tool` to pass, which consequently causes the `my-array-tool` test to
fail its assertion. The `got` log for the array test confirms that the
tool returns the correct response for the given input.
* **`invoke_my-exec-sql-tool`:** This test fails because the harness
sends the query `SELECT 1`, which is invalid syntax in Firebird.
Subsequent tests for this tool using DDL and DML (`create`, `select`,
`drop`) all pass, confirming its core functionality.
* **`invoke_select-fields-templateParams-tool`:** This test fails due to
a case-sensitivity mismatch in column names (`NAME` vs. `name`). The
expected output in the test harness is hardcoded with a lowercase name.
* **Authentication Tests:** All authentication-related tests are
failing. This appears to be a general issue with the local test
environment setup and is not specific to the Firebird implementation.
## Next Steps
I believe the implementation is ready for review. I am available to make
any requested changes.
Feat #935
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
## Add firestore-update-document tool
Adds a new tool for updating existing documents in Firestore
collections.
__What it does:__
- Updates documents at any path in Firestore
- Supports partial updates with field masks for selective field
modification
- Handles all Firestore data types (strings, numbers, booleans,
timestamps, geopoints, arrays, maps, etc.)
- Supports field deletion using updateMask
- Uses Firestore's native JSON format for type safety
- Can update nested fields within maps using dot notation
__Key parameters:__
- `documentPath`: The path of the document to update
- `documentData`: The document content in Firestore JSON format
- `updateMask`: Optional array of field paths for selective updates
- `returnData`: Optional flag to include updated document in response
__Special features:__
- When `updateMask` is provided, only specified fields are updated
- Can access nested fields with dot notation (e.g., 'address.city',
'user.profile.name')
- Without updateMask, performs a merge operation updating all provided
fields
## Description
---
This PR changes the help description in the prebuilt flag from hardcoded
source options to dynamically fetching it from the prebuilt configs.
## PR Checklist
---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/langchain-google-alloydb-pg-python/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change
🛠️ Fixes #<issue_number_goes_here>
## Add firestore-add-documents tool
Adds a new tool for creating documents in Firestore collections.
__What it does:__
- Adds documents to any Firestore collection
- Auto-generates unique document IDs
- Supports all Firestore data types (strings, numbers, booleans,
timestamps, geopoints, arrays, maps, etc.)
- Uses Firestore's native JSON format for type safety
__Key parameters:__
- `collectionPath`: Where to add the document
- `documentData`: The document content in Firestore JSON format
- `returnData`: Optional flag to include created document in response
---------
Co-authored-by: Averi Kitsch <akitsch@google.com>
Introduce Toolbox UI, which can be launched with the `--ui` flag.
This initial version of Toolbox UI allows users to test Toolbox by
inspecting tools/toolsets, modifying parameters, managing headers, and
executing API calls.
This PR supports TiDB in:
1. sources - tidb: As a data source;
2. tools - tidb - tidbsql: As a prepared SQL running tool;
3. tools - tidb - tidbexecutesql: As an arbitrary SQL running tool (for
development purposes).
And its corresponding docs.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
Fixes#861
This PR adds support for DuckDB which is a free, open-source, embedded,
in-process, relational database management system (RDBMS) designed for
analytical processing (OLAP)
---------
Co-authored-by: Averi Kitsch <akitsch@google.com>
This pull request introduces a new tool, `neo4j-schema`, for extracting
and processing comprehensive schema information from Neo4j databases. It
includes updates to the documentation, implementation of caching
mechanisms, helper utilities for schema transformation, and
corresponding unit tests. The most important changes are grouped by
theme below:
### Tool Integration:
- **`cmd/root.go`**: Added import for the new `neo4j-schema` tool to
integrate it into the application.
### Documentation:
- **`docs/en/resources/tools/neo4j/neo4j-schema.md`**: Added detailed
documentation for the `neo4j-schema` tool, describing its functionality,
caching behavior, and usage examples.
### Caching Implementation:
- **`internal/tools/neo4j/neo4jschema/cache/cache.go`**: Implemented a
thread-safe, in-memory cache with expiration and optional janitor for
cleaning expired items.
### Unit Tests:
- **`internal/tools/neo4j/neo4jschema/cache/cache_test.go`**: Added
comprehensive tests for the caching system, including functionality for
setting, retrieving, expiration, janitor cleanup, and concurrent access.
### Helper Utilities:
- **`internal/tools/neo4j/neo4jschema/helpers/helpers.go`**: Added
utility functions for processing schema data, including support for APOC
and native Cypher queries, and converting raw query results into
structured formats.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
- Users have the preference to choose their clients. Below example is
using Gemini CLI.
- Users can use the pre-built Dataplex tools by creating a settings.json
file under .gemini directory. The contents of settings.json would be as
follows:-
```
{
"mcpServers": {
"dataplex": {
"command": "./toolbox",
"args": ["--prebuilt","dataplex","--stdio"],
"env": {
"DATAPLEX_PROJECT": "test-project"
}
}
}
}
```
Fixes#831
---------
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
Co-authored-by: Mateusz Nowak <matnow@google.com>
Co-authored-by: Mateusz Nowak <kontakt@mateusznowak.pl>
Example:
```
alloydb-operations-get:
kind: wait-for-operation
source: alloydb-api-source
method: GET
path: /v1/projects/{{.projectId}}/locations/{{.locationId}}/operations/{{.operationId}}
description: "Makes API call to check whether operation is done or not. This tool is run first then wait for tool. if its still in create phase trigger it after 3 minutes. Print a message saying still not done. We will notify once its done."
pathParams:
- name: projectId
type: string
description: The dynamic path parameter
- name: locationId
type: string
description: The dynamic path parameter
default: us-central1
- name: operationId
type: string
description: Operation status check for previous task
```
This pull request introduces a new tool for executing arbitrary Cypher
queries against a Neo4j database (`neo4j-execute-cypher`) and implements
robust query classification functionality to distinguish between read
and write operations. The changes include updates to documentation, the
addition of a query classifier, and comprehensive test coverage for the
classifier.
### Addition of `neo4j-execute-cypher` tool:
- **Documentation**: Added a new markdown file `neo4j-execute-cypher.md`
that explains the tool's functionality, usage, and configuration
options, including the ability to enforce read-only mode for security.
- **Import statement**: Registered the new tool in the `cmd/root.go`
file to make it available in the toolbox.
### Query classification functionality:
- **Query classifier implementation**: Added `QueryClassifier` in
`classifier.go`, which classifies Cypher queries into read or write
operations based on keywords, procedures, and subquery analysis. It
supports handling edge cases like nested subqueries, multi-word
keywords, and invalid syntax.
- **Test coverage**: Created extensive tests in `classifier_test.go` to
validate the classifier's behavior across various query types, including
abuse cases, subqueries, and procedure calls. Tests ensure the
classifier is robust and does not panic on malformed queries.
---------
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
**Package Renaming**: The `neo4j` package has been renamed to
`neo4jcypher` to better reflect its functionality. This change affects
the tool's implementation
(`internal/tools/neo4j/neo4jcypher/neo4jcypher.go`) and its associated
tests (`internal/tools/neo4j/neo4jcypher/neo4jcypher_test.go`).
Firestore is a NoSQL document database built for automatic scaling, high
performance, and ease of application development. It's a fully managed,
serverless database that supports mobile, web, and server development.
This change adds Firestore as a source in toolbox
`
tools:
wait_for_tool:
kind: wait-for
description: "Tool puts chat to sleep for given timeout example 3m,10s,
5m etc. For tasks such as cluster creation no need to wait. It takes
longer time such as 8m for instance creation. Use timeout value as
default if nothing is provided"
timeout: 10s
`
Output:
`
prernakakkar@prernakakkar:~/senseai/genai-toolbox$ curl -X POST -H
"Content-Type: application/json" -d '{"duration": "5m"}'
http://127.0.0.1:5000/api/tool/wa
it_for_tool/invoke
{"result":"[\"Wait for 5m0s completed successfully.\"]"}
`
Allow fsnotify `CREATE` and `RENAME` events to trigger dynamic reload,
instead of requiring `WRITE`.
This is due to some cases with specific editors/OS, (ex: vim on Mac),
where file writes can potentially occur without ever triggering a
`WRITE` operation on the watched tools file, but solely through creation
and renaming of .swp files.
Allow Toolbox server to automatically update when users modify their
tool configuration file(s), instead of requiring a restart.
This feature is automatically enabled, but can be turned off with the
flag `--disable-reload`.
# Add Multiple YAML Configuration File Support
## 🎯 Overview
This PR introduces support for loading and merging multiple YAML
configuration files in Toolbox, addressing the need for modular
configuration management in complex deployments.
## ✨ New Features
### 1. Multiple Files Support (`--tools-files`)
- **Usage**: `--tools-files=file1.yaml,file2.yaml,file3.yaml`
- Load and intelligently merge multiple YAML configuration files
- Comma-separated file paths for maximum flexibility
### 2. Directory Support (`--tools-folder`)
- **Usage**: `--tools-folder=config-directory`
- Automatically discover and load all `.yaml` and `.yml` files from a
directory
- Simplifies configuration management for organized deployments
### 3. Smart Merging Logic
- **Sources/AuthServices/Tools**: Later files override earlier files
with same names
- **Toolsets**: Tools from same-named toolsets are combined without
duplicates
- Preserves all existing functionality while enabling composition
## 🔒 Safety & Validation
- **Mutual Exclusivity**: Prevents simultaneous use of `--tools-file`,
`--tools-files`, `--tools-folder`, and `--prebuilt`
- **Clear Error Messages**: Descriptive validation errors guide users to
correct usage
- **Comprehensive Error Handling**: Proper handling of missing files,
directories, and parsing errors
- **Full Backward Compatibility**: Existing configurations continue to
work unchanged
## 🏗️ Implementation Details
### Core Functions Added
- `mergeToolsFiles()` - Smart merging with configurable override rules
- `loadAndMergeToolsFiles()` - Multi-file loading and processing
- `loadAndMergeToolsFolder()` - Directory scanning and batch loading
### Command Structure Updates
- New `tools_files []string` field for multiple file paths
- New `tools_folder string` field for directory path
- Enhanced validation logic in `run()` function
- Updated flag definitions with proper descriptions
## 📋 Use Cases
### Organizational Benefits
- **Modular Configuration**: Separate database, API, and auth
configurations
- **Team Collaboration**: Multiple developers can work on different
config files
- **Environment Management**: Easy configuration swapping for different
environments
- **Scalability**: Large configurations can be broken into manageable
chunks
### Example Usage Patterns
```bash
# Multiple specific files
./toolbox --tools-files=database.yaml,apis.yaml,auth.yaml
# Directory-based loading
./toolbox --tools-folder=./production-configs
# Error case (properly handled)
./toolbox --tools-file=single.yaml --tools-folder=configs
# ERROR: --tools-file, --tools-files, and --tools-folder flags cannot be used simultaneously
---------
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
