Commit Graph

60 Commits

Author SHA1 Message Date
trehanshakuntG
60b26608dd feat!: update prebuilt-tool names to use consistent guidance (#1421)
## Description

---
This PR updates the naming conventions across prebuilt tool
configuration files to improve consistency:

__Changes:__

- __Toolset names__: Changed from hyphen-separated to
underscore-separated format (e.g., `firestore-database-tools` →
`firestore_database_tools`)
- __Tool names__: Removed product name prefixes for cleaner naming and
using underscore-separated format (e.g., `firestore-get-documents` →
`get_documents`, `alloydb-create-cluster` → `create_cluster`)
- __Copyright headers__: Added missing copyright headers to
configuration files
- __Test updates__: Updated `cmd/root_test.go` to reflect the new naming
conventions


## PR Checklist

---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change
2025-09-15 14:38:09 +05:30
Ajaykumar Yadav
681c2b4f3a feat(prebuilt/sqlite): prebuilt tools for the sqlite. (#1227)
## Description
Prebuilt tools for the sqlite
- [x] `list_table` with simple and detailed(trigger,index,column) for
each table
- [x]  `execute-sql` for executing any SQL statement for sqlite.
- [x]  added tests and done required changes in config.
- [x] **Documentation update**:Done

## PR Checklist

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/langchain-google-alloydb-pg-python/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involves a breaking change

🛠️ Fixes: https://github.com/googleapis/genai-toolbox/issues/1226

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-09-15 10:09:34 +05:30
Sri Varshitha
c181dabc91 feat(tools/alloydb-get-cluster): Add get-cluster tool for alloydb (#1420)
## Description

This pull request introduces a new custom tool kind
`alloydb-get-cluster` that retrieves detailed information for a specific
AlloyDB cluster.

### Example Configuration

```yaml
tools:
  get_cluster:
    kind: alloydb-get-cluster
    source: alloydb-admin-source
    description: Use this tool to retrieve detailed information for a specific AlloyDB cluster.
```

### Example Request
``` 
curl -X POST http://127.0.0.1:5000/api/tool/get_cluster/invoke \
-H "Content-Type: application/json" \
-d '{
    "projectId": "example-project",
    "locationId": "us-central1",
    "clusterId": "my-alloydb-cluster",
}'
```
## PR Checklist

---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #<issue_number_goes_here>
2025-09-12 15:39:45 +05:30
Sri Varshitha
93c1b30fce feat(tools/alloydb-list-instances): Add custom tool kind for AlloyDB list_instances (#1357)
## Description

---
This pull request introduces a new custom tool kind
`alloydb-list-instances` that allows users to list the AlloyDB instances
in a given project, cluster and location.

### Example Configuration

```yaml
tools:
  list_instances:
    kind: alloydb-list-instances
    source: alloydb-admin-source
    description: Use this tool to list all AlloyDB instances for a given project, cluster and location.
```

### Example Request
``` 
curl -X POST http://127.0.0.1:5000/api/tool/list_instances/invoke \
-H "Content-Type: application/json" \
-d '{
    "projectId": "example-project",
    "locationId": "us-central1",
    "clusterId": "example-cluster",
}'
```

## PR Checklist

---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #<issue_number_goes_here>
2025-09-12 15:08:23 +05:30
Sri Varshitha
3a8a65ceaa feat(tools/alloydb-list-users): Add new custom tool kind for AlloyDB list_users (#1377)
## Description

---
This pull request introduces a new custom tool kind `alloydb-list-users`
that allows users to list all database users within an AlloyDB cluster.

### Example Configuration

```yaml
tools:
  list_users:
    kind: alloydb-list-users
    source: alloydb-admin-source
    description: Use this tool to list all database users within an AlloyDB cluster
```

### Example Request
``` 
curl -X POST http://127.0.0.1:5000/api/tool/list_users/invoke \
-H "Content-Type: application/json" \
-d '{
    "projectId": "example-project",
    "locationId": "us-central1",
    "clusterId": "example-cluster",
}'
```
## PR Checklist

---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #<issue_number_goes_here>

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-09-12 14:44:21 +05:30
Sri Varshitha
d4a9eb0ce2 feat(tools/alloydb-list-cluster): Add custom tool kind for AlloyDB list_clusters (#1319)
## Description
---
This pull request introduces a new custom tool kind
`alloydb-list-clusters` that lists all AlloyDB clusters in a given
project and location.

### Example Configuration

```yaml
tools:
  list_clusters:
    kind: alloydb-list-clusters
    source: alloydb-admin-source
    description: Use this tool to list all AlloyDB clusters in a given project and location.
```

### Example Request
``` 
curl -X POST http://127.0.0.1:5000/api/tool/list_clusters/invoke \
-H "Content-Type: application/json" \
-d '{
    "projectId": "example-project",
    "locationId": "us-central1"
}'
```

## PR Checklist
---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/langchain-google-alloydb-pg-python/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #<issue_number_goes_here>

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-09-11 14:03:17 -07:00
Pete Hampton
e515d9254f feat(clickhouse): add list-databases tool (#1274)
## Description
---

Adds a tool to the list databases in a clickhouse cluster

<img width="1440" height="813" alt="Screenshot 2025-08-28 at 09 58 15"
src="https://github.com/user-attachments/assets/73643f5d-0c37-4e58-a81c-47bc3a2a5f8e"
/>


## PR Checklist
---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/langchain-google-alloydb-pg-python/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #<issue_number_goes_here>

---------

Co-authored-by: Pete Hampton <pjhampton@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-09-11 15:39:06 +00:00
Haoming Chen
81d239b053 feat(tools/bigquery-analyze-contribution): Add analyze contribution tool (#1223)
This tool creates a contribution analysis model and use ml.get_insights
to get the results.

---------

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-09-10 21:53:56 +00:00
nester-neo4j
f819e26443 feat(prebuiltconfig/neo4j): Add prebuiltconfig support for neo4j (#1352)
This pull request adds support and documentation for connecting Neo4j
databases using the Model Context Protocol (MCP) and the MCP Toolbox. It
introduces a new prebuilt configuration for Neo4j, updates documentation
to guide users through setup across multiple IDEs, and adds tests to
ensure the new configuration is loaded correctly.

**Neo4j MCP Integration:**

* Added a new prebuilt configuration file `neo4j.yaml` defining
`execute_cypher` and `schema` tools for Neo4j, with support for
environment-based configuration.
* Updated the documentation (`neo4j_mcp.md`) with step-by-step
instructions for connecting Neo4j to various IDEs (e.g., VS Code,
Cursor, Claude, Gemini) using MCP Toolbox.
* Expanded the reference documentation to include Neo4j, detailing
required environment variables, permissions, and available tools.

**Testing and Codebase Updates:**

* Updated test cases in `prebuiltconfigs_test.go` to include Neo4j in
the list of expected tool sources and verify the Neo4j configuration can
be fetched and loaded.
[[1]](diffhunk://#diff-fa866059efb09c74bb2aabec8cc71ca9fda3b595fbc2a56f05fb608782ad55b8R36)
[[2]](diffhunk://#diff-fa866059efb09c74bb2aabec8cc71ca9fda3b595fbc2a56f05fb608782ad55b8R102)
[[3]](diffhunk://#diff-fa866059efb09c74bb2aabec8cc71ca9fda3b595fbc2a56f05fb608782ad55b8R148-R150)

---------

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-09-08 23:02:33 -07:00
Pete Hampton
75a04a55dd feat(clickhouse): Add ClickHouse Source and Tools (#1088)
This PR introduces ClickHouse support by finishing off
https://github.com/googleapis/genai-toolbox/pull/713. We have had quite
a few requests internally and externally of people who want ClickHouse
support and wish to contribute but can't use the toolbox because of this
stalled PR. We also tried to reach out to @sidpan1 ourselves with no
success so we are going to assume the contributor doesn't have time to
complete. We would like to thank @sidpan1 for getting this going -
community contributions help make ClickHouse better for everyone.

This PR introduces

- Adds ClickHouse HTTPS and HTTP source support (drops native support
and compression in connection)
- Adds tools: `execute_sql`, ~~`describe_table`, `list_databases`,
`list_tables`~~
  - Adds unit and integration tests
  - Adds docs

Other contributors can build on this foundation

### Connect to local ClickHouse

```sh
CLICKHOUSE_HOST="localhost" \
  CLICKHOUSE_PORT=8123 \
  CLICKHOUSE_USER="default" \
  CLICKHOUSE_PASSWORD="" \
  CLICKHOUSE_DATABASE=default \
  CLICKHOUSE_PROTOCOL="http" \
  go run main.go --prebuilt clickhouse --ui --port 8080
```

### Connect to ClickHouse Cloud

```sh
CLICKHOUSE_HOST="tsmtweovmw.us-east-2.aws.clickhouse.cloud" \
  CLICKHOUSE_PORT=8443 \
  CLICKHOUSE_USER="default" \
  CLICKHOUSE_PASSWORD="[REDACTED]" \
  CLICKHOUSE_DATABASE=default \
  CLICKHOUSE_PROTOCOL=https \
  go run main.go --prebuilt clickhouse --ui --port 8080
```

### Run tests

```bash
go test -v ./tests/clickhouse/ -run TestClickHouse
go test -v ./tests/clickhouse/ -run TestClickHouseBasicConnection
```

<img width="1318" height="895" alt="Screenshot 2025-08-06 at 10 01 01"
src="https://github.com/user-attachments/assets/034d8f1b-10d6-4097-8033-5b0da93ad3fc"
/>

---------

Co-authored-by: Pete Hampton <pjhampton@users.noreply.github.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
Co-authored-by: duwenxin <duwenxin@google.com>
2025-08-27 18:42:10 -04:00
Yuan Teoh
03aa9fabac fix: update env var to allow empty string (#1260)
Update configuration file capability of reading from environment
variable:
* allow default empty string
* fix if statement to check if `parts[2]` is empty instead since it will
always have a length of 4.
* add unit tests.
2025-08-27 01:25:28 +00:00
Huan Chen
7651357d42 feat(tools/bigquery-ask-data-insights): add bigquery ask-data-insights tool (#932)
1. Add ask-data-insights tool based on conversational analytic API.
2. Add tokenSource for ask-data-insights tool, it uses access token
instead of client or restService.
3. Add a max row count to source, currently fixed to 50 and used only
for ask-data-insights tool. Later we may make it available for user to
make change and apply to bigquery-execute-sql and bigquery-sql to avoid
return too many data by accident.

---------

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-08-26 20:16:17 +00:00
trehanshakuntG
00101232a3 feat(tools/firestore-update-document): Add firestore-update-document tool (#1191)
## Add firestore-update-document tool

Adds a new tool for updating existing documents in Firestore
collections.

__What it does:__

- Updates documents at any path in Firestore
- Supports partial updates with field masks for selective field
modification
- Handles all Firestore data types (strings, numbers, booleans,
timestamps, geopoints, arrays, maps, etc.)
- Supports field deletion using updateMask
- Uses Firestore's native JSON format for type safety
- Can update nested fields within maps using dot notation

__Key parameters:__

- `documentPath`: The path of the document to update
- `documentData`: The document content in Firestore JSON format
- `updateMask`: Optional array of field paths for selective updates
- `returnData`: Optional flag to include updated document in response

__Special features:__

- When `updateMask` is provided, only specified fields are updated
- Can access nested fields with dot notation (e.g., 'address.city',
'user.profile.name')
- Without updateMask, performs a merge operation updating all provided
fields
2025-08-22 06:56:23 +00:00
Wenxin Du
212aaba74c feat(server): Fail-fast on environment variable substitution (#1177)
Fail immediately with "environment variable not found" error if an
environment variable ${ENV_VAR} is declared but not defined in a
tools.yaml file.

fix: https://github.com/googleapis/genai-toolbox/issues/1159
2025-08-20 14:19:50 -04:00
trehanshakuntG
ee4a70a0e8 feat(tools/firestore-add-documents): Add firestore-add-documents tool (#1107)
## Add firestore-add-documents tool

Adds a new tool for creating documents in Firestore collections.

__What it does:__

- Adds documents to any Firestore collection
- Auto-generates unique document IDs
- Supports all Firestore data types (strings, numbers, booleans,
timestamps, geopoints, arrays, maps, etc.)
- Uses Firestore's native JSON format for type safety

__Key parameters:__

- `collectionPath`: Where to add the document
- `documentData`: The document content in Firestore JSON format
- `returnData`: Optional flag to include created document in response

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-08-19 11:24:21 +05:30
Haoming Chen
2ad0ccf83d feat(tools/bigquery-forecast): Add bigqueryforecast tool (#1148)
This tool wraps the BigQuery's AI.FORECAST function to do the time
series forecasting.

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-08-18 13:45:54 -07:00
Anuj Jhunjhunwala
d940187c85 feat(tools/dataplex-search-aspect-types): Add support for dataplex-search-aspect-types tool (#1061)
Added support for search aspect types tool in Dataplex.
Fixes #1056

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-08-14 13:34:00 -07:00
prernakakkar-google
f5bcb9c755 feat(prebuiltconfig/alloydb-postgres): add tool to create user via Built in user type or IAM (#1130)
…ilt in or IAM

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-08-13 13:05:06 -07:00
prernakakkar-google
b42c139158 feat(prebuiltconfig/alloydb-admin): Add list cluster, instance and users (#1126)
…er tool

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-08-13 12:28:20 -07:00
Dr. Strangelove
42be3f550c feat(tools/looker): New Looker tools for dashboards (#1118)
* get_dashboards
* make_dashboard
* add_dashboard_element
2025-08-11 17:13:08 -04:00
Dr. Strangelove
61d9489344 feat(tools/looker): add looker-make-look tool to create Looks (#1099) 2025-08-07 12:48:40 -04:00
Anuj Jhunjhunwala
5fa1660fc8 feat(tools/dataplex-lookup-entry): Add support for dataplex-lookup-entry tool (#1009)
Added support for lookup entry tool in Dataplex.
Fixes #997

---------

Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-08-01 15:18:56 -04:00
Dr. Strangelove
327ddf0439 feat: new tool - looker-query-url (#1015) 2025-07-31 17:45:45 +00:00
Anuj Jhunjhunwala
30c16a559e feat: add Dataplex source and tool (#847)
- Users have the preference to choose their clients. Below example is
using Gemini CLI.

- Users can use the pre-built Dataplex tools by creating a settings.json
file under .gemini directory. The contents of settings.json would be as
follows:-

```
{
  "mcpServers": {
    "dataplex": {
      "command": "./toolbox",
      "args": ["--prebuilt","dataplex","--stdio"],
      "env": {
          "DATAPLEX_PROJECT": "test-project"
      }
    }
  }
}
```

Fixes #831

---------

Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
Co-authored-by: Mateusz Nowak <matnow@google.com>
Co-authored-by: Mateusz Nowak <kontakt@mateusznowak.pl>
2025-07-24 15:31:35 -07:00
prernakakkar-google
c600c30374 feat(prebuilt/mysql,prebuilt/mssql): add generic mysql and mssql prebuilt tools (#983)
Co-authored-by: Averi Kitsch <akitsch@google.com>
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-07-24 22:07:07 +05:30
prernakakkar-google
0b28b72aa0 feat: Add alloydb control plane as a prebuilt config (#937) 2025-07-24 00:14:43 +05:30
Dr. Strangelove
c67e01bcf9 feat: Looker MCP Server (#923)
Add support for Looker with the following prebuilt tools:
* get_models
* get_explores
* get_dimensions
* get_measures
* get_filters
* get_parameters
* query
* query_sql
* get_looks
* run_look

---------

Co-authored-by: duwenxin <duwenxin@google.com>
2025-07-23 18:12:06 +00:00
trehanshakuntG
2bb790e4f8 feat: Add Firestore as Source (#786)
Firestore is a NoSQL document database built for automatic scaling, high
performance, and ease of application development. It's a fully managed,
serverless database that supports mobile, web, and server development.

This change adds Firestore as a source in toolbox
2025-07-18 11:42:07 +05:30
AlexTalreja
aa8dbec970 fix: trigger reload on additional fsnotify operations (#854)
Allow fsnotify `CREATE` and `RENAME` events to trigger dynamic reload,
instead of requiring `WRITE`.

This is due to some cases with specific editors/OS, (ex: vim on Mac),
where file writes can potentially occur without ever triggering a
`WRITE` operation on the watched tools file, but solely through creation
and renaming of .swp files.
2025-07-10 16:39:26 -07:00
AlexTalreja
4c240ac3c9 feat: dynamic reloading for toolbox config (#800)
Allow Toolbox server to automatically update when users modify their
tool configuration file(s), instead of requiring a restart.

This feature is automatically enabled, but can be turned off with the
flag `--disable-reload`.
2025-07-08 17:28:12 -07:00
Mohd Mujtaba
40679d700e feat: add support for multiple YAML configuration files (#760)
# Add Multiple YAML Configuration File Support

## 🎯 Overview

This PR introduces support for loading and merging multiple YAML
configuration files in Toolbox, addressing the need for modular
configuration management in complex deployments.

##  New Features

### 1. Multiple Files Support (`--tools-files`)
- **Usage**: `--tools-files=file1.yaml,file2.yaml,file3.yaml`
- Load and intelligently merge multiple YAML configuration files
- Comma-separated file paths for maximum flexibility

### 2. Directory Support (`--tools-folder`)
- **Usage**: `--tools-folder=config-directory`
- Automatically discover and load all `.yaml` and `.yml` files from a
directory
- Simplifies configuration management for organized deployments

### 3. Smart Merging Logic
- **Sources/AuthServices/Tools**: Later files override earlier files
with same names
- **Toolsets**: Tools from same-named toolsets are combined without
duplicates
- Preserves all existing functionality while enabling composition

## 🔒 Safety & Validation

- **Mutual Exclusivity**: Prevents simultaneous use of `--tools-file`,
`--tools-files`, `--tools-folder`, and `--prebuilt`
- **Clear Error Messages**: Descriptive validation errors guide users to
correct usage
- **Comprehensive Error Handling**: Proper handling of missing files,
directories, and parsing errors
- **Full Backward Compatibility**: Existing configurations continue to
work unchanged

## 🏗️ Implementation Details

### Core Functions Added
- `mergeToolsFiles()` - Smart merging with configurable override rules
- `loadAndMergeToolsFiles()` - Multi-file loading and processing
- `loadAndMergeToolsFolder()` - Directory scanning and batch loading

### Command Structure Updates
- New `tools_files []string` field for multiple file paths
- New `tools_folder string` field for directory path
- Enhanced validation logic in `run()` function
- Updated flag definitions with proper descriptions

## 📋 Use Cases

### Organizational Benefits
- **Modular Configuration**: Separate database, API, and auth
configurations
- **Team Collaboration**: Multiple developers can work on different
config files
- **Environment Management**: Easy configuration swapping for different
environments
- **Scalability**: Large configurations can be broken into manageable
chunks

### Example Usage Patterns
```bash
# Multiple specific files
./toolbox --tools-files=database.yaml,apis.yaml,auth.yaml

# Directory-based loading
./toolbox --tools-folder=./production-configs

# Error case (properly handled)
./toolbox --tools-file=single.yaml --tools-folder=configs
# ERROR: --tools-file, --tools-files, and --tools-folder flags cannot be used simultaneously

---------

Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-06-30 14:00:49 -04:00
Wenxin Du
a1b60100c2 chore: Group tools by type (#743)
Group tools of the same type into the same folder so that they are more
discoverable and our tools are more organized as the number grows.
2025-06-30 11:37:48 -04:00
Kurtis Van Gent
cf96f4c249 fix: improve versionString (#658)
Reduces complexity of the version string calculation and fixes the ci/cd
pipeline to pass the correct parameters.

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-06-09 15:31:42 -07:00
Kurtis Van Gent
b4862825e8 refactor: implement dynamic tool registration (#613)
This PR refactors the tool configuration and loading mechanism to use a
dynamic registration pattern. Each tool package now registers itself
with a central registry, and the server configuration code uses this
registry to decode and initialize tools.

Key changes:
- Introduced tools.Register and tools.DecodeToolConfig for dynamic tool
handling.
- Removed direct imports of specific tool packages from
internal/server/config.go.
- Updated individual tool packages to include init() functions for
self-registration.
- Modified ToolKind constants to be local kind constants within each
tool package.
- Adjusted test files to reflect the changes in tool kind identifiers.

This change simplifies adding new tools and decouples the server
configuration from specific tool implementations.

---------

Co-authored-by: Yuan Teoh <yuanteoh@google.com>
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-06-04 10:19:42 -07:00
dishaprakash
a29c80012e feat: Adding support for the --prebuilt flag (#604)
Introduces a new --prebuilt <source_type> flag. This flag is mutually
exclusive with the existing --tools-file flag.

Added a new directory cmd/prebuiltconfigs/ to store the prebuilt
tools.yaml files (e.g., alloydb.yaml, postgres.yaml, etc.).
These YAML files are embedded into the Go binary using the //go:embed
directive.

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-05-28 14:56:42 -07:00
Yuan
1702ce1e00 feat: support MCP stdio transport protocol (#607)
Support MCP
[stdio](https://modelcontextprotocol.io/specification/2024-11-05/basic/transports#stdio)
transport protocol!

To run stdio with Toolbox, user have to use the `--stdio` flag.

Example of running MCP Toolbox with MCP Inspector via stdio transport
protocol: `npx @modelcontextprotocol/inspector ./toolbox --stdio`.

---------

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-05-28 10:10:34 -07:00
Wenxin Du
d9388ad57e feat: Add AuthRequired to Tool Manifest (#433)
Add `AuthRequired` to Tool Manifest so SDK could throw an error early
for unauthorized Tool invocations.
SDK changes:
https://github.com/googleapis/mcp-toolbox-sdk-python/pull/72/files

Also added `authRequired` to Neo4j and dgraph tools.
2025-04-23 12:52:04 -04:00
Wenxin Du
eadb678a7b feat: Support env replacement for tool.yaml (#462)
Environment variable replacement is needed so that users don't have to
hardcode their secrets in configuring `tools.yaml`.
Both formats `$ENV_NAME` and `${ENV_NAME}` are standard ways to declare
an environment variable.
However, some database statement placeholders that are already using the
`$ENV_NAME` format.
Therefore, we only support env var declaration using `${ENV_NAME}` to
disambiguate it from other usages.

Fixes issue: https://github.com/googleapis/genai-toolbox/issues/431
2025-04-23 07:33:02 -04:00
Yuan
34a7263fdc fix: add tools-file flag and deprecate tools_file (#384)
Add `tools-file` flag and deprecate `tools_file` flag. This is not a
breaking change. The old `tools_file` flag is still usable.

User will see the following message when using `tools_file` flag: 
```
Flag --tools_file has been deprecated, please use --tools-file instead
2025-04-03T10:09:12.803165-07:00 INFO "Initialized 2 sources."
```

Help command will reveal the new flag: 
```
...
      --telemetry-service-name string   Sets the value of the service.name resource attribute for telemetry data. (default "toolbox")
      --tools-file string               File path specifying the tool configuration. (default "tools.yaml")
  -v, --version                         version for toolbox
...
```

Fixes #383
2025-04-03 11:09:31 -07:00
Yuan
04cb5fbc3e feat!: deprecate authsource in favor of authservice (#297)
Rename existing `authSource` to `authService` through deprecation.
`AuthService` more clearly distinguishes it from `Sources` objects.

`authSources` will be converted into `authServices` after the
unmarshalling process. A warning log is shown if `authSources` are used
(for both within tools parameters and defining auth services):
```
2025-02-20T13:57:51.156025-08:00 WARN "`authSources` is deprecated, use `authServices` for parameters instead"
2025-02-20T13:57:51.156569-08:00 WARN "`authSources` is deprecated, use `authServices` instead"
2025-02-20T13:57:52.047584-08:00 INFO "Initialized 1 sources."
...
```

The manifest generated will continue to use `authSources` to keep
compatibility with the sdks:
```
{
"serverVersion":"0.1.0",
"tools":{
  "test_tool2":{
    "description":"Use this tool to test\n",
    "parameters":[{
      "name":"user_id",
      "type":"string",
      "description":"Auto-populated from Google login",
      "authSources":["my-google-auth"]
      }]
    }
  }
}
```



Test cases with `authSources` are kept for compatibility. Will be
removed when `authSources` are no longer supported.
2025-02-21 18:36:04 -08:00
Yuan
95d29f9e8a chore: pass logger via context through unmarshaller (#300)
Passing logger via context. With this, we will be able to add logs
during the unmarshaling process.
2025-02-21 13:02:23 -08:00
Yuan
ad58cd5855 fix: update tool invoke to return json (#266)
Return actual rows as `[]any` that contains `map` of results. Each `map`
represent a row, with the key being column name.
2025-02-05 13:45:01 -08:00
Yuan
a0ac5334d1 chore: return error for untested fields in tools.yaml (#239)
This only checks within `SourceConfig`, `ToolConfig`, and
`AuthSourceConfig`.

Error when an unknown field is provided:
`2025-01-27T22:43:46.988401-08:00 ERROR "unable to parse tool file at
\"tools.yaml\": unable to parse as \"cloud-sql-postgres\": [2:1] unknown
field \"extra\"\n 1 | database: test_database\n> 2 | extra: here\n ^\n 3
| instance: toolbox-cloudsql\n 4 | kind: cloud-sql-postgres\n 5 |
password: postgres\n 6 | "`

Error when a required field is not provided:
`2025-01-27T17:49:47.584846-08:00 ERROR "unable to parse tool file at
\"tools.yaml\": validation failed: Key: 'Config.Region' Error:Field
validation for 'Region' failed on the 'required' tag"`

---------

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
2025-02-03 15:30:27 -08:00
Yuan
1fcc20a846 feat: add support for OpenTelemetry (#205)
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-01-13 15:38:38 -08:00
Wenxin Du
abd1eb702c fix: Set server version (#150) 2024-12-18 17:15:55 -05:00
Wenxin Du
3118104ae1 feat: add auth_required to tools (#123)
Check if a tool invocation request contains required auth token.
2024-12-16 22:41:13 -05:00
Wenxin Du
380a6fbbd5 feat: Add Tool authenticated parameters (#80)
1. Add []ParamAuthSource to every Parameter type implementation to
support authenticated configs. Create new constructors for types with
auth.
2. Tool invocation API changes to parse auth header and authentecated
parameters.
3. Add authSources to Tool manifest.
2024-12-16 17:46:06 -05:00
Wenxin Du
77b0d43175 feat: Add Auth Source configuration (#71)
1. Add authentication source configs to Toolbox server.
2. Add Google auth implementation.
2024-12-13 12:22:42 -05:00
Yuan
6479c1dbe2 feat(source/cloudsql-pg): add configuration for public and private IP (#114)
Allow user to set if their database uses private or public ip. The
reason we add this is because the dialer require different
initialization with private and public ip.

By default, toolbox will use public ip.
2024-12-06 00:08:15 +00:00
Yuan
9a0f618efc feat: add --log-level and --logging-format flags (#97)
Logging support 4 different types of logging (debug, info, warn, error).
The default logging level is Info.

User will be able to set flag for log level (allowed values: "debug",
"info", "warn", "error"), example:
`go run . --log-level debug`

User will be able to set flag for logging format (allowed values:
"standard", "JSON"), example:
`go run . --logging-format json`

**sample http request log - std:**
server
```
2024-11-12T15:08:11.451377-08:00 INFO "Initalized 0 sources.\n"
```
httplog
```
2024-11-26T15:15:53.947287-08:00 INFO Response: 200 OK service: "httplog" httpRequest: {url: "http://127.0.0.1:5000/" method: "GET" path: "/" remoteIP: "127.0.0.1:64216" proto: "HTTP/1.1" requestID: "macbookpro.roam.interna/..." scheme: "http" header: {user-agent: "curl/8.7.1" accept: "*/*"}} httpResponse: {status: 200 bytes: 22 elapsed: 0.012417}
```

**sample http request log - structured:**
server
```
{
  "timestamp":"2024-11-04T16:45:11.987299-08:00",
  "severity":"ERROR",
  "logging.googleapis.com/sourceLocation":{
    "function":"github.com/googleapis/genai-toolbox/internal/log.(*StructuredLogger).Errorf",
    "file":"/Users/yuanteoh/github/genai-toolbox/internal/log/log.go","line":157
  },
  "message":"unable to parse tool file at \"tools.yaml\": \"cloud-sql-postgres1\" is not a valid kind of data source"
}
```
httplog
```
{
  "timestamp":"2024-11-26T15:12:49.290974-08:00",
  "severity":"INFO",
  "logging.googleapis.com/sourceLocation":{
      "function":"github.com/go-chi/httplog/v2.(*RequestLoggerEntry).Write",
      "file":"/Users/yuanteoh/go/pkg/mod/github.com/go-chi/httplog/v2@v2.1.1/httplog.go","line":173
  },
  "message":"Response: 200 OK",
  "service":"httplog",
  "httpRequest":{
      "url":"http://127.0.0.1:5000/",
      "method":"GET",
      "path":"/",
      "remoteIP":"127.0.0.1:64140",
      "proto":"HTTP/1.1",
      "requestID":"yuanteoh-macbookpro.roam.internal/NBrtYBu3q9-000001",
      "scheme":"http",
      "header":{"user-agent":"curl/8.7.1","accept":"*/*"}
  },
  "httpResponse":{"status":200,"bytes":22,"elapsed":0.0115}
}
```
2024-12-03 16:52:19 +00:00