Commit Graph

139 Commits

Author SHA1 Message Date
Yuan
fafed24858 fix: set default value to field's type during unmarshalling (#774)
When go-yaml decode into CommonParameter with Default being an any type,
int will be converted into []uint64.
It will fail the Parse() when the value is being used since it does not
belong to either of the int types.

Unmarshal `default` value into each field's type directly.

Fixes #771
2025-07-02 14:58:42 +00:00
Wenxin Du
a1b60100c2 chore: Group tools by type (#743)
Group tools of the same type into the same folder so that they are more
discoverable and our tools are more organized as the number grows.
2025-06-30 11:37:48 -04:00
Yuan
cbb4a33351 fix(tools/mysqlsql): Handle nil panic and connection leak in invoke (#758)
Copy fix from #757. 

The Invoke function had two bugs:

A panic would occur when scanning a row containing a NULL value in a
TEXT or VARCHAR column. The code did not check for nil before attempting
a type assertion on the scanned value.

The *sql.Rows result was not being closed on all code paths, leading to
connection leaks that could exhaust the database connection pool.

This change corrects both issues. A guard clause now checks for nil
values before processing, and rows.Close() is deferred to guarantee the
connection is released.
2025-06-26 20:54:59 +00:00
megatron0000
7badba42ee fix(tools/mysqlexecutesql): Handle nil panic and connection leak in Invoke (#757)
The Invoke function had two bugs:

1. A panic would occur when scanning a row containing a NULL value in a
TEXT or VARCHAR column. The code did not check for nil before attempting
a type assertion on the scanned value.

2. The *sql.Rows result was not being closed on all code paths, leading
to connection leaks that could exhaust the database connection pool.

This change corrects both issues. A guard clause now checks for nil
values before processing, and rows.Close() is deferred to guarantee the
connection is released.

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-06-26 13:12:45 -07:00
Wenxin Du
7a6644cf0c fix(bigquery,mssql): fix panic on tools with array param (#722)
Fix: https://github.com/googleapis/genai-toolbox/issues/701

Things done:
1. Replace the `AsReversedMap()` helper with `AsMap()`
2. BigQuery's QueryParameter only accepts typed slices as input, but our
arrays are passed in as []any. Therefore, add a logic to convert []any
to a typed array based on the item type.

Tested on MCP inspector:
<img width="409" alt="Screenshot 2025-06-16 at 5 15 55 PM"
src="https://github.com/user-attachments/assets/8053cad5-270e-4d82-b97c-856238c42154"
/>

---------

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
2025-06-25 22:54:26 -04:00
Yuan
474df57d62 feat: support MCP version 2025-03-26 (#755)
This feature includes the following:
* Implement initialize lifecycle (including version negotiation)
* Add the v20250326 schema
* Supporting the `DELETE` and `GET` endpoint for MCP.
* Supporting streamable HTTP (without SSE).
* Terminating sessions after timeout (default = 10 minutes from last
active).
* Toolbox do not support batch request. Will response with `Invalid
requests` if batch requests is received.
2025-06-26 00:34:37 +00:00
Wenxin Du
fd300dc606 feat: Add support for HTTP Tool pathParams (#726)
Allow users to specify dynamic path for HTTP tools.

fix: https://github.com/googleapis/genai-toolbox/issues/680
2025-06-20 14:53:49 -04:00
Yuan
4827771b78 feat: add support for optional parameters (#617)
Add a `default` field to parameters, that enables users to specify a
default value.

e.g.

```
  parameters:
    - name: name
      type: string
      default: "some-default-value"
      description: The name of the hotel.
```
if this parameter is invoked without specifying `name`, the parameter
would default to "some-default-value"


For parameter manifest, there will be an additional `Required` field.
The default `Required` field is true. If a `default` value is presented,
`Required: false`. Array parameter's item's `Required` field will
inherit the array's `Required` field.

Fixes #475
2025-06-20 10:46:59 -07:00
Yuan
67964d939f fix(postgres,mssql,cloudsqlmssql)!: encode source connection url for sources (#727)
Have to encode special character in connection url. Only needed for
`postgres`, `mssql`, `cloud-sql-mssql` sources.

Fixes #717
2025-06-18 15:32:46 -07:00
AlexTalreja
9197186b8b feat(tools/couchbase): add templateParameters field for couchbase (#723)
Add templateParameters to support non-filter parameters and DDL
statements.

Part of https://github.com/googleapis/genai-toolbox/issues/535
2025-06-18 21:27:41 +00:00
Yuan
f5f771b0f3 feat(tools/bigquery): add templateParameters field for bigquery (#699)
Add templateParameters to support non-filter parameters and DDL
statements.

Part of #535
2025-06-18 10:19:23 -07:00
Wenxin Du
4055b0c356 feat: Support disable SSL verification for HTTP Source (#674)
Fix issue: https://github.com/googleapis/genai-toolbox/issues/661
2025-06-16 17:22:09 -04:00
Yuan
1c067715fa feat(tools/bigtable): add templateParameters field for bigtable (#692)
Add templateParameters to support non-filter parameters and DDL
statements.

Added a new argument `ignoreInsert` at integration test. Bigtable only
allow `SELECT` statement. This is used to filter insert statement for
bigtable.

Part of #535
2025-06-12 21:18:32 +00:00
Wenxin Du
054ec198b9 feat: Add Valkey Source and Tool (#532) 2025-06-11 23:17:32 -04:00
Wenxin Du
f0aef29b0c feat: Add Redis Source and Tool (#519)
1. Added Redis Source and Tool
2. Moved some integration test helpers from tools.go to common.go
3. Make auth integration test want an input variable
2025-06-11 22:47:27 -04:00
Yuan
075dfa47e1 feat(tools/spanner): add templateParameters field for spanner (#691)
Add `templateParameters` to support non-filter parameters and DDL
statements.

Added a new argument `ignoreDdl` at integration test. Admin client is
needed to execute ddl statement in spanner. Toolbox does not use admin
client.

Part of #535
2025-06-11 23:54:13 +00:00
AlexTalreja
75e254c0a4 feat(tools/sqlitesql): add templateParameters field for sqlitesql (#687)
Add templateParameters to support non-filter parameters and DDL
statements.

Part of https://github.com/googleapis/genai-toolbox/issues/535
2025-06-10 13:08:52 -07:00
Yuan
4998f82852 fix: update path library (#678)
Using filepath.Join for embed filenames will not respect Windows
filepaths. https://github.com/golang/go/issues/44305

Fix #662
2025-06-09 10:00:26 -07:00
Yuan
b81fc6aa6c feat: add templateParameters field for mssqlsql (#671)
Add `templateParameters to support non-filter parameters and DDL
statements.

Part of https://github.com/googleapis/genai-toolbox/issues/535
2025-06-06 13:18:37 -07:00
Yuan
0a08d2c15d feat: add templateParameters field for mysqlsql (#663)
Add `templateParameters to support non-filter parameters and DDL
statements.

Part of #535
2025-06-05 21:37:30 +00:00
Yuan
518a0e4c70 refactor(tools/http): dedup to use GetParams (#659) 2025-06-05 10:54:08 -07:00
Yuan
d7ba2736eb refactor: concatenate template parameters and parameters (#660)
Refactor this to dedup from other tools.
2025-06-05 09:01:45 -07:00
Kurtis Van Gent
1c9ad5ea24 refactor: implement dynamic source registration (#614)
This commit refactors the source configuration and loading mechanism to
use a dynamic registration pattern. Each source package now registers
itself with a central registry via its init() function.

The server configuration code uses this registry to decode and
initialize sources, decoupling it from specific source implementations
and simplifying the addition of new sources.

Key changes:
- Introduced `sources.Register()` and `newConfig()` constructor in each
source package.
- Moved source package imports to `cmd/root.go` as blank imports to
trigger `init()` functions for self-registration.
- Removed direct imports of specific source packages from
`internal/server/config.go`.
- Renamed `SourceKind` constants to `Kind` within each source package.
- Updated tests to use the new `Kind` constants and reflect registration
changes.

---------

Co-authored-by: Yuan Teoh <yuanteoh@google.com>
2025-06-04 14:23:57 -07:00
AlexTalreja
b76346993f feat(tools/postgressql): add templateParameters field (#615)
Add new tool field, templateParameters, to support non-filter parameters 
and DDL statements

Fix #535 for postgressql tool.

---------

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-06-04 13:28:37 -07:00
Kurtis Van Gent
b4862825e8 refactor: implement dynamic tool registration (#613)
This PR refactors the tool configuration and loading mechanism to use a
dynamic registration pattern. Each tool package now registers itself
with a central registry, and the server configuration code uses this
registry to decode and initialize tools.

Key changes:
- Introduced tools.Register and tools.DecodeToolConfig for dynamic tool
handling.
- Removed direct imports of specific tool packages from
internal/server/config.go.
- Updated individual tool packages to include init() functions for
self-registration.
- Modified ToolKind constants to be local kind constants within each
tool package.
- Adjusted test files to reflect the changes in tool kind identifiers.

This change simplifies adding new tools and decouples the server
configuration from specific tool implementations.

---------

Co-authored-by: Yuan Teoh <yuanteoh@google.com>
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-06-04 10:19:42 -07:00
Mohd Mujtaba
ef94648455 fix(tools/mysqlsql): handled the null value for string case in mysqlsql tools (#641)
@Yuan325  please review and merge this if all looks ok.
In existing code, for the character field with null values it will panic
with error:
"panic: interface conversion: interface {} is nil, not []uint8"
2025-06-03 11:33:41 -07:00
Yuan
69d047af46 fix(server/stdio): notifications should not return a response (#638)
Currently the `stdio` transport protocol will throw a `ZodError` during
initialization. This is due to Toolbox writing `null` to stdout when it
received a notification. This is not expected hence the `ZodError`
occurs. Per the MCP protocol, notifications do not expect any response.

This fix added a condition to check if the responses is `nil` before
writing to stdout.
2025-06-02 16:59:39 -07:00
Yuan
ba8a6f3a3b chore: migrate golangci-lint to v2 (#630)
gosimple had been deprecated in favor of staticcheck:
https://github.com/golangci/golangci-lint/issues/357

Other requirements are all migrated.

`std-error-handling` exclusions is included because without that, it
will ask to check all error returns from (`Close()`, or `os.Setenv`s, or
`fmt.Fprint`s...
2025-05-31 02:50:17 +00:00
Averi Kitsch
3d10f85302 docs: add tool list to dev assist docs (#626)
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-05-29 22:52:35 +00:00
dishaprakash
a29c80012e feat: Adding support for the --prebuilt flag (#604)
Introduces a new --prebuilt <source_type> flag. This flag is mutually
exclusive with the existing --tools-file flag.

Added a new directory cmd/prebuiltconfigs/ to store the prebuilt
tools.yaml files (e.g., alloydb.yaml, postgres.yaml, etc.).
These YAML files are embedded into the Go binary using the //go:embed
directive.

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-05-28 14:56:42 -07:00
Huan Chen
0fd88b574b feat: add new bigquery tools. (#619)
Added five new tools and corresponding documents:
1. bigquery-execute-sql
2. bigquery-list-dataset-ids
3. bigquery-list-table-ids
4. bigquery-get-dataset-info
5.  bigquery-get-table-info

---------

Co-authored-by: duwenxin <duwenxin@google.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-05-28 14:38:08 -07:00
Yuan
1702ce1e00 feat: support MCP stdio transport protocol (#607)
Support MCP
[stdio](https://modelcontextprotocol.io/specification/2024-11-05/basic/transports#stdio)
transport protocol!

To run stdio with Toolbox, user have to use the `--stdio` flag.

Example of running MCP Toolbox with MCP Inspector via stdio transport
protocol: `npx @modelcontextprotocol/inspector ./toolbox --stdio`.

---------

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-05-28 10:10:34 -07:00
prernakakkar-google
6083a224aa Feat: Add Execute sql tool for SQL Server(MSSQL) (#585)
This tool can be used across mssql(SQL Server) sources.

`mssql-execute-sql` config is as below:

```
tools:
  mssql_execute_sql_tool:
     kind: "mssql-execute-sql"
     source: my-mssql-source
     description: Use this tool to execute sql.
```

The `mssql-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/mssql_execute_sql_tool/invoke
```

Reference for bug: b/416163913

---------

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Anubhav Dhawan <anubhavdhawan@google.com>
Co-authored-by: Huan Chen <142538604+Genesis929@users.noreply.github.com>
2025-05-22 00:10:55 +05:30
Huan Chen
eb52b66d82 fix: explicitly set query location for BigQuery queries (#586)
For issue https://github.com/googleapis/genai-toolbox/issues/567

---------

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-05-20 22:07:43 +00:00
Yuan
4b4fbc656a chore(tools/mysqlsql): only cast TEXT to string (#581)
Only cast to string if the column's `DatabaseTypeName` is `TEXT`.
2025-05-19 14:58:28 -07:00
Yuan
4d4b3ebeb9 chore: lint fix (#582) 2025-05-19 14:53:25 -07:00
trehanshakuntG
d65747a2dc feat: add spanner-execute-sql tool (#576)
This tool can be used across spanner sources.

`spanner-execute-sql` config is as below:

```
tools:
  spanner_execute_sql_tool:
    kind: "spanner-execute-sql"
    source: my-spanner-source
    description: Use this tool to execute sql.
```

The `spanner-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/spanner_execute_sql_tool/invoke
```

---------

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-05-19 14:48:11 -07:00
trehanshakuntG
8590061ae4 feat: add mysql-execute-sql tool (#577)
This tool can be used across mysql sources.

`mysql-execute-sql` config is as below:

```
tools:
  mysql_execute_sql_tool:
    kind: "mysql-execute-sql"
    source: my-mysql-source
    description: Use this tool to execute sql.
```

The `mysql-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/mysql_execute_sql_tool/invoke
```
2025-05-20 02:17:39 +05:30
trehanshakuntG
6512704e77 feat: Add support for read-only in Spanner tool (#563)
Allowing user to add `readOnly` field in spanner tools.

The existing tool doesn't work for reading schema tables since schema
tables can only be accessed through read-only transaction.

This PR also resolve #435 for Spanner tool.

---------

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-05-16 12:08:05 +05:30
Yuan
04dcf47912 fix: reinitialize required slice if nil (#571)
`slices.Concat` will return `nil` if arrays are all empty instead of an
empty slice `[]`. Fix this by setting `[]` if it's `nil`.

Fixes #564
2025-05-16 00:54:03 +00:00
Yuan
b58bf76dda fix: fix spellings in comments (#561) 2025-05-13 21:09:37 +00:00
Wenxin Du
e747b6e289 fix: prevent tool calls through MCP when auth is required (#544)
MCP does not support the `authRequired` feature. Disallow all MCP Tool
call to Tools with `authRequired` set.

Fixes: https://github.com/googleapis/genai-toolbox/issues/543
2025-05-07 15:24:13 -04:00
shyam-cb
d7390b06b7 feat: add Couchbase as Source and Tool (#307)
Added couchbase support to Genai Toolbox

---------

Co-authored-by: duwenxin <duwenxin@google.com>
2025-05-02 16:37:58 -04:00
Yuan
11ea7bc584 feat: add postgres-execute-sql tool (#490)
This tool can be used across all postgres sources.

`postgres-execute-sql` config is as below:

```
tools:
  postgres_execute_sql_tool:
    kind: "postgres-execute-sql"
    source: my-alloydb-source // or any other sources that is compatible with this tool
    description: Use this tool to execute sql.
```

The `postgres-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/postgres_execute_sql_tool/invoke
```
2025-05-01 17:43:41 +00:00
Yuan
b6cd99e859 chore(http): update invocation results formatting (#502)
Update http tools `Invoke` method.
* use `json.Unmarshal` for response body that are either list or json
map. This will allow us to return lists as is (e.g. `[{map}]` instead of
having to nest it in an additional list (e.g. `[]any{ [{map}] }`)
* for response body that are strings or fail to unmarshal, return it as
is.
2025-04-28 21:09:07 -07:00
Wenxin Du
d9388ad57e feat: Add AuthRequired to Tool Manifest (#433)
Add `AuthRequired` to Tool Manifest so SDK could throw an error early
for unauthorized Tool invocations.
SDK changes:
https://github.com/googleapis/mcp-toolbox-sdk-python/pull/72/files

Also added `authRequired` to Neo4j and dgraph tools.
2025-04-23 12:52:04 -04:00
Wenxin Du
0c0d7b8637 chore: Add user agent to Bigtable and BigQuery (#473)
Add user agent for metrics tracing.
2025-04-23 10:20:24 -04:00
Wenxin Du
eadb678a7b feat: Support env replacement for tool.yaml (#462)
Environment variable replacement is needed so that users don't have to
hardcode their secrets in configuring `tools.yaml`.
Both formats `$ENV_NAME` and `${ENV_NAME}` are standard ways to declare
an environment variable.
However, some database statement placeholders that are already using the
`$ENV_NAME` format.
Therefore, we only support env var declaration using `${ENV_NAME}` to
disambiguate it from other usages.

Fixes issue: https://github.com/googleapis/genai-toolbox/issues/431
2025-04-23 07:33:02 -04:00
waqarahmed6095
fc14cbfd07 feat: sqlite implementation (#438)
Adding sqlite implementation

---------

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
2025-04-22 22:25:53 -07:00
Huan Chen
8055aa519f feat: Add BigQuery source and tool (#463)
A `BigQuery` source can be added as the following example:

```yaml
sources:
  my-bigquery-source:
    kind: bigquery
    project: bigframes-dev
    location: us # This field is optional
```

A `BigQuery` tool can be added as below:
```yaml
tools:
  search-hotels-by-name:
    kind: bigquery-sql
    source: my-bigquery-source
    description: Search for hotels based on name.
    parameters:
      - name: name
        type: string
        description: The name of the hotel.
```

---------

Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-04-22 20:37:38 -06:00