Add `templateParameters` to support non-filter parameters and DDL
statements.
Added a new argument `ignoreDdl` at integration test. Admin client is
needed to execute ddl statement in spanner. Toolbox does not use admin
client.
Part of #535
Add new tool field, templateParameters, to support non-filter parameters
and DDL statements
Fix#535 for postgressql tool.
---------
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
This PR refactors the tool configuration and loading mechanism to use a
dynamic registration pattern. Each tool package now registers itself
with a central registry, and the server configuration code uses this
registry to decode and initialize tools.
Key changes:
- Introduced tools.Register and tools.DecodeToolConfig for dynamic tool
handling.
- Removed direct imports of specific tool packages from
internal/server/config.go.
- Updated individual tool packages to include init() functions for
self-registration.
- Modified ToolKind constants to be local kind constants within each
tool package.
- Adjusted test files to reflect the changes in tool kind identifiers.
This change simplifies adding new tools and decouples the server
configuration from specific tool implementations.
---------
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
@Yuan325 please review and merge this if all looks ok.
In existing code, for the character field with null values it will panic
with error:
"panic: interface conversion: interface {} is nil, not []uint8"
gosimple had been deprecated in favor of staticcheck:
https://github.com/golangci/golangci-lint/issues/357
Other requirements are all migrated.
`std-error-handling` exclusions is included because without that, it
will ask to check all error returns from (`Close()`, or `os.Setenv`s, or
`fmt.Fprint`s...
This tool can be used across spanner sources.
`spanner-execute-sql` config is as below:
```
tools:
spanner_execute_sql_tool:
kind: "spanner-execute-sql"
source: my-spanner-source
description: Use this tool to execute sql.
```
The `spanner-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/spanner_execute_sql_tool/invoke
```
---------
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
This tool can be used across mysql sources.
`mysql-execute-sql` config is as below:
```
tools:
mysql_execute_sql_tool:
kind: "mysql-execute-sql"
source: my-mysql-source
description: Use this tool to execute sql.
```
The `mysql-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/mysql_execute_sql_tool/invoke
```
Allowing user to add `readOnly` field in spanner tools.
The existing tool doesn't work for reading schema tables since schema
tables can only be accessed through read-only transaction.
This PR also resolve#435 for Spanner tool.
---------
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
This tool can be used across all postgres sources.
`postgres-execute-sql` config is as below:
```
tools:
postgres_execute_sql_tool:
kind: "postgres-execute-sql"
source: my-alloydb-source // or any other sources that is compatible with this tool
description: Use this tool to execute sql.
```
The `postgres-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/postgres_execute_sql_tool/invoke
```
Update http tools `Invoke` method.
* use `json.Unmarshal` for response body that are either list or json
map. This will allow us to return lists as is (e.g. `[{map}]` instead of
having to nest it in an additional list (e.g. `[]any{ [{map}] }`)
* for response body that are strings or fail to unmarshal, return it as
is.
Environment variable replacement is needed so that users don't have to
hardcode their secrets in configuring `tools.yaml`.
Both formats `$ENV_NAME` and `${ENV_NAME}` are standard ways to declare
an environment variable.
However, some database statement placeholders that are already using the
`$ENV_NAME` format.
Therefore, we only support env var declaration using `${ENV_NAME}` to
disambiguate it from other usages.
Fixes issue: https://github.com/googleapis/genai-toolbox/issues/431
A `BigQuery` source can be added as the following example:
```yaml
sources:
my-bigquery-source:
kind: bigquery
project: bigframes-dev
location: us # This field is optional
```
A `BigQuery` tool can be added as below:
```yaml
tools:
search-hotels-by-name:
kind: bigquery-sql
source: my-bigquery-source
description: Search for hotels based on name.
parameters:
- name: name
type: string
description: The name of the hotel.
```
---------
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
# Add Bigtable support
A `bigtable` source can be added as the following example
```
sources:
test-bigtable-source:
kind: "bigtable"
project: "sample-project"
instance: "sample-instance"
```
A `bigtable` tool can be added as below
```
tools:
get-test-tool-data:
kind: bigtable-sql
source: test-bigtable-source
description: Some description
statement: SELECT * FROM `test-table` WHERE address['state'] = @state;
parameters:
- name: state
type: string
description: Filter by state
```
---------
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
This PR is to remove the `NewGenericTool` function within tools.
This function was previously added when we are trying to consolidate the
three postgres tools. However, since we merged it into a single
`postgressql` tool, these functions are not used anymore.
Rename existing `authSource` to `authService` through deprecation.
`AuthService` more clearly distinguishes it from `Sources` objects.
`authSources` will be converted into `authServices` after the
unmarshalling process. A warning log is shown if `authSources` are used
(for both within tools parameters and defining auth services):
```
2025-02-20T13:57:51.156025-08:00 WARN "`authSources` is deprecated, use `authServices` for parameters instead"
2025-02-20T13:57:51.156569-08:00 WARN "`authSources` is deprecated, use `authServices` instead"
2025-02-20T13:57:52.047584-08:00 INFO "Initialized 1 sources."
...
```
The manifest generated will continue to use `authSources` to keep
compatibility with the sdks:
```
{
"serverVersion":"0.1.0",
"tools":{
"test_tool2":{
"description":"Use this tool to test\n",
"parameters":[{
"name":"user_id",
"type":"string",
"description":"Auto-populated from Google login",
"authSources":["my-google-auth"]
}]
}
}
}
```
Test cases with `authSources` are kept for compatibility. Will be
removed when `authSources` are no longer supported.
Check common parameters for required fields. Throw an error if name or
description is not provided.
Example of error when `description` field not provided:
```
2025-02-18T14:11:55.475101-08:00 ERROR "unable to parse tool file at \"tools.yaml\": unable to parse as \"postgres-sql\": unable to parse as \"array\": Key: 'CommonParameter.Desc' Error:Field validation for 'Desc' failed on the 'required' tag"
```
Fixes#291 by throwing a more descriptive error when items field is not provided.
Example of error when `items` field not provided for array parameter:
```
2025-02-18T14:07:58.77173-08:00 ERROR "unable to parse tool file at \"tools.yaml\": unable to parse as \"postgres-sql\": unable to parse as \"array\": unable to parse 'items' field: error parsing parameters: nothing to unmarshal"
```
`items` are not specified in the manifest for parameters, which means
the client doesn't parse them correctly when presented to the LLMs. This
PR adds `items` as part of manifest.
Example manifest:
```
{
"serverVersion": "0.1.0",
"tools": {
"tool_name": {
"description": "description of the tool...",
"parameters": [{
"name": "foos",
"type": "array",
"description": "A list of foo.",
"authSources": [],
"items": {"name": "foo", "type": "string", "description": "foobar.", "authSources": []}
}]
}
}
}
```
Fixes#292🦕
This only checks within `SourceConfig`, `ToolConfig`, and
`AuthSourceConfig`.
Error when an unknown field is provided:
`2025-01-27T22:43:46.988401-08:00 ERROR "unable to parse tool file at
\"tools.yaml\": unable to parse as \"cloud-sql-postgres\": [2:1] unknown
field \"extra\"\n 1 | database: test_database\n> 2 | extra: here\n ^\n 3
| instance: toolbox-cloudsql\n 4 | kind: cloud-sql-postgres\n 5 |
password: postgres\n 6 | "`
Error when a required field is not provided:
`2025-01-27T17:49:47.584846-08:00 ERROR "unable to parse tool file at
\"tools.yaml\": validation failed: Key: 'Config.Region' Error:Field
validation for 'Region' failed on the 'required' tag"`
---------
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
