Compare commits

...

18 Commits

Author SHA1 Message Date
Mend Renovate
08397cf398 chore(deps): update module cloud.google.com/go/cloudsqlconn to v1.18.1 (#1268)
This PR contains the following updates:

| Package | Change | Age | Confidence |
|---|---|---|---|
|
[cloud.google.com/go/cloudsqlconn](https://redirect.github.com/googlecloudplatform/cloud-sql-go-connector)
| `v1.18.0` -> `v1.18.1` |
[![age](https://developer.mend.io/api/mc/badges/age/go/cloud.google.com%2fgo%2fcloudsqlconn/v1.18.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/go/cloud.google.com%2fgo%2fcloudsqlconn/v1.18.0/v1.18.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>googlecloudplatform/cloud-sql-go-connector
(cloud.google.com/go/cloudsqlconn)</summary>

###
[`v1.18.1`](https://redirect.github.com/GoogleCloudPlatform/cloud-sql-go-connector/releases/tag/v1.18.1)

[Compare
Source](https://redirect.github.com/googlecloudplatform/cloud-sql-go-connector/compare/v1.18.0...v1.18.1)

##### Bug Fixes

- Use a new context for the domain name check loop.
([#&#8203;1007](https://redirect.github.com/GoogleCloudPlatform/cloud-sql-go-connector/issues/1007))
([908d0cf](908d0cf6a6))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/googleapis/genai-toolbox).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS44Mi43IiwidXBkYXRlZEluVmVyIjoiNDEuODIuNyIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
2025-08-27 19:07:20 -07:00
Pete Hampton
75a04a55dd feat(clickhouse): Add ClickHouse Source and Tools (#1088)
This PR introduces ClickHouse support by finishing off
https://github.com/googleapis/genai-toolbox/pull/713. We have had quite
a few requests internally and externally of people who want ClickHouse
support and wish to contribute but can't use the toolbox because of this
stalled PR. We also tried to reach out to @sidpan1 ourselves with no
success so we are going to assume the contributor doesn't have time to
complete. We would like to thank @sidpan1 for getting this going -
community contributions help make ClickHouse better for everyone.

This PR introduces

- Adds ClickHouse HTTPS and HTTP source support (drops native support
and compression in connection)
- Adds tools: `execute_sql`, ~~`describe_table`, `list_databases`,
`list_tables`~~
  - Adds unit and integration tests
  - Adds docs

Other contributors can build on this foundation

### Connect to local ClickHouse

```sh
CLICKHOUSE_HOST="localhost" \
  CLICKHOUSE_PORT=8123 \
  CLICKHOUSE_USER="default" \
  CLICKHOUSE_PASSWORD="" \
  CLICKHOUSE_DATABASE=default \
  CLICKHOUSE_PROTOCOL="http" \
  go run main.go --prebuilt clickhouse --ui --port 8080
```

### Connect to ClickHouse Cloud

```sh
CLICKHOUSE_HOST="tsmtweovmw.us-east-2.aws.clickhouse.cloud" \
  CLICKHOUSE_PORT=8443 \
  CLICKHOUSE_USER="default" \
  CLICKHOUSE_PASSWORD="[REDACTED]" \
  CLICKHOUSE_DATABASE=default \
  CLICKHOUSE_PROTOCOL=https \
  go run main.go --prebuilt clickhouse --ui --port 8080
```

### Run tests

```bash
go test -v ./tests/clickhouse/ -run TestClickHouse
go test -v ./tests/clickhouse/ -run TestClickHouseBasicConnection
```

<img width="1318" height="895" alt="Screenshot 2025-08-06 at 10 01 01"
src="https://github.com/user-attachments/assets/034d8f1b-10d6-4097-8033-5b0da93ad3fc"
/>

---------

Co-authored-by: Pete Hampton <pjhampton@users.noreply.github.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
Co-authored-by: duwenxin <duwenxin@google.com>
2025-08-27 18:42:10 -04:00
Mend Renovate
a5e74af104 chore(deps): update module github.com/go-chi/chi/v5 to v5.2.3 (#1265)
This PR contains the following updates:

| Package | Change | Age | Confidence |
|---|---|---|---|
| [github.com/go-chi/chi/v5](https://redirect.github.com/go-chi/chi) |
`v5.2.2` -> `v5.2.3` |
[![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fgo-chi%2fchi%2fv5/v5.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fgo-chi%2fchi%2fv5/v5.2.2/v5.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>go-chi/chi (github.com/go-chi/chi/v5)</summary>

###
[`v5.2.3`](https://redirect.github.com/go-chi/chi/releases/tag/v5.2.3)

[Compare
Source](https://redirect.github.com/go-chi/chi/compare/v5.2.2...v5.2.3)

#### What's Changed

- Add pathvalue example to README and implement PathValue handler. by
[@&#8203;catatsuy](https://redirect.github.com/catatsuy) in
[#&#8203;985](https://redirect.github.com/go-chi/chi/pull/985)
- Allow multiple whitespace between method & pattern by
[@&#8203;JRaspass](https://redirect.github.com/JRaspass) in
[#&#8203;1013](https://redirect.github.com/go-chi/chi/pull/1013)
- Avoid potential nil dereference by
[@&#8203;ProjectMutilation](https://redirect.github.com/ProjectMutilation)
in [#&#8203;1008](https://redirect.github.com/go-chi/chi/pull/1008)
- feat(mux): support http.Request.Pattern in Go 1.23 by
[@&#8203;Gusted](https://redirect.github.com/Gusted) in
[#&#8203;986](https://redirect.github.com/go-chi/chi/pull/986)
- fix/608 - Fix flaky Throttle middleware test by synchronizing token
usage by
[@&#8203;OtavioBernardes](https://redirect.github.com/OtavioBernardes)
in [#&#8203;1016](https://redirect.github.com/go-chi/chi/pull/1016)
- Optimize throttle middleware by avoiding unnecessary timer creation by
[@&#8203;vasayxtx](https://redirect.github.com/vasayxtx) in
[#&#8203;1011](https://redirect.github.com/go-chi/chi/pull/1011)
- Simplify wildcard replacement in route patterns by
[@&#8203;srpvpn](https://redirect.github.com/srpvpn) in
[#&#8203;1012](https://redirect.github.com/go-chi/chi/pull/1012)
- Replace methodTypString func with reverseMethodMap by
[@&#8203;JRaspass](https://redirect.github.com/JRaspass) in
[#&#8203;1018](https://redirect.github.com/go-chi/chi/pull/1018)

#### New Contributors

-
[@&#8203;ProjectMutilation](https://redirect.github.com/ProjectMutilation)
made their first contribution in
[#&#8203;1008](https://redirect.github.com/go-chi/chi/pull/1008)
- [@&#8203;Gusted](https://redirect.github.com/Gusted) made their first
contribution in
[#&#8203;986](https://redirect.github.com/go-chi/chi/pull/986)
- [@&#8203;OtavioBernardes](https://redirect.github.com/OtavioBernardes)
made their first contribution in
[#&#8203;1016](https://redirect.github.com/go-chi/chi/pull/1016)
- [@&#8203;srpvpn](https://redirect.github.com/srpvpn) made their first
contribution in
[#&#8203;1012](https://redirect.github.com/go-chi/chi/pull/1012)

**Full Changelog**:
<https://github.com/go-chi/chi/compare/v5.2.2...v5.2.3>

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/googleapis/genai-toolbox).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS44Mi43IiwidXBkYXRlZEluVmVyIjoiNDEuODIuNyIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-08-27 14:54:58 -07:00
Mend Renovate
58d7b3e5f8 chore(deps): update google.golang.org/genproto digest to ef028d9 (#1234)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[google.golang.org/genproto](https://redirect.github.com/googleapis/go-genproto)
| require | digest | `3122310` -> `ef028d9` |

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/googleapis/genai-toolbox).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS44Mi43IiwidXBkYXRlZEluVmVyIjoiNDEuODIuNyIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
2025-08-27 14:39:47 -07:00
Yuan Teoh
ca353e0b66 feat(server/mcp): support toolbox auth in mcp (#1140)
Existing `/mcp` endpoint of Toolbox does not support auth (authorized
invocation and authenticated parameters). This PR add support for
Toolbox auth to the `/mcp` endpoint.

Added integration test for MCP with auth.

Note that Toolbox auth is **NOT** supported in stdio transport protocol,
invocations of tools with auth will result in error.
2025-08-27 09:38:56 -07:00
Yuan Teoh
fc707fb561 test: add unit tests for unauthorized calls (#1262)
Our coverage is falling below minimum (40%). Add unit tests for
unauthorized calls.
2025-08-27 09:01:48 -07:00
Yuan Teoh
03aa9fabac fix: update env var to allow empty string (#1260)
Update configuration file capability of reading from environment
variable:
* allow default empty string
* fix if statement to check if `parts[2]` is empty instead since it will
always have a length of 4.
* add unit tests.
2025-08-27 01:25:28 +00:00
release-please[bot]
1a6dfe8d37 chore(main): release 0.13.0 (#1175)
🤖 I have created a release *beep* *boop*
---


##
[0.13.0](https://github.com/googleapis/genai-toolbox/compare/v0.12.0...v0.13.0)
(2025-08-27)


### ⚠ BREAKING CHANGES

* **prebuilt/alloydb:** Add bearer token support for
alloydb-wait-for-operation
([#1183](https://github.com/googleapis/genai-toolbox/issues/1183))


### Features

* Add capability to set default for environment variable in config
([#1248](https://github.com/googleapis/genai-toolbox/issues/1248))
([5bcd52e](5bcd52e7dc))
* **firebird:** Add Firebird SQL 2.5+ source and tool
([#1011](https://github.com/googleapis/genai-toolbox/issues/1011))
([4f6b806](4f6b806de9))
* **oceanbase:** Add Oceanbase source and tool
([#895](https://github.com/googleapis/genai-toolbox/issues/895))
([6fc4982](6fc49826d4))
* **server/mcp:** Support `ping` mechanism
([#1178](https://github.com/googleapis/genai-toolbox/issues/1178))
([5dcc66c](5dcc66c84f))
* **server:** Fail-fast on environment variable substitution
([#1177](https://github.com/googleapis/genai-toolbox/issues/1177))
([212aaba](212aaba74c))
* **server:** Implement Tool call auth error propagation
([#1235](https://github.com/googleapis/genai-toolbox/issues/1235))
([b94a021](b94a021ca1))
* **sources/bigquery:** Add support for user-credential passthrough
([#1067](https://github.com/googleapis/genai-toolbox/issues/1067))
([650e2e2](650e2e26f5))
* **tool/looker:** Add support for `description` field in looker tool
([#1199](https://github.com/googleapis/genai-toolbox/issues/1199))
([97f0dd2](97f0dd2acf))
* **tools/bigquery-ask-data-insights:** Add bigquery `ask-data-insights`
tool ([#932](https://github.com/googleapis/genai-toolbox/issues/932))
([7651357](7651357d42))
* **tools/bigquery-forecast:** Add bigqueryforecast tool
([#1148](https://github.com/googleapis/genai-toolbox/issues/1148))
([2ad0ccf](2ad0ccf83d))
* **tools/firestore-add-documents:** Add firestore-add-documents tool
([#1107](https://github.com/googleapis/genai-toolbox/issues/1107))
([ee4a70a](ee4a70a0e8))
* **tools/firestore-update-document:** Add firestore-update-document
tool ([#1191](https://github.com/googleapis/genai-toolbox/issues/1191))
([0010123](00101232a3))
* **tools/looker:** Control over whether hidden objects are surfaced
([#1222](https://github.com/googleapis/genai-toolbox/issues/1222))
([bc91559](bc91559cc4))
* **trino:** Add Trino source and tools
([#948](https://github.com/googleapis/genai-toolbox/issues/948))
([7dd123b](7dd123b3d7))


### Bug Fixes

* **tools/looker:** Lookergetdashboards uses proper Authorized helper
func ([#1255](https://github.com/googleapis/genai-toolbox/issues/1255))
([00866bc](00866bc7fc))
* **tools/mongodb-find-one:** ProjectPayload unmarshaling
([#1167](https://github.com/googleapis/genai-toolbox/issues/1167))
([8ea6a98](8ea6a98bd9))
* **tools/mysql:** Fix encoded text for mysql
([#1161](https://github.com/googleapis/genai-toolbox/issues/1161))
([a37cfa8](a37cfa841d)),
closes [#840](https://github.com/googleapis/genai-toolbox/issues/840)


---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-08-27 01:07:04 +00:00
Yuan Teoh
297f240955 chore: release 0.13.0 (#1259)
Release-As: 0.13.0
2025-08-27 00:25:21 +00:00
Mend Renovate
6ad6d39084 chore(deps): update module github.com/microsoft/go-mssqldb to v1.9.3 (#1236)
This PR contains the following updates:

| Package | Change | Age | Confidence |
|---|---|---|---|
|
[github.com/microsoft/go-mssqldb](https://redirect.github.com/microsoft/go-mssqldb)
| `v1.9.2` -> `v1.9.3` |
[![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fmicrosoft%2fgo-mssqldb/v1.9.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fmicrosoft%2fgo-mssqldb/v1.9.2/v1.9.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>microsoft/go-mssqldb
(github.com/microsoft/go-mssqldb)</summary>

###
[`v1.9.3`](https://redirect.github.com/microsoft/go-mssqldb/compare/v1.9.2...v1.9.3)

[Compare
Source](https://redirect.github.com/microsoft/go-mssqldb/compare/v1.9.2...v1.9.3)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/googleapis/genai-toolbox).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS44Mi43IiwidXBkYXRlZEluVmVyIjoiNDEuODIuNyIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-08-26 17:00:23 -07:00
Mend Renovate
f8929e3396 chore(deps): update module github.com/couchbase/gocb/v2 to v2.11.0 (#1249)
This PR contains the following updates:

| Package | Change | Age | Confidence |
|---|---|---|---|
|
[github.com/couchbase/gocb/v2](https://redirect.github.com/couchbase/gocb)
| `v2.10.1` -> `v2.11.0` |
[![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fcouchbase%2fgocb%2fv2/v2.11.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fcouchbase%2fgocb%2fv2/v2.10.1/v2.11.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>couchbase/gocb (github.com/couchbase/gocb/v2)</summary>

###
[`v2.11.0`](https://redirect.github.com/couchbase/gocb/compare/v2.10.1...v2.11.0)

[Compare
Source](https://redirect.github.com/couchbase/gocb/compare/v2.10.1...v2.11.0)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/googleapis/genai-toolbox).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS44Mi43IiwidXBkYXRlZEluVmVyIjoiNDEuODIuNyIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-08-26 16:50:25 -07:00
Dr. Strangelove
bc91559cc4 feat(tools/looker): Control over whether hidden objects are surfaced (#1222)
The following config is added to the looker source:

* show_hidden_models
* show_hidden_explores
* show_hidden_fields

## Description
---
> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

## PR Checklist
---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [ x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/langchain-google-alloydb-pg-python/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [ x] Ensure the tests and linter pass
- [ x] Code coverage does not decrease (if any source code was changed)
- [ x] Appropriate docs were updated (if necessary)
- [ x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1214
2025-08-26 15:39:44 -07:00
Wenxin Du
650e2e26f5 feat(sources/bigquery): add support for user-credential passthrough (#1067)
Support end-user credential passthrough with the BigQuery source and the
`bigquery-sql` tool.
Support for other BQ tools will be added in subsequent PRs.

Issue: https://github.com/googleapis/genai-toolbox/issues/813
2025-08-26 17:52:24 -04:00
Dr. Strangelove
00866bc7fc fix(tools/looker): lookergetdashboards uses proper Authorized helper func (#1255)
looker-getdash-board uses the Authorized helper function.

## PR Checklist
---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x ] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/langchain-google-alloydb-pg-python/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [ x] Ensure the tests and linter pass
- [ x] Code coverage does not decrease (if any source code was changed)
- [ x] Appropriate docs were updated (if necessary)
- [x ] Make sure to add `!` if this involve a breaking change
2025-08-26 20:32:08 +00:00
Dr. Strangelove
5bcd52e7dc feat: Add capability to set default for environment variable in config (#1248)
Support `${ENV_VALUE:default_value}`

🛠️ Fixes #1001
2025-08-26 20:28:55 +00:00
Huan Chen
7651357d42 feat(tools/bigquery-ask-data-insights): add bigquery ask-data-insights tool (#932)
1. Add ask-data-insights tool based on conversational analytic API.
2. Add tokenSource for ask-data-insights tool, it uses access token
instead of client or restService.
3. Add a max row count to source, currently fixed to 50 and used only
for ask-data-insights tool. Later we may make it available for user to
make change and apply to bigquery-execute-sql and bigquery-sql to avoid
return too many data by accident.

---------

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-08-26 20:16:17 +00:00
Wenxin Du
b94a021ca1 feat(server): implement Tool call auth error propagation (#1235)
For Toolbox protocol:
Before -  return 400 error for all tool invocation errors. 
After - Propagate auth-related errors (401 & 403) to the client if using
client credentials. If using ADC, raise 500 error instead.

For MCP protocol:
Before -  return 200 with error message in the response body.
After - Propagate auth-related errors (401 & 403) to the client if using
client credentials. If using ADC, raise 500 error instead.
2025-08-26 15:27:46 -04:00
Dr. Strangelove
81c36354cb fix(tools/looker): Minor fixes and refinements to Looker (#1215)
* fix not null expression 
* exclude dimensions that end in _raw
* Add tags to return from get_dimensions, get_measures, etc.
* Add synonyms to return from get_dimensions, get_measures, etc.
* Instruct Agent to only create pie chart visualizations with one
measure and one dimension.
* Added docs about the return types of
get_dimensions/get_measures/get_filters/get_parameters.

---------

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-08-26 18:55:23 +00:00
96 changed files with 4851 additions and 447 deletions

View File

@@ -575,6 +575,28 @@ steps:
firebird \
firebirdsql firebirdexecutesql
- id: "clickhouse"
name : golang:1
waitFor: ["compile-test-binary"]
entrypoint: /bin/bash
env:
- "GOPATH=/gopath"
- "CLICKHOUSE_DATABASE=$_CLICKHOUSE_DATABASE"
- "CLICKHOUSE_PORT=$_CLICKHOUSE_PORT"
- "CLICKHOUSE_PROTOCOL=$_CLICKHOUSE_PROTOCOL"
- "SERVICE_ACCOUNT_EMAIL=$SERVICE_ACCOUNT_EMAIL"
secretEnv: ["CLICKHOUSE_HOST", "CLICKHOUSE_USER", "CLIENT_ID"]
volumes:
- name: "go"
path: "/gopath"
args:
- -c
- |
.ci/test_with_coverage.sh \
"ClickHouse" \
clickhouse \
clickhouse
- id: "trino"
name: golang:1
waitFor: ["compile-test-binary"]
@@ -660,6 +682,10 @@ availableSecrets:
env: TIDB_USER
- versionName: projects/$PROJECT_ID/secrets/tidb_pass/versions/latest
env: TIDB_PASS
- versionName: projects/$PROJECT_ID/secrets/clickhouse_host/versions/latest
env: CLICKHOUSE_HOST
- versionName: projects/$PROJECT_ID/secrets/clickhouse_user/versions/latest
env: CLICKHOUSE_USER
- versionName: projects/$PROJECT_ID/secrets/firebird_user/versions/latest
env: FIREBIRD_USER
- versionName: projects/$PROJECT_ID/secrets/firebird_pass/versions/latest
@@ -707,6 +733,9 @@ substitutions:
_LOOKER_VERIFY_SSL: "true"
_TIDB_HOST: 127.0.0.1
_TIDB_PORT: "4000"
_CLICKHOUSE_DATABASE: "default"
_CLICKHOUSE_PORT: "8123"
_CLICKHOUSE_PROTOCOL: "http"
_FIREBIRD_HOST: 127.0.0.1
_FIREBIRD_PORT: "3050"
_TRINO_HOST: 127.0.0.1

View File

@@ -1,5 +1,36 @@
# Changelog
## [0.13.0](https://github.com/googleapis/genai-toolbox/compare/v0.12.0...v0.13.0) (2025-08-27)
### ⚠ BREAKING CHANGES
* **prebuilt/alloydb:** Add bearer token support for alloydb-wait-for-operation ([#1183](https://github.com/googleapis/genai-toolbox/issues/1183))
### Features
* Add capability to set default for environment variable in config ([#1248](https://github.com/googleapis/genai-toolbox/issues/1248)) ([5bcd52e](https://github.com/googleapis/genai-toolbox/commit/5bcd52e7dcd0773ded723585f4abe29d044e1540))
* **firebird:** Add Firebird SQL 2.5+ source and tool ([#1011](https://github.com/googleapis/genai-toolbox/issues/1011)) ([4f6b806](https://github.com/googleapis/genai-toolbox/commit/4f6b806de947efc4e12bdb50dff7781aedb7b966))
* **oceanbase:** Add Oceanbase source and tool ([#895](https://github.com/googleapis/genai-toolbox/issues/895)) ([6fc4982](https://github.com/googleapis/genai-toolbox/commit/6fc49826d43f46c84028e752ebebddf3d94b3d13))
* **server/mcp:** Support `ping` mechanism ([#1178](https://github.com/googleapis/genai-toolbox/issues/1178)) ([5dcc66c](https://github.com/googleapis/genai-toolbox/commit/5dcc66c84fa72c75ec50a9ac5198018212ec2979))
* **server:** Fail-fast on environment variable substitution ([#1177](https://github.com/googleapis/genai-toolbox/issues/1177)) ([212aaba](https://github.com/googleapis/genai-toolbox/commit/212aaba74c8b431de8a5f7b9822a0af4afcaaa0e))
* **server:** Implement Tool call auth error propagation ([#1235](https://github.com/googleapis/genai-toolbox/issues/1235)) ([b94a021](https://github.com/googleapis/genai-toolbox/commit/b94a021ca11c6637cf8038449483b5e75f2012b3))
* **sources/bigquery:** Add support for user-credential passthrough ([#1067](https://github.com/googleapis/genai-toolbox/issues/1067)) ([650e2e2](https://github.com/googleapis/genai-toolbox/commit/650e2e26f51bff75ce66343f64944d0a89a58b69))
* **tool/looker:** Add support for `description` field in looker tool ([#1199](https://github.com/googleapis/genai-toolbox/issues/1199)) ([97f0dd2](https://github.com/googleapis/genai-toolbox/commit/97f0dd2acf26caf28ecad65abea8779c196a27f1))
* **tools/bigquery-ask-data-insights:** Add bigquery `ask-data-insights` tool ([#932](https://github.com/googleapis/genai-toolbox/issues/932)) ([7651357](https://github.com/googleapis/genai-toolbox/commit/7651357d424a2b6656d8b6818cebc5c8a86ed053))
* **tools/bigquery-forecast:** Add bigqueryforecast tool ([#1148](https://github.com/googleapis/genai-toolbox/issues/1148)) ([2ad0ccf](https://github.com/googleapis/genai-toolbox/commit/2ad0ccf83df542340087742468d6762f81eedee6))
* **tools/firestore-add-documents:** Add firestore-add-documents tool ([#1107](https://github.com/googleapis/genai-toolbox/issues/1107)) ([ee4a70a](https://github.com/googleapis/genai-toolbox/commit/ee4a70a0e82b346b07b5b4c60dfa060da2273f50))
* **tools/firestore-update-document:** Add firestore-update-document tool ([#1191](https://github.com/googleapis/genai-toolbox/issues/1191)) ([0010123](https://github.com/googleapis/genai-toolbox/commit/00101232a39c70288aac5715649c184858d351e3))
* **tools/looker:** Control over whether hidden objects are surfaced ([#1222](https://github.com/googleapis/genai-toolbox/issues/1222)) ([bc91559](https://github.com/googleapis/genai-toolbox/commit/bc91559cc4e5b20385b84cc562b624fabf7e47a8))
* **trino:** Add Trino source and tools ([#948](https://github.com/googleapis/genai-toolbox/issues/948)) ([7dd123b](https://github.com/googleapis/genai-toolbox/commit/7dd123b3d76b8eb2b74b5d960959c1f90684b37e))
### Bug Fixes
* **tools/looker:** Lookergetdashboards uses proper Authorized helper func ([#1255](https://github.com/googleapis/genai-toolbox/issues/1255)) ([00866bc](https://github.com/googleapis/genai-toolbox/commit/00866bc7fc33115c547213e60316ae889735fdbb))
* **tools/mongodb-find-one:** ProjectPayload unmarshaling ([#1167](https://github.com/googleapis/genai-toolbox/issues/1167)) ([8ea6a98](https://github.com/googleapis/genai-toolbox/commit/8ea6a98bd9096ba97722e5f807366887e864004f))
* **tools/mysql:** Fix encoded text for mysql ([#1161](https://github.com/googleapis/genai-toolbox/issues/1161)) ([a37cfa8](https://github.com/googleapis/genai-toolbox/commit/a37cfa841d151b9995d4fab73cfc5e4d30d2cc57)), closes [#840](https://github.com/googleapis/genai-toolbox/issues/840)
## [0.12.0](https://github.com/googleapis/genai-toolbox/compare/v0.11.0...v0.12.0) (2025-08-14)

View File

@@ -117,7 +117,7 @@ To install Toolbox as a binary:
<!-- {x-release-please-start-version} -->
```sh
# see releases page for other versions
export VERSION=0.12.0
export VERSION=0.13.0
curl -O https://storage.googleapis.com/genai-toolbox/v$VERSION/linux/amd64/toolbox
chmod +x toolbox
```
@@ -130,7 +130,7 @@ You can also install Toolbox as a container:
```sh
# see releases page for other versions
export VERSION=0.12.0
export VERSION=0.13.0
docker pull us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:$VERSION
```
@@ -154,7 +154,7 @@ To install from source, ensure you have the latest version of
[Go installed](https://go.dev/doc/install), and then run the following command:
```sh
go install github.com/googleapis/genai-toolbox@v0.12.0
go install github.com/googleapis/genai-toolbox@v0.13.0
```
<!-- {x-release-please-end} -->

View File

@@ -43,6 +43,7 @@ import (
// Import tool packages for side effect of registration
_ "github.com/googleapis/genai-toolbox/internal/tools/alloydbainl"
_ "github.com/googleapis/genai-toolbox/internal/tools/bigquery/bigqueryconversationalanalytics"
_ "github.com/googleapis/genai-toolbox/internal/tools/bigquery/bigqueryexecutesql"
_ "github.com/googleapis/genai-toolbox/internal/tools/bigquery/bigqueryforecast"
_ "github.com/googleapis/genai-toolbox/internal/tools/bigquery/bigquerygetdatasetinfo"
@@ -51,6 +52,8 @@ import (
_ "github.com/googleapis/genai-toolbox/internal/tools/bigquery/bigquerylisttableids"
_ "github.com/googleapis/genai-toolbox/internal/tools/bigquery/bigquerysql"
_ "github.com/googleapis/genai-toolbox/internal/tools/bigtable"
_ "github.com/googleapis/genai-toolbox/internal/tools/clickhouse/clickhouseexecutesql"
_ "github.com/googleapis/genai-toolbox/internal/tools/clickhouse/clickhousesql"
_ "github.com/googleapis/genai-toolbox/internal/tools/couchbase"
_ "github.com/googleapis/genai-toolbox/internal/tools/dataplex/dataplexlookupentry"
_ "github.com/googleapis/genai-toolbox/internal/tools/dataplex/dataplexsearchaspecttypes"
@@ -119,6 +122,7 @@ import (
_ "github.com/googleapis/genai-toolbox/internal/sources/alloydbpg"
_ "github.com/googleapis/genai-toolbox/internal/sources/bigquery"
_ "github.com/googleapis/genai-toolbox/internal/sources/bigtable"
_ "github.com/googleapis/genai-toolbox/internal/sources/clickhouse"
_ "github.com/googleapis/genai-toolbox/internal/sources/cloudsqlmssql"
_ "github.com/googleapis/genai-toolbox/internal/sources/cloudsqlmysql"
_ "github.com/googleapis/genai-toolbox/internal/sources/cloudsqlpg"
@@ -264,8 +268,9 @@ type ToolsFile struct {
}
// parseEnv replaces environment variables ${ENV_NAME} with their values.
// also support ${ENV_NAME:default_value}.
func parseEnv(input string) (string, error) {
re := regexp.MustCompile(`\$\{(\w+)\}`)
re := regexp.MustCompile(`\$\{(\w+)(:(\w*))?\}`)
var err error
output := re.ReplaceAllStringFunc(input, func(match string) string {
@@ -276,6 +281,9 @@ func parseEnv(input string) (string, error) {
if value, found := os.LookupEnv(variableName); found {
return value
}
if parts[2] != "" {
return parts[3]
}
err = fmt.Errorf("environment variable not found: %q", variableName)
return ""
})
@@ -829,7 +837,7 @@ func run(cmd *Command) error {
}
cmd.logger.InfoContext(ctx, "Server ready to serve!")
if cmd.cfg.UI {
cmd.logger.InfoContext(ctx, fmt.Sprintf("Toolbox UI is up and running at: http://localhost:%d/ui", cmd.cfg.Port))
cmd.logger.InfoContext(ctx, fmt.Sprintf("Toolbox UI is up and running at: http://%s:%d/ui", cmd.cfg.Address, cmd.cfg.Port))
}
go func() {

View File

@@ -206,6 +206,72 @@ func TestServerConfigFlags(t *testing.T) {
}
}
func TestParseEnv(t *testing.T) {
tcs := []struct {
desc string
env map[string]string
in string
want string
err bool
errString string
}{
{
desc: "without default without env",
in: "${FOO}",
want: "",
err: true,
errString: `environment variable not found: "FOO"`,
},
{
desc: "without default with env",
env: map[string]string{
"FOO": "bar",
},
in: "${FOO}",
want: "bar",
},
{
desc: "with empty default",
in: "${FOO:}",
want: "",
},
{
desc: "with default",
in: "${FOO:bar}",
want: "bar",
},
{
desc: "with default with env",
env: map[string]string{
"FOO": "hello",
},
in: "${FOO:bar}",
want: "hello",
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
if tc.env != nil {
for k, v := range tc.env {
t.Setenv(k, v)
}
}
got, err := parseEnv(tc.in)
if tc.err {
if err == nil {
t.Fatalf("expected error not found")
}
if tc.errString != err.Error() {
t.Fatalf("incorrect error string: got %s, want %s", err, tc.errString)
}
}
if tc.want != got {
t.Fatalf("unexpected want: got %s, want %s", got, tc.want)
}
})
}
}
func TestToolFileFlag(t *testing.T) {
tcs := []struct {
desc string
@@ -1166,6 +1232,7 @@ func TestPrebuiltTools(t *testing.T) {
alloydb_admin_config, _ := prebuiltconfigs.Get("alloydb-postgres-admin")
alloydb_config, _ := prebuiltconfigs.Get("alloydb-postgres")
bigquery_config, _ := prebuiltconfigs.Get("bigquery")
clickhouse_config, _ := prebuiltconfigs.Get("clickhouse")
cloudsqlpg_config, _ := prebuiltconfigs.Get("cloud-sql-postgres")
cloudsqlmysql_config, _ := prebuiltconfigs.Get("cloud-sql-mysql")
cloudsqlmssql_config, _ := prebuiltconfigs.Get("cloud-sql-mssql")
@@ -1198,6 +1265,13 @@ func TestPrebuiltTools(t *testing.T) {
t.Setenv("ALLOYDB_POSTGRES_USER", "your_alloydb_user")
t.Setenv("ALLOYDB_POSTGRES_PASSWORD", "your_alloydb_password")
t.Setenv("CLICKHOUSE_PROTOCOL", "your_clickhouse_protocol")
t.Setenv("CLICKHOUSE_DATABASE", "your_clickhouse_database")
t.Setenv("CLICKHOUSE_PASSWORD", "your_clickhouse_password")
t.Setenv("CLICKHOUSE_USER", "your_clickhouse_user")
t.Setenv("CLICKHOUSE_HOST", "your_clickhosue_host")
t.Setenv("CLICKHOUSE_PORT", "8123")
t.Setenv("CLOUD_SQL_POSTGRES_PROJECT", "your_pg_project")
t.Setenv("CLOUD_SQL_POSTGRES_INSTANCE", "your_pg_instance")
t.Setenv("CLOUD_SQL_POSTGRES_DATABASE", "your_pg_db")
@@ -1279,7 +1353,17 @@ func TestPrebuiltTools(t *testing.T) {
wantToolset: server.ToolsetConfigs{
"bigquery-database-tools": tools.ToolsetConfig{
Name: "bigquery-database-tools",
ToolNames: []string{"execute_sql", "forecast", "get_dataset_info", "get_table_info", "list_dataset_ids", "list_table_ids"},
ToolNames: []string{"ask_data_insights", "execute_sql", "forecast", "get_dataset_info", "get_table_info", "list_dataset_ids", "list_table_ids"},
},
},
},
{
name: "clickhouse prebuilt tools",
in: clickhouse_config,
wantToolset: server.ToolsetConfigs{
"clickhouse-database-tools": tools.ToolsetConfig{
Name: "clickhouse-database-tools",
ToolNames: []string{"execute_sql"},
},
},
},

View File

@@ -1 +1 @@
0.12.0
0.13.0

View File

@@ -234,7 +234,7 @@
},
"outputs": [],
"source": [
"version = \"0.12.0\" # x-release-please-version\n",
"version = \"0.13.0\" # x-release-please-version\n",
"! curl -O https://storage.googleapis.com/genai-toolbox/v{version}/linux/amd64/toolbox\n",
"\n",
"# Make the binary executable\n",

View File

@@ -22,6 +22,11 @@ etc., you could use environment variables instead with the format `${ENV_NAME}`.
user: ${USER_NAME}
password: ${PASSWORD}
```
A default value can be specified like `${ENV_NAME:default}`.
```yaml
port: ${DB_PORT:3306}
```
### Sources

View File

@@ -86,7 +86,7 @@ To install Toolbox as a binary:
```sh
# see releases page for other versions
export VERSION=0.12.0
export VERSION=0.13.0
curl -O https://storage.googleapis.com/genai-toolbox/v$VERSION/linux/amd64/toolbox
chmod +x toolbox
```
@@ -97,7 +97,7 @@ You can also install Toolbox as a container:
```sh
# see releases page for other versions
export VERSION=0.12.0
export VERSION=0.13.0
docker pull us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:$VERSION
```
@@ -115,7 +115,7 @@ To install from source, ensure you have the latest version of
[Go installed](https://go.dev/doc/install), and then run the following command:
```sh
go install github.com/googleapis/genai-toolbox@v0.12.0
go install github.com/googleapis/genai-toolbox@v0.13.0
```
{{% /tab %}}

View File

@@ -105,7 +105,7 @@ In this section, we will download Toolbox, configure our tools in a
<!-- {x-release-please-start-version} -->
```bash
export OS="linux/amd64" # one of linux/amd64, darwin/arm64, darwin/amd64, or windows/amd64
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/$OS/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/$OS/toolbox
```
<!-- {x-release-please-end} -->

View File

@@ -62,19 +62,19 @@ to expose your developer assistant tools to a Firestore instance:
<!-- {x-release-please-start-version} -->
{{< tabpane persist=header >}}
{{< tab header="linux/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/linux/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/linux/amd64/toolbox
{{< /tab >}}
{{< tab header="darwin/arm64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/arm64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/arm64/toolbox
{{< /tab >}}
{{< tab header="darwin/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/amd64/toolbox
{{< /tab >}}
{{< tab header="windows/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/windows/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/windows/amd64/toolbox
{{< /tab >}}
{{< /tabpane >}}
<!-- {x-release-please-end} -->

View File

@@ -48,19 +48,19 @@ to expose your developer assistant tools to a Looker instance:
<!-- {x-release-please-start-version} -->
{{< tabpane persist=header >}}
{{< tab header="linux/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/linux/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/linux/amd64/toolbox
{{< /tab >}}
{{< tab header="darwin/arm64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/arm64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/arm64/toolbox
{{< /tab >}}
{{< tab header="darwin/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/amd64/toolbox
{{< /tab >}}
{{< tab header="windows/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/windows/amd64/toolbox.exe
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/windows/amd64/toolbox.exe
{{< /tab >}}
{{< /tabpane >}}
<!-- {x-release-please-end} -->

View File

@@ -37,19 +37,19 @@ description: "Connect your IDE to SQL Server using Toolbox."
<!-- {x-release-please-start-version} -->
{{< tabpane persist=header >}}
{{< tab header="linux/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/linux/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/linux/amd64/toolbox
{{< /tab >}}
{{< tab header="darwin/arm64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/arm64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/arm64/toolbox
{{< /tab >}}
{{< tab header="darwin/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/amd64/toolbox
{{< /tab >}}
{{< tab header="windows/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/windows/amd64/toolbox.exe
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/windows/amd64/toolbox.exe
{{< /tab >}}
{{< /tabpane >}}
<!-- {x-release-please-end} -->

View File

@@ -37,19 +37,19 @@ description: "Connect your IDE to MySQL using Toolbox."
<!-- {x-release-please-start-version} -->
{{< tabpane persist=header >}}
{{< tab header="linux/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/linux/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/linux/amd64/toolbox
{{< /tab >}}
{{< tab header="darwin/arm64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/arm64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/arm64/toolbox
{{< /tab >}}
{{< tab header="darwin/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/amd64/toolbox
{{< /tab >}}
{{< tab header="windows/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/windows/amd64/toolbox.exe
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/windows/amd64/toolbox.exe
{{< /tab >}}
{{< /tabpane >}}
<!-- {x-release-please-end} -->

View File

@@ -56,19 +56,19 @@ Omni](https://cloud.google.com/alloydb/omni/current/docs/overview).
<!-- {x-release-please-start-version} -->
{{< tabpane persist=header >}}
{{< tab header="linux/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/linux/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/linux/amd64/toolbox
{{< /tab >}}
{{< tab header="darwin/arm64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/arm64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/arm64/toolbox
{{< /tab >}}
{{< tab header="darwin/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/darwin/amd64/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/darwin/amd64/toolbox
{{< /tab >}}
{{< tab header="windows/amd64" lang="bash" >}}
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/windows/amd64/toolbox.exe
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/windows/amd64/toolbox.exe
{{< /tab >}}
{{< /tabpane >}}
<!-- {x-release-please-end} -->

View File

@@ -53,8 +53,11 @@ See guides, [Connect from your IDE](../how-to/connect-ide/_index.md), for detail
* **BigQuery User** (`roles/bigquery.user`) to execute queries and view metadata.
* **BigQuery Metadata Viewer** (`roles/bigquery.metadataViewer`) to view all datasets.
* **BigQuery Data Editor** (`roles/bigquery.dataEditor`) to create or modify datasets and tables.
* **Gemini for Google Cloud** (`roles/cloudaicompanion.user`) to use the conversational analytics API.
* **Tools:**
* `ask_data_insights`: Use this tool to perform data analysis, get insights, or answer complex questions about the contents of specific BigQuery tables. For more information on required roles, API setup, and IAM configuration, see the setup and authentication section of the [Conversational Analytics API documentation](https://cloud.google.com/gemini/docs/conversational-analytics-api/overview).
* `execute_sql`: Executes a SQL statement.
* `forecast`: Use this tool to forecast time series data.
* `get_dataset_info`: Gets dataset metadata.
* `get_table_info`: Gets table metadata.
* `list_dataset_ids`: Lists datasets.

View File

@@ -65,23 +65,40 @@ Connect your IDE to BigQuery using Toolbox.
BigQuery uses [Identity and Access Management (IAM)][iam-overview] to control
user and group access to BigQuery resources like projects, datasets, and tables.
Toolbox will use your [Application Default Credentials (ADC)][adc] to authorize
and authenticate when interacting with [BigQuery][bigquery-docs].
In addition to [setting the ADC for your server][set-adc], you need to ensure
the IAM identity has been given the correct IAM permissions for the queries
you intend to run. Common roles include `roles/bigquery.user` (which includes
permissions to run jobs and read data) or `roles/bigquery.dataViewer`. See
[Introduction to BigQuery IAM][grant-permissions] for more information on
applying IAM permissions and roles to an identity.
### Authentication via Application Default Credentials (ADC)
By **default**, Toolbox will use your [Application Default Credentials (ADC)][adc] to authorize and authenticate when interacting with [BigQuery][bigquery-docs].
When using this method, you need to ensure the IAM identity associated with your
ADC (such as a service account) has the correct permissions for the queries you
intend to run. Common roles include `roles/bigquery.user` (which includes
permissions to run jobs and read data) or `roles/bigbigquery.dataViewer`.
Follow this [guide][set-adc] to set up your ADC.
### Authentication via User's OAuth Access Token
If the `useClientOAuth` parameter is set to `true`, Toolbox will instead use the
OAuth access token for authentication. This token is parsed from the
`Authorization` header passed in with the tool invocation request. This method
allows Toolbox to make queries to [BigQuery][bigquery-docs] on behalf of the
client or the end-user.
When using this on-behalf-of authentication, you must ensure that the
identity used has been granted the correct IAM permissions. Currently,
this option is only supported by the following BigQuery tools:
- [`bigquery-sql`](../tools/bigquery/bigquery-sql.md)
Run SQL queries directly against BigQuery datasets.
[iam-overview]: https://cloud.google.com/bigquery/docs/access-control
[adc]: https://cloud.google.com/docs/authentication#adc
[set-adc]: https://cloud.google.com/docs/authentication/provide-credentials-adc
[grant-permissions]: https://cloud.google.com/bigquery/docs/access-control
## Example
Initialize a BigQuery source that uses ADC:
```yaml
sources:
my-bigquery-source:
@@ -89,10 +106,21 @@ sources:
project: "my-project-id"
```
Initialize a BigQuery source that uses the client's access token:
```yaml
sources:
my-bigquery-client-auth-source:
kind: "bigquery"
project: "my-project-id"
useClientOAuth: true
```
## Reference
| **field** | **type** | **required** | **description** |
|-----------|:--------:|:------------:|-------------------------------------------------------------------------------|
| kind | string | true | Must be "bigquery". |
| project | string | true | Id of the GCP project that the cluster was created in (e.g. "my-project-id"). |
| location | string | false | Specifies the location (e.g., 'us', 'asia-northeast1') in which to run the query job. This location must match the location of any tables referenced in the query. The default behavior is for it to be executed in the US multi-region |
| **field** | **type** | **required** | **description** |
|----------------|:--------:|:------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| kind | string | true | Must be "bigquery". |
| project | string | true | Id of the GCP project that the cluster was created in (e.g. "my-project-id"). |
| location | string | false | Specifies the location (e.g., 'us', 'asia-northeast1') in which to run the query job. This location must match the location of any tables referenced in the query. The default behavior is for it to be executed in the US multi-region |
| useClientOAuth | bool | false | If true, forwards the client's OAuth access token from the "Authorization" header to downstream queries. |

View File

@@ -0,0 +1,91 @@
---
title: "ClickHouse"
type: docs
weight: 1
description: >
ClickHouse is an open-source, OLTP database.
---
## About
[ClickHouse][clickhouse-docs] is a fast, open-source, column-oriented database
[clickhouse-docs]: https://clickhouse.com/docs
## Available Tools
- [`clickhouse-execute-sql`](../tools/clickhouse/clickhouse-execute-sql.md)
Execute parameterized SQL queries in ClickHouse with query logging.
- [`clickhouse-sql`](../tools/clickhouse/clickhouse-sql.md)
Execute SQL queries as prepared statements in ClickHouse.
## Requirements
### Database User
This source uses standard ClickHouse authentication. You will need to [create a
ClickHouse user][clickhouse-users] (or with [ClickHouse Cloud][clickhouse-cloud]) to connect to the database with. The user
should have appropriate permissions for the operations you plan to perform.
[clickhouse-cloud]: https://clickhouse.com/docs/getting-started/quick-start/cloud#connect-with-your-app
[clickhouse-users]: https://clickhouse.com/docs/en/sql-reference/statements/create/user
### Network Access
ClickHouse supports multiple protocols:
- **HTTPS protocol** (default port 8443) - Secure HTTP access (default)
- **HTTP protocol** (default port 8123) - Good for web-based access
## Example
### Secure Connection Example
```yaml
sources:
secure-clickhouse-source:
kind: clickhouse
host: clickhouse.example.com
port: "8443"
database: analytics
user: ${CLICKHOUSE_USER}
password: ${CLICKHOUSE_PASSWORD}
protocol: https
secure: true
```
### HTTP Protocol Example
```yaml
sources:
http-clickhouse-source:
kind: clickhouse
host: localhost
port: "8123"
database: logs
user: ${CLICKHOUSE_USER}
password: ${CLICKHOUSE_PASSWORD}
protocol: http
secure: false
```
{{< notice tip >}}
Use environment variable replacement with the format ${ENV_NAME}
instead of hardcoding your secrets into the configuration file.
{{< /notice >}}
## Reference
| **field** | **type** | **required** | **description** |
|-------------|:--------:|:------------:|------------------------------------------------------------------------------------|
| kind | string | true | Must be "clickhouse". |
| host | string | true | IP address or hostname to connect to (e.g. "127.0.0.1" or "clickhouse.example.com") |
| port | string | true | Port to connect to (e.g. "8443" for HTTPS, "8123" for HTTP) |
| database | string | true | Name of the ClickHouse database to connect to (e.g. "my_database"). |
| user | string | true | Name of the ClickHouse user to connect as (e.g. "analytics_user"). |
| password | string | false | Password of the ClickHouse user (e.g. "my-password"). |
| protocol | string | false | Connection protocol: "https" (default) or "http". |
| secure | boolean | false | Whether to use a secure connection (TLS). Default: false. |

View File

@@ -56,11 +56,14 @@ instead of hardcoding your secrets into the configuration file.
## Reference
| **field** | **type** | **required** | **description** |
| ------------- | :------: | :----------: | ----------------------------------------------------------------------------------------- |
| kind | string | true | Must be "looker". |
| base_url | string | true | The URL of your Looker server with no trailing /). |
| client_id | string | true | The client id assigned by Looker. |
| client_secret | string | true | The client secret assigned by Looker. |
| verify_ssl | string | true | Whether to check the ssl certificate of the server. |
| timeout | string | false | Maximum time to wait for query execution (e.g. "30s", "2m"). By default, 120s is applied. |
| **field** | **type** | **required** | **description** |
| -------------------- | :------: | :----------: | ----------------------------------------------------------------------------------------- |
| kind | string | true | Must be "looker". |
| base_url | string | true | The URL of your Looker server with no trailing /). |
| client_id | string | true | The client id assigned by Looker. |
| client_secret | string | true | The client secret assigned by Looker. |
| verify_ssl | string | true | Whether to check the ssl certificate of the server. |
| timeout | string | false | Maximum time to wait for query execution (e.g. "30s", "2m"). By default, 120s is applied. |
| show_hidden_models | string | false | Show or hide hidden models. (default: true) |
| show_hidden_explores | string | false | Show or hide hidden explores. (default: true) |
| show_hidden_fields | string | false | Show or hide hidden fields. (default: true) |

View File

@@ -0,0 +1,54 @@
---
title: "bigquery-conversational-analytics"
type: docs
weight: 1
description: >
A "bigquery-conversational-analytics" tool allows conversational interaction with a BigQuery source.
aliases:
- /resources/tools/bigquery-conversational-analytics
---
## About
A `bigquery-conversational-analytics` tool allows you to ask questions about your data in natural language.
This function takes a user's question (which can include conversational history for context)
and references to specific BigQuery tables, and sends them to a stateless conversational API.
The API uses a GenAI agent to understand the question, generate and execute SQL queries
and Python code, and formulate an answer. This function returns a detailed, sequential
log of this entire process, which includes any generated SQL or Python code, the data
retrieved, and the final text answer.
**Note**: This tool requires additional setup in your project. Please refer to the
official [Conversational Analytics API documentation](https://cloud.google.com/gemini/docs/conversational-analytics-api/overview)
for instructions.
It's compatible with the following sources:
- [bigquery](../sources/bigquery.md)
The tool takes the following input parameters:
* `user_query_with_context`: The user's question, potentially including conversation history and system instructions for context.
* `table_references`: A JSON string of a list of BigQuery tables to use as context. Each object in the list must contain `projectId`, `datasetId`, and `tableId`. Example: `'[{"projectId": "my-gcp-project", "datasetId": "my_dataset", "tableId": "my_table"}]'`
## Example
```yaml
tools:
ask_data_insights:
kind: bigquery-conversational-analytics
source: my-bigquery-source
description: |
Use this tool to perform data analysis, get insights, or answer complex
questions about the contents of specific BigQuery tables.
```
## Reference
| **field** | **type** | **required** | **description** |
|-------------|:------------------------------------------:|:------------:|--------------------------------------------------------------------------------------------------|
| kind | string | true | Must be "bigquery-conversational-analytics". |
| source | string | true | Name of the source for chat. |
| description | string | true | Description of the tool
that is passed to the LLM. |

View File

@@ -0,0 +1,7 @@
---
title: "ClickHouse"
type: docs
weight: 1
description: >
Tools for interacting with ClickHouse databases and tables.
---

View File

@@ -0,0 +1,46 @@
---
title: "clickhouse-execute-sql"
type: docs
weight: 1
description: >
A "clickhouse-execute-sql" tool executes a SQL statement against a ClickHouse
database.
aliases:
- /resources/tools/clickhouse-execute-sql
---
## About
A `clickhouse-execute-sql` tool executes a SQL statement against a ClickHouse
database. It's compatible with the [clickhouse](../../sources/clickhouse.md) source.
`clickhouse-execute-sql` takes one input parameter `sql` and runs the SQL
statement against the specified `source`. This tool includes query logging
capabilities for monitoring and debugging purposes.
> **Note:** This tool is intended for developer assistant workflows with
> human-in-the-loop and shouldn't be used for production agents.
## Example
```yaml
tools:
execute_sql_tool:
kind: clickhouse-execute-sql
source: my-clickhouse-instance
description: Use this tool to execute SQL statements against ClickHouse.
```
## Parameters
| **parameter** | **type** | **required** | **description** |
|---------------|:--------:|:------------:|----------------------------------------------------|
| sql | string | true | The SQL statement to execute against the database |
## Reference
| **field** | **type** | **required** | **description** |
|-------------|:--------:|:------------:|---------------------------------------------------------|
| kind | string | true | Must be "clickhouse-execute-sql". |
| source | string | true | Name of the ClickHouse source to execute SQL against. |
| description | string | true | Description of the tool that is passed to the LLM. |

View File

@@ -0,0 +1,81 @@
---
title: "clickhouse-sql"
type: docs
weight: 2
description: >
A "clickhouse-sql" tool executes SQL queries as prepared statements in ClickHouse.
aliases:
- /resources/tools/clickhouse-sql
---
## About
A `clickhouse-sql` tool executes SQL queries as prepared statements against a
ClickHouse database. It's compatible with the [clickhouse](../../sources/clickhouse.md) source.
This tool supports both template parameters (for SQL statement customization)
and regular parameters (for prepared statement values), providing flexible
query execution capabilities.
## Example
```yaml
tools:
my_analytics_query:
kind: clickhouse-sql
source: my-clickhouse-instance
description: Get user analytics for a specific date range
statement: |
SELECT
user_id,
count(*) as event_count,
max(timestamp) as last_event
FROM events
WHERE date >= ? AND date <= ?
GROUP BY user_id
ORDER BY event_count DESC
LIMIT ?
parameters:
- name: start_date
description: Start date for the query (YYYY-MM-DD format)
- name: end_date
description: End date for the query (YYYY-MM-DD format)
- name: limit
description: Maximum number of results to return
```
## Template Parameters Example
```yaml
tools:
flexible_table_query:
kind: clickhouse-sql
source: my-clickhouse-instance
description: Query any table with flexible columns
statement: |
SELECT {{columns}}
FROM {{table_name}}
WHERE created_date >= ?
LIMIT ?
templateParameters:
- name: columns
description: Comma-separated list of columns to select
- name: table_name
description: Name of the table to query
parameters:
- name: start_date
description: Start date filter
- name: limit
description: Maximum number of results
```
## Reference
| **field** | **type** | **required** | **description** |
|--------------------|:------------------:|:------------:|-----------------------------------------------------------|
| kind | string | true | Must be "clickhouse-sql". |
| source | string | true | Name of the ClickHouse source to execute SQL against. |
| description | string | true | Description of the tool that is passed to the LLM. |
| statement | string | true | The SQL statement template to execute. |
| parameters | array of Parameter | false | Parameters for prepared statement values. |
| templateParameters | array of Parameter | false | Parameters for SQL statement template customization. |

View File

@@ -35,6 +35,20 @@ tools:
explore_name looked up from get_explores.
```
The response is a json array with the following elements:
```json
{
"name": "field name",
"description": "field description",
"type": "field type",
"label": "field label",
"label_short": "field short label",
"tags": ["tags", ...],
"synonyms": ["synonyms", ...]
}
```
## Reference
| **field** | **type** | **required** | **description** |

View File

@@ -35,6 +35,21 @@ tools:
explore_name looked up from get_explores.
```
The response is a json array with the following elements:
```json
{
"name": "field name",
"description": "field description",
"type": "field type",
"label": "field label",
"label_short": "field short label",
"tags": ["tags", ...],
"synonyms": ["synonyms", ...]
}
```
## Reference
| **field** | **type** | **required** | **description** |

View File

@@ -35,6 +35,21 @@ tools:
explore_name looked up from get_explores.
```
The response is a json array with the following elements:
```json
{
"name": "field name",
"description": "field description",
"type": "field type",
"label": "field label",
"label_short": "field short label",
"tags": ["tags", ...],
"synonyms": ["synonyms", ...]
}
```
## Reference
| **field** | **type** | **required** | **description** |

View File

@@ -35,6 +35,21 @@ tools:
explore_name looked up from get_explores.
```
The response is a json array with the following elements:
```json
{
"name": "field name",
"description": "field description",
"type": "field type",
"label": "field label",
"label_short": "field short label",
"tags": ["tags", ...],
"synonyms": ["synonyms", ...]
}
```
## Reference
| **field** | **type** | **required** | **description** |

View File

@@ -220,7 +220,7 @@
},
"outputs": [],
"source": [
"version = \"0.12.0\" # x-release-please-version\n",
"version = \"0.13.0\" # x-release-please-version\n",
"! curl -O https://storage.googleapis.com/genai-toolbox/v{version}/linux/amd64/toolbox\n",
"\n",
"# Make the binary executable\n",

View File

@@ -179,7 +179,7 @@ to use BigQuery, and then run the Toolbox server.
<!-- {x-release-please-start-version} -->
```bash
export OS="linux/amd64" # one of linux/amd64, darwin/arm64, darwin/amd64, or windows/amd64
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/$OS/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/$OS/toolbox
```
<!-- {x-release-please-end} -->

View File

@@ -98,7 +98,7 @@ In this section, we will download Toolbox, configure our tools in a
<!-- {x-release-please-start-version} -->
```bash
export OS="linux/amd64" # one of linux/amd64, darwin/arm64, darwin/amd64, or windows/amd64
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/$OS/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/$OS/toolbox
```
<!-- {x-release-please-end} -->

View File

@@ -34,7 +34,7 @@ In this section, we will download Toolbox and run the Toolbox server.
<!-- {x-release-please-start-version} -->
```bash
export OS="linux/amd64" # one of linux/amd64, darwin/arm64, darwin/amd64, or windows/amd64
curl -O https://storage.googleapis.com/genai-toolbox/v0.12.0/$OS/toolbox
curl -O https://storage.googleapis.com/genai-toolbox/v0.13.0/$OS/toolbox
```
<!-- {x-release-please-end} -->

30
go.mod
View File

@@ -1,6 +1,6 @@
module github.com/googleapis/genai-toolbox
go 1.24
go 1.24.0
toolchain go1.25.0
@@ -8,17 +8,18 @@ require (
cloud.google.com/go/alloydbconn v1.15.5
cloud.google.com/go/bigquery v1.69.0
cloud.google.com/go/bigtable v1.38.0
cloud.google.com/go/cloudsqlconn v1.18.0
cloud.google.com/go/cloudsqlconn v1.18.1
cloud.google.com/go/dataplex v1.26.0
cloud.google.com/go/firestore v1.18.0
cloud.google.com/go/spanner v1.84.1
github.com/ClickHouse/clickhouse-go/v2 v2.36.0
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.53.0
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v1.29.0
github.com/cenkalti/backoff/v5 v5.0.3
github.com/couchbase/gocb/v2 v2.10.1
github.com/couchbase/gocb/v2 v2.11.0
github.com/couchbase/tools-common/http v1.0.9
github.com/fsnotify/fsnotify v1.9.0
github.com/go-chi/chi/v5 v5.2.2
github.com/go-chi/chi/v5 v5.2.3
github.com/go-chi/httplog/v2 v2.1.1
github.com/go-chi/render v1.0.3
github.com/go-goquery/goquery v1.0.1
@@ -30,7 +31,7 @@ require (
github.com/jackc/pgx/v5 v5.7.5
github.com/json-iterator/go v1.1.12
github.com/looker-open-source/sdk-codegen/go v0.25.10
github.com/microsoft/go-mssqldb v1.9.2
github.com/microsoft/go-mssqldb v1.9.3
github.com/nakagami/firebirdsql v0.9.15
github.com/neo4j/neo4j-go-driver/v5 v5.28.2
github.com/redis/go-redis/v9 v9.12.1
@@ -49,14 +50,22 @@ require (
go.opentelemetry.io/otel/trace v1.37.0
golang.org/x/oauth2 v0.30.0
google.golang.org/api v0.248.0
google.golang.org/genproto v0.0.0-20250818200422-3122310a409c
google.golang.org/genproto v0.0.0-20250826171959-ef028d996bc1
modernc.org/sqlite v1.38.2
)
require (
github.com/ClickHouse/ch-go v0.66.0 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/go-faster/city v1.0.1 // indirect
github.com/go-faster/errors v0.7.1 // indirect
github.com/paulmach/orb v0.11.1 // indirect
github.com/segmentio/asm v1.2.0 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b // indirect
gonum.org/v1/gonum v0.16.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
require (
@@ -79,7 +88,7 @@ require (
github.com/apache/arrow/go/v15 v15.0.2 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 // indirect
github.com/couchbase/gocbcore/v10 v10.7.1 // indirect
github.com/couchbase/gocbcore/v10 v10.8.0 // indirect
github.com/couchbase/gocbcoreps v0.1.3 // indirect
github.com/couchbase/goprotostellar v1.0.2 // indirect
github.com/couchbase/tools-common/errors v1.0.0 // indirect
@@ -132,7 +141,6 @@ require (
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/shopspring/decimal v1.2.0 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/spiffe/go-spiffe/v2 v2.5.0 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
@@ -145,7 +153,7 @@ require (
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.36.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.62.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0 // indirect
go.opentelemetry.io/contrib/propagators/aws v1.37.0 // indirect
go.opentelemetry.io/contrib/propagators/b3 v1.37.0 // indirect
@@ -164,10 +172,10 @@ require (
golang.org/x/time v0.12.0 // indirect
golang.org/x/tools v0.35.0 // indirect
golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250811230008-5f3141c8851a // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250818200422-3122310a409c // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c // indirect
google.golang.org/grpc v1.74.2 // indirect
google.golang.org/protobuf v1.36.7 // indirect
google.golang.org/protobuf v1.36.8 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
modernc.org/libc v1.66.3 // indirect
modernc.org/mathutil v1.7.1 // indirect

80
go.sum
View File

@@ -167,8 +167,8 @@ cloud.google.com/go/cloudbuild v1.9.0/go.mod h1:qK1d7s4QlO0VwfYn5YuClDGg2hfmLZEb
cloud.google.com/go/clouddms v1.3.0/go.mod h1:oK6XsCDdW4Ib3jCCBugx+gVjevp2TMXFtgxvPSee3OM=
cloud.google.com/go/clouddms v1.4.0/go.mod h1:Eh7sUGCC+aKry14O1NRljhjyrr0NFC0G2cjwX0cByRk=
cloud.google.com/go/clouddms v1.5.0/go.mod h1:QSxQnhikCLUw13iAbffF2CZxAER3xDGNHjsTAkQJcQA=
cloud.google.com/go/cloudsqlconn v1.18.0 h1:mP6TY/7I+nrnIh6vmbWCRJPxpFBZSL6AZhW6HaYC/OI=
cloud.google.com/go/cloudsqlconn v1.18.0/go.mod h1:58bxZZ17Mz5D83ddMT8x6w56yKpcmVXyaOwGWkzGcMw=
cloud.google.com/go/cloudsqlconn v1.18.1 h1:IIvs7QJ8eqKUUHSon13Joie9oH7/i7MJwNzBLG+FrhM=
cloud.google.com/go/cloudsqlconn v1.18.1/go.mod h1:58bxZZ17Mz5D83ddMT8x6w56yKpcmVXyaOwGWkzGcMw=
cloud.google.com/go/cloudtasks v1.5.0/go.mod h1:fD92REy1x5woxkKEkLdvavGnPJGEn8Uic9nWuLzqCpY=
cloud.google.com/go/cloudtasks v1.6.0/go.mod h1:C6Io+sxuke9/KNRkbQpihnW93SWDU3uXt92nu85HkYI=
cloud.google.com/go/cloudtasks v1.7.0/go.mod h1:ImsfdYWwlWNJbdgPIIGJWC+gemEGTBK/SunNQQNCAb4=
@@ -632,8 +632,8 @@ cloud.google.com/go/workflows v1.7.0/go.mod h1:JhSrZuVZWuiDfKEFxU0/F1PQjmpnpcoIS
cloud.google.com/go/workflows v1.8.0/go.mod h1:ysGhmEajwZxGn1OhGOGKsTXc5PyxOc0vfKf5Af+to4M=
cloud.google.com/go/workflows v1.9.0/go.mod h1:ZGkj1aFIOd9c8Gerkjjq7OW7I5+l6cSvT3ujaO/WwSA=
cloud.google.com/go/workflows v1.10.0/go.mod h1:fZ8LmRmZQWacon9UCX1r/g/DfAXx5VcPALq2CxzdePw=
dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
dario.cat/mergo v1.0.1 h1:Ra4+bf83h2ztPIQYNP99R6m+Y7KfnARDfID+a+vLl4s=
dario.cat/mergo v1.0.1/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
@@ -655,6 +655,10 @@ github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 h1:oygO0locgZJ
github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/ClickHouse/ch-go v0.66.0 h1:hLslxxAVb2PHpbHr4n0d6aP8CEIpUYGMVT1Yj/Q5Img=
github.com/ClickHouse/ch-go v0.66.0/go.mod h1:noiHWyLMJAZ5wYuq3R/K0TcRhrNA8h7o1AqHX0klEhM=
github.com/ClickHouse/clickhouse-go/v2 v2.36.0 h1:FJ03h8VdmBUhvR9nQEu5jRLdfG0c/HSxUjiNdOxRQww=
github.com/ClickHouse/clickhouse-go/v2 v2.36.0/go.mod h1:aijX64fKD1hAWu/zqWEmiGk7wRE8ZnpN0M3UvjsZG3I=
github.com/GoogleCloudPlatform/grpc-gcp-go/grpcgcp v1.5.3 h1:2afWGsMzkIcN8Qm4mgPJKZWyroE5QBszMiDMYEBrnfw=
github.com/GoogleCloudPlatform/grpc-gcp-go/grpcgcp v1.5.3/go.mod h1:dppbR7CwXD4pgtV9t3wD1812RaLDcBjtblcDF5f1vI0=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.27.0 h1:ErKg/3iS1AKcTkf3yixlZ54f9U1rljCkQyEXWUnIUxc=
@@ -684,6 +688,8 @@ github.com/ajstarks/deck/generate v0.0.0-20210309230005-c3f852c02e19/go.mod h1:T
github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af/go.mod h1:K08gAheRH3/J6wwsYMMT4xOr94bZjxIelGM0+d/wbFw=
github.com/ajstarks/svgo v0.0.0-20211024235047-1546f124cd8b/go.mod h1:1KcenG0jGWcpt8ov532z81sp/kMMUG485J2InIOyADM=
github.com/andybalholm/brotli v1.0.4/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/andybalholm/cascadia v1.3.3 h1:AG2YHrzJIm4BZ19iwJ/DAua6Btl3IwJX+VI4kktS1LM=
github.com/andybalholm/cascadia v1.3.3/go.mod h1:xNd9bqTn98Ln4DwST8/nG+H0yuB8Hmgu1YHNnWw0GeA=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
@@ -770,10 +776,10 @@ github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 h1:aQ3y1lwWyqYPiWZThqv
github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443/go.mod h1:W+zGtBO5Y1IgJhy4+A9GOqVhqLpfZi+vwmdNXUehLA8=
github.com/containerd/continuity v0.4.3 h1:6HVkalIp+2u1ZLH1J/pYX2oBVXlJZvh1X1A7bEZ9Su8=
github.com/containerd/continuity v0.4.3/go.mod h1:F6PTNCKepoxEaXLQp3wDAjygEnImnZ/7o4JzpodfroQ=
github.com/couchbase/gocb/v2 v2.10.1 h1:5r1jngGxw3dTZdtq6Xmjq3pdU6hOwRvynvbVIp58T64=
github.com/couchbase/gocb/v2 v2.10.1/go.mod h1:GGEJuYjrfnPHCQLcxTcIco+Puy63PS2p8QQd8FRw66I=
github.com/couchbase/gocbcore/v10 v10.7.1 h1:6jsNDtqyfoQ8Xg6kv99rzccc3CrHbp7FjeY+ahWXTF4=
github.com/couchbase/gocbcore/v10 v10.7.1/go.mod h1:Q8JWVenMCEOuRgrDQKApHbzzPif38HzefGgRVe9apAI=
github.com/couchbase/gocb/v2 v2.11.0 h1:OVB+KlVeXlKVtziKx/LWZT7DClLsoQHQFrI4wan5Ijc=
github.com/couchbase/gocb/v2 v2.11.0/go.mod h1:Y+lODSgyVzDSaf0Sy8sIzIa0RTAw3vlZUsjY6+FUq9Y=
github.com/couchbase/gocbcore/v10 v10.8.0 h1:zDcJyYqOirFyC8T/aVvNL4N9oj6GI4qtaBuTGGWCDb4=
github.com/couchbase/gocbcore/v10 v10.8.0/go.mod h1:OWKfU9R5Nm5V3QZBtfdZl5qCfgxtxTqOgXiNr4pn9/c=
github.com/couchbase/gocbcoreps v0.1.3 h1:fILaKGCjxFIeCgAUG8FGmRDSpdrRggohOMKEgO9CUpg=
github.com/couchbase/gocbcoreps v0.1.3/go.mod h1:hBFpDNPnRno6HH5cRXExhqXYRmTsFJlFHQx7vztcXPk=
github.com/couchbase/goprotostellar v1.0.2 h1:yoPbAL9sCtcyZ5e/DcU5PRMOEFaJrF9awXYu3VPfGls=
@@ -796,8 +802,8 @@ github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/r
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
github.com/docker/cli v26.1.4+incompatible h1:I8PHdc0MtxEADqYJZvhBrW9bo8gawKwwenxRM7/rLu8=
github.com/docker/cli v26.1.4+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/docker v27.1.1+incompatible h1:hO/M4MtV36kzKldqnA37IWhebRA+LnqqcqDja6kVaKY=
github.com/docker/docker v27.1.1+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v28.2.2+incompatible h1:CjwRSksz8Yo4+RmQ339Dp/D2tGO5JxwYeqtMOEe0LDw=
github.com/docker/docker v28.2.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
@@ -840,12 +846,16 @@ github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8
github.com/gabriel-vasile/mimetype v1.4.8 h1:FfZ3gj38NjllZIeJAmMhr+qKL8Wu+nOoI3GqacKw1NM=
github.com/gabriel-vasile/mimetype v1.4.8/go.mod h1:ByKUIKGjh1ODkGM1asKUbQZOLGrPjydw3hYPU2YU9t8=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-chi/chi/v5 v5.2.2 h1:CMwsvRVTbXVytCk1Wd72Zy1LAsAh9GxMmSNWLHCG618=
github.com/go-chi/chi/v5 v5.2.2/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
github.com/go-chi/chi/v5 v5.2.3 h1:WQIt9uxdsAbgIYgid+BpYc+liqQZGMHRaUwp0JUcvdE=
github.com/go-chi/chi/v5 v5.2.3/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
github.com/go-chi/httplog/v2 v2.1.1 h1:ojojiu4PIaoeJ/qAO4GWUxJqvYUTobeo7zmuHQJAxRk=
github.com/go-chi/httplog/v2 v2.1.1/go.mod h1:/XXdxicJsp4BA5fapgIC3VuTD+z0Z/VzukoB3VDc1YE=
github.com/go-chi/render v1.0.3 h1:AsXqd2a1/INaIfUSKq3G5uA8weYx20FOsM7uSoCyyt4=
github.com/go-chi/render v1.0.3/go.mod h1:/gr3hVkmYR0YlEy3LxCuVRFzEu9Ruok+gFqbIofjao0=
github.com/go-faster/city v1.0.1 h1:4WAxSZ3V2Ws4QRDrscLEDcibJY8uf41H6AhXDrNDcGw=
github.com/go-faster/city v1.0.1/go.mod h1:jKcUJId49qdW3L1qKHH/3wPeUstCVpVSXTM6vO3VcTw=
github.com/go-faster/errors v0.7.1 h1:MkJTnDoEdi9pDabt1dpWf7AA8/BaSYZqibYyhZ20AYg=
github.com/go-faster/errors v0.7.1/go.mod h1:5ySTjWFiphBs07IKuiL69nxdfd5+fzh1u7FPGZP2quo=
github.com/go-fonts/dejavu v0.1.0/go.mod h1:4Wt4I4OU2Nq9asgDCteaAaWZOV24E+0/Pwo0gppep4g=
github.com/go-fonts/latin-modern v0.2.0/go.mod h1:rQVLdDMK+mK1xscDwsqM5J8U2jrRa3T0ecnM9pNujks=
github.com/go-fonts/liberation v0.1.1/go.mod h1:K6qoJYypsmfVjWg8KOVDQhLc8UDgIK2HYqyqAO9z7GY=
@@ -933,6 +943,7 @@ github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiu
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v1.0.0 h1:Oy607GVXHs7RtbggtPBnr2RmDArIsAefDwvrdWvRhGs=
@@ -1088,6 +1099,7 @@ github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:C
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/asmfmt v1.3.2/go.mod h1:AG8TuvYojzulgDAMCnYn50l/5QV3Bs/tp6j0HLHbNSE=
github.com/klauspost/compress v1.13.6/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/klauspost/compress v1.15.9/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
@@ -1119,8 +1131,8 @@ github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-sqlite3 v1.14.14/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
github.com/microsoft/go-mssqldb v1.9.2 h1:nY8TmFMQOHpm2qVWo6y4I2mAmVdZqlGiMGAYt64Ibbs=
github.com/microsoft/go-mssqldb v1.9.2/go.mod h1:GBbW9ASTiDC+mpgWDGKdm3FnFLTUsLYN3iFL90lQ+PA=
github.com/microsoft/go-mssqldb v1.9.3 h1:hy4p+LDC8LIGvI3JATnLVmBOLMJbmn5X400mr5j0lPs=
github.com/microsoft/go-mssqldb v1.9.3/go.mod h1:GBbW9ASTiDC+mpgWDGKdm3FnFLTUsLYN3iFL90lQ+PA=
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8/go.mod h1:mC1jAcsrzbxHt8iiaC+zU4b1ylILSosueou12R++wfY=
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3/go.mod h1:RagcQ7I8IeTMnF8JTXieKnO4Z6JCsikNEzj0DwauVzE=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
@@ -1134,6 +1146,7 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe/go.mod h1:wL8QJuTMNUDYhXwkmfOly8iTdp5TEcJFWZD2D7SIkUc=
github.com/montanaflynn/stats v0.7.1 h1:etflOAAHORrCC44V+aR6Ftzort912ZU+YLiSTuV8eaE=
github.com/montanaflynn/stats v0.7.1/go.mod h1:etXPPgVO6n31NxCd9KQUMvCM+ve0ruNzt6R8Bnaayow=
github.com/nakagami/chacha20 v0.1.0 h1:2fbf5KeVUw7oRpAe6/A7DqvBJLYYu0ka5WstFbnkEVo=
@@ -1148,13 +1161,16 @@ github.com/onsi/gomega v1.36.2 h1:koNYke6TVk6ZmnyHrCXba/T/MoLBXFjeC1PtvYgw0A8=
github.com/onsi/gomega v1.36.2/go.mod h1:DdwyADRjrc825LhMEkD76cHR5+pUnjhUN8GlHlRPHzY=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M=
github.com/opencontainers/runc v1.1.13 h1:98S2srgG9vw0zWcDpFMn5TRrh8kLxa/5OFUstuUhmRs=
github.com/opencontainers/runc v1.1.13/go.mod h1:R016aXacfp/gwQBYw2FDGa9m+n6atbLWrYY8hNMT/sA=
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/ory/dockertest/v3 v3.11.0 h1:OiHcxKAvSDUwsEVh2BjxQQc/5EHz9n0va9awCtNGuyA=
github.com/ory/dockertest/v3 v3.11.0/go.mod h1:VIPxS1gwT9NpPOrfD3rACs8Y9Z7yhzO4SB194iUDnUI=
github.com/paulmach/orb v0.11.1 h1:3koVegMC4X/WeiXYz9iswopaTwMem53NzTJuTF20JzU=
github.com/paulmach/orb v0.11.1/go.mod h1:5mULz1xQfs3bmQm63QEJA6lNGujuRafwA5S/EnuLaLU=
github.com/paulmach/protoscan v0.2.1/go.mod h1:SpcSwydNLrxUGSDvXvO0P7g7AuhJ7lcKfDlhJCDw2gY=
github.com/phpdave11/gofpdf v1.4.2/go.mod h1:zpO6xFn9yxo3YLyMvW8HcKWVdbNqgIfOOp2dXMnm1mY=
github.com/phpdave11/gofpdi v1.0.12/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/7bXwjDoI=
github.com/phpdave11/gofpdi v1.0.13/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/7bXwjDoI=
@@ -1193,8 +1209,10 @@ github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWN
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/ruudk/golang-pdf417 v0.0.0-20181029194003-1af4ab5afa58/go.mod h1:6lfFZQK844Gfx8o5WFuvpxWRwnSoipWe/p622j1v06w=
github.com/ruudk/golang-pdf417 v0.0.0-20201230142125-a7e3863a1245/go.mod h1:pQAZKsJ8yyVxGRWYNEm9oFB8ieLgKFnamEyDmSA0BRk=
github.com/shopspring/decimal v1.2.0 h1:abSATXmQEYyShuxI4/vyW3tV1MrKAJzCZ/0zLUXYbsQ=
github.com/shopspring/decimal v1.2.0/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFRcu2hWCYk4o=
github.com/segmentio/asm v1.2.0 h1:9BQrFxC+YOHJlTlHGkTrFWf59nbL3XnCoFLTwDCI7ys=
github.com/segmentio/asm v1.2.0/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs=
github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k=
github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
@@ -1228,14 +1246,17 @@ github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOf
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/thlib/go-timezone-local v0.0.7 h1:fX8zd3aJydqLlTs/TrROrIIdztzsdFV23OzOQx31jII=
github.com/thlib/go-timezone-local v0.0.7/go.mod h1:/Tnicc6m/lsJE0irFMA0LfIwTBo4QP7A8IfyIv4zZKI=
github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
github.com/trinodb/trino-go-client v0.328.0 h1:X6hrGGysA3nvyVcz8kJbBS98srLNTNsnNYwRkMC1atA=
github.com/trinodb/trino-go-client v0.328.0/go.mod h1:e/nck9W6hy+9bbyZEpXKFlNsufn3lQGpUgDL1d5f1FI=
github.com/valkey-io/valkey-go v1.0.64 h1:3u4+b6D6zs9JQs254TLy4LqitCMHHr9XorP9GGk7XY4=
github.com/valkey-io/valkey-go v1.0.64/go.mod h1:bHmwjIEOrGq/ubOJfh5uMRs7Xj6mV3mQ/ZXUbmqpjqY=
github.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c=
github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=
github.com/xdg-go/scram v1.1.1/go.mod h1:RaEWvsqvNKKvBPvcKeFjrG2cJqOkHTiyTpzz23ni57g=
github.com/xdg-go/scram v1.1.2 h1:FHX5I5B4i4hKRVRBCFRxq1iQRej7WO3hhBuJf+UUySY=
github.com/xdg-go/scram v1.1.2/go.mod h1:RT/sEzTbU5y00aCK8UOx6R7YryM0iF1N2MOmC3kKLN4=
github.com/xdg-go/stringprep v1.0.3/go.mod h1:W3f5j4i+9rC0kuIEJL0ky1VpHXQU3ocBgklLGvcBnW8=
github.com/xdg-go/stringprep v1.0.4 h1:XLI/Ng3O1Atzq0oBs3TWm+5ZVgkq2aqdlvP9JtoZ6c8=
github.com/xdg-go/stringprep v1.0.4/go.mod h1:mPGuuIYwz7CmR2bT9j4GbQqutWS1zV24gijq1dTyGkM=
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb h1:zGWFAtiMcyryUHoUjUJX0/lt1H2+i2Ka2n+D3DImSNo=
@@ -1244,6 +1265,9 @@ github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHo
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74=
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d/go.mod h1:rHwXgn7JulP+udvsHwJoVG1YGAP6VLg4y9I5dyZdqmA=
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 h1:ilQV1hzziu+LLM3zUTJ0trRztfwgjqKnBWNtSRkbmwM=
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78/go.mod h1:aL8wCCfTfSfmXjznFBSZNN13rSJjlIOI1fUNAtF7rmI=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
@@ -1261,6 +1285,7 @@ github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
gitlab.com/nyarla/go-crypt v0.0.0-20160106005555-d9a5dc2b789b h1:7gd+rd8P3bqcn/96gOZa3F5dpJr/vEiDQYlNb/y2uNs=
gitlab.com/nyarla/go-crypt v0.0.0-20160106005555-d9a5dc2b789b/go.mod h1:T3BPAOm2cqquPa0MKWeNkmOM5RQsRhkrwMWonFMN7fE=
go.mongodb.org/mongo-driver v1.11.4/go.mod h1:PTSz5yu21bkT/wXpkS7WR5f0ddqw5quethTUn9WM+2g=
go.mongodb.org/mongo-driver v1.17.4 h1:jUorfmVzljjr0FLzYQsGP8cgN/qzzxlY9Vh0C9KFXVw=
go.mongodb.org/mongo-driver v1.17.4/go.mod h1:Hy04i7O2kC4RS06ZrhPRqj/u4DTYkFDAAccj+rVKqgQ=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
@@ -1276,8 +1301,8 @@ go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJyS
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/detectors/gcp v1.36.0 h1:F7q2tNlCaHY9nMKHR6XH9/qkp8FktLnIcy6jJNyOCQw=
go.opentelemetry.io/contrib/detectors/gcp v1.36.0/go.mod h1:IbBN8uAIIx734PTonTPxAxnjc2pQTxWNkwfstZ+6H2k=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 h1:q4XOmH/0opmeuJtPsbFNivyl7bCt7yRBbeEm2sC/XtQ=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0/go.mod h1:snMWehoOh2wsEwnvvwtDyFCxVeDAODenXHtn5vzrKjo=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.62.0 h1:rbRJ8BBoVMsQShESYZ0FkvcITu8X8QNwJogcLUmDNNw=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.62.0/go.mod h1:ru6KHrNtNHxM4nD/vd6QrLVWgKhxPYgblq4VAtNawTQ=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0 h1:Hf9xI/XLML9ElpiHVDNwvqI0hIFlzV8dgIr35kV1kRU=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0/go.mod h1:NfchwuyNoMcZ5MLHwPrODwUF1HWCXWrL31s8gSAdIKY=
go.opentelemetry.io/contrib/propagators/autoprop v0.62.0 h1:1+EHlhAe/tukctfePZRrDruB9vn7MdwyC+rf36nUSPM=
@@ -1330,6 +1355,7 @@ golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58=
golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
@@ -1439,6 +1465,7 @@ golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96b
golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220325170049-de3da57026de/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
@@ -1936,10 +1963,10 @@ google.golang.org/genproto v0.0.0-20230323212658-478b75c54725/go.mod h1:UUQDJDOl
google.golang.org/genproto v0.0.0-20230330154414-c0448cd141ea/go.mod h1:UUQDJDOlWu4KYeJZffbWgBkS1YFobzKbLVfK69pe0Ak=
google.golang.org/genproto v0.0.0-20230331144136-dcfb400f0633/go.mod h1:UUQDJDOlWu4KYeJZffbWgBkS1YFobzKbLVfK69pe0Ak=
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1/go.mod h1:nKE/iIaLqn2bQwXBg8f1g2Ylh6r5MN5CmZvuzZCgsCU=
google.golang.org/genproto v0.0.0-20250818200422-3122310a409c h1:ZERoum3uuqL0PRSc6SXielu26FN96T4BUGaaW0oL+c8=
google.golang.org/genproto v0.0.0-20250818200422-3122310a409c/go.mod h1:Q8kep885BJnK3Jt6QZXIFeLHSzoAQtlI1CCloQigiyU=
google.golang.org/genproto/googleapis/api v0.0.0-20250811230008-5f3141c8851a h1:DMCgtIAIQGZqJXMVzJF4MV8BlWoJh2ZuFiRdAleyr58=
google.golang.org/genproto/googleapis/api v0.0.0-20250811230008-5f3141c8851a/go.mod h1:y2yVLIE/CSMCPXaHnSKXxu1spLPnglFLegmgdY23uuE=
google.golang.org/genproto v0.0.0-20250826171959-ef028d996bc1 h1:Nm5SEGIguOIBDXs5rhfz2aKwEVWlgwC58UcmEnLDc8Y=
google.golang.org/genproto v0.0.0-20250826171959-ef028d996bc1/go.mod h1:Jz9LrroM7Mcm+a0QrLh4UpZ1B/WhjIbqwEcUf4y08nQ=
google.golang.org/genproto/googleapis/api v0.0.0-20250818200422-3122310a409c h1:AtEkQdl5b6zsybXcbz00j1LwNodDuH6hVifIaNqk7NQ=
google.golang.org/genproto/googleapis/api v0.0.0-20250818200422-3122310a409c/go.mod h1:ea2MjsO70ssTfCjiwHgI0ZFqcw45Ksuk2ckf9G468GA=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c h1:qXWI/sQtv5UKboZ/zUk7h+mrf/lXORyI+n9DKDAusdg=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c/go.mod h1:gw1tLEfykwDz2ET4a12jcXt4couGAm7IwsVaTy0Sflo=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
@@ -2003,10 +2030,11 @@ google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqw
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.29.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.36.7 h1:IgrO7UwFQGJdRNXH/sQux4R1Dj1WAKcLElzeeRaXV2A=
google.golang.org/protobuf v1.36.7/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc=
google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/ini.v1 v1.61.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=

View File

@@ -24,6 +24,7 @@ var expectedToolSources = []string{
"alloydb-postgres-admin",
"alloydb-postgres",
"bigquery",
"clickhouse",
"cloud-sql-mssql",
"cloud-sql-mysql",
"cloud-sql-postgres",
@@ -85,6 +86,7 @@ func TestGetPrebuiltTool(t *testing.T) {
alloydb_admin_config, _ := Get("alloydb-postgres-admin")
alloydb_config, _ := Get("alloydb-postgres")
bigquery_config, _ := Get("bigquery")
clickhouse_config, _ := Get("clickhouse")
cloudsqlpg_config, _ := Get("cloud-sql-postgres")
cloudsqlmysql_config, _ := Get("cloud-sql-mysql")
cloudsqlmssql_config, _ := Get("cloud-sql-mssql")
@@ -105,6 +107,9 @@ func TestGetPrebuiltTool(t *testing.T) {
if len(bigquery_config) <= 0 {
t.Fatalf("unexpected error: could not fetch bigquery prebuilt tools yaml")
}
if len(clickhouse_config) <= 0 {
t.Fatalf("unexpected error: could not fetch clickhouse prebuilt tools yaml")
}
if len(cloudsqlpg_config) <= 0 {
t.Fatalf("unexpected error: could not fetch cloud sql pg prebuilt tools yaml")
}

View File

@@ -4,6 +4,14 @@ sources:
project: ${BIGQUERY_PROJECT}
tools:
ask_data_insights:
kind: bigquery-conversational-analytics
source: bigquery-source
description: |
Use this tool to perform data analysis, get insights,
or answer complex questions about the contents of specific
BigQuery tables.
execute_sql:
kind: bigquery-execute-sql
source: bigquery-source
@@ -36,6 +44,7 @@ tools:
toolsets:
bigquery-database-tools:
- ask_data_insights
- execute_sql
- forecast
- get_dataset_info

View File

@@ -0,0 +1,19 @@
sources:
clickhouse-source:
kind: clickhouse
host: ${CLICKHOUSE_HOST}
port: ${CLICKHOUSE_PORT}
user: ${CLICKHOUSE_USER}
password: ${CLICKHOUSE_PASSWORD}
database: ${CLICKHOUSE_DATABASE}
protocol: ${CLICKHOUSE_PROTOCOL}
tools:
execute_sql:
kind: clickhouse-execute-sql
source: clickhouse-source
description: Use this tool to execute SQL.
toolsets:
clickhouse-database-tools:
- execute_sql

View File

@@ -4,8 +4,11 @@ sources:
base_url: ${LOOKER_BASE_URL}
client_id: ${LOOKER_CLIENT_ID}
client_secret: ${LOOKER_CLIENT_SECRET}
verify_ssl: ${LOOKER_VERIFY_SSL}
verify_ssl: ${LOOKER_VERIFY_SSL:true}
timeout: 600s
show_hidden_models: ${LOOKER_SHOW_HIDDEN_MODELS:true}
show_hidden_explores: ${LOOKER_SHOW_HIDDEN_EXPLORES:true}
show_hidden_fields: ${LOOKER_SHOW_HIDDEN_FIELDS:true}
tools:
get_models:
@@ -88,7 +91,8 @@ tools:
Filters are provided as a map of {"field.id": "condition",
"field.id2": "condition2", ...}. Do not put the field.id in
quotes. Filter expressions can be found at
https://cloud.google.com/looker/docs/filter-expressions.
https://cloud.google.com/looker/docs/filter-expressions. There
is one mistake in that, however, Use `not null` instead of `-NULL`.
Sorts can be specified like [ "field.id desc 0" ].
@@ -215,6 +219,7 @@ tools:
### Pie / Donut
* Pie charts must have exactly one dimension and one numerical measure.
* `type`: Must be `looker_pie`.
* `value_labels`: Where to display values (`'legend'`, `'labels'`).
* `label_type`: The format of data labels (`'labPer'`, `'labVal'`, `'lab'`, `'val'`, `'per'`).

View File

@@ -16,8 +16,10 @@ package server
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
@@ -165,6 +167,20 @@ func toolInvokeHandler(s *Server, w http.ResponseWriter, r *http.Request) {
return
}
// Extract OAuth access token from the "Authorization" header (currently for
// BigQuery end-user credentials usage only)
accessToken := tools.AccessToken(r.Header.Get("Authorization"))
// Check if this specific tool requires the standard authorization header
if tool.RequiresClientAuthorization() {
if accessToken == "" {
err = fmt.Errorf("tool requires client authorization but access token is missing from the request header")
s.logger.DebugContext(ctx, err.Error())
_ = render.Render(w, r, newErrResponse(err, http.StatusUnauthorized))
return
}
}
// Tool authentication
// claimsFromAuth maps the name of the authservice to the claims retrieved from it.
claimsFromAuth := make(map[string]map[string]any)
@@ -210,6 +226,12 @@ func toolInvokeHandler(s *Server, w http.ResponseWriter, r *http.Request) {
params, err := tool.ParseParams(data, claimsFromAuth)
if err != nil {
// If auth error, return 401
if errors.Is(err, tools.ErrUnauthorized) {
s.logger.DebugContext(ctx, fmt.Sprintf("error parsing authenticated parameters from ID token: %s", err))
_ = render.Render(w, r, newErrResponse(err, http.StatusUnauthorized))
return
}
err = fmt.Errorf("provided parameters were invalid: %w", err)
s.logger.DebugContext(ctx, err.Error())
_ = render.Render(w, r, newErrResponse(err, http.StatusBadRequest))
@@ -217,12 +239,34 @@ func toolInvokeHandler(s *Server, w http.ResponseWriter, r *http.Request) {
}
s.logger.DebugContext(ctx, fmt.Sprintf("invocation params: %s", params))
// Extract OAuth access token from the "Authorization" header (currently for
// BigQuery end-user credentials usage only)
accessToken := tools.AccessToken(r.Header.Get("Authorization"))
res, err := tool.Invoke(ctx, params, accessToken)
// Determine what error to return to the users.
if err != nil {
errStr := err.Error()
var statusCode int
// Upstream API auth error propagation
switch {
case strings.Contains(errStr, "Error 401"):
statusCode = http.StatusUnauthorized
case strings.Contains(errStr, "Error 403"):
statusCode = http.StatusForbidden
}
if statusCode == http.StatusUnauthorized || statusCode == http.StatusForbidden {
if tool.RequiresClientAuthorization() {
// Propagate the original 401/403 error.
s.logger.DebugContext(ctx, fmt.Sprintf("error invoking tool. Client credentials lack authorization to the source: %v", err))
_ = render.Render(w, r, newErrResponse(err, statusCode))
return
}
// ADC lacking permission or credentials configuration error.
internalErr := fmt.Errorf("unexpected auth error occured during Tool invocation: %w", err)
s.logger.ErrorContext(ctx, internalErr.Error())
_ = render.Render(w, r, newErrResponse(internalErr, http.StatusInternalServerError))
return
}
err = fmt.Errorf("error while invoking tool: %w", err)
s.logger.DebugContext(ctx, err.Error())
_ = render.Render(w, r, newErrResponse(err, http.StatusBadRequest))

View File

@@ -212,7 +212,7 @@ func TestToolGetEndpoint(t *testing.T) {
}
func TestToolInvokeEndpoint(t *testing.T) {
mockTools := []MockTool{tool1, tool2}
mockTools := []MockTool{tool1, tool2, tool4, tool5}
toolsMap, toolsets := setUpResources(t, mockTools)
r, shutdown := setUpServer(t, "api", toolsMap, toolsets)
defer shutdown()
@@ -247,6 +247,20 @@ func TestToolInvokeEndpoint(t *testing.T) {
want: "",
isErr: true,
},
{
name: "tool4",
toolName: tool4.Name,
requestBody: bytes.NewBuffer([]byte(`{}`)),
want: "",
isErr: true,
},
{
name: "tool5",
toolName: tool5.Name,
requestBody: bytes.NewBuffer([]byte(`{}`)),
want: "",
isErr: true,
},
}
for _, tc := range testCases {

View File

@@ -36,10 +36,12 @@ var _ tools.Tool = &MockTool{}
// MockTool is used to mock tools in tests
type MockTool struct {
Name string
Description string
Params []tools.Parameter
manifest tools.Manifest
Name string
Description string
Params []tools.Parameter
manifest tools.Manifest
unauthorized bool
requiresClientAuthrorization bool
}
func (t MockTool) Invoke(context.Context, tools.ParamValues, tools.AccessToken) (any, error) {
@@ -59,12 +61,15 @@ func (t MockTool) Manifest() tools.Manifest {
}
return tools.Manifest{Description: t.Description, Parameters: pMs}
}
func (t MockTool) Authorized(verifiedAuthServices []string) bool {
return true
// defaulted to true
return !t.unauthorized
}
func (t MockTool) RequiresClientAuthorization() bool {
return false
// defaulted to false
return t.requiresClientAuthrorization
}
func (t MockTool) McpManifest() tools.McpManifest {
@@ -111,6 +116,18 @@ var tool3 = MockTool{
},
}
var tool4 = MockTool{
Name: "unauthorized_tool",
Params: []tools.Parameter{},
unauthorized: true,
}
var tool5 = MockTool{
Name: "require_client_auth_tool",
Params: []tools.Parameter{},
requiresClientAuthrorization: true,
}
// setUpResources setups resources to test against
func setUpResources(t *testing.T, mockTools []MockTool) (map[string]tools.Tool, map[string]tools.Toolset) {
toolsMap := make(map[string]tools.Tool)

View File

@@ -218,6 +218,11 @@ func (c *ToolConfigs) UnmarshalYAML(ctx context.Context, unmarshal func(interfac
return fmt.Errorf("unable to unmarshal %q: %w", name, err)
}
// `authRequired` and `useClientOAuth` cannot be specified together
if v["authRequired"] != nil && v["useClientOAuth"] == true {
return fmt.Errorf("`authRequired` and `useClientOAuth` are mutually exclusive. Choose only one authentication method")
}
// Make `authRequired` an empty list instead of nil for Tool manifest
if v["authRequired"] == nil {
v["authRequired"] = []string{}

View File

@@ -19,9 +19,11 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
"sync"
"time"
@@ -142,7 +144,7 @@ func (s *stdioSession) readInputStream(ctx context.Context) error {
}
return err
}
v, res, err := processMcpMessage(ctx, []byte(line), s.server, s.protocol, "", "")
v, res, err := processMcpMessage(ctx, []byte(line), s.server, s.protocol, "", nil)
if err != nil {
// errors during the processing of message will generate a valid MCP Error response.
// server can continue to run.
@@ -330,6 +332,8 @@ func methodNotAllowed(s *Server, w http.ResponseWriter, r *http.Request) {
// httpHandler handles all mcp messages.
func httpHandler(s *Server, w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
ctx, span := s.instrumentation.Tracer.Start(r.Context(), "toolbox/server/mcp")
r = r.WithContext(ctx)
ctx = util.WithLogger(r.Context(), s.logger)
@@ -402,9 +406,11 @@ func httpHandler(s *Server, w http.ResponseWriter, r *http.Request) {
return
}
accessToken := tools.AccessToken(r.Header.Get("Authorization"))
v, res, err := processMcpMessage(ctx, body, s, protocolVersion, toolsetName, r.Header)
if err != nil {
s.logger.DebugContext(ctx, fmt.Errorf("error processing message: %w", err).Error())
}
v, res, err := processMcpMessage(ctx, body, s, protocolVersion, toolsetName, accessToken)
// notifications will return empty string
if res == nil {
// Notifications do not expect a response
@@ -412,9 +418,6 @@ func httpHandler(s *Server, w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusAccepted)
return
}
if err != nil {
s.logger.DebugContext(ctx, err.Error())
}
// for v20250326, add the `Mcp-Session-Id` header
if v == v20250326.PROTOCOL_VERSION {
@@ -434,13 +437,29 @@ func httpHandler(s *Server, w http.ResponseWriter, r *http.Request) {
s.logger.DebugContext(ctx, "unable to add to event queue")
}
}
if rpcResponse, ok := res.(jsonrpc.JSONRPCError); ok {
code := rpcResponse.Error.Code
switch code {
case jsonrpc.INTERNAL_ERROR:
w.WriteHeader(http.StatusInternalServerError)
case jsonrpc.INVALID_REQUEST:
errStr := err.Error()
if errors.Is(err, tools.ErrUnauthorized) {
w.WriteHeader(http.StatusUnauthorized)
} else if strings.Contains(errStr, "Error 401") {
w.WriteHeader(http.StatusUnauthorized)
} else if strings.Contains(errStr, "Error 403") {
w.WriteHeader(http.StatusForbidden)
}
}
}
// send HTTP response
render.JSON(w, r, res)
}
// processMcpMessage process the messages received from clients
func processMcpMessage(ctx context.Context, body []byte, s *Server, protocolVersion string, toolsetName string, accessToken tools.AccessToken) (string, any, error) {
func processMcpMessage(ctx context.Context, body []byte, s *Server, protocolVersion string, toolsetName string, header http.Header) (string, any, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return "", jsonrpc.NewError("", jsonrpc.INTERNAL_ERROR, err.Error(), nil), err
@@ -495,7 +514,7 @@ func processMcpMessage(ctx context.Context, body []byte, s *Server, protocolVers
err = fmt.Errorf("toolset does not exist")
return "", jsonrpc.NewError(baseMessage.Id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
res, err := mcp.ProcessMethod(ctx, protocolVersion, baseMessage.Id, baseMessage.Method, toolset, s.ResourceMgr.GetToolsMap(), body, accessToken)
res, err := mcp.ProcessMethod(ctx, protocolVersion, baseMessage.Id, baseMessage.Method, toolset, s.ResourceMgr.GetToolsMap(), s.ResourceMgr.GetAuthServiceMap(), body, header)
return "", res, err
}
}

View File

@@ -0,0 +1,39 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jsonrpc
import (
"reflect"
"testing"
)
func TestNewError(t *testing.T) {
var id interface{} = "foo"
code := 111
message := "foo bar"
want := JSONRPCError{
Jsonrpc: "2.0",
Id: "foo",
Error: Error{
Code: 111,
Message: "foo bar",
},
}
got := NewError(id, code, message, nil)
if !reflect.DeepEqual(want, got) {
t.Fatalf("unexpected error: got %+v, want %+v", got, want)
}
}

View File

@@ -18,8 +18,10 @@ import (
"context"
"encoding/json"
"fmt"
"net/http"
"slices"
"github.com/googleapis/genai-toolbox/internal/auth"
"github.com/googleapis/genai-toolbox/internal/server/mcp/jsonrpc"
mcputil "github.com/googleapis/genai-toolbox/internal/server/mcp/util"
v20241105 "github.com/googleapis/genai-toolbox/internal/server/mcp/v20241105"
@@ -93,14 +95,14 @@ func NotificationHandler(ctx context.Context, body []byte) error {
// ProcessMethod returns a response for the request.
// This is the Operation phase of the lifecycle for MCP client-server connections.
func ProcessMethod(ctx context.Context, mcpVersion string, id jsonrpc.RequestId, method string, toolset tools.Toolset, tools map[string]tools.Tool, body []byte, accessToken tools.AccessToken) (any, error) {
func ProcessMethod(ctx context.Context, mcpVersion string, id jsonrpc.RequestId, method string, toolset tools.Toolset, tools map[string]tools.Tool, authServices map[string]auth.AuthService, body []byte, header http.Header) (any, error) {
switch mcpVersion {
case v20250618.PROTOCOL_VERSION:
return v20250618.ProcessMethod(ctx, id, method, toolset, tools, body, accessToken)
return v20250618.ProcessMethod(ctx, id, method, toolset, tools, authServices, body, header)
case v20250326.PROTOCOL_VERSION:
return v20250326.ProcessMethod(ctx, id, method, toolset, tools, body, accessToken)
return v20250326.ProcessMethod(ctx, id, method, toolset, tools, authServices, body, header)
default:
return v20241105.ProcessMethod(ctx, id, method, toolset, tools, body, accessToken)
return v20241105.ProcessMethod(ctx, id, method, toolset, tools, authServices, body, header)
}
}

View File

@@ -14,7 +14,9 @@
package util
import "github.com/googleapis/genai-toolbox/internal/server/mcp/jsonrpc"
import (
"github.com/googleapis/genai-toolbox/internal/server/mcp/jsonrpc"
)
const (
// SERVER_NAME is the server name used in Implementation.

View File

@@ -18,22 +18,26 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"github.com/googleapis/genai-toolbox/internal/auth"
"github.com/googleapis/genai-toolbox/internal/server/mcp/jsonrpc"
"github.com/googleapis/genai-toolbox/internal/tools"
"github.com/googleapis/genai-toolbox/internal/util"
)
// ProcessMethod returns a response for the request.
func ProcessMethod(ctx context.Context, id jsonrpc.RequestId, method string, toolset tools.Toolset, tools map[string]tools.Tool, body []byte, accessToken tools.AccessToken) (any, error) {
func ProcessMethod(ctx context.Context, id jsonrpc.RequestId, method string, toolset tools.Toolset, tools map[string]tools.Tool, authServices map[string]auth.AuthService, body []byte, header http.Header) (any, error) {
switch method {
case PING:
return pingHandler(id)
case TOOLS_LIST:
return toolsListHandler(id, toolset, body)
case TOOLS_CALL:
return toolsCallHandler(ctx, id, tools, body, accessToken)
return toolsCallHandler(ctx, id, tools, authServices, body, header)
default:
err := fmt.Errorf("invalid method %s", method)
return jsonrpc.NewError(id, jsonrpc.METHOD_NOT_FOUND, err.Error(), nil), err
@@ -67,7 +71,7 @@ func toolsListHandler(id jsonrpc.RequestId, toolset tools.Toolset, body []byte)
}
// toolsCallHandler generate a response for tools call.
func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[string]tools.Tool, body []byte, accessToken tools.AccessToken) (any, error) {
func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, toolsMap map[string]tools.Tool, authServices map[string]auth.AuthService, body []byte, header http.Header) (any, error) {
// retrieve logger from context
logger, err := util.LoggerFromContext(ctx)
if err != nil {
@@ -83,12 +87,22 @@ func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[strin
toolName := req.Params.Name
toolArgument := req.Params.Arguments
logger.DebugContext(ctx, fmt.Sprintf("tool name: %s", toolName))
tool, ok := tools[toolName]
tool, ok := toolsMap[toolName]
if !ok {
err = fmt.Errorf("invalid tool name: tool with name %q does not exist", toolName)
return jsonrpc.NewError(id, jsonrpc.INVALID_PARAMS, err.Error(), nil), err
}
// Get access token
accessToken := tools.AccessToken(header.Get("Authorization"))
// Check if this specific tool requires the standard authorization header
if tool.RequiresClientAuthorization() {
if accessToken == "" {
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, "missing access token in the 'Authorization' header", nil), tools.ErrUnauthorized
}
}
// marshal arguments and decode it using decodeJSON instead to prevent loss between floats/int.
aMarshal, err := json.Marshal(toolArgument)
if err != nil {
@@ -102,10 +116,42 @@ func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[strin
return jsonrpc.NewError(id, jsonrpc.INTERNAL_ERROR, err.Error(), nil), err
}
// Tool authentication
// claimsFromAuth maps the name of the authservice to the claims retrieved from it.
// Since MCP doesn't support auth, an empty map will be use every time.
claimsFromAuth := make(map[string]map[string]any)
// if using stdio, header will be nil and auth will not be supported
if header != nil {
for _, aS := range authServices {
claims, err := aS.GetClaimsFromHeader(ctx, header)
if err != nil {
logger.DebugContext(ctx, err.Error())
continue
}
if claims == nil {
// authService not present in header
continue
}
claimsFromAuth[aS.GetName()] = claims
}
}
// Tool authorization check
verifiedAuthServices := make([]string, len(claimsFromAuth))
i := 0
for k := range claimsFromAuth {
verifiedAuthServices[i] = k
i++
}
// Check if any of the specified auth services is verified
isAuthorized := tool.Authorized(verifiedAuthServices)
if !isAuthorized {
err = fmt.Errorf("unauthorized Tool call: Please make sure your specify correct auth headers: %w", tools.ErrUnauthorized)
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
logger.DebugContext(ctx, "tool invocation authorized")
params, err := tool.ParseParams(data, claimsFromAuth)
if err != nil {
err = fmt.Errorf("provided parameters were invalid: %w", err)
@@ -113,14 +159,24 @@ func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[strin
}
logger.DebugContext(ctx, fmt.Sprintf("invocation params: %s", params))
if !tool.Authorized([]string{}) {
err = fmt.Errorf("unauthorized Tool call: `authRequired` is set for the target Tool")
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
// run tool invocation and generate response.
results, err := tool.Invoke(ctx, params, accessToken)
if err != nil {
errStr := err.Error()
// Missing authService tokens.
if errors.Is(err, tools.ErrUnauthorized) {
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
// Upstream auth error
if strings.Contains(errStr, "Error 401") || strings.Contains(errStr, "Error 403") {
if tool.RequiresClientAuthorization() {
// Error with client credentials should pass down to the client
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
// Auth error with ADC should raise internal 500 error
return jsonrpc.NewError(id, jsonrpc.INTERNAL_ERROR, err.Error(), nil), err
}
text := TextContent{
Type: "text",
Text: err.Error(),

View File

@@ -18,22 +18,26 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"github.com/googleapis/genai-toolbox/internal/auth"
"github.com/googleapis/genai-toolbox/internal/server/mcp/jsonrpc"
"github.com/googleapis/genai-toolbox/internal/tools"
"github.com/googleapis/genai-toolbox/internal/util"
)
// ProcessMethod returns a response for the request.
func ProcessMethod(ctx context.Context, id jsonrpc.RequestId, method string, toolset tools.Toolset, tools map[string]tools.Tool, body []byte, accessToken tools.AccessToken) (any, error) {
func ProcessMethod(ctx context.Context, id jsonrpc.RequestId, method string, toolset tools.Toolset, tools map[string]tools.Tool, authServices map[string]auth.AuthService, body []byte, header http.Header) (any, error) {
switch method {
case PING:
return pingHandler(id)
case TOOLS_LIST:
return toolsListHandler(id, toolset, body)
case TOOLS_CALL:
return toolsCallHandler(ctx, id, tools, body, accessToken)
return toolsCallHandler(ctx, id, tools, authServices, body, header)
default:
err := fmt.Errorf("invalid method %s", method)
return jsonrpc.NewError(id, jsonrpc.METHOD_NOT_FOUND, err.Error(), nil), err
@@ -67,7 +71,7 @@ func toolsListHandler(id jsonrpc.RequestId, toolset tools.Toolset, body []byte)
}
// toolsCallHandler generate a response for tools call.
func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[string]tools.Tool, body []byte, accessToken tools.AccessToken) (any, error) {
func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, toolsMap map[string]tools.Tool, authServices map[string]auth.AuthService, body []byte, header http.Header) (any, error) {
// retrieve logger from context
logger, err := util.LoggerFromContext(ctx)
if err != nil {
@@ -83,12 +87,22 @@ func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[strin
toolName := req.Params.Name
toolArgument := req.Params.Arguments
logger.DebugContext(ctx, fmt.Sprintf("tool name: %s", toolName))
tool, ok := tools[toolName]
tool, ok := toolsMap[toolName]
if !ok {
err = fmt.Errorf("invalid tool name: tool with name %q does not exist", toolName)
return jsonrpc.NewError(id, jsonrpc.INVALID_PARAMS, err.Error(), nil), err
}
// Get access token
accessToken := tools.AccessToken(header.Get("Authorization"))
// Check if this specific tool requires the standard authorization header
if tool.RequiresClientAuthorization() {
if accessToken == "" {
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, "missing access token in the 'Authorization' header", nil), tools.ErrUnauthorized
}
}
// marshal arguments and decode it using decodeJSON instead to prevent loss between floats/int.
aMarshal, err := json.Marshal(toolArgument)
if err != nil {
@@ -102,10 +116,42 @@ func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[strin
return jsonrpc.NewError(id, jsonrpc.INTERNAL_ERROR, err.Error(), nil), err
}
// Tool authentication
// claimsFromAuth maps the name of the authservice to the claims retrieved from it.
// Since MCP doesn't support auth, an empty map will be use every time.
claimsFromAuth := make(map[string]map[string]any)
// if using stdio, header will be nil and auth will not be supported
if header != nil {
for _, aS := range authServices {
claims, err := aS.GetClaimsFromHeader(ctx, header)
if err != nil {
logger.DebugContext(ctx, err.Error())
continue
}
if claims == nil {
// authService not present in header
continue
}
claimsFromAuth[aS.GetName()] = claims
}
}
// Tool authorization check
verifiedAuthServices := make([]string, len(claimsFromAuth))
i := 0
for k := range claimsFromAuth {
verifiedAuthServices[i] = k
i++
}
// Check if any of the specified auth services is verified
isAuthorized := tool.Authorized(verifiedAuthServices)
if !isAuthorized {
err = fmt.Errorf("unauthorized Tool call: Please make sure your specify correct auth headers: %w", tools.ErrUnauthorized)
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
logger.DebugContext(ctx, "tool invocation authorized")
params, err := tool.ParseParams(data, claimsFromAuth)
if err != nil {
err = fmt.Errorf("provided parameters were invalid: %w", err)
@@ -113,14 +159,23 @@ func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[strin
}
logger.DebugContext(ctx, fmt.Sprintf("invocation params: %s", params))
if !tool.Authorized([]string{}) {
err = fmt.Errorf("unauthorized Tool call: `authRequired` is set for the target Tool")
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
// run tool invocation and generate response.
results, err := tool.Invoke(ctx, params, accessToken)
if err != nil {
errStr := err.Error()
// Missing authService tokens.
if errors.Is(err, tools.ErrUnauthorized) {
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
// Upstream auth error
if strings.Contains(errStr, "Error 401") || strings.Contains(errStr, "Error 403") {
if tool.RequiresClientAuthorization() {
// Error with client credentials should pass down to the client
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
// Auth error with ADC should raise internal 500 error
return jsonrpc.NewError(id, jsonrpc.INTERNAL_ERROR, err.Error(), nil), err
}
text := TextContent{
Type: "text",
Text: err.Error(),

View File

@@ -18,22 +18,26 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"github.com/googleapis/genai-toolbox/internal/auth"
"github.com/googleapis/genai-toolbox/internal/server/mcp/jsonrpc"
"github.com/googleapis/genai-toolbox/internal/tools"
"github.com/googleapis/genai-toolbox/internal/util"
)
// ProcessMethod returns a response for the request.
func ProcessMethod(ctx context.Context, id jsonrpc.RequestId, method string, toolset tools.Toolset, tools map[string]tools.Tool, body []byte, accessToken tools.AccessToken) (any, error) {
func ProcessMethod(ctx context.Context, id jsonrpc.RequestId, method string, toolset tools.Toolset, tools map[string]tools.Tool, authServices map[string]auth.AuthService, body []byte, header http.Header) (any, error) {
switch method {
case PING:
return pingHandler(id)
case TOOLS_LIST:
return toolsListHandler(id, toolset, body)
case TOOLS_CALL:
return toolsCallHandler(ctx, id, tools, body, accessToken)
return toolsCallHandler(ctx, id, tools, authServices, body, header)
default:
err := fmt.Errorf("invalid method %s", method)
return jsonrpc.NewError(id, jsonrpc.METHOD_NOT_FOUND, err.Error(), nil), err
@@ -67,7 +71,7 @@ func toolsListHandler(id jsonrpc.RequestId, toolset tools.Toolset, body []byte)
}
// toolsCallHandler generate a response for tools call.
func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[string]tools.Tool, body []byte, accessToken tools.AccessToken) (any, error) {
func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, toolsMap map[string]tools.Tool, authServices map[string]auth.AuthService, body []byte, header http.Header) (any, error) {
// retrieve logger from context
logger, err := util.LoggerFromContext(ctx)
if err != nil {
@@ -83,12 +87,22 @@ func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[strin
toolName := req.Params.Name
toolArgument := req.Params.Arguments
logger.DebugContext(ctx, fmt.Sprintf("tool name: %s", toolName))
tool, ok := tools[toolName]
tool, ok := toolsMap[toolName]
if !ok {
err = fmt.Errorf("invalid tool name: tool with name %q does not exist", toolName)
return jsonrpc.NewError(id, jsonrpc.INVALID_PARAMS, err.Error(), nil), err
}
// Get access token
accessToken := tools.AccessToken(header.Get("Authorization"))
// Check if this specific tool requires the standard authorization header
if tool.RequiresClientAuthorization() {
if accessToken == "" {
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, "missing access token in the 'Authorization' header", nil), tools.ErrUnauthorized
}
}
// marshal arguments and decode it using decodeJSON instead to prevent loss between floats/int.
aMarshal, err := json.Marshal(toolArgument)
if err != nil {
@@ -102,10 +116,42 @@ func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[strin
return jsonrpc.NewError(id, jsonrpc.INTERNAL_ERROR, err.Error(), nil), err
}
// Tool authentication
// claimsFromAuth maps the name of the authservice to the claims retrieved from it.
// Since MCP doesn't support auth, an empty map will be use every time.
claimsFromAuth := make(map[string]map[string]any)
// if using stdio, header will be nil and auth will not be supported
if header != nil {
for _, aS := range authServices {
claims, err := aS.GetClaimsFromHeader(ctx, header)
if err != nil {
logger.DebugContext(ctx, err.Error())
continue
}
if claims == nil {
// authService not present in header
continue
}
claimsFromAuth[aS.GetName()] = claims
}
}
// Tool authorization check
verifiedAuthServices := make([]string, len(claimsFromAuth))
i := 0
for k := range claimsFromAuth {
verifiedAuthServices[i] = k
i++
}
// Check if any of the specified auth services is verified
isAuthorized := tool.Authorized(verifiedAuthServices)
if !isAuthorized {
err = fmt.Errorf("unauthorized Tool call: Please make sure your specify correct auth headers: %w", tools.ErrUnauthorized)
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
logger.DebugContext(ctx, "tool invocation authorized")
params, err := tool.ParseParams(data, claimsFromAuth)
if err != nil {
err = fmt.Errorf("provided parameters were invalid: %w", err)
@@ -113,14 +159,23 @@ func toolsCallHandler(ctx context.Context, id jsonrpc.RequestId, tools map[strin
}
logger.DebugContext(ctx, fmt.Sprintf("invocation params: %s", params))
if !tool.Authorized([]string{}) {
err = fmt.Errorf("unauthorized Tool call: `authRequired` is set for the target Tool")
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
// run tool invocation and generate response.
results, err := tool.Invoke(ctx, params, accessToken)
if err != nil {
errStr := err.Error()
// Missing authService tokens.
if errors.Is(err, tools.ErrUnauthorized) {
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
// Upstream auth error
if strings.Contains(errStr, "Error 401") || strings.Contains(errStr, "Error 403") {
if tool.RequiresClientAuthorization() {
// Error with client credentials should pass down to the client
return jsonrpc.NewError(id, jsonrpc.INVALID_REQUEST, err.Error(), nil), err
}
// Auth error with ADC should raise internal 500 error
return jsonrpc.NewError(id, jsonrpc.INTERNAL_ERROR, err.Error(), nil), err
}
text := TextContent{
Type: "text",
Text: err.Error(),

View File

@@ -39,7 +39,7 @@ const protocolVersion20250326 = "2025-03-26"
const protocolVersion20250618 = "2025-06-18"
const serverName = "Toolbox"
var tool1InputSchema = map[string]any{
var basicInputSchema = map[string]any{
"type": "object",
"properties": map[string]any{},
"required": []any{},
@@ -67,7 +67,7 @@ var tool3InputSchema = map[string]any{
}
func TestMcpEndpointWithoutInitialized(t *testing.T) {
mockTools := []MockTool{tool1, tool2, tool3}
mockTools := []MockTool{tool1, tool2, tool3, tool4, tool5}
toolsMap, toolsets := setUpResources(t, mockTools)
r, shutdown := setUpServer(t, "mcp", toolsMap, toolsets)
defer shutdown()
@@ -116,7 +116,7 @@ func TestMcpEndpointWithoutInitialized(t *testing.T) {
"tools": []any{
map[string]any{
"name": "no_params",
"inputSchema": tool1InputSchema,
"inputSchema": basicInputSchema,
},
map[string]any{
"name": "some_params",
@@ -127,6 +127,14 @@ func TestMcpEndpointWithoutInitialized(t *testing.T) {
"description": "some description",
"inputSchema": tool3InputSchema,
},
map[string]any{
"name": "unauthorized_tool",
"inputSchema": basicInputSchema,
},
map[string]any{
"name": "require_client_auth_tool",
"inputSchema": basicInputSchema,
},
},
},
},
@@ -169,6 +177,76 @@ func TestMcpEndpointWithoutInitialized(t *testing.T) {
},
},
},
{
name: "call tool1 unauthorized tool",
url: "/",
body: jsonrpc.JSONRPCRequest{
Jsonrpc: jsonrpcVersion,
Id: "tools-call-tool1",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "no_params",
},
},
want: map[string]any{
"jsonrpc": "2.0",
"id": "tools-call-tool1",
"result": map[string]any{
"content": []any{
map[string]any{
"type": "text",
"text": `"no_params"`,
},
},
},
},
},
{
name: "call tool4 unauthorized tool",
url: "/",
body: jsonrpc.JSONRPCRequest{
Jsonrpc: jsonrpcVersion,
Id: "tools-call-tool4",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "unauthorized_tool",
},
},
want: map[string]any{
"jsonrpc": "2.0",
"id": "tools-call-tool4",
"error": map[string]any{
"code": -32600.0,
"message": "unauthorized Tool call: Please make sure your specify correct auth headers: unauthorized",
},
},
},
{
name: "call tool5 unauthorized tool",
url: "/",
body: jsonrpc.JSONRPCRequest{
Jsonrpc: jsonrpcVersion,
Id: "tools-call-tool5",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "require_client_auth_tool",
},
},
want: map[string]any{
"jsonrpc": "2.0",
"id": "tools-call-tool5",
"error": map[string]any{
"code": -32600.0,
"message": "missing access token in the 'Authorization' header",
},
},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
@@ -258,7 +336,7 @@ func runInitializeLifecycle(t *testing.T, ts *httptest.Server, protocolVersion s
}
func TestMcpEndpoint(t *testing.T) {
mockTools := []MockTool{tool1, tool2, tool3}
mockTools := []MockTool{tool1, tool2, tool3, tool4, tool5}
toolsMap, toolsets := setUpResources(t, mockTools)
r, shutdown := setUpServer(t, "mcp", toolsMap, toolsets)
defer shutdown()
@@ -334,11 +412,12 @@ func TestMcpEndpoint(t *testing.T) {
}
testCases := []struct {
name string
url string
isErr bool
body any
want map[string]any
name string
url string
isErr bool
body any
wantStatusCode int
want map[string]any
}{
{
name: "basic notification",
@@ -349,6 +428,7 @@ func TestMcpEndpoint(t *testing.T) {
Method: "notification",
},
},
wantStatusCode: http.StatusAccepted,
},
{
name: "ping",
@@ -360,6 +440,7 @@ func TestMcpEndpoint(t *testing.T) {
Method: "ping",
},
},
wantStatusCode: http.StatusOK,
want: map[string]any{
"jsonrpc": "2.0",
"id": "ping-test-123",
@@ -376,6 +457,7 @@ func TestMcpEndpoint(t *testing.T) {
Method: "tools/list",
},
},
wantStatusCode: http.StatusOK,
want: map[string]any{
"jsonrpc": "2.0",
"id": "tools-list",
@@ -383,7 +465,7 @@ func TestMcpEndpoint(t *testing.T) {
"tools": []any{
map[string]any{
"name": "no_params",
"inputSchema": tool1InputSchema,
"inputSchema": basicInputSchema,
},
map[string]any{
"name": "some_params",
@@ -394,6 +476,14 @@ func TestMcpEndpoint(t *testing.T) {
"description": "some description",
"inputSchema": tool3InputSchema,
},
map[string]any{
"name": "unauthorized_tool",
"inputSchema": basicInputSchema,
},
map[string]any{
"name": "require_client_auth_tool",
"inputSchema": basicInputSchema,
},
},
},
},
@@ -408,6 +498,7 @@ func TestMcpEndpoint(t *testing.T) {
Method: "tools/list",
},
},
wantStatusCode: http.StatusOK,
want: map[string]any{
"jsonrpc": "2.0",
"id": "tools-list-tool1",
@@ -415,7 +506,7 @@ func TestMcpEndpoint(t *testing.T) {
"tools": []any{
map[string]any{
"name": "no_params",
"inputSchema": tool1InputSchema,
"inputSchema": basicInputSchema,
},
},
},
@@ -432,6 +523,7 @@ func TestMcpEndpoint(t *testing.T) {
Method: "tools/list",
},
},
wantStatusCode: http.StatusOK,
want: map[string]any{
"jsonrpc": "2.0",
"id": "tools-list-invalid-toolset",
@@ -450,6 +542,7 @@ func TestMcpEndpoint(t *testing.T) {
Id: "missing-method",
Request: jsonrpc.Request{},
},
wantStatusCode: http.StatusOK,
want: map[string]any{
"jsonrpc": "2.0",
"id": "missing-method",
@@ -470,6 +563,7 @@ func TestMcpEndpoint(t *testing.T) {
Method: "foo",
},
},
wantStatusCode: http.StatusOK,
want: map[string]any{
"jsonrpc": "2.0",
"id": "invalid-method",
@@ -490,6 +584,7 @@ func TestMcpEndpoint(t *testing.T) {
Method: "foo",
},
},
wantStatusCode: http.StatusOK,
want: map[string]any{
"jsonrpc": "2.0",
"id": "invalid-jsonrpc-version",
@@ -519,6 +614,7 @@ func TestMcpEndpoint(t *testing.T) {
},
},
},
wantStatusCode: http.StatusOK,
want: map[string]any{
"jsonrpc": "2.0",
"error": map[string]any{
@@ -527,6 +623,79 @@ func TestMcpEndpoint(t *testing.T) {
},
},
},
{
name: "call tool1 unauthorized tool",
url: "/",
body: jsonrpc.JSONRPCRequest{
Jsonrpc: jsonrpcVersion,
Id: "tools-call-tool1",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "no_params",
},
},
wantStatusCode: http.StatusOK,
want: map[string]any{
"jsonrpc": "2.0",
"id": "tools-call-tool1",
"result": map[string]any{
"content": []any{
map[string]any{
"type": "text",
"text": `"no_params"`,
},
},
},
},
},
{
name: "call tool4 unauthorized tool",
url: "/",
body: jsonrpc.JSONRPCRequest{
Jsonrpc: jsonrpcVersion,
Id: "tools-call-tool4",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "unauthorized_tool",
},
},
wantStatusCode: http.StatusUnauthorized,
want: map[string]any{
"jsonrpc": "2.0",
"id": "tools-call-tool4",
"error": map[string]any{
"code": -32600.0,
"message": "unauthorized Tool call: Please make sure your specify correct auth headers: unauthorized",
},
},
},
{
name: "call tool5 unauthorized tool",
url: "/",
body: jsonrpc.JSONRPCRequest{
Jsonrpc: jsonrpcVersion,
Id: "tools-call-tool5",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "require_client_auth_tool",
},
},
wantStatusCode: http.StatusUnauthorized,
want: map[string]any{
"jsonrpc": "2.0",
"id": "tools-call-tool5",
"error": map[string]any{
"code": -32600.0,
"message": "missing access token in the 'Authorization' header",
},
},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
@@ -540,10 +709,15 @@ func TestMcpEndpoint(t *testing.T) {
}
resp, body, err := runRequest(ts, http.MethodPost, tc.url, bytes.NewBuffer(reqMarshal), header)
if err != nil {
t.Fatalf("unexpected error during request: %s", err)
}
if resp.StatusCode != tc.wantStatusCode {
t.Errorf("StatusCode mismatch: got %d, want %d", resp.StatusCode, tc.wantStatusCode)
}
// Notifications don't expect a response.
if tc.want != nil {
if contentType := resp.Header.Get("Content-type"); contentType != "application/json" {

View File

@@ -21,8 +21,10 @@ import (
bigqueryapi "cloud.google.com/go/bigquery"
"github.com/goccy/go-yaml"
"github.com/googleapis/genai-toolbox/internal/sources"
"github.com/googleapis/genai-toolbox/internal/tools"
"github.com/googleapis/genai-toolbox/internal/util"
"go.opentelemetry.io/otel/trace"
"golang.org/x/oauth2"
"golang.org/x/oauth2/google"
bigqueryrestapi "google.golang.org/api/bigquery/v2"
"google.golang.org/api/option"
@@ -33,6 +35,8 @@ const SourceKind string = "bigquery"
// validate interface
var _ sources.SourceConfig = Config{}
type BigqueryClientCreator func(tokenString tools.AccessToken) (*bigqueryapi.Client, *bigqueryrestapi.Service, error)
func init() {
if !sources.Register(SourceKind, newConfig) {
panic(fmt.Sprintf("source kind %q already registered", SourceKind))
@@ -62,17 +66,19 @@ func (r Config) SourceConfigKind() string {
func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.Source, error) {
// Initializes a BigQuery Google SQL source
client, restService, err := initBigQueryConnection(ctx, tracer, r.Name, r.Project, r.Location)
client, restService, tokenSource, clientCreator, err := initBigQueryConnection(ctx, tracer, r.Name, r.Project, r.Location)
if err != nil {
return nil, err
}
s := &Source{
Name: r.Name,
Kind: SourceKind,
Client: client,
RestService: restService,
Location: r.Location,
Name: r.Name,
Kind: SourceKind,
Client: client,
RestService: restService,
TokenSource: tokenSource,
MaxQueryResultRows: 50,
ClientCreator: clientCreator,
}
return s, nil
@@ -82,11 +88,13 @@ var _ sources.Source = &Source{}
type Source struct {
// BigQuery Google SQL struct with client
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *bigqueryapi.Client
RestService *bigqueryrestapi.Service
Location string `yaml:"location"`
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *bigqueryapi.Client
RestService *bigqueryrestapi.Service
TokenSource oauth2.TokenSource
MaxQueryResultRows int
ClientCreator BigqueryClientCreator
}
func (s *Source) SourceKind() string {
@@ -102,38 +110,96 @@ func (s *Source) BigQueryRestService() *bigqueryrestapi.Service {
return s.RestService
}
func (s *Source) BigQueryTokenSource() oauth2.TokenSource {
return s.TokenSource
}
func (s *Source) GetMaxQueryResultRows() int {
return s.MaxQueryResultRows
}
func (s *Source) BigQueryClientCreator() BigqueryClientCreator {
return s.ClientCreator
}
func initBigQueryConnection(
ctx context.Context,
tracer trace.Tracer,
name string,
project string,
location string,
) (*bigqueryapi.Client, *bigqueryrestapi.Service, error) {
) (*bigqueryapi.Client, *bigqueryrestapi.Service, oauth2.TokenSource, BigqueryClientCreator, error) {
ctx, span := sources.InitConnectionSpan(ctx, tracer, SourceKind, name)
defer span.End()
cred, err := google.FindDefaultCredentials(ctx, bigqueryapi.Scope)
if err != nil {
return nil, nil, fmt.Errorf("failed to find default Google Cloud credentials with scope %q: %w", bigqueryapi.Scope, err)
return nil, nil, nil, nil, fmt.Errorf("failed to find default Google Cloud credentials with scope %q: %w", bigqueryapi.Scope, err)
}
userAgent, err := util.UserAgentFromContext(ctx)
if err != nil {
return nil, nil, err
return nil, nil, nil, nil, err
}
// Initialize the high-level BigQuery client
client, err := bigqueryapi.NewClient(ctx, project, option.WithUserAgent(userAgent), option.WithCredentials(cred))
if err != nil {
return nil, nil, fmt.Errorf("failed to create BigQuery client for project %q: %w", project, err)
return nil, nil, nil, nil, fmt.Errorf("failed to create BigQuery client for project %q: %w", project, err)
}
client.Location = location
// Initialize the low-level BigQuery REST service using the same credentials
restService, err := bigqueryrestapi.NewService(ctx, option.WithUserAgent(userAgent), option.WithCredentials(cred))
if err != nil {
return nil, nil, nil, nil, fmt.Errorf("failed to create BigQuery v2 service: %w", err)
}
clientCreator := newBigQueryClientCreator(ctx, project, location, userAgent)
return client, restService, cred.TokenSource, clientCreator, nil
}
// initBigQueryConnectionWithOAuthToken initialize a BigQuery client with an
// OAuth access token.
func initBigQueryConnectionWithOAuthToken(
ctx context.Context,
project string,
location string,
userAgent string,
tokenString tools.AccessToken,
) (*bigqueryapi.Client, *bigqueryrestapi.Service, error) {
// Construct token source
token := &oauth2.Token{
AccessToken: string(tokenString),
}
ts := oauth2.StaticTokenSource(token)
// Initialize the BigQuery client with tokenSource
client, err := bigqueryapi.NewClient(ctx, project, option.WithUserAgent(userAgent), option.WithTokenSource(ts))
if err != nil {
return nil, nil, fmt.Errorf("failed to create BigQuery client for project %q: %w", project, err)
}
client.Location = location
// Initialize the low-level BigQuery REST service using the same credentials
restService, err := bigqueryrestapi.NewService(ctx, option.WithUserAgent(userAgent), option.WithTokenSource(ts))
if err != nil {
return nil, nil, fmt.Errorf("failed to create BigQuery v2 service: %w", err)
}
return client, restService, nil
}
// newBigQueryClientCreator sets the project parameters for the init helper
// function. The returned function takes in an OAuth access token and uses it to
// create a BQ client.
func newBigQueryClientCreator(
ctx context.Context,
project string,
location string,
userAgent string,
) func(tools.AccessToken) (*bigqueryapi.Client, *bigqueryrestapi.Service, error) {
return func(tokenString tools.AccessToken) (*bigqueryapi.Client, *bigqueryrestapi.Service, error) {
return initBigQueryConnectionWithOAuthToken(ctx, project, location, userAgent, tokenString)
}
}

View File

@@ -0,0 +1,145 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package clickhouse
import (
"context"
"database/sql"
"fmt"
"net/url"
"time"
_ "github.com/ClickHouse/clickhouse-go/v2"
"github.com/goccy/go-yaml"
"github.com/googleapis/genai-toolbox/internal/sources"
"go.opentelemetry.io/otel/trace"
)
const SourceKind string = "clickhouse"
// validate interface
var _ sources.SourceConfig = Config{}
func init() {
if !sources.Register(SourceKind, newConfig) {
panic(fmt.Sprintf("source kind %q already registered", SourceKind))
}
}
func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (sources.SourceConfig, error) {
actual := Config{Name: name}
if err := decoder.DecodeContext(ctx, &actual); err != nil {
return nil, err
}
return actual, nil
}
type Config struct {
Name string `yaml:"name" validate:"required"`
Kind string `yaml:"kind" validate:"required"`
Host string `yaml:"host" validate:"required"`
Port string `yaml:"port" validate:"required"`
Database string `yaml:"database" validate:"required"`
User string `yaml:"user" validate:"required"`
Password string `yaml:"password"`
Protocol string `yaml:"protocol"`
Secure bool `yaml:"secure"`
}
func (r Config) SourceConfigKind() string {
return SourceKind
}
func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.Source, error) {
pool, err := initClickHouseConnectionPool(ctx, tracer, r.Name, r.Host, r.Port, r.User, r.Password, r.Database, r.Protocol, r.Secure)
if err != nil {
return nil, fmt.Errorf("unable to create pool: %w", err)
}
err = pool.PingContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to connect successfully: %w", err)
}
s := &Source{
Name: r.Name,
Kind: SourceKind,
Pool: pool,
}
return s, nil
}
var _ sources.Source = &Source{}
type Source struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Pool *sql.DB
}
func (s *Source) SourceKind() string {
return SourceKind
}
func (s *Source) ClickHousePool() *sql.DB {
return s.Pool
}
func validateConfig(protocol string) error {
validProtocols := map[string]bool{"http": true, "https": true}
if protocol != "" && !validProtocols[protocol] {
return fmt.Errorf("invalid protocol: %s, must be one of: http, https", protocol)
}
return nil
}
func initClickHouseConnectionPool(ctx context.Context, tracer trace.Tracer, name, host, port, user, pass, dbname, protocol string, secure bool) (*sql.DB, error) {
//nolint:all // Reassigned ctx
ctx, span := sources.InitConnectionSpan(ctx, tracer, SourceKind, name)
defer span.End()
if protocol == "" {
protocol = "https"
}
if err := validateConfig(protocol); err != nil {
return nil, err
}
encodedUser := url.QueryEscape(user)
encodedPass := url.QueryEscape(pass)
var dsn string
scheme := protocol
if protocol == "http" && secure {
scheme = "https"
}
dsn = fmt.Sprintf("%s://%s:%s@%s:%s/%s", scheme, encodedUser, encodedPass, host, port, dbname)
if scheme == "https" {
dsn += "?secure=true&skip_verify=false"
}
pool, err := sql.Open("clickhouse", dsn)
if err != nil {
return nil, fmt.Errorf("sql.Open: %w", err)
}
pool.SetMaxOpenConns(25)
pool.SetMaxIdleConns(5)
pool.SetConnMaxLifetime(5 * time.Minute)
return pool, nil
}

View File

@@ -0,0 +1,348 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package clickhouse
import (
"context"
"strings"
"testing"
"github.com/goccy/go-yaml"
"github.com/google/go-cmp/cmp"
"github.com/googleapis/genai-toolbox/internal/testutils"
"go.opentelemetry.io/otel"
)
func TestConfigSourceConfigKind(t *testing.T) {
config := Config{}
if config.SourceConfigKind() != SourceKind {
t.Errorf("Expected %s, got %s", SourceKind, config.SourceConfigKind())
}
}
func TestNewConfig(t *testing.T) {
tests := []struct {
name string
yaml string
expected Config
}{
{
name: "all fields specified",
yaml: `
name: test-clickhouse
kind: clickhouse
host: localhost
port: "8443"
user: default
password: "mypass"
database: mydb
protocol: https
secure: true
`,
expected: Config{
Name: "test-clickhouse",
Kind: "clickhouse",
Host: "localhost",
Port: "8443",
User: "default",
Password: "mypass",
Database: "mydb",
Protocol: "https",
Secure: true,
},
},
{
name: "minimal configuration with defaults",
yaml: `
name: minimal-clickhouse
kind: clickhouse
host: 127.0.0.1
port: "8123"
user: testuser
database: testdb
`,
expected: Config{
Name: "minimal-clickhouse",
Kind: "clickhouse",
Host: "127.0.0.1",
Port: "8123",
User: "testuser",
Password: "",
Database: "testdb",
Protocol: "",
Secure: false,
},
},
{
name: "http protocol",
yaml: `
name: http-clickhouse
kind: clickhouse
host: clickhouse.example.com
port: "8123"
user: analytics
password: "securepass"
database: analytics_db
protocol: http
secure: false
`,
expected: Config{
Name: "http-clickhouse",
Kind: "clickhouse",
Host: "clickhouse.example.com",
Port: "8123",
User: "analytics",
Password: "securepass",
Database: "analytics_db",
Protocol: "http",
Secure: false,
},
},
{
name: "https with secure connection",
yaml: `
name: secure-clickhouse
kind: clickhouse
host: secure.clickhouse.io
port: "8443"
user: secureuser
password: "verysecure"
database: production
protocol: https
secure: true
`,
expected: Config{
Name: "secure-clickhouse",
Kind: "clickhouse",
Host: "secure.clickhouse.io",
Port: "8443",
User: "secureuser",
Password: "verysecure",
Database: "production",
Protocol: "https",
Secure: true,
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
decoder := yaml.NewDecoder(strings.NewReader(string(testutils.FormatYaml(tt.yaml))))
config, err := newConfig(context.Background(), tt.expected.Name, decoder)
if err != nil {
t.Fatalf("Failed to create config: %v", err)
}
clickhouseConfig, ok := config.(Config)
if !ok {
t.Fatalf("Expected Config type, got %T", config)
}
if diff := cmp.Diff(tt.expected, clickhouseConfig); diff != "" {
t.Errorf("Config mismatch (-want +got):\n%s", diff)
}
})
}
}
func TestNewConfigInvalidYAML(t *testing.T) {
tests := []struct {
name string
yaml string
expectError bool
}{
{
name: "invalid yaml syntax",
yaml: `
name: test-clickhouse
kind: clickhouse
host: [invalid
`,
expectError: true,
},
{
name: "missing required fields",
yaml: `
name: test-clickhouse
kind: clickhouse
`,
expectError: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
decoder := yaml.NewDecoder(strings.NewReader(string(testutils.FormatYaml(tt.yaml))))
_, err := newConfig(context.Background(), "test-clickhouse", decoder)
if tt.expectError && err == nil {
t.Errorf("Expected error but got none")
}
if !tt.expectError && err != nil {
t.Errorf("Expected no error but got: %v", err)
}
})
}
}
func TestSource_SourceKind(t *testing.T) {
source := &Source{}
if source.SourceKind() != SourceKind {
t.Errorf("Expected %s, got %s", SourceKind, source.SourceKind())
}
}
func TestValidateConfig(t *testing.T) {
tests := []struct {
name string
protocol string
expectError bool
}{
{
name: "valid https protocol",
protocol: "https",
expectError: false,
},
{
name: "valid http protocol",
protocol: "http",
expectError: false,
},
{
name: "invalid protocol",
protocol: "invalid",
expectError: true,
},
{
name: "invalid protocol - native not supported",
protocol: "native",
expectError: true,
},
{
name: "empty values use defaults",
protocol: "",
expectError: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := validateConfig(tt.protocol)
if tt.expectError && err == nil {
t.Errorf("Expected error but got none")
}
if !tt.expectError && err != nil {
t.Errorf("Expected no error but got: %v", err)
}
})
}
}
func TestInitClickHouseConnectionPoolDSNGeneration(t *testing.T) {
tracer := otel.Tracer("test")
ctx := context.Background()
tests := []struct {
name string
host string
port string
user string
pass string
dbname string
protocol string
secure bool
shouldErr bool
}{
{
name: "http protocol with defaults",
host: "localhost",
port: "8123",
user: "default",
pass: "",
dbname: "default",
protocol: "http",
secure: false,
shouldErr: true,
},
{
name: "https protocol with secure",
host: "localhost",
port: "8443",
user: "default",
pass: "",
dbname: "default",
protocol: "https",
secure: true,
shouldErr: true,
},
{
name: "special characters in password",
host: "localhost",
port: "8443",
user: "test@user",
pass: "pass@word:with/special&chars",
dbname: "default",
protocol: "https",
secure: true,
shouldErr: true,
},
{
name: "invalid protocol should fail",
host: "localhost",
port: "9000",
user: "default",
pass: "",
dbname: "default",
protocol: "invalid",
secure: false,
shouldErr: true,
},
{
name: "empty protocol defaults to https",
host: "localhost",
port: "8443",
user: "user",
pass: "pass",
dbname: "testdb",
protocol: "",
secure: true,
shouldErr: true,
},
{
name: "http with secure flag should upgrade to https",
host: "example.com",
port: "8443",
user: "user",
pass: "pass",
dbname: "db",
protocol: "http",
secure: true,
shouldErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
pool, err := initClickHouseConnectionPool(ctx, tracer, "test", tt.host, tt.port, tt.user, tt.pass, tt.dbname, tt.protocol, tt.secure)
if !tt.shouldErr && err != nil {
t.Errorf("Expected no error, got: %v", err)
}
if pool != nil {
pool.Close()
}
})
}
}

View File

@@ -39,7 +39,14 @@ func init() {
}
func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (sources.SourceConfig, error) {
actual := Config{Name: name, SslVerification: "true", Timeout: "600s"} // Default Ssl,timeout
actual := Config{
Name: name,
SslVerification: true,
Timeout: "600s",
ShowHiddenModels: true,
ShowHiddenExplores: true,
ShowHiddenFields: true,
} // Default Ssl,timeout, ShowHidden
if err := decoder.DecodeContext(ctx, &actual); err != nil {
return nil, err
}
@@ -47,13 +54,16 @@ func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (sources
}
type Config struct {
Name string `yaml:"name" validate:"required"`
Kind string `yaml:"kind" validate:"required"`
BaseURL string `yaml:"base_url" validate:"required"`
ClientId string `yaml:"client_id" validate:"required"`
ClientSecret string `yaml:"client_secret" validate:"required"`
SslVerification string `yaml:"verify_ssl"`
Timeout string `yaml:"timeout"`
Name string `yaml:"name" validate:"required"`
Kind string `yaml:"kind" validate:"required"`
BaseURL string `yaml:"base_url" validate:"required"`
ClientId string `yaml:"client_id" validate:"required"`
ClientSecret string `yaml:"client_secret" validate:"required"`
SslVerification bool `yaml:"verify_ssl"`
Timeout string `yaml:"timeout"`
ShowHiddenModels bool `yaml:"show_hidden_models"`
ShowHiddenExplores bool `yaml:"show_hidden_explores"`
ShowHiddenFields bool `yaml:"show_hidden_fields"`
}
func (r Config) SourceConfigKind() string {
@@ -77,14 +87,14 @@ func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.So
return nil, fmt.Errorf("unable to parse Timeout string as time.Duration: %s", err)
}
if r.SslVerification != "true" {
if !r.SslVerification {
logger.WarnContext(ctx, "Insecure HTTP is enabled for Looker source %s. TLS certificate verification is skipped.\n", r.Name)
}
cfg := rtl.ApiSettings{
AgentTag: userAgent,
BaseUrl: r.BaseURL,
ApiVersion: "4.0",
VerifySsl: (r.SslVerification == "true"),
VerifySsl: r.SslVerification,
Timeout: int32(duration.Seconds()),
ClientId: r.ClientId,
ClientSecret: r.ClientSecret,
@@ -98,11 +108,14 @@ func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.So
logger.DebugContext(ctx, fmt.Sprintf("logged in as user %v %v.\n", *me.FirstName, *me.LastName))
s := &Source{
Name: r.Name,
Kind: SourceKind,
Timeout: r.Timeout,
Client: sdk,
ApiSettings: &cfg,
Name: r.Name,
Kind: SourceKind,
Timeout: r.Timeout,
Client: sdk,
ApiSettings: &cfg,
ShowHiddenModels: r.ShowHiddenModels,
ShowHiddenExplores: r.ShowHiddenExplores,
ShowHiddenFields: r.ShowHiddenFields,
}
return s, nil
@@ -111,11 +124,14 @@ func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.So
var _ sources.Source = &Source{}
type Source struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Timeout string `yaml:"timeout"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Timeout string `yaml:"timeout"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
ShowHiddenModels bool `yaml:"show_hidden_models"`
ShowHiddenExplores bool `yaml:"show_hidden_explores"`
ShowHiddenFields bool `yaml:"show_hidden_fields"`
}
func (s *Source) SourceKind() string {

View File

@@ -43,13 +43,16 @@ func TestParseFromYamlLooker(t *testing.T) {
`,
want: map[string]sources.SourceConfig{
"my-looker-instance": looker.Config{
Name: "my-looker-instance",
Kind: looker.SourceKind,
BaseURL: "http://example.looker.com/",
ClientId: "jasdl;k;tjl",
ClientSecret: "sdakl;jgflkasdfkfg",
Timeout: "600s",
SslVerification: "true",
Name: "my-looker-instance",
Kind: looker.SourceKind,
BaseURL: "http://example.looker.com/",
ClientId: "jasdl;k;tjl",
ClientSecret: "sdakl;jgflkasdfkfg",
Timeout: "600s",
SslVerification: true,
ShowHiddenModels: true,
ShowHiddenExplores: true,
ShowHiddenFields: true,
},
},
},

View File

@@ -0,0 +1,523 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package bigqueryconversationalanalytics
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
bigqueryapi "cloud.google.com/go/bigquery"
yaml "github.com/goccy/go-yaml"
"github.com/googleapis/genai-toolbox/internal/sources"
bigqueryds "github.com/googleapis/genai-toolbox/internal/sources/bigquery"
"github.com/googleapis/genai-toolbox/internal/tools"
"golang.org/x/oauth2"
)
const kind string = "bigquery-conversational-analytics"
const instructions = `**INSTRUCTIONS - FOLLOW THESE RULES:**
1. **CONTENT:** Your answer should present the supporting data and then provide a conclusion based on that data.
2. **OUTPUT FORMAT:** Your entire response MUST be in plain text format ONLY.
3. **NO CHARTS:** You are STRICTLY FORBIDDEN from generating any charts, graphs, images, or any other form of visualization.`
func init() {
if !tools.Register(kind, newConfig) {
panic(fmt.Sprintf("tool kind %q already registered", kind))
}
}
func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.ToolConfig, error) {
actual := Config{Name: name}
if err := decoder.DecodeContext(ctx, &actual); err != nil {
return nil, err
}
return actual, nil
}
type compatibleSource interface {
BigQueryClient() *bigqueryapi.Client
BigQueryTokenSource() oauth2.TokenSource
GetMaxQueryResultRows() int
}
type BQTableReference struct {
ProjectID string `json:"projectId"`
DatasetID string `json:"datasetId"`
TableID string `json:"tableId"`
}
// Structs for building the JSON payload
type UserMessage struct {
Text string `json:"text"`
}
type Message struct {
UserMessage UserMessage `json:"userMessage"`
}
type BQDatasource struct {
TableReferences []BQTableReference `json:"tableReferences"`
}
type DatasourceReferences struct {
BQ BQDatasource `json:"bq"`
}
type ImageOptions struct {
NoImage map[string]any `json:"noImage"`
}
type ChartOptions struct {
Image ImageOptions `json:"image"`
}
type Options struct {
Chart ChartOptions `json:"chart"`
}
type InlineContext struct {
DatasourceReferences DatasourceReferences `json:"datasourceReferences"`
Options Options `json:"options"`
}
type CAPayload struct {
Project string `json:"project"`
Messages []Message `json:"messages"`
InlineContext InlineContext `json:"inlineContext"`
}
// validate compatible sources are still compatible
var _ compatibleSource = &bigqueryds.Source{}
var compatibleSources = [...]string{bigqueryds.SourceKind}
type Config struct {
Name string `yaml:"name" validate:"required"`
Kind string `yaml:"kind" validate:"required"`
Source string `yaml:"source" validate:"required"`
Description string `yaml:"description" validate:"required"`
AuthRequired []string `yaml:"authRequired"`
}
// validate interface
var _ tools.ToolConfig = Config{}
func (cfg Config) ToolConfigKind() string {
return kind
}
func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error) {
// verify source exists
rawS, ok := srcs[cfg.Source]
if !ok {
return nil, fmt.Errorf("no source named %q configured", cfg.Source)
}
// verify the source is compatible
s, ok := rawS.(compatibleSource)
if !ok {
return nil, fmt.Errorf("invalid source for %q tool: source kind must be one of %q", kind, compatibleSources)
}
userQueryParameter := tools.NewStringParameter("user_query_with_context", "The user's question, potentially including conversation history and system instructions for context.")
tableRefsParameter := tools.NewStringParameter("table_references", `A JSON string of a list of BigQuery tables to use as context. Each object in the list must contain 'projectId', 'datasetId', and 'tableId'. Example: '[{"projectId": "my-gcp-project", "datasetId": "my_dataset", "tableId": "my_table"}]'`)
parameters := tools.Parameters{userQueryParameter, tableRefsParameter}
mcpManifest := tools.McpManifest{
Name: cfg.Name,
Description: cfg.Description,
InputSchema: parameters.McpManifest(),
}
// finish tool setup
t := Tool{
Name: cfg.Name,
Kind: kind,
Parameters: parameters,
AuthRequired: cfg.AuthRequired,
Client: s.BigQueryClient(),
TokenSource: s.BigQueryTokenSource(),
manifest: tools.Manifest{Description: cfg.Description, Parameters: parameters.Manifest(), AuthRequired: cfg.AuthRequired},
mcpManifest: mcpManifest,
MaxQueryResultRows: s.GetMaxQueryResultRows(),
}
return t, nil
}
// validate interface
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
Client *bigqueryapi.Client
TokenSource oauth2.TokenSource
manifest tools.Manifest
mcpManifest tools.McpManifest
MaxQueryResultRows int
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
// Get credentials for the API call
if t.TokenSource == nil {
return nil, fmt.Errorf("authentication error: found credentials but they are missing a valid token source")
}
token, err := t.TokenSource.Token()
if err != nil {
return nil, fmt.Errorf("failed to get token from credentials: %w", err)
}
// Extract parameters from the map
mapParams := params.AsMap()
userQuery, _ := mapParams["user_query_with_context"].(string)
finalQueryText := fmt.Sprintf("%s\n**User Query and Context:**\n%s", instructions, userQuery)
tableRefsJSON, _ := mapParams["table_references"].(string)
var tableRefs []BQTableReference
if tableRefsJSON != "" {
if err := json.Unmarshal([]byte(tableRefsJSON), &tableRefs); err != nil {
return nil, fmt.Errorf("failed to parse 'table_references' JSON string: %w", err)
}
}
// Construct URL, headers, and payload
projectID := t.Client.Project()
location := t.Client.Location
if location == "" {
location = "us"
}
caURL := fmt.Sprintf("https://geminidataanalytics.googleapis.com/v1alpha/projects/%s/locations/%s:chat", projectID, location)
headers := map[string]string{
"Authorization": fmt.Sprintf("Bearer %s", token.AccessToken),
"Content-Type": "application/json",
}
payload := CAPayload{
Project: fmt.Sprintf("projects/%s", projectID),
Messages: []Message{{UserMessage: UserMessage{Text: finalQueryText}}},
InlineContext: InlineContext{
DatasourceReferences: DatasourceReferences{
BQ: BQDatasource{TableReferences: tableRefs},
},
Options: Options{Chart: ChartOptions{Image: ImageOptions{NoImage: map[string]any{}}}},
},
}
// Call the streaming API
response, err := getStream(caURL, payload, headers, t.MaxQueryResultRows)
if err != nil {
return nil, fmt.Errorf("failed to get response from conversational analytics API: %w", err)
}
return response, nil
}
func (t Tool) ParseParams(data map[string]any, claims map[string]map[string]any) (tools.ParamValues, error) {
return tools.ParseParams(t.Parameters, data, claims)
}
func (t Tool) Manifest() tools.Manifest {
return t.manifest
}
func (t Tool) McpManifest() tools.McpManifest {
return t.mcpManifest
}
func (t Tool) Authorized(verifiedAuthServices []string) bool {
return tools.IsAuthorized(t.AuthRequired, verifiedAuthServices)
}
func (t Tool) RequiresClientAuthorization() bool {
return false
}
// StreamMessage represents a single message object from the streaming API response.
type StreamMessage struct {
SystemMessage *SystemMessage `json:"systemMessage,omitempty"`
Error *ErrorResponse `json:"error,omitempty"`
}
// SystemMessage contains different types of system-generated content.
type SystemMessage struct {
Text *TextResponse `json:"text,omitempty"`
Schema *SchemaResponse `json:"schema,omitempty"`
Data *DataResponse `json:"data,omitempty"`
}
// TextResponse contains textual parts of a message.
type TextResponse struct {
Parts []string `json:"parts"`
}
// SchemaResponse contains schema-related information.
type SchemaResponse struct {
Query *SchemaQuery `json:"query,omitempty"`
Result *SchemaResult `json:"result,omitempty"`
}
// SchemaQuery holds the question that prompted a schema lookup.
type SchemaQuery struct {
Question string `json:"question"`
}
// SchemaResult contains the datasources with their schemas.
type SchemaResult struct {
Datasources []Datasource `json:"datasources"`
}
// Datasource represents a data source with its reference and schema.
type Datasource struct {
BigQueryTableReference *BQTableReference `json:"bigqueryTableReference,omitempty"`
Schema *BQSchema `json:"schema,omitempty"`
}
// BQSchema defines the structure of a BigQuery table.
type BQSchema struct {
Fields []BQField `json:"fields"`
}
// BQField describes a single column in a BigQuery table.
type BQField struct {
Name string `json:"name"`
Type string `json:"type"`
Description string `json:"description"`
Mode string `json:"mode"`
}
// DataResponse contains data-related information, like queries and results.
type DataResponse struct {
Query *DataQuery `json:"query,omitempty"`
GeneratedSQL string `json:"generatedSql,omitempty"`
Result *DataResult `json:"result,omitempty"`
}
// DataQuery holds information about a data retrieval query.
type DataQuery struct {
Name string `json:"name"`
Question string `json:"question"`
}
// DataResult contains the schema and rows of a query result.
type DataResult struct {
Schema BQSchema `json:"schema"`
Data []map[string]any `json:"data"`
}
// ErrorResponse represents an error message from the API.
type ErrorResponse struct {
Code float64 `json:"code"` // JSON numbers are float64 by default
Message string `json:"message"`
}
func getStream(url string, payload CAPayload, headers map[string]string, maxRows int) (string, error) {
payloadBytes, err := json.Marshal(payload)
if err != nil {
return "", fmt.Errorf("failed to marshal payload: %w", err)
}
req, err := http.NewRequest("POST", url, bytes.NewBuffer(payloadBytes))
if err != nil {
return "", fmt.Errorf("failed to create request: %w", err)
}
for k, v := range headers {
req.Header.Set(k, v)
}
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return "", fmt.Errorf("failed to send request: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return "", fmt.Errorf("API returned non-200 status: %d %s", resp.StatusCode, string(body))
}
var messages []map[string]any
decoder := json.NewDecoder(resp.Body)
// The response is a JSON array, so we read the opening bracket.
if _, err := decoder.Token(); err != nil {
if err == io.EOF {
return "", nil // Empty response is valid
}
return "", fmt.Errorf("error reading start of json array: %w", err)
}
for decoder.More() {
var msg StreamMessage
if err := decoder.Decode(&msg); err != nil {
if err == io.EOF {
break
}
return "", fmt.Errorf("error decoding stream message: %w", err)
}
var newMessage map[string]any
if msg.SystemMessage != nil {
if msg.SystemMessage.Text != nil {
newMessage = handleTextResponse(msg.SystemMessage.Text)
} else if msg.SystemMessage.Schema != nil {
newMessage = handleSchemaResponse(msg.SystemMessage.Schema)
} else if msg.SystemMessage.Data != nil {
newMessage = handleDataResponse(msg.SystemMessage.Data, maxRows)
}
} else if msg.Error != nil {
newMessage = handleError(msg.Error)
}
messages = appendMessage(messages, newMessage)
}
var acc strings.Builder
for i, msg := range messages {
jsonBytes, err := json.MarshalIndent(msg, "", " ")
if err != nil {
return "", fmt.Errorf("error marshalling message: %w", err)
}
acc.Write(jsonBytes)
if i < len(messages)-1 {
acc.WriteString("\n")
}
}
return acc.String(), nil
}
func formatBqTableRef(tableRef *BQTableReference) string {
return fmt.Sprintf("%s.%s.%s", tableRef.ProjectID, tableRef.DatasetID, tableRef.TableID)
}
func formatSchemaAsDict(data *BQSchema) map[string]any {
headers := []string{"Column", "Type", "Description", "Mode"}
if data == nil {
return map[string]any{"headers": headers, "rows": []any{}}
}
var rows [][]any
for _, field := range data.Fields {
rows = append(rows, []any{field.Name, field.Type, field.Description, field.Mode})
}
return map[string]any{"headers": headers, "rows": rows}
}
func formatDatasourceAsDict(datasource *Datasource) map[string]any {
var sourceName string
if datasource.BigQueryTableReference != nil {
sourceName = formatBqTableRef(datasource.BigQueryTableReference)
}
var schema map[string]any
if datasource.Schema != nil {
schema = formatSchemaAsDict(datasource.Schema)
}
return map[string]any{"source_name": sourceName, "schema": schema}
}
func handleTextResponse(resp *TextResponse) map[string]any {
return map[string]any{"Answer": strings.Join(resp.Parts, "")}
}
func handleSchemaResponse(resp *SchemaResponse) map[string]any {
if resp.Query != nil {
return map[string]any{"Question": resp.Query.Question}
}
if resp.Result != nil {
var formattedSources []map[string]any
for _, ds := range resp.Result.Datasources {
formattedSources = append(formattedSources, formatDatasourceAsDict(&ds))
}
return map[string]any{"Schema Resolved": formattedSources}
}
return nil
}
func handleDataResponse(resp *DataResponse, maxRows int) map[string]any {
if resp.Query != nil {
return map[string]any{
"Retrieval Query": map[string]any{
"Query Name": resp.Query.Name,
"Question": resp.Query.Question,
},
}
}
if resp.GeneratedSQL != "" {
return map[string]any{"SQL Generated": resp.GeneratedSQL}
}
if resp.Result != nil {
var headers []string
for _, f := range resp.Result.Schema.Fields {
headers = append(headers, f.Name)
}
totalRows := len(resp.Result.Data)
var compactRows [][]any
numRowsToDisplay := totalRows
if numRowsToDisplay > maxRows {
numRowsToDisplay = maxRows
}
for _, rowVal := range resp.Result.Data[:numRowsToDisplay] {
var rowValues []any
for _, header := range headers {
rowValues = append(rowValues, rowVal[header])
}
compactRows = append(compactRows, rowValues)
}
summary := fmt.Sprintf("Showing all %d rows.", totalRows)
if totalRows > maxRows {
summary = fmt.Sprintf("Showing the first %d of %d total rows.", numRowsToDisplay, totalRows)
}
return map[string]any{
"Data Retrieved": map[string]any{
"headers": headers,
"rows": compactRows,
"summary": summary,
},
}
}
return nil
}
func handleError(resp *ErrorResponse) map[string]any {
return map[string]any{
"Error": map[string]any{
"Code": int(resp.Code),
"Message": resp.Message,
},
}
}
func appendMessage(messages []map[string]any, newMessage map[string]any) []map[string]any {
if newMessage == nil {
return messages
}
if len(messages) > 0 {
if _, ok := messages[len(messages)-1]["Data Retrieved"]; ok {
messages = messages[:len(messages)-1]
}
}
return append(messages, newMessage)
}

View File

@@ -0,0 +1,72 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package bigqueryconversationalanalytics_test
import (
"testing"
yaml "github.com/goccy/go-yaml"
"github.com/google/go-cmp/cmp"
"github.com/googleapis/genai-toolbox/internal/server"
"github.com/googleapis/genai-toolbox/internal/testutils"
"github.com/googleapis/genai-toolbox/internal/tools/bigquery/bigqueryconversationalanalytics"
)
func TestParseFromYamlBigQueryConversationalAnalytics(t *testing.T) {
ctx, err := testutils.ContextWithNewLogger()
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
tcs := []struct {
desc string
in string
want server.ToolConfigs
}{
{
desc: "basic example",
in: `
tools:
example_tool:
kind: bigquery-conversational-analytics
source: my-instance
description: some description
`,
want: server.ToolConfigs{
"example_tool": bigqueryconversationalanalytics.Config{
Name: "example_tool",
Kind: "bigquery-conversational-analytics",
Source: "my-instance",
Description: "some description",
AuthRequired: []string{},
},
},
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got := struct {
Tools server.ToolConfigs `yaml:"tools"`
}{}
// Parse contents
err := yaml.UnmarshalContext(ctx, testutils.FormatYaml(tc.in), &got)
if err != nil {
t.Fatalf("unable to unmarshal: %s", err)
}
if diff := cmp.Diff(tc.want, got.Tools); diff != "" {
t.Fatalf("incorrect parse: diff %v", diff)
}
})
}
}

View File

@@ -23,6 +23,7 @@ import (
bigqueryapi "cloud.google.com/go/bigquery"
yaml "github.com/goccy/go-yaml"
"github.com/googleapis/genai-toolbox/internal/sources"
bigqueryds "github.com/googleapis/genai-toolbox/internal/sources/bigquery"
"github.com/googleapis/genai-toolbox/internal/tools"
bigqueryrestapi "google.golang.org/api/bigquery/v2"
@@ -48,6 +49,7 @@ func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.T
type compatibleSource interface {
BigQueryClient() *bigqueryapi.Client
BigQueryRestService() *bigqueryrestapi.Service
BigQueryClientCreator() bigqueryds.BigqueryClientCreator
}
// validate compatible sources are still compatible
@@ -62,6 +64,7 @@ type Config struct {
Description string `yaml:"description" validate:"required"`
Statement string `yaml:"statement" validate:"required"`
AuthRequired []string `yaml:"authRequired"`
UseClientOAuth bool `yaml:"useClientOAuth"`
Parameters tools.Parameters `yaml:"parameters"`
TemplateParameters tools.Parameters `yaml:"templateParameters"`
}
@@ -101,15 +104,18 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
t := Tool{
Name: cfg.Name,
Kind: kind,
AuthRequired: cfg.AuthRequired,
UseClientOAuth: cfg.UseClientOAuth,
Parameters: cfg.Parameters,
TemplateParameters: cfg.TemplateParameters,
AllParams: allParameters,
Statement: cfg.Statement,
AuthRequired: cfg.AuthRequired,
Client: s.BigQueryClient(),
RestService: s.BigQueryRestService(),
manifest: tools.Manifest{Description: cfg.Description, Parameters: paramManifest, AuthRequired: cfg.AuthRequired},
mcpManifest: mcpManifest,
Statement: cfg.Statement,
Client: s.BigQueryClient(),
RestService: s.BigQueryRestService(),
ClientCreator: s.BigQueryClientCreator(),
manifest: tools.Manifest{Description: cfg.Description, Parameters: paramManifest, AuthRequired: cfg.AuthRequired},
mcpManifest: mcpManifest,
}
return t, nil
}
@@ -121,14 +127,17 @@ type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
AuthRequired []string `yaml:"authRequired"`
UseClientOAuth bool `yaml:"useClientOAuth"`
Parameters tools.Parameters `yaml:"parameters"`
TemplateParameters tools.Parameters `yaml:"templateParameters"`
AllParams tools.Parameters `yaml:"allParams"`
Statement string
Client *bigqueryapi.Client
RestService *bigqueryrestapi.Service
manifest tools.Manifest
mcpManifest tools.McpManifest
Statement string
Client *bigqueryapi.Client
RestService *bigqueryrestapi.Service
ClientCreator bigqueryds.BigqueryClientCreator
manifest tools.Manifest
mcpManifest tools.McpManifest
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
@@ -208,11 +217,23 @@ func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken
lowLevelParams = append(lowLevelParams, lowLevelParam)
}
query := t.Client.Query(newStatement)
bqClient := t.Client
restService := t.RestService
var query *bigqueryapi.Query
// Initialize new client if using user OAuth token
if t.UseClientOAuth {
bqClient, restService, err = t.ClientCreator(accessToken)
if err != nil {
return nil, fmt.Errorf("error creating client from OAuth access token: %w", err)
}
}
query = bqClient.Query(newStatement)
query.Parameters = highLevelParams
query.Location = t.Client.Location
dryRunJob, err := dryRunQuery(ctx, t.RestService, t.Client.Project(), t.Client.Location, newStatement, lowLevelParams, query.ConnectionProperties)
dryRunJob, err := dryRunQuery(ctx, restService, t.Client.Project(), t.Client.Location, newStatement, lowLevelParams, query.ConnectionProperties)
if err != nil {
// This is a fallback check in case the switch logic was bypassed.
return nil, fmt.Errorf("final query validation failed: %w", err)
@@ -277,8 +298,9 @@ func (t Tool) Authorized(verifiedAuthServices []string) bool {
}
func (t Tool) RequiresClientAuthorization() bool {
return false
return t.UseClientOAuth
}
func BQTypeStringFromToolType(toolType string) (string, error) {
switch toolType {
case "string":

View File

@@ -0,0 +1,191 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package clickhouse
import (
"context"
"database/sql"
"fmt"
yaml "github.com/goccy/go-yaml"
"github.com/googleapis/genai-toolbox/internal/sources"
"github.com/googleapis/genai-toolbox/internal/tools"
)
type compatibleSource interface {
ClickHousePool() *sql.DB
}
var compatibleSources = []string{"clickhouse"}
const executeSQLKind string = "clickhouse-execute-sql"
func init() {
if !tools.Register(executeSQLKind, newExecuteSQLConfig) {
panic(fmt.Sprintf("tool kind %q already registered", executeSQLKind))
}
}
func newExecuteSQLConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.ToolConfig, error) {
actual := Config{Name: name}
if err := decoder.DecodeContext(ctx, &actual); err != nil {
return nil, err
}
return actual, nil
}
type Config struct {
Name string `yaml:"name" validate:"required"`
Kind string `yaml:"kind" validate:"required"`
Source string `yaml:"source" validate:"required"`
Description string `yaml:"description" validate:"required"`
AuthRequired []string `yaml:"authRequired"`
}
var _ tools.ToolConfig = Config{}
func (cfg Config) ToolConfigKind() string {
return executeSQLKind
}
func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error) {
rawS, ok := srcs[cfg.Source]
if !ok {
return nil, fmt.Errorf("no source named %q configured", cfg.Source)
}
s, ok := rawS.(compatibleSource)
if !ok {
return nil, fmt.Errorf("invalid source for %q tool: source kind must be one of %q", executeSQLKind, compatibleSources)
}
sqlParameter := tools.NewStringParameter("sql", "The SQL statement to execute.")
parameters := tools.Parameters{sqlParameter}
mcpManifest := tools.McpManifest{
Name: cfg.Name,
Description: cfg.Description,
InputSchema: parameters.McpManifest(),
}
t := ExecuteSQLTool{
Name: cfg.Name,
Kind: executeSQLKind,
Parameters: parameters,
AuthRequired: cfg.AuthRequired,
Pool: s.ClickHousePool(),
manifest: tools.Manifest{Description: cfg.Description, Parameters: parameters.Manifest(), AuthRequired: cfg.AuthRequired},
mcpManifest: mcpManifest,
}
return t, nil
}
var _ tools.Tool = ExecuteSQLTool{}
type ExecuteSQLTool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
Pool *sql.DB
manifest tools.Manifest
mcpManifest tools.McpManifest
}
func (t ExecuteSQLTool) Invoke(ctx context.Context, params tools.ParamValues, token tools.AccessToken) (any, error) {
paramsMap := params.AsMap()
sql, ok := paramsMap["sql"].(string)
if !ok {
return nil, fmt.Errorf("unable to cast sql parameter %s", paramsMap["sql"])
}
results, err := t.Pool.QueryContext(ctx, sql)
if err != nil {
return nil, fmt.Errorf("unable to execute query: %w", err)
}
defer results.Close()
cols, err := results.Columns()
if err != nil {
return nil, fmt.Errorf("unable to retrieve rows column name: %w", err)
}
// create an array of values for each column, which can be re-used to scan each row
rawValues := make([]any, len(cols))
values := make([]any, len(cols))
for i := range rawValues {
values[i] = &rawValues[i]
}
colTypes, err := results.ColumnTypes()
if err != nil {
return nil, fmt.Errorf("unable to get column types: %w", err)
}
var out []any
for results.Next() {
err := results.Scan(values...)
if err != nil {
return nil, fmt.Errorf("unable to parse row: %w", err)
}
vMap := make(map[string]any)
for i, name := range cols {
// ClickHouse driver may return specific types that need handling
switch colTypes[i].DatabaseTypeName() {
case "String", "FixedString":
if rawValues[i] != nil {
// Handle potential []byte to string conversion if needed
if b, ok := rawValues[i].([]byte); ok {
vMap[name] = string(b)
} else {
vMap[name] = rawValues[i]
}
} else {
vMap[name] = nil
}
default:
vMap[name] = rawValues[i]
}
}
out = append(out, vMap)
}
if err := results.Err(); err != nil {
return nil, fmt.Errorf("errors encountered by results.Scan: %w", err)
}
return out, nil
}
func (t ExecuteSQLTool) ParseParams(data map[string]any, claims map[string]map[string]any) (tools.ParamValues, error) {
return tools.ParseParams(t.Parameters, data, claims)
}
func (t ExecuteSQLTool) Manifest() tools.Manifest {
return t.manifest
}
func (t ExecuteSQLTool) McpManifest() tools.McpManifest {
return t.mcpManifest
}
func (t ExecuteSQLTool) Authorized(verifiedAuthServices []string) bool {
return tools.IsAuthorized(t.AuthRequired, verifiedAuthServices)
}
func (t ExecuteSQLTool) RequiresClientAuthorization() bool {
return false
}

View File

@@ -0,0 +1,70 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package clickhouse
import (
"testing"
yaml "github.com/goccy/go-yaml"
"github.com/google/go-cmp/cmp"
"github.com/googleapis/genai-toolbox/internal/server"
"github.com/googleapis/genai-toolbox/internal/testutils"
)
func TestParseFromYamlClickHouseExecuteSQL(t *testing.T) {
ctx, err := testutils.ContextWithNewLogger()
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
tcs := []struct {
desc string
in string
want server.ToolConfigs
}{
{
desc: "basic example",
in: `
tools:
example_tool:
kind: clickhouse-execute-sql
source: my-instance
description: some description
`,
want: server.ToolConfigs{
"example_tool": Config{
Name: "example_tool",
Kind: "clickhouse-execute-sql",
Source: "my-instance",
Description: "some description",
AuthRequired: []string{},
},
},
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got := struct {
Tools server.ToolConfigs `yaml:"tools"`
}{}
err := yaml.UnmarshalContext(ctx, testutils.FormatYaml(tc.in), &got)
if err != nil {
t.Fatalf("unable to unmarshal: %s", err)
}
if diff := cmp.Diff(tc.want, got.Tools); diff != "" {
t.Fatalf("incorrect parse: diff %v", diff)
}
})
}
}

View File

@@ -0,0 +1,207 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package clickhouse
import (
"context"
"database/sql"
"fmt"
yaml "github.com/goccy/go-yaml"
"github.com/googleapis/genai-toolbox/internal/sources"
"github.com/googleapis/genai-toolbox/internal/tools"
)
type compatibleSource interface {
ClickHousePool() *sql.DB
}
var compatibleSources = []string{"clickhouse"}
const sqlKind string = "clickhouse-sql"
func init() {
if !tools.Register(sqlKind, newSQLConfig) {
panic(fmt.Sprintf("tool kind %q already registered", sqlKind))
}
}
func newSQLConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.ToolConfig, error) {
actual := Config{Name: name}
if err := decoder.DecodeContext(ctx, &actual); err != nil {
return nil, err
}
return actual, nil
}
type Config struct {
Name string `yaml:"name" validate:"required"`
Kind string `yaml:"kind" validate:"required"`
Source string `yaml:"source" validate:"required"`
Description string `yaml:"description" validate:"required"`
Statement string `yaml:"statement" validate:"required"`
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
TemplateParameters tools.Parameters `yaml:"templateParameters"`
}
var _ tools.ToolConfig = Config{}
func (cfg Config) ToolConfigKind() string {
return sqlKind
}
func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error) {
rawS, ok := srcs[cfg.Source]
if !ok {
return nil, fmt.Errorf("no source named %q configured", cfg.Source)
}
s, ok := rawS.(compatibleSource)
if !ok {
return nil, fmt.Errorf("invalid source for %q tool: source kind must be one of %q", sqlKind, compatibleSources)
}
allParameters, paramManifest, paramMcpManifest, _ := tools.ProcessParameters(cfg.TemplateParameters, cfg.Parameters)
mcpManifest := tools.McpManifest{
Name: cfg.Name,
Description: cfg.Description,
InputSchema: paramMcpManifest,
}
t := Tool{
Name: cfg.Name,
Kind: sqlKind,
Parameters: cfg.Parameters,
TemplateParameters: cfg.TemplateParameters,
AllParams: allParameters,
Statement: cfg.Statement,
AuthRequired: cfg.AuthRequired,
Pool: s.ClickHousePool(),
manifest: tools.Manifest{Description: cfg.Description, Parameters: paramManifest, AuthRequired: cfg.AuthRequired},
mcpManifest: mcpManifest,
}
return t, nil
}
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
TemplateParameters tools.Parameters `yaml:"templateParameters"`
AllParams tools.Parameters `yaml:"allParams"`
Pool *sql.DB
Statement string
manifest tools.Manifest
mcpManifest tools.McpManifest
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, token tools.AccessToken) (any, error) {
paramsMap := params.AsMap()
newStatement, err := tools.ResolveTemplateParams(t.TemplateParameters, t.Statement, paramsMap)
if err != nil {
return nil, fmt.Errorf("unable to extract template params: %w", err)
}
newParams, err := tools.GetParams(t.Parameters, paramsMap)
if err != nil {
return nil, fmt.Errorf("unable to extract standard params: %w", err)
}
sliceParams := newParams.AsSlice()
results, err := t.Pool.QueryContext(ctx, newStatement, sliceParams...)
if err != nil {
return nil, fmt.Errorf("unable to execute query: %w", err)
}
cols, err := results.Columns()
if err != nil {
return nil, fmt.Errorf("unable to retrieve rows column name: %w", err)
}
rawValues := make([]any, len(cols))
values := make([]any, len(cols))
for i := range rawValues {
values[i] = &rawValues[i]
}
colTypes, err := results.ColumnTypes()
if err != nil {
return nil, fmt.Errorf("unable to get column types: %w", err)
}
var out []any
for results.Next() {
err := results.Scan(values...)
if err != nil {
return nil, fmt.Errorf("unable to parse row: %w", err)
}
vMap := make(map[string]any)
for i, name := range cols {
switch colTypes[i].DatabaseTypeName() {
case "String", "FixedString":
if rawValues[i] != nil {
// Handle potential []byte to string conversion if needed
if b, ok := rawValues[i].([]byte); ok {
vMap[name] = string(b)
} else {
vMap[name] = rawValues[i]
}
} else {
vMap[name] = nil
}
default:
vMap[name] = rawValues[i]
}
}
out = append(out, vMap)
}
err = results.Close()
if err != nil {
return nil, fmt.Errorf("unable to close rows: %w", err)
}
if err := results.Err(); err != nil {
return nil, fmt.Errorf("errors encountered by results.Scan: %w", err)
}
return out, nil
}
func (t Tool) ParseParams(data map[string]any, claims map[string]map[string]any) (tools.ParamValues, error) {
return tools.ParseParams(t.AllParams, data, claims)
}
func (t Tool) Manifest() tools.Manifest {
return t.manifest
}
func (t Tool) McpManifest() tools.McpManifest {
return t.mcpManifest
}
func (t Tool) Authorized(verifiedAuthServices []string) bool {
return tools.IsAuthorized(t.AuthRequired, verifiedAuthServices)
}
func (t Tool) RequiresClientAuthorization() bool {
return false
}

View File

@@ -0,0 +1,276 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package clickhouse
import (
"testing"
"github.com/goccy/go-yaml"
"github.com/google/go-cmp/cmp"
"github.com/googleapis/genai-toolbox/internal/server"
"github.com/googleapis/genai-toolbox/internal/sources"
"github.com/googleapis/genai-toolbox/internal/sources/clickhouse"
"github.com/googleapis/genai-toolbox/internal/testutils"
"github.com/googleapis/genai-toolbox/internal/tools"
)
func TestConfigToolConfigKind(t *testing.T) {
config := Config{}
if config.ToolConfigKind() != sqlKind {
t.Errorf("Expected %s, got %s", sqlKind, config.ToolConfigKind())
}
}
func TestParseFromYamlClickHouseSQL(t *testing.T) {
ctx, err := testutils.ContextWithNewLogger()
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
tcs := []struct {
desc string
in string
want server.ToolConfigs
}{
{
desc: "basic example",
in: `
tools:
example_tool:
kind: clickhouse-sql
source: my-instance
description: some description
statement: SELECT 1
`,
want: server.ToolConfigs{
"example_tool": Config{
Name: "example_tool",
Kind: "clickhouse-sql",
Source: "my-instance",
Description: "some description",
Statement: "SELECT 1",
AuthRequired: []string{},
},
},
},
{
desc: "with parameters",
in: `
tools:
param_tool:
kind: clickhouse-sql
source: test-source
description: Test ClickHouse tool
statement: SELECT * FROM test_table WHERE id = $1
parameters:
- name: id
type: string
description: Test ID
`,
want: server.ToolConfigs{
"param_tool": Config{
Name: "param_tool",
Kind: "clickhouse-sql",
Source: "test-source",
Description: "Test ClickHouse tool",
Statement: "SELECT * FROM test_table WHERE id = $1",
Parameters: tools.Parameters{
tools.NewStringParameter("id", "Test ID"),
},
AuthRequired: []string{},
},
},
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got := struct {
Tools server.ToolConfigs `yaml:"tools"`
}{}
err := yaml.UnmarshalContext(ctx, testutils.FormatYaml(tc.in), &got)
if err != nil {
t.Fatalf("unable to unmarshal: %s", err)
}
if diff := cmp.Diff(tc.want, got.Tools); diff != "" {
t.Fatalf("incorrect parse: diff %v", diff)
}
})
}
}
func TestSQLConfigInitializeValidSource(t *testing.T) {
config := Config{
Name: "test-tool",
Kind: sqlKind,
Source: "test-clickhouse",
Description: "Test tool",
Statement: "SELECT 1",
Parameters: tools.Parameters{},
}
// Create a mock ClickHouse source
mockSource := &clickhouse.Source{}
sources := map[string]sources.Source{
"test-clickhouse": mockSource,
}
tool, err := config.Initialize(sources)
if err != nil {
t.Fatalf("Expected no error, got: %v", err)
}
clickhouseTool, ok := tool.(Tool)
if !ok {
t.Fatalf("Expected Tool type, got %T", tool)
}
if clickhouseTool.Name != "test-tool" {
t.Errorf("Expected name 'test-tool', got %s", clickhouseTool.Name)
}
}
func TestSQLConfigInitializeMissingSource(t *testing.T) {
config := Config{
Name: "test-tool",
Kind: sqlKind,
Source: "missing-source",
Description: "Test tool",
Statement: "SELECT 1",
Parameters: tools.Parameters{},
}
sources := map[string]sources.Source{}
_, err := config.Initialize(sources)
if err == nil {
t.Fatal("Expected error for missing source, got nil")
}
expectedErr := `no source named "missing-source" configured`
if err.Error() != expectedErr {
t.Errorf("Expected error %q, got %q", expectedErr, err.Error())
}
}
// mockIncompatibleSource is a mock source that doesn't implement the compatibleSource interface
type mockIncompatibleSource struct{}
func (m *mockIncompatibleSource) SourceKind() string {
return "mock"
}
func TestSQLConfigInitializeIncompatibleSource(t *testing.T) {
config := Config{
Name: "test-tool",
Kind: sqlKind,
Source: "incompatible-source",
Description: "Test tool",
Statement: "SELECT 1",
Parameters: tools.Parameters{},
}
mockSource := &mockIncompatibleSource{}
sources := map[string]sources.Source{
"incompatible-source": mockSource,
}
_, err := config.Initialize(sources)
if err == nil {
t.Fatal("Expected error for incompatible source, got nil")
}
if err.Error() == "" {
t.Error("Expected non-empty error message")
}
}
func TestToolManifest(t *testing.T) {
tool := Tool{
manifest: tools.Manifest{
Description: "Test description",
Parameters: []tools.ParameterManifest{},
},
}
manifest := tool.Manifest()
if manifest.Description != "Test description" {
t.Errorf("Expected description 'Test description', got %s", manifest.Description)
}
}
func TestToolMcpManifest(t *testing.T) {
tool := Tool{
mcpManifest: tools.McpManifest{
Name: "test-tool",
Description: "Test description",
},
}
manifest := tool.McpManifest()
if manifest.Name != "test-tool" {
t.Errorf("Expected name 'test-tool', got %s", manifest.Name)
}
if manifest.Description != "Test description" {
t.Errorf("Expected description 'Test description', got %s", manifest.Description)
}
}
func TestToolAuthorized(t *testing.T) {
tests := []struct {
name string
authRequired []string
verifiedAuthServices []string
expectedAuthorized bool
}{
{
name: "no auth required",
authRequired: []string{},
verifiedAuthServices: []string{},
expectedAuthorized: true,
},
{
name: "auth required and verified",
authRequired: []string{"google"},
verifiedAuthServices: []string{"google"},
expectedAuthorized: true,
},
{
name: "auth required but not verified",
authRequired: []string{"google"},
verifiedAuthServices: []string{},
expectedAuthorized: false,
},
{
name: "auth required but different service verified",
authRequired: []string{"google"},
verifiedAuthServices: []string{"aws"},
expectedAuthorized: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tool := Tool{
AuthRequired: tt.authRequired,
}
authorized := tool.Authorized(tt.verifiedAuthServices)
if authorized != tt.expectedAuthorized {
t.Errorf("Expected authorized %t, got %t", tt.expectedAuthorized, authorized)
}
})
}
}

View File

@@ -16,6 +16,7 @@ package lookercommon
import (
"context"
"fmt"
"strings"
"github.com/googleapis/genai-toolbox/internal/tools"
"github.com/googleapis/genai-toolbox/internal/util"
@@ -24,14 +25,14 @@ import (
)
const (
DimensionsFields = "fields(dimensions(name,type,label,label_short,description))"
FiltersFields = "fields(filters(name,type,label,label_short,description))"
MeasuresFields = "fields(measures(name,type,label,label_short,description))"
ParametersFields = "fields(parameters(name,type,label,label_short,description))"
DimensionsFields = "fields(dimensions(name,type,label,label_short,description,synonyms,tags,hidden))"
FiltersFields = "fields(filters(name,type,label,label_short,description,synonyms,tags,hidden))"
MeasuresFields = "fields(measures(name,type,label,label_short,description,synonyms,tags,hidden))"
ParametersFields = "fields(parameters(name,type,label,label_short,description,synonyms,tags,hidden))"
)
// ExtractLookerFieldProperties extracts common properties from Looker field objects.
func ExtractLookerFieldProperties(ctx context.Context, fields *[]v4.LookmlModelExploreField) ([]any, error) {
func ExtractLookerFieldProperties(ctx context.Context, fields *[]v4.LookmlModelExploreField, showHiddenFields bool) ([]any, error) {
data := make([]any, 0)
// Handle nil fields pointer
@@ -48,6 +49,12 @@ func ExtractLookerFieldProperties(ctx context.Context, fields *[]v4.LookmlModelE
for _, v := range *fields {
logger.DebugContext(ctx, "Got response element of %v\n", v)
if v.Name != nil && strings.HasSuffix(*v.Name, "_raw") {
continue
}
if !showHiddenFields && v.Hidden != nil && *v.Hidden {
continue
}
vMap := make(map[string]any)
if v.Name != nil {
vMap["name"] = *v.Name
@@ -64,6 +71,12 @@ func ExtractLookerFieldProperties(ctx context.Context, fields *[]v4.LookmlModelE
if v.Description != nil {
vMap["description"] = *v.Description
}
if v.Tags != nil {
vMap["tags"] = *v.Tags
}
if v.Synonyms != nil {
vMap["synonyms"] = *v.Synonyms
}
logger.DebugContext(ctx, "Converted to %v\n", vMap)
data = append(data, vMap)
}

View File

@@ -132,7 +132,7 @@ func TestExtractLookerFieldProperties(t *testing.T) {
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got, err := lookercommon.ExtractLookerFieldProperties(ctx, &tc.fields)
got, err := lookercommon.ExtractLookerFieldProperties(ctx, &tc.fields, true)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -150,7 +150,7 @@ func TestExtractLookerFieldPropertiesWithNilFields(t *testing.T) {
t.Fatalf("unexpected error: %s", err)
}
got, err := lookercommon.ExtractLookerFieldProperties(ctx, nil)
got, err := lookercommon.ExtractLookerFieldProperties(ctx, nil, true)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}

View File

@@ -184,7 +184,7 @@ func (t Tool) McpManifest() tools.McpManifest {
}
func (t Tool) Authorized(verifiedAuthServices []string) bool {
return true
return tools.IsAuthorized(t.AuthRequired, verifiedAuthServices)
}
func (t Tool) RequiresClientAuthorization() bool {

View File

@@ -93,7 +93,8 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
Parameters: parameters.Manifest(),
AuthRequired: cfg.AuthRequired,
},
mcpManifest: mcpManifest,
mcpManifest: mcpManifest,
ShowHiddenFields: s.ShowHiddenFields,
}, nil
}
@@ -101,14 +102,15 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
ShowHiddenFields bool
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
@@ -136,7 +138,7 @@ func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken
return nil, fmt.Errorf("error processing get_dimensions response: %w", err)
}
data, err := lookercommon.ExtractLookerFieldProperties(ctx, resp.Fields.Dimensions)
data, err := lookercommon.ExtractLookerFieldProperties(ctx, resp.Fields.Dimensions, t.ShowHiddenFields)
if err != nil {
return nil, fmt.Errorf("error extracting get_dimensions response: %w", err)
}

View File

@@ -93,7 +93,8 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
Parameters: parameters.Manifest(),
AuthRequired: cfg.AuthRequired,
},
mcpManifest: mcpManifest,
mcpManifest: mcpManifest,
ShowHiddenExplores: s.ShowHiddenExplores,
}, nil
}
@@ -101,14 +102,15 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
ShowHiddenExplores bool
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
@@ -122,7 +124,7 @@ func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken
return nil, fmt.Errorf("'model' must be a string, got %T", mapParams["model"])
}
resp, err := t.Client.LookmlModel(model, "explores(name,label,group_label)", t.ApiSettings)
resp, err := t.Client.LookmlModel(model, "explores(name,description,label,group_label,hidden)", t.ApiSettings)
if err != nil {
return nil, fmt.Errorf("error making get_explores request: %s", err)
}
@@ -130,6 +132,9 @@ func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken
var data []any
for _, v := range *resp.Explores {
logger.DebugContext(ctx, "Got response element of %v\n", v)
if !t.ShowHiddenExplores && v.Hidden != nil && *v.Hidden {
continue
}
vMap := make(map[string]any)
if v.Name != nil {
vMap["name"] = *v.Name

View File

@@ -93,7 +93,8 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
Parameters: parameters.Manifest(),
AuthRequired: cfg.AuthRequired,
},
mcpManifest: mcpManifest,
mcpManifest: mcpManifest,
ShowHiddenFields: s.ShowHiddenFields,
}, nil
}
@@ -101,14 +102,15 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
ShowHiddenFields bool
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
@@ -136,7 +138,7 @@ func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken
return nil, fmt.Errorf("error processing get_filters response: %w", err)
}
data, err := lookercommon.ExtractLookerFieldProperties(ctx, resp.Fields.Filters)
data, err := lookercommon.ExtractLookerFieldProperties(ctx, resp.Fields.Filters, t.ShowHiddenFields)
if err != nil {
return nil, fmt.Errorf("error extracting get_filters response: %w", err)
}

View File

@@ -93,7 +93,8 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
Parameters: parameters.Manifest(),
AuthRequired: cfg.AuthRequired,
},
mcpManifest: mcpManifest,
mcpManifest: mcpManifest,
ShowHiddenFields: s.ShowHiddenFields,
}, nil
}
@@ -101,14 +102,15 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
ShowHiddenFields bool
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
@@ -136,7 +138,7 @@ func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken
return nil, fmt.Errorf("error processing get_measures response: %w", err)
}
data, err := lookercommon.ExtractLookerFieldProperties(ctx, resp.Fields.Measures)
data, err := lookercommon.ExtractLookerFieldProperties(ctx, resp.Fields.Measures, t.ShowHiddenFields)
if err != nil {
return nil, fmt.Errorf("error extracting get_measures response: %w", err)
}

View File

@@ -92,7 +92,8 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
Parameters: parameters.Manifest(),
AuthRequired: cfg.AuthRequired,
},
mcpManifest: mcpManifest,
mcpManifest: mcpManifest,
ShowHiddenModels: s.ShowHiddenModels,
}, nil
}
@@ -100,14 +101,15 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
ShowHiddenModels bool
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
@@ -117,7 +119,7 @@ func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken
}
excludeEmpty := false
excludeHidden := false
excludeHidden := !t.ShowHiddenModels
includeInternal := true
req := v4.RequestAllLookmlModels{

View File

@@ -93,7 +93,8 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
Parameters: parameters.Manifest(),
AuthRequired: cfg.AuthRequired,
},
mcpManifest: mcpManifest,
mcpManifest: mcpManifest,
ShowHiddenFields: s.ShowHiddenFields,
}, nil
}
@@ -101,14 +102,15 @@ func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error)
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
Name string `yaml:"name"`
Kind string `yaml:"kind"`
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
ShowHiddenFields bool
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
@@ -136,7 +138,7 @@ func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken
return nil, fmt.Errorf("error processing get_parameters response: %w", err)
}
data, err := lookercommon.ExtractLookerFieldProperties(ctx, resp.Fields.Parameters)
data, err := lookercommon.ExtractLookerFieldProperties(ctx, resp.Fields.Parameters, t.ShowHiddenFields)
if err != nil {
return nil, fmt.Errorf("error extracting get_parameters response: %w", err)
}

View File

@@ -107,7 +107,7 @@ func parseFromAuthService(paramAuthServices []ParamAuthService, claimsMap map[st
}
return v, nil
}
return nil, fmt.Errorf("missing or invalid authentication header")
return nil, fmt.Errorf("missing or invalid authentication header: %w", ErrUnauthorized)
}
// CheckParamRequired checks if a parameter is required based on the required and default field.

View File

@@ -16,6 +16,7 @@ package tools
import (
"context"
"errors"
"fmt"
"slices"
@@ -91,6 +92,8 @@ type McpManifest struct {
InputSchema McpToolsSchema `json:"inputSchema,omitempty"`
}
var ErrUnauthorized = errors.New("unauthorized")
// Helper function that returns if a tool invocation request is authorized
func IsAuthorized(authRequiredSources []string, verifiedAuthServices []string) bool {
if len(authRequiredSources) == 0 {

View File

@@ -165,12 +165,12 @@ func TestAlloyDBPgToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, failInvocationWant, createTableStatement := tests.GetPostgresWants()
select1Want, failInvocationWant, createTableStatement, mcpSelect1Want := tests.GetPostgresWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want)
tests.RunMCPToolCallMethod(t, failInvocationWant)
tests.RunMCPToolCallMethod(t, failInvocationWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}

View File

@@ -158,15 +158,17 @@ func TestBigQueryToolEndpoints(t *testing.T) {
datasetInfoWant := "\"Location\":\"US\",\"DefaultTableExpiration\":0,\"Labels\":null,\"Access\":"
tableInfoWant := "{\"Name\":\"\",\"Location\":\"US\",\"Description\":\"\",\"Schema\":[{\"Name\":\"id\""
ddlWant := `"Query executed successfully and returned no content."`
dataInsightsWant := `(?s)Schema Resolved.*Retrieval Query.*SQL Generated.*Answer`
// Partial message; the full error message is too long.
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"final query validation failed: failed to insert dry run job: googleapi: Error 400: Syntax error: Unexpected identifier \"SELEC\" at [1:1]`
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"f0_\":1}"}]}}`
createColArray := `["id INT64", "name STRING", "age INT64"]`
selectEmptyWant := `"The query returned 0 rows."`
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want, tests.DisableOptionalNullParamTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunToolInvokeTest(t, select1Want, tests.DisableOptionalNullParamTest(), tests.EnableClientAuthTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want, tests.EnableMcpClientAuthTest())
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam,
tests.WithCreateColArray(createColArray),
tests.WithDdlWant(ddlWant),
@@ -182,6 +184,7 @@ func TestBigQueryToolEndpoints(t *testing.T) {
runBigQueryGetDatasetInfoToolInvokeTest(t, datasetName, datasetInfoWant)
runBigQueryListTableIdsToolInvokeTest(t, datasetName, tableName)
runBigQueryGetTableInfoToolInvokeTest(t, datasetName, tableName, tableInfoWant)
runBigQueryConversationalAnalyticsInvokeTest(t, datasetName, tableName, dataInsightsWant)
}
// getBigQueryParamToolInfo returns statements and param for my-tool for bigquery kind
@@ -420,6 +423,19 @@ func addBigQueryPrebuiltToolsConfig(t *testing.T, config map[string]any) map[str
"my-google-auth",
},
}
tools["my-conversational-analytics-tool"] = map[string]any{
"kind": "bigquery-conversational-analytics",
"source": "my-instance",
"description": "Tool to ask BigQuery conversational analytics",
}
tools["my-auth-conversational-analytics-tool"] = map[string]any{
"kind": "bigquery-conversational-analytics",
"source": "my-instance",
"description": "Tool to ask BigQuery conversational analytics",
"authRequired": []string{
"my-google-auth",
},
}
config["tools"] = tools
return config
}
@@ -453,7 +469,13 @@ func addBigQuerySqlToolConfig(t *testing.T, config map[string]any, toolStatement
map[string]any{"name": "bool_array", "type": "array", "description": "an array of boolean values", "items": map[string]any{"name": "item", "type": "boolean", "description": "desc"}},
},
}
tools["my-client-auth-tool"] = map[string]any{
"kind": "bigquery-sql",
"source": "my-instance",
"description": "Tool to test client authorization.",
"useClientOAuth": true,
"statement": "SELECT 1",
}
config["tools"] = tools
return config
}
@@ -1349,3 +1371,94 @@ func runBigQueryGetTableInfoToolInvokeTest(t *testing.T, datasetName, tableName,
})
}
}
func runBigQueryConversationalAnalyticsInvokeTest(t *testing.T, datasetName, tableName, dataInsightsWant string) {
// Get ID token
idToken, err := tests.GetGoogleIdToken(tests.ClientId)
if err != nil {
t.Fatalf("error getting Google ID token: %s", err)
}
tableRefsJSON := fmt.Sprintf(`[{"projectId":"%s","datasetId":"%s","tableId":"%s"}]`, BigqueryProject, datasetName, tableName)
invokeTcs := []struct {
name string
api string
requestHeader map[string]string
requestBody io.Reader
want string
isErr bool
}{
{
name: "invoke my-conversational-analytics-tool successfully",
api: "http://127.0.0.1:5000/api/tool/my-conversational-analytics-tool/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(fmt.Sprintf(
`{"user_query_with_context": "What are the names in the table?", "table_references": %q}`,
tableRefsJSON,
))),
want: dataInsightsWant,
isErr: false,
},
{
name: "invoke my-auth-conversational-analytics-tool with auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-conversational-analytics-tool/invoke",
requestHeader: map[string]string{"my-google-auth_token": idToken},
requestBody: bytes.NewBuffer([]byte(fmt.Sprintf(
`{"user_query_with_context": "What are the names in the table?", "table_references": %q}`,
tableRefsJSON,
))),
want: dataInsightsWant,
isErr: false,
},
{
name: "invoke my-auth-conversational-analytics-tool without auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-conversational-analytics-tool/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{"user_query_with_context": "What are the names in the table?"}`)),
isErr: true,
},
}
for _, tc := range invokeTcs {
t.Run(tc.name, func(t *testing.T) {
// Send Tool invocation request
req, err := http.NewRequest(http.MethodPost, tc.api, tc.requestBody)
if err != nil {
t.Fatalf("unable to create request: %s", err)
}
req.Header.Add("Content-type", "application/json")
for k, v := range tc.requestHeader {
req.Header.Add(k, v)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("unable to send request: %s", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
if tc.isErr {
return
}
bodyBytes, _ := io.ReadAll(resp.Body)
t.Fatalf("response status code is not 200, got %d: %s", resp.StatusCode, string(bodyBytes))
}
var body map[string]interface{}
err = json.NewDecoder(resp.Body).Decode(&body)
if err != nil {
t.Fatalf("error parsing response body: %v", err)
}
got, ok := body["result"].(string)
if !ok {
t.Fatalf("unable to find result in response body")
}
wantPattern := regexp.MustCompile(tc.want)
if !wantPattern.MatchString(got) {
t.Fatalf("response did not match the expected pattern.\nFull response:\n%s", got)
}
})
}
}

View File

@@ -121,6 +121,7 @@ func TestBigtableToolEndpoints(t *testing.T) {
select1Want := "[{\"$col1\":1}]"
myToolById4Want := `[{"id":4,"name":""}]`
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to prepare statement: rpc error: code = InvalidArgument desc = Syntax error: Unexpected identifier \"SELEC\" [at 1:1]"}],"isError":true}}`
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"$col1\":1}"}]}}`
nameFieldArray := `["CAST(cf['name'] AS string) as name"]`
nameColFilter := "CAST(cf['name'] AS string)"
@@ -129,7 +130,7 @@ func TestBigtableToolEndpoints(t *testing.T) {
tests.RunToolInvokeTest(t, select1Want,
tests.WithMyToolById4Want(myToolById4Want),
)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam,
tests.WithNameFieldArray(nameFieldArray),
tests.WithNameColFilter(nameColFilter),

File diff suppressed because it is too large Load Diff

View File

@@ -159,12 +159,12 @@ func TestCloudSQLMSSQLToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, createTableStatement := tests.GetMSSQLWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := tests.GetMSSQLWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want, tests.DisableArrayTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}

View File

@@ -146,12 +146,12 @@ func TestCloudSQLMySQLToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, createTableStatement := tests.GetMySQLWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := tests.GetMySQLWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want, tests.DisableArrayTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}

View File

@@ -150,12 +150,12 @@ func TestCloudSQLPgSimpleToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, createTableStatement := tests.GetPostgresWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := tests.GetPostgresWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}

View File

@@ -400,27 +400,30 @@ func GetMySQLTmplToolStatement() (string, string) {
}
// GetPostgresWants return the expected wants for postgres
func GetPostgresWants() (string, string, string) {
func GetPostgresWants() (string, string, string, string) {
select1Want := "[{\"?column?\":1}]"
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: ERROR: syntax error at or near \"SELEC\" (SQLSTATE 42601)"}],"isError":true}}`
createTableStatement := `"CREATE TABLE t (id SERIAL PRIMARY KEY, name TEXT)"`
return select1Want, mcpMyFailToolWant, createTableStatement
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"?column?\":1}"}]}}`
return select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want
}
// GetMSSQLWants return the expected wants for mssql
func GetMSSQLWants() (string, string, string) {
func GetMSSQLWants() (string, string, string, string) {
select1Want := "[{\"\":1}]"
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: mssql: Could not find stored procedure 'SELEC'."}],"isError":true}}`
createTableStatement := `"CREATE TABLE t (id INT IDENTITY(1,1) PRIMARY KEY, name NVARCHAR(MAX))"`
return select1Want, mcpMyFailToolWant, createTableStatement
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"\":1}"}]}}`
return select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want
}
// GetMySQLWants return the expected wants for mysql
func GetMySQLWants() (string, string, string) {
func GetMySQLWants() (string, string, string, string) {
select1Want := "[{\"1\":1}]"
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: Error 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SELEC 1' at line 1"}],"isError":true}}`
createTableStatement := `"CREATE TABLE t (id SERIAL PRIMARY KEY, name TEXT)"`
return select1Want, mcpMyFailToolWant, createTableStatement
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"1\":1}"}]}}`
return select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want
}
// SetupPostgresSQLTable creates and inserts data into a table of tool
@@ -511,14 +514,15 @@ func SetupMySQLTable(t *testing.T, ctx context.Context, pool *sql.DB, createStat
}
// GetRedisWants return the expected wants for redis
func GetRedisValkeyWants() (string, string, string, string, string, string) {
func GetRedisValkeyWants() (string, string, string, string, string, string, string) {
select1Want := "[\"PONG\"]"
mcpMyFailToolWant := `unknown command 'SELEC 1;', with args beginning with: \""}]}}`
invokeParamWant := "[{\"id\":\"1\",\"name\":\"Alice\"},{\"id\":\"3\",\"name\":\"Sid\"}]"
invokeIdNullWant := `[{"id":"4","name":""}]`
nullWant := `["null"]`
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"\"PONG\""}]}}`
mcpInvokeParamWant := `{"jsonrpc":"2.0","id":"my-tool","result":{"content":[{"type":"text","text":"{\"id\":\"1\",\"name\":\"Alice\"}"},{"type":"text","text":"{\"id\":\"3\",\"name\":\"Sid\"}"}]}}`
return select1Want, mcpMyFailToolWant, invokeParamWant, invokeIdNullWant, nullWant, mcpInvokeParamWant
return select1Want, mcpMyFailToolWant, invokeParamWant, invokeIdNullWant, nullWant, mcpSelect1Want, mcpInvokeParamWant
}
func GetRedisValkeyToolsConfig(sourceConfig map[string]any, toolKind string) map[string]any {

View File

@@ -138,13 +138,14 @@ func TestCouchbaseToolEndpoints(t *testing.T) {
// Get configs for tests
select1Want := "[{\"$1\":1}]"
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: parsing failure | {\"statement\":\"SELEC 1;\"`
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"$1\":1}"}]}}`
tmplSelectId1Want := "[{\"age\":21,\"id\":1,\"name\":\"Alex\"}]"
selectAllWant := "[{\"age\":21,\"id\":1,\"name\":\"Alex\"},{\"age\":100,\"id\":2,\"name\":\"Alice\"}]"
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunToolInvokeWithTemplateParameters(t, collectionNameTemplateParam,
tests.WithTmplSelectId1Want(tmplSelectId1Want),
tests.WithSelectAllWant(selectAllWant),

View File

@@ -127,7 +127,7 @@ func TestFirebirdToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, failInvocationWant, createTableStatement := getFirebirdWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := getFirebirdWants()
nullWant := `[{"id":4,"name":null}]`
select1Statement := `"SELECT 1 AS \"constant\" FROM RDB$DATABASE;"`
templateParamCreateColArray := `["id INTEGER","name VARCHAR(255)","age INTEGER"]`
@@ -137,7 +137,7 @@ func TestFirebirdToolEndpoints(t *testing.T) {
tests.RunToolInvokeTest(t, select1Want,
tests.WithNullWant(nullWant),
tests.DisableArrayTest())
tests.RunMCPToolCallMethod(t, failInvocationWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want, tests.WithSelect1Statement(select1Statement))
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam,
tests.WithCreateColArray(templateParamCreateColArray))
@@ -303,11 +303,12 @@ func getFirebirdAuthToolInfo(tableName string) ([]string, string, string, []any)
return createStatements, insertStatement, toolStatement, params
}
func getFirebirdWants() (string, string, string) {
func getFirebirdWants() (string, string, string, string) {
select1Want := `[{"constant":1}]`
failInvocationWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: Dynamic SQL Error\nSQL error code = -104\nToken unknown - line 1, column 1\nSELEC\n"}],"isError":true}}`
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: Dynamic SQL Error\nSQL error code = -104\nToken unknown - line 1, column 1\nSELEC\n"}],"isError":true}}`
createTableStatement := `"CREATE TABLE t (id INTEGER PRIMARY KEY, name VARCHAR(50))"`
return select1Want, failInvocationWant, createTableStatement
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"constant\":1}"}]}}`
return select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want
}
func getFirebirdToolsConfig(sourceConfig map[string]any, toolKind, paramToolStatement, idParamToolStmt, nameParamToolStmt, arrayToolStatement, authToolStatement string) map[string]any {

View File

@@ -624,10 +624,10 @@ func TestLooker(t *testing.T) {
wantResult = "{\"group_label\":\"System Activity\",\"label\":\"Content Usage\",\"name\":\"content_usage\"}"
tests.RunToolInvokeParametersTest(t, "get_explores", []byte(`{"model": "system__activity"}`), wantResult)
wantResult = "{\"description\":\"Number of times this content has been viewed via the Looker API\",\"label\":\"Content Usage API Count\",\"label_short\":\"API Count\",\"name\":\"content_usage.api_count\",\"type\":\"number\"}"
wantResult = "{\"description\":\"Number of times this content has been viewed via the Looker API\",\"label\":\"Content Usage API Count\",\"label_short\":\"API Count\",\"name\":\"content_usage.api_count\",\"synonyms\":[],\"tags\":[],\"type\":\"number\"}"
tests.RunToolInvokeParametersTest(t, "get_dimensions", []byte(`{"model": "system__activity", "explore": "content_usage"}`), wantResult)
wantResult = "{\"description\":\"The total number of views via the Looker API\",\"label\":\"Content Usage API Total\",\"label_short\":\"API Total\",\"name\":\"content_usage.api_total\",\"type\":\"sum\"}"
wantResult = "{\"description\":\"The total number of views via the Looker API\",\"label\":\"Content Usage API Total\",\"label_short\":\"API Total\",\"name\":\"content_usage.api_total\",\"synonyms\":[],\"tags\":[],\"type\":\"sum\"}"
tests.RunToolInvokeParametersTest(t, "get_measures", []byte(`{"model": "system__activity", "explore": "content_usage"}`), wantResult)
wantResult = "[]"

View File

@@ -112,7 +112,7 @@ func TestMongoDBToolEndpoints(t *testing.T) {
tests.WithMyToolId3NameAliceWant(myToolId3NameAliceWant),
tests.WithMyToolById4Want(myToolById4Want),
)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant,
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, select1Want,
tests.WithMcpMyToolId3NameAliceWant(mcpMyToolId3NameAliceWant),
)

View File

@@ -132,12 +132,12 @@ func TestMSSQLToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, createTableStatement := tests.GetMSSQLWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := tests.GetMSSQLWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want, tests.DisableArrayTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}

View File

@@ -123,12 +123,12 @@ func TestMySQLToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, createTableStatement := tests.GetMySQLWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := tests.GetMySQLWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want, tests.DisableArrayTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}

View File

@@ -125,12 +125,12 @@ func TestOceanBaseToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, createTableStatement := getOceanBaseWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := getOceanBaseWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want, tests.DisableArrayTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}
@@ -164,11 +164,12 @@ func getOceanBaseTmplToolStatement() (string, string) {
}
// OceanBase specific expected results
func getOceanBaseWants() (string, string, string) {
func getOceanBaseWants() (string, string, string, string) {
select1Want := "[{\"1\":1}]"
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: Error 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your OceanBase version for the right syntax to use near 'SELEC 1;' at line 1"}],"isError":true}}`
createTableStatement := `"CREATE TABLE t (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, name VARCHAR(255))"`
return select1Want, mcpMyFailToolWant, createTableStatement
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"1\":1}"}]}}`
return select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want
}
// Add OceanBase Execute SQL configuration

View File

@@ -23,6 +23,7 @@ type InvokeTestConfig struct {
nullWant string
supportOptionalNullParam bool
supportArrayParam bool
supportClientAuth bool
}
type InvokeTestOption func(*InvokeTestConfig)
@@ -68,11 +69,21 @@ func DisableArrayTest() InvokeTestOption {
}
}
// EnableClientAuthTest runs the client authorization tests.
// Only enable it if your source supports the `useClientOAuth` configuration.
// Currently, this should only be used with the BigQuery tests.
func EnableClientAuthTest() InvokeTestOption {
return func(c *InvokeTestConfig) {
c.supportClientAuth = true
}
}
/* Configurations for RunMCPToolCallMethod() */
// MCPTestConfig represents the various configuration options for mcp tool call tests.
type MCPTestConfig struct {
myToolId3NameAliceWant string
supportClientAuth bool
}
type McpTestOption func(*MCPTestConfig)
@@ -85,6 +96,15 @@ func WithMcpMyToolId3NameAliceWant(s string) McpTestOption {
}
}
// EnableMcpClientAuthTest runs the client authorization tests.
// Only enable it if your source supports the `useClientOAuth` configuration.
// Currently, this should only be used with the BigQuery tests.
func EnableMcpClientAuthTest() McpTestOption {
return func(c *MCPTestConfig) {
c.supportClientAuth = true
}
}
/* Configurations for RunExecuteSqlToolInvokeTest() */
// ExecuteSqlTestConfig represents the various configuration options for RunExecuteSqlToolInvokeTest()

View File

@@ -129,12 +129,12 @@ func TestPostgres(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, createTableStatement := tests.GetPostgresWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := tests.GetPostgresWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}

View File

@@ -98,7 +98,7 @@ func TestRedisToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, invokeParamWant, invokeIdNullWant, nullWant, mcpInvokeParamWant := tests.GetRedisValkeyWants()
select1Want, mcpMyFailToolWant, invokeParamWant, invokeIdNullWant, nullWant, mcpSelect1Want, mcpInvokeParamWant := tests.GetRedisValkeyWants()
// Run tests
tests.RunToolGetTest(t)
@@ -107,7 +107,7 @@ func TestRedisToolEndpoints(t *testing.T) {
tests.WithMyToolById4Want(invokeIdNullWant),
tests.WithNullWant(nullWant),
)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant,
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want,
tests.WithMcpMyToolId3NameAliceWant(mcpInvokeParamWant),
)
}

View File

@@ -155,6 +155,7 @@ func TestSpannerToolEndpoints(t *testing.T) {
toolInvokeMyToolById4Want := `[{"id":"4","name":null}]`
mcpMyFailToolWant := `"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute client: unable to parse row: spanner: code = \"InvalidArgument\", desc = \"Syntax error: Unexpected identifier \\\\\\\"SELEC\\\\\\\" [at 1:1]\\\\nSELEC 1;\\\\n^\"`
mcpMyToolId3NameAliceWant := `{"jsonrpc":"2.0","id":"my-tool","result":{"content":[{"type":"text","text":"{\"id\":\"1\",\"name\":\"Alice\"}"},{"type":"text","text":"{\"id\":\"3\",\"name\":\"Sid\"}"}]}}`
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"\":\"1\"}"}]}}`
tmplSelectAllWwant := "[{\"age\":\"21\",\"id\":\"1\",\"name\":\"Alex\"},{\"age\":\"100\",\"id\":\"2\",\"name\":\"Alice\"}]"
tmplSelectId1Want := "[{\"age\":\"21\",\"id\":\"1\",\"name\":\"Alex\"}]"
@@ -164,7 +165,7 @@ func TestSpannerToolEndpoints(t *testing.T) {
tests.WithMyToolId3NameAliceWant(invokeParamWant),
tests.WithMyToolById4Want(toolInvokeMyToolById4Want),
)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, tests.WithMcpMyToolId3NameAliceWant(mcpMyToolId3NameAliceWant))
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want, tests.WithMcpMyToolId3NameAliceWant(mcpMyToolId3NameAliceWant))
tests.RunToolInvokeWithTemplateParameters(
t, tableNameTemplateParam,
tests.WithSelectAllWant(tmplSelectAllWwant),

View File

@@ -156,10 +156,11 @@ func TestSQLiteToolEndpoint(t *testing.T) {
// Get configs for tests
select1Want := "[{\"1\":1}]"
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: SQL logic error: near \"SELEC\": syntax error (1)"}],"isError":true}}`
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"1\":1}"}]}}`
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want, tests.DisableArrayTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}

View File

@@ -76,11 +76,12 @@ func initTiDBConnectionPool(host, port, user, pass, dbname string, useSSL bool)
}
// getTiDBWants return the expected wants for tidb
func getTiDBWants() (string, string, string) {
func getTiDBWants() (string, string, string, string) {
select1Want := "[{\"1\":1}]"
mcpMyFailToolWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: Error 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your TiDB version for the right syntax to use line 1 column 5 near \"SELEC 1;\" "}],"isError":true}}`
createTableStatement := `"CREATE TABLE t (id SERIAL PRIMARY KEY, name TEXT)"`
return select1Want, mcpMyFailToolWant, createTableStatement
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"1\":1}"}]}}`
return select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want
}
// addTiDBExecuteSqlConfig gets the tools config for `tidb-execute-sql`
@@ -154,12 +155,12 @@ func TestTiDBToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, createTableStatement := getTiDBWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := getTiDBWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want, tests.DisableArrayTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam)
}

View File

@@ -25,6 +25,7 @@ import (
"testing"
"github.com/googleapis/genai-toolbox/internal/server/mcp/jsonrpc"
"github.com/googleapis/genai-toolbox/internal/sources"
)
// RunToolGet runs the tool get endpoint
@@ -253,6 +254,7 @@ func RunToolInvokeTest(t *testing.T, select1Want string, options ...InvokeTestOp
nullWant: "null",
supportOptionalNullParam: true,
supportArrayParam: true,
supportClientAuth: false,
}
// Apply provided options
@@ -266,122 +268,175 @@ func RunToolInvokeTest(t *testing.T, select1Want string, options ...InvokeTestOp
t.Fatalf("error getting Google ID token: %s", err)
}
// Get access token
accessToken, err := sources.GetIAMAccessToken(t.Context())
if err != nil {
t.Fatalf("error getting access token from ADC: %s", err)
}
// Test tool invoke endpoint
invokeTcs := []struct {
name string
api string
requestHeader map[string]string
requestBody io.Reader
want string
isErr bool
name string
api string
enabled bool
requestHeader map[string]string
requestBody io.Reader
wantStatusCode int
wantBody string
}{
{
name: "invoke my-simple-tool",
api: "http://127.0.0.1:5000/api/tool/my-simple-tool/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
want: select1Want,
isErr: false,
name: "invoke my-simple-tool",
api: "http://127.0.0.1:5000/api/tool/my-simple-tool/invoke",
enabled: true,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantBody: select1Want,
wantStatusCode: http.StatusOK,
},
{
name: "invoke my-tool",
api: "http://127.0.0.1:5000/api/tool/my-tool/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{"id": 3, "name": "Alice"}`)),
want: configs.myToolId3NameAliceWant,
isErr: false,
name: "invoke my-tool",
api: "http://127.0.0.1:5000/api/tool/my-tool/invoke",
enabled: true,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{"id": 3, "name": "Alice"}`)),
wantBody: configs.myToolId3NameAliceWant,
wantStatusCode: http.StatusOK,
},
{
name: "invoke my-tool-by-id with nil response",
api: "http://127.0.0.1:5000/api/tool/my-tool-by-id/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{"id": 4}`)),
want: configs.myToolById4Want,
isErr: false,
name: "invoke my-tool-by-id with nil response",
api: "http://127.0.0.1:5000/api/tool/my-tool-by-id/invoke",
enabled: true,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{"id": 4}`)),
wantBody: configs.myToolById4Want,
wantStatusCode: http.StatusOK,
},
{
name: "invoke my-tool-by-name with nil response",
api: "http://127.0.0.1:5000/api/tool/my-tool-by-name/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
want: configs.nullWant,
isErr: !configs.supportOptionalNullParam,
name: "invoke my-tool-by-name with nil response",
api: "http://127.0.0.1:5000/api/tool/my-tool-by-name/invoke",
enabled: configs.supportOptionalNullParam,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantBody: configs.nullWant,
wantStatusCode: http.StatusOK,
},
{
name: "Invoke my-tool without parameters",
api: "http://127.0.0.1:5000/api/tool/my-tool/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
isErr: true,
name: "Invoke my-tool without parameters",
api: "http://127.0.0.1:5000/api/tool/my-tool/invoke",
enabled: true,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantBody: "",
wantStatusCode: http.StatusBadRequest,
},
{
name: "Invoke my-tool with insufficient parameters",
api: "http://127.0.0.1:5000/api/tool/my-tool/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{"id": 1}`)),
isErr: true,
name: "Invoke my-tool with insufficient parameters",
api: "http://127.0.0.1:5000/api/tool/my-tool/invoke",
enabled: true,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{"id": 1}`)),
wantBody: "",
wantStatusCode: http.StatusBadRequest,
},
{
name: "invoke my-array-tool",
api: "http://127.0.0.1:5000/api/tool/my-array-tool/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{"idArray": [1,2,3], "nameArray": ["Alice", "Sid", "RandomName"], "cmdArray": ["HGETALL", "row3"]}`)),
want: configs.myToolId3NameAliceWant,
isErr: !configs.supportArrayParam,
name: "invoke my-array-tool",
api: "http://127.0.0.1:5000/api/tool/my-array-tool/invoke",
enabled: configs.supportArrayParam,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{"idArray": [1,2,3], "nameArray": ["Alice", "Sid", "RandomName"], "cmdArray": ["HGETALL", "row3"]}`)),
wantBody: configs.myToolId3NameAliceWant,
wantStatusCode: http.StatusOK,
},
{
name: "Invoke my-auth-tool with auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-tool/invoke",
requestHeader: map[string]string{"my-google-auth_token": idToken},
requestBody: bytes.NewBuffer([]byte(`{}`)),
want: "[{\"name\":\"Alice\"}]",
isErr: false,
name: "Invoke my-auth-tool with auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-tool/invoke",
enabled: true,
requestHeader: map[string]string{"my-google-auth_token": idToken},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantBody: "[{\"name\":\"Alice\"}]",
wantStatusCode: http.StatusOK,
},
{
name: "Invoke my-auth-tool with invalid auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-tool/invoke",
requestHeader: map[string]string{"my-google-auth_token": "INVALID_TOKEN"},
requestBody: bytes.NewBuffer([]byte(`{}`)),
isErr: true,
name: "Invoke my-auth-tool with invalid auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-tool/invoke",
enabled: true,
requestHeader: map[string]string{"my-google-auth_token": "INVALID_TOKEN"},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantStatusCode: http.StatusUnauthorized,
},
{
name: "Invoke my-auth-tool without auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-tool/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
isErr: true,
name: "Invoke my-auth-tool without auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-tool/invoke",
enabled: true,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantStatusCode: http.StatusUnauthorized,
},
{
name: "Invoke my-auth-required-tool with auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-required-tool/invoke",
enabled: true,
requestHeader: map[string]string{"my-google-auth_token": idToken},
requestBody: bytes.NewBuffer([]byte(`{}`)),
isErr: false,
want: select1Want,
wantBody: select1Want,
wantStatusCode: http.StatusOK,
},
{
name: "Invoke my-auth-required-tool with invalid auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-required-tool/invoke",
requestHeader: map[string]string{"my-google-auth_token": "INVALID_TOKEN"},
requestBody: bytes.NewBuffer([]byte(`{}`)),
isErr: true,
name: "Invoke my-auth-required-tool with invalid auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-required-tool/invoke",
enabled: true,
requestHeader: map[string]string{"my-google-auth_token": "INVALID_TOKEN"},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantStatusCode: http.StatusUnauthorized,
},
{
name: "Invoke my-auth-required-tool without auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-tool/invoke",
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
isErr: true,
name: "Invoke my-auth-required-tool without auth token",
api: "http://127.0.0.1:5000/api/tool/my-auth-tool/invoke",
enabled: true,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantStatusCode: http.StatusUnauthorized,
},
{
name: "Invoke my-client-auth-tool with auth token",
api: "http://127.0.0.1:5000/api/tool/my-client-auth-tool/invoke",
enabled: configs.supportClientAuth,
requestHeader: map[string]string{"Authorization": accessToken},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantBody: select1Want,
wantStatusCode: http.StatusOK,
},
{
name: "Invoke my-client-auth-tool without auth token",
api: "http://127.0.0.1:5000/api/tool/my-client-auth-tool/invoke",
enabled: configs.supportClientAuth,
requestHeader: map[string]string{},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantStatusCode: http.StatusUnauthorized,
},
{
name: "Invoke my-client-auth-tool with invalid auth token",
api: "http://127.0.0.1:5000/api/tool/my-client-auth-tool/invoke",
enabled: configs.supportClientAuth,
requestHeader: map[string]string{"Authorization": "Bearer invalid-token"},
requestBody: bytes.NewBuffer([]byte(`{}`)),
wantStatusCode: http.StatusUnauthorized,
},
}
for _, tc := range invokeTcs {
t.Run(tc.name, func(t *testing.T) {
if !tc.enabled {
return
}
// Send Tool invocation request
req, err := http.NewRequest(http.MethodPost, tc.api, tc.requestBody)
if err != nil {
t.Fatalf("unable to create request: %s", err)
}
req.Header.Add("Content-type", "application/json")
// Add headers
for k, v := range tc.requestHeader {
req.Header.Add(k, v)
}
@@ -391,19 +446,22 @@ func RunToolInvokeTest(t *testing.T, select1Want string, options ...InvokeTestOp
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
if tc.isErr {
return
}
bodyBytes, _ := io.ReadAll(resp.Body)
t.Fatalf("response status code is not 200, got %d: %s", resp.StatusCode, string(bodyBytes))
// Check status code
if resp.StatusCode != tc.wantStatusCode {
body, _ := io.ReadAll(resp.Body)
t.Errorf("StatusCode mismatch: got %d, want %d. Response body: %s", resp.StatusCode, tc.wantStatusCode, string(body))
}
// skip response body check
if tc.wantBody == "" {
return
}
// Check response body
var body map[string]interface{}
err = json.NewDecoder(resp.Body).Decode(&body)
if err != nil {
t.Fatalf("error parsing response body")
t.Fatalf("error parsing response body: %s", err)
}
got, ok := body["result"].(string)
@@ -411,8 +469,8 @@ func RunToolInvokeTest(t *testing.T, select1Want string, options ...InvokeTestOp
t.Fatalf("unable to find result in response body")
}
if got != tc.want {
t.Fatalf("unexpected value: got %q, want %q", got, tc.want)
if got != tc.wantBody {
t.Fatalf("unexpected value: got %q, want %q", got, tc.wantBody)
}
})
}
@@ -763,11 +821,12 @@ func RunInitialize(t *testing.T, protocolVersion string) string {
}
// RunMCPToolCallMethod runs the tool/call for mcp endpoint
func RunMCPToolCallMethod(t *testing.T, myFailToolWant string, options ...McpTestOption) {
func RunMCPToolCallMethod(t *testing.T, myFailToolWant, select1Want string, options ...McpTestOption) {
// Resolve options
// Default values for MCPTestConfig
configs := &MCPTestConfig{
myToolId3NameAliceWant: `{"jsonrpc":"2.0","id":"my-tool","result":{"content":[{"type":"text","text":"{\"id\":1,\"name\":\"Alice\"}"},{"type":"text","text":"{\"id\":3,\"name\":\"Sid\"}"}]}}`,
supportClientAuth: false,
}
// Apply provided options
@@ -776,22 +835,32 @@ func RunMCPToolCallMethod(t *testing.T, myFailToolWant string, options ...McpTes
}
sessionId := RunInitialize(t, "2024-11-05")
header := map[string]string{}
if sessionId != "" {
header["Mcp-Session-Id"] = sessionId
// Get access token
accessToken, err := sources.GetIAMAccessToken(t.Context())
if err != nil {
t.Fatalf("error getting access token from ADC: %s", err)
}
idToken, err := GetGoogleIdToken(ClientId)
if err != nil {
t.Fatalf("error getting Google ID token: %s", err)
}
// Test tool invoke endpoint
invokeTcs := []struct {
name string
api string
requestBody jsonrpc.JSONRPCRequest
requestHeader map[string]string
want string
name string
api string
enabled bool // switch to turn on/off the test case
requestBody jsonrpc.JSONRPCRequest
requestHeader map[string]string
wantStatusCode int
wantBody string
}{
{
name: "MCP Invoke my-tool",
api: "http://127.0.0.1:5000/mcp",
enabled: true,
requestHeader: map[string]string{},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
@@ -807,11 +876,13 @@ func RunMCPToolCallMethod(t *testing.T, myFailToolWant string, options ...McpTes
},
},
},
want: configs.myToolId3NameAliceWant,
wantStatusCode: http.StatusOK,
wantBody: configs.myToolId3NameAliceWant,
},
{
name: "MCP Invoke invalid tool",
api: "http://127.0.0.1:5000/mcp",
enabled: true,
requestHeader: map[string]string{},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
@@ -824,11 +895,13 @@ func RunMCPToolCallMethod(t *testing.T, myFailToolWant string, options ...McpTes
"arguments": map[string]any{},
},
},
want: `{"jsonrpc":"2.0","id":"invalid-tool","error":{"code":-32602,"message":"invalid tool name: tool with name \"foo\" does not exist"}}`,
wantStatusCode: http.StatusOK,
wantBody: `{"jsonrpc":"2.0","id":"invalid-tool","error":{"code":-32602,"message":"invalid tool name: tool with name \"foo\" does not exist"}}`,
},
{
name: "MCP Invoke my-tool without parameters",
api: "http://127.0.0.1:5000/mcp",
enabled: true,
requestHeader: map[string]string{},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
@@ -841,11 +914,13 @@ func RunMCPToolCallMethod(t *testing.T, myFailToolWant string, options ...McpTes
"arguments": map[string]any{},
},
},
want: `{"jsonrpc":"2.0","id":"invoke-without-parameter","error":{"code":-32602,"message":"provided parameters were invalid: parameter \"id\" is required"}}`,
wantStatusCode: http.StatusOK,
wantBody: `{"jsonrpc":"2.0","id":"invoke-without-parameter","error":{"code":-32602,"message":"provided parameters were invalid: parameter \"id\" is required"}}`,
},
{
name: "MCP Invoke my-tool with insufficient parameters",
api: "http://127.0.0.1:5000/mcp",
enabled: true,
requestHeader: map[string]string{},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
@@ -858,12 +933,14 @@ func RunMCPToolCallMethod(t *testing.T, myFailToolWant string, options ...McpTes
"arguments": map[string]any{"id": 1},
},
},
want: `{"jsonrpc":"2.0","id":"invoke-insufficient-parameter","error":{"code":-32602,"message":"provided parameters were invalid: parameter \"name\" is required"}}`,
wantStatusCode: http.StatusOK,
wantBody: `{"jsonrpc":"2.0","id":"invoke-insufficient-parameter","error":{"code":-32602,"message":"provided parameters were invalid: parameter \"name\" is required"}}`,
},
{
name: "MCP Invoke my-auth-required-tool",
api: "http://127.0.0.1:5000/mcp",
requestHeader: map[string]string{},
enabled: true,
requestHeader: map[string]string{"my-google-auth_token": idToken},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
Id: "invoke my-auth-required-tool",
@@ -875,11 +952,106 @@ func RunMCPToolCallMethod(t *testing.T, myFailToolWant string, options ...McpTes
"arguments": map[string]any{},
},
},
want: "{\"jsonrpc\":\"2.0\",\"id\":\"invoke my-auth-required-tool\",\"error\":{\"code\":-32600,\"message\":\"unauthorized Tool call: `authRequired` is set for the target Tool\"}}",
wantStatusCode: http.StatusOK,
wantBody: select1Want,
},
{
name: "MCP Invoke my-auth-required-tool with invalid auth token",
api: "http://127.0.0.1:5000/mcp",
requestHeader: map[string]string{"my-google-auth_token": "INVALID_TOKEN"},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
Id: "invoke my-auth-required-tool with invalid token",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "my-auth-required-tool",
"arguments": map[string]any{},
},
},
wantStatusCode: http.StatusUnauthorized,
wantBody: "{\"jsonrpc\":\"2.0\",\"id\":\"invoke my-auth-required-tool with invalid token\",\"error\":{\"code\":-32600,\"message\":\"unauthorized Tool call: Please make sure your specify correct auth headers: unauthorized\"}}",
},
{
name: "MCP Invoke my-auth-required-tool without auth token",
api: "http://127.0.0.1:5000/mcp",
requestHeader: map[string]string{},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
Id: "invoke my-auth-required-tool without token",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "my-auth-required-tool",
"arguments": map[string]any{},
},
},
wantStatusCode: http.StatusUnauthorized,
wantBody: "{\"jsonrpc\":\"2.0\",\"id\":\"invoke my-auth-required-tool without token\",\"error\":{\"code\":-32600,\"message\":\"unauthorized Tool call: Please make sure your specify correct auth headers: unauthorized\"}}",
},
{
name: "MCP Invoke my-client-auth-tool",
enabled: configs.supportClientAuth,
api: "http://127.0.0.1:5000/mcp",
requestHeader: map[string]string{"Authorization": accessToken},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
Id: "invoke my-client-auth-tool",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "my-client-auth-tool",
"arguments": map[string]any{},
},
},
wantStatusCode: http.StatusOK,
wantBody: "{\"jsonrpc\":\"2.0\",\"id\":\"invoke my-client-auth-tool\",\"result\":{\"content\":[{\"type\":\"text\",\"text\":\"{\\\"f0_\\\":1}\"}]}}",
},
{
name: "MCP Invoke my-client-auth-tool without access token",
enabled: configs.supportClientAuth,
api: "http://127.0.0.1:5000/mcp",
requestHeader: map[string]string{},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
Id: "invoke my-client-auth-tool",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "my-client-auth-tool",
"arguments": map[string]any{},
},
},
wantStatusCode: http.StatusUnauthorized,
wantBody: "{\"jsonrpc\":\"2.0\",\"id\":\"invoke my-client-auth-tool\",\"error\":{\"code\":-32600,\"message\":\"missing access token in the 'Authorization' header\"}",
},
{
name: "MCP Invoke my-client-auth-tool with invalid access token",
enabled: configs.supportClientAuth,
api: "http://127.0.0.1:5000/mcp",
requestHeader: map[string]string{"Authorization": "Bearer invalid-token"},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
Id: "invoke my-client-auth-tool",
Request: jsonrpc.Request{
Method: "tools/call",
},
Params: map[string]any{
"name": "my-client-auth-tool",
"arguments": map[string]any{},
},
},
wantStatusCode: http.StatusUnauthorized,
},
{
name: "MCP Invoke my-fail-tool",
api: "http://127.0.0.1:5000/mcp",
enabled: true,
requestHeader: map[string]string{},
requestBody: jsonrpc.JSONRPCRequest{
Jsonrpc: "2.0",
@@ -892,36 +1064,56 @@ func RunMCPToolCallMethod(t *testing.T, myFailToolWant string, options ...McpTes
"arguments": map[string]any{"id": 1},
},
},
want: myFailToolWant,
wantStatusCode: http.StatusOK,
wantBody: myFailToolWant,
},
}
for _, tc := range invokeTcs {
t.Run(tc.name, func(t *testing.T) {
if !tc.enabled {
return
}
reqMarshal, err := json.Marshal(tc.requestBody)
if err != nil {
t.Fatalf("unexpected error during marshaling of request body")
}
_, respBody := runRequest(t, http.MethodPost, tc.api, bytes.NewBuffer(reqMarshal), header)
got := string(bytes.TrimSpace(respBody))
// add headers
headers := map[string]string{}
if sessionId != "" {
headers["Mcp-Session-Id"] = sessionId
}
for key, value := range tc.requestHeader {
headers[key] = value
}
if !strings.Contains(got, tc.want) {
t.Fatalf("Expected substring not found:\ngot: %q\nwant: %q (to be contained within got)", got, tc.want)
httpResponse, respBody := runRequest(t, http.MethodPost, tc.api, bytes.NewBuffer(reqMarshal), headers)
// Check status code
if httpResponse.StatusCode != tc.wantStatusCode {
t.Errorf("StatusCode mismatch: got %d, want %d", httpResponse.StatusCode, tc.wantStatusCode)
}
// Check response body
got := string(bytes.TrimSpace(respBody))
if !strings.Contains(got, tc.wantBody) {
t.Fatalf("Expected substring not found:\ngot: %q\nwant: %q (to be contained within got)", got, tc.wantBody)
}
})
}
}
func runRequest(t *testing.T, method, url string, body io.Reader, header map[string]string) (*http.Response, []byte) {
func runRequest(t *testing.T, method, url string, body io.Reader, headers map[string]string) (*http.Response, []byte) {
// Send request
req, err := http.NewRequest(method, url, body)
if err != nil {
t.Fatalf("unable to create request: %s", err)
}
req.Header.Add("Content-type", "application/json")
for k, v := range header {
req.Header.Add(k, v)
req.Header.Set("Content-type", "application/json")
for k, v := range headers {
req.Header.Set(k, v)
}
resp, err := http.DefaultClient.Do(req)

View File

@@ -148,11 +148,12 @@ func getTrinoTmplToolStatement() (string, string) {
}
// getTrinoWants return the expected wants for trino
func getTrinoWants() (string, string, string) {
func getTrinoWants() (string, string, string, string) {
select1Want := `[{"_col0":1}]`
failInvocationWant := `{"jsonrpc":"2.0","id":"invoke-fail-tool","result":{"content":[{"type":"text","text":"unable to execute query: trino: query failed (200 OK): \"USER_ERROR: line 1:1: mismatched input 'SELEC'. Expecting: 'ALTER', 'ANALYZE', 'CALL', 'COMMENT', 'COMMIT', 'CREATE', 'DEALLOCATE', 'DELETE', 'DENY', 'DESC', 'DESCRIBE', 'DROP', 'EXECUTE', 'EXPLAIN', 'GRANT', 'INSERT', 'MERGE', 'PREPARE', 'REFRESH', 'RESET', 'REVOKE', 'ROLLBACK', 'SET', 'SHOW', 'START', 'TRUNCATE', 'UPDATE', 'USE', 'WITH', \u003cquery\u003e\""}],"isError":true}}`
createTableStatement := `"CREATE TABLE t (id BIGINT NOT NULL, name VARCHAR(255))"`
return select1Want, failInvocationWant, createTableStatement
mcpSelect1Want := `{"jsonrpc":"2.0","id":"invoke my-auth-required-tool","result":{"content":[{"type":"text","text":"{\"_col0\":1}"}]}}`
return select1Want, failInvocationWant, createTableStatement, mcpSelect1Want
}
// setupTrinoTable creates and inserts data into a table of tool
@@ -255,12 +256,12 @@ func TestTrinoToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, createTableStatement := getTrinoWants()
select1Want, mcpMyFailToolWant, createTableStatement, mcpSelect1Want := getTrinoWants()
// Run tests
tests.RunToolGetTest(t)
tests.RunToolInvokeTest(t, select1Want, tests.DisableArrayTest())
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want)
tests.RunExecuteSqlToolInvokeTest(t, createTableStatement, select1Want)
tests.RunToolInvokeWithTemplateParameters(t, tableNameTemplateParam, tests.WithInsert1Want(`[{"rows":1}]`))
}

View File

@@ -101,7 +101,7 @@ func TestValkeyToolEndpoints(t *testing.T) {
}
// Get configs for tests
select1Want, mcpMyFailToolWant, invokeParamWant, invokeIdNullWant, nullWant, mcpInvokeParamWant := tests.GetRedisValkeyWants()
select1Want, mcpMyFailToolWant, invokeParamWant, invokeIdNullWant, nullWant, mcpSelect1Want, mcpInvokeParamWant := tests.GetRedisValkeyWants()
// Run tests
tests.RunToolGetTest(t)
@@ -110,7 +110,7 @@ func TestValkeyToolEndpoints(t *testing.T) {
tests.WithMyToolById4Want(invokeIdNullWant),
tests.WithNullWant(nullWant),
)
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant,
tests.RunMCPToolCallMethod(t, mcpMyFailToolWant, mcpSelect1Want,
tests.WithMcpMyToolId3NameAliceWant(mcpInvokeParamWant),
)
}