Commit Graph

50 Commits

Author SHA1 Message Date
Yuan Teoh
293c1d6889 feat!: update configuration file v2 (#2369)
This PR introduces a significant update to the Toolbox configuration
file format, which is one of the primary **breaking changes** required
for the implementation of the Advanced Control Plane.

# Summary of Changes
The configuration schema has been updated to enforce resource isolation
and facilitate atomic, incremental updates.
* Resource Isolation: Resource definitions are now separated into
individual blocks, using a distinct structure for each resource type
(Source, Tool, Toolset, etc.). This improves readability, management,
and auditing of configuration files.
* Field Name Modification: Internal field names have been modified to
align with declarative methodologies. Specifically, the configuration
now separates kind (general resource type, e.g., Source) from type
(specific implementation, e.g., Postgres).

# User Impact
Existing tools.yaml configuration files are now in an outdated format.
Users must eventually update their files to the new YAML format.

# Mitigation & Compatibility
Backward compatibility is maintained during this transition to ensure no
immediate user action is required for existing files.
* Immediate Backward Compatibility: The source code includes a
pre-processing layer that automatically detects outdated configuration
files (v1 format) and converts them to the new v2 format under the hood.
* [COMING SOON] Migration Support: The new toolbox migrate subcommand
will be introduced to allow users to automatically convert their old
configuration files to the latest format.

# Example
Example for config file v2:
```
kind: sources
name: my-pg-instance
type: cloud-sql-postgres
project: my-project
region: my-region
instance: my-instance
database: my_db
user: my_user
password: my_pass
---
kind: authServices
name: my-google-auth
type: google
clientId: testing-id
---
kind: tools
name: example_tool
type: postgres-sql
source: my-pg-instance
description: some description
statement: SELECT * FROM SQL_STATEMENT;
parameters:
- name: country
  type: string
  description: some description
---
kind: tools
name: example_tool_2
type: postgres-sql
source: my-pg-instance
description: returning the number one
statement: SELECT 1;
---
kind: toolsets
name: example_toolset
tools:
- example_tool
```

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
2026-01-27 16:58:43 -08:00
Siddharth Ravi
cf0fc515b5 feat: add tool to list store procedure (#2156)
Adds the following tools for Postgres:
(1) list_stored_procedure: Retrieves stored procedure metadata returning
schema name, procedure name, procedure owner, language, definition, and
description, filtered by optional role name (procedure owner), schema
name, and limit (default 20).

<img width="3808" height="1181" alt="image"
src="https://github.com/user-attachments/assets/43513a04-95ce-478f-a59f-3e5dafdb6b23"
/>

<img width="2654" height="1288" alt="image"
src="https://github.com/user-attachments/assets/84aca162-3779-4daa-ae2f-61620560589f"
/>


> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1738
2026-01-06 08:36:45 +00:00
gRedHeadphone
0641da0353 feat(tools/mysql-get-query-plan): tool impl + docs + tests (#2123)
## Description

Tool mysql-get-query-plan implementation, along with tests and docs.
Tool used to get information about how MySQL executes a SQL statement
(EXPLAIN).

## PR Checklist

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1692

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-12-19 01:02:16 +00:00
Siddharth Ravi
78b02f08c3 feat: add list-table-stats-tool to list table statistics. (#2055)
Adds the following tools for Postgres:
(1) list_table_stats: Lists table statistics in the database. .

<img width="3446" height="1304" alt="image"
src="https://github.com/user-attachments/assets/68951edc-8d99-460e-a1ac-2d3da9388baf"
/>

<img width="2870" height="1338" alt="image"
src="https://github.com/user-attachments/assets/100a3b7d-202d-4dfd-b046-5dab4390ba41"
/>


> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1738
2025-12-10 11:11:33 +00:00
Srividya Reddy
bea9705450 feat(tools/postgres): Add new postgres-list-roles tool (#2038)
## Description
Adds a postgresql custom list_roles tool, that lists all the
user-created roles in the instance. It provides details about each
role's attributes and memberships.


> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution
![Uploading Screenshot 2025-11-26 at 1.16.42 AM.png…]()

<img width="1065" height="145" alt="Screenshot 2025-11-26 at 12 59
56 AM"
src="https://github.com/user-attachments/assets/d90131b1-d369-4108-b4db-ee5dc9aafe38"
/>


## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #<1738>

Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-12-09 21:48:20 +00:00
Srividya Reddy
32367a472f feat(tools/postgres): add list_pg_settings, list_database_stats tools for postgres (#2030)
## Description
Adds the following tools for Postgres:
(1) list_pg_settings: List configuration parameters for the PostgreSQL
server.
(2) list_database_stats: Lists the key performance and activity
statistics for each database in the postgreSQL
  server.

> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

list_pg_settings:
<img width="1526" height="803" alt="Screenshot 2025-11-25 at 10 19
48 AM"
src="https://github.com/user-attachments/assets/73634b9b-4936-4bf0-a94b-6b31fe3642a1"
/>
<img width="1064" height="715" alt="Screenshot 2025-11-25 at 10 27
19 AM"
src="https://github.com/user-attachments/assets/36c13585-27e4-4294-b451-1c1a963c0d6c"
/>

list_database_stats:
<img width="1511" height="779" alt="Screenshot 2025-11-25 at 10 21
12 AM"
src="https://github.com/user-attachments/assets/d283e018-ea81-427d-b1b4-7aaf79b9696b"
/>
<img width="1017" height="506" alt="Screenshot 2025-11-25 at 10 27
47 AM"
src="https://github.com/user-attachments/assets/47b72bd7-7114-4f2a-8a9d-cecc80bf47e9"
/>



## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #<1738>

Co-authored-by: Averi Kitsch <akitsch@google.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-12-09 11:55:53 -08:00
Pranjul Kalsi
3b40fea25e feat(sources/mariadb): add MariaDB source and MySQL tools integration (#1908)
## Description
This PR 
1. Adds **MariaDB** as a Source - Implementation is similar to **MySQL**
source
2. Utilises pre implemented **MySQL** Tools
- `mysql-execute-sql`
- `mysql-list-active-queries`
- `mysql-list-table-fragmentation`
- `mysql-list-tables`
- `mysql-list-tables-missing-unique-indexes`
- `mysql-sql`
**Note:** After discussion with @duwenxin99 in issue #1768, I initially
assumed MariaDB required new tools due to different metadata structures
and system tables. That is true for older MariaDB versions, but current
MySQL tooling already works with MariaDB (verified), so a separate tool
set was not needed.

3. Adds a source doc for **MariaDB** in docs 
4. Adds MariaDB integration tests using the existing MySQL test flow.
Note: The test file is based on the MySQL integration test, but
`GetMariaDBWants()` and
`RunMariDBListTablesTest()` are implemented because MariaDB returns
different metadata
and list-tables output, so the assertions must be MariaDB-specific.
5. Updates CI 

Lastly
I considered adding a MariaDB-exclusive Galera cluster monitoring tool,
but skipped it because it requires a multi-node Galera setup for
integration testing and would significantly increase CI complexity with
unclear usage demand.

## PR Checklist
- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1712 #1768

---------

Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-12-09 18:45:05 +00:00
Srividya Reddy
5ad7c6127b feat(tools/postgres-list-tablespaces): add new postgres-list-tablespaces tool (#1934)
## Description
Adds a postgresql custom list_tablespaces tool, that returns the details
of tablespaces present in database.

<img width="1719" height="698" alt="Screenshot 2025-11-12 at 9 11 13 AM"
src="https://github.com/user-attachments/assets/03964a1b-27e0-4da8-85a2-57db905163ed"
/>
<img width="1077" height="141" alt="Screenshot 2025-11-12 at 9 12 42 AM"
src="https://github.com/user-attachments/assets/f93f5692-eb62-4f30-8192-40c8873d4d00"
/>


> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

Lists all tablespaces in the database. Returns the tablespace name,
owner name, size in bytes, internal object ID, the access control list
regarding permissions, and any specific tablespace options.

## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1738
2025-12-04 20:38:54 +00:00
Srividya Reddy
f4b1f0a680 feat(tools/postgres-list-publication-tables): add new postgres-list-publication-tables tool (#1919)
## Description
Adds a postgresql custom list_publication_tables tool, that returns the
details of publication tables present in database.

Test Output:

<img width="845" height="239" alt="Screenshot 2025-11-11 at 12 50 59 AM"
src="https://github.com/user-attachments/assets/b7606e44-c5f6-4fc7-865e-7efadd112eff"
/>

<img width="1529" height="648" alt="Screenshot 2025-11-11 at 1 15 18 AM"
src="https://github.com/user-attachments/assets/6192b772-f0bc-4fb4-8032-ca487434d77c"
/>


> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1738

Co-authored-by: Averi Kitsch <akitsch@google.com>
Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-12-04 19:26:45 +00:00
Siddharth Ravi
9f76026925 feat: add list-query-stats and get-column-cardinality functions (#1976)
Adds the following tools for Postgre
(1) get_column_cardinality - to fetch the estimates of the distinct
column counts in the table for particular column or all columns
(2) list-query-stats - to obtain the query level statistics in a
database. This tool requires pg_stat_statements extension as a
prerequisite.

<img width="2428" height="1368" alt="image"
src="https://github.com/user-attachments/assets/5d9b22f0-6b09-4abe-8411-b1139387e259"
/>

<img width="3774" height="1010" alt="image"
src="https://github.com/user-attachments/assets/b1d9fdf1-8a3b-4afe-ab98-63226a7e3705"
/>


PR Checklist
[Y] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
[Y] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
[Y] Ensure the tests and linter pass
[Y] Code coverage does not decrease (if any source code was changed)
[Y] Appropriate docs were updated (if necessary)
[Y] Make sure to add ! if this involve a breaking change

🛠️ Fixes https://github.com/googleapis/genai-toolbox/issues/1691

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-11-26 23:43:03 +00:00
Siddharth Ravi
5abad5d56c feat(tools/postgres): add long_running_transactions, list_locks and replication_stats tools (#1751)
Adds the following tools for Postgre
(1) long_running_transactions - reports transactions that exceed a
configured duration threshold.
(2) list_locks - lists active locks in the database, including the
associated process, lock type, relation, mode, and the query holding or
waiting on the lock.
(3) replication_stats - reports replication-related metrics for WAL
streaming replicas, including lag sizes presented in human-readable
form.

<img width="3020" height="1420" alt="image"
src="https://github.com/user-attachments/assets/e7d70063-b90c-4464-90ec-1bd810c02cac"
/>

<img width="3036" height="1374" alt="image"
src="https://github.com/user-attachments/assets/f7cf584b-ae01-455c-9c9c-acca3166a549"
/>

<img width="2682" height="868" alt="image"
src="https://github.com/user-attachments/assets/dd10646c-4521-4d8f-a111-760d6eb01249"
/>


## PR Checklist
- [Y] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [Y] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [Y] Ensure the tests and linter pass
- [Y] Code coverage does not decrease (if any source code was changed)
- [Y] Appropriate docs were updated (if necessary)
- [Y] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1691 #1715

---------

Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-11-18 19:23:10 +00:00
Yuan Teoh
4aabb4aaca chore: move parameters to internal/util (#1907)
To facilitate the transition of moving invocation implementation to
Source, we will have to move parameter to `internal/util`. This approach
is crucial because certain parameters may not be fully resolvable
pre-implementation. Since both `internal/sources` and `internal/tools`
will need access to `parameters`, it will be more relevant to move
parameters implementation to utils.
2025-11-13 21:37:12 +00:00
Srividya Reddy
897c63dcea feat(tools): add list_indexes, list_sequences tools for postgres (#1765)
## Description

Adds the following tools for Postgres:
(1) list_indexes:  Lists available user indexes in a given database. 
(2) list_sequences: Lists all the sequences in the database ordered by
sequence name.

list_indexes:
<img width="1708" height="816" alt="Screenshot 2025-10-21 at 10 24
30 PM"
src="https://github.com/user-attachments/assets/d1888f53-6013-4beb-b0dc-b94f3d66135e"
/>

<img width="1914" height="1017" alt="Screenshot 2025-10-22 at 8 23
16 AM"
src="https://github.com/user-attachments/assets/7c185ef4-4550-4bb2-8051-0eca8f40dc7b"
/>

list_sequences:

<img width="2539" height="902" alt="Screenshot 2025-10-22 at 8 04 25 AM"
src="https://github.com/user-attachments/assets/f22cfaad-412b-4df2-99f3-ee813f538ff7"
/>

<img width="1082" height="610" alt="Screenshot 2025-10-22 at 8 06 07 AM"
src="https://github.com/user-attachments/assets/102ed3ca-33f6-409f-9ba1-363c9e15d5d3"
/>



> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1738
2025-11-13 21:17:03 +00:00
Srividya Reddy
a4c9287aec feat(tools/postgres): add list_triggers, database_overview tools for postgres (#1912)
## Description

Adds the following tools for Postgres:
(1) list_triggers: Lists triggers in the database. .
(2) database_overview: Fetches the current state of the PostgreSQL
server.

list_triggers:
<img width="1712" height="703" alt="Screenshot 2025-11-09 at 8 16 53 PM"
src="https://github.com/user-attachments/assets/1974e303-b559-4efc-b129-444ba97c7715"
/>

<img width="874" height="513" alt="Screenshot 2025-11-09 at 8 19 43 PM"
src="https://github.com/user-attachments/assets/59ddcd15-224b-4e9a-906d-ec2645835873"
/>

database_overview:

<img width="1521" height="683" alt="Screenshot 2025-11-09 at 8 53 03 PM"
src="https://github.com/user-attachments/assets/4ae86e74-aa78-410c-a9cc-f33ae3268fb6"
/>

<img width="850" height="241" alt="Screenshot 2025-11-09 at 8 49 53 PM"
src="https://github.com/user-attachments/assets/abae2c7a-5f3e-4433-86de-3606e3298ec5"
/>


> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1738
2025-11-12 14:35:23 -08:00
Yuan Teoh
1af43db6f2 test: add prebuilt tools test to all postgres source (#1874)
Move postgres prebuilt integration tests to `common.go` and `tool.go`.
Run those tests from alloydbpg and cloudsqlpg as well.

alloydbpg and cloudsqlpg integration test coverage calculate against the
whole `internal/tools/postgres/` folder. If not added, the coverage will
eventually drop below minimum requirement.
2025-11-06 15:54:38 -08:00
Srividya Reddy
1a19cac7cd feat(tools/postgres-list-schemas): add new postgres-list-schemas tool (#1741)
## Description
Add a read-only PostgreSQL custom list_schemas tool, that returns the
schemas present in the database excluding system and temporary schemas.
Returns the schema name, schema owner, grants, number of functions,
number of tables, and number of views within each schema.

<img width="1985" height="1043" alt="Screenshot 2025-10-20 at 7 45
45 PM"
src="https://github.com/user-attachments/assets/8c4f0bb8-587c-489a-8795-efa79e92b06f"
/>

<img width="3372" height="1694" alt="3NpZG7W6h3XGsM7"
src="https://github.com/user-attachments/assets/370b5440-cc48-4c4e-82ea-4fd508cbcf2b"
/>

> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #<issue_number_goes_here>

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-11-06 00:54:01 +00:00
Wenxin Du
fa8f7df24e ci: Make test helper function generic (#1672)
Make it reusable for other types of execute-sql tools
2025-10-09 22:11:40 +00:00
Dr. Strangelove
2d5a93e312 feat(tools/looker): Enable access to the Conversational Analytics API for Looker (#1596)
## Description

This enables the Conversational Analytics API for Looker. The prebuilt
config is separate since it is not a good
idea to use the Looker prebuilt config with CA. Agents get confused as
to whether they should query the data directly
or use the CA tool.
2025-10-02 17:45:41 +00:00
Sri Varshitha
918753d396 chore(tests): Add database cleanup functions (#1528)
## Description

---
This change adds helper functions to cleanup the test databases
(PostgreSQL, MySQL, and MSSQL) by dropping any orphan tables from
previous test runs. These functions can be called at the beginning of
each test run.

## PR Checklist

---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1304

---------

Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
2025-09-30 08:56:58 +05:30
Yuan Teoh
cb692b5883 tests: add mssql prebuilt tests to cloud sql mssql integration (#1501)
## Description

---
Add mssql's prebuilt tools tests to cloud-sql-mssql integration tests.

Cloud sql mssql's integration test coverage check against the mssql
package since those tools are compatible. Hence, when we add new tools
to mssql, we will have to add those integration tests against cloud sql
mssql as well.
2025-09-25 22:37:09 +00:00
Yuan Teoh
4768abd95c style: run linter (#1518) 2025-09-18 22:24:59 +00:00
shuzhou-gc
9eb821a6dc feat(tools/mysql-list-tables-missing-index): Add a new tool to list tables that do not have primary or unique keys in a MySQL instance (#1493)
## Description

---
A `mysql-list-tables-missing-index` tool searches tables that do not
have primary or unique indices in a MySQL database. It's compatible
with:

- cloud-sql-mysql
- mysql

`mysql-list-tables-missing-index` outputs table names, including
`table_schema` and `table_name` in JSON format. It takes 2 optional
input parameters:

- `table_schema` (optional): Only check tables in this specific
schema/database. Search all visible tables in all visible databases if
not specified.
- `limit` (optional):  max number of queries to return, default `50`.

## Example

```yaml
tools:
  list_tables_missing_index:
    kind: mysql-list-tables-missing-index
    source: my-mysql-instance
    description: Find tables that do not have primary or unique key constraint. A primary key or unique key is the only mechanism that guaranttes a row is unique. Without them, the database-level protection against data integrity issues will be missing.
```
The response is a json array with the following fields:
```json
{
  "table_schema": "the schema/database this table belongs to",
  "table_name": "name of the table",
}
```

## Reference

| **field** | **type** | **required** | **description** |

|-------------|:------------------------------------------:|:------------:|--------------------------------------------------------------------------------------------------|
| kind | string | true | Must be "mysql-list-active-queries". |
| source | string | true | Name of the source the SQL should execute on.
|
| description | string | true | Description of the tool that is passed
to the LLM. |
## PR Checklist

---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-09-18 11:17:15 -07:00
shuzhou-gc
fe651d822f feat(tools/mysql-list-table-fragmentation): Add a new tool to list table fragmentation in a MySQL instance (#1479)
## Description

A `mysql-list-table-fragmentation` tool checks table fragmentation of
MySQL tables by calculating the size of the data and index files in
bytes and comparing with free space allocated to each table. This tool
calculates `fragmentation_percentage` which represents the proportion of
free space relative to the total data and index size. It's compatible
with

- cloud-sql-mysql
- mysql

`mysql-list-table-fragmentation` outputs detailed information as JSON ,
ordered by the fragmentation percentage in descending order.
This tool takes 4 optional input parameters:

- `table_schema` (optional): The database where fragmentation check is
to be executed. Check all tables visible to the current user if not
specified.
- `table_name` (optional): Name of the table to be checked. Check all
tables visible to the current user if not specified.
- `data_free_threshold_bytes` (optional): Only show tables with at least
this much free space in bytes. Default 1.
- `limit` (optional): Max rows to return, default 10.

## Example

```yaml
tools:
  list_table_fragmentation:
    kind: mysql-list-table-fragmentation
    source: my-mysql-instance
    description: List table fragmentation in MySQL, by calculating the size of the data and index files and free space allocated to each table. The query calculates fragmentation percentage which represents the proportion of free space relative to the total data and index size. Storage can be reclaimed for tables with high fragmentation using OPTIMIZE TABLE.
```
The response is a json array with the following fields:
```json
{
  "table_schema": "The schema/database this table belongs to",
  "table_name": "Name of this table",
  "data_size": "Size of the table data in bytes",
  "index_size": "Size of the table's indexes in bytes",
  "data_free": "Free space (bytes) available in the table's data file",
  "fragmentation_percentage": "How much fragementation this table has",
}
```

## PR Checklist

---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change

---------

Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-09-18 10:24:45 -07:00
Yuan Teoh
874ee0ca9e tests: add mysql prebuilt tests to cloud sql mysql integration (#1500)
## Description

---
Add mysql's prebuilt tools tests to `cloud-sql-mysql` integration tests.


Cloud sql mysql's integration test coverage check against the mysql
folder since those tools are compatible. Hence, when we add new tools to
`mysql`, we will have to add those integration tests against cloud sql
mysql as well.
2025-09-17 22:49:13 +00:00
Niraj Nandre
0cd3f16f87 feat(prebuilt/cloud-sql-mysql): Add env var support for IP Type (#1232) (#1347)
This PR fixes #1232 by allowing the `CLOUD_SQL_MYSQL_IP_TYPE`
environment variable to control the connection type for the
`cloud-sql-mysql` prebuilt tool, defaulting to "PUBLIC".

**Changes:**

* **`internal/prebuiltconfigs/tools/cloud-sql-mysql.yaml`:** Updated
`ipType` to use `${CLOUD_SQL_MYSQL_IP_TYPE:PUBLIC}`.
* **`internal/sources/cloudsqlmysql/cloud_sql_mysql.go`:** Removed
`validate:"required"` from the `IPType` field in the `Config` struct to
allow the default to be used when the env var is unset.
* **Documentation:** Updated relevant documentation to reflect the new
`CLOUD_SQL_MYSQL_IP_TYPE` environment variable usage.

**Testing:**

Built a local Docker image and tested on a GCE VM:

*   Setting `CLOUD_SQL_MYSQL_IP_TYPE="private"` connects via Private IP.
*   Setting `CLOUD_SQL_MYSQL_IP_TYPE="PUBLIC"` connects via Public IP.
*   Leaving `CLOUD_SQL_MYSQL_IP_TYPE` unset defaults to Public IP.

All tests initialized the toolbox successfully.

Fixes #1232



## Description

---
> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

## PR Checklist

---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #1232

---------

Co-authored-by: Averi Kitsch <akitsch@google.com>
2025-09-10 21:32:49 +00:00
Yuan Teoh
ca353e0b66 feat(server/mcp): support toolbox auth in mcp (#1140)
Existing `/mcp` endpoint of Toolbox does not support auth (authorized
invocation and authenticated parameters). This PR add support for
Toolbox auth to the `/mcp` endpoint.

Added integration test for MCP with auth.

Note that Toolbox auth is **NOT** supported in stdio transport protocol,
invocations of tools with auth will result in error.
2025-08-27 09:38:56 -07:00
Yuan Teoh
8d6f3bbe2c chore: add options for all tests (#1169)
Add options for `RunToolInvokeTest()`, `RunMCPToolCallMethod()`,
`RunExecuteSqlToolInvokeTest()`.

When I was trying to update integration test for Firebird. They have a
different syntax from the usual `SELECT 1` that the other DBs are using.
If we want to update the `SELECT 1` statement in the test cases, we will
have to add a new argument and update all the other integration tests.

With `Options`, we can just add customization without having the need to
update all other integration tests. This allow the integration tests to
be more flexible. With this, we can also remove the
`GetNonSpannerWant()` since those can be set as default values for the
`Options`.
2025-08-22 10:14:36 -07:00
Abhinav Misra
7dd123b3d7 feat(trino): Add Trino source and tools (#948)
Add support for Trino distributed SQL query engine with complete
implementation including sources, tools, documentation, and tests.

## Summary

- Add Trino source implementation with multiple authentication methods
- Create trino-execute-sql tool for arbitrary SQL execution
- Create trino-sql tool for parameterized queries
- Add comprehensive documentation ~~and prebuilt configurations~~
- Include unit tests for all components

## Features

### Authentication Support
- Basic authentication (username/password)
- JWT token authentication
- Kerberos authentication
- SSL/TLS connections

### Tools
- **trino-execute-sql**: Execute arbitrary SQL queries
- **trino-sql**: Execute parameterized SQL with prepared statements
- Prebuilt tools for common operations (list catalogs, schemas, tables,
etc.)

### Cross-Catalog Queries
- Support for federated queries across multiple data sources
- Query Hive, PostgreSQL, MySQL, and other connectors in single queries
- Leverage Trino's distributed query engine capabilities

## Test Plan
- [x] Unit tests pass for source DSN generation
- [x] Unit tests pass for tool parameter validation
- [x] Project builds successfully with new dependencies
- [x] Documentation includes comprehensive examples

Ref #880

---------

Co-authored-by: Yuan Teoh <yuanteoh@google.com>
Co-authored-by: Yuan Teoh <45984206+Yuan325@users.noreply.github.com>
2025-08-22 00:03:04 +00:00
Yuan Teoh
7eff0f9ac7 test: integration tests for null optional parameters (#889)
Note: null optional parameters currently doesn't work with BigQuery.
2025-07-21 17:32:44 +00:00
Wenxin Du
2bdcc0841a fix: nil parameter error when arrays are used (#801)
- Spanner: convert array to typed before querying, as Spanner does not
accept non-typed array []any
- BigQuery: fix https://github.com/googleapis/genai-toolbox/issues/793
- Bigtable: add the required `ElemType` for array-type params 
- Redis/Valkey: change indexing to append to avoid extra spaces

Add integration tests for array parameters, skipped for the sources not
supporting arrays:
- SQLite
- Cloud SQL MSSQL
- Cloud SQL MySQL
- MSSQL
- MySQL
2025-07-09 17:40:49 -04:00
Yuan Teoh
04e2529ba9 test: add null column test case (#768)
Add integration tests to check for `null` columns. ref #757
2025-07-08 20:20:16 +00:00
Yuan
9b2dfcc553 chore: update int test variable name to be consistent (#766)
Update `_` variables to camelcase.
2025-07-04 05:21:39 +00:00
AlexTalreja
9197186b8b feat(tools/couchbase): add templateParameters field for couchbase (#723)
Add templateParameters to support non-filter parameters and DDL
statements.

Part of https://github.com/googleapis/genai-toolbox/issues/535
2025-06-18 21:27:41 +00:00
Yuan
f5f771b0f3 feat(tools/bigquery): add templateParameters field for bigquery (#699)
Add templateParameters to support non-filter parameters and DDL
statements.

Part of #535
2025-06-18 10:19:23 -07:00
Yuan
12b6636a9b refactor: update template param integration test to use options (#707) 2025-06-18 17:00:47 +00:00
Wenxin Du
054ec198b9 feat: Add Valkey Source and Tool (#532) 2025-06-11 23:17:32 -04:00
Wenxin Du
f0aef29b0c feat: Add Redis Source and Tool (#519)
1. Added Redis Source and Tool
2. Moved some integration test helpers from tools.go to common.go
3. Make auth integration test want an input variable
2025-06-11 22:47:27 -04:00
Yuan
075dfa47e1 feat(tools/spanner): add templateParameters field for spanner (#691)
Add `templateParameters` to support non-filter parameters and DDL
statements.

Added a new argument `ignoreDdl` at integration test. Admin client is
needed to execute ddl statement in spanner. Toolbox does not use admin
client.

Part of #535
2025-06-11 23:54:13 +00:00
Yuan
b81fc6aa6c feat: add templateParameters field for mssqlsql (#671)
Add `templateParameters to support non-filter parameters and DDL
statements.

Part of https://github.com/googleapis/genai-toolbox/issues/535
2025-06-06 13:18:37 -07:00
Yuan
5638ef520a chore: fix typos in tests (#672) 2025-06-06 14:43:57 +00:00
Yuan
0a08d2c15d feat: add templateParameters field for mysqlsql (#663)
Add `templateParameters to support non-filter parameters and DDL
statements.

Part of #535
2025-06-05 21:37:30 +00:00
Yuan
71250e1ced test: add templateParam postgres-sql tests to other pg source (#664)
Add the template parameters integration test to `postgres` source and
`cloudsql-postgres` source.
2025-06-05 21:33:35 +00:00
AlexTalreja
b76346993f feat(tools/postgressql): add templateParameters field (#615)
Add new tool field, templateParameters, to support non-filter parameters 
and DDL statements

Fix #535 for postgressql tool.

---------

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2025-06-04 13:28:37 -07:00
Wenxin Du
0ddc7240b7 chore: Move test helper functions to common.go (#647)
1. Move test helper functions to common.go
2. Move HTTP helper to its own test file
2025-06-03 16:22:08 -07:00
Yuan
e615e355d5 chore: dedup execute sql integration tests (#600)
Dedup the execute sql integration tests across postgres, mysql, and sql
server.
2025-05-22 22:21:07 +00:00
prernakakkar-google
6083a224aa Feat: Add Execute sql tool for SQL Server(MSSQL) (#585)
This tool can be used across mssql(SQL Server) sources.

`mssql-execute-sql` config is as below:

```
tools:
  mssql_execute_sql_tool:
     kind: "mssql-execute-sql"
     source: my-mssql-source
     description: Use this tool to execute sql.
```

The `mssql-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/mssql_execute_sql_tool/invoke
```

Reference for bug: b/416163913

---------

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Anubhav Dhawan <anubhavdhawan@google.com>
Co-authored-by: Huan Chen <142538604+Genesis929@users.noreply.github.com>
2025-05-22 00:10:55 +05:30
trehanshakuntG
8590061ae4 feat: add mysql-execute-sql tool (#577)
This tool can be used across mysql sources.

`mysql-execute-sql` config is as below:

```
tools:
  mysql_execute_sql_tool:
    kind: "mysql-execute-sql"
    source: my-mysql-source
    description: Use this tool to execute sql.
```

The `mysql-execute-sql` tool takes one parameter. Example request as
follow:
```
curl -X POST -H "Content-Type: application/json" -d '{"sql": "SELECT 1"}' http://127.0.0.1:5000/api/tool/mysql_execute_sql_tool/invoke
```
2025-05-20 02:17:39 +05:30
Yuan
c26b2f4d9e test: add integration tests for postgres-execute-sql tool (#494)
Add `postgres-execute-sql` to compatible source's integration tests.
2025-05-06 15:42:22 -06:00
Yuan
a6c17c96f3 test: cleanup integration test (#498)
Remove cleaning `\\` or `\` in `got` and `want` strings of test cases.
This is to prevent future the need to manually clean strings before
comparing. This also allows us to be aware of the amount of backspace to
prevent accidental escaping in src code.

Update integration test that is affected. This includes moving the
`want` of test cases for `my-param-tool` invocation to each sources
since spanner have a different format (return id as strings instead of
int).
2025-04-29 11:57:47 -07:00
Yuan
59f4452755 ci: optimize integration tests with shared compile step (#477)
Reduces test time >50%.

Added a step to compile all test files and run tests based on the name.
Also separated all tests to it's own package.
2025-04-23 15:18:59 -07:00