Move source-related queries from `Invoke()` function into Source.
The following sources were updated in this PR:
* mssql
* cloudsql-mssql
* mysql
* cloudsql-mysql
This is an effort to generalizing tools to work with any Source that
implements a specific interface. This will provide a better segregation
of the roles for Tools vs Source.
Tool's role will be limited to the following:
* Resolve any pre-implementation steps or parameters (e.g. template
parameters)
* Retrieving Source
* Calling the source's implementation
## Description
This PR adds the support for IAM authentication in the Cloud SQL MySQL
source
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [x] Make sure to add `!` if this involve a breaking change
🛠️ Fixes #<issue_number_goes_here>
To keep a persistent backend storage for configuration, we will have to
keep a single source of truth. This involves supporting bi-directional
conversion between Config and Source.
This PR make the following changes:
* Embed Config in Source
* Add `ToConfig()` to extract Config from Source.
## Description
> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [ ] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change
🛠️ Fixes #<issue_number_goes_here>
## Description
---
Audit your applications by using the following sql to track program name
```
SELECT
session_connect_attrs.ATTR_VALUE AS program_name,
processlist.*
FROM
information_schema.processlist
LEFT JOIN
performance_schema.session_connect_attrs
ON
(processlist.ID = session_connect_attrs.PROCESSLIST_ID AND session_connect_attrs.ATTR_NAME = "program_name");
```
## PR Checklist
---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [ ] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change
🛠️ Fixes #<issue_number_goes_here>
This PR fixes#1232 by allowing the `CLOUD_SQL_MYSQL_IP_TYPE`
environment variable to control the connection type for the
`cloud-sql-mysql` prebuilt tool, defaulting to "PUBLIC".
**Changes:**
* **`internal/prebuiltconfigs/tools/cloud-sql-mysql.yaml`:** Updated
`ipType` to use `${CLOUD_SQL_MYSQL_IP_TYPE:PUBLIC}`.
* **`internal/sources/cloudsqlmysql/cloud_sql_mysql.go`:** Removed
`validate:"required"` from the `IPType` field in the `Config` struct to
allow the default to be used when the env var is unset.
* **Documentation:** Updated relevant documentation to reflect the new
`CLOUD_SQL_MYSQL_IP_TYPE` environment variable usage.
**Testing:**
Built a local Docker image and tested on a GCE VM:
* Setting `CLOUD_SQL_MYSQL_IP_TYPE="private"` connects via Private IP.
* Setting `CLOUD_SQL_MYSQL_IP_TYPE="PUBLIC"` connects via Public IP.
* Leaving `CLOUD_SQL_MYSQL_IP_TYPE` unset defaults to Public IP.
All tests initialized the toolbox successfully.
Fixes#1232
## Description
---
> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution
## PR Checklist
---
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [x] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [x] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change
🛠️Fixes#1232
---------
Co-authored-by: Averi Kitsch <akitsch@google.com>
This commit refactors the source configuration and loading mechanism to
use a dynamic registration pattern. Each source package now registers
itself with a central registry via its init() function.
The server configuration code uses this registry to decode and
initialize sources, decoupling it from specific source implementations
and simplifying the addition of new sources.
Key changes:
- Introduced `sources.Register()` and `newConfig()` constructor in each
source package.
- Moved source package imports to `cmd/root.go` as blank imports to
trigger `init()` functions for self-registration.
- Removed direct imports of specific source packages from
`internal/server/config.go`.
- Renamed `SourceKind` constants to `Kind` within each source package.
- Updated tests to use the new `Kind` constants and reflect registration
changes.
---------
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
User agent will be set as "genai-toolbox/" + versionString.
Instead of a panic, Toolbox will throw an error if the source fail to
retrieve `user agent`.
This will also be consistent with setting and getting logger from
context.
This only checks within `SourceConfig`, `ToolConfig`, and
`AuthSourceConfig`.
Error when an unknown field is provided:
`2025-01-27T22:43:46.988401-08:00 ERROR "unable to parse tool file at
\"tools.yaml\": unable to parse as \"cloud-sql-postgres\": [2:1] unknown
field \"extra\"\n 1 | database: test_database\n> 2 | extra: here\n ^\n 3
| instance: toolbox-cloudsql\n 4 | kind: cloud-sql-postgres\n 5 |
password: postgres\n 6 | "`
Error when a required field is not provided:
`2025-01-27T17:49:47.584846-08:00 ERROR "unable to parse tool file at
\"tools.yaml\": validation failed: Key: 'Config.Region' Error:Field
validation for 'Region' failed on the 'required' tag"`
---------
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Add user agent to cloud databases that provides us anonymized data
request count, number of users, number of projects, and other
environment settings.
User agent is using the format: `genai-toolbox/$version+metadata`
Add integration test for CloudSQL for MySQL.
Added other integration tests' tag into `.golangci.yaml`, and fixing
lint errors.
Moved getCloudSQLDialOpts to `common_test.go` since it is used across
all three cloud sql integration tests.
Add CloudSQL for MySQL source and tool.
CloudSQLMySQL source is initialize with the following config:
```
sources:
my-cloudsqlmysql-source:
kind: cloud-sql-mysql
project: my-project-name
region: my-region
instance: my-instance-name
user: my_user
password: my_pass
database: my_db
# ipType: public # The default dialect is public.
```
MySQL tool is initialize with the following config.
```
tools:
test_tool:
kind: mysql
source: my-cloudsqlmysql-source
description: >
Testing tool.
statement: "SELECT 1;"
```
