Different databases require different types for `Params` field when
adding parameters to their statement. e.g. alloydb, cloudsql, and
postgres uses `pgxpool` to query and build sql statement, whereas
spanner uses `Spanner` library.
Added a new `ParamValue` struct. `ParseParams` helper function parses
arbitraryJSON object into `[]ParamValue`, and the tool's invoke will
convert `[]ParamValue` into it's required type.
---------
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Allow user to set if their database uses private or public ip. The
reason we add this is because the dialer require different
initialization with private and public ip.
By default, toolbox will use public ip.
---------
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Logging support 4 different types of logging (debug, info, warn, error).
The default logging level is Info.
User will be able to set flag for log level (allowed values: "debug",
"info", "warn", "error"), example:
`go run . --log-level debug`
User will be able to set flag for logging format (allowed values:
"standard", "JSON"), example:
`go run . --logging-format json`
**sample http request log - std:**
server
```
2024-11-12T15:08:11.451377-08:00 INFO "Initalized 0 sources.\n"
```
httplog
```
2024-11-26T15:15:53.947287-08:00 INFO Response: 200 OK service: "httplog" httpRequest: {url: "http://127.0.0.1:5000/" method: "GET" path: "/" remoteIP: "127.0.0.1:64216" proto: "HTTP/1.1" requestID: "macbookpro.roam.interna/..." scheme: "http" header: {user-agent: "curl/8.7.1" accept: "*/*"}} httpResponse: {status: 200 bytes: 22 elapsed: 0.012417}
```
**sample http request log - structured:**
server
```
{
"timestamp":"2024-11-04T16:45:11.987299-08:00",
"severity":"ERROR",
"logging.googleapis.com/sourceLocation":{
"function":"github.com/googleapis/genai-toolbox/internal/log.(*StructuredLogger).Errorf",
"file":"/Users/yuanteoh/github/genai-toolbox/internal/log/log.go","line":157
},
"message":"unable to parse tool file at \"tools.yaml\": \"cloud-sql-postgres1\" is not a valid kind of data source"
}
```
httplog
```
{
"timestamp":"2024-11-26T15:12:49.290974-08:00",
"severity":"INFO",
"logging.googleapis.com/sourceLocation":{
"function":"github.com/go-chi/httplog/v2.(*RequestLoggerEntry).Write",
"file":"/Users/yuanteoh/go/pkg/mod/github.com/go-chi/httplog/v2@v2.1.1/httplog.go","line":173
},
"message":"Response: 200 OK",
"service":"httplog",
"httpRequest":{
"url":"http://127.0.0.1:5000/",
"method":"GET",
"path":"/",
"remoteIP":"127.0.0.1:64140",
"proto":"HTTP/1.1",
"requestID":"yuanteoh-macbookpro.roam.internal/NBrtYBu3q9-000001",
"scheme":"http",
"header":{"user-agent":"curl/8.7.1","accept":"*/*"}
},
"httpResponse":{"status":200,"bytes":22,"elapsed":0.0115}
}
```
This PR introduces the following breaking change: The
`alloydb-pg-generic`, `cloud-sql-pg-generic`, and
`postgres-generic-tool` have been replaced by the `postgres-sql` tool,
which works with all 3 Postgres sources.
If you were using of the the previous tools, you will need to update it
as follows:
```diff
example_tool:
- kind: cloud-sql-pg-generic
+ kind: postgres-sql
source: my-cloud-sql-pg-instance
description: some description
statement: |
SELECT * FROM SQL_STATEMENT;
parameters:
- name: country
type: string
description: some description
```
I'm proposing this change for the following reasons:
1. It provides greater flexibility between postgres-compatible sources
-- you can change between "postgres" and "alloydb-postgres" without
issue
2. The name "postgres-sql" is more clear that "postgres-generic" -- it
indicates it's a tool that runs SQL on the source
3. It's easier for us to maintain feature compatibility across a single
"postgres-sql" tool
Moves all of the "source" and "tool" implementations into their own
packages. This layout makes it a bit more clear where the
implementations are, and seems likely to scale more cleanly as more
sources and tools are added.
Adds support for "array" type parameters. Uses a subet of JSONSchema for
specification, in that arrays can be specified in the following way:
```yaml
parameters:
name: "my_array"
type: "array"
description: "some description"
items:
type: "integer"
```
1. Calculate tool manifests when server starts.
2. Add toolset manifest endpoints.
---------
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Makes the following changes:
- Fills in previously stubbed "Invoke()" function for the Cloud SQL
Generic Tool
- Updates API to /{tool_name}/invoke to for invocation of said tool
- Updates response to use JSON
- Correctly returns error messages for invalid http codes
Add `Toolset` implementation to the `tools` package:
- struct and configs.
- Custom `UnmarshalYAML` function.
- Initialization function that validates if tools specified for the
toolset exist.
This PR adds preliminary parsing of parameters. Currently it only
supports 4 types: string, int, float32, and bool. Almost certainly we
will need to introduce more complicated parsing configuration (to handle
objects and arrays), but my initial attempts got quickly complicated, so
I simplified in the short term.
This also makes 2 breaking changes to config.yaml:
- changes "parameters" to be a list over object -- this is because
parameter ordering is important, and needs to be preserved
- removed the "required" field from parameter objects -- we need to
determine how to handle optional parameters in SQL queries
