Add debug logs to Toolbox.
For example when a http fail, it will just show Error at the http level,
but not log with actual error message. err message are returned to the
api as following `{"status":"Internal Server Error","error":"error while
invoking tool: unable to execute client: spanner: code =
\"InvalidArgument\", desc = \"invalid session pool\""}`.
After adding this, if user/dev run toolbox with `--log-level=debug`, it
will output the following (debug log in addition to the error for http
request):
```
2025-01-08T14:16:25.040824-08:00 DEBUG "error while invoking tool: unable to execute client: spanner: code = \"InvalidArgument\", desc = \"invalid session pool\""
2025-01-08T14:16:25.040968-08:00 ERROR Response: 500 Server Error service: "httplog" httpRequest: {url: "http://127.0.0.1:5000/api/tool/test_tool_two/invoke" method: "POST" path: "/api/tool/test_tool_two/invoke" remoteIP: "127.0.0.1:51708" proto: "HTTP/1.1" requestID: "yuanteoh-macbookpro.roam.internal/N7LNMcLIUH-000001" scheme: "http" header: {user-agent: "curl/8.7.1" accept: "*/*" content-type: "application/json" content-length: "2"}} httpResponse: {status: 500 bytes: 167 elapsed: 0.301917}
```
Adds logic to make the server shutdown gracefully, including better
respecting cancelled contexts and providing up to 10 seconds to finish
current connections.
1. Add []ParamAuthSource to every Parameter type implementation to
support authenticated configs. Create new constructors for types with
auth.
2. Tool invocation API changes to parse auth header and authentecated
parameters.
3. Add authSources to Tool manifest.
End to end integration test for cloudsql postgres.
Include checks for one tool's get (manifest) and post (invoke) endpoint.
Integration tests are excluded from regular unit tests.
Logging support 4 different types of logging (debug, info, warn, error).
The default logging level is Info.
User will be able to set flag for log level (allowed values: "debug",
"info", "warn", "error"), example:
`go run . --log-level debug`
User will be able to set flag for logging format (allowed values:
"standard", "JSON"), example:
`go run . --logging-format json`
**sample http request log - std:**
server
```
2024-11-12T15:08:11.451377-08:00 INFO "Initalized 0 sources.\n"
```
httplog
```
2024-11-26T15:15:53.947287-08:00 INFO Response: 200 OK service: "httplog" httpRequest: {url: "http://127.0.0.1:5000/" method: "GET" path: "/" remoteIP: "127.0.0.1:64216" proto: "HTTP/1.1" requestID: "macbookpro.roam.interna/..." scheme: "http" header: {user-agent: "curl/8.7.1" accept: "*/*"}} httpResponse: {status: 200 bytes: 22 elapsed: 0.012417}
```
**sample http request log - structured:**
server
```
{
"timestamp":"2024-11-04T16:45:11.987299-08:00",
"severity":"ERROR",
"logging.googleapis.com/sourceLocation":{
"function":"github.com/googleapis/genai-toolbox/internal/log.(*StructuredLogger).Errorf",
"file":"/Users/yuanteoh/github/genai-toolbox/internal/log/log.go","line":157
},
"message":"unable to parse tool file at \"tools.yaml\": \"cloud-sql-postgres1\" is not a valid kind of data source"
}
```
httplog
```
{
"timestamp":"2024-11-26T15:12:49.290974-08:00",
"severity":"INFO",
"logging.googleapis.com/sourceLocation":{
"function":"github.com/go-chi/httplog/v2.(*RequestLoggerEntry).Write",
"file":"/Users/yuanteoh/go/pkg/mod/github.com/go-chi/httplog/v2@v2.1.1/httplog.go","line":173
},
"message":"Response: 200 OK",
"service":"httplog",
"httpRequest":{
"url":"http://127.0.0.1:5000/",
"method":"GET",
"path":"/",
"remoteIP":"127.0.0.1:64140",
"proto":"HTTP/1.1",
"requestID":"yuanteoh-macbookpro.roam.internal/NBrtYBu3q9-000001",
"scheme":"http",
"header":{"user-agent":"curl/8.7.1","accept":"*/*"}
},
"httpResponse":{"status":200,"bytes":22,"elapsed":0.0115}
}
```
Moves all of the "source" and "tool" implementations into their own
packages. This layout makes it a bit more clear where the
implementations are, and seems likely to scale more cleanly as more
sources and tools are added.
1. Calculate tool manifests when server starts.
2. Add toolset manifest endpoints.
---------
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Add `Toolset` implementation to the `tools` package:
- struct and configs.
- Custom `UnmarshalYAML` function.
- Initialization function that validates if tools specified for the
toolset exist.
