Update source code `Kind` to `Type`. It's only changes within our code.
Changes to yaml tag (that will affect users) will be done in later PRs.
This is a breaking change since it updates telemetry's span attribute
from `source_kind` to `source_type`.
Related #817
Future updates will include:
* Updating a preprocessing function to convert config file from v1 to v2
* Update unmarshal function for ToolsFile to convert config file (test
will fail since the yaml tag is not yet updated).
* Update yaml tag (test will pass).
Check for naming validation for Tool. This validation follows the MCP
SEP986 [naming
guidance](1b1eb60ec4/docs/specification/draft/server/tools.mdx (tool-names)).
Name will be checked before MCP initialization (where specs version is
confirmed). Hence, we will be implementing this across all versions and
endpoints.
This will be a breaking change for user that currently uses other
special character as name (other than `_`, `-`, `.`)
## Description
Previously added `allowed-origins` (for CORs) is not sufficient for
preventing DNS rebinding attacks. We'll have to check host headers.
To test, run Toolbox with the following:
```
go run . --allowed-hosts=127.0.0.1:5000
```
Test with the following:
```
// curl successfully
curl -H "Host: 127.0.0.1:5000" http://127.0.0.1:5000
// will show Invalid Host Header error
curl -H "Host: attacker:5000" http://127.0.0.1:5000
```
## PR Checklist
> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:
- [ ] Make sure you reviewed
[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change
🛠️ Fixes #<issue_number_goes_here>
To keep a persistent backend storage for configuration, we will have to
keep a single source of truth. This involves supporting bi-directional
conversion for Prompt and Promptsets.
This PR make the following changes:
* Separate Prompt from PromptConfig
* Embed Config in Resources (for both prompt and promptset)
* Add `ToConfig()` to extract Config from Resources (for both prompt and
promptset)
To keep a persistent backend storage for configuration, we will have to
keep a single source of truth. This involves supporting bi-directional
conversion between ToolsetConfig and Toolset.
This PR make the following changes:
* Embed ToolsetConfig in Toolset
* Add `ToConfig()` to extract ToolsetConfig from Toolset.
To keep a persistent backend storage for configuration, we will have to
keep a single source of truth. This involves supporting bi-directional
conversion between Config and Source.
This PR make the following changes:
* Embed Config in Source
* Add `ToConfig()` to extract Config from Source.
Allow Toolbox server to automatically update when users modify their
tool configuration file(s), instead of requiring a restart.
This feature is automatically enabled, but can be turned off with the
flag `--disable-reload`.
Adds logic to make the server shutdown gracefully, including better
respecting cancelled contexts and providing up to 10 seconds to finish
current connections.
Logging support 4 different types of logging (debug, info, warn, error).
The default logging level is Info.
User will be able to set flag for log level (allowed values: "debug",
"info", "warn", "error"), example:
`go run . --log-level debug`
User will be able to set flag for logging format (allowed values:
"standard", "JSON"), example:
`go run . --logging-format json`
**sample http request log - std:**
server
```
2024-11-12T15:08:11.451377-08:00 INFO "Initalized 0 sources.\n"
```
httplog
```
2024-11-26T15:15:53.947287-08:00 INFO Response: 200 OK service: "httplog" httpRequest: {url: "http://127.0.0.1:5000/" method: "GET" path: "/" remoteIP: "127.0.0.1:64216" proto: "HTTP/1.1" requestID: "macbookpro.roam.interna/..." scheme: "http" header: {user-agent: "curl/8.7.1" accept: "*/*"}} httpResponse: {status: 200 bytes: 22 elapsed: 0.012417}
```
**sample http request log - structured:**
server
```
{
"timestamp":"2024-11-04T16:45:11.987299-08:00",
"severity":"ERROR",
"logging.googleapis.com/sourceLocation":{
"function":"github.com/googleapis/genai-toolbox/internal/log.(*StructuredLogger).Errorf",
"file":"/Users/yuanteoh/github/genai-toolbox/internal/log/log.go","line":157
},
"message":"unable to parse tool file at \"tools.yaml\": \"cloud-sql-postgres1\" is not a valid kind of data source"
}
```
httplog
```
{
"timestamp":"2024-11-26T15:12:49.290974-08:00",
"severity":"INFO",
"logging.googleapis.com/sourceLocation":{
"function":"github.com/go-chi/httplog/v2.(*RequestLoggerEntry).Write",
"file":"/Users/yuanteoh/go/pkg/mod/github.com/go-chi/httplog/v2@v2.1.1/httplog.go","line":173
},
"message":"Response: 200 OK",
"service":"httplog",
"httpRequest":{
"url":"http://127.0.0.1:5000/",
"method":"GET",
"path":"/",
"remoteIP":"127.0.0.1:64140",
"proto":"HTTP/1.1",
"requestID":"yuanteoh-macbookpro.roam.internal/NBrtYBu3q9-000001",
"scheme":"http",
"header":{"user-agent":"curl/8.7.1","accept":"*/*"}
},
"httpResponse":{"status":200,"bytes":22,"elapsed":0.0115}
}
```
Moves all of the "source" and "tool" implementations into their own
packages. This layout makes it a bit more clear where the
implementations are, and seems likely to scale more cleanly as more
sources and tools are added.
1. Calculate tool manifests when server starts.
2. Add toolset manifest endpoints.
---------
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
