Commit Graph

136 Commits

Author SHA1 Message Date
Twisha Bansal
bd6a7455f1 chore: fix link (#168) 2024-12-25 20:01:45 +00:00
Twisha Bansal
6edcf36a2b chore: added period (#169) 2024-12-25 11:55:43 -08:00
Yuan
809e547a48 feat: update log with given context (#147)
Update logging with the given context.
2024-12-23 19:55:25 -08:00
Yuan
bf276e5f56 ci: combine integration cloudbuild job (#166) 2024-12-23 15:29:35 -08:00
Twisha Bansal
938f7b41d7 chore: fix comments (#163) 2024-12-23 09:28:14 -07:00
Anubhav Dhawan
003ce510a1 feat(llamaindex-sdk): Implement OAuth support for LlamaIndex. (#159)
* Update docs for Authenticated Tools.
* Implement test cases for OAuth in LlamaIndex.
* Sync SDK dependency versions b/w LangChain and LlamaIndex.
2024-12-23 14:24:22 +05:30
Wenxin Du
5ae30c2cff ci: Add AlloyDB integration test (#158)
1. Test AlloyDB as a source connection.
2. Adding individual go build tags to separate integration test runs for
different sources.
3. Modify integration test command to only run tests in the `./tests`
directory.
2024-12-22 13:35:43 -05:00
Anubhav Dhawan
3ac7dcfb9c doc(langchain-sdk): Improve doc and tests for LangChain SDK. (#154) 2024-12-19 18:56:25 +05:30
release-please[bot]
21a9146404 chore(main): release 0.0.4 (#153)
🤖 I have created a release *beep* *boop*
---


##
[0.0.4](https://github.com/googleapis/genai-toolbox/compare/v0.0.3...v0.0.4)
(2024-12-18)


### Features

* Add `auth_required` to tools
([#123](https://github.com/googleapis/genai-toolbox/issues/123))
([3118104](3118104ae1))
* Add Auth Source configuration
([#71](https://github.com/googleapis/genai-toolbox/issues/71))
([77b0d43](77b0d43175))
* Add Tool authenticated parameters
([#80](https://github.com/googleapis/genai-toolbox/issues/80))
([380a6fb](380a6fbbd5))
* **langchain-sdk:** Correctly parse Manifest API response as JSON
([#143](https://github.com/googleapis/genai-toolbox/issues/143))
([2c8633c](2c8633c3eb))
* **langchain-sdk:** Support authentication in LangChain Toolbox SDK.
([#133](https://github.com/googleapis/genai-toolbox/issues/133))
([23fa912](23fa912a80))


### Bug Fixes

* Fix release image version tag
([#136](https://github.com/googleapis/genai-toolbox/issues/136))
([6d19ff9](6d19ff96e4))
* **langchain-sdk:** Correct test name to ensure execution and full
coverage.
([#145](https://github.com/googleapis/genai-toolbox/issues/145))
([d820ac3](d820ac3767))
* Set server version
([#150](https://github.com/googleapis/genai-toolbox/issues/150))
([abd1eb7](abd1eb702c))


### Miscellaneous Chores

* Release 0.0.4
([#152](https://github.com/googleapis/genai-toolbox/issues/152))
([86ec12f](86ec12f8c5))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
v0.0.4
2024-12-18 18:48:13 -05:00
Wenxin Du
86ec12f8c5 chore: release 0.0.4 (#152)
Release-As: 0.0.4
2024-12-18 15:29:02 -08:00
Wenxin Du
abd1eb702c fix: Set server version (#150) 2024-12-18 17:15:55 -05:00
Anubhav Dhawan
46f8bb3bc4 doc(langchain-sdk): Add guidance for configuring and using authenticated tools in Toolbox. (#146) 2024-12-18 12:54:57 -05:00
Wenxin Du
8ca7e91240 docs: Add auth source docs (#132)
Add authsources and authenticated parameter docs. Format some other .md
docs.

---------

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
2024-12-17 13:49:43 -05:00
Anubhav Dhawan
d820ac3767 fix(langchain-sdk): Correct test name to ensure execution and full coverage. (#145) 2024-12-17 12:05:10 +00:00
Anubhav Dhawan
854f69384d chore(langchain-sdk): Fix warning about async session not closed after langchain utils tests. (#144) 2024-12-17 15:58:21 +05:30
Anubhav Dhawan
2c8633c3eb feat(langchain-sdk): Correctly parse Manifest API response as JSON (#143)
The Manifest API returns a JSON payload. Previously, it was parsed as
YAML, which worked due to YAML's superset relationship with JSON. This
change explicitly parses the response as JSON for improved robustness
and security by ensuring strict adherence to the expected format.
2024-12-17 15:29:18 +05:30
Wenxin Du
3118104ae1 feat: add auth_required to tools (#123)
Check if a tool invocation request contains required auth token.
2024-12-16 22:41:13 -05:00
Wenxin Du
380a6fbbd5 feat: Add Tool authenticated parameters (#80)
1. Add []ParamAuthSource to every Parameter type implementation to
support authenticated configs. Create new constructors for types with
auth.
2. Tool invocation API changes to parse auth header and authentecated
parameters.
3. Add authSources to Tool manifest.
2024-12-16 17:46:06 -05:00
Yuan
bb30286e65 docs(spanner): add docs (#128) 2024-12-16 11:03:14 -08:00
Anubhav Dhawan
8cc4fe384e chore(langchain-sdk): Match error message while testing tool invocation failure modes. (#141) 2024-12-16 12:05:40 +00:00
Twisha Bansal
119333bf2d tests(llamaindex-sdk): Added unit tests for llamaindex SDK client and utils. (#134)
Test coverage:

| Name    | Stmts | Miss | Cover |
| -------- | ------- | ---- | ----- |
| src/toolbox_llamaindex_sdk/__init__.py  | 2   |  0 | 100% |
| src/toolbox_llamaindex_sdk/client.py | 43    | 2 | 95% |
| src/toolbox_llamaindex_sdk/utils.py    | 48   | 0 | 100%|
| TOTAL | 93 | 2 | 98% |
2024-12-16 12:30:57 +05:30
Lawrence Wu
9860da8ecb chore: Update langchain README.md to run (#139)
Fixed missing or incorrect imports and added an example of how to pass
invoke the compiled `langgraph` agent
2024-12-13 20:38:14 -06:00
Lawrence Wu
6fe0328a69 chore: update example in README (#135)
`int` is not valid in the `tools.yaml`

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2024-12-13 20:37:29 -06:00
Anubhav Dhawan
2b3be816f1 doc: Update README.md (#138) 2024-12-13 18:05:04 +00:00
Wenxin Du
77b0d43175 feat: Add Auth Source configuration (#71)
1. Add authentication source configs to Toolbox server.
2. Add Google auth implementation.
2024-12-13 12:22:42 -05:00
Yuan
daa679e9c3 chore(deps): update dependencies (#137) 2024-12-12 14:40:07 -08:00
Yuan
7e8d751d3f ci(cloudsql-pg): add end to end integration test (#113)
End to end integration test for cloudsql postgres.

Include checks for one tool's get (manifest) and post (invoke) endpoint.

Integration tests are excluded from regular unit tests.
2024-12-12 13:56:04 -08:00
Yuan
6d19ff96e4 fix: fix release image version tag (#136) 2024-12-12 12:05:38 -08:00
dependabot[bot]
eab98d9c51 chore(deps): bump golang.org/x/crypto from 0.28.0 to 0.31.0 (#131)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from
0.28.0 to 0.31.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b4f1988a35"><code>b4f1988</code></a>
ssh: make the public key cache a 1-entry FIFO cache</li>
<li><a
href="7042ebcbe0"><code>7042ebc</code></a>
openpgp/clearsign: just use rand.Reader in tests</li>
<li><a
href="3e90321ac7"><code>3e90321</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="8c4e668694"><code>8c4e668</code></a>
x509roots/fallback: update bundle</li>
<li><a
href="6018723c74"><code>6018723</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="71ed71b4fa"><code>71ed71b</code></a>
README: don't recommend go get</li>
<li><a
href="750a45fe5e"><code>750a45f</code></a>
sha3: add MarshalBinary, AppendBinary, and UnmarshalBinary</li>
<li><a
href="36b172546b"><code>36b1725</code></a>
sha3: avoid trailing permutation</li>
<li><a
href="80ea76eb17"><code>80ea76e</code></a>
sha3: fix padding for long cSHAKE parameters</li>
<li><a
href="c17aa50fbd"><code>c17aa50</code></a>
sha3: avoid buffer copy</li>
<li>Additional commits viewable in <a
href="https://github.com/golang/crypto/compare/v0.28.0...v0.31.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/crypto&package-manager=go_modules&previous-version=0.28.0&new-version=0.31.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/googleapis/genai-toolbox/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
2024-12-12 09:04:08 -08:00
Anubhav Dhawan
23fa912a80 feat(langchain-sdk): Support authentication in LangChain Toolbox SDK. (#133)
This PR adds support for authentication in the `ToolboxClient`.

Note: This is a resubmission of #117
2024-12-12 13:01:57 +05:30
Twisha Bansal
01623edba3 tests(langchain-sdk): Added unit tests for langchain SDK client and utils. (#118)
Test coverage:

| Name    | Stmts | Miss | Cover |
| -------- | ------- | ---- | ----- |
| src/toolbox_langchain_sdk/__init__.py  | 2   |  0 | 100% |
| src/toolbox_langchain_sdk/client.py | 43    | 2 | 95% |
| src/toolbox_langchain_sdk/utils.py    | 48   | 0 | 100%|
| TOTAL | 93 | 2 | 98% |
2024-12-12 07:05:17 +00:00
Twisha Bansal
218e377554 chore(langchain-ci): use working dir for shell commands (#126) 2024-12-12 10:31:48 +05:30
Twisha Bansal
7bb0a13766 chore(llamaindex-ci): use working dir for shell commands (#127) 2024-12-12 10:21:23 +05:30
Yuan
827d845759 docs: add ip_type reference to alloydb and cloudsql doc (#130)
Update source docs to include `ip_type` reference for config.
2024-12-11 22:16:10 +00:00
Yuan
cb3357f305 fix!(spanner): use googlesql for dialect name (#129)
Google updated the official name from `google_standard_sql` to
`googlesql`.
2024-12-11 14:11:34 -08:00
Andrew Brook
07ddf6e26c chore: modify README.md to make tool description consistent (#125) 2024-12-11 14:48:20 -07:00
release-please[bot]
504fbf508b chore(main): release 0.0.3 (#75)
🤖 I have created a release *beep* *boop*
---


##
[0.0.3](https://github.com/googleapis/genai-toolbox/compare/v0.0.2...v0.0.3)
(2024-12-10)


### Features

* Add --log-level and --logging-format flags
([#97](https://github.com/googleapis/genai-toolbox/issues/97))
([9a0f618](9a0f618efc))
* Add options for command
([#110](https://github.com/googleapis/genai-toolbox/issues/110))
([5c690c5](5c690c5c30))
* Add Spanner source and tool
([#90](https://github.com/googleapis/genai-toolbox/issues/90))
([890914a](890914aae0))
* Add std logger
([#95](https://github.com/googleapis/genai-toolbox/issues/95))
([6a8feb5](6a8feb51f0))
* Add structured logger
([#96](https://github.com/googleapis/genai-toolbox/issues/96))
([5e20417](5e20417551))
* **source/alloydb-pg:** Add configuration for public and private IP
([#103](https://github.com/googleapis/genai-toolbox/issues/103))
([e88ec40](e88ec409d1))
* **source/cloudsql-pg:** Add configuration for public and private IP
([#114](https://github.com/googleapis/genai-toolbox/issues/114))
([6479c1d](6479c1dbe2))


### Bug Fixes

* Fix go test workflow
([#84](https://github.com/googleapis/genai-toolbox/issues/84))
([8c2c373](8c2c373d35))
* Fix issue causing client session to not close properly while closing
SDK. ([#81](https://github.com/googleapis/genai-toolbox/issues/81))
([9d360e1](9d360e16ea))
* Fix test cases for ip_type
([#115](https://github.com/googleapis/genai-toolbox/issues/115))
([5528bec](5528bec8ed))
* Fix the errors showing up after setting up mypy type checker.
([#74](https://github.com/googleapis/genai-toolbox/issues/74))
([522bbef](522bbefa7b))
* **llamaindex-sdk:** Fix issue causing client session to not close
properly while closing SDK.
([#82](https://github.com/googleapis/genai-toolbox/issues/82))
([fa03376](fa03376bbc))


### Miscellaneous Chores

* Release 0.0.3
([#122](https://github.com/googleapis/genai-toolbox/issues/122))
([626e12f](626e12fdb3))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
v0.0.3
2024-12-09 19:39:11 -05:00
Wenxin Du
626e12fdb3 chore: release 0.0.3 (#122)
Release-As: 0.0.3
2024-12-09 18:44:47 -05:00
Yuan
45a38ca9f0 chore: separate listen and server into two functions (#121)
Separate listen and server into two distinct functions to prevent race
condition during testing.
2024-12-09 22:57:53 +00:00
Twisha Bansal
2a394ec72d ci(langchain-sdk): Update file format check to run inside the sdk directory (#119) 2024-12-09 15:52:25 +00:00
Twisha Bansal
536f00b6a6 ci(llamaindex-sdk): Update file format check to run inside the sdk directory (#120) 2024-12-09 21:18:13 +05:30
Yuan
890914aae0 feat: add Spanner source and tool (#90)
Add Spanner source and tool.

Spanner source is initialize with the following config:
```
sources:
    my-spanner-source:
        kind: spanner
        project: my-project-name
        instance: my-instance-name
        database: my_db
        # dialect: postgresql # The default dialect is google_standard_sql.
```

Spanner tool (with gsql dialect) is initialize with the following
config.
```
tools:
    get_flight_by_id:
        kind: spanner
        source: my-cloud-sql-source
        description: >
            Use this tool to list all airports matching search criteria. Takes 
            at least one of country, city, name, or all and returns all matching
            airports. The agent can decide to return the results directly to 
            the user.
        statement: "SELECT * FROM flights WHERE id = @id"
        parameters:
        - name: id
          type: int
          description: 'id' represents the unique ID for each flight. 
```

Spanner tool (with postgresql dialect) is initialize with the following
config.
```
tools:
    get_flight_by_id:
        kind: spanner
        source: my-cloud-sql-source
        description: >
            Use this tool to list all airports matching search criteria. Takes 
            at least one of country, city, name, or all and returns all matching
            airports. The agent can decide to return the results directly to 
            the user.
        statement: "SELECT * FROM flights WHERE id = $1"
        parameters:
        - name: id
          type: int
          description: 'id' represents the unique ID for each flight. 
```

Note: the only difference in config for both dialects is the sql
statement.

---------

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
2024-12-06 16:38:03 -08:00
Yuan
5528bec8ed fix: fix test cases for ip_type (#115)
Set ip_type to be non-case sensitive.
Add more test case to test ip_type with caps.
Fix test name for cloudsql pg.
2024-12-06 11:53:57 -08:00
Twisha Bansal
7991d64f3c ci(llamaindex): Update env variable to use typecheck build from sdks/llamaindex/src (#116)
Running lint type checker (mypy) on the sdk tests can raise errors like:
https://github.com/googleapis/genai-toolbox/actions/runs/12194679407/job/34018934437
because it's looking in the wrong place for type definitions.

To fix this, we'll tell the type checker (mypy) to specifically include
the toolbox_llamaindex_sdk module from our SDK source code when it's
checking the tests. This will ensure it uses the correct type
information and avoids these errors.
2024-12-06 13:46:31 +05:30
Twisha Bansal
b81df9ece7 ci(langchain): Added auto lint and integration test config (#73)
This PR adds CI workflows for linting and integration tests to the
Langchain SDK.

Linting is failing due to type checking errors in the SDK (to be fixed
separately).
Integration tests are failing and need updates to match the latest SDK
code (also to be fixed separately).
2024-12-06 07:32:55 +00:00
Yuan
3df68f1b69 chore: set out and err io.Writer during initialization (#112)
Logger might be created pre- or post- flag. The logger defined pre- and
post- flag should be using the same out and err writer.

This will ensure that the server uses the same out and err writer
defined with the `WithStreams` options.
2024-12-05 16:14:43 -08:00
Yuan
6479c1dbe2 feat(source/cloudsql-pg): add configuration for public and private IP (#114)
Allow user to set if their database uses private or public ip. The
reason we add this is because the dialer require different
initialization with private and public ip.

By default, toolbox will use public ip.
2024-12-06 00:08:15 +00:00
Yuan
e815dc49f4 chore: update params type (#98)
Different databases require different types for `Params` field when
adding parameters to their statement. e.g. alloydb, cloudsql, and
postgres uses `pgxpool` to query and build sql statement, whereas
spanner uses `Spanner` library.

Added a new `ParamValue` struct. `ParseParams` helper function parses
arbitraryJSON object into `[]ParamValue`, and the tool's invoke will
convert `[]ParamValue` into it's required type.

---------

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
2024-12-05 16:05:09 -08:00
Yuan
e88ec409d1 feat(source/alloydb-pg): add configuration for public and private IP (#103)
Allow user to set if their database uses private or public ip. The
reason we add this is because the dialer require different
initialization with private and public ip.

By default, toolbox will use public ip.

---------

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
2024-12-05 23:05:53 +00:00
Yuan
5c690c5c30 feat: add options for command (#110)
Add Option to configure a Command.

Currently we are only supporting `WithLogger` Option.
2024-12-05 09:02:47 -08:00