mirror of https://github.com/googleapis/genai-toolbox.git synced 2026-01-08 23:18:04 -05:00

Files

Yuan e66cb423a7 docs(deploy/cloud-run): update deploy command (#109 )

Update deploy command to include `--allow-unauthenticated`.
This is optional (hence commented out) and might only be needed for
certain projects.

2024-12-04 10:18:58 -08:00

4.9 KiB

Raw Blame History

Deploy Toolbox to Cloud Run

Before you begin

Install the Google Cloud CLI.
Set the PROJECT_ID environment variable:
```
export PROJECT_ID="my-project-id"
```

Initialize gcloud CLI:

gcloud init
gcloud config set project $PROJECT_ID

Make sure you've set up and initialized your database.

You must have the following APIs enabled:

gcloud services enable run.googleapis.com \
                       cloudbuild.googleapis.com \
                       artifactregistry.googleapis.com \
                       iam.googleapis.com

To create an IAM account, you must have the following IAM permissions (or roles):
- Create Service Account role (roles/iam.serviceAccountCreator)
To deploy from source, you must have the following set of roles:
- Cloud Run Developer (roles/run.developer)
- Service Account User role (roles/iam.serviceAccountUser)

Note

If you are under a domain restriction organization policy restricting unauthenticated invocations for your project, you will need to access your deployed service as described under Testing private services.

Note

If you are using VPC-based sources (such as AlloyDB), make sure your Cloud Run service and the database are in the same VPC network.

Create a service account

Create a backend service account if you don't already have one:
```
gcloud iam service-accounts create toolbox-identity
```

Grant permissions to use secret manager:

gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member serviceAccount:toolbox-identity@$PROJECT_ID.iam.gserviceaccount.com \
    --role roles/secretmanager.secretAccessor

Grant additional permissions to the service account that are specific to the source, e.g.:
- AlloyDB for PostgreSQL
- Cloud SQL for PostgreSQL

Configuration

Set up configuration for tools.yml.

Deploy to Cloud Run

Upload tools.yaml as a secret:
```
gcloud secrets create tools --data-file=tools.yaml
```
If you already have a secret and want to update the secret version, execute the following:
```
gcloud secrets versions add tools --data-file=tools.yaml
```

Set env var of the container image that you want to use for cloud run:

export IMAGE=us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:latest

From the root genai-toolbox directory, deploy Toolbox to Cloud Run using the following command:

gcloud run deploy toolbox \
    --image $IMAGE \
    --service-account toolbox-identity \
    --region us-central1 \
    --set-secrets "/app/tools.yaml=tools:latest" \
    --args="--tools_file=/app/tools.yaml","--address=0.0.0.0","--port=8080"
    # --allow-unauthenticated # https://cloud.google.com/run/docs/authenticating/public#gcloud

If you are using a VPC network, use the command below:

gcloud run deploy toolbox \
    --image $IMAGE \
    --service-account toolbox-identity \
    --region us-central1 \
    --set-secrets "/app/tools.yaml=tools:latest" \
    --args="--tools_file=/app/tools.yaml","--address=0.0.0.0","--port=8080" \
    --network default \
    --subnet default
    # --allow-unauthenticated # https://cloud.google.com/run/docs/authenticating/public#gcloud

Connecting to Cloud Run

Next, we will use gcloud to authenticate requests to our Cloud Run instance:

Run the run services proxy to proxy connections to Cloud Run:
```
gcloud run services proxy toolbox --port=8080 --region=us-central1
```
If you are prompted to install the proxy, reply Y to install.
Finally, use curl to verify the endpoint works:
```
curl http://127.0.0.1:8080
```

Connecting with Toolbox Client SDK

Next, we will use Toolbox with client SDK:

Below is a list of Client SDKs that are supported:
- LangChain / LangGraph
- LlamaIndex
Run the following to retrieve a non-deterministic URL for the cloud run service:
```
gcloud run services describe toolbox --format 'value(status.url)'
```

Import and initialize the toolbox client with the URL retrieved above:

from toolbox_langchain_sdk import ToolboxClient

# Replace with the cloud run service URL generated above
toolbox = ToolboxClient("http://URL")

4.9 KiB Raw Blame History