genai-toolbox/docs/en/how-to/deploy_toolbox.md

title, type, weight, description

title	type	weight	description
Deploy to Cloud Run	docs	3	How to set up and configure Toolbox to run on Cloud Run.

Before you begin

1. Install the Google Cloud CLI.

2. Set the PROJECT_ID environment variable:

   export PROJECT_ID="my-project-id"
   

3. Initialize gcloud CLI:

   gcloud init
   gcloud config set project $PROJECT_ID
   

4. Make sure you've set up and initialized your database.

5. You must have the following APIs enabled:

   gcloud services enable run.googleapis.com \
                          cloudbuild.googleapis.com \
                          artifactregistry.googleapis.com \
                          iam.googleapis.com \
                          secretmanager.googleapis.com
   
   

6. To create an IAM account, you must have the following IAM permissions (or roles):

   • Create Service Account role (roles/iam.serviceAccountCreator)

7. To create a secret, you must have the following roles:

   • Secret Manager Admin role (roles/secretmanager.admin)

8. To deploy to Cloud Run, you must have the following set of roles:

   • Cloud Run Developer (roles/run.developer)
   • Service Account User role (roles/iam.serviceAccountUser)

{{< notice note >}} If you are using sources that require VPC-access (such as AlloyDB or Cloud SQL over private IP), make sure your Cloud Run service and the database are in the same VPC network. {{< /notice >}}

Create a service account

1. Create a backend service account if you don't already have one:

   gcloud iam service-accounts create toolbox-identity
   

2. Grant permissions to use secret manager:

   gcloud projects add-iam-policy-binding $PROJECT_ID \
       --member serviceAccount:toolbox-identity@$PROJECT_ID.iam.gserviceaccount.com \
       --role roles/secretmanager.secretAccessor
   

3. Grant additional permissions to the service account that are specific to the source, e.g.:

   • AlloyDB for PostgreSQL
   • Cloud SQL for PostgreSQL

Configure tools.yaml file

Create a tools.yaml file that contains your configuration for Toolbox. For details, see the configuration section.

Deploy to Cloud Run

1. Upload tools.yaml as a secret:

   gcloud secrets create tools --data-file=tools.yaml
   

   If you already have a secret and want to update the secret version, execute the following:

   gcloud secrets versions add tools --data-file=tools.yaml
   

2. Set an environment variable to the container image that you want to use for cloud run:

   export IMAGE=us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:latest
   

3. Deploy Toolbox to Cloud Run using the following command:

   gcloud run deploy toolbox \
       --image $IMAGE \
       --service-account toolbox-identity \
       --region us-central1 \
       --set-secrets "/app/tools.yaml=tools:latest" \
       --args="--tools-file=/app/tools.yaml","--address=0.0.0.0","--port=8080"
       # --allow-unauthenticated # https://cloud.google.com/run/docs/authenticating/public#gcloud
   

   If you are using a VPC network, use the command below:

   gcloud run deploy toolbox \
       --image $IMAGE \
       --service-account toolbox-identity \
       --region us-central1 \
       --set-secrets "/app/tools.yaml=tools:latest" \
       --args="--tools-file=/app/tools.yaml","--address=0.0.0.0","--port=8080" \
       # TODO(dev): update the following to match your VPC if necessary 
       --network default \
       --subnet default
       # --allow-unauthenticated # https://cloud.google.com/run/docs/authenticating/public#gcloud
   

Connecting with Toolbox Client SDK

You can connect to Toolbox Cloud Run instances directly through the SDK.

1. Set up Cloud Run Invoker role access to your Cloud Run service.

2. (Only for local runs) Set up Application Default Credentials for the principle you set up the Cloud Run Invoker role access to.

3. Run the following to retrieve a non-deterministic URL for the cloud run service:

   gcloud run services describe toolbox --format 'value(status.url)'
   

4. Import and initialize the toolbox client with the URL retrieved above:

   from toolbox_core import ToolboxClient, auth_methods
   
   # Replace with the Cloud Run service URL generated in the previous step.
   URL = "https://cloud-run-url.app"
   
   auth_token_provider = auth_methods.aget_google_id_token(URL) # can also use sync method
   
   async with ToolboxClient(
       URL,
       client_headers={"Authorization": auth_token_provider},
   ) as toolbox:
   

Now, you can use this client to connect to the deployed Cloud Run instance!