genai-toolbox/docs/en/how-to/deploy_gke.md at 4d4f169e42576bf7ba69f7ca5f8f63881f851369

mirror of https://github.com/googleapis/genai-toolbox.git synced 2026-01-08 15:14:00 -05:00

Files

Yuan Teoh 862868f284 feat: add allowed-origins flag (#1984 )

Support `allowed-origins` flag to allow secure deployment of Toolbox.
Current Toolbox is **insecure by default**, which allows all origin
(`*`). This PR also updated docs to notify user of the new
`allowed-origins` flag in the Cloud Run, kubernetes, and docker
deployment docs.

This PR was tested manually by mocking a browser access:
1. Created a HTML file with Javascript fetch named
`malicious-client.html`:
```
<!DOCTYPE html>
<html>
<head>
    <title>Malicious CORS Test</title>
</head>
<body>
    <h1>Attempting to access API at http://127.0.0.1:5000/mcp</h1>
    <p>Check the **Chrome Developer Console** (F12 -> Console tab) for the result.</p>

    <script>
        fetch('http://127.0.0.1:5000/mcp', {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                // The browser automatically adds the 'Origin' header based on where this HTML is served from (http://localhost:8000)
            },
            body: JSON.stringify({
                "jsonrpc": "2.0",
                "id": 1,
                "method": "tools/list"
            })
        })
        .then(response => {
            console.log('Success (but check console for CORS enforcement details):', response);
            return response.json();
        })
        .then(data => console.log('Data received (only if CORS passes):', data))
        .catch(error => console.error('Fetch Error:', error));
    </script>

</body>
</html>
```
2. Run `python3 -m http.server 8000`
3. Open `http://localhost:8000/malicious-client.html` in browser.
4. Tried without `--allowed-origins` flag -- success.
     Tried with `--allowed-origins=http://localhost:8000` -- success.
     Tried with `--allowed-origins=http://foo.com` -- unsuccessful.

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>

2025-11-27 17:03:53 +00:00

7.0 KiB

Raw Blame History

title, type, weight, description

title	type	weight	description
Deploy to Kubernetes	docs	4	How to set up and configure Toolbox to deploy on Kubernetes with Google Kubernetes Engine (GKE).

Before you begin

Set the PROJECT_ID environment variable:
```
export PROJECT_ID="my-project-id"
```
Install the gcloud CLI.

Initialize gcloud CLI:

gcloud init
gcloud config set project $PROJECT_ID

You must have the following APIs enabled:

gcloud services enable artifactregistry.googleapis.com \
                       cloudbuild.googleapis.com \
                       container.googleapis.com \
                       iam.googleapis.com

kubectl is used to manage Kubernetes, the cluster orchestration system used by GKE. Verify if you have kubectl installed:
```
kubectl version --client
```
If needed, install kubectl component using the Google Cloud CLI:
```
gcloud components install kubectl
```

Create a service account

Specify a name for your service account with an environment variable:
```
export SA_NAME=toolbox
```

Create a backend service account:

gcloud iam service-accounts create $SA_NAME

Grant any IAM roles necessary to the IAM service account. Each source has a list of necessary IAM permissions listed on its page. The example below is for cloud sql postgres source:
```
gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com \
    --role roles/cloudsql.client
```

Deploy to Kubernetes

Set environment variables:

export CLUSTER_NAME=toolbox-cluster
export DEPLOYMENT_NAME=toolbox
export SERVICE_NAME=toolbox-service
export REGION=us-central1
export NAMESPACE=toolbox-namespace
export SECRET_NAME=toolbox-config
export KSA_NAME=toolbox-service-account

Create a GKE cluster.

gcloud container clusters create-auto $CLUSTER_NAME \
    --location=us-central1

Get authentication credentials to interact with the cluster. This also configures kubectl to use the cluster.

gcloud container clusters get-credentials $CLUSTER_NAME \
    --region=$REGION \
    --project=$PROJECT_ID

View the current context for kubectl.
```
kubectl config current-context
```
Create namespace for the deployment.
```
kubectl create namespace $NAMESPACE
```

Create a Kubernetes Service Account (KSA).

kubectl create serviceaccount $KSA_NAME --namespace $NAMESPACE

Enable the IAM binding between Google Service Account (GSA) and Kubernetes Service Account (KSA).

gcloud iam service-accounts add-iam-policy-binding \
    --role="roles/iam.workloadIdentityUser" \
    --member="serviceAccount:$PROJECT_ID.svc.id.goog[$NAMESPACE/$KSA_NAME]" \
    $SA_NAME@$PROJECT_ID.iam.gserviceaccount.com

Add annotation to KSA to complete binding:

kubectl annotate serviceaccount \
    $KSA_NAME \
    iam.gke.io/gcp-service-account=$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com \
    --namespace $NAMESPACE

Prepare the Kubernetes secret for your tools.yaml file.

kubectl create secret generic $SECRET_NAME \
    --from-file=./tools.yaml \
    --namespace=$NAMESPACE

Create a Kubernetes manifest file (k8s_deployment.yaml) to build deployment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: toolbox
  namespace: toolbox-namespace
spec:
  selector:
    matchLabels:
      app: toolbox
  template:
    metadata:
      labels:
        app: toolbox
    spec:
      serviceAccountName: toolbox-service-account
      containers:
        - name: toolbox
          # Recommend to use the latest version of toolbox
          image: us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:latest
          args: ["--address", "0.0.0.0"]
          ports:
            - containerPort: 5000
          volumeMounts:
            - name: toolbox-config
              mountPath: "/app/tools.yaml"
              subPath: tools.yaml
              readOnly: true
      volumes:
        - name: toolbox-config
          secret:
            secretName: toolbox-config
            items:
            - key: tools.yaml
              path: tools.yaml

{{< notice tip >}}
To prevent DNS rebinding attack, use the --allowed-origins flag to specify a list of origins permitted to access the server. E.g. args: ["--address", "0.0.0.0", "--allowed-origins", "https://foo.bar"] {{< /notice >}}

Create the deployment.

kubectl apply -f k8s_deployment.yaml --namespace $NAMESPACE

Check the status of deployment.

kubectl get deployments --namespace $NAMESPACE

Create a Kubernetes manifest file (k8s_service.yaml) to build service.

apiVersion: v1
kind: Service
metadata:
  name: toolbox-service
  namespace: toolbox-namespace
  annotations:
    cloud.google.com/l4-rbs: "enabled"
spec:
  selector:
    app: toolbox
  ports:
    - port: 5000
      targetPort: 5000
  type: LoadBalancer

Create the service.

kubectl apply -f k8s_service.yaml --namespace $NAMESPACE

You can find your IP address created for your service by getting the service information through the following.
```
kubectl describe services $SERVICE_NAME --namespace $NAMESPACE
```

To look at logs, run the following.

kubectl logs -f deploy/$DEPLOYMENT_NAME --namespace $NAMESPACE

You might have to wait a couple of minutes. It is ready when you can see EXTERNAL-IP with the following command:
```
kubectl get svc -n $NAMESPACE
```
Access toolbox locally.
```
curl <EXTERNAL-IP>:5000
```

Clean up resources

Delete secret.

kubectl delete secret $SECRET_NAME --namespace $NAMESPACE

Delete deployment.

kubectl delete deployment $DEPLOYMENT_NAME --namespace $NAMESPACE

Delete the application's service.

kubectl delete service $SERVICE_NAME --namespace $NAMESPACE

Delete the Kubernetes cluster.

gcloud container clusters delete $CLUSTER_NAME \
    --location=$REGION

7.0 KiB Raw Blame History

Before you begin

Create a service account

Deploy to Kubernetes

Clean up resources

7.0 KiB

Raw Blame History