How Do You Deploy an Embedding Model on GCP?

Deploying an embedding model on GCP with SIE requires configuring a GPU cluster using the SIE Terraform module for Google Cloud, deploying the inference server with Helm, and connecting via the SIE SDK. Your data stays within your GCP project, costs are up to 50× lower than managed API pricing, and the full deployment takes under 30 minutes.

Why deploy on GCP?

Google Cloud is a strong choice for self-hosted embedding inference when:

• Your application stack already runs on GCP (GKE, Cloud Run, BigQuery)
• You need to keep data in a specific region for compliance (GDPR, data residency)
• You want to use Google’s A100 or L4 TPU-adjacent GPU fleet
• You’re optimising costs with GCP committed use discounts or spot VMs

SIE’s GCP Terraform module provisions everything needed: a GKE cluster, GPU node pools, autoscaling, and networking, all within your own GCP project.

Prerequisites

• A GCP project with billing enabled
• GPU quota approved for your target region (request via GCP console if needed)
• Terraform installed (>= 1.3)
• Helm installed (>= 3.0)
• gcloud CLI authenticated (gcloud auth application-default login)
• kubectl installed

Step 1: Configure with Terraform

# main.tf
module "sie" {
  source  = "superlinked/sie/google"
  version = "~> 1.0"

  project_id = "your-gcp-project-id"
  region     = "us-central1"

  # One L4 pool on spot capacity; add pools for other GPU types
  gpu_node_pools = [
    {
      name           = "l4-pool"
      machine_type   = "g2-standard-8"
      gpu_type       = "nvidia-l4"
      gpu_count      = 1
      min_node_count = 0
      max_node_count = 6
      spot           = true
    },
  ]
}

output "kubectl_config_command" {
  value = module.sie.kubectl_config_command
}

output "workload_identity_annotation" {
  value = module.sie.workload_identity_annotation
}

terraform init
terraform plan
terraform apply   # provisions the GKE cluster and GPU node pool

Terraform creates a GKE cluster with an L4 GPU node pool and the Workload Identity service account the SIE pods run under. The GCP deployment guide covers multi-pool setups.

Step 2: Deploy the inference server

# Authenticate kubectl with your new GKE cluster
$(terraform output -raw kubectl_config_command)

# The annotation output is "key=email"; the chart takes the email.
WI_SA=$(terraform output -raw workload_identity_annotation | cut -d= -f2)

# Install the chart from GHCR. Worker pools default to off, so enable the L4 pool.
helm upgrade --install sie oci://ghcr.io/superlinked/charts/sie-cluster \
  -n sie --create-namespace \
  --set "serviceAccount.annotations.iam\.gke\.io/gcp-service-account=$WI_SA" \
  --set workers.pools.l4.enabled=true \
  --set workers.pools.l4.bundles.default.minReplicas=1

To scale workers with demand, add --set keda.install=true --set autoscaling.enabled=true.

Step 3: Verify

kubectl get pods -n sie

# Reach the gateway from your machine
kubectl -n sie port-forward svc/sie-sie-cluster-gateway 8080:8080 &

curl http://localhost:8080/health
# → JSON with the gateway status, its workers and the models they serve

Step 4: Start encoding

from sie_sdk import SIEClient
from sie_sdk.types import Item

client = SIEClient("http://localhost:8080")  # the port-forward from step 3

results = client.encode(
    "BAAI/bge-m3",
    [
        Item(text="self-hosted inference on GCP"),
        Item(text="keep embeddings within your Google Cloud project"),
    ],
)

print(f"Encoded {len(results)} docs, dim={len(results[0]['dense'])}")

GCP GPU selection guide

GCP L4 spot VMs offer excellent price-performance for BGE-M3 and similar models. Spot preemption is handled by SIE’s cluster gracefully, with requests rerouted to healthy nodes.

Enabling GPU quota on GCP

GCP requires explicit quota approval for GPU instances. If you see quota errors during terraform apply:

1. Go to IAM & Admin → Quotas in the GCP Console
2. Search for NVIDIA_A100_GPUS or NVIDIA_L4_GPUS in your target region
3. Click Edit Quotas, request the number you need (start with 4-8)
4. Approval typically takes a few hours for small quotas

Cost comparison: GCP self-hosted vs managed APIs

Encoding 100M documents with BGE-M3 (avg 256 tokens each):

The self-hosted cost varies by GPU utilisation and run time, but the savings are substantial at scale.

Frequently asked questions

Does SIE support GKE Autopilot? SIE works with both GKE Standard and GKE Autopilot. Autopilot simplifies node management but has more constraints on GPU configuration. Standard mode gives more control for GPU workloads.

Can I use Workload Identity with SIE? Yes. The Terraform module configures Workload Identity by default, so SIE pods can access GCS buckets or Secret Manager without service account key files.

What regions have good GPU availability on GCP? us-central1 and us-east1 typically have the best A100 and L4 availability. europe-west4 is the best European option. Check GCP’s GPU availability dashboard before choosing a region.

Related resources

• SIE deployment documentation
• How do you deploy on AWS?
• What is self-hosted inference?
• SIE vs TEI vs OpenAI benchmark
• What is GPU utilisation in inference?