---
title: Kubernetes in Azure
description: Deploy SIE on Azure AKS with GPU autoscaling.
canonical_url: https://superlinked.com/docs/deployment/cloud-azure
last_updated: 2026-07-22
---

Deploy SIE to Azure Kubernetes Service (AKS) with GPU node pools, KEDA autoscaling, and Terraform automation.

## Prerequisites

There are two install paths for AKS. Confirm the items under the path you plan to take before running any commands.

### Path A. Terraform and Helm (module provisions the cluster)

1. **Azure subscription** with billing enabled. For CI, use a federated service principal via the GitHub Actions Azure OIDC flow; see the [module README](https://github.com/superlinked/terraform-azure-sie#ci-authentication-github-actions) for the setup.
2. **RBAC roles on the subscription** sufficient to create VNet, AKS, identity, storage, and (optionally) ACR resources. `Contributor` plus `User Access Administrator` works; the latter covers the role assignments the module creates for the model cache and image pulls. For the least-privilege set (Network Contributor, AKS Contributor, AKS RBAC Cluster Admin, AcrPush, Monitoring Contributor, Key Vault Contributor, RBAC Admin), see [scoped role assignments](https://learn.microsoft.com/azure/role-based-access-control/best-practices).
3. **GPU quota in your target region** (default: `westeurope`), requested from the Quotas blade in the Azure portal. Spot pools draw from `Total Regional Spot vCPUs`; on-demand pools draw from the per-family quota (*Standard NCASv3_T4 Family vCPUs* for T4, *Standard NVADSA10v5 Family vCPUs* for A10, *Standard NCadsH100v5 Family vCPUs* for H100). The `dev-nc4ast4-spot` example uses spot. `Standard_NC4as_T4_v3` is 4 vCPU per node and the example scales 0–5 nodes, so anything ≥ 20 is sufficient:

   ```bash
   az vm list-usage --location westeurope --output table | grep -iE "NCASv3_T4|Spot"
   ```

   H100 quota approvals are the slowest; file the request before the first apply if you plan to use `gpu_class = "h100"`.
4. **Region with GPU VM availability.** T4 (`NC4as_T4_v3`) and A10 (`NV6ads_A10_v5`) are available in most major regions; A100 (`NC24ads_A100_v4`) and H100 (`NC40ads_H100_v5`) availability is narrower. Check before changing region.
5. **Local tooling:** Terraform ≥ 1.14, Azure CLI (`az`), `kubectl`, [`kubelogin`](https://github.com/Azure/kubelogin), and `helm` ≥ 3.13.
6. **Authenticated:**

   ```bash
   az login
   az account set --subscription "<subscription-id>"
   ```

### Path B. Helm into an existing AKS cluster

1. Cluster meets the generic [Kubernetes Cluster Prerequisites](/docs/deployment/#kubernetes-cluster-prerequisites) (k8s version, GPU device plugin, ingress controller, network egress).
2. **GPU node pool** with an N-series VM size (`Standard_NC4as_T4_v3`, `Standard_NV6ads_A10_v5`, `Standard_NC24ads_A100_v4`, or `Standard_NC40ads_H100_v5`) and the `nvidia.com/gpu=present:NoSchedule` taint. Spot pools also carry the AKS-managed `kubernetes.azure.com/scalesetpriority=spot:NoSchedule` taint; the chart's AKS overlay tolerates both.
3. **NVIDIA Device Plugin DaemonSet installed.** AKS preinstalls the NVIDIA driver on N-series node images but not the device plugin. Verify `kubectl describe node` shows a schedulable `nvidia.com/gpu` resource; without the plugin, GPU nodes never advertise it and worker pods stay `Pending`. The Terraform module installs it via Helm during cluster bootstrap. The exception is the preview [AKS-managed GPU node pools](https://learn.microsoft.com/en-us/azure/aks/aks-managed-gpu-nodes): those install the device plugin for you but do not support the cluster autoscaler during preview, so scale-from-zero cannot work on them. This guide, and the Terraform module, target standard GPU pools.
4. **Workload Identity enabled** on the cluster (OIDC issuer + webhook), with a user-assigned managed identity that has a federated credential for the `sie:sie-server` ServiceAccount and blob-read access to your model-cache container. Pass the identity's client ID to the chart as the `azure.workload.identity/client-id` ServiceAccount annotation at install time; the AKS values overlay adds the `azure.workload.identity/use: "true"` pod label the webhook keys off.
5. **ACR decision.** Let the chart pull public images from GHCR (default), or mirror to ACR and point `gateway.image.repository`, `workers.common.image.repository`, and `config.image.repository` at it.
6. **`kubectl` authenticated** against the target cluster (`az aks get-credentials --resource-group <rg> --name <cluster>`, then `kubelogin convert-kubeconfig -l azurecli` for AAD-enabled clusters).

---

## Architecture

The architecture mirrors the [GCP](/docs/deployment/cloud-gcp/) and [AWS](/docs/deployment/cloud-aws/) deployments, with gateway, config, worker pods, and KEDA autoscaling:

![AKS cluster architecture with Gateway, Config service, T4 and A10 worker pools, KEDA, and Prometheus](/diagrams/aks-arch.svg)

**Components:**
- **AKS Cluster** with AAD-RBAC, a Workload Identity OIDC issuer, and the built-in per-pool cluster autoscaler (no separate autoscaler chart)
- **NVIDIA Device Plugin** for GPU scheduling; the driver itself ships on AKS N-series node images
- **Workload Identity** for Blob Storage access, so no static credentials reach a pod
- **NATS Core and JetStream** for queued work, result inboxes, SIE server sidecar health, and config deltas
- **GPU worker pods** that run the SIE server sidecar beside the Python `sie-server` adapter; the sidecar pulls JetStream work and calls the adapter over IPC
- **KEDA and Prometheus** for autoscaling based on gateway and queue metrics
- **Grafana and DCGM Exporter** for dashboards and GPU metrics

---

## Terraform Setup

The examples in [`superlinked/terraform-azure-sie`](https://github.com/superlinked/terraform-azure-sie) consume the published `superlinked/sie/azure` Terraform registry module, pinned to a known-good version:

| Example | GPU | Approx. spot cost | Best for |
|---------|-----|-------------------|----------|
| [`dev-nc4ast4-spot`](https://github.com/superlinked/terraform-azure-sie/tree/main/examples/dev-nc4ast4-spot) | T4 (NC4as_T4_v3) | ~$0.15/hr | Minimal-cost development |
| [`dev-nv6adsa10-spot`](https://github.com/superlinked/terraform-azure-sie/tree/main/examples/dev-nv6adsa10-spot) | A10 (NV6ads_A10_v5) | ~$0.35/hr | Larger embedding bundles (24 GiB VRAM) |

Costs are approximate West Europe spot list prices; check the [Azure pricing calculator](https://azure.microsoft.com/pricing/calculator/) for your region.

Both examples run GPU nodes on **Spot capacity**: cheap, but evictable at any time and without an SLA. They are development setups; for production, configure a regular-priority (non-Spot) GPU pool through the module variables.

### Prerequisites

See [Path A. Terraform and Helm](#path-a-terraform-and-helm-module-provisions-the-cluster) in the Prerequisites section at the top of this page.

### Deploy

```bash
git clone https://github.com/superlinked/terraform-azure-sie.git
cd terraform-azure-sie/examples/dev-nc4ast4-spot

# Required: populates the CAF `Owner` tag on every resource
export TF_VAR_owner="you@example.com"

# Initialize and apply (VNet, AKS control plane, system + GPU pools,
# workload identity, model-cache storage account)
terraform init
terraform apply
```

The example `main.tf` pins the module version:

```hcl
module "sie_aks" {
  source  = "superlinked/sie/azure"
  version = "0.6.18"

  location     = var.location
  project_name = var.project_name
  owner        = var.owner

  gpu_node_pools = [
    {
      name       = "t4spot"
      gpu_class  = "t4"
      node_count = 0 # scale-to-zero when idle
      max_count  = 5
      spot       = true
    },
  ]
}
```

Two module behaviors to know about before the first apply:

- **`storage_use_azuread = true`** is required on your `azurerm` provider block when `create_model_cache = true` (the default). The module disables shared-key auth on the model-cache storage account, so the provider needs AAD tokens to finish the apply. Every shipped example already sets this.
- **`grant_admin_to_creator = true`** gives the identity running Terraform the `AKS RBAC Cluster Admin` role on the new cluster, so `kubectl` works right after apply. Set it unless your identity already holds that role at subscription scope (the cluster is AAD-RBAC; subscription `Owner` alone does not grant Kubernetes data-plane access).

The module does not ship its own state backend or CI-identity bootstrap; those are subscription-wide concerns you own once and reuse across clusters. Each example includes a `backend.hcl.example` for an `azurerm` remote-state backend.

Each `gpu_node_pools` entry picks its GPU via `gpu_class`:

| `gpu_class` | VM size | GPU | VRAM |
|-------------|---------|-----|------|
| `t4` | `Standard_NC4as_T4_v3` | 1x T4 | 16 GB |
| `a10` | `Standard_NV6ads_A10_v5` | 1x A10 | 24 GB |
| `a100` | `Standard_NC24ads_A100_v4` | 1x A100 | 80 GB |
| `h100` | `Standard_NC40ads_H100_v5` | 1x H100 | 80 GB |

Adding A100 or H100 once quota is granted is a values-only change: append an entry to `gpu_node_pools`. For multi-GPU worker pods, override `gpu_node_pools[*].vm_size` to a SKU with more GPUs and set the matching `workers.pools.<name>.gpu.count` on the Helm side. See the [module variables reference](https://github.com/superlinked/terraform-azure-sie/blob/main/variables.tf).

### Variables

Key configuration options for the `superlinked/sie/azure` module:

| Variable | Default | Description |
|----------|---------|-------------|
| `location` | `westeurope` | Azure region |
| `project_name` | `sie` | Name prefix for all resources |
| `owner` | (required) | UPN for the CAF `Owner` tag, e.g. `alice@example.com` |
| `gpu_node_pools` | `[]` | GPU pool definitions (`name`, `gpu_class`, `spot`, `node_count`, `max_count`) |
| `grant_admin_to_creator` | `false` | Grant the applying identity cluster-admin RBAC on the new cluster |
| `create_model_cache` | `true` | Provision the blob container backing the model cache and payload store |
| `create_acr` | `false` | Provision a Premium ACR for custom images (the chart pulls GHCR by default) |
| `enable_private_cluster` | `false` | Private API server endpoint |
| `api_server_authorized_ip_ranges` | `[]` | CIDRs allowed to reach a public API server |
| `deletion_protection` | `true` | `CanNotDelete` lock on the cluster (set `false` for dev) |

The module also fills the four Cloud Adoption Framework taxonomy tags (`Environment`, `Owner`, `CostCenter`, `Workload`) from input variables, so subscriptions with a CAF tag-baseline policy accept the cluster without exceptions.

### What Gets Created

| Resource | Purpose |
|----------|---------|
| AKS Cluster | Control plane with AAD-RBAC, OIDC issuer, and Workload Identity |
| GPU Node Pool | Auto-scaling `Standard_NC4as_T4_v3` T4 spot nodes (0–5), NVIDIA driver preinstalled |
| System Node Pool | `Standard_B4ms` for system workloads, spread across zones |
| VNet + NAT Gateway | Three subnets (system, GPU, private endpoints) with predictable egress IPs |
| NVIDIA Device Plugin | GPU scheduling in Kubernetes (AKS does not install it) |
| Workload Identity | User-assigned identity + federated credential for the `sie:sie-server` ServiceAccount |
| Storage Account | Blob container holding the model cache (`models/`) and payload store (`payloads/`) |
| ACR | Optional (`create_acr = true`). The chart pulls public images from GHCR by default. |
| Private Endpoints | Optional (`enable_private_endpoints = true`): ACR and Storage on Private Link |

---

## Helm Installation

Once the cluster is up, configure `kubectl` and install the `sie-cluster` chart with the AKS values overlay. The overlay installs more than the plain GCP and AWS commands do: KEDA, kube-prometheus-stack, log collection, and nginx ingress come on by default, the `t4` and `a10` machine profiles are pre-wired, and worker pods get the GPU and spot tolerations plus the `azure.workload.identity/use` pod label. The overlay URL is pinned to the `v0.6.18` tag to match the chart version; bump both together when upgrading.

```bash
# Configure kubectl from the terraform output, then convert the kubeconfig
# to Azure CLI token auth (the cluster authenticates kubectl through AAD)
$(terraform output -raw kubectl_config_command)
kubelogin convert-kubeconfig -l azurecli

# Install SIE. The -f flag pulls the AKS overlay from the chart's source repo
# at the matching release tag; model_cache_helm_args wires in the blob cache.
helm upgrade --install sie-cluster oci://ghcr.io/superlinked/charts/sie-cluster \
  --version 0.6.18 \
  -f https://raw.githubusercontent.com/superlinked/sie/v0.6.18/deploy/helm/sie-cluster/values-aks.yaml \
  --namespace sie --create-namespace \
  --set "serviceAccount.annotations.azure\.workload\.identity/client-id=$(terraform output -raw sie_workload_identity_client_id)" \
  $(terraform output -raw model_cache_helm_args)

# Wait for rollout
kubectl -n sie get pods -w
```

For gated Hugging Face models, add `--set hfToken.create=true --set-file hfToken.value=/path/to/token-file` (`--set-file` keeps the token out of argv and shell history; `--set hfToken.value="$HF_TOKEN"` also works but exposes the token to `ps` and CI traces). For the smoke test below (`BAAI/bge-m3`) it is optional; in that case omit the flags entirely (an empty-token secret fails later on any gated-model request).

The overlay ships the `t4` pool enabled at `minReplicas: 0`, so the cluster starts with zero GPU nodes and KEDA provisions them on demand. To keep one worker always warm instead, add `--set workers.pools.t4.bundles.default.minReplicas=1`.

### Smoke Test

```bash
# The AKS overlay sets fullnameOverride=sie, so the service is sie-gateway
kubectl -n sie port-forward svc/sie-gateway 8080:8080 &

# Install the Python SDK. Requires Python 3.12; see the SDK README for newer or older Python notes.
pip install sie-sdk

python3 -c "
from sie_sdk import SIEClient

client = SIEClient('http://localhost:8080')
result = client.encode('BAAI/bge-m3', {'text': 'hello world'},
                       gpu='t4', wait_for_capacity=True, provision_timeout_s=600)
print(result['dense'].shape)  # (1024,)
"
```

The first request after scale-from-zero takes ~5–10 minutes (node provisioning + image pull + model loading). See [Scale-from-Zero](/docs/deployment/autoscaling/) for the full flow.

### Available GPU Types

The AKS overlay defines two machine profiles out of the box:

| GPU Type | Header Value | VM Size |
|----------|--------------|---------|
| NVIDIA T4 | `t4` | Standard_NC4as_T4_v3 |
| NVIDIA A10 | `a10` | Standard_NV6ads_A10_v5 |

Azure's [GPU VM lineup](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview#gpu-accelerated) has no L4 size; the A10 (24 GB VRAM) fills that slot. Route requests with `gpu="t4"` or `gpu="a10"` in the SDK, or the `X-SIE-MACHINE-PROFILE` header over HTTP; see [GPU Selection](/docs/deployment/cloud-gcp/#gpu-selection). For A100 or H100 pools, add a `machineProfiles` entry and a `workers.pools` entry in your values file, following the `t4` blocks in [`values-aks.yaml`](https://github.com/superlinked/sie/blob/main/deploy/helm/sie-cluster/values-aks.yaml), alongside the Terraform `gpu_node_pools` entry.

### Cleanup

```bash
helm uninstall sie-cluster -n sie
terraform destroy
```

Spot GPU pools can be evicted with no warning; Azure Spot has no EC2-style 2-minute interruption notice, and evicted VMs are deleted. Always destroy dev clusters when not in use.

---

## Model Cache and Payload Store on Blob Storage

Two object-store-backed features share one blob container, created by default (`create_model_cache = true`):

- **Model cache**: pre-staged weights under `models/`, so workers cold-start from Blob Storage instead of re-downloading from Hugging Face on every pod spin-up.
- **Payload store**: work-item payloads over the 1 MiB NATS in-band budget (images, long documents) under `payloads/`. The gateway writes them, the worker reads them once, and a runtime TTL plus a blob lifecycle rule garbage-collects them after a day.

The `model_cache_helm_args` Terraform output composes the Helm flags for you. Written out as values:

```yaml
workers:
  common:
    clusterCache:
      enabled: true
      url: abfs://sie-cache@<storage-account>.dfs.core.windows.net/models
```

The chart derives `payloadStore.url` from the cache URL, so one `--set` covers both. Workload Identity handles authentication: the module grants the workload identity `Storage Blob Data Reader` scoped to `models/` and `Storage Blob Data Contributor` scoped to `payloads/`. No storage keys reach a pod; shared-access-key auth is disabled on the account entirely.

One network note: the storage account's ACL defaults to `Deny`, allowing only the cluster's subnets. To populate the cache from a laptop or CI runner (`sie-admin cache populate --target $(terraform output -raw model_cache_bucket_url)/`), add your egress IP to `storage_allowed_ip_ranges`.

---

## Differences from GCP and AWS

| Feature | GCP (GKE) | AWS (EKS) | Azure (AKS) |
|---------|-----------|-----------|-------------|
| GPU scheduling | Native GKE support | NVIDIA Device Plugin required | NVIDIA Device Plugin required (module installs it; driver preinstalled) |
| IAM for pods | Workload Identity | IRSA | Workload Identity (managed identity + federated credential) |
| Model cache storage | GCS (`gs://`) | S3 (`s3://`) | Blob Storage (`abfs://`) |
| Node provisioning | GKE Autopilot / NAP | Karpenter or Cluster Autoscaler | Built-in per-pool cluster autoscaler |
| Spot instances | Spot VMs | Spot Instances (2-min notice) | Spot VMSS (no eviction notice, extra `scalesetpriority` taint) |
| Entry-level GPU | L4 | L4 (`g6`) | T4 / A10 (no L4 SKU) |

---

## Security Considerations

The dev examples expose the AKS API server publicly (`allow_public_api_server = true`). For production:

- **Restrict the API server** with `api_server_authorized_ip_ranges`, or set `enable_private_cluster = true`
- **Enable authentication** via oauth2-proxy or static gateway tokens
- **Use a private load balancer** for internal-only access. The `azure-load-balancer-internal` annotation applies to a Service, so set it on the gateway Service (under `ingress.annotations` it has no effect and the endpoint stays public):

```yaml
gateway:
  service:
    type: LoadBalancer
    annotations:
      service.beta.kubernetes.io/azure-load-balancer-internal: "true"
```

- **Put ACR and Storage on Private Link** with `enable_private_endpoints = true`; their public endpoints are disabled entirely

Out of the box the module enforces AAD-RBAC (no local admin accounts in production mode), TLS 1.2 minimum on Storage and ACR, and NAT-gateway egress with predictable IPs.

---

## Standalone Docker on Azure

For simpler deployments, run standalone SIE on a GPU VM:

```bash
# On a Standard_NC4as_T4_v3 (NVIDIA T4) VM with the NVIDIA driver installed
sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker

docker run --gpus all -p 8080:8080 \
  -v ~/.cache/huggingface:/app/.cache/huggingface \
  ghcr.io/superlinked/sie-server:latest-cuda12-default
```

This is simpler than AKS and suitable for single-instance production workloads.

---

## What's Next

- [Upgrade Runbook](/docs/deployment/upgrades/) - pre-upgrade checklist, rolling updates, and rollback
- [Scale-from-Zero](/docs/deployment/autoscaling/) - understanding the 202 flow and cold starts
- [Monitoring](/docs/deployment/monitoring/) - metrics, alerts, and dashboards
- [Kubernetes in GCP](/docs/deployment/cloud-gcp/) - equivalent GKE deployment
- [Kubernetes in AWS](/docs/deployment/cloud-aws/) - equivalent EKS deployment