---
title: TEI → SIE
description: Replace N single-model HuggingFace TEI containers with one SIE cluster. Same checkpoints, one process for dense, sparse, multivector, and rerank.
canonical_url: https://superlinked.com/docs/migrate/tei
last_updated: 2026-08-25
---

[Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference)
is HuggingFace's single-model embedding/reranking server. The migration
to SIE is the headline use case for platform engineers:
**N TEI containers → 1 SIE cluster.**

## Why migrate

- **Multi-model in one process.** SIE serves dense, sparse, multivector,
  rerank, and vision models from one cluster. TEI is one model per
  container, so N models means N deployments, N health checks, N
  autoscalers.
- **Query-time model selection.** Choose the model on every request.
  Adding a new model in TEI means redeploying a container; in SIE it
  means hot-loading via the config API.
- **Typed sparse + multivector outputs.** TEI exposes `/embed_sparse`
  and `/embed_all` as separate endpoints; outputs aren't typed. SIE
  has typed `dense`, `sparse`, and `multivector` outputs in one call.
- **Apple Silicon.** SIE runs on `--device mps` for local development.
  TEI's CPU image works on macOS but is significantly slower.

### "Why not just put N TEI containers behind one ingress?"

Fair question. For two or three stable models, that's a perfectly good
answer and you should not migrate. SIE earns its keep when you have:

- **Several models in active use**, where the per-container fixed
  overhead (RAM, sidecar tax, scrape targets, alert routes) starts to
  dominate.
- **A long tail of "sometimes" models**, like domain rerankers,
  language variants, experimental checkpoints. SIE's LRU lets you list
  them all in one bundle and load on demand. N TEIs means N standing
  pods or N scale-to-zero cold starts.
- **Mixed modalities in the same request path** (dense plus rerank,
  or dense plus sparse plus multivector). One round-trip to SIE
  replaces two or three to different TEI services with different DNS,
  different timeouts, different OTel spans.
- **Per-request model choice.** Route English to one model and
  multilingual to another. With TEI you build the routing layer
  yourself. With SIE the cluster does the routing.

If your shape is "two pinned models, both at high QPS, both pegging
their GPUs", N TEI behind ingress is simpler and you should keep it.

## What stays the same

- **Model checkpoints.** Every BERT / Sentence-Transformers / cross-encoder
  model TEI serves works in SIE on the same checkpoint, in the same
  vector space.

## Before

```bash
# One container per model
docker run -d -p 8088:80 \
  ghcr.io/huggingface/text-embeddings-inference:cpu-1.6 \
  --model-id BAAI/bge-small-en-v1.5

docker run -d -p 8089:80 \
  ghcr.io/huggingface/text-embeddings-inference:cpu-1.6 \
  --model-id BAAI/bge-reranker-v2-m3
```

#### Python

```python
import httpx

texts = ["The mitochondrion is the powerhouse of the cell."]
query = "What is the powerhouse of the cell?"
docs = [
    "Mitochondria are the powerhouse of the cell.",
    "The Eiffel Tower is in Paris.",
]

# Embed
embed = httpx.post("http://localhost:8088/embed",
                   json={"inputs": texts}).json()

# Rerank (different container)
rerank = httpx.post("http://localhost:8089/rerank",
                    json={"query": query, "texts": docs}).json()
```

#### TypeScript

```typescript
const texts = ["The mitochondrion is the powerhouse of the cell."];
const query = "What is the powerhouse of the cell?";
const docs = [
  "Mitochondria are the powerhouse of the cell.",
  "The Eiffel Tower is in Paris.",
];

// Embed
const embed = await fetch("http://localhost:8088/embed", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ inputs: texts }),
}).then(r => r.json());

// Rerank (different container)
const rerank = await fetch("http://localhost:8089/rerank", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ query, texts: docs }),
}).then(r => r.json());
```

## After

```bash
# One cluster, both models
docker run -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  -e SIE_PRELOAD_MODELS=BAAI/bge-small-en-v1.5,BAAI/bge-reranker-v2-m3 \
  ghcr.io/superlinked/sie-server:latest-cpu-default
# BAAI/bge-small-en-v1.5 is not in the default bundle.
# See /docs/engine/bundles/ to add it, or swap in a shipped encoder.
```

#### Python

```python
from sie_sdk import SIEClient
from sie_sdk.types import Item

client = SIEClient("http://localhost:8080")

texts = ["The mitochondrion is the powerhouse of the cell."]
query = "What is the powerhouse of the cell?"
docs = [
    "Mitochondria are the powerhouse of the cell.",
    "The Eiffel Tower is in Paris.",
]

# Embed
embed = client.encode(
    "BAAI/bge-small-en-v1.5",
    [Item(text=t) for t in texts],
)

# Rerank: same cluster, different model
rerank = client.score(
    "BAAI/bge-reranker-v2-m3",
    Item(text=query),
    [Item(text=d) for d in docs],
)
```

#### TypeScript

```typescript
import { SIEClient } from "@superlinked/sie-sdk";

const client = new SIEClient("http://localhost:8080");

const texts = ["The mitochondrion is the powerhouse of the cell."];
const query = "What is the powerhouse of the cell?";
const docs = [
  "Mitochondria are the powerhouse of the cell.",
  "The Eiffel Tower is in Paris.",
];

// Embed
const embed = await client.encode(
  "BAAI/bge-small-en-v1.5",
  texts.map(t => ({ text: t })),
);

// Rerank: same cluster, different model
const rerank = await client.score(
  "BAAI/bge-reranker-v2-m3",
  { text: query },
  docs.map(d => ({ text: d })),
);

await client.close();
```

## Mapping

| TEI                                            | SIE equivalent                                            |
|------------------------------------------------|-----------------------------------------------------------|
| `--model-id BAAI/bge-small-en-v1.5`            | bundle config + `docker run -e SIE_PRELOAD_MODELS=...`     |
| One container per model                        | One cluster, model selected per request                   |
| `POST /embed`                                  | `client.encode(model, items)`                             |
| `POST /rerank`                                 | `client.score(model, query, items)`                       |
| `POST /embed_sparse`                           | `client.encode(..., output_types=["sparse"])`             |
| `POST /embed_all` (multivector)                | `client.encode(..., output_types=["multivector"])`        |
| `--auto-truncate` / `--max-batch-tokens`       | Per-model in SIE bundle config                            |
| `/v1/embeddings` (OpenAI-compatible, optional) | `/v1/embeddings` on SIE (always-on)                       |
| `--dtype float16` / `bfloat16`                 | Per-model in adapter config                               |
| `/health`                                      | `/health` on the SIE gateway; workers expose `/healthz` and `/readyz` |
| `/metrics`                                     | No app-local `/metrics`; metrics export over OTLP to the bundled collector, which Prometheus scrapes; pre-built Grafana dashboards available |

## Sparse and multivector

SIE's typed outputs replace TEI's separate endpoints:

#### Python

```python
# Sparse (SPLADE)
sparse = client.encode("naver/splade-v3", item, output_types=["sparse"])
# sparse["sparse"] is a SparseResult TypedDict: ["indices"] and ["values"] (numpy arrays)

# Multivector (ColBERT)
mv = client.encode("jinaai/jina-colbert-v2", item, output_types=["multivector"])
# mv["multivector"] is np.ndarray of shape [n_tokens, dim]
```

#### TypeScript

```typescript
// Sparse (SPLADE)
const sparse = await client.encode("naver/splade-v3", item, { outputTypes: ["sparse"] });
// sparse.sparse is { indices: Int32Array, values: Float32Array }

// Multivector (ColBERT)
const mv = await client.encode("jinaai/jina-colbert-v2", item, { outputTypes: ["multivector"] });
// mv.multivector is Float32Array[] of shape [n_tokens][dim]
```

## Re-embed required?

**No** when you stay on the same checkpoint. Numerical drift between
TEI's backend (Candle / CTranslate2 / ONNX, depending on flags) and
SIE's PyTorch backend is small in practice; verify on a slice of your
own corpus before cutting over.

## Run it yourself

```bash
# Bring up TEI on a known checkpoint.
docker run -d -p 8088:80 \
  ghcr.io/huggingface/text-embeddings-inference:cpu-1.6 \
  --model-id sentence-transformers/all-MiniLM-L6-v2

# Bring up SIE with the same checkpoint and a reranker.
docker run -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  -e SIE_PRELOAD_MODELS=sentence-transformers/all-MiniLM-L6-v2,BAAI/bge-reranker-v2-m3 \
  ghcr.io/superlinked/sie-server:latest-cpu-default
```

Run the 'before' and 'after' snippets from this page against both.
On the same checkpoint, expect cosine at or above 0.999.