Fastembed → SIE

Fastembed is the Qdrant-maintained library that runs ONNX embedding models in-process via onnxruntime. SIE serves the same models out-of-process over HTTP. The Node port fastembed-js mirrors the same surface from TypeScript.

Why migrate

• Out-of-process serving. Every Python worker that imports Fastembed gets its own copy of the model in RAM. SIE loads weights once per worker pod regardless of how many app processes connect.
• Shared GPU. Fastembed is CPU-only by default (GPU support requires a separate ONNX runtime build). SIE serves on CPU, MPS, or CUDA without changing client code.
• Multi-model. SIE can serve dense, sparse, ColBERT, rerankers, and vision models from one cluster. Fastembed does not cover sparse or cross-encoder rerankers natively.

What stays the same

• Model checkpoint (e.g. BAAI/bge-small-en-v1.5).
• Vector dimension.
• Cosine semantics. Numerical drift between Fastembed (ONNX) and SIE (PyTorch) is small in practice; verify on a slice of your own corpus (see Run it yourself) before deciding whether an existing index can stay.

Before

Python

from fastembed import TextEmbedding

encoder = TextEmbedding(model_name="BAAI/bge-small-en-v1.5")
[vec] = list(encoder.embed(["The mitochondrion is the powerhouse of the cell."]))

TypeScript

import { EmbeddingModel, FlagEmbedding } from "fastembed";

const encoder = await FlagEmbedding.init({ model: EmbeddingModel.BGESmallEN15 });
const [vec] = await encoder.embed(["The mitochondrion is the powerhouse of the cell."]).next().then(r => r.value);

After

Python

from sie_sdk import SIEClient
from sie_sdk.types import Item

client = SIEClient("http://localhost:8080")
result = client.encode(
    "BAAI/bge-small-en-v1.5",
    Item(text="The mitochondrion is the powerhouse of the cell."),
)
vec = result["dense"]  # np.ndarray, shape [384]

TypeScript

import { SIEClient } from "@superlinked/sie-sdk";

const client = new SIEClient("http://localhost:8080");
const result = await client.encode(
  "BAAI/bge-small-en-v1.5",
  { text: "The mitochondrion is the powerhouse of the cell." },
);
const vec = result.dense; // Float32Array, length 384

await client.close();

Re-embed required?

No if you keep the same checkpoint and the drift check on a slice of your own corpus passes (see Run it yourself). Yes if that comparison shows an unacceptable retrieval regression, or if you take the migration as a chance to upgrade to a stronger model.

Run it yourself

sentence-transformers/all-MiniLM-L6-v2 is the common-denominator small model both Fastembed and SIE ship by default.

docker run -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  -e SIE_PRELOAD_MODELS=sentence-transformers/all-MiniLM-L6-v2 \
  ghcr.io/superlinked/sie-server:latest-cpu-default
uv add fastembed

Run the ‘before’ and ‘after’ snippets from this page. Expected: identical dim (384), cosine at or above 0.999.

Using BAAI/bge-small-en-v1.5 (or any other model)

Most Fastembed users actually run BAAI/bge-small-en-v1.5. SIE ships a model config for it at packages/sie_server/models/BAAI__bge-small-en-v1.5.yaml, but that config only defines a candle profile, so the cpu-default image cannot serve it. Migration mechanics are otherwise identical. Either run an image built with the candle bundle, or add a default profile with the sentence-transformer adapter to the existing YAML:

profiles:
  default:
    max_batch_tokens: 16384
    compute_precision: null
    adapter_path: sie_server.adapters.sentence_transformer:SentenceTransformerDenseAdapter
    adapter_options:
      loadtime:
        trust_remote_code: false
      runtime:
        pooling: cls
        normalize: true

Then sie-server serve -m BAAI/bge-small-en-v1.5. The sentence-transformers__all-MiniLM-L6-v2.yaml model config is the closest working reference; its default profile uses the same adapter.