SIE vs vLLM: Which Should You Deploy for Multi-Model Workloads?

Short version: this is the wrong question asked the right way. vLLM and SIE (Superlinked Inference Engine) are not competing for the same slot. vLLM spreads one large generative model across GPUs for token throughput; SIE packs many small models onto shared GPUs behind one API. Touch both and you deploy both. If every model in your pipeline is under a few billion parameters, you may not need vLLM at all.

The useful version of the question is: which layer does my workload actually live in.

SIE is open source and free under Apache 2.0. Stop reading and start cutting per-token costs whenever you like: the engine runs on your own hardware, from a laptop to a Kubernetes cluster.

★ Star the repo on GitHub Run the 2-minute quickstart →

vLLM and SIE solve inverse problems

vLLM is an LLM serving engine. Its job is to take one large generative model and get as many tokens per second out of a GPU or a group of GPUs as the hardware allows, through continuous batching, paged attention and tensor parallelism. Everything in its design assumes the model is the fixed thing and the traffic flows through it.

SIE assumes the opposite. The models are the variable thing: dozens of them, mostly in the 100M to 4B range, each request short, and the expensive part is switching between them without leaving GPUs idle. It serves them through four operations, encode, score, extract and generate, with models loading on demand and sharing GPU memory through least-recently-used eviction.

Put plainly: vLLM parallelizes one model across many GPUs. SIE packs many models onto few GPUs. Those are inverse optimizations, and neither one degrades gracefully into the other.

They are already complementary, in three concrete ways

This is not diplomatic hedging. There are specific mechanisms.

SIE selects SGLang as one of its compute backends. SIE runs its own backends written in PyTorch and Candle in Rust, and it also selects SGLang automatically for models where SGLang is the best serving path, sweeping parameters per model to pick it (engine docs). SGLang is vLLM’s closest peer in the generative serving tier. So choosing SIE does not mean giving up that class of runtime performance, it means not having to wire it up per model yourself.

A real agent shows the shape of the split. Take the contract review agent built on the OpenAI Agents SDK: one orchestrator driving six tools across nine models. A granite-guardian model screens input. A Qwen3-4B investigator calls tools in any order: Qwen3-0.6B to classify the document, GLiNER to extract entities, LightOnOCR plus Qwen3.5-4B to read a scanned signature page, BAAI/bge-m3 plus a Qwen3 reranker to search clauses, a Qwen3-4B reasoning sub-agent for clause risk, and sqlcoder for text-to-SQL over the obligations database. A final Qwen3-4B synthesizer emits a schema-valid review.

Nine models, one agent, and every model in it is small. That whole workload runs on one SIE cluster with no vLLM in the picture, because nothing in it needs a 120B model. Run against CUAD, a set of 500-plus real commercial contracts filed with the SEC, it caught a contract that wrote the same late-delivery penalty as both 30% and 5%, flagged a force-majeure clause missing its required certificate, and marked execution uncertain rather than inventing a signature when a scan was unreadable (Infer() Summit writeup).

The moment a frontier model enters, vLLM has a job. Swap that Qwen3-4B synthesizer for a large open model because the reasoning genuinely needs it, and now you have a generative tier with different serving requirements: long context, KV cache pressure, tokens streaming to a user. That is vLLM’s problem, and SIE keeps serving the eight small models around it. One agent, two control planes, talking over HTTP.

What breaks when you force either one to do the other’s job

Both failure modes are common enough to be worth naming.

Running one vLLM instance per encoder gives you a GPU per model, held whether or not traffic arrives. Under bursty traffic that layout averages roughly 21% utilization; you are mostly paying for idle. No eviction, so an untouched model still owns its VRAM. And you still do not get encode, score or extract as operations, so every reranker and extractor call gets shaped into a generation call or handled by a separate service you also maintain.

Going the other way, using SIE where you need a large generative model, does not work either. It is not a drop-in replacement for vLLM and it is not trying to be. If your requirement is one very large model saturating a multi-GPU node with long-context streaming, deploy vLLM, or put a fleet-scale control plane above it. That comparison is in llm-d vs Dynamo vs SIE.

At a glance

How the split looks in code

The practical version is that small-model work goes through one client and generation goes wherever you serve it. Nothing clever is required at the boundary. Sketch of the split (setup omitted):

from sie_sdk import SIEClient
from sie_sdk.types import Item

client = SIEClient("http://localhost:8080")

# Embedding, reranking and extraction on SIE
vectors = client.encode("BAAI/bge-m3", Item(text=user_query))
ranked = client.score(
    "BAAI/bge-reranker-v2-m3",
    Item(text=user_query),
    [Item(text=c) for c in candidates],
)
fields = client.extract(
    "urchade/gliner_multi-v2.1",
    Item(text=document),
    labels=["party", "date", "amount"],
)

# Generation on whichever engine serves your large model
answer = llm.complete(prompt=build_prompt(ranked, fields))

If the generative model is also small, that last line can be a generate call on the same client and the second deployment disappears.

Where to deploy each

SIE has three deployment modes, and the same engine and model IDs run in all of them, so the choice is operational rather than architectural (deployment docs).

Managed on SIE Cloud, where Superlinked runs the cluster and you get an API key, no idle GPUs and no ops. Self-hosted on Kubernetes, with Terraform modules for AWS and GCP and Helm for AKS, scaling to zero between jobs and running per-tenant pools. Or locally on an NVIDIA GPU or Apple Silicon with one command, using the same SDK and model IDs so nothing changes when you move up.

vLLM you deploy yourself alongside it, or under a fleet-scale control plane if the generative tier is large enough to need one.

FAQ

Is SIE a drop-in replacement for vLLM? No. vLLM serves generative LLMs, SIE serves the models around them, and SIE selects SGLang internally for some models rather than competing with that tier.

So which do I deploy for a multi-model workload? Count models and serving shape together. Small specialist fleets with short requests usually fit SIE alone. A large or long-context generative tier with streaming or high concurrency usually needs vLLM; mixed pipelines often run both.

Does running both mean two sets of GPUs? Usually yes, and that is the point. The two tiers want different hardware profiles and different scaling behavior; sharing GPUs between them causes the interference you are trying to avoid.

Can I A/B test two encoders without a second deployment? Yes. Name the model per request against one endpoint. That is the specific cost that one-model-per-instance serving imposes and that model packing removes.

Does SIE compete with SGLang? It uses it. SGLang is one of the backends SIE selects from automatically, so you get its performance without wiring it to each model.

Star the repo on GitHub · Read the engine docs · Browse the model catalog