--- title: What is SIE? description: SIE (Superlinked Inference Engine) is an open-source inference server for AI models. It runs encoders, rerankers, extractors, and generation models on your own GPU, serving 100+ models through four simple API primitives. canonical_url: https://superlinked.com/docs last_updated: 2026-08-08 --- **SIE (Superlinked Inference Engine) is an open-source inference server for AI models.** It runs encoders, rerankers, entity extractors, and generation models on your own infrastructure, from a laptop to a production Kubernetes cluster, without managing per-model deployments or paying per-token API costs. SIE exposes four primitives: - **[Encode](https://superlinked.com/docs/encode/)** converts text or images to vectors for semantic search and RAG - **[Score](https://superlinked.com/docs/score/)** reranks query-document pairs for higher-precision retrieval - **[Extract](https://superlinked.com/docs/extract/)** pulls entities and structured data from unstructured text - **[Generate](https://superlinked.com/docs/generate/)** runs text generation on open LLMs you host yourself 100+ models are supported out of the box. The server handles batching, GPU sharing, and model switching automatically. Browse the full [model catalog](https://superlinked.com/models). > SIE is built by [Superlinked](https://superlinked.com/), the team behind the Superlinked vector compute framework. [Read the launch post](https://superlinked.com/blog/launch). --- ## Get Started | I want to... | Go to | | --- | --- | | **Get my first vectors in 2 minutes** | [Quickstart](https://superlinked.com/docs/quickstart/) | | **Embed text or images** | [Encode Overview](https://superlinked.com/docs/encode/) | | **Rerank search results** | [Score Overview](https://superlinked.com/docs/score/) | | **Extract entities from text** | [Extract Overview](https://superlinked.com/docs/extract/) | | **Generate text with a small LLM** | [Generate Overview](https://superlinked.com/docs/generate/) | | **Choose the right model** | [Model Selection Guide](https://superlinked.com/docs/choosing/) | | **See all 100+ models** | [Model Catalog](https://superlinked.com/models) | | **Deploy to production** | [Deployment Overview](https://superlinked.com/docs/deployment/) | | **Connect to LangChain or LlamaIndex** | [Integrations](https://superlinked.com/docs/integrations/) | --- ## Why Does SIE Exist? LLM inference tools are designed for one large model spread across many GPUs. Small model inference is the opposite problem: you run many models (encoders, rerankers, extractors) on one GPU and need fast switching between them. **What makes SIE different from other inference servers:** 1. **Compute engine abstraction.** SIE wraps PyTorch, SGLang, Flash Attention, and Apple MLX behind four uniform primitives. The server picks the best engine per model automatically. 2. **Multi-model GPU sharing.** Many models can share one GPU via LRU eviction. One SIE instance serves any model at query time without pre-loading everything. 3. **Same code, laptop to cloud.** The same Docker image runs locally and in a production Kubernetes cluster. There is no separate production mode. 4. **Validated correctness.** Every supported model has quality and latency targets checked in CI. --- ## How Does SIE Compare to Alternatives? | | SIE | TEI (HuggingFace) | OpenAI API | |---|---|---|---| | Self-hosted | Yes | Yes | No | | Multi-model on one GPU | Yes | No (one model per server) | N/A | | Encode + Score + Extract + Generate | Yes | Encode only | Encode + generate only | | 100+ supported models | Yes | Varies | Limited | | Open source | Yes | Yes | No | | No per-token cost | Yes | Yes | No | See the [retrieval strategy benchmark](https://superlinked.com/docs/examples/benchmark/) for a worked evaluation of eight retrieval strategies on SIE with measured NDCG@10 numbers. --- ## Frequently Asked Questions **What is SIE used for?** SIE is used to generate embeddings for semantic search and RAG pipelines, rerank search results to improve precision, extract entities from unstructured text, and run text generation on small self-hosted LLMs. All of this runs on your own infrastructure. See [superlinked.com](https://superlinked.com/) for more on what you can build. **Does SIE support GPU inference?** Yes. SIE runs on CPU or GPU. For production inference at scale, a GPU is strongly recommended. See [Hardware and Capacity](https://superlinked.com/docs/deployment/resources/) for GPU sizing guidance. **How many models can SIE run at the same time?** SIE loads models on demand and evicts the least-recently-used models when GPU memory fills up. An L4 GPU (24GB) holds several standard models resident at once; the exact count depends on model size and batch settings. The full catalog stays addressable at query time: models load on demand, and each requested model still has to fit in the GPU's memory to load. **Is SIE open source?** Yes. SIE is open source and available on [GitHub](https://github.com/superlinked/sie). The core inference server is free to use. Superlinked also offers managed cloud deployment. [Contact us](https://superlinked.com/) to learn more. **How is SIE different from the Superlinked framework?** The [Superlinked framework](https://superlinked.com/) is a higher-level Python SDK for building multi-attribute search and recommendation systems. SIE is the inference layer underneath it. You can use SIE standalone or as part of a full Superlinked stack.