Bundles
Why Bundles
Section titled “Why Bundles”Python ML libraries often have conflicting dependency requirements. Models using trust_remote_code=True or specialized backends can pin incompatible versions of transformers, torch, or sglang. SIE solves this with bundles: each bundle is a self-contained environment with compatible dependencies, built into its own Docker image.
Each bundle is a YAML file under packages/sie_server/bundles/ that lists the adapters it enables and the pinned dependency versions needed by those adapters. At build time, the Dockerfile selects one bundle via the BUNDLE build arg and installs only that bundle’s deps.
Published Bundles
Section titled “Published Bundles”Seven bundles are published to GHCR as sie-server images. The default bundle covers every model unless it needs SGLang, transformers>=5, or another dedicated engine.
| Bundle | Purpose | Key Models |
|---|---|---|
default | Everything that runs on standard transformers + Flash Attention | BGE-M3, E5, Stella, Qwen3-Embedding 0.6B and 8B, GritLM, NV-Embed, gte-Qwen2-7B, llama-embed-nemotron-8b, ColBERT, ColPali, ColQwen2, GLiNER, GLiREL, GLiClass, Florence-2, Donut, CLIP, SigLIP, Grounding DINO, OwlV2, SPLADE, Whisper, and more |
sglang | Text generation served through the SGLang runtime | Qwen3, Qwen3.5-4B and Qwen3.6-27B generation, Qwen3Guard, granite-guardian-3.0-2b, sqlcoder-7b-2, Hy-MT2-1.8B |
sglang-vision-extract | Document OCR VLMs served through SGLang | LightOnOCR-2-1B, LightOnOCR-3-4B, GLM-OCR, PaddleOCR-VL-1.5 |
sglang-cu130 | SGLang 0.5.20 on CUDA 13, for models the sglang pin cannot load | Gemma 4, Qwen3.6-35B-A3B, Qwen3.8-27B-FP8, Qwen3.5-122B-A10B-FP8, GLM-5.3-Flash |
transformers5 | Adapters that require transformers>=5 | SAM 3, Parakeet-TDT, GLiNER2.5, GLiGuard, the privacy filters, TopK-Embed, visual SPLADE |
ctranslate2 | Translation on the CTranslate2 runtime | MADLAD-400-3B |
tensorrt-llm | TensorRT-LLM runtime for T5, BART and mBART-family encoder-decoders on CUDA 13 | No catalog model uses it yet |
Key Models lists where each model’s default profile runs. A model’s other profiles can need another image: Qwen3.6-27B runs on sglang by default, while its long-context and thinking profiles run on sglang-cu130. Qwen3.6-35B-A3B is the reverse. Each profile’s adapter, and so its bundle, is in the model’s YAML under packages/sie_server/models/.
The previous
glinerandflorence2bundles no longer exist; their adapters and dependencies were folded intodefaultonce the underlying version conflicts were resolved. The CUDA 13gemmabundle was renamedsglang-cu130.
The repo’s bundles/ directory holds eleven bundle YAML files. The other four have no sie-server image of their own:
sglang-embeddingserves large LLM embeddings on thesglangimage. Start that image withSIE_BUNDLE=sglang-embedding, or setimageBundle: sglangon the bundle lane in the Helm chart.remoteforwards requests to an upstream API. The Helm chart runs it on thetransformers5image.candleis the Rust engine. Its worker ships as a separatesie-server-rustimage, built for L4 GPUs (cuda12-sm89), which the Helm chart runs forengine: candlelanes.fakeserves deterministic weightless models for tests.
Bundle Contents
Section titled “Bundle Contents”default
Section titled “default”The default bundle is the broad, general-purpose image. It bundles the standard transformers, sentence-transformers, Flash Attention, and the NER/vision adapters.
Included adapter families:
- Dense encoders: BERT (flash), ModernBERT (flash), BGE-M3, Qwen2, XLM-RoBERTa, Nomic, GTE, Stella, sentence-transformers, PyTorch embedding
- Cross-encoders / rerankers: BERT, ModernBERT, Qwen2, Jina (flash), NLI classification
- Multi-vector / late-interaction: ColBERT, ColBERT + ModernBERT, ColBERT + rotary, ColPali, ColQwen2, NeMo ColEmbed
- Sparse: SPLADE (flash), GTE sparse (flash)
- Vision and vision-language: CLIP, SigLIP, Grounding DINO, OwlV2, Florence-2, Donut
- Zero-shot NER / extraction: GLiNER, GLiREL, GLiClass
sglang
Section titled “sglang”The SGLang generation bundle. It enables the SGLang generation adapter with pinned SGLang, grammar-backend, and kernel dependencies for serving text-generation models.
sglang-embedding
Section titled “sglang-embedding”A companion bundle wired to the SGLang embedding adapter for large LLM embeddings (4B+ params). It shares the sglang dependency stack, so it runs on the sglang image; the split exists for worker pool isolation.
Included models:
Qwen/Qwen3-Embedding-4Bintfloat/e5-mistral-7b-instructLinq-AI-Research/Linq-Embed-MistralSalesforce/SFR-Embedding-Mistral,Salesforce/SFR-Embedding-2_R
sglang-vision-extract
Section titled “sglang-vision-extract”Serves document OCR models through SGLang. It pins the same SGLang stack as sglang but ships as its own CUDA 12 image. LightOnOCR, GLM-OCR and PaddleOCR-VL run here by default; each also has a transformers profile that runs on transformers5 (LightOnOCR, GLM-OCR) or default (PaddleOCR-VL).
sglang-cu130
Section titled “sglang-cu130”SGLang 0.5.20 on CUDA 13 with torch cu130. It holds the models that need a newer engine than the sglang pin: Gemma 4, Qwen3.8-27B-FP8, Qwen3.5-122B-A10B-FP8, GLM-5.3-Flash, the default and h100-fp8 profiles of Qwen3.6-35B-A3B, and the long-context and thinking profiles of Qwen3.6-27B. It ships only as a cuda13 image.
transformers5
Section titled “transformers5”Published to GHCR for adapters that require transformers>=5: SAM 3 detection, Parakeet-TDT speech-to-text, GLiNER2.5 entities and decisions, GLiGuard (GLiNER2 classification), the OpenAI and GLiNER2 privacy filters, TopK-Embed, and the sparse vision (multimodal SparseEncoder) adapter. The original GLiNER2 base and large models stay on default.
ctranslate2
Section titled “ctranslate2”The CTranslate2 generation adapter, pinned to ctranslate2==4.8.1. MADLAD-400-3B translation runs here.
tensorrt-llm
Section titled “tensorrt-llm”A CUDA 13 TensorRT-LLM lane for direct-from-Hugging-Face T5, BART and mBART-family encoder-decoders. No catalog model uses it yet.
Docker Images
Section titled “Docker Images”default, sglang, transformers5 and ctranslate2 ship for both cpu and cuda12. sglang-vision-extract ships for cuda12 only, and sglang-cu130 and tensorrt-llm for cuda13 only. The image tag format is {version}-{platform}-{bundle}, with a floating latest-{platform}-{bundle} tag that tracks the most recent release.
The cpu-sglang image cannot serve text generation. Off CUDA, the generation adapter switches to the Apple Silicon MLX backend, which the Linux image does not include. Run generation models on a CUDA image.
# Default bundle (CPU)docker run -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cpu-default
# Default bundle (CUDA 12, recommended for GPU)docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-default
# SGLang bundle (CUDA 12)docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-sglang
# SGLang embedding models, on the SGLang imagedocker run --gpus all -p 8080:8080 -e SIE_BUNDLE=sglang-embedding ghcr.io/superlinked/sie-server:latest-cuda12-sglang
# Document OCR VLMs (CUDA 12)docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-sglang-vision-extract
# Transformers 5 bundle (CUDA 12)docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-transformers5
# SGLang on CUDA 13docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda13-sglang-cu130
# Pin to a specific releasedocker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:v0.10.1-cuda12-defaultBundle Selection
Section titled “Bundle Selection”Choose a bundle based on the models you need. Each page in the model catalog shows the docker run command for the image that serves that model.
- Start with
default. It covers dense, sparse, multi-vector, cross-encoder, vision, and extraction models, which is the overwhelming majority of use cases. - Use
sglangwhen you need text generation through the SGLang runtime. Run it as a second container alongsidedefaultand route requests by model name. Large LLM embedders run on the same image withSIE_BUNDLE=sglang-embedding. - Use
sglang-vision-extractfor LightOnOCR, GLM-OCR and PaddleOCR-VL. - Use
sglang-cu130for Gemma 4 and the other CUDA 13 models. It needs a host driver that supports CUDA 13. - Use
transformers5for models that needtransformers>=5: SAM 3, Parakeet, GLiNER2.5, GLiGuard, and visual SPLADE. - Use
ctranslate2for MADLAD-400 translation.
Models are loaded on first request. The bundle only determines which models are available inside a given image.
What’s Next
Section titled “What’s Next”- Model Catalog - complete list of supported models
- Docker Deployment - tags, GPU configuration, and Docker Compose
- Deployment Overview - from single container to Kubernetes