Skip to content
Most AI tasks no longer need a frontier model Read launch post

Bundles

Python ML libraries often have conflicting dependency requirements. Models using trust_remote_code=True or specialized backends can pin incompatible versions of transformers, torch, or sglang. SIE solves this with bundles: each bundle is a self-contained environment with compatible dependencies, built into its own Docker image.

Each bundle is a YAML file under packages/sie_server/bundles/ that lists the adapters it enables and the pinned dependency versions needed by those adapters. At build time, the Dockerfile selects one bundle via the BUNDLE build arg and installs only that bundle’s deps.


Seven bundles are published to GHCR as sie-server images. The default bundle covers every model unless it needs SGLang, transformers>=5, or another dedicated engine.

BundlePurposeKey Models
defaultEverything that runs on standard transformers + Flash AttentionBGE-M3, E5, Stella, Qwen3-Embedding 0.6B and 8B, GritLM, NV-Embed, gte-Qwen2-7B, llama-embed-nemotron-8b, ColBERT, ColPali, ColQwen2, GLiNER, GLiREL, GLiClass, Florence-2, Donut, CLIP, SigLIP, Grounding DINO, OwlV2, SPLADE, Whisper, and more
sglangText generation served through the SGLang runtimeQwen3, Qwen3.5-4B and Qwen3.6-27B generation, Qwen3Guard, granite-guardian-3.0-2b, sqlcoder-7b-2, Hy-MT2-1.8B
sglang-vision-extractDocument OCR VLMs served through SGLangLightOnOCR-2-1B, LightOnOCR-3-4B, GLM-OCR, PaddleOCR-VL-1.5
sglang-cu130SGLang 0.5.20 on CUDA 13, for models the sglang pin cannot loadGemma 4, Qwen3.6-35B-A3B, Qwen3.8-27B-FP8, Qwen3.5-122B-A10B-FP8, GLM-5.3-Flash
transformers5Adapters that require transformers>=5SAM 3, Parakeet-TDT, GLiNER2.5, GLiGuard, the privacy filters, TopK-Embed, visual SPLADE
ctranslate2Translation on the CTranslate2 runtimeMADLAD-400-3B
tensorrt-llmTensorRT-LLM runtime for T5, BART and mBART-family encoder-decoders on CUDA 13No catalog model uses it yet

Key Models lists where each model’s default profile runs. A model’s other profiles can need another image: Qwen3.6-27B runs on sglang by default, while its long-context and thinking profiles run on sglang-cu130. Qwen3.6-35B-A3B is the reverse. Each profile’s adapter, and so its bundle, is in the model’s YAML under packages/sie_server/models/.

The previous gliner and florence2 bundles no longer exist; their adapters and dependencies were folded into default once the underlying version conflicts were resolved. The CUDA 13 gemma bundle was renamed sglang-cu130.

The repo’s bundles/ directory holds eleven bundle YAML files. The other four have no sie-server image of their own:

  • sglang-embedding serves large LLM embeddings on the sglang image. Start that image with SIE_BUNDLE=sglang-embedding, or set imageBundle: sglang on the bundle lane in the Helm chart.
  • remote forwards requests to an upstream API. The Helm chart runs it on the transformers5 image.
  • candle is the Rust engine. Its worker ships as a separate sie-server-rust image, built for L4 GPUs (cuda12-sm89), which the Helm chart runs for engine: candle lanes.
  • fake serves deterministic weightless models for tests.

The default bundle is the broad, general-purpose image. It bundles the standard transformers, sentence-transformers, Flash Attention, and the NER/vision adapters.

Included adapter families:

  • Dense encoders: BERT (flash), ModernBERT (flash), BGE-M3, Qwen2, XLM-RoBERTa, Nomic, GTE, Stella, sentence-transformers, PyTorch embedding
  • Cross-encoders / rerankers: BERT, ModernBERT, Qwen2, Jina (flash), NLI classification
  • Multi-vector / late-interaction: ColBERT, ColBERT + ModernBERT, ColBERT + rotary, ColPali, ColQwen2, NeMo ColEmbed
  • Sparse: SPLADE (flash), GTE sparse (flash)
  • Vision and vision-language: CLIP, SigLIP, Grounding DINO, OwlV2, Florence-2, Donut
  • Zero-shot NER / extraction: GLiNER, GLiREL, GLiClass

The SGLang generation bundle. It enables the SGLang generation adapter with pinned SGLang, grammar-backend, and kernel dependencies for serving text-generation models.

A companion bundle wired to the SGLang embedding adapter for large LLM embeddings (4B+ params). It shares the sglang dependency stack, so it runs on the sglang image; the split exists for worker pool isolation.

Included models:

  • Qwen/Qwen3-Embedding-4B
  • intfloat/e5-mistral-7b-instruct
  • Linq-AI-Research/Linq-Embed-Mistral
  • Salesforce/SFR-Embedding-Mistral, Salesforce/SFR-Embedding-2_R

Serves document OCR models through SGLang. It pins the same SGLang stack as sglang but ships as its own CUDA 12 image. LightOnOCR, GLM-OCR and PaddleOCR-VL run here by default; each also has a transformers profile that runs on transformers5 (LightOnOCR, GLM-OCR) or default (PaddleOCR-VL).

SGLang 0.5.20 on CUDA 13 with torch cu130. It holds the models that need a newer engine than the sglang pin: Gemma 4, Qwen3.8-27B-FP8, Qwen3.5-122B-A10B-FP8, GLM-5.3-Flash, the default and h100-fp8 profiles of Qwen3.6-35B-A3B, and the long-context and thinking profiles of Qwen3.6-27B. It ships only as a cuda13 image.

Published to GHCR for adapters that require transformers>=5: SAM 3 detection, Parakeet-TDT speech-to-text, GLiNER2.5 entities and decisions, GLiGuard (GLiNER2 classification), the OpenAI and GLiNER2 privacy filters, TopK-Embed, and the sparse vision (multimodal SparseEncoder) adapter. The original GLiNER2 base and large models stay on default.

The CTranslate2 generation adapter, pinned to ctranslate2==4.8.1. MADLAD-400-3B translation runs here.

A CUDA 13 TensorRT-LLM lane for direct-from-Hugging-Face T5, BART and mBART-family encoder-decoders. No catalog model uses it yet.


default, sglang, transformers5 and ctranslate2 ship for both cpu and cuda12. sglang-vision-extract ships for cuda12 only, and sglang-cu130 and tensorrt-llm for cuda13 only. The image tag format is {version}-{platform}-{bundle}, with a floating latest-{platform}-{bundle} tag that tracks the most recent release.

The cpu-sglang image cannot serve text generation. Off CUDA, the generation adapter switches to the Apple Silicon MLX backend, which the Linux image does not include. Run generation models on a CUDA image.

# Default bundle (CPU)
docker run -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cpu-default
# Default bundle (CUDA 12, recommended for GPU)
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-default
# SGLang bundle (CUDA 12)
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-sglang
# SGLang embedding models, on the SGLang image
docker run --gpus all -p 8080:8080 -e SIE_BUNDLE=sglang-embedding ghcr.io/superlinked/sie-server:latest-cuda12-sglang
# Document OCR VLMs (CUDA 12)
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-sglang-vision-extract
# Transformers 5 bundle (CUDA 12)
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-transformers5
# SGLang on CUDA 13
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda13-sglang-cu130
# Pin to a specific release
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:v0.10.1-cuda12-default

Choose a bundle based on the models you need. Each page in the model catalog shows the docker run command for the image that serves that model.

  1. Start with default. It covers dense, sparse, multi-vector, cross-encoder, vision, and extraction models, which is the overwhelming majority of use cases.
  2. Use sglang when you need text generation through the SGLang runtime. Run it as a second container alongside default and route requests by model name. Large LLM embedders run on the same image with SIE_BUNDLE=sglang-embedding.
  3. Use sglang-vision-extract for LightOnOCR, GLM-OCR and PaddleOCR-VL.
  4. Use sglang-cu130 for Gemma 4 and the other CUDA 13 models. It needs a host driver that supports CUDA 13.
  5. Use transformers5 for models that need transformers>=5: SAM 3, Parakeet, GLiNER2.5, GLiGuard, and visual SPLADE.
  6. Use ctranslate2 for MADLAD-400 translation.

Models are loaded on first request. The bundle only determines which models are available inside a given image.


Talk to us

Tell us about your use case and we'll get back to you shortly.