Skip to content
Most AI tasks no longer need a frontier model Read launch post

Swap an OCR model with one identifier change

OCR is rarely a single-model problem. A real pipeline has three concerns:

  1. Recognition (image to text). VLM-OCRs like PaddleOCR-VL, LightOnOCR or GLM-OCR take a whole document and emit Markdown.
  2. Structured extraction (image to JSON). End-to-end document models like Donut on CORD skip the text intermediate entirely and emit nested JSON directly.
  3. Zero-shot NER on the recognized text (text to typed fields). When you want to declare entity labels at query time instead of fine-tuning a new model.

This demo wires all three behind one SIE server. Pick a sample document on the left, swap any of the three models in the dropdowns, watch SIE hot-swap the underlying architecture with a single identifier change.

# Recognition: VLM-OCR (transformers profile, runs on CPU), returns Markdown
client.extract(
"PaddlePaddle/PaddleOCR-VL-1.5:transformers",
Item(images=[image_bytes]),
options={"task": "ocr"},
)
# Structured: end-to-end Donut, returns JSON tree
client.extract("naver-clova-ix/donut-base-finetuned-cord-v2", Item(images=[image_bytes]))
# NER: zero-shot, returns typed entities
client.extract(
"urchade/gliner_multi-v2.1",
Item(text=recognized_markdown),
labels=["merchant", "total", "date"],
)

Each panel in the UI has a “See the SIE call” disclosure that shows the exact line of code that just ran. Swap a dropdown, the snippet updates with the one parameter that changed.

You could build this with three SaaS APIs (one OCR provider, one document AI provider, one NER provider). It would work. It would also be three auth flows, three rate-limit budgets, three SDKs, and three deployment stories.

SIE collapses that into one process:

  • One server, three of the four primitives. encode, score, extract. This demo uses extract for all three model classes.
  • One SDK call. client.extract(model_id, item) works for VLM-OCR, end-to-end document AI, and zero-shot NER. Swap the model ID alone.
  • Open source, runs in your VPC. Customer documents never leave the host running the compose. Compliance teams stop blocking you.
  • Same code laptop to Kubernetes. SIE ships a Helm chart, KEDA autoscaling, and Terraform modules. The code in this demo runs unchanged against a production cluster; only the URL changes.

The fastest path is the hosted Hugging Face Space. It runs an earlier build of this demo on a free CPU, so its model lineup differs from the one below.

For local Docker (no API key, runs entirely on your machine), each sample click takes about 20 to 30 s on an 8-core x86_64 host. Apple Silicon runs the linux/amd64 image through Rosetta, so clicks are slower there:

git clone https://github.com/superlinked/sie
cd sie/examples/document-ocr
npm install
npm start

npm start runs docker compose up -d (boots ghcr.io/superlinked/sie-server:latest-cpu-default and preloads PaddleOCR-VL-1.5’s transformers profile, Donut on CORD, Donut on DocVQA and urchade/gliner_multi-v2.1), then starts a Node UI server and opens http://localhost:3032.

On Linux with an NVIDIA GPU, npm run start:gpu uses compose.gpu.yml and ghcr.io/superlinked/sie-server:latest-cuda12-sglang-vision-extract instead. It preloads LightOnOCR-2-1B and loads GLM-OCR on demand. The two do not fit on one GPU together, so switching between them reloads the model. Of the demo’s models, that image serves only these two. It also serves LightOnOCR-3-4B and PaddleOCR-VL-1.5’s default SGLang profile, which the demo does not use. The structured and NER stages need the CPU compose. Both scripts start the UI on port 3032, so stop the npm start UI (Ctrl+C) before running npm run start:gpu.

  • Recognition (default PaddlePaddle/PaddleOCR-VL-1.5:transformers): Paddle’s 0.9B VLM-OCR through its transformers profile, which runs on CPU. Produces Markdown directly. LightOnOCR-2-1B and GLM-OCR are the GPU alternates; they run only on the GPU compose, and the UI disables any model the running server does not list.
  • Structured extraction (default naver-clova-ix/donut-base-finetuned-cord-v2): fine-tuned for the CORD receipt schema. The alternate is Donut on DocVQA, the same architecture fine-tuned for visual question answering.
  • Zero-shot NER (default urchade/gliner_multi-v2.1): 280M, multilingual, declare labels at query time. Alternates include GLiNER large, GLiNER PII, and NuMind’s NuNER Zero (different architecture, same SDK call).
  • Click any sample (receipt, invoice, business-card, event-poster, slide, letter) and watch all three model classes run in one pipeline. The footer prints per-stage timings.
  • Swap the recognition dropdown (on the GPU compose, between LightOnOCR and GLM-OCR). The “See the SIE call” disclosure updates with the new model ID, nothing else changes.
  • Swap NER from gliner_multi-v2.1 to NuNER_Zero. Different model family entirely (NuMind vs urchade), same SDK call, same labels.
  • Compare two samples back to back: click receipt.png (Donut on CORD reads the subtotal, tax and total, because receipts are its fine-tuning distribution), then click letter.png (the opposite: Donut still emits a CORD-shaped tree, with the sender’s name as the total price; recognition + GLiNER carry the pipeline).
  • extract for all three model classes

The example lives in examples/document-ocr in the main SIE repo.

Talk to us

Tell us about your use case and we'll get back to you shortly.