---
title: "PDF-to-Markdown is easy to demo and hard to trust. Here's how to test it."
description: "Walk through the SIE document-to-markdown example: a PDF-to-Markdown pipeline with a deterministic evaluation harness attached."
canonical_url: https://superlinked.com/blog/document-to-markdown
last_updated: 2026-08-10
---

*This article walks through [document-to-markdown](https://github.com/superlinked/sie/tree/main/examples/document-to-markdown), a workable example from the open-source [SIE repo](https://github.com/superlinked/sie) on GitHub: a PDF-to-Markdown pipeline with a deterministic evaluation harness attached. SIE is Superlinked's self-hosted inference engine, one cluster that serves embedding, reranking, OCR, vision, entity-extraction, and generation models behind three primitives (extract, encode, score). The example is a complete project; four commands take you from fetching the PDFs to a graded eval. Built by Superlinked.*

Every PDF conversion library has the same demo: feed in a clean page, get out clean Markdown, ship it. Then a real document arrives. A financial report with nested tables. A two-column investor deck. A government form where the labels and the fields are only related by position on the page. The conversion still runs, still produces Markdown, and somewhere in the middle a table has silently lost a column.

Silent is the problem. Nobody reviews conversion output line by line, so the corruption flows straight into your RAG index or your data pipeline, and you find out weeks later when an answer cites a number that doesn't exist in the source.

The document-to-markdown example in the SIE repo treats conversion as something you verify, not something you trust. It converts four genuinely difficult public PDFs through SIE's `docling` model and then runs 25 deterministic checks against the output: exact facts that must appear, section order where linear order applies, and Markdown tables that must survive intact.

## The test corpus is the point

The four documents were picked to break naive converters in four different ways.

NVIDIA's Q4 FY2025 CFO commentary is dense financial prose packed with figures where a single transposed number changes the meaning. The SiriusPoint investor presentation is a designed slide deck, which means layout-driven reading order that plain text extraction scrambles. The Docling research paper is academic two-column formatting with references and figures. And the FEMA form is the hardest category of all: form labels and values whose relationship is purely spatial.

Results, from the actual eval run: NVIDIA passed 6/6 checks in 50.3 seconds, SiriusPoint 6/6 in 16.3s, the Docling paper 6/6 in 11.7s, and the FEMA form 7/7 in 10.0s. All 25 checks green, with per-document timing you can compare against your own hardware.

## What the pipeline looks like

The conversion itself is one call:

```python
result = client.extract(
    "docling",
    Item(document=Path("data/pdfs/nvidia-q4-fy2025-cfo-commentary.pdf")),
)
markdown = result["data"]["markdown"]
```

That's the whole integration surface. `docling` runs behind SIE's extract endpoint, so the same client call works against a local container or the SIE Cloud API; you point it at whichever with environment variables.

The full workflow is four commands:

```bash
cd examples/document-to-markdown
uv sync
uv run fetch-documents
uv run convert-documents --run-id local
uv run eval-documents runs/local
```

Fetch pulls the four public PDFs, convert produces the Markdown, eval runs the 25 checks and reports pass/fail per document. Because the checks are deterministic (string presence, order assertions, table structure), a regression in the extraction model or your deployment shows up as a red check, not as a vibe.

## Why deterministic checks beat eyeballing

The alternative to this harness is what most teams actually do: convert a few documents, scroll through the output, and declare it fine. That review catches formatting weirdness and misses the failures that matter. A table that dropped its rightmost column still looks like a table. A section that moved still reads fluently. Your eyes verify plausibility; they don't verify facts.

Deterministic checks invert the economics. Writing "the string '\$39.3 billion' must appear" takes thirty seconds and catches a transposed revenue figure forever, on every future run, at zero marginal cost. The 25 checks in this repo took someone maybe an hour to write, and they now guard four documents against every model upgrade, dependency bump, and infrastructure change from here on. Manual review guards exactly one conversion, once.

There's a second benefit that only shows up over time: the checks document what your pipeline actually depends on. Six months from now, when someone asks whether it's safe to swap extraction models, the answer is a command, and the harness re-grades the converted output without redoing the conversion.

## Honest about the edges

The README states its own boundary: this approach covers standard digital documents well, and encrypted files, handwriting, or diagram-heavy pages may need an OCR or vision model instead. That candor is useful. It tells you exactly when to stay on the docling path and when to reach for a different extract model, which in SIE is an identifier change rather than a new integration.

## Steal the harness, not just the converter

The conversion quality is good; the evaluation pattern is the transferable asset. Swap in your own PDFs, write ten checks for the facts your pipeline actually depends on (the revenue figure, the section ordering, the table that feeds your ETL), and you have a regression suite for document extraction. Run it whenever you upgrade a model or change infrastructure. Document conversion stops being a leap of faith and becomes a tested dependency, which is what it should have been all along.

Four difficult PDFs, 25 checks, about 88 seconds of sequential SIE conversion time on the recorded L4 run before file and eval overhead. Clone it and run the eval before you believe any converter's demo, including this one.

**Try it on GitHub:** [superlinked/sie/examples/document-to-markdown](https://github.com/superlinked/sie/tree/main/examples/document-to-markdown)