---
title: "One month late, payment recouped: reproducing a CMS finding with five models"
description: "Walk through the SIE prior-authorization-review-agent example: a five-model pipeline that reproduces a published CMS documentation finding with a validated evidence chain."
canonical_url: https://superlinked.com/blog/prior-authorization-review-agent
last_updated: 2026-08-10
---

*This article walks through [prior-authorization-review-agent](https://github.com/superlinked/sie/tree/main/examples/prior-authorization-review-agent), a workable example from the open-source [SIE repo](https://github.com/superlinked/sie) on GitHub: a five-model pipeline that reproduces a published CMS documentation finding with a validated evidence chain. SIE is Superlinked's self-hosted inference engine, one cluster that serves embedding, reranking, OCR, vision, entity-extraction, and generation models behind three primitives (extract, encode, score). The example is a complete project run entirely on the public CMS record; no patient data is involved. Built by Superlinked.*

CMS published a documentation example that turns on a single month. A supplier billed L1851, a lower limb orthosis code. The coverage rule requires a face-to-face encounter within six months before delivery. The encounter in the record happened seven months before delivery. Result: insufficient documentation, payment recouped.

Six months required, seven months actual. The gap is one month, the reasoning is published, and the outcome is official. That makes this case something rare in healthcare AI: a ground-truth documentation review you can test against without touching patient data or guessing what the right answer should have been.

The prior-authorization-review-agent example in the SIE repo reproduces CMS's finding end to end, and the way it gets there matters as much as the answer.

## Five models, three source groups, zero shortcuts

The pipeline chains five models on one SIE deployment, each stage consuming the previous stage's output.

Docling parses the CMS excerpt into structured markdown. BAAI/bge-m3 embeds and retrieves candidate passages with cosine ranking. Qwen3-Reranker-4B reorders them so the controlling requirement language outranks incidental mentions. Then two extraction models split the work: GLiNER (urchade/gliner_multi-v2.1) pulls entities and requirement spans, while GLiNER2 (fastino/gliner2-large-v1) processes three bounded source groups separately: the requirements, the submission facts, and the outcomes.

That three-group separation is quiet but important. The six-month rule comes from requirement text. The seven-month encounter comes from the submission record. The recoupment comes from the outcome. Keeping the groups bounded means the pipeline can't accidentally read the conclusion into the evidence, or the rule into the facts.

The extracted spans are the exact critical phrases: "L1851", "6 months", "7 months", "insufficient documentation", "recoups payment". Each one is validated deterministically against the source fragments. And the timing violation itself, the one-month gap, emerges from comparing the extracted month counts in code. No model is asked to do arithmetic or render the verdict; the models find what the documents say, and the comparison is a subtraction you can read.

## All five facts or nothing

The validator requires five facts before proceeding: the HCPCS code, the six-month window, the seven-month actual gap, CMS's conclusion, and the recoupment statement. Missing or conflicting model outputs halt the run.

In a domain where a wrong automated finding means a wrongly denied claim or a missed compliance problem, halting is the correct failure mode. The pipeline either reproduces the complete evidence chain or it tells you it couldn't; there is no middle state where it produces a confident summary from partial extractions.

The run outputs make that chain inspectable: 12+ JSON files and markdown documents covering parsing, embeddings, reranking, per-group extractions, validated mappings, and latency metrics. Every step between the CMS excerpt and the final finding is on disk.

## Published ground truth is the scarce resource

Healthcare document AI has a validation problem that money can't easily solve. Real claims data is locked behind privacy walls. Synthetic data validates nothing, because the difficulty lives in exactly the messy specifics a generator won't invent. And internal test cases carry internal assumptions; the team that built the pipeline decides what the right answer was, then confirms it.

A published CMS example escapes all of that. The agency itself stated the requirement, the facts, and the outcome, so "correct" is defined by the regulator rather than the vendor. When this pipeline reproduces the finding, the claim being made is checkable by anyone with the CMS document and an afternoon. That's a small evaluation, but it's an honest one, and a pipeline that fails on published ground truth was never going to survive real claims anyway.

## Running it

```bash
cd examples/prior-authorization-review-agent
cp .env.example .env
uv sync
uv run review-pa --run-id local
uv run eval-pa runs/local
```

Local server by default; set `SIE_CLUSTER_URL` and `SIE_API_KEY` for a hosted cluster. The eval step confirms your run reproduces the published result.

## The honest scope of healthcare document AI

Notice what this example refuses to do. It doesn't decide whether the patient should have received the orthosis. It doesn't make a coverage call. It traces whether specific temporal and procedural requirements were met in the documentation, against a case where CMS already published the answer.

That's a narrower ambition than "AI prior authorization", and a far more defensible one. Utilization review teams drown in exactly this work: checking dates against windows, codes against requirements, records against rules. A pipeline that does the tracing with a validated evidence chain, and halts rather than guesses, is the version of this technology a compliance officer can actually sign off on.

The CMS example is bundled with the repo. Run it, open the evidence files, and follow the one-month gap from source text to finding.

**Try it on GitHub:** [superlinked/sie/examples/prior-authorization-review-agent](https://github.com/superlinked/sie/tree/main/examples/prior-authorization-review-agent)