# Doc field extraction page sources ## Recorded model output - Model: [`Qwen/Qwen3.8-27B-FP8`](https://huggingface.co/Qwen/Qwen3.8-27B-FP8) - Endpoint: SIE Cloud, `https://api.superlinked.com`, `POST /v1/generate/Qwen__Qwen3.8-27B-FP8` - Call shape: the generated snippet for this task, so one page image plus `grammar.json_schema` with `strict: true`, and the fixed prompt `Extract the document fields described by the JSON schema. Return only the JSON object.` - Run date: 2026-09-15. Requests, raw responses, the run manifest, the scored evaluation and the image checksums live in `apps/site/tests/fixtures/reference/doc-field-extraction/`. - Re-run with `python3 apps/site/tests/fixtures/reference/doc-field-extraction/run.py`. The registry lists this task on `Qwen/Qwen3.8-27B-FP8` for both lanes, and production serves that model. Proof calls raise `max_new_tokens` from the snippet's 512 to 2048 so a long table is never cut; the playground call keeps 512, which is what the displayed snippet sends. Generation is sampled. The snippet sends no temperature, so individual fields move between runs: an earlier complete run of these same eight documents scored 174 of 223 fields, with a different mix of misses. Every number on the page comes from the one complete run pinned in `manifest.json`, not from an average or a best-of. ## Pre-registration `inputs.json` was written before any model call. It fixes the eight documents, the JSON schema for each, the expected value of every field, and the comparison rules. Expected values were read from the source pixels and, where a document has one, from its PDF text layer. Nothing in `expected` changed after the run. One later change is recorded in `inputs.json` and kept in `diagnostics/playground-v1/`: the first playground schema asked for `payment_terms`, and the call returned the printed `NET 30` plus a long boilerplate paragraph that is not on the invoice. The field was replaced with `currency` and that call re-run. ### Audit of every displayed miss, 2026-09-16 Every one of the 43 misses was re-read individually against the page beside it, to make sure the page never prints a miss next to output a reader can see is correct. The check asks, for each miss, whether the returned value appears verbatim in the page's text layer. It over-reports, so each hit was judged against the document rather than banked. All 43 hold up as real. Two classes needed a decision and are recorded here: - **The 14 element rows on the NIST certificate are decomposition failures, not transcription errors.** The schema asks for `element` and `symbol` as separate required fields, so splitting `Carbon (C)` into `Carbon` and `C` is the instruction the model was given. On all 14 rows it wrote the printed cell into `element` and the symbol into `symbol`, so the name field carries the symbol twice. `srm_number` is scored the same way: the page prints `SRM 1155a` and no sibling field claims the prefix, so returning the phrase is arguable, but appending label text to an identifier field is the same behaviour and is counted the same. Both were considered for correction and both were left as misses. - **Seven Greek-bill rows return a number that really is printed on the bill, from the wrong line.** `supply_charges_eur` returned 14.25, the transport charge, where 287.98 is printed for the supply subtotal. Real errors. The symmetry check ran the same test over all 138 currently-matching text expectations, so it could cut against the model as easily as for it. It produced one candidate, `line_items[0].item` on the invoice, which turned out to be a text-layer wrapping artifact: the cell prints across two lines with other columns interleaved. A further 24 expectations on raster documents are **unverifiable**, meaning Tesseract did not find them, which does not prove they are absent from the page; they are not counted as confirmed either way. ## Display steps, not model output - **Field marks.** The model returns values only; it never reports where on the page a value came from. `regions.py` builds a Tesseract 5.5.1 word layer for each image and matches a returned value to a run of consecutive words by exact normalized text. Only a value that also matches the printed page carries a mark, and a checkbox field is matched against its printed label, never the tick itself. Fields with no text match stay unmarked. - **Scoring.** `evaluate.py` compares returned values with the pre-registered ones: strings after Unicode NFKC, case folding, straightened quotes, hyphenated dashes, removed whitespace and commas, and one ignored trailing period; numbers within 1e-9; enums and booleans exactly. ## What the page displays Eight documents were recorded and all eight stay in the evidence record. The proof grid shows three of them, because each card is a document scan plus a field ledger and dense proof items are capped at three: | Displayed | Document | Score | Why this one | |---|---|---|---| | yes | NIST SRM 1155a certificate | 56 of 71 | Dense table, 42 of 42 numbers and 14 of 14 symbols exact, and the element/symbol decomposition finding | | yes | FAA 8130-3, rebuilt fuel control | 15 of 16 | A ticked box and an empty box both arrive as typed booleans | | yes | Walmart receipt, photographed sideways | 4 of 11 | Schema-valid JSON with invented values | | no | FAA 8130-3, export flap | 16 of 16 | | | no | OSHA Form 300 injury log | 48 of 53 | | | no | Wolters Kluwer invoice | 14 of 17 | | | no | USPS CN 22 customs declaration | 15 of 17 | | | no | DEI electricity bill, Greece | 12 of 22 | | The three shown are not the three best. The only document that matched every field is not displayed, the worst recorded result is, and the line under the grid reports the totals over all eight together with the worst result the page no longer shows. ## Source documents ### FAA Form 8130-3 authorized release certificates - Document: - Publisher: Federal Aviation Administration, Order 8130.21H, Appendix A, Figures A-10 and A-14 - License: U.S. government work, public domain - Derivation: the embedded figure images on PDF pages 56 and 60, extracted unchanged with `pdfimages` - The two samples carry opposite boxes: A-10 ticks "Other regulation specified in Block 12" in block 14a, A-14 ticks "Approved design data" in block 13a. ### OSHA Form 300, log of work-related injuries and illnesses - Document: - Publisher: Occupational Safety and Health Administration, "Forms for Recording Work-Related Injuries and Illnesses", page 6 - License: U.S. government work, public domain - The names and cases are OSHA's own illustrative entries on its published "How to Fill Out the Log" example, not records of real people. - Derivation: the embedded 2621x1824 image on page 6, cropped to the header and filled rows and resized to 2000 px wide - Case 4 is revised on the form: the original column J selection is struck through and column H is selected with 3 days away. ### NIST SRM 1155a certificate of analysis - Document: - Publisher: National Institute of Standards and Technology, Certificate of Analysis for Standard Reference Material 1155a, stainless steel (AISI 316), page 3 - License: U.S. government work, public domain - Derivation: page 3 rendered at 150 dpi with `pdftoppm`, grayscale. The page also carries Table 2, reference values, which the schema excludes. ### Wolters Kluwer Health invoice - Source page: - Publisher: Wolters Kluwer Health, uploaded to Wikimedia Commons - License: public domain on Commons, ineligible for copyright - Derivation: page 1 rendered at 150 dpi with `pdftoppm`; the contact name on the Attention line is covered with a dark box before the run, so no personal name reaches the model or the page. ### USPS First-Class Package International label with CN 22 customs declaration - Source page: - Publisher: USPS label, scanned by a Commons contributor - License: public domain on Commons, ineligible for copyright - Derivation: the 4934x3708 scan resized to 1800 px wide. The uploader had already blurred the recipient name and street. ### DEI electricity bill, Greece - Source page: - Publisher: Public Power Corporation (DEI) bill, scanned by Rakoon - License: CC0 1.0 - Derivation: the top 2420 px of the 2481x3603 scan, the meter table and the charge breakdown, resized to 1600 px wide. The payment stub carrying the service address is cropped away. ### Walmart grocery receipt, photographed sideways - Source page: - Publisher: Cryptogoth, Wikimedia Commons - License: [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) - Derivation: the 4032x3024 photo resized to 2016x1512, orientation left as photographed, so the receipt text runs bottom to top.