# Speech to text page sources ## Recorded model output - Model: `openai/whisper-large-v3-turbo` - Hugging Face revision in the SIE model config: `41f01f3fe87f28c78e2fbf8b568835947dd65ed9` - Served revision reported by SIE Cloud (`x-sie-model-revision`): `10333b84de80b402376b626eb25366fb081d3faeb893eb4b01cf32e8c27e4aff` - Model card: - Endpoint: `https://api.superlinked.com`, SIE server version 0.7.3 - Run: 2026-09-16, started 02:15:22 UTC, 24 calls, all HTTP 200 - Runner: `apps/site/tests/fixtures/reference/speech-to-text/run.py` (Python standard library) Every clip was sent twice as the exact MP3 bytes this page serves: 1. The playground snippet call, `POST /v1/extract/openai%2Fwhisper-large-v3-turbo` with one `audio` item and `params.instruction` set to `Transcribe the audio verbatim.` The page shows this call's `data.text`. 2. A control call with no `params`. It is recorded for comparison only: 8.8% pooled word error rate and the same 56 of 61 key terms, against 8.1% for the snippet call. The saved requests replace the base64 audio with the clip's SHA-256 and byte count. The raw responses, run manifest and scores sit beside the runner and are not served by the website. ## Pre-registration The clips, their start and end times, the human reference transcripts, the key terms and the scoring rules were committed before any model call, in `inputs.json` and `collect_inputs.py`. Nothing was reselected after the run. - Clips are cut exactly at the published human timing, with no padding, then encoded as mono 16 kHz, 32 kbit/s MP3. - Word error rate is (substitutions + deletions + insertions) divided by the number of reference words, from a word-level Levenshtein alignment. Both texts first pass through OpenAI Whisper's English text normalizer (`EnglishTextNormalizer`, ported to the standard library in `whisper_normalizer.py` from Transformers 4.57.6 and checked against it), using `normalizer.json` from the model repository at the revision above. The normalizer lowercases, removes punctuation and filler words such as "um" and "uh", and writes spoken numbers as digits. - A key term counts as found when a contiguous run of normalized transcript words, joined without spaces and with periods between digits removed, equals the normalized term. ## Scoring amendments, written after the run The key-term rule above was amended twice once the transcripts were in. Both amendments are in `scoring_amendment.json`, with their reasons and the registered amounts, and both are scoring changes over responses already recorded: no model call was repeated for either. - The pre-registered rule: a key term counts as found when a contiguous run of normalized transcript words, joined without spaces and with periods between digits removed, equals the normalized term. - **Money amounts, which can only remove hits.** A key term that states a money amount counts as found only when the transcript writes that same amount. It applies on top of the pre-registered rule, so a term must satisfy that rule first. In the AMI project kickoff the speaker says "twelve fifty Euros, so fifty percent of the selling price" of twenty-five Euros, and Whisper wrote `€1250`. The normalizer turns the reference and the transcript into the same string, so no string rule can separate them; the spoken amount has to be stated explicitly. A reader searching for that figure would find a number a hundred times too large. - **Term content, which can only add hits.** A key term counts as found when the transcript spells its content. Content is the words and the digits; the decimal point inside a number, a currency symbol and a leading article are not content, and both sides drop them before comparing. The amount itself is never dropped, so `twelve fifty Euros` still fails against `€1250`. This amendment was made after inspecting the miss list: the comparison had been currency-symbol- and article-sensitive, so it printed two transcription errors a reader could disprove from the transcript shown beside them. `eleven ninety nine` had failed against the model's `€11.99`, and `a hundred percent` against the model's `100%`, though both amounts are correct. The rule lives in `matching.py`, which both the scorer and the display highlighting import, so the two cannot drift. Measured over the recorded responses, each rule applied to all 61 terms: the pre-registered rule alone finds 55 of 61, the money amendment removes one, and the content amendment adds two, so the published figure is **56 of 61**, the same for the snippet call and the control call. The intermediate figure, 54, is what this page carried before the content amendment. Word error rate is untouched by both. Result across all 12 clips: 48 edits over 594 reference words (8.1% pooled word error rate) and 56 of 61 key terms found. ## What the page displays All 12 clips are scored, counted in the figures above and held in the page data. The page plays 5 of them: the hero clip, three in the proof grid and the playground clip. The grid was cut from eleven rows to three in September 2026 so the section reads in one pass; the cut is a display decision and removed no clip, transcript or score. The three rows are one per recording setting, chosen to carry different arguments: | Row | Recorded id | Why it is displayed | | --- | --- | --- | | Telehealth consultation | `primock-uti-antibiotics` | A 4.3% word error rate, and the drug name still came back wrong | | Supreme Court argument | `scotus-irs-levy-notices` | The highest word error rate of the 12, 20.0%, with every key term still found | | Meeting room microphone | `ami-project-finance` | The money amendment case, where a match hid a hundredfold error | The hero plays `ami-remote-control-chip` and the playground plays `scotus-nrc-atomic-energy-act`. The remaining seven clips are scored and not played. Two of the five key-term misses are in the displayed grid; the line under the grid states that split on the page itself, computed from all 12 scored clips rather than from the three cards, and site CI compares the sentence with the built page. ## What the misses are Every one of the five is a transcription error, not a matching artifact. The two marked "displayed" are visible in the proof grid; the rest are counted in the totals above and in the page data. - `GeForce` came back as `G-Force` (NVIDIA argument). - `DSH payment` came back as `DISH payment` (Medicare argument). - `Piriton` came back as `Pyroton`. - `Trimethoprim` came back as `trimethoprine` (displayed). - `twelve fifty Euros` came back as `€1250`, a hundred times the spoken amount (displayed). The normalizer hides this, so the money amendment catches it. The page keeps a note beside that clip explaining the figure. ## Primary sources ### Supreme Court of the United States oral arguments Public domain. Audio from the Court's argument audio page; reference text from the official transcript by Heritage Reporting Corporation, published by the Court. Clip bounds come from the Oyez sentence alignment of the same recording (); only its start and stop times are used, and the Oyez recording matches the Court's file length within 0.04 seconds. - *NVIDIA Corp. v. E. Ohman J:or Fonder AB*, No. 23-970, argued 2024-11-13. Audio: . Transcript: . - `scotus-nvidia-crypto-share.mp3`: 914.30 to 944.12 s, transcript page 18, line 2 (Neal Kumar Katyal). - `scotus-nvidia-geforce-units.mp3`: 2999.03 to 3020.93 s, transcript page 54, line 15 (Deepak Gupta). - *Nuclear Regulatory Commission v. Texas*, No. 23-1300, argued 2025-03-05. Audio: . Transcript: . - `scotus-nrc-atomic-energy-act.mp3`: 1490.95 to 1509.06 s, transcript page 31, line 10 (Malcolm L. Stewart). - *Advocate Christ Medical Center v. Kennedy*, No. 23-715, argued 2024-11-05. Audio: . Transcript: . - `scotus-medicare-dsh-payment.mp3`: 3849.59 to 3867.02 s, transcript page 75, line 5 (Ephraim McDowell). - *Commissioner of Internal Revenue v. Zuch*, No. 24-416, argued 2025-04-22. Audio: . Transcript: . - `scotus-irs-levy-notices.mp3`: 2306.09 to 2322.59 s, transcript page 47, line 10 (Shay Dvoretzky). ### PriMock57 primary care consultations Babylon Health, *PriMock57: A Dataset Of Primary Care Mock Consultations* (Papadopoulos Korfiatis et al., ACL 2022), . Licensed under Creative Commons Attribution 4.0 International (). The consultations are mock: clinicians talk to colleagues acting as patients over a video call. Audio is the doctor channel; references are the human utterance-level TextGrid transcripts, cut at the utterance's own start and end. Changes: the clip is trimmed to one utterance and re-encoded as MP3; on the page, transcriber tags such as `` are dropped from the displayed reference. - `primock-chlorphenamine-dose.mp3`: `day2_consultation06_doctor`, 476.738 to 485.563 s. - `primock-inhaler-plan.mp3`: `day3_consultation04_doctor`, 411.283 to 425.558 s. - `primock-uti-antibiotics.mp3`: `day1_consultation09_doctor`, 444.679 to 454.459 s. - `primock-ibuprofen-dose.mp3`: `day1_consultation13_doctor`, 516.130 to 529.360 s. ### AMI Meeting Corpus University of Edinburgh Centre for Speech Technology Research, AMI Meeting Corpus, . Licensed under Creative Commons Attribution 4.0 International (). The meetings are scenario design meetings recorded in an instrumented room. Audio is the first microphone of the table array (`Array1-01`), so every speaker in the room is audible. References come from the manual word annotations (`ami_public_manual_1.6.2.zip`, `words/*.words.xml`): every word from every speaker whose midpoint falls inside the window, in time order, with AMI's letter spellings joined (`T_A_` to `TA`). Changes: the clip is trimmed to the window and re-encoded as MP3. - `ami-remote-control-chip.mp3`: meeting ES2004c, 452.66 to 478.53 s (hero). - `ami-production-cost.mp3`: meeting ES2009d, 423.75 to 451.32 s. - `ami-project-finance.mp3`: meeting ES2002a, 472.41 to 495.00 s. ## What the page does not claim The page makes no claim about speaker names, timestamps, punctuation or language detection. The snippet call returned `language: null` for every clip, and the page shows that value as returned. The hero's search is a plain substring count over the returned `data.text`.