Why did we open-source our inference engine? Read the post

Turn documents into typed records for a ninth of what Claude Opus 5.5 charges

Send your JSON Schema with the text and get back a record shaped to it, filled in even where the text only implies the answer.

No credit card, or run it in your own cluster.

ODI complaint 11180068

TL* THE CONTACT OWNED A 2016 KIA SPORTAGE . WHILE THE CONTACT'S SON WAS MAKING A RIGHT TURN AT 5 MPH , HE LOST CONTROL OF THE VEHICLE. THE STEERING SEIZED, WHICH CAUSED THE VEHICLE TO CRASH INTO A TREE AND BURST INTO FLAMES . IT WAS UNKNOWN IF THE AIR BAGS DEPLOYED . THE CONTACT'S SON SUFFERED A CONCUSSION AND SOUGHT MEDICAL ATTENTION. A POLICE REPORT WAS FILED. THE FIRE DEPARTMENT EXTINGUISHED THE FIRE. THE VEHICLE WAS DEEMED TOTALED BY THE CONTACT'S INSURANCE COMPANY. THE DEALER AND MANUFACTURER WERE NOT CONTACTED. THE FAILURE MILEAGE AND VIN WERE UNKNOWN . *TT*JB

Returned JSON
airbags
"unknown"
crash
true
fire
true
injured_people
1
make
"KIA"
model
"SPORTAGE"
model_year
2016
odometer_miles
null
speed_mph
5
PRICE
$ / 1M input tokens $ / 1M output tokens ↓
$0.72 SIE Qwen3.8 27B
$2 Anthropic Claude Sonnet 5
$2 OpenAI GPT-6 Sol
$4 Anthropic Claude Opus 5.5
OpenAI GPT-6 Astra $10
$0.72 SIE Qwen3.8 27B
$10 Anthropic Claude Sonnet 5
$10 OpenAI GPT-6 Sol
$20 Anthropic Claude Opus 5.5
OpenAI GPT-6 Astra $50
QUALITY
Schema validity · JSONSchemaBench ↑
0.95
0.95
0.93
0.93
SIE Qwen3.8 27B 0.83
LATENCY
p50 s ↓
1.6s
1.6s
2.3s
2.6s
SIE Qwen3.8 27B 7.0s

Send your schema with the document and get the record back

Run this example
import json
from sie_sdk import SIEClient
​
client = SIEClient(
api_key="API keysk-sie-…",
base_url="https://api.superlinked.com",
)
​
schema = json.loads("""
schema{
"type": "object",
"additionalProperties": false,
"properties": {
"check_engine_light_on": {"type": "boolean"},
"oil_pressure_sensor_faulty": {"type": ["boolean", "null"]},
"part_to_replace": {"type": ["string", "null"]},
"cosmetic_issues": {"type": "array", "items": {"type": "string"}},
"additional_deficiencies_known": {"type": "boolean"}
},
"required": ["check_engine_light_on", "oil_pressure_sensor_faulty", "part_to_replace", "cosmetic_issues", "additional_deficiencies_known"]
}
""")
instruction = "instructionFill the maintenance record from this GSA Auctions condition note. Use null when the note does not state a value."
messages = [
{"role": "system", "content": f"{instruction}\n\nRequired JSON schema:\n{json.dumps(schema, indent=2)}"},
{"role": "user", "content": "textCondition & Markings Usable/good condition. Check engine light is on with initial code indicating faulty oil pressure sensor. Mechanics inspection determined sensor is functional but oil pump requires replacement. Minor dents, scratches, and visual blemishes consistent with normal wear and age. Additional deficiencies/information unknown."},
]
result = client.chat_completions(
"modelQwen/Qwen3.8-27B-FP8",
messages,
max_completion_tokens=512,
response_format={
"type": "json_schema",
"json_schema": {"name": "structured_output", "strict": True, "schema": schema},
},
)
print(json.loads(result["choices"][0]["message"]["content"]))
import { SIEClient } from '@superlinked/sie-sdk';

const client = new SIEClient('https://api.superlinked.com', {
  apiKey: 'sk-sie-…',
});

const schema = {
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "check_engine_light_on": {"type": "boolean"},
    "oil_pressure_sensor_faulty": {"type": ["boolean", "null"]},
    "part_to_replace": {"type": ["string", "null"]},
    "cosmetic_issues": {"type": "array", "items": {"type": "string"}},
    "additional_deficiencies_known": {"type": "boolean"}
  },
  "required": ["check_engine_light_on", "oil_pressure_sensor_faulty", "part_to_replace", "cosmetic_issues", "additional_deficiencies_known"]
};
const instruction = "Fill the maintenance record from this GSA Auctions condition note. Use null when the note does not state a value.";
const result = await client.chatCompletions({
  model: 'Qwen/Qwen3.8-27B-FP8',
  messages: [
    {
      role: 'system',
      content: `${instruction}\n\nRequired JSON schema:\n${JSON.stringify(schema, null, 2)}`,
    },
    { role: 'user', content: "Condition & Markings\nUsable/good condition.\nCheck engine light is on with initial code indicating faulty oil pressure sensor.\nMechanics inspection determined sensor is functional but oil pump requires replacement.\nMinor dents, scratches, and visual blemishes consistent with normal wear and age.\nAdditional deficiencies/information unknown." },
  ],
  max_completion_tokens: 512,
  response_format: {
    type: 'json_schema',
    json_schema: { name: 'structured_output', strict: true, schema },
  },
});
const content = result.choices[0].message.content;
if (typeof content !== 'string') throw new Error('The model returned no JSON content.');
console.log(JSON.parse(content));
curl https://api.superlinked.com/v1/chat/completions \
  -H "Authorization: Bearer sk-sie-…" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"Qwen/Qwen3.8-27B-FP8\",\"messages\":[{\"role\":\"system\",\"content\":\"Fill the maintenance record from this GSA Auctions condition note. Use null when the note does not state a value.\\n\\nRequired JSON schema:\\n{\\n  \\\"type\\\": \\\"object\\\",\\n  \\\"additionalProperties\\\": false,\\n  \\\"properties\\\": {\\n    \\\"check_engine_light_on\\\": {\\n      \\\"type\\\": \\\"boolean\\\"\\n    },\\n    \\\"oil_pressure_sensor_faulty\\\": {\\n      \\\"type\\\": [\\n        \\\"boolean\\\",\\n        \\\"null\\\"\\n      ]\\n    },\\n    \\\"part_to_replace\\\": {\\n      \\\"type\\\": [\\n        \\\"string\\\",\\n        \\\"null\\\"\\n      ]\\n    },\\n    \\\"cosmetic_issues\\\": {\\n      \\\"type\\\": \\\"array\\\",\\n      \\\"items\\\": {\\n        \\\"type\\\": \\\"string\\\"\\n      }\\n    },\\n    \\\"additional_deficiencies_known\\\": {\\n      \\\"type\\\": \\\"boolean\\\"\\n    }\\n  },\\n  \\\"required\\\": [\\n    \\\"check_engine_light_on\\\",\\n    \\\"oil_pressure_sensor_faulty\\\",\\n    \\\"part_to_replace\\\",\\n    \\\"cosmetic_issues\\\",\\n    \\\"additional_deficiencies_known\\\"\\n  ]\\n}\"},{\"role\":\"user\",\"content\":\"Condition & Markings\\nUsable/good condition.\\nCheck engine light is on with initial code indicating faulty oil pressure sensor.\\nMechanics inspection determined sensor is functional but oil pump requires replacement.\\nMinor dents, scratches, and visual blemishes consistent with normal wear and age.\\nAdditional deficiencies/information unknown.\"}],\"max_completion_tokens\":512,\"response_format\":{\"type\":\"json_schema\",\"json_schema\":{\"name\":\"structured_output\",\"strict\":true,\"schema\":{\"type\":\"object\",\"additionalProperties\":false,\"properties\":{\"check_engine_light_on\":{\"type\":\"boolean\"},\"oil_pressure_sensor_faulty\":{\"type\":[\"boolean\",\"null\"]},\"part_to_replace\":{\"type\":[\"string\",\"null\"]},\"cosmetic_issues\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"additional_deficiencies_known\":{\"type\":\"boolean\"}},\"required\":[\"check_engine_light_on\",\"oil_pressure_sensor_faulty\",\"part_to_replace\",\"cosmetic_issues\",\"additional_deficiencies_known\"]}}}}"
Build the "Structured output" capability into my app using the Superlinked Inference Engine (SIE).

Context
- SIE is an OpenAI-style inference API. Python SDK: `from sie_sdk import SIEClient`; TypeScript: `@superlinked/sie-sdk`.
- Base URL: https://api.superlinked.com (or my regional endpoint). Auth: Bearer key from env `SIE_API_KEY` (never hard-code it).
- Model: Qwen/Qwen3.8-27B-FP8 (OpenAI-compatible endpoint: /v1/chat/completions). Keep the model id configurable.

Task
- Input: a block of text.
- Behaviour: return schema-valid JSON extracted from the text
- Send one POST /v1/chat/completions request (SDK: chat_completions / chatCompletions) with the instruction and the JSON schema text in a system message, the source text as the user message, max_completion_tokens 512, and response_format {type: "json_schema", json_schema: {name: "structured_output", strict: true, schema}}. Parse choices[0].message.content as JSON and fail loudly when it is missing or invalid.

Deliverables
- A typed client wrapper, an application-level function for this task, error handling for timeouts/empty input, and unit tests with a stubbed client.
- Wire it into my existing stack (ask me which framework if unclear) and add a short usage example.
Output
{
  "additional_deficiencies_known": false,
  "check_engine_light_on": true,
  "cosmetic_issues": [
    "minor dents",
    "scratches",
    "visual blemishes"
  ],
  "oil_pressure_sensor_faulty": false,
  "part_to_replace": "oil pump"
}
Schema-valid4 of 4 checked fields right
  • ✓check_engine_light_onis true
  • ✓oil_pressure_sensor_faultyis false
  • ✓part_to_replacecontains "oil pump", any case
  • ✓additional_deficiencies_knownis false
  • –cosmetic_issuesschema-validated, not scored
Validated with jsonschema 4.21.1 Draft202012Validator; finished with stop in 6,836 ms
  • Emits schema-valid JSON from raw text
  • Conforms to the schema you define

Complaints, filings and listings come back in the shape your schema asks for

  • 10 of 10 records valid against their schema, first call
  • 7 nulls returned, each where the document had none

A crash complaint, with speed, fire and injuries typed

Document

TL* THE CONTACTS DAUGHTER OWNED A 2019 HONDA CR-V. WHILE THE CONTACT'S DAUGHTER WAS DRIVING 60 MPH, SHE CRASHED INTO THE REAR OF A COMMERCIAL TRUCK. THERE WERE NO WARNING INDICATORS ILLUMINATED. THE AIR BAGS DID NOT DEPLOY. …

Record

airbags
"did not deploy"
crash
true
fire
true
injured_people
1
make
"Honda"
model
"CR-V"
model_year
2019

A CEO appointment, salary and dates as typed values

Document

Item 5.02. Departure of Directors or Principal Officers; Election of Directors; Appointment of Principal Officers; Compensatory Arrangements of Certain Officers. …

Record

appointee
"Nick Konat"
appointee_age
49
base_salary_in_new_role_usd
1000000
board_action_date
"2026-08-31"
company
"Sprouts Farmers Market, Inc."
effective_date
"2027-01-04"
internal_promotion
true

A listing whose two mileages disagree, flagged true

Document

Specifications Make: IHC Inc Model: 1754 Model Year: 1984 Transmission Type: Automatic No of Cylinders: 08 Fuel Type: Gasoline Body Style: Utility Mileage: 52970 Color: Blue Color Gradient: Medium Open Recall: No VIN: 1HTCHWN9EHA26990 Condi …

Record

airbags_deployed
null
highway_title
null
listed_mileage
52970
make
"IHC Inc"
mileage_conflict
true
model
"1754"
model_year
1984

Deploy your way

Self-host with K8s

Easy & scalable deployment in your own cloud.

  • Terraform to your cloud in minutes
  • Apache-2.0, same engine as Cloud
  • Scales to zero, no bill between jobs
  • Per-tenant pools, no noisy neighbors
Agent prompt
Deploy SIE to our AWS account with the superlinked/sie/aws Terraform module. Docs: superlinked.com/docs/deploymentDeploy SIE to our GCP project with the superlinked/sie/google Terraform module. Docs: superlinked.com/docs/deploymentDeploy SIE to our Azure AKS cluster via helm install. Requirements: superlinked.com/docs/deployment
Deploy guide

Run locally

Run the same models on your own machine.

  • Runs on NVIDIA GPU or Apple Silicon
  • One command, no Docker or cluster
  • All 100+ Cloud models, fully offline
  • Same SDK and IDs, no code changes
pip install "sie-server[local]" && sie-server servepip install "sie-server[local]" && sie-server serve --device cuda
Quickstart

$1,008 a month instead of $8,800 on Claude Opus 5.5

Cost
One million documents a month at 1,200 input tokens and 200 output tokens each, priced at list on both sides.
Risk
Open weights, and the model ID is pinned, so the record you parse today comes back the same way next quarter.
Engineering time
The request is the OpenAI chat completions format with your schema as the response format, so migrating is a base URL and a model name.

Contact us

Tell us about your use case and we'll get back to you shortly.

Apply for an inference grant

Free capacity on our hosted cluster for selected projects. Tell us what you run and we reply by email.