Why did we open-source our inference engine? Read the post

Retrieve the passage that meets every condition

Superlinked gives your agent one API to encode technical docs and regulations as token vectors, so retrieval checks every condition in a question.

Get started
Query

create an index concurrently on a partitioned table

Answer passage #1 of 4 MaxSim 11.309

Concurrent builds for indexes on partitioned tables are currently not supported. However, you may concurrently build the index on each partition individually and then finally create the partitioned index non-concurrently in order to reduce the time where writes to the partitioned table will be locked out. In this case, building the partitioned index is a metadata only operation.

Closest other #2 of 4 MaxSim 11.105

When CREATE INDEX is invoked on a partitioned table, the default behavior is to recurse to all partitions to ensure they all have matching indexes. Each partition is first checked to determine whether an equivalent index already exists, and if so, that index will become attached as a partition index to the index being created, which will become its parent index.

Query token Best answer token Best other token
[CLS] [CLS] 0.915 [CLS] 0.908
[Q] [D] 0.928 [D] 0.922
create create 0.934 CRE 0.956
an the 0.969 an 0.966
index index 0.917 DEX 0.928
concurrently concurrently 0.891 When 0.666
on on 0.969 on 0.969
a the 0.964 a 0.967
partition partition 0.923 partition 0.926
ed ed 0.972 ed 0.968
table table 0.959 table 0.961
[SEP] is 0.968 which 0.970

Every ranking decision comes with the token scores behind it

View evidence
Frost over the fuel tanks

can an airline take off with frost on the underside of the wing above the fuel tanks

Answer passage #1 of 4 MaxSim 19.753

(b) No person may take off an aircraft when frost, ice, or snow is adhering to the wings, control surfaces, propellers, engine inlets, or other critical surfaces of the aircraft or when the takeoff would not be in compliance with paragraph (c) of this section. Takeoffs with frost under the wing in the area of the fuel tanks may be authorized by the Administrator.

Closest other #2 of 4 MaxSim 19.454

(d) A certificate holder may continue to operate under this section without a program as required in paragraph (c) of this section, if it includes in its operations specifications a requirement that, any time conditions are such that frost, ice, or snow may reasonably be expected to adhere to the aircraft, no aircraft will take off unless it has been checked to ensure that the wings, control surfaces, and other critical surfaces are free of frost, ice, and snow. The check must occur within five minutes prior to beginning takeoff. This check must be accomplished from outside the aircraft.

Query token Best answer token Best other token
fuel fuel 0.952 it 0.866
tanks tanks 0.946 it 0.869
19 more query tokens differ by less than 0.04
14 CFR 121.629, Operation in icing conditions
Too few in-sync replicas

error a producer with acks=all gets when too few in-sync replicas acknowledge the write

Answer passage #1 of 4 MaxSim 21.360

When a producer sets acks to "all" (or "-1"), this configuration specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend).

Closest other #2 of 4 MaxSim 21.073

acks=all This means the leader will wait for the full set of in-sync replicas to acknowledge the record. This guarantees that the record will not be lost as long as at least one in-sync replica remains alive. This is the strongest available guarantee. This is equivalent to the acks=-1 setting.

Query token Best answer token Best other token
[Q] [D] 0.872 [D] 0.819
producer producer 0.944 the 0.792
in this 0.932 in 0.972
sync the 0.834 sync 0.945
write write 0.944 be 0.835
18 more query tokens differ by less than 0.04
Apache Kafka 4.0 documentation, topic and producer configs
Response caching

keep every browser and proxy cache from storing a response

Answer passage #2 of 4 MaxSim 11.718

The no-store response directive indicates that any caches of any kind (private or shared) should not store this response.

Closest other #1 of 4 MaxSim 11.746

The private response directive indicates that the response can be stored only in a private cache (e.g., local caches in browsers).

Query token Best answer token Best other token
browser this 0.845 browsers 0.921
12 more query tokens differ by less than 0.04
MDN Web Docs, Cache-Control header

The answer passage scored highest in 7 of the 8 recorded searches. This grid holds 3 of the 8 and the hero one more, and every search the answer passage lost is in this grid.

Score your own question against a passage, token by token

View on GitHub
from sie_sdk import SIEClient
client = SIEClient(
api_key="sk-sie-…",
base_url="https://api.superlinked.com",
)
res = client.encode(
"modellightonai/GTE-ModernColBERT-v1",
[{"text": "querypartitioned table concurrently"}, {"text": "passageConcurrent builds for indexes on partitioned tables are currently not supported."}],
output_types=["multivector"],
)
q = res[0]["multivector"] # [n_query_tokens, dim]
d = res[1]["multivector"] # [n_doc_tokens, dim]
# MaxSim late interaction: sum each query token's best doc match
score = float(sum(max(qt @ dt for dt in d) for qt in q))
print(score)
import { SIEClient } from '@superlinked/sie-sdk';

const client = new SIEClient('https://api.superlinked.com', {
  apiKey: 'sk-sie-…',
});

const res = await client.encode(
  'lightonai/GTE-ModernColBERT-v1',
  [{ text: "partitioned table concurrently" }, { text: "Concurrent builds for indexes on partitioned tables are currently not supported." }],
  { outputTypes: ['multivector'] },
);
const [q, d] = [res[0]!.multivector!, res[1]!.multivector!]; // per-token vectors
const dot = (a: Float32Array, b: Float32Array) => a.reduce((s, x, i) => s + x * b[i], 0);
const score = q.reduce((sum, qt) => sum + Math.max(...d.map((dt) => dot(qt, dt))), 0);
console.log(score);
curl https://api.superlinked.com/v1/encode/lightonai%2FGTE-ModernColBERT-v1 \
  -H "Authorization: Bearer sk-sie-…" \
  -H "Content-Type: application/json" \
  -d "{\"items\":[{\"text\":\"partitioned table concurrently\"},{\"text\":\"Concurrent builds for indexes on partitioned tables are currently not supported.\"}],\"params\":{\"output_types\":[\"multivector\"]}}"
Build the "Multi-vector" capability into my app using the Superlinked Inference Engine (SIE).

Context
- SIE is an OpenAI-style inference API. Python SDK: `from sie_sdk import SIEClient`; TypeScript: `@superlinked/sie-sdk`.
- Base URL: https://api.superlinked.com (or my regional endpoint). Auth: Bearer key from env `SIE_API_KEY` (never hard-code it).
- Model: lightonai/GTE-ModernColBERT-v1 (SIE primitive: /encode). Keep the model id configurable.

Task
- Input: a query string plus a list of candidate documents.
- Behaviour: return the candidates ranked by late-interaction (ColBERT) relevance
- Encode the query and candidates in one batched /encode call, then rank by cosine similarity in the client. Do not call the API per candidate.

Deliverables
- A typed client wrapper, an application-level function for this task, error handling for timeouts/empty input, and unit tests with a stubbed client.
- Wire it into my existing stack (ask me which framework if unclear) and add a short usage example.
Output
partition
ed
table
concurrently
Con
0.948
0.975
0.950
0.964
current
0.899
0.922
0.894
0.970
builds
0.922
0.940
0.921
0.904
for
0.954
0.981
0.962
0.953
indexes
0.927
0.942
0.935
0.895
on
0.962
0.984
0.965
0.953
partition
0.979
0.962
0.924
0.899
ed
0.976
0.988
0.954
0.932
tables
0.933
0.948
0.983
0.903
are
0.942
0.971
0.942
0.948
currently
0.882
0.920
0.888
0.913
not
0.925
0.952
0.921
0.934
supported
0.879
0.917
0.902
0.898
MaxSim 6.875 over every query token · Columns are query tokens, rows the passage · each query token's strongest match carries the accent
  • Matches at the token level with MaxSim
  • Keeps detail a single vector blurs away

Multi-vector retrieval quality and latency

PRICE
$ / 1M input tokens
SIE bge-m3 multivector $0.0498
Jina ColBERT v2 $0.05
Jina Embeddings v4 multi $0.05
SIE GTE ModernColBERT $0.0554
QUALITY
nDCG@10 · SciFact
SIE GTE ModernColBERT 0.76
0.71
SIE bge-m3 multivector 0.70
0.69
LATENCY
p50 ms
55ms
88ms
179ms SIE GTE ModernColBERT
SIE bge-m3 multivector 354ms

Deploy your way

Self-host with K8s

Easy & scalable deployment in your own cloud.

  • Terraform to your cloud in minutes
  • Apache-2.0, same engine as Cloud
  • Scales to zero, no bill between jobs
  • Per-tenant pools, no noisy neighbors
Agent prompt
Deploy SIE to our AWS account with the superlinked/sie/aws Terraform module. Docs: superlinked.com/docs/deploymentDeploy SIE to our GCP project with the superlinked/sie/google Terraform module. Docs: superlinked.com/docs/deploymentDeploy SIE to our Azure AKS cluster via helm install. Requirements: superlinked.com/docs/deployment
Deploy guide

Run locally

Run the same models on your own machine.

  • Runs on NVIDIA GPU or Apple Silicon
  • One command, no Docker or cluster
  • All 100+ Cloud models, fully offline
  • Same SDK and IDs, no code changes
pip install "sie-server[local]" && sie-server servepip install "sie-server[local]" && sie-server serve --device cuda
Quickstart

Contact us

Tell us about your use case and we'll get back to you shortly.

Apply for an inference grant

Free capacity on our hosted cluster for selected projects. Tell us what you run and we reply by email.