Why did we open-source our inference engine? Read the post

Find the right page without flattening it

Superlinked gives your agent one API to search rendered PDF pages, keep their layout intact and return the evidence that matches each query.

Get started
Query

Explain the design principles and functional purposes of the hand signal system described for communication during aircraft refueling operations.

Text-only search Exact page #17

General refueling precautions

Visual page search Exact page #1

The hand-signal page

68 pages U.S. Air Force hot-refueling manual

Layout moves the right page up the ranking

View evidence

Run visual search on your own PDF pages

import numpy as np
from sie_sdk import SIEClient
client = SIEClient(
api_key="API keysk-sie-…",
base_url="https://api.superlinked.com",
)
query = "queryDid JPMorganChase execute more than half of its planned $30 billion stock repurchase program by year-end?"
pages = [
imagejpmorgan-2024-p106.jpgbrowse,
imagejpmorgan-2024-p107.jpgbrowse,
imagejpmorgan-2024-p108.jpgbrowse,
]
q = client.encode(
"modelvidore/colpali-v1.3-hf",
[{"text": query}],
output_types=["multivector"],
is_query=True,
)[0]["multivector"]
docs = client.encode(
"modelvidore/colpali-v1.3-hf",
[{"images": [page]} for page in pages],
output_types=["multivector"],
is_query=False,
)
def maxsim(query_vectors, page_vectors):
return sum(max(qt @ dt for dt in page_vectors) for qt in query_vectors)
scores = [maxsim(q, doc["multivector"]) for doc in docs]
for i in np.argsort(scores)[::-1]:
print(f"{scores[i]:.3f} {pages[i]}")
page_0=$(base64 < 'jpmorgan-2024-p106.jpg' | tr -d '\n')
page_1=$(base64 < 'jpmorgan-2024-p107.jpg' | tr -d '\n')
page_2=$(base64 < 'jpmorgan-2024-p108.jpg' | tr -d '\n')
curl https://api.superlinked.com/v1/encode/vidore%2Fcolpali-v1.3-hf \
  -H "Authorization: Bearer sk-sie-…" \
  -H "Content-Type: application/json" \
  -d "{\"items\":[{\"text\":\"Did JPMorganChase execute more than half of its planned \$30 billion stock repurchase program by year-end?\"}],\"params\":{\"output_types\":[\"multivector\"],\"is_query\":true}}"

curl https://api.superlinked.com/v1/encode/vidore%2Fcolpali-v1.3-hf \
  -H "Authorization: Bearer sk-sie-…" \
  -H "Content-Type: application/json" \
  -d "{\"items\":[{\"images\":[{\"data\":\"$page_0\",\"format\":\"jpeg\"}]},{\"images\":[{\"data\":\"$page_1\",\"format\":\"jpeg\"}]},{\"images\":[{\"data\":\"$page_2\",\"format\":\"jpeg\"}]}],\"params\":{\"output_types\":[\"multivector\"],\"is_query\":false}}"
Build the "Visual document search" capability into my app using the Superlinked Inference Engine (SIE).

Context
- SIE is an OpenAI-style inference API. Python SDK: `from sie_sdk import SIEClient`; TypeScript: `@superlinked/sie-sdk`.
- Base URL: https://api.superlinked.com (or my regional endpoint). Auth: Bearer key from env `SIE_API_KEY` (never hard-code it).
- Model: vidore/colpali-v1.3-hf (SIE primitive: /encode). Keep the model id configurable.

Task
- Input: a text query plus one or more document page images.
- Behaviour: return document pages ranked by late-interaction relevance to the query
- Call the selected SIE primitive once per request and map the response into your domain type.

Deliverables
- A typed client wrapper, an application-level function for this task, error handling for timeouts/empty input, and unit tests with a stubbed client.
- Wire it into my existing stack (ask me which framework if unclear) and add a short usage example.
Output
jpmorgan 2024 p1071top match
jpmorgan-2024-p107.jpg
19.540
jpmorgan 2024 p1062
jpmorgan-2024-p106.jpg
15.664
jpmorgan 2024 p1083
jpmorgan-2024-p108.jpg
13.439
  • Ranks page pixels without flattening the layout
  • Matches query tokens to tables, charts and text

Visual document retrieval quality and latency

PRICE
$ / 1M input tokens $ / 1k images
$0.02 SIE ColPali
$0.04 SIE ColQwen2.5
$0.05 Jina Embeddings v4 multimodal
Voyage multimodal-3.5 · 1MP $0.12
Cohere Embed v4 multimodal $0.12
$0.1 Google Multimodal Embeddings
Voyage multimodal-3.5 · 1MP $0.6
QUALITY
nDCG@5 · ViDoRe
SIE ColQwen2.5 0.89
0.87
0.86
SIE ColPali 0.81
0.79
0.78
LATENCY
p50 ms
30ms SIE ColPali
38ms
44ms
SIE ColQwen2.5 55ms
70ms
78ms

Deploy your way

Self-host with K8s

Easy & scalable deployment in your own cloud.

  • Terraform to your cloud in minutes
  • Apache-2.0, same engine as Cloud
  • Scales to zero, no bill between jobs
  • Per-tenant pools, no noisy neighbors
Agent prompt
Deploy SIE to our AWS account with the superlinked/sie/aws Terraform module. Docs: superlinked.com/docs/deploymentDeploy SIE to our GCP project with the superlinked/sie/google Terraform module. Docs: superlinked.com/docs/deploymentDeploy SIE to our Azure AKS cluster via helm install. Requirements: superlinked.com/docs/deployment
Deploy guide

Run locally

Run the same models on your own machine.

  • Runs on NVIDIA GPU or Apple Silicon
  • One command, no Docker or cluster
  • All 100+ Cloud models, fully offline
  • Same SDK and IDs, no code changes
pip install "sie-server[local]" && sie-server servepip install "sie-server[local]" && sie-server serve --device cuda
Quickstart

Contact us

Tell us about your use case and we'll get back to you shortly.

Apply for an inference grant

Free capacity on our hosted cluster for selected projects. Tell us what you run and we reply by email.