Most AI tasks no longer need a frontier model Read launch post

colpali v1.3 hf

An open model that searches document pages from their images. vidore/colpali-v1.3-hf

  • Accuracy

    3 ViDoRe v3 sets, mean nDCG@10 · not measured

  • Time per request

    Not timed one request at a time

  • Serverless price

Call it

from pathlib import Path
from urllib.request import urlopen
from sie_sdk import SIEClient
​
client = SIEClient("http://localhost:8080")
​
model = "modelvidore/colpali-v1.3-hf"
query = "queryDid JPMorganChase execute more than half of its planned $30 billion stock repurchase program by year-end?"
pages = [
imagejpmorgan-2024-p106.jpgbrowse,
imagejpmorgan-2024-p107.jpgbrowse,
imagejpmorgan-2024-p108.jpgbrowse,
]
images = [urlopen(page).read() if page.startswith("https://") else Path(page).read_bytes() for page in pages]
​
q = client.encode(
model, [{"text": query}],
output_types=["multivector"], is_query=True,
)[0]["multivector"]
docs = client.encode(
model, [{"images": [image]} for image in images],
output_types=["multivector"], is_query=False,
)
​
print(q)
print(docs)
import { readFile } from 'node:fs/promises';
import { SIEClient } from '@superlinked/sie-sdk';

const client = new SIEClient('http://localhost:8080');

const model = 'vidore/colpali-v1.3-hf';
const query = "Did JPMorganChase execute more than half of its planned $30 billion stock repurchase program by year-end?";
const pages = [
  "jpmorgan-2024-p106.jpg",
  "jpmorgan-2024-p107.jpg",
  "jpmorgan-2024-p108.jpg",
];
const sources = pages;
const images = await Promise.all(sources.map(async (source) => {
  if (!source.startsWith('https://')) return readFile(source);
  const response = await fetch(source);
  if (!response.ok) throw new Error('Could not load the original page image.');
  return new Uint8Array(await response.arrayBuffer());
}));

const [q] = await client.encode(
  model, [{ text: query }],
  { outputTypes: ['multivector'], isQuery: true },
);
const docs = await client.encode(
  model, images.map((image) => ({ images: [image] })),
  { outputTypes: ['multivector'], isQuery: false },
);
console.log(q, docs);
page_0=$(base64 < 'jpmorgan-2024-p106.jpg' | tr -d '\n')
page_1=$(base64 < 'jpmorgan-2024-p107.jpg' | tr -d '\n')
page_2=$(base64 < 'jpmorgan-2024-p108.jpg' | tr -d '\n')
curl http://localhost:8080/v1/encode/vidore%2Fcolpali-v1.3-hf \
  -H "Content-Type: application/json" \
  -d "{\"items\":[{\"text\":\"Did JPMorganChase execute more than half of its planned \$30 billion stock repurchase program by year-end?\"}],\"params\":{\"output_types\":[\"multivector\"],\"options\":{\"is_query\":true}}}"

curl http://localhost:8080/v1/encode/vidore%2Fcolpali-v1.3-hf \
  -H "Content-Type: application/json" \
  -d "{\"items\":[{\"images\":[{\"data\":\"$page_0\",\"format\":\"jpeg\"}]},{\"images\":[{\"data\":\"$page_1\",\"format\":\"jpeg\"}]},{\"images\":[{\"data\":\"$page_2\",\"format\":\"jpeg\"}]}],\"params\":{\"output_types\":[\"multivector\"],\"options\":{\"is_query\":false}}}"
Build the "Visual document search" capability into my app using the Superlinked Inference Engine (SIE).

Context
- SIE is an OpenAI-style inference API. Python SDK: `from sie_sdk import SIEClient`; TypeScript: `@superlinked/sie-sdk`.
- Base URL: http://localhost:8080, my own SIE server, which takes no API key.
- Model: vidore/colpali-v1.3-hf (SIE primitive: /encode). Keep the model id configurable.

Task
- Input: a text query plus one or more document page images.
- Behaviour: return native token vectors for the query and document page images
- Preserve the original page image bytes. Encode the text query with output_types ["multivector"] and is_query true, then all page images in one batch with output_types ["multivector"] and is_query false. Python: client.encode with output_types and is_query. TypeScript: client.encode with outputTypes and isQuery. Return the native token vectors without application scoring or ranking.

Deliverables
- A typed client wrapper, an application-level function for this task, error handling for timeouts/empty input, and unit tests with a stubbed client.
- Wire it into my existing stack (ask me which framework if unclear) and add a short usage example.

Next to the other page search models

Next to the other page search models
Model 3 ViDoRe v3 sets, mean nDCG@10 Time per request Serverless price Size
Tomoro ColQwen3 4B TomoroAI Not measured Not timed $2.00 per 1M tokens $0.20 per 1,000 images 4.4B
nemotron colembed vl 4b v2 nvidia 0.720 Not timed Run in your cloud 4.8B
v splade quality naver 0.527 Not timed Run in your cloud 330M
colqwen2.5 v0.2 vidore Not measured Not timed Run in your cloud 7.0B
All 6 page search models

Facts

Size
2.9B parameters
License
gemma
Reads
Text (up to 2,048 tokens), image
Endpoint
encode
Evaluation
Accuracy
Run on SIE against the public dataset named in each figure
3 ViDoRe v3 sets
The mean over Vidore3ComputerScienceRetrieval, Vidore3FinanceEnRetrieval and Vidore3PharmaceuticalsRetrieval
Prices
Serverless list prices, effective 10 Nov 2026

Run colpali v1.3 hf in your own cloud, or start on Serverless

Native Inference

Kubernetes in your own cloud, backed by our engineers.

  • Terraform modules for your cloud
  • Apache-2.0, same engine as Serverless
  • GPU workers scale to zero between jobs
  • Separate worker pools for each tenant
Agent prompt Deploy SIE to our AWS account with the superlinked/sie/aws Terraform module. Docs: superlinked.com/docs/deployment.Deploy SIE to our Google Cloud project with the superlinked/sie/google Terraform module. Docs: superlinked.com/docs/deployment.Deploy SIE to our Azure subscription with the superlinked/sie/azure Terraform module. Docs: superlinked.com/docs/deployment.Deploy SIE to our Alibaba Cloud account with the superlinked/sie/alicloud Terraform module. Docs: superlinked.com/docs/deployment.

SIE Local

The open-source server on your own machine.

  • Runs on your NVIDIA GPU
  • One container, no cluster
  • Process your data fully offline
  • Same SDK and model IDs as Serverless
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-default

Talk to us

Tell us about your use case and we'll get back to you shortly.

Apply for an inference grant

Free capacity on our hosted cluster for selected projects. Tell us what you run and we reply by email.