Most AI tasks no longer need a frontier model Read launch post

mxbai colbert large v1

An open multi-vector model that matches a query to documents token by token. mixedbread-ai/mxbai-colbert-large-v1

  • Accuracy

    8 retrieval sets, mean nDCG@10 · not measured

  • Time per request

    Not timed one request at a time

  • Serverless price

Call it

from sie_sdk import SIEClient
​
client = SIEClient("http://localhost:8080")
​
query = "queryrefund duplicate subscription"
passages = [
"passage 1We reimburse accidental double payments on any recurring membership.",
]
​
q = client.encode(
"modelmixedbread-ai/mxbai-colbert-large-v1",
[{"text": query}],
output_types=["multivector"],
is_query=True,
)[0]["multivector"]
docs = client.encode(
"modelmixedbread-ai/mxbai-colbert-large-v1",
[{"text": passage} for passage in passages],
output_types=["multivector"],
is_query=False,
)
​
print(q)
print(docs)
import { SIEClient } from '@superlinked/sie-sdk';

const client = new SIEClient('http://localhost:8080');

const query = "refund duplicate subscription";
const passages = [
  "We reimburse accidental double payments on any recurring membership.",
];
const model = 'mixedbread-ai/mxbai-colbert-large-v1';

const [q] = await client.encode(model, [{ text: query }], {
  outputTypes: ['multivector'], isQuery: true,
});
const docs = await client.encode(model, passages.map((text) => ({ text })), {
  outputTypes: ['multivector'], isQuery: false,
});
console.log({ query: q, documents: docs });
# Encode the question as a query, then the passages as documents; rank by MaxSim.
curl http://localhost:8080/v1/encode/mixedbread-ai%2Fmxbai-colbert-large-v1 \
  -H "Content-Type: application/json" \
  -d "{\"items\":[{\"text\":\"refund duplicate subscription\"}],\"params\":{\"output_types\":[\"multivector\"],\"options\":{\"is_query\":true}}}"

curl http://localhost:8080/v1/encode/mixedbread-ai%2Fmxbai-colbert-large-v1 \
  -H "Content-Type: application/json" \
  -d "{\"items\":[{\"text\":\"We reimburse accidental double payments on any recurring membership.\"}],\"params\":{\"output_types\":[\"multivector\"],\"options\":{\"is_query\":false}}}"
Build the "Multi-vector" capability into my app using the Superlinked Inference Engine (SIE).

Context
- SIE is an OpenAI-style inference API. Python SDK: `from sie_sdk import SIEClient`; TypeScript: `@superlinked/sie-sdk`.
- Base URL: http://localhost:8080, my own SIE server, which takes no API key.
- Model: mixedbread-ai/mxbai-colbert-large-v1 (SIE primitive: /encode). Keep the model id configurable.

Task
- Input: a query string plus a list of candidate documents.
- Behaviour: return the query and passage token vectors from the native encode calls
- Send two POST requests to http://localhost:8080/v1/encode/mixedbread-ai%2Fmxbai-colbert-large-v1. Encode one query item {text: query} with params {output_types: ["multivector"], options: {is_query: true}}, then all candidate passages in one batch of {text: passage} items with params {output_types: ["multivector"], options: {is_query: false}}. Do not call the API per candidate. Read the returned per-token vectors from items[i].multivector.values. Return the native token vectors without adding application scoring or ranking. In Python, use SIEClient.encode(..., output_types=["multivector"], is_query=True) for the query and encode(..., output_types=["multivector"], is_query=False) for the passage batch; read each decoded ["multivector"] array. In TypeScript, use client.encode with outputTypes: ["multivector"] and isQuery: true for the query, then isQuery: false for the passage batch; the SDK decodes the native token vectors.

Deliverables
- A typed client wrapper, an application-level function for this task, error handling for timeouts/empty input, and unit tests with a stubbed client.
- Wire it into my existing stack (ask me which framework if unclear) and add a short usage example.

Next to the other multi-vector models

Next to the other multi-vector models
Model 8 retrieval sets, mean nDCG@10 Time per request Serverless price Size
GTE ModernColBERT lightonai Not measured Not timed $0.0332 per 1M tokens 149M
mLateOn lightonai 0.561 Not timed Run in your cloud 307M
Reason ModernColBERT lightonai 0.519 Not timed Run in your cloud 149M
answerai colbert small v1 answerdotai 0.466 Not timed Run in your cloud 33M
All 9 multi-vector models

Measured on public datasets

Measured on public datasets
Dataset nDCG@10
SciFact 0.631
NanoFiQA2018Retrieval 0.458
FiQA2018 0.364
NFCorpus 0.307
SCIDOCS 0.160

Facts

Size
335M parameters
License
Apache-2.0
Reads
Text, up to 512 tokens
Endpoint
encode
Evaluation
Accuracy
Run on SIE against the public dataset named in each figure
8 retrieval sets
The mean over NFCorpus, FiQA2018, SciFact, SCIDOCS, CQADupstackPhysicsRetrieval, CosQA, StackOverflowQA and LegalBenchConsumerContractsQA
Prices
Serverless list prices, effective 10 Nov 2026

Run mxbai colbert large v1 in your own cloud, or start on Serverless

Native Inference

Kubernetes in your own cloud, backed by our engineers.

  • Terraform modules for your cloud
  • Apache-2.0, same engine as Serverless
  • GPU workers scale to zero between jobs
  • Separate worker pools for each tenant
Agent prompt Deploy SIE to our AWS account with the superlinked/sie/aws Terraform module. Docs: superlinked.com/docs/deployment.Deploy SIE to our Google Cloud project with the superlinked/sie/google Terraform module. Docs: superlinked.com/docs/deployment.Deploy SIE to our Azure subscription with the superlinked/sie/azure Terraform module. Docs: superlinked.com/docs/deployment.Deploy SIE to our Alibaba Cloud account with the superlinked/sie/alicloud Terraform module. Docs: superlinked.com/docs/deployment.

SIE Local

The open-source server on your own machine.

  • Runs on your NVIDIA GPU
  • One container, no cluster
  • Process your data fully offline
  • Same SDK and model IDs as Serverless
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-default

Talk to us

Tell us about your use case and we'll get back to you shortly.

Apply for an inference grant

Free capacity on our hosted cluster for selected projects. Tell us what you run and we reply by email.