Most AI tasks no longer need a frontier model Read launch post

mxbai rerank base v2

An open model that scores how well each passage fits a query. mixedbread-ai/mxbai-rerank-base-v2

  • Accuracy

    0.664

    AskUbuntu, nDCG@10

  • Time per request

    Not timed one request at a time

  • Serverless price

Call it

from sie_sdk import SIEClient
​
client = SIEClient("http://localhost:8080")
​
query = "queryWhy does my model take 30 seconds to answer the first request?"
documents = [
"document 1Cold storage keeps old checkpoints cheap to retain.",
"document 2Subsequent requests are fast once the model is warm.",
"document 3The first request is slow because the model loads into GPU memory; keep one replica warm.",
]
items = [{"id": str(i), "text": d} for i, d in enumerate(documents)]
ranked = client.score("modelmixedbread-ai/mxbai-rerank-base-v2", {"text": query}, items)
for r in ranked["scores"]:
print(r["score"], documents[int(r["item_id"])])
import { SIEClient } from '@superlinked/sie-sdk';

const client = new SIEClient('http://localhost:8080');

const query = "Why does my model take 30 seconds to answer the first request?";
const documents = [
  "Cold storage keeps old checkpoints cheap to retain.",
  "Subsequent requests are fast once the model is warm.",
  "The first request is slow because the model loads into GPU memory; keep one replica warm.",
];
const items = documents.map((text, i) => ({ id: String(i), text }));
const ranked = await client.score('mixedbread-ai/mxbai-rerank-base-v2', { text: query }, items);
console.log(ranked.scores.map(({ itemId, score }) => ({ document: documents[Number(itemId)], score })));
curl http://localhost:8080/v1/score/mixedbread-ai%2Fmxbai-rerank-base-v2 \
  -H "Content-Type: application/json" \
  -d "{\"query\":{\"text\":\"Why does my model take 30 seconds to answer the first request?\"},\"items\":[{\"id\":\"0\",\"text\":\"Cold storage keeps old checkpoints cheap to retain.\"},{\"id\":\"1\",\"text\":\"Subsequent requests are fast once the model is warm.\"},{\"id\":\"2\",\"text\":\"The first request is slow because the model loads into GPU memory; keep one replica warm.\"}]}"
Build the "Rerank" capability into my app using the Superlinked Inference Engine (SIE).

Context
- SIE is an OpenAI-style inference API. Python SDK: `from sie_sdk import SIEClient`; TypeScript: `@superlinked/sie-sdk`.
- Base URL: http://localhost:8080, my own SIE server, which takes no API key.
- Model: mixedbread-ai/mxbai-rerank-base-v2 (SIE primitive: /score). Keep the model id configurable.

Task
- Input: a query string plus a list of candidate documents.
- Behaviour: reorder the candidates by true relevance to the query using the reranking model
- Call the selected SIE primitive once per request and map the response into your domain type.

Deliverables
- A typed client wrapper, an application-level function for this task, error handling for timeouts/empty input, and unit tests with a stubbed client.
- Wire it into my existing stack (ask me which framework if unclear) and add a short usage example.

Next to the other rerankers

Next to the other rerankers
Model AskUbuntu, nDCG@10 Time per request Serverless price Size
Qwen3 Reranker 4B Qwen 0.697 55 ms per query $0.0425 per 1M tokens 4.0B
mxbai rerank large v2 mixedbread-ai 0.691 Not timed Run in your cloud 1.5B
bge reranker v2 m3 BAAI 0.676 Not timed Run in your cloud 568M
gte reranker modernbert base Alibaba-NLP 0.670 Not timed Run in your cloud 150M
All 22 rerankers

Measured on public datasets

Measured on public datasets
Dataset Metric Value
CMedQAv2Reranking MAP@10 0.803
CMedQAv1Reranking MAP@10 0.798
StackOverflowQA nDCG@10 0.911
LegalBenchConsumerContractsQA nDCG@10 0.828
SciFact nDCG@10 0.789
AskUbuntuDupQuestions nDCG@10 0.664
CQADupstackPhysicsRetrieval nDCG@10 0.495
FiQA2018 nDCG@10 0.467
NFCorpus nDCG@10 0.382
CosQA nDCG@10 0.339
SCIDOCS nDCG@10 0.176

Facts

Size
494M parameters
License
Apache-2.0
Reads
Text, up to 8,192 tokens
Endpoint
score
Evaluation
Accuracy
Run on SIE against the public dataset named in each figure
Reranking
Reorders the top 50 from gte multilingual base
Time
SIE timed on a GPU server with the client alongside, one request at a time
Prices
Serverless list prices

Run mxbai rerank base v2 in your own cloud, or start on Serverless

SIE Cluster

Kubernetes in your own cloud, backed by our engineers.

  • Terraform modules for your cloud
  • Apache-2.0, same engine as Serverless
  • GPU workers scale to zero between jobs
  • Separate worker pools for each tenant
Agent prompt Deploy SIE to our AWS account with the superlinked/sie/aws Terraform module. Docs: superlinked.com/docs/deployment.Deploy SIE to our Google Cloud project with the superlinked/sie/google Terraform module. Docs: superlinked.com/docs/deployment.Deploy SIE to our Azure subscription with the superlinked/sie/azure Terraform module. Docs: superlinked.com/docs/deployment.Deploy SIE to our Alibaba Cloud account with the superlinked/sie/alicloud Terraform module. Docs: superlinked.com/docs/deployment.

SIE Local

The open-source server on your own machine.

  • Runs on your NVIDIA GPU
  • One container, no cluster
  • Process your data fully offline
  • Same SDK and model IDs as Serverless
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-default

Talk to us

Tell us about your use case and we'll get back to you shortly.

Apply for an inference grant

Free capacity on our hosted cluster for selected projects. Tell us what you run and we reply by email.