Why did we open-source our inference engine? Read the post

mixedbread-ai/mxbai-rerank-large-v2 (Score)

Architecture
Parameters
435M
Tasks
Score
Outputs
Score
Max Sequence Length
8,192 tokens
License

Benchmarks

CosQA

technology retrieval en

Performance L4 b1 c16
Query TPS 1.9K
Query p50 535.4ms

FiQA2018

finance retrieval en

Performance L4 b1 c16
Query TPS 1.3K
Query p50 1.4s

LegalBenchConsumerContractsQA

legal retrieval en

Performance L4 b1 c16
Query TPS 7.5K
Query p50 767.2ms

NFCorpus

medical retrieval en

Performance L4 b1 c16
Query TPS 2.3K
Query p50 1.7s

SciFact

scientific retrieval en

Performance L4 b1 c16
Query TPS 2.2K
Query p50 1.7s

Self-hosted inference for search & document processing

Cut API costs by 50x, boost quality with 85+ SOTA models, and keep your data in your own cloud.

Github
1.5K

Contact us

Tell us about your use case and we'll get back to you shortly.