jinaai/jina-colbert-v2
Primitive: /score · Score ·
XLM-RoBERTa
Trained by Jina AI.
MultilingualLong context
Overview
Hardware: — drives latency, throughput & cost
| Size | 559M params |
|---|---|
| Tasks | /encode · /score |
| License | cc-by-nc-4.0 |
| Languages | multilingual, af, am, ar, as, az, be, bg, bn, br, bs, ca, cs, cy, da, de, el, en, eo, es, et, eu, fa, fi, fr, fy, ga, gd, gl, gu, ha, he, hi, hr, hu, hy, id, is, it, ja, jv, ka, kk, km, kn, ko, ku, ky, la, lo, lt, lv, mg, mk, ml, mn, mr, ms, my, ne, nl, no, om, or, pa, pl, ps, pt, ro, ru, sa, sd, si, sk, sl, so, sq, sr, su, sv, sw, ta, te, th, tl, tr, ug, uk, ur, uz, vi, xh, yi, zh |
| Latency | 226 ms |
| Throughput | 1.4K tok/s |
| Cost | $0.158 /1M tok |
Cost is approximate — computed from list GPU prices; your actual price depends on the provider you deploy SIE with.
Scoring
| Inputs | text |
|---|---|
| Max sequence length | 8,192 |
Benchmarks
AskUbuntuDupQuestions
Duplicate question detection from AskUbuntu
Corpus: 6,743 Queries: 360
muvera
Quality
ndcg at 10 0.6481
map at 10 0.4896
mrr at 10 0.7363
default
Quality
ndcg at 10 0.6391
map at 10 0.4833
mrr at 10 0.7290
CMedQAv1-reranking
Chinese medical question answering reranking (v1)
Corpus: 100,000 Queries: 2,000
muvera
Quality
ndcg at 10 0.3980
map at 10 0.3319
mrr at 10 0.4200
default
Quality
ndcg at 10 0.5026
map at 10 0.4465
mrr at 10 0.5249
CMedQAv2-reranking
Chinese medical question answering reranking (v2)
Corpus: 108,000 Queries: 4,000
muvera
Quality
ndcg at 10 0.4235
map at 10 0.3545
mrr at 10 0.4379
default
Quality
ndcg at 10 0.5066
map at 10 0.4502
mrr at 10 0.5342
CQADupstackPhysicsRetrieval
Duplicate question retrieval from StackExchange Physics
Corpus: 38,314 Queries: 1,039
default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.3491
mrr at 10 0.4005
ndcg at 10 0.4044
Performance L4-SPOT b1 c16
Corpus 3.3K tok/s
Corpus p50 325.6ms
Query 163 tok/s
Query p50 490.5ms
Performance L4 b1 c16
Corpus 22.9K tok/s
Corpus p50 83.9ms
Query 2.9K tok/s
Query p50 59.2ms
muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.2633
mrr at 10 0.3078
ndcg at 10 0.3167
CosQA
Code search with natural language queries
Corpus: 6,267 Queries: 500
default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.2083
mrr at 10 0.1928
ndcg at 10 0.2676
Performance L4-SPOT b1 c16
Corpus 798 tok/s
Corpus p50 555.8ms
Query 127 tok/s
Query p50 364.0ms
Performance L4 b1 c16
Corpus 13.4K tok/s
Corpus p50 65.7ms
Query 1.6K tok/s
Query p50 60.4ms
muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.1775
mrr at 10 0.2119
ndcg at 10 0.2339
FiQA2018
Financial opinion mining and question answering
Corpus: 57,599 Queries: 648
default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.3236
mrr at 10 0.4864
ndcg at 10 0.4039
Performance L4-SPOT b1 c16
Corpus 1.7K tok/s
Corpus p50 656.4ms
Query 218 tok/s
Query p50 412.6ms
Performance L4 b1 c16
Corpus 25.8K tok/s
Corpus p50 95.6ms
Query 3.0K tok/s
Query p50 60.7ms
muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.1793
mrr at 10 0.3034
ndcg at 10 0.2390
LegalBenchConsumerContractsQA
Question answering on consumer contracts
Corpus: 153 Queries: 396
default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.7114
mrr at 10 0.7122
ndcg at 10 0.7625
Performance L4-SPOT b1 c16
Corpus 5.8K tok/s
Corpus p50 568.2ms
Query 216 tok/s
Query p50 517.1ms
Performance L4 b1 c16
Corpus 27.5K tok/s
Corpus p50 274.0ms
Query 3.1K tok/s
Query p50 74.2ms
muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.5524
mrr at 10 0.5516
ndcg at 10 0.6238
MMarcoReranking
Multilingual MARCO passage reranking (Chinese)
default
Quality
map at 10 0.3444
mrr at 10 0.3518
muvera
Quality
ndcg at 10 0.2753
map at 10 0.2211
mrr at 10 0.2264
NFCorpus
Biomedical literature search from NutritionFacts.org
Corpus: 3,593 Queries: 323
default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.1423
mrr at 10 0.5709
ndcg at 10 0.3578
Performance L4-SPOT b1 c16
Corpus 3.0K tok/s
Corpus p50 622.8ms
Query 77 tok/s
Query p50 440.1ms
Performance L4 b1 c16
Corpus 30.3K tok/s
Corpus p50 156.2ms
Query 1.2K tok/s
Query p50 63.6ms
muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.1046
mrr at 10 0.4801
ndcg at 10 0.2894
SCIDOCS
Citation prediction, document classification, and recommendation for scientific papers
Corpus: 25,656 Queries: 1,000
default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.1046
mrr at 10 0.3099
ndcg at 10 0.1783
Performance L4-SPOT b1 c16
Corpus 2.8K tok/s
Corpus p50 521.5ms
Query 165 tok/s
Query p50 520.0ms
Performance L4 b1 c16
Corpus 26.7K tok/s
Corpus p50 108.8ms
Query 2.7K tok/s
Query p50 62.0ms
muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.0868
mrr at 10 0.2564
ndcg at 10 0.1525
SciFact
Scientific claim verification using research literature
Corpus: 5,183 Queries: 300
default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.6253
mrr at 10 0.6368
ndcg at 10 0.6684
Performance L4-SPOT b1 c16
Corpus 4.6K tok/s
Corpus p50 397.7ms
Query 269 tok/s
Query p50 436.5ms
Performance L4 b1 c16
Corpus 29.3K tok/s
Corpus p50 145.2ms
Query 3.9K tok/s
Query p50 65.2ms
muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.5539
mrr at 10 0.5659
ndcg at 10 0.5988
StackOverflowQA
Programming question answering from Stack Overflow
Corpus: 19,931 Queries: 1,994
default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.5717
mrr at 10 0.5717
ndcg at 10 0.6082
Performance L4-SPOT b1 c16
Corpus 3.4K tok/s
Corpus p50 554.5ms
Query 3.9K tok/s
Query p50 505.2ms
Performance L4 b1 c16
Corpus 19.2K tok/s
Corpus p50 162.2ms
Query 40.0K tok/s
Query p50 88.2ms
muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base
Quality
map at 10 0.3764
mrr at 10 0.3764
ndcg at 10 0.4247
T2Reranking
Chinese passage ranking benchmark
muvera
Quality
ndcg at 10 0.7016
map at 10 0.5271
mrr at 10 0.7507
default
Quality
ndcg at 10 0.7321
map at 10 0.5607
mrr at 10 0.7809
Compare (0)Compare →