← Catalog
answerdotai/answerai-colbert-small-v1 Add to compare Open comparison →
Primitive: /score · Score ·
BERT
answerai-colbert-small-v1 is a new, proof-of-concept model by Answer.AI, showing the strong performance multi-vector models with the new JaColBERTv2.5 training recipe and some extra tweaks can reach, even with just 33 million parameters.
View on Hugging Face →
Overview
Hardware:
L4 RTX-PRO-6000 — drives latency, throughput & cost
Size 33M params Tasks /encode · /score License apache-2.0 Languages en Latency 122 ms Throughput 1.7K tok/s Cost $0.128 /1M tok
Cost is approximate — computed from list GPU prices; your actual price depends on the provider you deploy SIE with.
Scoring Inputs text Max sequence length 512
Benchmarks Duplicate question detection from AskUbuntu
Corpus: 6,743 Queries: 360
Quality
ndcg at 10 0.6259
map at 10 0.4680
mrr at 10 0.7165
Reference →
Quality
ndcg at 10 0.2088
map at 10 0.1602
mrr at 10 0.2210
Quality
ndcg at 10 0.2311
map at 10 0.1838
mrr at 10 0.2409
Duplicate question retrieval from StackExchange Physics
Corpus: 38,314 Queries: 1,039
Quality
ndcg at 10 0.4336
map at 10 0.3778
mrr at 10 0.4431
Performance L4-SPOT b1 c16
Corpus 3.9K tok/s
Corpus p50 203.2ms
Query 186 tok/s
Query p50 300.7ms
Performance L4 b1 c16
Corpus 37.3K tok/s
Corpus p50 54.7ms
Query 3.6K tok/s
Query p50 47.1ms
Reference →
Code search with natural language queries
Corpus: 6,267 Queries: 500
Quality
ndcg at 10 0.3084
map at 10 0.2328
mrr at 10 0.2479
Performance L4-SPOT b1 c16
Corpus 1.1K tok/s
Corpus p50 345.4ms
Query 102 tok/s
Query p50 466.2ms
Performance L4 b1 c16
Corpus 15.7K tok/s
Corpus p50 53.9ms
Query 2.0K tok/s
Query p50 47.4ms
Reference →
Financial opinion mining and question answering
Corpus: 57,599 Queries: 648
Quality
ndcg at 10 0.4361
map at 10 0.3564
mrr at 10 0.5317
Performance L4-SPOT b1 c16
Corpus 3.4K tok/s
Corpus p50 384.8ms
Query 174 tok/s
Query p50 547.3ms
Performance L4 b1 c16
Corpus 43.1K tok/s
Corpus p50 59.1ms
Query 3.7K tok/s
Query p50 50.0ms
Reference →
Question answering on consumer contracts
Corpus: 153 Queries: 396
Quality
ndcg at 10 0.7823
map at 10 0.7278
mrr at 10 0.7293
Performance L4-SPOT b1 c16
Corpus 11.2K tok/s
Corpus p50 286.1ms
Query 254 tok/s
Query p50 500.2ms
Performance L4 b1 c16
Corpus 83.1K tok/s
Corpus p50 83.9ms
Query 4.8K tok/s
Query p50 52.2ms
Reference →
Multilingual MARCO passage reranking (Chinese)
Quality
ndcg at 10 0.0812
map at 10 0.0603
mrr at 10 0.0603
Reference →
Biomedical literature search from NutritionFacts.org
Corpus: 3,593 Queries: 323
Quality
ndcg at 10 0.3744
map at 10 0.2715
mrr at 10 0.5828
Performance L4-SPOT b1 c16
Corpus 5.7K tok/s
Corpus p50 300.1ms
Query 210 tok/s
Query p50 178.7ms
Performance L4 b1 c16
Corpus 60.7K tok/s
Corpus p50 69.4ms
Query 1.4K tok/s
Query p50 52.3ms
Reference →
Citation prediction, document classification, and recommendation for scientific papers
Corpus: 25,656 Queries: 1,000
Quality
ndcg at 10 0.1892
map at 10 0.1112
mrr at 10 0.3353
Performance L4-SPOT b1 c16
Corpus 2.9K tok/s
Corpus p50 501.5ms
Query 222 tok/s
Query p50 353.0ms
Performance L4 b1 c16
Corpus 48.4K tok/s
Corpus p50 57.9ms
Query 3.8K tok/s
Query p50 46.7ms
Reference →
Scientific claim verification using research literature
Corpus: 5,183 Queries: 300
Quality
ndcg at 10 0.7485
map at 10 0.7060
mrr at 10 0.7181
Performance L4-SPOT b1 c16
Corpus 4.9K tok/s
Corpus p50 443.1ms
Query 332 tok/s
Query p50 430.3ms
Performance L4 b1 c16
Corpus 61.5K tok/s
Corpus p50 63.3ms
Query 5.2K tok/s
Query p50 50.5ms
Reference →
Programming question answering from Stack Overflow
Corpus: 19,931 Queries: 1,994
Quality
ndcg at 10 0.6171
map at 10 0.5703
mrr at 10 0.5799
Performance L4-SPOT b1 c16
Corpus 4.4K tok/s
Corpus p50 379.7ms
Query 5.2K tok/s
Query p50 432.3ms
Performance L4 b1 c16
Corpus 55.3K tok/s
Corpus p50 60.2ms
Query 73.2K tok/s
Query p50 64.7ms
Reference →
Chinese passage ranking benchmark
Quality
ndcg at 10 0.6821
map at 10 0.5041
mrr at 10 0.7178
Reference →
