← Catalog
answerdotai/answerai-colbert-small-v1 Add to compare Open comparison →
Primitive: /encode · Encode ·
BERT
answerai-colbert-small-v1 is a new, proof-of-concept model by Answer.AI, showing the strong performance multi-vector models with the new JaColBERTv2.5 training recipe and some extra tweaks can reach, even with just 33 million parameters.
Multi-vector
View on Hugging Face →
Overview
Hardware:
L4 RTX-PRO-6000 — drives latency, throughput & cost
Size 33M params Tasks /encode · /score License apache-2.0 Languages en Latency 48 ms Throughput 59.1K tok/s Cost $0.0038 /1M tok
Cost is approximate — computed from list GPU prices; your actual price depends on the provider you deploy SIE with.
Embedding Output types Multi-Vec Dimensions multivector: 96 Max sequence length 512 Inputs text
Benchmarks Duplicate question retrieval from StackExchange Physics
Corpus: 38,314 Queries: 1,039
Quality
ndcg at 10 0.4154
map at 10 0.3645
mrr at 10 0.4213
Performance L4 b1 c16
Corpus 44.9K tok/s
Corpus p50 45.2ms
Query 4.5K tok/s
Query p50 37.7ms
Reference →
Code search with natural language queries
Corpus: 6,267 Queries: 500
Quality
ndcg at 10 0.2844
map at 10 0.2180
mrr at 10 0.2069
Performance L4 b1 c16
Corpus 19.0K tok/s
Corpus p50 43.5ms
Query 2.3K tok/s
Query p50 40.6ms
Reference →
Financial opinion mining and question answering
Corpus: 57,599 Queries: 648
Quality
ndcg at 10 0.4103
map at 10 0.3338
mrr at 10 0.4965
Performance L4 b1 c16
Corpus 49.3K tok/s
Corpus p50 47.9ms
Query 4.5K tok/s
Query p50 40.0ms
Reference →
Question answering on consumer contracts
Corpus: 153 Queries: 396
Quality
ndcg at 10 0.7840
map at 10 0.7315
mrr at 10 0.7315
Performance L4 b1 c16
Corpus 115.6K tok/s
Corpus p50 62.7ms
Query 6.4K tok/s
Query p50 40.8ms
Reference →
Biomedical literature search from NutritionFacts.org
Corpus: 3,593 Queries: 323
Quality
ndcg at 10 0.3715
map at 10 0.1440
mrr at 10 0.5870
Performance L4 b1 c16
Corpus 75.8K tok/s
Corpus p50 55.4ms
Query 1.9K tok/s
Query p50 41.4ms
Reference →
Smaller subset of the FiQA financial QA dataset
Quality
ndcg at 10 0.5563
map at 10 0.4718
mrr at 10 0.6192
Performance L4 b1 c16
Corpus 43.6K tok/s
Corpus p50 44.3ms
Query 4.0K tok/s
Query p50 35.0ms
Reference →
Citation prediction, document classification, and recommendation for scientific papers
Corpus: 25,656 Queries: 1,000
Quality
ndcg at 10 0.1778
map at 10 0.1046
mrr at 10 0.3078
Performance L4 b1 c16
Corpus 59.1K tok/s
Corpus p50 47.2ms
Query 4.7K tok/s
Query p50 38.2ms
Reference →
Scientific claim verification using research literature
Corpus: 5,183 Queries: 300
Quality
ndcg at 10 0.7405
map at 10 0.7015
mrr at 10 0.7120
Performance L4 b1 c16
Corpus 75.3K tok/s
Corpus p50 51.0ms
Query 6.7K tok/s
Query p50 39.6ms
Reference →
Programming question answering from Stack Overflow
Corpus: 19,931 Queries: 1,994
Quality
ndcg at 10 0.5461
map at 10 0.5130
mrr at 10 0.5130
Performance L4 b1 c16
Corpus 62.1K tok/s
Corpus p50 53.0ms
Query 88.2K tok/s
Query p50 52.4ms
Reference →
