← Catalog
colbert-ir/colbertv2.0
Open comparison →
Primitive: /encode · Encode ·
BERT
[](https://colab.research.google.com/github/stanford-futuredata/ColBERT/blob/main/docs/intro2new.ipynb)
Multi-vector
View on Hugging Face →
Overview
Hardware:
— drives latency, throughput & cost
| Size | 110M params |
|---|
| Tasks | /encode · /score |
|---|
| License | mit |
|---|
| Languages | en |
|---|
| Latency | 66 ms |
|---|
| Throughput | 43.0K tok/s |
|---|
| Cost | $0.0052 /1M tok |
|---|
Cost is approximate — computed from list GPU prices; your actual price depends on the provider you deploy SIE with.
Embedding
| Output types | Multi-Vec |
|---|
| Dimensions | multivector: 128 |
|---|
| Max sequence length | 512 |
|---|
| Inputs | text |
|---|
Benchmarks
Duplicate question retrieval from StackExchange Physics
Corpus: 38,314 Queries: 1,039
Performance L4 b1 c16
Corpus 34.0K tok/s
Corpus p50 56.9ms
Query 3.6K tok/s
Query p50 48.5ms
Reference →
Code search with natural language queries
Corpus: 6,267 Queries: 500
Performance L4 b1 c16
Corpus 16.0K tok/s
Corpus p50 52.1ms
Query 1.9K tok/s
Query p50 50.5ms
Reference →
Financial opinion mining and question answering
Corpus: 57,599 Queries: 648
Performance L4 b1 c16
Corpus 38.8K tok/s
Corpus p50 60.6ms
Query 3.8K tok/s
Query p50 49.2ms
Reference →
Question answering on consumer contracts
Corpus: 153 Queries: 396
Performance L4 b1 c16
Corpus 78.1K tok/s
Corpus p50 94.6ms
Query 5.5K tok/s
Query p50 48.3ms
Reference →
Biomedical literature search from NutritionFacts.org
Corpus: 3,593 Queries: 323
Quality
ndcg at 10 0.2647
map at 10 0.0913
mrr at 10 0.4494
Performance L4 b1 c16
Corpus 52.6K tok/s
Corpus p50 78.8ms
Query 1.5K tok/s
Query p50 51.3ms
Reference →
Smaller subset of the FiQA financial QA dataset
default
Quality
ndcg at 10 0.4675
map at 10 0.3893
mrr at 10 0.5201
Performance L4 b1 c16
Corpus 43.0K tok/s
Corpus p50 44.7ms
Query 3.9K tok/s
Query p50 34.6ms
muvera
Quality
ndcg at 10 0.4675
map at 10 0.3893
mrr at 10 0.5201
Reference →
Citation prediction, document classification, and recommendation for scientific papers
Corpus: 25,656 Queries: 1,000
Performance L4 b1 c16
Corpus 41.4K tok/s
Corpus p50 65.7ms
Query 3.5K tok/s
Query p50 50.5ms
Reference →
Scientific claim verification using research literature
Corpus: 5,183 Queries: 300
Performance L4 b1 c16
Corpus 53.9K tok/s
Corpus p50 70.6ms
Query 5.4K tok/s
Query p50 48.9ms
Reference →
Programming question answering from Stack Overflow
Corpus: 19,931 Queries: 1,994
Performance L4 b1 c16
Corpus 49.0K tok/s
Corpus p50 69.7ms
Query 64.6K tok/s
Query p50 73.5ms
Reference →
