--- title: answerdotai/answerai-colbert-small-v1 (Score) description: answerai-colbert-small-v1 is a new, proof-of-concept model by Answer.AI, showing the strong performance multi-vector models with the new JaC. BERT, 33M parameters. canonical_url: https://superlinked.com/models/answerdotai-answerai-colbert-small-v1--score last_updated: 2026-05-25 --- # answerdotai/answerai-colbert-small-v1 (Score) answerai-colbert-small-v1 is a new, proof-of-concept model by Answer.AI, showing the strong performance multi-vector models with the new JaColBERTv2.5 training recipe and some extra tweaks can reach, even with just 33 million parameters. Source: [answerdotai/answerai-colbert-small-v1 on HuggingFace](https://huggingface.co/answerdotai/answerai-colbert-small-v1) ## Overview | Field | Value | |-------|-------| | Architecture | BERT | | Parameters | 33M | | Tasks | Encode, Score | | Outputs | Multi-Vec | | Dimensions | Multi-Vec: 96 | | Max sequence length | 512 tokens | | License | apache-2.0 | | Inputs | text | | Languages | en | ## Benchmarks ### AskUbuntuDupQuestions Domain: technology · Task: reranking · Language: en Duplicate question detection from AskUbuntu Corpus: 6,743 · Queries: 360 **Quality:** ndcg at 10: 0.6259 · map at 10: 0.4680 · mrr at 10: 0.7165 [Reference](https://github.com/taolei87/askubuntu) ### CMedQAv1-reranking Domain: general · Task: reranking · Language: en **Quality:** ndcg at 10: 0.2088 · map at 10: 0.1602 · mrr at 10: 0.2210 ### CMedQAv2-reranking Domain: general · Task: reranking · Language: en **Quality:** ndcg at 10: 0.2311 · map at 10: 0.1838 · mrr at 10: 0.2409 ### CQADupstackPhysicsRetrieval Domain: scientific · Task: retrieval · Language: en Duplicate question retrieval from StackExchange Physics Corpus: 38,314 · Queries: 1,039 **Quality:** ndcg at 10: 0.4336 · map at 10: 0.3778 · mrr at 10: 0.4431 **Performance (L4-SPOT b1 c16):** Corpus 3.9K tok/s · Corpus p50 203.2ms · Query 186 tok/s · Query p50 300.7ms **Performance (L4 b1 c16):** Corpus 37.3K tok/s · Corpus p50 54.7ms · Query 3.6K tok/s · Query p50 47.1ms [Reference](http://nlp.cis.unimelb.edu.au/resources/cqadupstack/) ### CosQA Domain: technology · Task: retrieval · Language: en Code search with natural language queries Corpus: 6,267 · Queries: 500 **Quality:** ndcg at 10: 0.3084 · map at 10: 0.2328 · mrr at 10: 0.2479 **Performance (L4-SPOT b1 c16):** Corpus 1.1K tok/s · Corpus p50 345.4ms · Query 102 tok/s · Query p50 466.2ms **Performance (L4 b1 c16):** Corpus 15.7K tok/s · Corpus p50 53.9ms · Query 2.0K tok/s · Query p50 47.4ms [Reference](https://arxiv.org/abs/2105.13239) ### FiQA2018 Domain: finance · Task: retrieval · Language: en Financial opinion mining and question answering Corpus: 57,599 · Queries: 648 **Quality:** ndcg at 10: 0.4361 · map at 10: 0.3564 · mrr at 10: 0.5317 **Performance (L4-SPOT b1 c16):** Corpus 3.4K tok/s · Corpus p50 384.8ms · Query 174 tok/s · Query p50 547.3ms **Performance (L4 b1 c16):** Corpus 43.1K tok/s · Corpus p50 59.1ms · Query 3.7K tok/s · Query p50 50.0ms [Reference](https://sites.google.com/view/fiqa/) ### LegalBenchConsumerContractsQA Domain: legal · Task: retrieval · Language: en Question answering on consumer contracts Corpus: 153 · Queries: 396 **Quality:** ndcg at 10: 0.7823 · map at 10: 0.7278 · mrr at 10: 0.7293 **Performance (L4-SPOT b1 c16):** Corpus 11.2K tok/s · Corpus p50 286.1ms · Query 254 tok/s · Query p50 500.2ms **Performance (L4 b1 c16):** Corpus 83.1K tok/s · Corpus p50 83.9ms · Query 4.8K tok/s · Query p50 52.2ms [Reference](https://huggingface.co/datasets/nguha/legalbench) ### MMarcoReranking Domain: general · Task: reranking · Language: zh Multilingual MARCO passage reranking (Chinese) **Quality:** ndcg at 10: 0.0812 · map at 10: 0.0603 · mrr at 10: 0.0603 [Reference](https://arxiv.org/abs/2304.03679) ### NFCorpus Domain: medical · Task: retrieval · Language: en Biomedical literature search from NutritionFacts.org Corpus: 3,593 · Queries: 323 **Quality:** ndcg at 10: 0.3744 · map at 10: 0.2715 · mrr at 10: 0.5828 **Performance (L4-SPOT b1 c16):** Corpus 5.7K tok/s · Corpus p50 300.1ms · Query 210 tok/s · Query p50 178.7ms **Performance (L4 b1 c16):** Corpus 60.7K tok/s · Corpus p50 69.4ms · Query 1.4K tok/s · Query p50 52.3ms [Reference](https://www.cl.uni-heidelberg.de/statnlpgroup/nfcorpus/) ### SCIDOCS Domain: scientific · Task: retrieval · Language: en Citation prediction, document classification, and recommendation for scientific papers Corpus: 25,656 · Queries: 1,000 **Quality:** ndcg at 10: 0.1892 · map at 10: 0.1112 · mrr at 10: 0.3353 **Performance (L4-SPOT b1 c16):** Corpus 2.9K tok/s · Corpus p50 501.5ms · Query 222 tok/s · Query p50 353.0ms **Performance (L4 b1 c16):** Corpus 48.4K tok/s · Corpus p50 57.9ms · Query 3.8K tok/s · Query p50 46.7ms [Reference](https://allenai.org/data/scidocs) ### SciFact Domain: scientific · Task: retrieval · Language: en Scientific claim verification using research literature Corpus: 5,183 · Queries: 300 **Quality:** ndcg at 10: 0.7485 · map at 10: 0.7060 · mrr at 10: 0.7181 **Performance (L4-SPOT b1 c16):** Corpus 4.9K tok/s · Corpus p50 443.1ms · Query 332 tok/s · Query p50 430.3ms **Performance (L4 b1 c16):** Corpus 61.5K tok/s · Corpus p50 63.3ms · Query 5.2K tok/s · Query p50 50.5ms [Reference](https://github.com/allenai/scifact) ### StackOverflowQA Domain: technology · Task: retrieval · Language: en Programming question answering from Stack Overflow Corpus: 19,931 · Queries: 1,994 **Quality:** ndcg at 10: 0.6171 · map at 10: 0.5703 · mrr at 10: 0.5799 **Performance (L4-SPOT b1 c16):** Corpus 4.4K tok/s · Corpus p50 379.7ms · Query 5.2K tok/s · Query p50 432.3ms **Performance (L4 b1 c16):** Corpus 55.3K tok/s · Corpus p50 60.2ms · Query 73.2K tok/s · Query p50 64.7ms [Reference](https://arxiv.org/abs/2407.02883) ### T2Reranking Domain: general · Task: reranking · Language: zh Chinese passage ranking benchmark **Quality:** ndcg at 10: 0.6821 · map at 10: 0.5041 · mrr at 10: 0.7178 [Reference](https://arxiv.org/abs/2304.03679)