--- title: answerdotai/answerai-colbert-small-v1 (Score) description: answerai-colbert-small-v1 is a new, proof-of-concept model by Answer.AI, showing the strong performance multi-vector models with the new JaC. BERT, 33M parameters. canonical_url: https://superlinked.com/models/answerdotai-answerai-colbert-small-v1--score last_updated: 2026-10-14 --- # answerdotai/answerai-colbert-small-v1 (Score) answerai-colbert-small-v1 is a new, proof-of-concept model by Answer.AI, showing the strong performance multi-vector models with the new JaColBERTv2.5 training recipe and some extra tweaks can reach, even with just 33 million parameters. Source: [answerdotai/answerai-colbert-small-v1 on HuggingFace](https://huggingface.co/answerdotai/answerai-colbert-small-v1) ## Overview | Field | Value | |-------|-------| | Architecture | BERT | | Parameters | 33M | | Tasks | Encode, Score | | Outputs | Multi-Vec | | Dimensions | Multi-Vec: 96 | | Max sequence length | 512 tokens | | License | apache-2.0 | | Inputs | text | | Languages | en | ## Benchmarks ### AskUbuntuDupQuestions Domain: technology · Task: reranking · Language: en Duplicate question detection from AskUbuntu Corpus: 6,743 · Queries: 360 **Variant: muvera** **Quality:** ndcg at 10: 0.6331 · map at 10: 0.4738 · mrr at 10: 0.7266 **Variant: default** **Quality:** ndcg at 10: 0.6259 · map at 10: 0.4680 · mrr at 10: 0.7165 [Reference](https://github.com/taolei87/askubuntu) ### CMedQAv1-reranking Domain: medical · Task: reranking · Language: zh Chinese medical question answering reranking (v1) Corpus: 100,000 · Queries: 2,000 **Variant: muvera** **Quality:** ndcg at 10: 0.2192 · map at 10: 0.1670 · mrr at 10: 0.2283 **Variant: default** **Quality:** ndcg at 10: 0.2088 · map at 10: 0.1602 · mrr at 10: 0.2210 [Reference](https://github.com/zhangsheng93/cMedQA) ### CMedQAv2-reranking Domain: medical · Task: reranking · Language: zh Chinese medical question answering reranking (v2) Corpus: 108,000 · Queries: 4,000 **Variant: muvera** **Quality:** ndcg at 10: 0.2356 · map at 10: 0.1835 · mrr at 10: 0.2397 **Variant: default** **Quality:** ndcg at 10: 0.2311 · map at 10: 0.1838 · mrr at 10: 0.2409 [Reference](https://github.com/zhangsheng93/cMedQA2) ### CQADupstackPhysicsRetrieval Domain: scientific · Task: retrieval · Language: en Duplicate question retrieval from StackExchange Physics Corpus: 38,314 · Queries: 1,039 **Variant: default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.3646 · mrr at 10: 0.4214 · ndcg at 10: 0.4155 **Performance (L4-SPOT b1 c16):** Corpus 3.9K tok/s · Corpus p50 203.2ms · Query 186 tok/s · Query p50 300.7ms **Performance (L4 b1 c16):** Corpus 37.3K tok/s · Corpus p50 54.7ms · Query 3.6K tok/s · Query p50 47.1ms **Variant: muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.3163 · mrr at 10: 0.3688 · ndcg at 10: 0.3702 [Reference](http://nlp.cis.unimelb.edu.au/resources/cqadupstack/) ### CosQA Domain: technology · Task: retrieval · Language: en Code search with natural language queries Corpus: 6,267 · Queries: 500 **Variant: default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.2256 · mrr at 10: 0.2148 · ndcg at 10: 0.2897 **Performance (L4-SPOT b1 c16):** Corpus 1.1K tok/s · Corpus p50 345.4ms · Query 102 tok/s · Query p50 466.2ms **Performance (L4 b1 c16):** Corpus 15.7K tok/s · Corpus p50 53.9ms · Query 2.0K tok/s · Query p50 47.4ms **Variant: muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.1953 · mrr at 10: 0.2087 · ndcg at 10: 0.2536 [Reference](https://arxiv.org/abs/2105.13239) ### FiQA2018 Domain: finance · Task: retrieval · Language: en Financial opinion mining and question answering Corpus: 57,599 · Queries: 648 **Variant: default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.3337 · mrr at 10: 0.4965 · ndcg at 10: 0.4103 **Performance (L4-SPOT b1 c16):** Corpus 3.4K tok/s · Corpus p50 384.8ms · Query 174 tok/s · Query p50 547.3ms **Performance (L4 b1 c16):** Corpus 43.1K tok/s · Corpus p50 59.1ms · Query 3.7K tok/s · Query p50 50.0ms **Variant: muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.2223 · mrr at 10: 0.3540 · ndcg at 10: 0.2900 [Reference](https://sites.google.com/view/fiqa/) ### LegalBenchConsumerContractsQA Domain: legal · Task: retrieval · Language: en Question answering on consumer contracts Corpus: 153 · Queries: 396 **Variant: default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.7315 · mrr at 10: 0.7315 · ndcg at 10: 0.7840 **Performance (L4-SPOT b1 c16):** Corpus 11.2K tok/s · Corpus p50 286.1ms · Query 254 tok/s · Query p50 500.2ms **Performance (L4 b1 c16):** Corpus 83.1K tok/s · Corpus p50 83.9ms · Query 4.8K tok/s · Query p50 52.2ms **Variant: muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.6346 · mrr at 10: 0.6346 · ndcg at 10: 0.6991 [Reference](https://huggingface.co/datasets/nguha/legalbench) ### MMarcoReranking Domain: general · Task: reranking · Language: zh Multilingual MARCO passage reranking (Chinese) **Variant: muvera** **Quality:** ndcg at 10: 0.0725 · map at 10: 0.0502 · mrr at 10: 0.0507 **Variant: default** **Quality:** map at 10: 0.0602 · mrr at 10: 0.0602 [Reference](https://arxiv.org/abs/2304.03679) ### NFCorpus Domain: medical · Task: retrieval · Language: en Biomedical literature search from NutritionFacts.org Corpus: 3,593 · Queries: 323 **Variant: default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.1437 · mrr at 10: 0.5860 · ndcg at 10: 0.3706 **Performance (L4-SPOT b1 c16):** Corpus 5.7K tok/s · Corpus p50 300.1ms · Query 210 tok/s · Query p50 178.7ms **Performance (L4 b1 c16):** Corpus 60.7K tok/s · Corpus p50 69.4ms · Query 1.4K tok/s · Query p50 52.3ms **Variant: muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.1186 · mrr at 10: 0.5151 · ndcg at 10: 0.3192 [Reference](https://www.cl.uni-heidelberg.de/statnlpgroup/nfcorpus/) ### SCIDOCS Domain: scientific · Task: retrieval · Language: en Citation prediction, document classification, and recommendation for scientific papers Corpus: 25,656 · Queries: 1,000 **Variant: default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.1045 · mrr at 10: 0.3076 · ndcg at 10: 0.1777 **Performance (L4-SPOT b1 c16):** Corpus 2.9K tok/s · Corpus p50 501.5ms · Query 222 tok/s · Query p50 353.0ms **Performance (L4 b1 c16):** Corpus 48.4K tok/s · Corpus p50 57.9ms · Query 3.8K tok/s · Query p50 46.7ms **Variant: muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.0973 · mrr at 10: 0.2854 · ndcg at 10: 0.1681 [Reference](https://allenai.org/data/scidocs) ### SciFact Domain: scientific · Task: retrieval · Language: en Scientific claim verification using research literature Corpus: 5,183 · Queries: 300 **Variant: default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.7014 · mrr at 10: 0.7120 · ndcg at 10: 0.7404 **Performance (L4-SPOT b1 c16):** Corpus 4.9K tok/s · Corpus p50 443.1ms · Query 332 tok/s · Query p50 430.3ms **Performance (L4 b1 c16):** Corpus 61.5K tok/s · Corpus p50 63.3ms · Query 5.2K tok/s · Query p50 50.5ms **Variant: muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.6522 · mrr at 10: 0.6645 · ndcg at 10: 0.6933 [Reference](https://github.com/allenai/scifact) ### StackOverflowQA Domain: technology · Task: retrieval · Language: en Programming question answering from Stack Overflow Corpus: 19,931 · Queries: 1,994 **Variant: default_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.5125 · mrr at 10: 0.5125 · ndcg at 10: 0.5457 **Performance (L4-SPOT b1 c16):** Corpus 4.4K tok/s · Corpus p50 379.7ms · Query 5.2K tok/s · Query p50 432.3ms **Performance (L4 b1 c16):** Corpus 55.3K tok/s · Corpus p50 60.2ms · Query 73.2K tok/s · Query p50 64.7ms **Variant: muvera_candidates-k-50_candidates-model-Alibaba-NLP__gte-multilingual-base** **Quality:** map at 10: 0.4498 · mrr at 10: 0.4498 · ndcg at 10: 0.4880 [Reference](https://arxiv.org/abs/2407.02883) ### T2Reranking Domain: general · Task: reranking · Language: zh Chinese passage ranking benchmark **Variant: muvera** **Quality:** ndcg at 10: 0.6591 · map at 10: 0.4793 · mrr at 10: 0.6847 **Variant: default** **Quality:** ndcg at 10: 0.6821 · map at 10: 0.5041 · mrr at 10: 0.7178 [Reference](https://arxiv.org/abs/2304.03679)