SPLADE Mini This is a SPLADE sparse retrieval model based on BERT Mini (11M) that was trained by distilling a Cross Encoder on the MSMARCO dataset. The cross encoder used was ms marco MiniLM L6 v2. This tiny SPLADE model is 6x smaller than Naver's official splade v3 distilbert while having 85% of it's performance on the MSMARCO benchmark. This model is small enough to be used without a GPU on a dataset of a few thousand documents. Collection: https://huggingface.co/collections/rasyosef/splade tiny msmarco 687c548c0691d95babf65b70 Distillation Dataset: https://huggingface.co/datasets/yosefw/msmarco train distil v2 Code: https://github.com/rasyosef/splade tiny msmarco Performance The splade models were evaluated on 55 thousand queries and 8.84 million documents from the MSMARCO dataset. Size ( Params) MRR@10 (MS MARCO dev) : : : BM25 18.0 rasyosef/splade tiny 4.4M 30.9 rasyosef/splade mini 11.2M 34.1 rasyosef/splade small 28.8M 35.4 naver/splade v3 distilbert 67.0M 38.7 Usage Direct Usage (Sentence Transformers) First install the Sentence Transformers library: Then you can load this model and run inference. Click to see the direct usage in Transformers Click to expand Model Details M…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy