opensearch neural sparse encoding multilingual v1 Select the model The model should be selected considering search relevance, model inference and retrieval efficiency(FLOPS). We benchmark models' performance on MIRACL benchmark (we exclude th since the uncased backbone can not encode it). We recommend to use it with max ratio pruning. Model Inference free for Retrieval Model Parameters AVG NDCG@10 AVG FLOPS AVG EMB SIZE opensearch neural sparse encoding multilingual v1 ✔️ 160M 0.629 1.3 138 opensearch neural sparse encoding multilingual v1; prune ratio 0.1 ✔️ 160M 0.626 0.8 75 Overview Paper : Towards Competitive Search Relevance For Inference Free Learned Sparse Retrievers Fine tuning sample : opensearch sparse model tuning sample This is a learned sparse retrieval model. It encodes the documents to 105879 dimensional sparse vectors . For queries, it just use a tokenizer and a weight look up table to generate sparse vectors. The non zero dimension index means the corresponding token in the vocabulary, and the weight means the importance of the token. And the similarity score is the inner product of query/document sparse vectors. OpenSearch neural sparse feature supports learned sp…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy