jina reranker v3 GGUF GGUF quantizations of jina reranker v3 using llama.cpp. A 0.6B parameter multilingual listwise reranker quantized for efficient inference. Requirements Python 3.8+ llama.cpp binaries ( llama embedding and llama tokenize ) Hanxiao's llama.cpp fork recommended: https://github.com/hanxiao/llama.cpp Installation Files jina reranker v3 BF16.gguf Quantized model weights (BF16, 1.1GB) projector.safetensors MLP projector weights (3MB) rerank.py Reranker implementation Usage API GGUFReranker.rerank(query, documents, top n=None, return embeddings=False, instruction=None) Arguments: query (str): Search query documents (List[str]): Documents to rerank top n (int, optional): Return only top N results return embeddings (bool): Include embeddings in output instruction (str, optional): Custom ranking instruction Returns: List of dicts with keys: index , relevance score , document , and optionally embedding Citation If you find jina reranker v3 useful in your research, please cite the original paper: License This MLX implementation follows the same CC BY NC 4.0 license as the original model. For commercial usage inquiries, please contact Jina AI.
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy