Llamacpp imatrix Quantizations of Ministral 8B Instruct 2410 This is based on the officially merged safetensors for Ministral, however there may still be changes required to llama.cpp for full performance Using llama.cpp release b3930 for quantization. Original model: https://huggingface.co/mistralai/Ministral 8B Instruct 2410 All quants made using imatrix option with dataset from here Run them in LM Studio Prompt format What's new: Update to official repo Download a file (not the whole branch) from below: Filename Quant type File Size Split Description Ministral 8B Instruct 2410 f16.gguf f16 16.05GB false Full F16 weights. Ministral 8B Instruct 2410 Q8 0.gguf Q8 0 8.53GB false Extremely high quality, generally unneeded but max available quant. Ministral 8B Instruct 2410 Q6 K L.gguf Q6 K L 6.85GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . Ministral 8B Instruct 2410 Q6 K.gguf Q6 K 6.59GB false Very high quality, near perfect, recommended . Ministral 8B Instruct 2410 Q5 K L.gguf Q5 K L 6.06GB false Uses Q8 0 for embed and output weights. High quality, recommended . Ministral 8B Instruct 2410 Q5 K M.gguf Q5 K M 5.72GB false High qualit…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy