Llamacpp imatrix Quantizations of Dolphin Mistral 24B Venice Edition by cognitivecomputations Using llama.cpp release b5835 for quantization. Original model: https://huggingface.co/cognitivecomputations/Dolphin Mistral 24B Venice Edition All quants made using imatrix option with dataset from here Run them in LM Studio Run them directly with llama.cpp, or any other llama.cpp based project Prompt format What's new: Original model updated Download a file (not the whole branch) from below: Filename Quant type File Size Split Description Dolphin Mistral 24B Venice Edition bf16.gguf bf16 47.15GB false Full BF16 weights. Dolphin Mistral 24B Venice Edition Q8 0.gguf Q8 0 25.05GB false Extremely high quality, generally unneeded but max available quant. Dolphin Mistral 24B Venice Edition Q6 K L.gguf Q6 K L 19.67GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . Dolphin Mistral 24B Venice Edition Q6 K.gguf Q6 K 19.35GB false Very high quality, near perfect, recommended . Dolphin Mistral 24B Venice Edition Q5 K L.gguf Q5 K L 17.18GB false Uses Q8 0 for embed and output weights. High quality, recommended . Dolphin Mistral 24B Venice Edition Q5 K M.gg…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy