Llamacpp imatrix Quantizations of Dolphin3.0 R1 Mistral 24B by cognitivecomputations Using llama.cpp release b4585 for quantization. Original model: https://huggingface.co/cognitivecomputations/Dolphin3.0 R1 Mistral 24B All quants made using imatrix option with dataset from here Run them in LM Studio Run them directly with llama.cpp, or any other llama.cpp based project Prompt format Recommended reasoning system prompt For reasoning, it's recommended you use the following system prompt: Download a file (not the whole branch) from below: Filename Quant type File Size Split Description Dolphin3.0 R1 Mistral 24B f32.gguf f32 94.30GB true Full F32 weights. Dolphin3.0 R1 Mistral 24B Q8 0.gguf Q8 0 25.05GB false Extremely high quality, generally unneeded but max available quant. Dolphin3.0 R1 Mistral 24B Q6 K L.gguf Q6 K L 19.67GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . Dolphin3.0 R1 Mistral 24B Q6 K.gguf Q6 K 19.35GB false Very high quality, near perfect, recommended . Dolphin3.0 R1 Mistral 24B Q5 K L.gguf Q5 K L 17.18GB false Uses Q8 0 for embed and output weights. High quality, recommended . Dolphin3.0 R1 Mistral 24B Q5 K M.gguf Q5…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy