Llamacpp imatrix Quantizations of DeepSeek R1 Distill Qwen 7B Using llama.cpp release b4514 for quantization. Original model: https://huggingface.co/deepseek ai/DeepSeek R1 Distill Qwen 7B All quants made using imatrix option with dataset from here Run them in LM Studio Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Split Description DeepSeek R1 Distill Qwen 7B f32.gguf f32 30.47GB false Full F32 weights. DeepSeek R1 Distill Qwen 7B f16.gguf f16 15.24GB false Full F16 weights. DeepSeek R1 Distill Qwen 7B Q8 0.gguf Q8 0 8.10GB false Extremely high quality, generally unneeded but max available quant. DeepSeek R1 Distill Qwen 7B Q6 K L.gguf Q6 K L 6.52GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . DeepSeek R1 Distill Qwen 7B Q6 K.gguf Q6 K 6.25GB false Very high quality, near perfect, recommended . DeepSeek R1 Distill Qwen 7B Q5 K L.gguf Q5 K L 5.78GB false Uses Q8 0 for embed and output weights. High quality, recommended . DeepSeek R1 Distill Qwen 7B Q5 K M.gguf Q5 K M 5.44GB false High quality, recommended . DeepSeek R1 Distill Qwen 7B Q5 K S.gguf Q5 K S 5.32GB false High quality, recomm…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy