Llamacpp imatrix Quantizations of DeepSeek R1 Distill Qwen 1.5B Using llama.cpp release b4514 for quantization. Original model: https://huggingface.co/deepseek ai/DeepSeek R1 Distill Qwen 1.5B All quants made using imatrix option with dataset from here Run them in LM Studio Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Split Description DeepSeek R1 Distill Qwen 1.5B f32.gguf f32 7.11GB false Full F32 weights. DeepSeek R1 Distill Qwen 1.5B f16.gguf f16 3.56GB false Full F16 weights. DeepSeek R1 Distill Qwen 1.5B Q8 0.gguf Q8 0 1.89GB false Extremely high quality, generally unneeded but max available quant. DeepSeek R1 Distill Qwen 1.5B Q6 K L.gguf Q6 K L 1.58GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . DeepSeek R1 Distill Qwen 1.5B Q6 K.gguf Q6 K 1.46GB false Very high quality, near perfect, recommended . DeepSeek R1 Distill Qwen 1.5B Q5 K L.gguf Q5 K L 1.43GB false Uses Q8 0 for embed and output weights. High quality, recommended . DeepSeek R1 Distill Qwen 1.5B Q5 K M.gguf Q5 K M 1.29GB false High quality, recommended . DeepSeek R1 Distill Qwen 1.5B Q4 K L.gguf Q4 K L 1.29GB false Us…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy