Llamacpp imatrix Quantizations of DeepSeek R1 0528 Qwen3 8B by deepseek ai Using llama.cpp release b5524 for quantization. Original model: https://huggingface.co/deepseek ai/DeepSeek R1 0528 Qwen3 8B All quants made using imatrix option with dataset from here Run them in LM Studio Run them directly with llama.cpp, or any other llama.cpp based project Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Split Description DeepSeek R1 0528 Qwen3 8B bf16.gguf bf16 16.39GB false Full BF16 weights. DeepSeek R1 0528 Qwen3 8B Q8 0.gguf Q8 0 8.71GB false Extremely high quality, generally unneeded but max available quant. DeepSeek R1 0528 Qwen3 8B Q6 K L.gguf Q6 K L 7.03GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . DeepSeek R1 0528 Qwen3 8B Q6 K.gguf Q6 K 6.73GB false Very high quality, near perfect, recommended . DeepSeek R1 0528 Qwen3 8B Q5 K L.gguf Q5 K L 6.24GB false Uses Q8 0 for embed and output weights. High quality, recommended . DeepSeek R1 0528 Qwen3 8B Q5 K M.gguf Q5 K M 5.85GB false High quality, recommended . DeepSeek R1 0528 Qwen3 8B Q5 K S.gguf Q5 K S 5.72GB false High quality, recommen…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy