Llamacpp imatrix Quantizations of QwQ 32B ArliAI RpR v4 by ArliAI Using llama.cpp release b5432 for quantization. Original model: https://huggingface.co/ArliAI/QwQ 32B ArliAI RpR v4 All quants made using imatrix option with dataset from here Run them in LM Studio Run them directly with llama.cpp, or any other llama.cpp based project Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Split Description QwQ 32B ArliAI RpR v4 bf16.gguf bf16 65.54GB true Full BF16 weights. QwQ 32B ArliAI RpR v4 Q8 0.gguf Q8 0 34.82GB false Extremely high quality, generally unneeded but max available quant. QwQ 32B ArliAI RpR v4 Q6 K L.gguf Q6 K L 27.26GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . QwQ 32B ArliAI RpR v4 Q6 K.gguf Q6 K 26.89GB false Very high quality, near perfect, recommended . QwQ 32B ArliAI RpR v4 Q5 K L.gguf Q5 K L 23.74GB false Uses Q8 0 for embed and output weights. High quality, recommended . QwQ 32B ArliAI RpR v4 Q5 K M.gguf Q5 K M 23.26GB false High quality, recommended . QwQ 32B ArliAI RpR v4 Q5 K S.gguf Q5 K S 22.64GB false High quality, recommended . QwQ 32B ArliAI RpR v4 Q4 1.gguf Q4…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy