Llamacpp imatrix Quantizations of Qwen2.5 VL 7B NSFW Caption V3 by thesby Using llama.cpp release b5849 for quantization. Original model: https://huggingface.co/thesby/Qwen2.5 VL 7B NSFW Caption V3 All quants made using imatrix option with dataset from here Run them in LM Studio Run them directly with llama.cpp, or any other llama.cpp based project Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Split Description Qwen2.5 VL 7B NSFW Caption V3 bf16.gguf bf16 15.24GB false Full BF16 weights. Qwen2.5 VL 7B NSFW Caption V3 Q8 0.gguf Q8 0 8.10GB false Extremely high quality, generally unneeded but max available quant. Qwen2.5 VL 7B NSFW Caption V3 Q6 K L.gguf Q6 K L 6.52GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . Qwen2.5 VL 7B NSFW Caption V3 Q6 K.gguf Q6 K 6.25GB false Very high quality, near perfect, recommended . Qwen2.5 VL 7B NSFW Caption V3 Q5 K L.gguf Q5 K L 5.78GB false Uses Q8 0 for embed and output weights. High quality, recommended . Qwen2.5 VL 7B NSFW Caption V3 Q5 K M.gguf Q5 K M 5.44GB false High quality, recommended . Qwen2.5 VL 7B NSFW Caption V3 Q5 K S.gguf Q5 K S 5.32GB fa…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy