Llamacpp imatrix Quantizations of Hunyuan 7B Instruct by tencent Using llama.cpp release b6076 for quantization. Original model: https://huggingface.co/tencent/Hunyuan 7B Instruct All quants made using imatrix option with dataset from here Run them in LM Studio Run them directly with llama.cpp, or any other llama.cpp based project Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Split Description Hunyuan 7B Instruct bf16.gguf bf16 15.01GB false Full BF16 weights. Hunyuan 7B Instruct Q8 0.gguf Q8 0 7.98GB false Extremely high quality, generally unneeded but max available quant. Hunyuan 7B Instruct Q6 K L.gguf Q6 K L 6.29GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . Hunyuan 7B Instruct Q6 K.gguf Q6 K 6.16GB false Very high quality, near perfect, recommended . Hunyuan 7B Instruct Q5 K L.gguf Q5 K L 5.50GB false Uses Q8 0 for embed and output weights. High quality, recommended . Hunyuan 7B Instruct Q5 K M.gguf Q5 K M 5.37GB false High quality, recommended . Hunyuan 7B Instruct Q5 K S.gguf Q5 K S 5.23GB false High quality, recommended . Hunyuan 7B Instruct Q4 1.gguf Q4 1 4.80GB false Legacy f…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy