openchat 3.6 8b 20240522 IMat GGUF Llama.cpp imatrix quantization of openchat/openchat 3.6 8b 20240522 Original Model: openchat/openchat 3.6 8b 20240522 Original dtype: BF16 ( bfloat16 ) Quantized by: llama.cpp b3006 IMatrix dataset: here openchat 3.6 8b 20240522 IMat GGUF Files IMatrix Common Quants All Quants Downloading using huggingface cli Inference Simple chat template Chat template with system prompt Llama.cpp FAQ Why is the IMatrix not applied everywhere? How do I merge a split GGUF? Files IMatrix Status: ✅ Available Link: here Common Quants Filename Quant type File Size Status Uses IMatrix Is Split openchat 3.6 8b 20240522.Q8 0.gguf Q8 0 8.54GB ✅ Available ⚪ Static 📦 No openchat 3.6 8b 20240522.Q6 K.gguf Q6 K 6.60GB ✅ Available ⚪ Static 📦 No openchat 3.6 8b 20240522.Q4 K.gguf Q4 K 4.92GB ✅ Available 🟢 IMatrix 📦 No openchat 3.6 8b 20240522.Q3 K.gguf Q3 K 4.02GB ✅ Available 🟢 IMatrix 📦 No openchat 3.6 8b 20240522.Q2 K.gguf Q2 K 3.18GB ✅ Available 🟢 IMatrix 📦 No All Quants Filename Quant type File Size Status Uses IMatrix Is Split openchat 3.6 8b 20240522.FP16.gguf F16 16.07GB ✅ Available ⚪ Static 📦 No openchat 3.6 8b 20240522.BF16.gguf BF16 16.07GB ✅ Available ⚪ Sta…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy