Carnice V2 27B GGUF GGUF exports for kai os/carnice v2 27b , a merged BF16 SFT of Qwen/Qwen3.6 27B for Hermes style agent traces. Recommended Files File Size class Use : carnice v2 27b IQ2 M.gguf 9.4GB Best 16GB GPU target. Built with a Carnice/Hermes imatrix calibration pass. carnice v2 27b Q2 K.gguf 10GB Safest 16GB GPU fallback. More compatible than IQ quants, lower quality than imatrix IQ2 M. carnice v2 27b Q4 K M.gguf 16GB Balanced local quality tier. May require shorter context or partial CPU offload on a 16GB GPU. carnice v2 27b Q5 K M.gguf 18GB Better quality tier for 24GB+ or split/offload setups. carnice v2 27b Q8 0.gguf 27GB Near lossless quant tier for high memory systems. carnice v2 27b bf16.gguf 51GB Full BF16 GGUF export. For a 16GB GPU, start with IQ2 M if your runtime supports IQ quants and this Qwen3.5/Qwen3.6 GGUF architecture. If the runtime is older or fails to load IQ quants, use Q2 K . Benchmarks From The Source SFT Metric Qwen3.6 27B base Carnice SFT : : IFEval prompt strict, limit 20 85.0% 90.0% IFEval prompt loose, limit 20 85.0% 90.0% IFEval instruction strict, limit 20 90.0% 93.3% IFEval instruction loose, limit 20 90.0% 93.3% Held out assistant token ev…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy