Gemma 4 31B IT Abliterated (Q4 K M GGUF) Abliterated variant of google/gemma 4 31b it with refusal behavior removed using heretic. Removes ~64% of refusals while preserving model capabilities (KL divergence 0.27). Files File Size Description gemma 4 31b it abliterated t126 Q4 K M.gguf 18 GB Q4 K M quantization Abliteration Method Tool: heretic v1.2.0 Approach: Bayesian optimized refusal direction removal via LoRA based weight modification Optimization: 200 trials (60 random exploration + 140 TPE guided), selected from Pareto front balancing refusal count vs KL divergence Targets: 120 modules across 60 layers ( self attn.o proj + mlp.down proj ) Datasets: 400 harmful vs 400 harmless prompts (mlabonne/harmful behaviors, mlabonne/harmless alpaca) Hardware: NVIDIA H100 80GB, ~1 hour optimization Gemma 4 Compatibility Note Gemma 4's vision encoder uses Gemma4ClippableLinear layers not supported by PEFT. Resolved by restricting LoRA targeting to language model layers via full module paths instead of leaf name matching. Usage Quantization Format: GGUF Q4 K M (4.87 BPW) Size: ~18GB Target hardware: RTX 4090 (24GB), RTX 5090 (32GB), any GPU with 20GB+ VRAM
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy