Huihui gemma 4 31B it abliterated v2 NVFP4 This repository contains an NVFP4 compressed version of huihui ai/Huihui gemma 4 31B it abliterated v2. The goal of this build is to produce an NVFP4 checkpoint that actually fits on a single 32 GB RTX 5090 while preserving the multimodal pipeline and the abliteration intervention. The existing NVIDIA reference release keeps every self attn layer in BF16, which pushes the on disk weights past 32 GB and makes single GPU serving impossible. This build quantizes self attn as well, so only the vision tower, embeddings, and lm head remain in BF16. What Changed In v2 (vs v1) The source model ( huihui ai/... abliterated v2 ) differs from v1 in one key way: the first 5 text layers (layers 0–4) are no longer abliterated . They retain the original google/gemma 4 31B it weights. Layers 5–59 are still abliterated but with a refusal direction recomputed excluding those early layers. Per the source model card, this produces lower perplexity (better quality) while maintaining the same level of refusal removal: Model PPL Gap vs base google/gemma 4 31B it (base) 14874.75 — v1 abliterated 13335.55 1539 v2 abliterated 13161.29 1713 (v1 − 174) In local side b…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy