gemma 4 31B it uncensored heretic FP8 FP8 block quantized version of llmfan46/gemma 4 31B it uncensored heretic. Quantization Details Method: FP8 BLOCK (weight block size=[128,128], activation scheme=dynamic) Tool: llmcompressor model free ptq Format: compressed tensors (vLLM native) Ignored layers: vision tower, lm head, embedding layers Model size: ~31.5 GB (vs ~62 GB BF16) Evaluation Results Compared against RedHatAI/gemma 4 31B it FP8 block using identical evaluation settings (lm eval 0.4.11, vLLM, 2 seeds averaged). Benchmark This Model RedHatAI FP8 Delta GSM8k Platinum (5 shot) 97.68% 97.82% 0.14 MMLU CoT (5 shot) 89.97% 90.70% 0.73 MMLU Pro (5 shot) 84.52% 84.92% 0.40 IFEval inst strict 93.41% 93.84% 0.43 IFEval prompt strict 90.76% 91.31% 0.55 Generation parameters: temperature=1.0, top p=0.95, top k=64, max gen toks=8192 Usage Disclaimer This is an uncensored model. The quantizer (kakrotto) is not responsible for the model's outputs or any misuse. This FP8 quantization preserves the original model's behavior. Please use responsibly. Attribution Source model: llmfan46/gemma 4 31B it uncensored heretic Quantization recipe reference: RedHatAI/gemma 4 31B it FP8 block
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy