pekkAi/Gemma 4 12B it abliterated NVFP4 This model pekkAi/Gemma 4 12B it abliterated NVFP4 was quantized to NVFP4 format from huihui ai/Huihui gemma 4 12B it abliterated using Model Optimizer. Static NVFP4 weight scales from MSE FP8 scale sweep and dynamic NVFP4 inputs to MLP/MoE layers, plus FP8 KV cache cast mode. Tested working on RTX 5090 32Gb py3.13+torch2.11+cu130 Serving (RTX 5090, 32GB) Installing SGLang with modelopt ptq support (Thanks to AxionML) Original Model Card huihui ai/Huihui gemma 4 12B it abliterated This is an uncensored version of google/gemma 4 12B it created with abliteration (see remove refusals with transformers to know more about it). This is a crude, proof of concept implementation to remove refusals from an LLM model without using TransformerLens. Note For this model, both the thinking mode and the non thinking mode have been completely abliterated. Only layers 23 28 have been abliterated. Important Notice : The weights of the earliest released google/gemma 4 12B it model have issues. We have already re ablated and re uploaded the model. If you have already downloaded it, please re download. Usage Warnings Risk of Sensitive or Controversial Outputs : Th…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy