huihui ai/Huihui Ornith 1.0 9B abliterated MTP GGUF This is an uncensored version of deepreinforce ai/Ornith 1.0 9B created with abliteration (see remove refusals with transformers to know more about it). This is a crude, proof of concept implementation to remove refusals from an LLM model without using TransformerLens. Note We will directly perform ablation on the GGUF files from protoLabsAI/Ornith 1.0 9B MTP GGUF. Some weights quantized in Q3 K, Q4 K, Q5 K and Q6 K will be converted to Q8 0. This time, the weights of the first 5 layers, so the final weights will increase, but the change will not be significant. We performed a simple quantization comparison between Q8 0 and MXFP4 on the Q3 version of protoLabsAI/Ornith 1.0 9B MTP GGUF. Clearly, Q8 0 has the largest weight size, but its PPL is smaller than the original. While MXFP4 can reduce the model weight size, it increases the PPL. name Size PPL weight Ornith 1.0 9B MTP GGUF/ornith 9b mtp kl IQ3 M.gguf 4.67GB 7.2065 +/ 0.04554 Q6 K(output), IQ3 S(ffn down/ssm out) Huihui Ornith 1.0 9B abliterated MTP GGUF Q8/ornith 9b mtp kl IQ3 M.gguf 5.99GB 7.1733 +/ 0.04588 Q8 0(output/ffn down/ssm out) Huihui Ornith 1.0 9B abliterated MTP…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy