Qwopus3.5 122B A10B Kimi-K2.6 Distill Healed Abliterated - Custom GGUF Quantizations

CRITICAL COMPATIBILITY WARNING

These are iqk format quantizations and are EXCLUSIVE to the ik_llama.cpp fork.

They will NOT work on mainline llama.cpp, standard LM Studio, standard Text Generation WebUI, or KoboldCPP.

You must compile and run this using ikawrakow's llama.cpp fork, or a UI where you have manually swapped the backend to an ik_llama.cpp build.

This repository contains custom, mixed-precision ik_llama.cpp GGUF quantizations for OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated, a Kimi-K2.6 distilled, healed, abliterated Qwen3.5 122B A10B MoE model.

These quants use different precision levels for different layer types, keeping attention, SSM, shared expert, output, and MTP/NextN tensors at higher precision while compressing the routed experts, which make up the bulk of the model's size.

⚠️ Disclaimer: The "Vibes Test"

These quantizations have NOT been formally tested for perplexity.

They were compiled as an experiment to see how the model handles shifting bottlenecks. There is no guarantee that they are mathematically optimal or perform flawlessly.

If they pass the vibes test for you, enjoy!

Credits & Acknowledgments

• Base model: OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated
• Functional MTP discussion: OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated/discussions/2
• imatrix source: The imatrix was sourced from mradermacher/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated-i1-GGUF and converted from GGUF to legacy .dat format for ik_llama.cpp compatibility.
• Community chat template: froggeric/Qwen-Fixed-Chat-Templates
• Quantization recipes: Heavily based on the blending logic from ubergarm/Qwen3.5-122B-A10B-GGUF.

Quantization Recipes

All variants use the same custom tensor buckets: attention, SSM, shared experts, routed experts, embeddings/output, and MTP/NextN tensors.

IQ6_K

Highest quality routed expert quantization in this set.

IQ5_K

High quality routed expert quantization with IQ5_K experts.

IQ5_KS

High quality routed expert quantization using IQ5_KS experts.

IQ4_K

Balanced 4-bit routed expert quantization with high precision on always-active tensors.

IQ4_KS

Smaller 4-bit routed expert quantization with compressed embeddings, output, and MTP tensors.

IQ4_KSS

Ubergarm-style split routed expert recipe.

IQ3_K

Lower size recipe with IQ3_K routed experts and IQ6_K on many always-active tensors.

IQ2_KL

Maximum compression variant in this set.

How to Run

1. Clone and build the ik_llama.cpp fork from ikawrakow/ik_llama.cpp.
2. Use the compiled llama-server or llama-cli from that specific build.
3. For chat templating, use the model's embedded template or the community template credited above, depending on your frontend.

Example llama-server launch command:

./llama-server -m Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated-IQ4_KS.gguf -c 8192 -ngl 99 -fa --jinja