InstinctRazor — Qwen3.5 122B A10B · IQ3 XXS GGUF A 122B hybrid Gated DeltaNet MoE (256 experts, 8 active) — packed to 48 GiB so it runs on one 80 GB GPU (or a small card + CPU offload). Quantized from the original BF16 with an importance matrix (math + code + general calibration), via llama.cpp. Framework, recipe, and full reproduction: https://github.com/General Instinct/InstinctRazor Speed (llama.cpp, this artifact) 1× H100 80GB , all layers on GPU: 115.9 tok/s decode (prefill ≈2541 tok/s). Small card + CPU expert offload ( n cpu moe 48 , peak ≈7.6 GiB VRAM): 45.7 tok/s decode — runs on an 8 GB GPU + ≈48 GiB system RAM. Run Requires a llama.cpp build with qwen3 5 moe support (upstream, 2026 02+). Scope & roadmap This GGUF matches or beats the footprint matched A4B on knowledge, reasoning, and multimodal MMMU. Where it still trails — code (LiveCodeBench v6) and math / multimodal math — the loss is largely token inefficiency introduced by quantization, and is the target of OPD (on policy distillation) , a separate framework we'll open source later. Eval absolutes are subject to a same harness validation gate; see the GitHub results/RESULTS.md for full per number provenance. Attribu…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy