GLM 5.2 — "e waste edition" GGUF Imatrix GGUF quantizations of GLM 5.2 (745B total / ~40B active, glm dsa arch, DeepSeek style MoE with 256 routed + 1 shared expert, 8 active/token, MLA attention) tuned for MoE inference on older / "e waste" hardware — pre AVX 512 CPUs and previous gen datacenter GPUs (MI100 / gfx908). The guiding choice across every build ( Q4 K / Q3 K / Q2 K ) is K quants for the routed experts, not codebook i quants (IQ2/IQ3) : K quant dequantization is dramatically faster on pre AVX 512 CPUs and on gfx908, where expert dequant — not bandwidth — is the decode bottleneck. An i quant that looks smaller on paper is slower to run on this class of hardware. (The experimental Q2 K XL build keeps the same principle — experts stay Q2 K — except for its 13 coldest, rarely fired layers; see below.) Available quantizations Variant Size Routed experts Non experts Target GLM 5.2 Q4 K XL 403.8 GiB Q4 K Q8 0 Highest quality ; big RAM CPU expert boxes (≈512 GB). Too big for 10×32 GB 0 spill GLM 5.2 Q3 K XL 312.9 GiB Q3 K Q8 0 Max quality on GPU; experts spill to CPU on ub 2048 is a free +45% on prefill here purely because the 0 spill fit leaves ~20 GiB of VRAM headroom to spend…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy