GLM 5.2 W4AFP8 中文文档（Chinese documentation）：ModelScope 模型卡 GLM 5.2 W4AFP8 is a 4 bit quantization of GLM 5.2 that runs the full 1M token context on a single 8×H200 node. Two things make it useful: The full 1M context fits on one node. Quantizing the Mixture of Experts weights to 4 bit cuts their footprint from 755 GB to 368 GB, which frees enough memory for the complete 1M token KV cache. The FP8 release reaches only about 500K tokens on the same hardware. No measurable quality loss. Reasoning, instruction following, long context, and tool calling benchmarks all land within measurement noise of the FP8 model. [!NOTE] This model keeps GLM 5.2's full 753B parameters. The ~387B in the sidebar is a counting quirk: two 4 bit weights share each int8 slot, so the metadata counts storage slots, not parameters. Tested with SGLang only (v0.5.13.post1). The 4 bit layout and speculative decoding path are SGLang specific and untried on vLLM or other engines. Benchmarks Measured on 8×H200 with SGLang at GLM 5.2's recommended sampling (temperature 1.0, top p 0.95) with reasoning enabled. The reference column shows the published GLM 5.2 results. Benchmark GLM 5.2 W4AFP8 GLM 5.2 reference GPQA Diamo…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy