Qwen3.5 9B NVFP4 This is a quantized version of Qwen/Qwen3.5 9B. This model accepts text and images as inputs and generates text as outputs. The weights and activations were quantized to FP4 using llm compressor with 512 calibration samples from nvidia/Nemotron Post Training Dataset v2, reducing the model size from 18.0 GB to 11.5 GB (~1.6x reduction) while maintaining 97.3% average accuracy recovery. Quantization Details Scheme : NVFP4 Calibration : 512 samples (256 reasoning on + 256 reasoning off) from Nemotron Post Training Dataset v2 Max sequence length : 4096 Inference This model is supported in vLLM 0.17.0. To serve the model: Evaluation Evaluated with lm evaluation harness, 0 shot, thinking mode ON. Benchmark Qwen3.5 9B Qwen3.5 9B NVFP4 (this model) Recovery GPQA Diamond 78.79% 74.24% 94.2% IFEval 94.48% 92.69% 98.1% MMLU Redux 91.80% 91.39% 99.6% Average 88.36% 86.11% 97.3%
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy