Qwen3VL 8B Instruct NVFP4 This is an NVFP4 quantized version of Qwen3 VL 8B Instruct, a powerful vision language model for multimodal understanding and generation tasks. Model Details Base Model Base Model : Qwen/Qwen3 VL 8B Instruct Architecture : Qwen3VLForConditionalGeneration Model Type : Vision Language Model (VLM) Quantization Details Quantization Method : NVFP4 with SmoothQuant Quantization Tool : llmcompressor Smoothing Strength : 0.8 Calibration Dataset : lmms lab/flickr30k (512 samples from test split) Max Sequence Length : 32,768 tokens Quantization Configuration Weight Quantization : NVFP4 (4 bit NVIDIA Floating Point) Strategy: Channel wise Observer: MinMax Symmetric: True Activation Quantization : NVFP4 Strategy: Token wise Dynamic scaling: Enabled Symmetric: True Excluded Modules The following modules were excluded from quantization to maintain model quality: lm head (language model head) Visual encoder modules ( model.visual. ) MLP gate projections ( . mlp.gate$ ) Model Use Installation Basic Usage Using with vLLM For faster inference, you can use this model with vLLM: Performance Memory Benefits Reduced Memory Footprint : NVFP4 quantization significantly reduces mo…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy