Qwopus3.6 27B v2 NVFP4 Parameters: 27B (base model) Quantized to NVFP4 Vision Encoder + MTP kept in BF16 Base Model : Jackrong/Qwopus3.6 27B v2 Overview This repository contains a quantized version of Qwopus3.6 27B v2 (based on the Qwen3.5 architecture). The model has been quantized from BF16 to NVFP4 format using the NVIDIA TensorRT Model Optimizer ( modelopt ). To maintain high performance and accuracy, a hybrid precision strategy was applied: the core LLM weights are quantized to 4 bit, while the Vision Encoder and Multi Token Prediction (MTP) modules are strictly preserved in their native BF16 precision. The resulting checkpoint is fully compatible with vLLM for high throughput, low latency inference. Hardware Recommendation Target Hardware: Highly recommended for NVIDIA Blackwell architecture GPUs (e.g., NVIDIA RTX 50 series, RTX PRO series, and GB100/GB10 chips) which feature native hardware support for FP4. Model Specifications Original Size: ~54 GB (BF16) Quantized Size: ~18 GB (~0.33x compression ratio) Format: Hugging Face safetensors (compatible with vLLM via quantization modelopt ) Preserved Modules: Vision tower and MTP heads remain in BF16 to prevent degradation in im…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy