Qwen3 8B FP8 dynamic Model Overview Model Architecture: Qwen3ForCausalLM Input: Text Output: Text Model Optimizations: Activation quantization: FP8 Weight quantization: FP8 Intended Use Cases: Reasoning. Function calling. Subject matter experts via fine tuning. Multilingual instruction following. Translation. Out of scope: Use in any manner that violates applicable laws or regulations (including trade compliance laws). Release Date: 05/02/2025 Version: 1.0 Validated on: RHOAI 2.24, RHAIIS 3.2.1 Model Developers: RedHat (Neural Magic) ModelCar Storage URI: oci://registry.redhat.io/rhelai1/modelcar qwen3 8b fp8 dynamic:1.5 Validated on RHOAI 2.24: quay.io/modh/vllm:rhoai 2.24 cuda Validated on RHAIIS 3.2.1: http://registry.redhat.io/rhaiis/vllm cuda rhel9:3.2.1 Validated on vLLM: 0.10.0 Model Optimizations This model was obtained by quantizing activations and weights of Qwen3 8B to FP8 data type. This optimization reduces the number of bits used to represent weights and activations from 16 to 8, reducing GPU memory requirements (by approximately 50%) and increasing matrix multiply compute throughput (by approximately 2x). Weight quantization also reduces disk size requirements by app…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy