Qwen3 30B A3B Instruct 2507.w8a8 Model Overview Model Architecture: Qwen3ForCausalLM Input: Text Output: Text Model Optimizations: Weight quantization: INT8 Intended Use Cases: Reasoning. Function calling. Subject matter experts via fine tuning. Multilingual instruction following. Translation. Out of scope: Use in any manner that violates applicable laws or regulations (including trade compliance laws). Release Date: 05/05/2025 Version: 1.0 Model Developers: RedHat (Neural Magic) Model Optimizations This model was obtained by quantizing the weights of Qwen/Qwen3 30B A3B Instruct 2507 to INT8 data type. This optimization reduces the number of bits used to represent weights and activations from 16 to 8, reducing GPU memory requirements (by approximately 50%) and increasing matrix multiply compute throughput (by approximately 2x). Weight quantization also reduces disk size requirements by approximately 50%. Only weights and activations of the linear operators within transformers blocks are quantized. Weights are quantized with a symmetric static per channel scheme, whereas activations are quantized with a symmetric dynamic per token scheme. A combination of the SmoothQuant and GPTQ al…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy