Ministral 3 8B Instruct 2512 AWQ INT4 Model Details Quantization Details Quantization Method: AWQ Bits: 4 Group Size: 32 Calibration Dataset: 5CD AI/LLaVA CoT o1 Instruct Quantization Tool: llm compressor Memory Usage Type Ministral 3 8B Instruct 2512 BF16 Ministral 3 8B Instruct 2512 AWQ 4bit : : : : : : Memory Size 33.2 GB 13.4 GB Evaluations Benchmarks Ministral 3 8B Instruct 2512 BF16 Ministral 3 8B Instruct 2512 AWQ 4bit : : : : : : Perplexity 1.53717 1.54435 Evaluation Context Length: 16384 Inference Prerequisite Basic Usage Additional Information Changelog v1.0.0 Initial quantized release Authors Name: Ton Cao Contacts: ton@cyan.kiwi Ministral 3 8B Instruct 2512 A balanced model in the Ministral 3 family, Ministral 3 8B is a powerful, efficient tiny language model with vision capabilities. This model is the instruct post trained version in FP8 , fine tuned for instruction tasks, making it ideal for chat and instruction based use cases. The Ministral 3 family is designed for edge deployment, capable of running on a wide range of hardware. Ministral 3 8B can even be deployed locally, capable of fitting in 12GB of VRAM in FP8, and less if further quantized. Learn more in our bl…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy