Ministral 3 3B Instruct 2512 AWQ INT4 Model Details Quantization Details Quantization Method: AWQ Bits: 4 Group Size: 32 Calibration Dataset: 5CD AI/LLaVA CoT o1 Instruct Quantization Tool: llm compressor Memory Usage Type Ministral 3 3B Instruct 2512 BF16 Ministral 3 3B Instruct 2512 AWQ 4bit : : : : : : Memory Size 14.3 GB 7.7 GB Evaluations Benchmarks Ministral 3 3B Instruct 2512 BF16 Ministral 3 3B Instruct 2512 AWQ 4bit : : : : : : Perplexity 1.58746 1.59943 Evaluation Context Length: 16384 Inference Prerequisite Basic Usage Additional Information Changelog v1.0.0 Initial quantized release Authors Name: Ton Cao Contacts: ton@cyan.kiwi Ministral 3 3B Instruct 2512 BF16 The smallest model in the Ministral 3 family, Ministral 3 3B is a powerful, efficient tiny language model with vision capabilities. This model is the instruct post trained version, fine tuned for instruction tasks, making it ideal for chat and instruction based use cases. The Ministral 3 family is designed for edge deployment, capable of running on a wide range of hardware. Ministral 3 3B can even be deployed locally, capable of fitting in 16GB of VRAM in BF16, and less than 8GB of RAM/VRAM when quantized. We pro…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy