WHISPER SMALL GGUF Quantized Models Quantized versions of openai/whisper small in GGUF format. Directory Structure Format Compatibility Root directory ( whisper small .gguf ): Use with Candle (Rust ML framework) Tensor names include model. prefix (e.g., model.encoder.conv1.weight ) Compatible with Neurolang application Requires config small.json and tokenizer small.json whisper.cpp/ directory: Use with whisper.cpp (C++ implementation) Tensor names without model. prefix (e.g., encoder.conv1.weight ) Compatible with whisper.cpp CLI tools Both directories contain .gguf files, not .bin files Available Formats Format Quality Use Case q2 k Smallest Extreme compression q3 k Small Mobile devices q4 0 Good Legacy compatibility q4 k Good Recommended for production q4 1 Good+ Legacy with bias q5 0 Very Good Legacy compatibility q5 k Very Good High quality q5 1 Very Good+ Legacy with bias q6 k Excellent Near lossless q8 0 Excellent Minimal loss, benchmarking Usage With Candle (Rust) For this model, you need to modify the example code in candle. To try whisper in candle faster and easier, it's better to use the tiny model → https://huggingface.co/oxide lab/whisper tiny GGUF Command line example…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy