Llamacpp imatrix Quantizations of gpt oss 120b by openai Using llama.cpp release b6115 for quantization. Original model: https://huggingface.co/openai/gpt oss 120b All quants made using imatrix option with dataset from here Run them in LM Studio Run them directly with llama.cpp, or any other llama.cpp based project Prompt format No prompt format found, check original model page Download a file (not the whole branch) from below: Use this one: Filename Quant type File Size Split Description gpt oss 120b MXFP4 MOE.gguf MXFP4 MOE 63.39GB true Special format for OpenAI's gpt oss models, see: https://github.com/ggml org/llama.cpp/pull/15091 The reason is, the FFN (feed forward networks) of gpt oss do not behave nicely when quantized to anything other than MXFP4, so they are kept at that level for everything. The rest of these are provided for your own interest in case you feel like experimenting, but the size savings is basically non existent so I would not recommend running them, they are provided simply for show: Filename Quant type File Size Split Description gpt oss 120b bf16.gguf bf16 65.37GB true Full BF16 weights. gpt oss 120b Q6 K.gguf Q6 K 63.28GB true Q6 K with all FFN kept at…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy