Llamacpp imatrix Quantizations of Magnum Picaro 0.7 v2 12b by Trappu Using llama.cpp release b5074 for quantization. Original model: https://huggingface.co/Trappu/Magnum Picaro 0.7 v2 12b All quants made using imatrix option with dataset from here Run them in LM Studio Run them directly with llama.cpp, or any other llama.cpp based project Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Split Description Magnum Picaro 0.7 v2 12b bf16.gguf bf16 24.50GB false Full BF16 weights. Magnum Picaro 0.7 v2 12b Q8 0.gguf Q8 0 13.02GB false Extremely high quality, generally unneeded but max available quant. Magnum Picaro 0.7 v2 12b Q6 K L.gguf Q6 K L 10.38GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . Magnum Picaro 0.7 v2 12b Q6 K.gguf Q6 K 10.06GB false Very high quality, near perfect, recommended . Magnum Picaro 0.7 v2 12b Q5 K L.gguf Q5 K L 9.14GB false Uses Q8 0 for embed and output weights. High quality, recommended . Magnum Picaro 0.7 v2 12b Q5 K M.gguf Q5 K M 8.73GB false High quality, recommended . Magnum Picaro 0.7 v2 12b Q5 K S.gguf Q5 K S 8.52GB false High quality, recommended . Magnum Pic…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy