Llamacpp imatrix Quantizations of Dolphin3.0 Qwen2.5 1.5B Using llama.cpp release b4418 for quantization. Original model: https://huggingface.co/cognitivecomputations/Dolphin3.0 Qwen2.5 1.5B All quants made using imatrix option with dataset from here Run them in LM Studio Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Split Description Dolphin3.0 Qwen2.5 1.5B f32.gguf f32 6.18GB false Full F32 weights. Dolphin3.0 Qwen2.5 1.5B f16.gguf f16 3.09GB false Full F16 weights. Dolphin3.0 Qwen2.5 1.5B Q8 0.gguf Q8 0 1.65GB false Extremely high quality, generally unneeded but max available quant. Dolphin3.0 Qwen2.5 1.5B Q6 K L.gguf Q6 K L 1.33GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . Dolphin3.0 Qwen2.5 1.5B Q6 K.gguf Q6 K 1.27GB false Very high quality, near perfect, recommended . Dolphin3.0 Qwen2.5 1.5B Q5 K L.gguf Q5 K L 1.18GB false Uses Q8 0 for embed and output weights. High quality, recommended . Dolphin3.0 Qwen2.5 1.5B Q5 K M.gguf Q5 K M 1.13GB false High quality, recommended . Dolphin3.0 Qwen2.5 1.5B Q5 K S.gguf Q5 K S 1.10GB false High quality, recommended . Dolphin3.0 Qwen2.5 1.5B…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy