Llamacpp imatrix Quantizations of DeepSeek Coder V2 Lite Instruct Using llama.cpp release b3166 for quantization. Original model: https://huggingface.co/deepseek ai/DeepSeek Coder V2 Lite Instruct All quants made using imatrix option with dataset from here Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Description DeepSeek Coder V2 Lite Instruct Q8 0 L.gguf Q8 0 L 17.09GB Experimental , uses f16 for embed and output weights. Please provide any feedback of differences. Extremely high quality, generally unneeded but max available quant. DeepSeek Coder V2 Lite Instruct Q8 0.gguf Q8 0 16.70GB Extremely high quality, generally unneeded but max available quant. DeepSeek Coder V2 Lite Instruct Q6 K L.gguf Q6 K L 14.56GB Experimental , uses f16 for embed and output weights. Please provide any feedback of differences. Very high quality, near perfect, recommended . DeepSeek Coder V2 Lite Instruct Q6 K.gguf Q6 K 14.06GB Very high quality, near perfect, recommended . DeepSeek Coder V2 Lite Instruct Q5 K L.gguf Q5 K L 12.37GB Experimental , uses f16 for embed and output weights. Please provide any feedback of differences. High quality, recommended…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy