About weighted/imatrix quants of https://huggingface.co/stepenZEN/DeepSeek R1 Distill Llama 8B Abliterated static quants are available at https://huggingface.co/mradermacher/DeepSeek R1 Distill Llama 8B Abliterated GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi part files. Provided Quants (sorted by size, not necessarily quality. IQ quants are often preferable over similar sized non IQ quants) Link Type Size/GB Notes : : : : PART 1 PART 2 i1 IQ1 S 4.1 for the desperate PART 1 PART 2 i1 IQ1 M 4.4 mostly desperate PART 1 PART 2 i1 IQ2 XXS 4.9 PART 1 PART 2 i1 IQ2 XS 5.3 PART 1 PART 2 i1 IQ2 S 5.6 PART 1 PART 2 i1 IQ2 M 6.0 PART 1 PART 2 i1 Q2 K S 6.1 very low quality PART 1 PART 2 i1 Q2 K 6.5 IQ3 XXS probably better PART 1 PART 2 i1 IQ3 XXS 6.6 lower quality PART 1 PART 2 i1 IQ3 XS 7.1 PART 1 PART 2 i1 Q3 K S 7.4 IQ3 XS probably better PART 1 PART 2 i1 IQ3 S 7.5 beats Q3 K PART 1 PART 2 i1 IQ3 M 7.7 PART 1 PART 2 i1 Q3 K M 8.1 IQ3 S probably better PART 1 PART 2 i1 Q3 K L 8.7 IQ3 M probably better PART 1 PART 2 i1 IQ4 XS 9.0 PART 1 PART 2 i1 Q4 0 9.5 fast, low quality PART 1 PART 2 i1 IQ4…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy