DeepSeek V4 Flash — GGUF for ds4 This quants are specific for the DS4 inference engine. They may work with other inference engines or not (they should, but not the MTP model which requires a specific loader). https://github.com/antirez/ds4 Files File Size Routed experts ( ffn {gate,up,down} exps ) Everything else : DeepSeek V4 Flash IQ2XXS w2Q2K AProjQ8 SExpQ8 OutQ8 chat v2.gguf 80.8 GiB IQ2 XXS (gate, up) + Q2 K (down) Q8 0 attn proj / shared experts / output, F16 router + embed + indexer + compressor + HC, F32 norms / sinks / bias DeepSeek V4 Flash Q4KExperts F16HC F16Compressor F16Indexer Q8Attn Q8Shared Q8Out chat v2.gguf 153.3 GiB Q4 K (all three) same as above DeepSeek V4 Flash MTP Q4K Q8 0 F32.gguf 3.6 GiB MTP / speculative decoding support (optional, not standalone). Use q2 on 128 GB Mac machines, q4 on machines with ≥ 256 GB RAM, pair either with MTP for optional speculative decoding. Quantization recipe The filename is the spec. In detail, for the q2 file: Tensor class Quant Notes blk. .ffn gate exps , blk. .ffn up exps IQ2 XXS routed expert up/gate blk. .ffn down exps Q2 K routed expert down (K quant for quality) blk. .ffn {gate,up,down} shexp Q8 0 shared experts blk. .a…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy