DeepSeek V4 Flash native FP4 / FP8 GGUF Native, 1:1 conversion of deepseek ai/DeepSeek V4 Flash from the original safetensors into a single GGUF file that preserves the model's native low precision weights: Dense weights: FP8 E4M3 ( F8 E4M3 B128 , 128 element blocks with one E8M0 scale) MoE expert weights: MXFP4 ( MXFP4 ) This file is not derived from a higher precision intermediate; the FP4 and FP8 codes from the upstream checkpoint are written directly into the GGUF. File File Size Quant DeepSeek V4 Flash FP4 FP8 native.gguf ~146 GB F8 E4M3 + MXFP4 Loading This GGUF requires a llama.cpp build with native F8 E4M3 B128 and MXFP4 support and the DeepSeek V4 Flash architecture. Stock upstream llama.cpp cannot load this file. Reference (WIP) build that can both produce and run this GGUF: https://github.com/nisparks/llama.cpp/tree/wip/deepseek v4 support That branch adds: GGML TYPE F8 E4M3 B128 (ggml type 42) LLAMA FTYPE MOSTLY F8 E4M3 MXFP4 (ftype 41, exposed as F8 E4M3 MXFP4 / moe f8 e4m3 mxfp4 ) CUDA dequant / MMVQ kernels for F8 E4M3 B128 Loader / converter / gguf py support Custom DeepSeek V4 Flash model graph The branch is an active WIP, expect rough edges. Notes DeepSeek V4 Flas…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy