THIS IS A WIP (Work In Progress) DeepSeek V4 Flash · GGUF GGUF quantizations of deepseek ai/DeepSeek V4 Flash for use with the V4 aware llama.cpp fork at cchuter/llama.cpp @ feat/v4 port cuda . 📦 Required: V4 aware llama.cpp fork. These quants don't load on upstream ggml org/llama.cpp — V4 architecture support (compressor decode, hyperconnection, lightning indexer, FP8 KV simulation, NextN heads) lives only in the fork: Full build + run instructions in Loading below. 🖥️ Supported backends: Apple Silicon (Metal), NVIDIA CUDA (Ada/Blackwell), and CPU. All 5 V4 custom ops ( ggml dsv4 rope tail , ggml dsv4 hc split sinkhorn , ggml dsv4 hc weighted sum , ggml dsv4 hc expand , ggml dsv4 fp8 kv quantize ) have Metal kernels AND CUDA kernels in this fork (validated 19/19 on RTX 5090, CUDA 12.8, SM 120 native). The CUDA FP8 path is gated behind CUDA ARCH = 890 ; older NVIDIA hardware (Volta/Turing/Ampere) uses a software emulated FP8 path that builds cleanly under DCMAKE CUDA ARCHITECTURES=70 but hasn't been runtime validated yet. CUDA testers wanted — file issues at the fork if you hit problems. ROCm / Vulkan / Metal on AMD have no V4 kernels and will fail at the first dsv4 op. Available…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy