DeepSeek V4 Pro · GGUF GGUF quantizations of deepseek ai/DeepSeek V4 Pro for use with the V4 aware llama.cpp fork at cchuter/llama.cpp @ feat/v4 port cuda . 📦 Required: V4 aware llama.cpp fork. These quants don't load on upstream ggml org/llama.cpp — V4 architecture support (compressor decode, hyperconnection, lightning indexer, FP8 KV simulation, NextN heads) lives only in the fork: Full build + run instructions in Loading below. 🖥️ Supported backends: Apple Silicon (Metal), NVIDIA CUDA (Ada/Blackwell), and CPU. All 5 V4 custom ops ( ggml dsv4 rope tail , ggml dsv4 hc split sinkhorn , ggml dsv4 hc weighted sum , ggml dsv4 hc expand , ggml dsv4 fp8 kv quantize ) have Metal kernels AND CUDA kernels in this fork (validated 19/19 on RTX 5090, CUDA 12.8, SM 120 native). The CUDA FP8 path is gated behind CUDA ARCH = 890 ; older NVIDIA hardware (Volta/Turing/Ampere) uses a software emulated FP8 path that builds cleanly under DCMAKE CUDA ARCHITECTURES=70 but hasn't been runtime validated yet. CUDA testers wanted — file issues at the fork if you hit problems. V4 Pro's size also means most quants need multi GPU or CPU+GPU partial offload; see size note below. ROCm / Vulkan / Metal on AMD…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy