Read our How to Run DiffusionGemma Guide! See Unsloth Dynamic 2.0 GGUFs for our quantization benchmarks. Jun 12 Update: You can now run DiffusionGemma in Unsloth Studio. See our Guide . You can also fine tune DiffusionGemma in Unsloth . Inference settings are auto set. Example of DiffusionGemma (4 bit GGUF) running in Unsloth Studio with executable code: DiffusionGemma Hugging Face GitHub Launch Blog Documentation License : Apache 2.0 Authors : Google DeepMind Run with llama.cpp (GGUF) These GGUFs run with the DiffusionGemma build of llama.cpp (the DiffusionGemma PR ggml org/llama.cpp 24423). DiffusionGemma is a block diffusion architecture, so it needs that branch plus the dedicated llama diffusion cli runner the standard llama cli / llama server cannot generate from it yet. 1. Build the branch 2. Download a GGUF Quant Size Notes : : : BF16 47 GB full precision reference Q8 0 25 GB near lossless, recommended Q6 K 21 GB Q5 K M 18 GB Q4 K M 16 GB smallest, fits a single 24 GB GPU 3. Run it (chat) n / n predict is the only length knob you need: it auto derives the diffusion block count and grows the batch/context to fit. On a single GPU the prompt prefix KV cache turns on automatical…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy