Qwen-Image-Edit-Rapid-AIO-NSFW-v23 — NVFP4 GGUF
This repository contains NVFP4 (4-bit NVIDIA FP4) quantized GGUF components of the Phr00t/Qwen-Image-Edit-Rapid-AIO-NSFW-v23 merged checkpoint, converted for use with ComfyUI-GGUF.
Components
| File | Size | Description |
|---|---|---|
qwen-v23-diffusion-NVFP4.gguf | 11.5 GB | MMDiT diffusion backbone (~20B params) — NVFP4 quantized |
text_encoder/text_encoder-NVFP4.gguf | 3.84 GB | Qwen2.5-VL-7B text encoder — NVFP4 quantized |
vae/vae.safetensors | 0.25 GB | VAE decoder (BF16, original format) |
Hardware Requirements
- GPU: NVIDIA Blackwell (RTX 50-series) with CUDA 13.0+
- VRAM: ~10 GB for diffusion model + text encoder + VAE (NVFP4 saves ~4 GB vs FP8)
- System RAM: 32 GB+ recommended
Usage with ComfyUI
- Install ComfyUI-GGUF
- Place
qwen-v23-diffusion-NVFP4.ggufinComfyUI/models/checkpoints/ - Place
text_encoder/text_encoder-NVFP4.ggufinComfyUI/models/clip/ - Place
vae/vae.safetensorsinComfyUI/models/vae/ - Load the checkpoint normally — ComfyUI-GGUF will automatically detect the NVFP4 format
Quantization Details
| Property | Value |
|---|---|
| Quantization (Diffusion) | NVFP4 (NVIDIA FP4, 4-bit) |
| Quantization (Text Encoder) | NVFP4 (NVIDIA FP4, 4-bit) |
| GGML Quant Type | 40 (NVFP4) |
| GGUF File Type | 39 (LLAMA_FTYPE_MOSTLY_NVFP4) |
| Bits Per Weight (Text Encoder) | ~4.66 |
| Backend | CUDA 13.2, compute 12.0 |
| llama.cpp Commit | dd7cad7 |
| Source Precision | F8_E4M3 → F16 → NVFP4 |
Conversion Process
The original FP8 merged checkpoint was:
- Split into three components (diffusion, text encoder, VAE)
- Each component's FP8 tensors manually converted to F16 via streaming binary I/O with row-chunked FP8→F16 conversion to avoid OOM on large tensors
- Written as F16 GGUF using
ggufPython library (v0.19.0) - Quantized to NVFP4 using
llama-quantize.exe(commitdd7cad7, CUDA 13.2) - Architecture binary-patched in output GGUF to match expected ComfyUI-GGUF target
Text Encoder Notes
- The text encoder (Qwen2.5-VL-7B, 7.75 GB F8_E4M3 → 3.84 GB NVFP4) uses the
qwen2GGUF architecture - Input GGUF was written with both
qwen2.*andllama.*metadata keys for quantizer compatibility, then patched toqwen2after quantization - Row-chunked FP8→F16 conversion (256 rows at a time) prevents OOM on the 545M-element embedding table
- Embedding weights are tied (
lm_headshares weights withtoken_embd) — standard for Qwen2 models
Available Quantizations
Currently only NVFP4 is provided. Other quantizations (Q4_K_M, Q5_K_M, Q6_K, Q8_0) are possible using the same toolchain.
Notes
- This is a NSFW checkpoint. Use at your own discretion.
- The text encoder is now also NVFP4 GGUF, saving ~4 GB (was 7.75 GB as safetensors) while remaining compatible with ComfyUI-GGUF.
- The VAE remains in safetensors format (BF16) for maximum compatibility.
- NVFP4 is a lossy quantization — expect minor quality degradation compared to the original FP8 model.
- Requires NVIDIA Blackwell GPU (RTX 50-series) with CUDA 13.0+ for NVFP4 compute support.
Links
- Original model: Phr00t/Qwen-Image-Edit-Rapid-AIO-NSFW-v23
- ComfyUI-GGUF: city96/ComfyUI-GGUF
- llama.cpp: ggml-org/llama.cpp