NVIDIA Nemotron Labs 3 Puzzle 75B-A9B GGUF
Community GGUF conversion of nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16.
Important llama.cpp compatibility notice
This model was converted with the still-unmerged ggml-org/llama.cpp PR #25444, pinned to commit af49ef5cd990d039dbf360dd3a9f3b5dafdd1726, plus a narrowly scoped converter compatibility fix for the official BF16 checkpoint's model.layers.* tensor prefix and bounded writeback for large lazy tensors on the high-RAM Colab runtime. Until equivalent support is merged into mainline llama.cpp, use a build containing PR #25444 to load these files.
PR #25444 adds NemotronHPuzzleForCausalLM / nemotron_h_puzzle support, heterogeneous per-layer MoE settings, and the model's two-block MTP draft head. This repository is not an official NVIDIA or llama.cpp release.
Files
NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16.gguf— BF16 master GGUFNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q2_K.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q3_K_S.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q3_K_M.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q3_K_L.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-IQ4_XS.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_K_S.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_K_M.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_K_S.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_K_M.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q6_K.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q8_0.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_0.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_1.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_0.ggufNVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_1.gguf
No importance matrix was used for any quantization in this repository.
Shape-fallback warning
Puzzle uses heterogeneous expert widths from 1280 to 2688. Some expert down-projection rows, especially width 2688, are not divisible by the 256-element block size used by K-quants. The pinned llama.cpp quantizer therefore applies its compatible fallback automatically on those tensors: Q2_K/Q3_K to Q4_0, IQ4_XS to IQ4_NL, Q4_K to Q5_0, Q5_K to Q5_1, and Q6_K to Q8_0. Consequently, named K-quant files are mixed recipes rather than a single tensor type everywhere. Required norms, scales, and other small tensors may remain F32.
Artifact integrity and quantizer compatibility are checked automatically, but successful conversion does not prove task-quality parity with BF16. Evaluate very small formats before practical use.
Provenance
- Source revision:
8fe5546888e9bd03fdbf52d808adebdfca901b52 - Source safetensors:
156,596,801,168bytes across 31 model shards plusmtp.safetensors - llama.cpp PR: #25444
- PR commit:
af49ef5cd990d039dbf360dd3a9f3b5dafdd1726 - Patched converter SHA-256:
1aaa36ac789fc6eceebefe19d4d80c3c9dc56185a4a3e956411bc0478ee46afc - Patched large-write GGUF writer SHA-256:
350db0132703b3b4025ee61e344b7d7400b9c3a2692b87dd6ca186d82423fa30 - Conversion: pinned safetensors in hybrid RAM/disk storage, then local
convert_hf_to_gguf.py --outtype bf16 - Quantization: pinned
llama-quantize, without--imatrix - License: the source model is governed by the OpenMDW License Agreement 1.1
Follow the prompting, safety, license, and usage guidance in the original NVIDIA model card.