m51Lab-MiniMax-M2.7-REAP-139B-A10B-NVFP4-GGUF

NVFP4 mixed-precision GGUF quantization of the REAP-40% pruned MiniMax M2.7 model, optimised for NVIDIA Blackwell GPUs. The NVFP4 tensor types use native Blackwell FP4 hardware acceleration for maximum inference speed, while smaller tensors (norms, biases, router, embeddings) are preserved in Q4_K for accuracy.

About the Source Model

m51Lab-MiniMax-M2.7-REAP-139B-A10B is a REAP-40% pruned variant of MiniMax M2.7, reducing total parameters from 229B to 139B while preserving the 10B active parameters per token. REAP (Router-weighted Expert Activation Pruning) prunes 40% of experts per MoE block (256 → 154) based on router-gated activation patterns.

• Base model: MiniMaxAI/MiniMax-M2.7 (229B MoE, 62 layers)
• Pruning method: REAP (Lasby et al., 2025, arXiv:2510.13999)
• Pruning rate: 40% of experts per MoE block
• Active parameters: ~10B per token
• Architecture: Sparse MoE decoder-only Transformer
  • Hidden dim: 3072
  • Attention heads: 48 query / 8 KV (GQA)
  • Experts: 154 per block (top-8 activated)
  • Expert routing: Sigmoid gating with learnable bias
  • Context length: 196,608 tokens
  • Vocab: 200,064 (GPT-2 tokenizer)

Evaluation (from source)

HumanEval pass@1 (on completed): 83.3% (90/108)

All evaluations conducted on the Q4_K_M base variant. NVFP4 mixed quantization targets the same tensor patterns used by other Q4_K_M quality benchmarks, so quality should be equivalent or better (NVFP4 at 4.52 BPW vs Q4_K_M at 4.84 BPW on the output, but with hardware-accelerated inference on Blackwell).

Original Creator: m51Lab / dervig | License: Modified MIT (inherited from MiniMaxAI/MiniMax-M2.7)

Quantization Details

This is a mixed NVFP4/Q4_K quantization created using llama-quantize with --tensor-type-file on a Q4_K_M imatrix-quantized base.

Tensor Mapping

Size Comparison

The size reduction comes primarily from ffn_down_exps tensors which were Q6_K in the base (568 MB each) and are now NVFP4 (390 MB each) — a 31% reduction on those 62 tensors alone.

Performance

Benchmarked on single NVIDIA RTX PRO 6000 Blackwell Workstation Edition (97 GB VRAM) with llama.cpp build 9253 (commit 29f148222), CUDA sm_120, native Blackwell FP4 acceleration. Strict GPU 0 isolation (CUDA_VISIBLE_DEVICES=0).

Recommended Configuration

Measured Throughput (canonical, "Give me a history of dotnet" prompt, 300 generated tokens)

Context Ceiling Analysis (f16 KV)

Key Findings

• Decode is purely bandwidth-bound: batch size 512–4096 all yield ~120 tok/s with q8_0 KV and ~133 tok/s with f16 KV. Larger batch sizes give no throughput benefit.
• f16 KV is faster than q8_0 on this MoE on Blackwell: ~11–13% decode speedup, with zero quality concern (f16 > q8_0 by definition, so it satisfies the no-below-q8 constraint).
• Context scaling doesn't affect decode speed: 20K and 131K contexts both yield ~120 tok/s (q8_0 KV) — context only affects VRAM and prefill.
• f16 KV costs exactly 2× q8_0 VRAM: 128 MiB/1K tokens for q8_0, 256 MiB/1K tokens for f16.
• Bandwidth ceiling is ~135 tok/s on this single 6000 Pro — no spec, batch, or context tuning exceeds this.

NVFP4 on Blackwell uses native FP4 tensor core acceleration, providing significantly higher throughput than equivalent bit-width integer quants on the same hardware. The expert weights (ffn_*_exps) — which are the largest tensors in the model — benefit most from this acceleration.

Usage

llama.cpp Server (Recommended)

llama-server \
  -m m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf \
  --host 127.0.0.1 --port 8080 \
  -ngl 99 \
  -c 80000 \
  -b 512 -ub 512 \
  --threads 64 \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --split-mode none \
  --main-gpu 0 \
  --cont-batching \
  --no-warmup \
  --spec-type ngram-mod \
  --spec-ngram-mod-n-match 24 \
  --spec-ngram-mod-n-min 4 \
  --spec-ngram-mod-n-max 48

For maximum context (use q8_0 KV to fit 131K):

  -c 131000 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0
  # (omit the --spec-type args for max context + max speed; f16 KV hits OOM at 90K+)

Recommended Sampling Parameters

• Temperature: 0.6–1.0
• top_p: 0.95
• top_k: 20–40
• min_p: 0.01
• repeat_penalty: 1.1

Speculative Decoding

This model does not include MTP (Multi-Token Prediction) heads — the REAP pruning process does not preserve the MTP module from the original MiniMax M2.7. N-gram speculative decoding is supported and yields ~25% accept rate, contributing ~3-5 tok/s extra:

llama-server \
  -m m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf \
  -ngl 99 -c 20000 \
  --spec-type ngram-mod \
  --spec-ngram-mod-n-match 24 \
  --spec-ngram-mod-n-min 4 \
  --spec-ngram-mod-n-max 48

Hardware Requirements

• Required: NVIDIA Blackwell GPU (RTX 5090, RTX PRO 6000, B200, etc.) for NVFP4 hardware acceleration
• VRAM: ~62 GB minimum for model weights (single GPU), additional VRAM for KV cache
  • q8_0 KV: 128 MiB per 1K context tokens
  • f16 KV: 256 MiB per 1K context tokens
• Tested: Single RTX PRO 6000 (97 GB) — full benchmark sweep at contexts 20K–131K
• System RAM: 128 GB+ recommended
• CPU: 32+ cores recommended (Threadripper-class) for prompt cache management

Reproduction

# 1. Download the Q4_K_M imatrix source and imatrix file
hf download mradermacher/m51Lab-MiniMax-M2.7-REAP-139B-A10B-i1-GGUF \
  m51Lab-MiniMax-M2.7-REAP-139B-A10B.i1-Q4_K_M.gguf \
  m51Lab-MiniMax-M2.7-REAP-139B-A10B.imatrix.gguf \
  --local-dir ./source

# 2. Create tensor-type file
cat > nvfp4-tensor-types.txt << 'EOF'
attn_q=nvfp4
attn_k=nvfp4
attn_v=nvfp4
attn_output=nvfp4
ffn_gate_exps=nvfp4
ffn_up_exps=nvfp4
ffn_down_exps=nvfp4
EOF

# 3. Quantize
llama-quantize \
  --allow-requantize \
  --imatrix ./source/m51Lab-MiniMax-M2.7-REAP-139B-A10B.imatrix.gguf \
  --tensor-type-file nvfp4-tensor-types.txt \
  ./source/m51Lab-MiniMax-M2.7-REAP-139B-A10B.i1-Q4_K_M.gguf \
  m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf \
  Q4_K

File Information

• Model file: m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf (~74 GB)
• Format: GGUF v3
• Architecture: minimax-m2
• Quantization: Mixed NVFP4 (large weights) / Q4_K (small weights)
• No mmproj required — text-only model

Acknowledgments

• m51Lab — REAP-40% pruning of MiniMax M2.7 and base GGUF conversion
• dervig — Publishing the pruned model on HuggingFace
• MiniMax AI — base MiniMax M2.7 model
• mradermacher — Q4_K_M imatrix quantization and i1-GGUF variant
• ggml-org/llama.cpp — NVFP4 quantization support and inference engine (build 9253+)
• Cerebras Research — REAP method (Lasby et al., 2025)

License

Modified MIT License — inherited from MiniMaxAI/MiniMax-M2.7. See the base model repository for full license terms.

Citation

@article{lasby2025reap,
  title   = {REAP the Experts: Why Pruning Prevails for One-Shot MoE compression},
  author  = {Lasby, Mike and Hussein, Ahmed and Spyra, Jayden and Chkroun, Ivan
             and Suleiman, Oriol Sans and Ioannou, Nikoli and Hyder, Ammar Ali
             and Jacobs, Sam and Chaturvedi, Sachin and Mishra, Shreyanshu
             and Aboutalebi, Hossei and Rugol, Vasileios},
  journal = {arXiv preprint arXiv:2510.13999},
  year    = {2025}
}

@misc{minimax_m2_7,
  title  = {MiniMax-M2.7},
  author = {MiniMax AI},
  year   = {2026},
  url    = {https://huggingface.co/MiniMaxAI/MiniMax-M2.7}
}