Qwen3.6-27B-FABQ-RC-GGUF

https://github.com/toxzak-svg/fabq-rc

Model Description

FABQ-RC (Fisher-Adaptive Binary Quantization with Residual Codebooks) is a 1-bit quantization method for large language models that adapts per layer rather than using a fixed blocksize. This quantization of Qwen3.6-27B achieves ~1.18 bits per parameter while maintaining quality through four key innovations.

How It Works

FABQ-RC combines four innovations:

1. Fisher-Weighted Channel Importance — Uses Fisher Information (expected gradient²) to determine which channels actually matter for the loss. This is more directly relevant than Hessian (curvature) or magnitude alone.

2. Mixed-Precision Core Allocation — Top 5% Fisher channels → int4 (preserve accuracy). Bottom 95% → binary ±1 (maximum compression).

3. Adaptive Blocksize — Each layer gets its own optimal blocksize from {64, 128, 256, 512}, chosen by minimizing Fisher-weighted reconstruction error. Homogeneous layers use larger blocks; heterogeneous layers use smaller ones.

4. Residual Codebook — After binary quantization, systematic residuals remain. FABQ-RC clusters these using 4 tiered k-means codebooks (64 centroids each, Fisher quartile-based), enabling non-linear correction that beats BiLLM's linear approximation.

FP32 Weights
    │
    ▼
Stage 1: Fisher-Weighted Channel Importance
    │  Compute per-channel Fisher Information
    │  Sort channels by expected loss impact
    ▼
Stage 2: Mixed-Precision Allocation
    │  Top 5% channels → int4 (preserve accuracy)
    │  Bottom 95% channels → binary ±1 (max compression)
    ▼
Stage 3: Adaptive Blocksize Selection
    │  Per-layer sweep {64, 128, 256, 512}
    │  Pick blocksize minimizing Fisher-weighted reconstruction error
    ▼
Stage 4: Residual Codebook Clustering
    │  4 tiered codebooks × 64 centroids (Fisher quartile-based)
    │  4-bit indices per block (16 centroids per layer cluster)
    ▼
FABQ-RC GGUF