EuroLLM 22B Instruct GPTQ (4 bit) This is a 4 bit GPTQ quantized version of utter project/EuroLLM 22B Instruct 2512. Model Details Attribute Value Original Model EuroLLM 22B Instruct 2512 Quantization GPTQ 4 bit Group Size 128 Activation Order desc act=True Calibration Samples 512 Sequence Length 1024 Calibration Dataset OpenHermes 2.5 (ChatML format) Size ~14GB Quantization Configuration Quantization Quality The quantization achieved excellent loss values across all 53 transformer layers: Attention projections: 10⁻⁵ to 10⁻⁶ range MLP layers: 10⁻⁵ range No error explosion or instability Languages Supported EuroLLM supports all 24 official EU languages plus additional European languages: Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Irish, Italian, Latvian, Lithuanian, Maltese, Norwegian, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish, Turkish, Ukrainian Usage With GPTQModel (Recommended) With vLLM With Transformers + AutoGPTQ Chat Template This model uses the ChatML format: Hardware Requirements Minimum : 16GB VRAM (RTX 4080, RTX A4000) Recommended : 24GB VRAM (RTX 4090, RTX A5000, RTX 6000) Supports mult…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy