OfflineAid — Gemma 4 E4B fine tune (Q4 K M GGUF) Q4 K M quantized GGUF of the OfflineAid Stage 1 fine tune. Drop in for llama.cpp and Ollama. Base: unsloth/gemma 4 E4B it LoRA: helenk/gemma 4 E4B lora — Unsloth fine tune on Kaggle T4 Merged fp16 source: helenk/gemma 4 E4B finetune Quantize chain: peft.merge and unload → llama.cpp/convert hf to gguf.py → llama quantize Q4 K M File size: ~5.0 GB Tier A held out eval (vs stock + RAG) Held out: 111 rows stratified per language (37 EN + 37 ZH + 37 AR) from the 1,113 row helenkwok/offlineaid corpus, seed=3407. Both models served by Ollama (Q4 K M). Greedy decoding, explicit "Answer in {language}" directive. Language Metric stock + RAG ft + RAG Δ : : : EN ROUGE L F1 0.688 0.699 +0.011 EN Format OK % 91.9% 94.6% +2.7 pp ZH Format OK % 45.9% 62.2% +16.3 pp AR ROUGE L F1 0.085 0.139 +63% AR Format OK % 21.6% 54.1% +32.4 pp (2.5×) all ROUGE L F1 0.334 0.355 +0.021 all Format OK % 53.2% 70.3% +17.0 pp The fine tune's value lives in multilingual robustness, especially Arabic (format OK 21.6% → 54.1%, ROUGE L +63%). Reproducible via bash scripts/tier a pipeline eval only.sh in the OfflineAid repo. Use with Ollama Or via a local Modelfile : Use w…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy