OfflineAid — Gemma 4 E4B fine-tune (Q4_K_M GGUF)
Q4_K_M-quantized GGUF of the OfflineAid Stage-1 fine-tune. Drop-in for llama.cpp and Ollama.
- Base:
unsloth/gemma-4-E4B-it - LoRA:
helenk/gemma-4-E4B-lora— Unsloth fine-tune on Kaggle T4 - Merged fp16 source:
helenk/gemma-4-E4B-finetune - Quantize chain:
peft.merge_and_unload→llama.cpp/convert_hf_to_gguf.py→llama-quantize Q4_K_M - File size: ~5.0 GB
Tier A held-out eval (vs stock + RAG)
Held-out: 111 rows stratified per-language (37 EN + 37 ZH + 37 AR) from the 1,113-row helenkwok/offlineaid corpus, seed=3407. Both models served by Ollama (Q4_K_M). Greedy decoding, explicit "Answer in {language}" directive.
| Language | Metric | stock + RAG | ft + RAG | Δ |
|---|---|---|---|---|
| EN | ROUGE-L F1 | 0.688 | 0.699 | +0.011 |
| EN | Format-OK % | 91.9% | 94.6% | +2.7 pp |
| ZH | Format-OK % | 45.9% | 62.2% | +16.3 pp |
| AR | ROUGE-L F1 | 0.085 | 0.139 | +63% |
| AR | Format-OK % | 21.6% | 54.1% | +32.4 pp (2.5×) |
| all | ROUGE-L F1 | 0.334 | 0.355 | +0.021 |
| all | Format-OK % | 53.2% | 70.3% | +17.0 pp |
The fine-tune's value lives in multilingual robustness, especially Arabic (format-OK 21.6% → 54.1%, ROUGE-L +63%). Reproducible via bash scripts/tier_a_pipeline_eval_only.sh in the OfflineAid repo.
Use with Ollama
ollama pull hf.co/helenk/gemma-4-E4B-finetune-GGUF
Or via a local Modelfile:
FROM /path/to/gemma-4-E4B-offlineaid-Q4_K_M.gguf
RENDERER gemma4
PARSER gemma4
PARAMETER num_ctx 32768
PARAMETER stop "<turn|>"
PARAMETER temperature 0.0
ollama create offlineaid-e4b -f Modelfile
ollama run offlineaid-e4b
Use with llama.cpp
./llama-cli \
-m gemma-4-E4B-offlineaid-Q4_K_M.gguf \
-p "Answer in Simplified Chinese.\n\nQUESTION: ..." \
--temp 0.0 -n 256
Intended use
Stage 3 of the OfflineAid pipeline — Mac-side pack-builder agent loop. Pixel 7 production deployment uses stock Gemma 4 E2B + retrieval, not this fine-tune; see the project writeup for the architectural rationale.
License
Inherits Google's Gemma Terms of Use. Training data (helenkwok/offlineaid) is CC-BY-4.0.
Sibling repos
- Merged fp16 safetensors (~16 GB):
helenk/gemma-4-E4B-finetune - LoRA adapter (~50 MB):
helenk/gemma-4-E4B-lora - Smaller E2B variant:
helenk/gemma-4-E2B-finetune-GGUF