💻 Gemma4-12B-Coder (GGUF) — Composer 2.5 × Fable 5 ✨

🐣 Tiny footprint, big brain — a local coding model for everyone

No matter your GPU. No matter your RAM. If you've got ~4.5 GB of VRAM or unified memory free, you can run your own private, offline coding assistant right now. 🚀 This is the v1 / code edition — distilled from real chain-of-thought so it thinks through a problem before writing the solution. 🧠💻 All local, all yours, no API, no cloud.

🎯 What it is

A focused fine-tune of Gemma 4 12B on verifiable Python coding data — every training example's reasoning leads to code that actually passed its tests. The result reasons in the open (edge cases, complexity, approach) and then emits a clean, runnable solution. 💚

📚 Training data (the interesting part 🍳)

This is a distillation of two complementary chain-of-thought sources, both over verifiable Python coding tasks (algorithmic / function-level problems that come with deterministic tests):

• 🥇 Main set — Composer 2.5 real CoT. Genuine, model-authored reasoning traces. The teacher solved each problem, its code was run against the task's tests, and only the passing solutions were kept. So the reasoning you're learning from leads to code that actually works.
• 🥈 Aux set — Fable 5 (released today! 🎉). A clever twist: we took the problems where Composer 2.5 got it wrong and handed them to Fable 5 to redo — re-deriving a fresh, self-consistent chain-of-thought and a correct solution, again gated on passing the tests. This recovers the hard cases the main teacher missed. These traces are synthetic (rationalized CoT), and are tagged separately so the two sources stay distinguishable.

The recipe: real CoT for the bulk of solid coverage, plus synthetic "second-attempt" CoT to patch the failures — both verified by execution before anything entered training. ✅

🗺️ Roadmap — v2 (if there's interest! 💚)

This is v1. If the likes / downloads add up, I'll ship a v2 that:

• Leans harder into the Fable 5 data as the primary signal,
• keeps a portion of Composer 2.5 real CoT for coverage,
• and pushes for the benchmarks 🏁.

⭐ Like & download if you'd like to see v2 — that's the signal I'm watching!

🐢 Upload status — sorry, and a heartfelt PSA 🙏

I'm very sorry the upload has been so slow — as of right now, not all files have finished uploading yet. But please don't worry: I will get everything up. 💪

✅ Update: all files are up — every quant (Q2_K / Q4_K_M / Q6_K / Q8_0) is fully uploaded. Enjoy! 🎉

And a sincere plea while I'm at it: please, do NOT use any Verizon WiFi. I happen to be on their WiFi, and my uploads keep stalling. I've tried to fix it many, many times and it's still broken. So let me say it once more, loud and clear: stay away from Verizon WiFi. 📵 Thank you so much for your patience! 💚

📦 Pick your size (GGUF quants)

🧮 "Will it fit?" — context length cheat-sheet

Rough estimates 🤓 (assumes q8_0 KV cache + ~1.5 GB overhead; use q4_0 KV cache for ≈2× more context!). Max context is 131K. "—" = won't fit, pick a smaller quant. ✂️

💡 Apple Silicon / integrated GPUs with unified memory count too — same numbers, just slower than a dGPU. 💡 Low on room? Drop a quant or switch KV cache to q4_0 and your context roughly doubles.

🚀 How to run it (super easy)

Option A — llama.cpp (recommended) 🦙

1. Grab a quant above (e.g. …-Q4_K_M.gguf) and llama-server from llama.cpp.

   ⚠️ Needs a recent llama.cpp (this is the gemma4_unified architecture — older builds won't load it).

2. Run a server (Windows .bat shown — tweak --port, --ctx-size to taste):

@echo off
cd /d C:\llama.cpp
llama-server.exe ^
  -m C:\models\gemma4-coding-Q4_K_M.gguf ^
  --ctx-size 16384 ^
  --n-gpu-layers 99 ^
  --no-mmap ^
  -fa on ^
  --cache-type-k q8_0 --cache-type-v q8_0 ^
  --temp 1.0 --top-p 0.95 --top-k 64 ^
  --host 0.0.0.0 --port 18080
pause

3. Open http://localhost:18080 and chat. 🎉 (Tip: bump --ctx-size per the table; use q4_0 KV for more.)

Option B — one-click apps 🖱️

Works in LM Studio, Jan, Ollama, etc. — just import the GGUF, pick your quant, go. 🐾

🧠 Thinking mode

This model thinks in Gemma's native thought channel before answering — exactly how it was trained. Keep enable_thinking=true (the default chat template handles it). Recommended sampling: temp 1.0, top_p 0.95, top_k 64. For coding you can also go greedy (temp 0) for more deterministic solutions.

⚠️ Good to know

• Reduced refusals: the training data is task-focused with no safety hedging, so this refuses less than the base model. It is not safety-aligned — add your own guardrails for production. Use responsibly. 🙏
• Specialized for Python / algorithmic coding. Reasoning quality is strongest in that domain; general-knowledge facts/numbers should still be double-checked.
• English-centric.

📚 Base & License

• Base model: google/gemma-4-12B-it. Subject to the Gemma Terms of Use (derivatives must comply).
• Personal/hobby project — shared as-is, no warranty. Have fun, and happy hacking! 🐾✨