Llamacpp imatrix Quantizations of functiongemma 270m it by google Using llama.cpp release b7429 for quantization. Original model: https://huggingface.co/google/functiongemma 270m it All quants made using imatrix option with dataset from here combined with a subset of combined all small.parquet from Ed Addario here Run them in LM Studio Run them directly with llama.cpp, or any other llama.cpp based project Prompt format Download a file (not the whole branch) from below: Filename Quant type File Size Split Description functiongemma 270m it bf16.gguf bf16 0.54GB false Full BF16 weights. functiongemma 270m it Q8 0.gguf Q8 0 0.29GB false Extremely high quality, generally unneeded but max available quant. functiongemma 270m it Q6 K L.gguf Q6 K L 0.28GB false Uses Q8 0 for embed and output weights. Very high quality, near perfect, recommended . functiongemma 270m it Q6 K.gguf Q6 K 0.28GB false Very high quality, near perfect, recommended . functiongemma 270m it Q5 K L.gguf Q5 K L 0.26GB false Uses Q8 0 for embed and output weights. High quality, recommended . functiongemma 270m it Q5 K M.gguf Q5 K M 0.26GB false High quality, recommended . functiongemma 270m it Q5 K S.gguf Q5 K S 0.26GB f…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy