Model Details This model is an int4 model with group size 128 and symmetric quantization of google/gemma 4 26B A4B it generated by intel/auto round. Please follow the license of the original model. Transformes inference ~~~python from transformers import AutoProcessor, AutoModelForCausalLM MODEL ID = "Intel/gemma 4 26B A4B it int4 AutoRound/" Load model processor = AutoProcessor.from pretrained(MODEL ID) model = AutoModelForCausalLM.from pretrained( MODEL ID, dtype="auto", device map="auto" ) Prompt add image before text messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://raw.githubusercontent.com/google gemma/cookbook/refs/heads/main/Demos/sample data/GoldenGate.png"}, {"type": "text", "text": "What is shown in this image?"} ] } ] Process input inputs = processor.apply chat template( messages, tokenize=True, return dict=True, return tensors="pt", add generation prompt=True, ).to(model.device) input len = inputs["input ids"].shape[ 1] Generate output outputs = model.generate( inputs, max new tokens=512) response = processor.decode(outputs[0][input len:], skip special tokens=False) Parse output print(processor.parse response(response)) ~~~ {'content': 'An o…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy