Model Details This model is an W4A16 model with group size 128 and symmetric quantization of google/gemma 4 12B it generated by intel/auto round. Please follow the license of the original model. Transformes inference pip uninstall gptqmodel ~~~python from transformers import AutoProcessor, AutoModelForMultimodalLM MODEL ID = "Intel/gemma 4 12B it int4 AutoRound" Load model processor = AutoProcessor.from pretrained(MODEL ID) model = AutoModelForMultimodalLM.from pretrained( MODEL ID, dtype="auto", device map="auto" ) Prompt add image before text messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://qianwen res.oss accelerate.aliyuncs.com/Qwen3.5/demo/RealWorld/RealWorld 04.png"}, {"type": "text", "text": "What is shown in this image?"} ] } ] Process input inputs = processor.apply chat template( messages, tokenize=True, return dict=True, return tensors="pt", add generation prompt=True, ).to(model.device) input len = inputs["input ids"].shape[ 1] Generate output outputs = model.generate( inputs, max new tokens=512) response = processor.decode(outputs[0][input len:], skip special tokens=False) Parse output print(processor.parse response(response)) ~~~ {'content'…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy