Model Card for Model ID Transformers compatible pixtral checkpoints. Make sure to install from source or wait for v4.45! Usage example You should get an output similar to the below: Usage with chat template You can also use a chat template to format your chat history for Pixtral. Make sure that the images argument to the processor contains the images in the order that they appear in the chat, so that the model understands where each image is supposed to go. Here's an example with text and multiple images interleaved in the same message: From transformers =v4.48, you can also pass image url or local path to the conversation history, and let the chat template handle the rest. Chat template will load the image for you and return inputs in torch.Tensor which you can pass directly to model.generate() . You should get something like this: While it may appear that spacing in the input is disrupted, this is caused by us skipping special tokens for display, and actually "Can this animal" and "live here" are correctly separated by image tokens. Try decoding with special tokens included to see exactly what the model sees!
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy