LLaVA Model Card Below is the model card of Llava model 7b, which is copied from the original Llava model card that you can find here. Check out also the Google Colab demo to run Llava on a free tier Google Colab instance: Or check out our Spaces demo! Model details Model type: LLaVA is an open source chatbot trained by fine tuning LLaMA/Vicuna on GPT generated multimodal instruction following data. It is an auto regressive language model, based on the transformer architecture. Model date: LLaVA v1.5 7B was trained in September 2023. Paper or resources for more information: https://llava vl.github.io/ How to use the model First, make sure to have transformers = 4.35.3 . The model supports multi image and multi prompt generation. Meaning that you can pass multiple images in your prompt. Make sure also to follow the correct prompt template ( USER: xxx\nASSISTANT: ) and add the token to the location where you want to query images: Using pipeline : Below we used "llava hf/llava 1.5 7b hf" checkpoint. Using pure transformers : Below is an example script to run generation in float16 precision on a GPU device: From transformers =v4.48, you can also pass image url or local path to the conv…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy