VipLLaVA Model Card Below is the model card of VipLlava model 7b, which is copied from the original Llava model card that you can find here. Check out also the Google Colab demo to run Llava on a free tier Google Colab instance (the model works similarly as Llava): Or check out our Spaces demo! Model details Model type: LLaVA is an open source chatbot trained by fine tuning LLaMA/Vicuna on GPT generated multimodal instruction following data. It is an auto regressive language model, based on the transformer architecture. Vip LlaVa enhances the training protocol of Llava by marking images and interact with the model using natural cues like a “red bounding box” or “pointed arrow” during training. Model date: ViP LLaVa was released in December 2023. Paper or resources for more information: https://vip llava.github.io/ How to use the model First, make sure to have transformers = 4.35.3 . The model supports multi image and multi prompt generation. Meaning that you can pass multiple images in your prompt. Make sure also to follow the correct prompt template and add the token to the location where you want to query images: According to the official code base, it is recommeneded to use this…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy