LLaVA NeXT Video Model Card Check out also the Google Colab demo to run Llava on a free tier Google Colab instance: Disclaimer: The team releasing LLaVa NeXT Video did not write a model card for this model so this model card has been written by the Hugging Face team. 📄 Model details Model type: LLaVA Next Video is an open source chatbot trained by fine tuning LLM on multimodal instruction following data. The model is buit on top of LLaVa NeXT by tuning on a mix of video and image data to achieve better video understanding capabilities. The videos were sampled uniformly to be 32 frames per clip. The model is a current SOTA among open source models on VideoMME bench. Base LLM: lmsys/vicuna 7b v1.5 Model date: LLaVA Next Video 7B was trained in April 2024. Paper or resources for more information: https://github.com/LLaVA VL/LLaVA NeXT 📚 Training dataset Image 558K filtered image text pairs from LAION/CC/SBU, captioned by BLIP. 158K GPT generated multimodal instruction following data. 500K academic task oriented VQA data mixture. 50K GPT 4V data mixture. 40K ShareGPT data. Video 100K VideoChatGPT Instruct. 📊 Evaluation dataset A collection of 4 benchmarks, including 3 academic VQA b…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy