GGUF Quantized LLaVA 1.6 Mistral 7B Updated quants and projector from PR 5267 Provided files Name Quant method Bits Size Use case llava v1.6 mistral 7b.Q3 K XS.gguf Q3 K XS 3 2.99 GB very small, high quality loss llava v1.6 mistral 7b.Q3 K M.gguf Q3 K M 3 3.52 GB very small, high quality loss llava v1.6 mistral 7b.Q4 K M.gguf Q4 K M 4 4.37 GB medium, balanced quality recommended llava v1.6 mistral 7b.Q5 K S.gguf Q5 K S 5 5.00 GB large, low quality loss recommended llava v1.6 mistral 7b.Q5 K M.gguf Q5 K M 5 5.13 GB large, very low quality loss recommended llava v1.6 mistral 7b.Q6 K.gguf Q6 K 6 5.94 GB very large, extremely low quality loss llava v1.6 mistral 7b.Q8 0.gguf Q8 0 8 7.7 GB very large, extremely low quality loss not recommended ORIGINAL LLaVA Model Card Model details Model type: LLaVA is an open source chatbot trained by fine tuning LLM on multimodal instruction following data. It is an auto regressive language model, based on the transformer architecture. Base LLM: mistralai/Mistral 7B Instruct v0.2 Model date: LLaVA v1.6 Mistral 7B was trained in December 2023. Paper or resources for more information: https://llava vl.github.io/ License mistralai/Mistral 7B Instruct v0.…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy