license name: qwen research license link: https://huggingface.co/Qwen/Qwen2.5 VL 3B Instruct/blob/main/LICENSE language: en pipeline tag: image text to text tags: multimodal library name: transformers Qwen2.5 VL 3B Instruct GGUF Models These files have been built using a imatrix file and latest llama.cpp build. You must use a fork of llama.cpp to use vision with the model. How to Use Qwen 2.5 VL Instruct with llama.cpp To utilize the experimental support for Qwen 2.5 VL in llama.cpp , follow these steps: Note this uses a fork of llama.cpp. At this time the main branch does not support vision for this model 1. Clone the lastest llama.cpp Fork : 2. Build the Llama.cpp : Build llama.cpp as usual : https://github.com/ggml org/llama.cpp building the project Once llama.cpp is built Copy the ./llama.cpp.qwen2vl/build/bin/llama qwen2 vl cli to a chosen folder. 3. Download the Qwen 2.5 VL gguf file : https://huggingface.co/Mungert/Qwen2.5 VL 3B Instruct GGUF/tree/main Choose a gguf file without the mmproj in the name Example gguf file : https://huggingface.co/Mungert/Mungert/Qwen2.5 VL 3B Instruct GGUF/resolve/main/Qwen2.5 VL 3B Instruct q8 0.gguf Copy this file to your chosen folder. 4. Do…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy