Llava v1.5 7B GGUF Original Model liuhaotian/llava v1.5 7b Run with LlamaEdge LlamaEdge version: v0.16.2 Prompt template Prompt type: vicuna llava Prompt string Context size: 4096 Run as LlamaEdge service Quantized GGUF Models Name Quant method Bits Size Use case llava v1.5 7b Q2 K.gguf Q2 K 2 2.53 GB smallest, significant quality loss not recommended for most purposes llava v1.5 7b Q3 K L.gguf Q3 K L 3 3.6 GB small, substantial quality loss llava v1.5 7b Q3 K M.gguf Q3 K M 3 3.3 GB very small, high quality loss llava v1.5 7b Q3 K S.gguf Q3 K S 3 2.95 GB very small, high quality loss llava v1.5 7b Q4 0.gguf Q4 0 4 3.83 GB legacy; small, very high quality loss prefer using Q3 K M llava v1.5 7b Q4 K M.gguf Q4 K M 4 4.08 GB medium, balanced quality recommended llava v1.5 7b Q4 K S.gguf Q4 K S 4 3.86 GB small, greater quality loss llava v1.5 7b Q5 0.gguf Q5 0 5 4.65 GB legacy; medium, balanced quality prefer using Q4 K M llava v1.5 7b Q5 K M.gguf Q5 K M 5 4.78 GB large, very low quality loss recommended llava v1.5 7b Q5 K S.gguf Q5 K S 5 4.65 GB large, low quality loss recommended llava v1.5 7b Q6 K.gguf Q6 K 6 5.53 GB very large, extremely low quality loss llava v1.5 7b Q8 0.gguf Q8 0…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy