Qwen1.5 0.5B Chat GGUF Introduction Qwen1.5 is the beta version of Qwen2, a transformer based decoder only language model pretrained on a large amount of data. In comparison with the previous released Qwen, the improvements include: 8 model sizes, including 0.5B, 1.8B, 4B, 7B, 14B, 32B and 72B dense models, and an MoE model of 14B with 2.7B activated; Significant performance improvement in human preference for chat models; Multilingual support of both base and chat models; Stable support of 32K context length for models of all sizes No need of trust remote code . For more details, please refer to our blog post and GitHub repo. In this repo, we provide quantized models in the GGUF formats, including q2 k , q3 k m , q4 0 , q4 k m , q5 0 , q5 k m , q6 k and q8 0 . To demonstrate their model quality, we follow llama.cpp to evaluate their perplexity on wiki test set. Results are shown below: Size fp16 q8 0 q6 k q5 k m q5 0 q4 k m q4 0 q3 k m q2 k 0.5B 34.20 34.22 34.31 33.80 34.02 34.27 36.74 38.25 62.14 1.8B 15.99 15.99 15.99 16.09 16.01 16.22 16.54 17.03 19.99 4B 13.20 13.21 13.28 13.24 13.27 13.61 13.44 13.67 15.65 7B 14.21 14.24 14.35 14.32 14.12 14.35 14.47 15.11 16.57 14B 10.91 10…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy