Tongyi DeepResearch 30B A3B GGUF Models Model Generation Details This model was generated using llama.cpp at commit a2054e3a8 . Quantization Beyond the IMatrix I've been experimenting with a new quantization approach that selectively elevates the precision of key layers beyond what the default IMatrix configuration provides. In my testing, standard IMatrix quantization underperforms at lower bit depths, especially with Mixture of Experts (MoE) models. To address this, I'm using the tensor type option in llama.cpp to manually "bump" important layers to higher precision. You can see the implementation here: 👉 Layer bumping with llama.cpp While this does increase model file size, it significantly improves precision for a given quantization level. I'd love your feedback—have you tried this? How does it perform for you? Click here to get info on choosing the right GGUF model format Introduction We present Tongyi DeepResearch , an agentic large language model featuring 30 billion total parameters, with only 3 billion activated per token. Developed by Tongyi Lab, the model is specifically designed for long horizon, deep information seeking tasks. Tongyi DeepResearch demonstrates state of…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy