Quants with iMatrix for : https://huggingface.co/TeeZee/Kyllene 34B v1.1 Non iMatrix quants (more choice in higher bitrate quants) : https://huggingface.co/TeeZee/Kyllene 34B v1.1 GGUF/tree/main TeeZee's Kyllene 34B v1.1 model is one of the best Yi 34b merge around with those of BruceTheMoose. But it has a little thing which distinguishes it : It uses Gryphe's MergeMonster as a tool to trim out the GPTisms, Yisms, and Llamaisms, and give a more natural output. The clearing of any problematic gptism, llamaism, or yiism which was specified to MergeMonster is noticeable And it's like the model is freed of these sequences which represent some form of "EOS chains of tokens" in many models, this in the sense that they conclude many outputs, this ofc in an unwanted way It's quite a step in the right direction which should become the standard practice. That make me wonder about the future, when we'll get Miqu 70b models properly finetuned with the best datatsets AND with the Mistralisms trimmed out as well. Available quants : Full offload possible on 48GB VRAM with a huge context size : Q8 0 Full offload possible on 36 GB VRAM with a huge context size : Q5 K S Full offload possible on 24GB…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy