Bielik 11B v2.3 Instruct GGUF This repo contains GGUF format model files for SpeakLeash's Bielik 11B v.2.3 Instruct. DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations! Available quantization formats: q4 k m: Uses Q6 K for half of the attention.wv and feed forward.w2 tensors, else Q4 K q5 k m: Uses Q6 K for half of the attention.wv and feed forward.w2 tensors, else Q5 K q6 k: Uses Q8 K for all tensors q8 0: Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. Ollama Modfile The GGUF file can be used with Ollama. To do this, you need to import the model using the configuration defined in the Modfile. For model eg. Bielik 11B v2.3 Instruct.Q4 K M.gguf (full path to model location) Modfile looks like: Model description: Developed by: SpeakLeash & ACK Cyfronet AGH Language: Polish Model type: causal decoder only Quant from: Bielik 11B v2.3 Instruct Finetuned from: Bielik 11B v2 License: Apache 2.0 and Terms of Use About GGUF GGUF is a new format introduced by the llama.cpp team on August 21st 2023. Here is an incomplete list of clients and libraries that are known to support GGUF: llama.cpp.…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy