Resharded Resharded version of https://huggingface.co/tiiuae/falcon 7b instruct for low RAM enviroments (e.g. Colab, Kaggle) in safetensors Tutorial: https://medium.com/@vilsonrodrigues/run your private llm falcon 7b instruct with less than 6gb of gpu using 4 bit quantization ff1d4ffbabcc ✨ Falcon 7B Instruct Falcon 7B Instruct is a 7B parameters causal decoder only model built by TII based on Falcon 7B and finetuned on a mixture of chat/instruct datasets. It is made available under the Apache 2.0 license. Paper coming soon 😊. 🤗 To get started with Falcon (inference, finetuning, quantization, etc.), we recommend reading this great blogpost fron HF! Why use Falcon 7B Instruct? You are looking for a ready to use chat/instruct model based on Falcon 7B. Falcon 7B is a strong base model, outperforming comparable open source models (e.g., MPT 7B, StableLM, RedPajama etc.), thanks to being trained on 1,500B tokens of RefinedWeb enhanced with curated corpora. See the OpenLLM Leaderboard. It features an architecture optimized for inference , with FlashAttention (Dao et al., 2022) and multiquery (Shazeer et al., 2019). ⚠️ Falcon is now available as a core model in the transformers library!…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy