🚀 Falcon 7B Falcon 7B is a 7B parameters causal decoder only model built by TII and trained on 1,500B tokens of RefinedWeb enhanced with curated corpora. It is made available under the Apache 2.0 license. Paper coming soon 😊. 🤗 To get started with Falcon (inference, finetuning, quantization, etc.), we recommend reading this great blogpost fron HF! Why use Falcon 7B? It outperforms comparable open source models (e.g., MPT 7B, StableLM, RedPajama etc.), thanks to being trained on 1,500B tokens of RefinedWeb enhanced with curated corpora. See the OpenLLM Leaderboard. It features an architecture optimized for inference , with FlashAttention (Dao et al., 2022) and multiquery (Shazeer et al., 2019). It is made available under a permissive Apache 2.0 license allowing for commercial use , without any royalties or restrictions. ⚠️ This is a raw, pretrained model, which should be further finetuned for most usecases. If you are looking for a version better suited to taking generic instructions in a chat format, we recommend taking a look at Falcon 7B Instruct. 🔥 Looking for an even more powerful model? Falcon 40B is Falcon 7B's big brother! 💥 Falcon LLMs require PyTorch 2.0 for use with…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy