Falcon RW 1B Falcon RW 1B is a 1B parameters causal decoder only model built by TII and trained on 350B tokens of RefinedWeb. It is made available under the Apache 2.0 license. See the 📓 paper on arXiv for more details. RefinedWeb is a high quality web dataset built by leveraging stringent filtering and large scale deduplication. Falcon RW 1B, trained on RefinedWeb only, matches or outperforms comparable models trained on curated data. ⚠️ Falcon is now available as a core model in the transformers library! To use the in library version, please install the latest version of transformers with pip install git+https://github.com/huggingface/transformers.git , then simply remove the trust remote code=True argument from from pretrained() . ⚠️ This model is intended for use as a research artifact , to study the influence of training on web data alone. If you are interested in state of the art models, we recommend using Falcon 7B/40B, both trained on 1,000 billion tokens. 💥 Falcon LLMs require PyTorch 2.0 for use with transformers ! Model Card for Falcon RW 1B Model Details Model Description Developed by: https://www.tii.ae; Model type: Causal decoder only; Language(s) (NLP): English; Li…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy