This model was quantized by SanctumAI. To leave feedback, join our community in Discord. OpenELM 3B Instruct GGUF Model creator: apple Original model : OpenELM 3B Instruct Model Summary: Sachin Mehta, Mohammad Hossein Sekhavat, Qingqing Cao, Maxwell Horton, Yanzi Jin, Chenfan Sun, Iman Mirzadeh, Mahyar Najibi, Dmitry Belenko, Peter Zatloukal, Mohammad Rastegari We introduce OpenELM , a family of Open E fficient L anguage M odels. OpenELM uses a layer wise scaling strategy to efficiently allocate parameters within each layer of the transformer model, leading to enhanced accuracy. We pretrained OpenELM models using the CoreNet library. We release both pretrained and instruction tuned models with 270M, 450M, 1.1B and 3B parameters. We release the complete framework, encompassing data preparation, training, fine tuning, and evaluation procedures, alongside multiple pre trained checkpoints and training logs, to facilitate open research. Our pre training dataset contains RefinedWeb, deduplicated PILE, a subset of RedPajama, and a subset of Dolma v1.6, totaling approximately 1.8 trillion tokens. Please check license agreements and terms of these datasets before using them. Prompt Template…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy