Table of Contents 0. TL;DR 1. Model Details 2. Training Details 3. Usage 4. Evaluation 5. Citation TL;DR Model Details Model Description Developed by: https://www.tii.ae Model type: Causal decoder only Architecture: Hybrid Transformers + Mamba architecture Language(s) (NLP): English, Multilingual License: Falcon LLM License Training details For more details about the training protocol of this model, please refer to the Falcon H1 technical blogpost. Usage Currently to use this model you can either rely on Hugging Face transformers , vLLM or our custom fork of llama.cpp library. Inference Make sure to install the latest version of transformers or vllm , eventually install these packages from source: Refer to the official vLLM documentation for more details on building vLLM from source. 🤗 transformers Refer to the snippet below to run H1 models using 🤗 transformers: vLLM For vLLM, simply start a server by executing the command below: llama.cpp While we are working on integrating our architecture directly into llama.cpp library, you can install our fork of the library and use it directly: https://github.com/tiiuae/llama.cpp Falcon H1 Use the same installing guidelines as llama.cpp .…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy