Model Card for Zamba2 7B Instruct Zamba2 7B Instruct is obtained from Zamba2 7B by fine tuning on instruction following and chat datasets. Zamba2 7B Instruct is a hybrid model composed of state space (Mamba2) and transformer blocks. Zamba2 7B Instruct long context has been extended from 4k to 16k context by adjusting the rope frequency in the attention blocks. Quick start Prerequisites To use Zamba2 7B instruct, install transformers from source: 1. git clone https://github.com/huggingface/transformers.git 2. cd transformers && pip install . To install dependencies necessary to run Mamba2 kernels, install mamba ssm from source (due to compatibility issues with PyTorch) as well as causal conv1d : 1. git clone https://github.com/state spaces/mamba.git 2. cd mamba && git checkout v2.1.0 && pip install . 3. pip install causal conv1d You can run the model without using the optimized Mamba2 kernels, but it is not recommended as it will result in significantly higher latency and memory usage. Inference To use the context extended version of Zamba, please load the model with use long context=True , i.e.: Performance Zamba2 7B Instruct punches dramatically above its weight, achieving extreme…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy