Model Card for Mistral Small Instruct 2409 Mistral Small Instruct 2409 is an instruct fine tuned version with the following characteristics: 22B parameters Vocabulary to 32768 Supports function calling 32k sequence length Usage Examples vLLM (recommended) We recommend using this model with the vLLM library to implement production ready inference pipelines. Installation Make sure you install vLLM = v0.6.1.post1 : Also make sure you have mistral common = 1.4.1 installed: You can also make use of a ready to go docker image. Offline Server You can also use Mistral Small in a server/client setting. 1. Spin up a server: Note: Running Mistral Small on a single GPU requires at least 44 GB of GPU RAM. If you want to divide the GPU requirement over multiple devices, please add e.g. tensor parallel=2 2. And ping the client: Mistral inference We recommend using mistral inference to quickly try out / "vibe check" the model. Install Make sure to have mistral inference = 1.4.1 installed. Download Chat After installing mistral inference , a mistral chat CLI command should be available in your environment. You can chat with the model using Instruct following Function calling Usage in Hugging Face T…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy