SmolLM Table of Contents 1. Model Summary 2. Limitations 3. Training 4. License 5. Citation Model Summary SmolLM is a series of state of the art small language models available in three sizes: 135M, 360M, and 1.7B parameters. These models are built on Cosmo Corpus, a meticulously curated high quality training dataset. Cosmo Corpus includes Cosmopedia v2 (28B tokens of synthetic textbooks and stories generated by Mixtral), Python Edu (4B tokens of educational Python samples from The Stack), and FineWeb Edu (220B tokens of deduplicated educational web samples from FineWeb). SmolLM models have shown promising results when compared to other models in their size categories across various benchmarks testing common sense reasoning and world knowledge. For detailed information on training, benchmarks and performance, please refer to our full blog post. This is the SmolLM 135M Generation Running the model on CPU/GPU/multi GPU Using full precision Using torch.bfloat16 Quantized Versions through bitsandbytes Using 8 bit precision (int8) Limitations While SmolLM models have been trained on a diverse dataset including educational content and synthetic texts, they have limitations. The models pr…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy