SmolLM Table of Contents 1. Model Summary 2. Limitations 3. Training 4. License 5. Citation Model Summary SmolLM is a series of state of the art small language models available in three sizes: 135M, 360M, and 1.7B parameters. These models are built on Cosmo Corpus, a meticulously curated high quality training dataset. Cosmo Corpus includes Cosmopedia v2 (28B tokens of synthetic textbooks and stories generated by Mixtral), Python Edu (4B tokens of educational Python samples from The Stack), and FineWeb Edu (220B tokens of deduplicated educational web samples from FineWeb). SmolLM models have shown promising results when compared to other models in their size categories across various benchmarks testing common sense reasoning and world knowledge. For detailed information on training, benchmarks and performance, please refer to our full blog post. This is the SmolLM 360M Generation Running the model on CPU/GPU/multi GPU Using full precision python pip install accelerate import torch from transformers import AutoTokenizer, AutoModelForCausalLM checkpoint = "HuggingFaceTB/SmolLM 360M" tokenizer = AutoTokenizer.from pretrained(checkpoint) for fp16 use torch dtype=torch.float16 instead mo…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy