SmolLM Corpus This dataset is a curated collection of high quality educational and synthetic data designed for training small language models. You can find more details about the models trained on this dataset in our SmolLM blog post. Dataset subsets Cosmopedia v2 Cosmopedia v2 is an enhanced version of Cosmopedia, the largest synthetic dataset for pre training, consisting of over 39 million textbooks, blog posts, and stories generated by Mixtral 8x7B Instruct v0.1. Most of the samples are generated by prompting the model to generate content on specific topics using a web page referred to as a "seed sample," as shown in Figure 1. We use web samples to increase diversity and expand the range of prompts. You can find more details in this blog post. Dataset Features prompt (string) : The input prompt used to generate the text. text (string) : The generated text content. token length (int64) : The length of the text in tokens (Mistral 7B tokenizer). audience (string) : The intended audience for the content. format (string) : The format of the content (e.g., textbook, story). seed data (string) : The seed sample used to generate the text. Loading the dataset Python Edu The python edu su…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy