NemoCurator FineWeb Mixtral Edu Classifier Model Overview This is a text classification model designed to determine the educational value of a piece of text (score 0 5 from low to high). It is similar to the FineWeb Edu classifier and was trained on the same text samples, but using annotations from Mixtral 8x22B Instruct. In contrast, the original FineWeb Edu classifier was trained using annotations from Llama 3 70B Instruct. The NeMo Curator FineWeb Mixtral Edu classifier was used as part of a classifier ensemble in the creation of the Nemotron CC dataset. The models were finetuned starting from the Snowflake/snowflake arctic embed m model. License GOVERNING TERMS: Use of this model is governed by the NVIDIA Open Model License Agreement. Additional Information: Apache 2.0. References The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale Nemotron CC: Transforming Common Crawl into a Refined Long Horizon Pretraining Dataset Arctic Embed: Scalable, Efficient, and Accurate Text Embedding Models Model Architecture Architecture type: Transformer (BERT) Network architecture: Snowflake/snowflake arctic embed m How To Use in NeMo Curator NeMo Curator improves generative…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy