math fasttext classifier Dataset This is part of my fasttext classifier collection for curating pretraining dataset. This classifier classifies a text into Maths or Others. The model is trained over 1.6M records, which is a 50:50 mix of maths and non maths in website and achieved a test F1 score of 0.99 (too good to be true?). It is an intended upsampling of maths data. The classifier can be used for LLM pretraining data curation, to enhance capability in mathematics. It is ultra fast ⚡ with a throughtput of ~2000 doc/s with CPU. Don't underestimate the "old" fasttext classiifer! It is indeed a good and scalable practice. For example, QWEN2.5 MATH leverages fasttext to curate pretraining data, althought its classifier is not open sourced. 🛠️Usage 📊Evaluation full version ⚠️Known Limitation The classifier does not handle short text well, which might not be surprising. Citation
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy