Liberating 3T of the finest tokens from PDFs What is this? As we run out of web pages to process, the natural question has always been: what to do next? Only a few knew about a data source that everyone avoided for ages, due to its incredible extraction cost and complexity: PDFs . 📄 FinePDFs is exactly that. It is the largest publicly available corpus sourced exclusively from PDFs, containing about 3 trillion tokens across 475 million documents in 1733 languages . Compared to HTML datasets, despite being only mildly filtered, it achieves results nearly on par with state of the art collections such as the SmolLM 3 Web mixture. More importantly, when mixed with HTML based corpora, it delivers a significant performance boost across benchmarks 🚀. The data was sourced from 105 CommonCrawl snapshots, spanning the summer of 2013 to February 2025 , as well as refetched from the internet, and processed using 🏭 datatrove , our large scale data processing library. This carefully deduplicated and filtered dataset comprises roughly 3.65 terabytes of 3T tokens. For PII and copyright opt out see Personal and Sensitive Information and opt out . As is tradition, the dataset is fully reproducible…
Runs entirely in your browser via DuckDB-Wasm — this dataset's real data file is loaded once, then queried locally. Nothing is sent to a server.
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy