FineWeb + FineWeb 2 with served from region annotations (URL only, resolved only) A region annotated, hive partitioned view of HuggingFaceFW/fineweb (the sample 100BT slice) and HuggingFaceFW/fineweb 2 . Every record carries an ISO 3166 1 alpha 2 region label produced by the URL only resolution pipeline described in the companion paper. Records with region == "XX" (unresolved) have been dropped ; to access them, use the original FineWeb / FineWeb 2 datasets and rejoin on (source dataset, id) . Records: 2,960,753,780 (resolved only) Locales (language script × region): 30,012 Languages (resolved): 525 • Regions: 243 Layout: hive partitioned data/language script={ls}/region={r}/{source dataset} .parquet Resolution rate, not precision. The 57.20 % combined resolution rate (against the full input including XX) is recall like — a measure of whether any label was produced, not whether it is correct. Label precision is not yet evaluated. Use the region confidence column to filter ( high / medium / low are the lookup backed tiers; ccTLD and url hint are weaker; rows with no signal were dropped). Schema Column Type Notes text string Document body, verbatim from the source dataset. id string…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy