DCAD 2000: A Multilingual Dataset across 2000+ Languages with Data Cleaning as Anomaly Detection (NeurIPS 2025) 😊 2025.9.19: DCAD 2000: A Multilingual Dataset across 2000+ Languages with Data Cleaning as Anomaly Detection, has been accepted at NeurIPS 2025 Datasets and Benchmarks Track. + Paper: A Multilingual Dataset across 2000+ Languages with Data Cleaning as Anomaly Detection + Github: https://github.com/yl shen/DCAD 2000 + Dataset (HuggingFace): openbmb/DCAD 2000 🛠️ 2025.11.29: Dataset Storage Optimization Update To address Hugging Face storage capacity limitations and ensure the smooth release of new research from the OpenBMB community, we have performed a major dataset optimization by removing the top 20 languages by data volume: Language Code Data Size (rows) eng Latn 1,311,089,898 rus Cyrl 858,533,615 cmn Hani 713,970,414 deu Latn 668,616,647 spa Latn 604,454,122 fra Latn 513,533,126 jpn Jpan 491,465,489 ita Latn 311,421,021 por Latn 271,475,185 pol Latn 223,213,297 nld Latn 219,161,133 ind Latn 156,915,320 tur Latn 143,311,483 vie Latn 87,769,030 fas Arab 82,798,155 kor Hang 79,219,710 swe Latn 77,323,851 hun Latn 70,788,907 ukr Cyrl 67,869,004 ell Grek 67,476,934 The a…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy