Dataset Card for "oscar" Table of Contents Dataset Description Dataset Summary Supported Tasks and Leaderboards Languages Dataset Structure Data Instances Data Fields Data Splits Dataset Creation Curation Rationale Source Data Annotations Personal and Sensitive Information Considerations for Using the Data Social Impact of Dataset Discussion of Biases Other Known Limitations Additional Information Dataset Curators Licensing Information Citation Information Contributions Dataset Description Homepage: https://oscar corpus.com Repository: github.com/oscar corpus/corpus Paper: Towards a Cleaner Document Oriented Multilingual Crawled Corpus Point of Contact: Contact Dataset Summary OSCAR or O pen S uper large C rawled A ggregated co R pus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the ungoliant architecture. Data is distributed by language in both original and deduplicated form. We are aware of the virus warnings issue. See discussion here for more info! Usage Supported Tasks and Leaderboards OSCAR is mainly intended to pretrain language models and word representations. Languages All the data is distributed by languag…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy