Dataset Card GPT-NL Public Corpus

The GPT-NL Public Corpus is the largest permissively licensed Dutch-language resource available for large language model pretraining. It consists of 29 curated collections totaling over 524 billion tokens, including 36B Dutch, 207B English, 232B code, and 48B German/Danish tokens. All data is sourced under permissive licensing and redistributed under a CC-BY license. For more details please refer to our Public Corpus article.

Dataset Description

• Curated by: GPT-NL Team
• Language(s) (NLP): Dutch, English, German, Danish, Frisian, Code
• License: CC-BY. If you use (part of) the Public Corpus, please cite like stated below.
  Note: this is the license of the corpus. However, all datapoints have specific licenses attributed in the metadata field. These licenses are based on the license of the original source dataset that was curated or augmented for the purpose of creating the GPT-NL public corpus. Author names are given for collections with a CC-BY license. For Code datasets, we link the original Github link where the original license conditions can be found.

Dataset Structure

We have collected metadata of all 29 collections through doing interviews, doing research on the reporting about these datasets and by examing quality signals from our curation pipeline per dataset. This result in metadata on a datapoint-level and metadata on a dataset-level. The meta-data on a dataset-level is shown here. The fields on datapoint-level are shown below:

General Metadata

Quality Signal Metadata

For more quality signal information (for the Dutch subsets), please check out https://huggingface.co/datasets/tvosch/GPT-NL-propella-annotations!

Dataset Creation

Curation Rationale

The GPT-NL Corpus is intended to enable LLM applications that are secure, transparent and trustworthy. The GPT-NL Corpus is used in the training of GPT-NL, a Dutch language model developed in line with European and Dutch rules and public values. The data and information on this page are meant to facilitate researchers and developers when evaluation and creating responsible AI solutions and is meant to provide transparency in the data-sources used when training GPT-NL.

Source Data

Table: Overview of datasets in the GPT-NL Public Corpus

Data Collection and Processing

As highlighted in the table above, we have done data collection in four major ways:

1. Collected data, open-license data that we aggregated in collections (after crawling/digitalization efforts).
2. Selected data from already existing permissively licensed collections
3. Synthetic data, created from already existing collections, translated to llm-training ready format.
4. Filtered web-crawls, created from filtering permissively licensed data from common crawl dumps.

We have created our data curation pipeline implementation, including heuristic filtering, language filtering, PII and harmful language removal. Check out our Github!

Who are the source data producers?

The data in the GPT-NL is sourced from many different places. Therefore we have asked all data contributors to fill in a comprehensive survey about their dataset and the authors of the data. The results of these surveys with more in-detail information about the sourcing for individual collections can be found here (for the GPT-NL Public and Private Corpus).

Personal and Sensitive Information

The PII Removal curation step is added to remove all structured instances of personal data such as email addresses, phone numbers, and credit card numbers, for both Public Persons and Non-Public Persons. Subsequently, it removes any remaining personal data such as names for Non-public Persons using contextual anonymization techniques. For public Persons (persons with a dedicated WikiPedia page), some personal information is kept (e.g. name, age) for the purpose of model performance. If you notice any personal data that is either sensitive or from non-public Persons, please inform us via the .

Bias, Risks, and Limitations

As with any corpus of this size, there will always be text that is potentially problematic. The highest risk of outdated views embedded in text is present in the archive/public domain collections. For Dutch content, this dataset is heavily skewed towards professional text from governmental, archival or judicial environments. Keep in mind that models trained on this corpus will likely suffer in performance in domains and language use out of this scope.

Recommendations

The collections in this corpus can be used for training meaningful compliant machine learning models, including large language models. The collected and synthetic collections are to our knowledge, not readily available anywhere else. Furthermore, since PII removal has been done already for all collections in this set, it is easier to create applications that are GDPR-compliant. When doing so, analysis the individual collections you aim by inspecting the data and looking at the global information given in the metadata.

How to Contribute

A lot of public (Germanic) data available is not yet included in this set. If you have or know of permissive data (CC-0, CC-BY, Public Domain) that is not yet included (from pdfs from (public) institutions to voice-data we can transcribe), please send an email to us. If approved, we will evaluate the data, curate it and put it online as part of the Public Corpus in an effort to create more high-quality compliant Germanic data for AI models and applications.

Acknowledgements

The creation of the GPT-NL public corpus is done for the GPT-NL project which is funded by the Ministry of Economic Affairs of the Netherlands and executed by TNO in collaboration with SURF and The Netherlands Forensic Institute (NFI). For the collected datasets Openraadsinfromatie, Officiële bekendmakingen, Koninklijke Bibliotheek, Naturalis, PBL, and European Parliament the Open State Foundation completed the sourcing of the content. We want to thank the proactive stance of the people from the content departments of the Rijksoverheid, VNG, Woogle, Koninklijke Bibliotheek, De Rechtspraak, Tweede Kamer, Nationaal Archief, Utrechts Archief, Noord-Hollands Archief, Zeeuws Archief, Wikiwijs, PBL, Naturalis, and DANS-KNAW.

Thanks to the Open State Foundation for their help with the collections of Openraadsinformatie, Officiële bekendmakingen, PBL, Koninklijke Bibliotheek, and European Parliament.

Thanks to the kind help from the following parties in contributing their collections:

Thanks to PleIAs for their work on Common Corpus.

Citation

If you are using the GPT-NL Public Corpus, please cite our work via:

@misc{vanoort2026gptnlpubliccorpuspermissively,     
      title={GPT-NL Public Corpus: A Permissively Licensed, Dutch-First Dataset for LLM Pre-training},    
      author={Jesse van Oort and Frank Brinkkemper and Erik de Graaf and Bram Vanroy and Saskia Lensink},  
      year={2026},  
      eprint={2604.00920},  
      archivePrefix={arXiv},  
      primaryClass={cs.CL},  
      url={https://arxiv.org/abs/2604.00920},   
}

Dataset Card Contact

If you have any suggestions, possible contributions or questions with respect to the public corpus, please feel free to reach out to us.

GPT-NL Team Contact