MLS-Sidon

Overview

This dataset is a cleansed version of Multilingual LibriSpeech (MLS) with Sidon speech restoration mode for Speech Synthesis and Spoken Language Modeling.

The dataset is provided in WebDataset format for efficient large-scale training.

• Source: Multilingual LibriSpeech
• Languages: English, German, French, Spanish, Italian, Polish, Dutch, Portuguese
• Format: WebDataset (.tar shards)
• License: CC-BY-4.0

Dataset Structure

Each sample in the dataset contains:

• flac — audio file (48 kHz, single channel)
• metadata.json (optional) — metadata including language, speaker ID, and original MLS reference

Example (inside a .tar shard):

000001.flac
000001.metadata.json
000002.flac
000002.metadata.json
...

How to Use

With 🤗 Datasets

You can load the WebDataset directly with Hugging Face’s datasets library:

import datasets
from IPython.display import Audio
from huggingface_hub import hf_hub_download
import yaml

base_url = "https://huggingface.co/datasets/sarulab-speech/mls_sidon/resolve/main/"
language = 'english'
split = 'test'
data_file_path = hf_hub_download(repo_id="sarulab-speech/mls_sidon", repo_type="dataset", filename="paths.yaml")
paths = yaml.load(open(data_file_path, "r"), Loader=yaml.FullLoader)

ds = datasets.load_dataset("webdataset", data_files=[base_url + p for p in paths['english'][split]],streaming=True)['train']

sample = next(iter(ds))
audio = sample['flac']
print(sample['metadata.json'])
Audio(audio['array'], rate=audio['sampling_rate'])

Replace language with the language (e.g., english, german).

Citation

If you use this dataset, please cite Sidon and the original MLS paper:

@misc{nakata2025sidonfastrobustopensource,
      title={Sidon: Fast and Robust Open-Source Multilingual Speech Restoration for Large-scale Dataset Cleansing}, 
      author={Wataru Nakata and Yuki Saito and Yota Ueda and Hiroshi Saruwatari},
      year={2025},
      eprint={2509.17052},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2509.17052}, 
}

@inproceedings{pratap2020mls,
  title     = {MLS: A Large-Scale Multilingual Dataset for Speech Research},
  author    = {Pratap, Vineel and Xu, Qiantong and Sriram, Anuroop and others},
  booktitle = {Interspeech},
  year      = {2020}
}

License

This dataset is released under CC-BY-4.0.

Acknowledgements

• Original data: Multilingual LibriSpeech (MLS)