BLIP3o-Pretrain-JourneyDB

BLIP3o Pretrain JourneyDB Dataset

This collection contains 4 million JourneyDB images.

Download

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="BLIP3o/BLIP3o-Pretrain-JourneyDB",
    repo_type="dataset"
)

Load Dataset without Extracting

You don’t need to unpack the .tar archives, use WebDataset support in 🤗datasets instead:

from datasets import load_dataset
import glob

data_files = glob.glob("/your/data/path/*.tar")
train_dataset = load_dataset(
    "webdataset",
    data_files=data_files,
    cache_dir="/your/hf/cache",
    split="train",
    num_proc=128
)

Feel free to comment on it when you have any issues.