gift-pretrain-full-4096

gift-pretrain-full-4096 Full counterpart to jeremycochoy/gift-pretrain-small-4096: every series of every arrow file in Salesforce/GiftEvalPretrain, cropped into non-overlapping 4096-point windows and globally shuffled. The small companion sub-samples 10 series per sub-dataset; this one keeps everything. 6,376 source arrow files across 152 sub-datasets fully consumed 42,571,692 windows of length 4096 (float32) 4,274 parquet shards, ~619 GB total (zstd) Layout .… See the full description on the dataset page: https://huggingface.co/datasets/jeremycochoy/gift-pretrain-full-4096.