TokenShrink OCR Dataset Introduction This is a large scale dataset containing 120,000 images, designed for Optical Character Recognition (OCR) tasks. All images are derived from the ImageNet database, providing a challenging collection of text against complex backgrounds, varied lighting conditions, and diverse fonts. Dataset Structure All image files are stored in a sharded structure. All data ( train , validation , test ) has been split into small folders, each containing 1,000 files. The directory structure in the remote repository is as follows: How to Use You can easily load all sharded data using the datasets library, the imagefolder loader, and a glob (wildcard) pattern. Install Dependencies python from datasets import load dataset REPO ID = "LukB4UJump/TokenShrink OCR" IMAGE EXTENSION = "jpg" data files = { "train": f"{REPO ID}::train/ / .{IMAGE EXTENSION}", Matches train/000/ .jpg, train/001/ .jpg ... "validation": f"{REPO ID}::validation/ / .{IMAGE EXTENSION}", "test": f"{REPO ID}::test/ / .{IMAGE EXTENSION}" } Use the "imagefolder" loader streaming=True allows you to access the data without downloading all 120k images, saving disk space dataset = load dataset( "imagefold…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy