Vero-2.5M-unfiltered

[!Note] This repository contains the full unfiltered dataset used to construct Vero-600k and Vero-1.6M, before question and answer filtering. Note that task categories are not balanced in this dataset.

Vero is a fully open reinforcement learning (RL) recipe for training and evaluating multi-task visual reasoning with vision-language models. This repository contains the Vero-2.5M-unfiltered dataset, a curation of 2.5M reinforcement learning samples from 59 datasets across 6 diverse visual reasoning categories.

Highlights

• Scale: 2.5M RL samples from 59 datasets.
• Diversity: Covers 6 broad categories: STEM Reasoning, Chart & OCR, Spatial & Action, Knowledge & Recognition, Grounding & Counting, and Instruction Following.
• Task-Routed Rewards: Designed to handle heterogeneous answer formats across diverse tasks.
• Open Recipe: Fully open release of models, training code, evaluation suite, and dataset.

Dataset Structure

The dataset is organized into six broad task categories:

1. STEM reasoning
2. Chart and OCR
3. Spatial reasoning and action
4. Knowledge and recognition
5. Grounding, counting, and visual search
6. Captioning and instruction following

For detailed dataset format, curation details, and reward routing metadata, see the Data Guide.

Sample Usage

To download and format the Vero-2.5M dataset for RL training using the official setup script (note that you will need to specify this Vero-2.5M-unfiltered in the script):

# Clone the repository
git clone https://github.com/zlab-princeton/vero.git
cd vero

# Run the formatting script
python scripts/download_and_format_vero_600k.py