Vero-1.6M

[!Note] This repository contains an expanded version of Vero-600K, using the same dataset curation and filtering process, but with a larger set. Note that task categories are not balanced in this dataset.

Vero is a fully open reinforcement learning (RL) recipe for training and evaluating multi-task visual reasoning with vision-language models. This repository contains the Vero-600K dataset, a curation of 600K reinforcement learning samples from 59 datasets across 6 diverse visual reasoning categories.

Highlights

• Scale: 1.6M curated RL samples from 59 datasets.
• Diversity: Covers 6 broad categories: STEM Reasoning, Chart & OCR, Spatial & Action, Knowledge & Recognition, Grounding & Counting, and Instruction Following.
• Task-Routed Rewards: Designed to handle heterogeneous answer formats across diverse tasks.
• Open Recipe: Fully open release of models, training code, evaluation suite, and dataset.

Dataset Structure

The dataset is organized into six broad task categories:

1. STEM reasoning
2. Chart and OCR
3. Spatial reasoning and action
4. Knowledge and recognition
5. Grounding, counting, and visual search
6. Captioning and instruction following

For detailed dataset format, curation details, and reward routing metadata, see the Data Guide.

Sample Usage

To download and format the Vero-1.6M dataset for RL training using the official setup script (note that you will need to specify this Vero-1.6M in the script):

# Clone the repository
git clone https://github.com/zlab-princeton/vero.git
cd vero

# Run the formatting script
python scripts/download_and_format_vero_600k.py