RoboCasa Pretraining Dataset
Video trajectory dataset for robot manipulation pretraining in the RoboCasa kitchen simulation environment. Contains two data splits: synthetically generated MimicGen demonstrations and human-teleoperated demonstrations.
Dataset Summary
| Split | Examples | Description |
|---|---|---|
robocasa_pretrain_mimicgen | 536,030 | MimicGen-generated trajectories |
robocasa_pretrain_human | 32,043 | Human-teleoperated trajectories |
| Total | 568,073 |
Each example corresponds to one robot manipulation trajectory rendered as an MP4 video clip, paired with structured metadata.
Data Structure
Each split is a Parquet index paired with video files stored as trajectory_XXXX.mp4 in batch subdirectories.
Fields
| Field | Type | Description |
|---|---|---|
id | string | Unique trajectory identifier |
task | string | Task name (e.g., kitchen manipulation task) |
lang_vector | float32[] | Language embedding vector for the task instruction |
data_source | string | Data collection method (mimicgen or human) |
frames | string | Path to the corresponding MP4 video file |
is_robot | bool | Whether the trajectory was executed by a robot |
quality_label | string | Trajectory quality annotation |
partial_success | float32 | Partial task completion score [0, 1] |
Usage
from datasets import load_dataset
# Load MimicGen split
ds_mim = load_dataset("myconnects/robocasa-pretrain", "robocasa_pretrain_mimicgen")
# Load Human split
ds_hum = load_dataset("myconnects/robocasa-pretrain", "robocasa_pretrain_human")
About RoboCasa
RoboCasa is a large-scale simulation framework for training generalist robot policies in diverse kitchen environments. It provides photorealistic kitchen scenes with a wide variety of objects and tasks, designed for pretraining visuomotor policies.
Citation
If you use this dataset, please cite the RoboCasa paper:
@inproceedings{robocasa2024,
title = {RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots},
author = {Soroush Nasiriany and Abhiram Maddukuri and Lance Zhang and Adeet Parikh and Aaron Lo and Abhishek Joshi and Ajay Mandlekar and Yuke Zhu},
booktitle = {Robotics: Science and Systems},
year = {2024}
}