BEHAVIOR-1K MP-Collected — turning_on_radio

Combined dataset for BEHAVIOR-1K task 0 (turning_on_radio):

• 1154 success demos + 846 failure demos collected by a hybrid motion-planner + X-VLA policy pipeline on instances 301–700 (private test set, 400 instances × 5 episodes)
• 200 success demos from the original BEHAVIOR-1K teleoperated dataset (behavior-1k/2025-challenge-demos), merged into success/

Total: 1354 success + 846 failure = 2200 episodes (~157 GB).

Layout

success/2025-challenge-demos/
  data/task-0000/episode_XXXXXXXX.parquet                # simplified obs/action
  annotations/task-0000/episode_XXXXXXXX.json            # skill + primitive annotation
  meta/episodes/task-0000/episode_XXXXXXXX.json          # per-episode metadata (MP-collected only)
  meta/episodes/task-0000/episode_XXXXXXXX_bddl_transitions.json
  trajectories/task-0000/episode_XXXXXXXX.hdf5           # raw OmniGibson trajectory (MP-collected only)
  videos/task-0000/
    observation.images.rgb.head/episode_XXXXXXXX.mp4              # all 1354 demos
    observation.images.rgb.left_wrist/episode_XXXXXXXX.mp4        # all 1354 demos
    observation.images.rgb.right_wrist/episode_XXXXXXXX.mp4       # all 1354 demos
    observation.images.rgb.external/episode_XXXXXXXX.mp4          # MP-collected only (1154)
    observation.images.depth.head/episode_XXXXXXXX.mp4            # MP-collected only (1154)
    observation.images.depth.left_wrist/episode_XXXXXXXX.mp4      # MP-collected only (1154)
    observation.images.depth.right_wrist/episode_XXXXXXXX.mp4     # MP-collected only (1154)
failure/2025-challenge-demos/   (same structure, MP-collected failures only)

How to use the dataset

Download

huggingface-cli download Hoshipu/behavior-1k-mp-collected-turning-on-radio \
    --repo-type=dataset \
    --local-dir ./behavior-1k-mp-collected

Or from Python:

from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="Hoshipu/behavior-1k-mp-collected-turning-on-radio",
    repo_type="dataset",
    local_dir="./behavior-1k-mp-collected",
)

Files are stored individually and match the BEHAVIOR-1K canonical layout — no extraction step required.

demo_id ranges

Coverage by artifact

• parquet, annotations: all 2200 episodes
• meta + bddl_transitions, hdf5, depth videos, rgb.external video: only MP-collected (2000 episodes); HF-original demos lack these
• rgb.head / rgb.left_wrist / rgb.right_wrist videos: all 2200 episodes
• phase_segments: only MP-collected (2000 episodes); see below

Per-episode skill annotation

Each annotations/.../episode_XXXXXXXX.json follows the HF challenge schema with 4 skills (move to, pick up from, press, place on) and 3 primitives. For MP-collected demos, the skill boundaries are derived from the orchestrator's per-action enter/exit step numbers.

Orchestrator phase segments (MP-collected only)

phase_segments/task-0000/episode_XXXXXXXX_orchestrator_phases.json exposes the finer-grained sub-phase trace of the hybrid motion-planner + X-VLA orchestrator that produced each MP-collected episode. The 4 skill annotations group runs of related sub-phases; these files preserve the underlying 6–7 sub-phases with exact step boundaries and per-phase metadata (matched retrieval demos, IK convergence errors, grasp variant, etc.).

Schema

{
  "episode_id":   7002,
  "task_id":      0,
  "instance_id":  700,
  "trial_index":  2,
  "success":      true,
  "total_steps":  1525,
  "source_log":   "2910_b1k_mp_collect.err",
  "subphases": [
    {"name": "navigate_to_radio",      "executor": "policy", "expected_phase": 0, "step_lo": 0,    "step_hi": 50,   "n_steps": 51},
    {"name": "pick_up_radio_approach", "executor": "mp",     "expected_phase": 1, "step_lo": 51,   "step_hi": 500,  "n_steps": 450,
        "metadata": {"matched_demo": 460, "grasp_variant": "B", "approach_dist": 2.5393, "bridge_len": 100, "traj_len": 350, "total_len": 450}},
    {"name": "pick_up_radio_grasp",    "executor": "mp",     "expected_phase": 1, "step_lo": 501,  "step_hi": 577,  "n_steps": 77,
        "metadata": {"grasp_variant": "B", "radio_xyz": [...], "grasp_xyz": [...], "grasp_quat": [...],
                     "ik_waypoints": [{"wp": 1, "of": 2, "status": "CONVERGED", "err_rad": 0.0056, "retries": 0}, ...]}},
    {"name": "close_right_gripper",    "executor": "mp",     "expected_phase": 1, "step_lo": 578,  "step_hi": 607,  "n_steps": 30,
        "metadata": {"source": "state23", "r_grip_target": -1.0, "hold_frames": 30, "prev_r_grip_state": -0.265}},
    {"name": "press_radio",            "executor": "mp",     "expected_phase": 2, "step_lo": 608,  "step_hi": 1517, "n_steps": 910,
        "metadata": {"matched_demo": 930, "press_dist": 0.2211, "bridge_len": 716, "traj_len": 194, "total_len": 910}},
    {"name": "wrap_up_policy",         "executor": "policy", "expected_phase": 3, "step_lo": 1518, "step_hi": 1524, "n_steps": 7,
        "metadata": {"max_frames": 300}}
  ]
}

Sub-phase list

step_lo / step_hi are inclusive frame indices into the parquet (step_hi == total_steps - 1 for the last sub-phase). Phases that didn't execute in a given episode (e.g. put_down_radio when the radio toggled on during press_radio) are simply absent.