LIBERO Plus: In depth Robustness Analysis of Vision Language Action Models 📄 Paper 🏗️ Repo 🌐 Website 🔥 Overview This repository contains the official implementation and benchmark for our paper "In depth Robustness Analysis for Vision Language Action Models". We systematically expose the hidden vulnerabilities of contemporary VLA models through comprehensive robustness evaluation across seven perturbation dimensions. You can simply replace the original libero with a pip install e . without modifying your code. 🚀 Key Findings Significant Fragility : VLA models exhibit extreme sensitivity to camera viewpoints and robot initial states, with performance dropping from 95% to below 30% under modest perturbations Language Ignorance : Models largely ignore language instructions, functioning more like Vision Action models Negative Compositional Generalization : Combined perturbations reveal complex interaction effects beyond independent factors 📊 LIBERO plus Benchmark 7 Perturbation Dimensions We introduce LIBERO plus , a comprehensive benchmark with 10,030 tasks spanning: 1. Objects Layout Confounding objects and target object displacement 2. Camera Viewpoints Position, orientation, a…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy