GUI World: A Dataset for GUI Orientated Multimodal Large Language Models Dataset: GUI World Overview GUI World introduces a comprehensive benchmark for evaluating MLLMs in dynamic and complex GUI environments. It features extensive annotations covering six GUI scenarios and eight types of GUI oriented questions. The dataset assesses state of the art ImageLLMs and VideoLLMs, highlighting their limitations in handling dynamic and multi step tasks. It provides valuable insights and a foundation for future research in enhancing the understanding and interaction capabilities of MLLMs with dynamic GUI content. This dataset aims to advance the development of robust GUI agents capable of perceiving and interacting with both static and dynamic GUI elements. How to use GUI World See Github for further details. Based on GUI World, we train the first VideoLLM GUI Vid with powerful GUI understanding capability. License This work is licensed under a Creative Commons Attribution 4.0 International License. Citation
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy