arxiv.org/abs/2411.18363 1. Introduction 📚 TL;DR: ChatRex is an MLLM skilled in perception that can respond to questions while simultaneously grounding its answers to the referenced objects. ChatRex is a Multimodal Large Language Model (MLLM) designed to seamlessly integrate fine grained object perception and robust language understanding. By adopting a decoupled architecture with a retrieval based approach for object detection and leveraging high resolution visual inputs, ChatRex addresses key challenges in perception tasks. It is powered by the Rexverse 2M dataset with diverse image region text annotations. ChatRex can be applied to various scenarios requiring fine grained perception, such as object detection, grounded conversation, grounded image captioning and region understanding. 2. Installation 🛠️ 2.1 Download Pre trained UPN Models We provide model checkpoints for both the Universal Proposal Network (UPN) and the ChatRex model . You can download the pre trained models from the following links: UPN Checkpoint ChatRex 7B Checkpoint Or you can also using the following command to download the pre trained models: 2.2 Verify Installation To verify the installation of the Univer…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy