1. Introduction Janus Pro is a novel autoregressive framework that unifies multimodal understanding and generation. It addresses the limitations of previous approaches by decoupling visual encoding into separate pathways, while still utilizing a single, unified transformer architecture for processing. The decoupling not only alleviates the conflict between the visual encoder’s roles in understanding and generation, but also enhances the framework’s flexibility. Janus Pro surpasses previous unified model and matches or exceeds the performance of task specific models. The simplicity, high flexibility, and effectiveness of Janus Pro make it a strong candidate for next generation unified multimodal models. Github Repository 2. Model Summary Janus Pro is a unified understanding and generation MLLM, which decouples visual encoding for multimodal understanding and generation. Janus Pro is constructed based on the DeepSeek LLM 1.5b base/DeepSeek LLM 7b base. For multimodal understanding, it uses the SigLIP L as the vision encoder, which supports 384 x 384 image input. For image generation, Janus Pro uses the tokenizer from here with a downsample rate of 16. 3. Quick Start Please refer to G…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy