Kandinsky 2.1 Kandinsky 2.1 inherits best practices from Dall E 2 and Latent diffusion while introducing some new ideas. It uses the CLIP model as a text and image encoder, and diffusion image prior (mapping) between latent spaces of CLIP modalities. This approach increases the visual performance of the model and unveils new horizons in blending images and text guided image manipulation. The Kandinsky model is created by Arseniy Shakhmatov, Anton Razzhigaev, Aleksandr Nikolich, Igor Pavlov, Andrey Kuznetsov and Denis Dimitrov Usage Kandinsky 2.1 is available in diffusers! Text to image Text Guided Image to Image Generation Interpolate Model Architecture Overview Kandinsky 2.1 is a text conditional diffusion model based on unCLIP and latent diffusion, composed of a transformer based image prior model, a unet diffusion model, and a decoder. The model architectures are illustrated in the figure below the chart on the left describes the process to train the image prior model, the figure in the center is the text to image generation process, and the figure on the right is image interpolation. Specifically, the image prior model was trained on CLIP text and image embeddings generated wit…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy