I JEPA Model (Huge, fine tuned on IN1K) I JEPA is a method for self supervised learning. At a high level, I JEPA predicts the representations of part of an image from the representations of other parts of the same image: 1. without relying on pre specified invariances to hand crafted data transformations, which tend to be biased for particular downstream tasks, 2. and without having the model fill in pixel level details, which tend to result in learning less semantically meaningful representations. How does it work? As opposed to generative methods that have a pixel decoder, I JEPA has a predictor that makes predictions in latent space. The predictor in I JEPA can be seen as a primitive (and restricted) world model that is able to model spatial uncertainty in a static image from a partially observable context. This world model is semantic in the sense that it predicts high level information about unseen regions in the image, rather than pixel level details. We trained a stochastic decoder that maps the I JEPA predicted representations back in pixel space as sketches. The model correctly captures positional uncertainty and produces high level object parts with the correct pose (e.g.…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy