Vision Transformer (small sized model) trained using DINOv2, with registers Vision Transformer (ViT) model introduced in the paper Vision Transformers Need Registers by Darcet et al. and first released in this repository. Disclaimer: The team releasing DINOv2 with registers did not write a model card for this model so this model card has been written by the Hugging Face team. Model description The Vision Transformer (ViT) is a transformer encoder model (BERT like) originally introduced to do supervised image classification on ImageNet. Next, people figured out ways to make ViT work really well on self supervised image feature extraction (i.e. learning meaningful features, also called embeddings) on images without requiring any labels. Some example papers here include DINOv2 and MAE. The authors of DINOv2 noticed that ViTs have artifacts in attention maps. It’s due to the model using some image patches as “registers”. The authors propose a fix: just add some new tokens (called "register" tokens), which you only use during pre training (and throw away afterwards). This results in: no artifacts interpretable attention maps and improved performances. Visualization of attention maps of…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy