Vision Transformer (base sized model) Vision Transformer (ViT) model pre trained on ImageNet 21k (14 million images, 21,843 classes) at resolution 224x224. It was introduced in the paper An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale by Dosovitskiy et al. and first released in this repository. However, the weights were converted from the timm repository by Ross Wightman, who already converted the weights from JAX to PyTorch. Credits go to him. Disclaimer: The team releasing ViT did not write a model card for this model so this model card has been written by the Hugging Face team. Model description The Vision Transformer (ViT) is a transformer encoder model (BERT like) pretrained on a large collection of images in a supervised fashion, namely ImageNet 21k, at a resolution of 224x224 pixels. Images are presented to the model as a sequence of fixed size patches (resolution 32x32), which are linearly embedded. One also adds a [CLS] token to the beginning of a sequence to use it for classification tasks. One also adds absolute position embeddings before feeding the sequence to the layers of the Transformer encoder. Note that this model does not provide any fi…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy