Document Image Transformer (base sized model) Document Image Transformer (DiT) model pre trained on IIT CDIP (Lewis et al., 2006), a dataset that includes 42 million document images. It was introduced in the paper DiT: Self supervised Pre training for Document Image Transformer by Li et al. and first released in this repository. Note that DiT is identical to the architecture of BEiT. Disclaimer: The team releasing DiT did not write a model card for this model so this model card has been written by the Hugging Face team. Model description The Document Image Transformer (DiT) is a transformer encoder model (BERT like) pre trained on a large collection of images in a self supervised fashion. The pre training objective for the model is to predict visual tokens from the encoder of a discrete VAE (dVAE), based on masked patches. Images are presented to the model as a sequence of fixed size patches (resolution 16x16), which are linearly embedded. One also adds absolute position embeddings before feeding the sequence to the layers of the Transformer encoder. By pre training the model, it learns an inner representation of images that can then be used to extract features useful for downstrea…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy