Model card for vit large patch16 dinov3.lvd1689m A DINOv3 ViT model image feature encoder. Distilled on LVD 1689M from the DINOv3 ViT 7B model. Model Notes The original model weights ended up with all QKV projection biases being zeroes. For timm , have disabled the QKV bias ( qkv bias=False ) for the models and not loaded the zero weights. For some model sizes there are variants with qkvb in the name that have the bias enabled ( qkv bias=True ), but zero, to match the behaviour of transformers and original models. The original models keep RoPE periods as a persistent bfloat16 buffer. timm generates float32 periods at init. This results in some numerical differences, however the timm approach should be less problematic running on devices without bfloat16 support, and appears to work as well if not slightly better for fine tuning. model.rope.periods = model.rope.periods.to(torch.bfloat16).to(torch.float32) will truncate the periods to bfloat16 and result in matching outputs. Model Details Model Type: Image Feature Encoder Model Stats: Params (M): 303.1 GMACs: 82.4 Activations (M): 90.6 Image size: 256 x 256 Original: https://github.com/facebookresearch/dinov3 License: DINOv3 Dataset:…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy