ViT from Scratch — CIFAR 10 (IT3103 Week 15) A small Vision Transformer trained from scratch on CIFAR 10 for teaching purposes. Final test accuracy: 76.25% . This is a plain PyTorch nn.Module (see the ViT / TransformerBlock / PatchEmbedding classes in the course notebook), not a 🤗 Transformers model. To use it, recreate the architecture from the notebook and load the weights: Config: {"architecture": "ViT (from scratch, custom nn.Module)", "img size": 32, "patch size": 4, "in channels": 3, "num classes": 10, "embed dim": 256, "depth": 4, "num heads": 8, "mlp ratio": 4.0, "dropout": 0.1, "classes": ["airplane", "automobile", "bird", "cat", "deer", "dog", "frog", "horse", "ship", "truck"], "normalize mean": [0.4914, 0.4822, 0.4465], "normalize std": [0.247, 0.2435, 0.2616], "final test acc": 76.25}
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy