Computer Vision

Compute a ViT sequence length and see why resolution is expensive.

1. Take a 224×224 image with 16×16 patches: 224 ÷ 16 = 14, so 14 × 14 = 196 patches, plus one class token.
2. Now double the resolution to 448×448. Patches go to 28 × 28 = 784 — four times as many.
3. Attention cost scales with the square of sequence length, so that is roughly 16× the attention compute.
4. Compare with a CNN, where doubling resolution costs about 4× — linear in pixel count.

What you should see: This is the concrete reason ViTs are hungry at high resolution, and why CNNs remain competitive when you need fine detail on a budget.

Choose between fine-tuning a backbone and training a head.

1. Count your labelled images per class. Under ~100, freeze the backbone and train only the classifier head.
2. With a few thousand per class, unfreeze the top blocks and fine-tune at a low learning rate.
3. Either way, check how close your images are to the pretraining data. Natural photos transfer well; X-rays and satellite imagery transfer much less.
4. Split the data before you look at it, and keep the test set untouched.

What you should see: A small dataset with a frozen backbone often beats full fine-tuning, which overfits fast. Domain distance matters more than dataset size.

Pick the right kind of segmentation for your question.

1. State what you need to know. "How much of this field is crop?" is semantic — per-pixel classes, instances irrelevant.
2. "How many plants are there?" is instance segmentation — you must separate touching objects.
3. If you need neither and just want to cut one object out interactively, a promptable model like SAM needs no training at all.
4. Check whether your objects touch. Overlapping instances are where semantic masks quietly stop being usable.

What you should see: Choosing instance segmentation when semantic would do costs labelling effort you never needed. The touching-objects test is the deciding question.

Find the edge of what a VLM actually perceives.

1. Give a VLM a photo and ask it to describe the scene. It will do well.
2. Now ask counting questions: how many objects of a given type. Then ask about spatial relations — what is to the left of what.
3. Ask it to read small text in the image.
4. Ask a question about something absent: "What colour is the car?" when there is no car.

What you should see: Description is strong; counting, precise spatial reasoning and small text are weak. The absent-object question is the important one — a model that invents an answer will do so in production too.