LoRA fine-tuning trains a small set of adapter weights while leaving the base model frozen. The result is a file measured in megabytes rather than gigabytes, which makes it cheap to store, quick to share, and easy to swap at serving time. This is an end-to-end path using marketplace hardware. ## Decide whether you need it at all Before renting anything, test whether prompting solves your problem. Write down ten representative inputs with the outputs you want, then try to hit them with a system prompt and two or three few-shot examples in the Playground. Count how many come out right. If prompting gets eight or nine, stop. Fine-tuning to fix two examples costs more than it returns, and you will maintain the result forever. If it stalls well below that, continue. ## Prepare the data LoRA is more sensitive to data quality than to hyperparameters. A few hundred consistent, correct examples beat several thousand noisy ones. Formatting must be consistent, because the model learns the format as readily as the content. If half your examples wrap answers in JSON and half do not, you have taught it that both are acceptable and it will pick one at random. Hold out a test split before you look at the data, and do not train on it. ## Rent and train Pick a machine with enough VRAM for the base model plus optimizer state and gradients. LoRA reduces the trainable parameters dramatically, but the frozen base model still occupies memory, and this is where sizing usually goes wrong. TRL provides the training loop; the docs cover supervised fine-tuning and preference methods including DPO. Start with defaults, change one thing at a time, and keep the run short enough to iterate. A first run exists to prove the pipeline works, not to produce a final model. ## Measure against a baseline Run your evaluation set against the base model **before** training and write down the score. This is the step most often skipped, and without it there is nothing to compare against afterwards — you cannot recover the number later. Then run the identical evaluation against the fine-tuned result. Falling training loss is not evidence of a better model; it is evidence of a model fitting your training set. If the gain is small, check whether a better prompt on the base model closes it. That option costs nothing to serve and nothing to maintain. ## Ship it Push the adapter to a repository on the hub, where it is versioned and can be deployed to an endpoint. Because the adapter is small, keeping several around for different tasks is practical in a way that keeping several full fine-tunes is not. Then stop the instance. Training done and machine still running is the most common way a cheap experiment becomes an expensive one.

Fine-tuning open models with LoRA on rented 5090s