Quickstart

The fastest way to see TRL in action is to run a supervised fine-tune on a small instruction dataset.

from datasets import load_dataset
from trl import SFTConfig, SFTTrainer

dataset = load_dataset("trl-lib/Capybara", split="train")

trainer = SFTTrainer(
    model="Qwen/Qwen2.5-0.5B",
    train_dataset=dataset,
    args=SFTConfig(output_dir="Qwen2.5-0.5B-SFT"),
)
trainer.train()

That's the whole loop — TRL handles tokenization, packing, the chat template, and checkpointing.

Launch on multiple GPUs

Every TRL script is an accelerate script. Save the code above as sft.py and run:

accelerate launch --multi_gpu --num_processes 4 sft.py

Add a DeepSpeed ZeRO-3 config for large models:

accelerate launch --config_file deepspeed_zero3.yaml sft.py

Next, dig into each method: SFT, DPO, and GRPO.