All concepts
Instruction Tuning
Fine-tune a pretrained model on instruction-response examples so it follows tasks.
Transformers & LLMs · Advanced · ~8 min
In plain English
A raw model just continues text. Instruction tuning trains it on thousands of (request → good answer) pairs until following an instruction becomes its default behaviour.
Why it's worth your time
It's the step between 'a model that predicts text' and 'a model you can talk to' — and the format is where most fine-tunes go wrong.
If you remember three things
- Supervised fine-tuning on instruction/response pairs
- Quality and diversity of instructions beat raw volume
- The chat template must match exactly at train and serve time
Overview
Fine-tuning a pretrained language model on (instruction, response) pairs so it reliably follows tasks instead of just continuing text. Supervised imitation of high-quality responses teaches format adherence and helpfulness, turning a raw base model into a usable chat or tool-using assistant.
How it works
- Start: Base LM A pretrained model predicts text but may not follow user instructions reliably.
- Base LM -> Instruction Data Examples pair an instruction with an ideal response.
- Instruction Data -> Supervised FT Train the model to imitate high-quality responses.
- Supervised FT -> Eval Check helpfulness, safety, format adherence, and regressions.
- Eval -> Instruct Model The model becomes better at chat, tools, and user-facing tasks.
In an interview
Instruction tuning is supervised fine-tuning on instruction-response examples. A base model predicts plausible text but doesn't reliably do what you ask; training it to imitate curated ideal responses aligns it to follow instructions, hold formats, and handle chat and tools. It's the step before preference optimization like RLHF or DPO.
Production defaults
- Data
- a few thousand diverse, well-written examples. Curation matters far more than count
- Loss
- compute it on the response tokens only, not on the instruction
- Template
- use the model's own chat template verbatim. A mismatched template is the most common silent fine-tuning bug
What breaks
- Fine-tuned model ignores instructions — Template mismatch between training and inference. Print the exact tokenized string in both and diff them.
- Answers are the right shape but shallow — Templated training data taught format, not reasoning. Diversify difficulty and phrasing.