Knowledge distillation
Training a smaller student model on a larger teacher's evaluated outputs so a defined task can run at lower cost or latency, or with stronger isolation.
Category
How a model is chosen and adapted
Related terms
Distillation transfers evaluated behavior from a larger or specialist teacher model into a smaller student on a defined task. The teacher supplies probability targets, reviewed responses, labels, or rationales; the student trains to reproduce them.
When it is the right move
The case is a real deployment benefit, not elegance: latency the larger model cannot meet, operating cost at production volume, isolation requirements that rule out the teacher's runtime, or hardware the teacher does not fit. If none of those bind, the distillation adds an artifact and a maintenance obligation for nothing.
What has to be true first
Teacher rights and access. Data and output rights. Base-model and provider terms that permit training on those outputs. Privacy boundaries that permit the corpus to exist. A representative evaluation set for the student's actual task. Without all of them, the student cannot be shipped regardless of how well it scores.
The student is its own artifact
A distilled student does not inherit the teacher's accuracy, safety behavior, privacy properties, permissions, or operating controls. It is evaluated, versioned, approved, monitored, and rolled back on its own terms. Where the student is worse, that has to be measured and accepted rather than assumed away.
Distillation also never moves institutional policy, identity, permissions, approval authority, or stop conditions into weights. Those remain external and inspectable no matter which model runs.
Often confused with
How it differs from similar terms.
| knowledge distillation | Compared with | What's different |
|---|---|---|
| Distillation | Fine-tuning | Distillation learns from a teacher model's outputs. Fine-tuning learns from task examples directly. |
| Student model | Teacher model | The student is a separate approved artifact. It inherits none of the teacher's guarantees automatically. |
Want these terms worked through on one of your own decisions? Start with a Decision Blueprint