0XP
All concepts
Concept 07 of 090/3 mastered

How It Learned

From raw text to helpful assistant

A model arrives in three stages. Pretraining on enormous amounts of text teaches it language. Instruction tuning teaches it to answer rather than continue. Preference training teaches it which answer people actually wanted.

Nobody wrote the rules
Page 01 / 06

Nobody wrote the rules

No linguist sat down and taught these models grammar, and no engineer typed in a list of facts. The behaviour you interact with was produced in two phases with very different jobs, and almost every question people ask about model behaviour is really a question about which phase produced it.

ONE QUESTION: “HOW DO I RESET MY PASSWORD?”1AFTER PRETRAININGWhat is a good way to reset it? How do I change my password?Continues the text. Does not answer.2AFTER INSTRUCTION TUNINGGo to Settings, choose Security, then Reset password.Answers, but flatly.3AFTER PREFERENCE FEEDBACKOpen Settings → Security → Reset password. You will get an email link; it expires in an hour.Answers the way people preferred.

Pretraining is the phase where the model learns what language and the world look like, by predicting text over and over across an enormous amount of it. What comes out knows a great deal and is close to unusable, because nothing has taught it that a question is a thing you answer.

Post-training is everything done afterwards to turn that into an assistant, and it has two parts. The first teaches the model to answer rather than continue. The second teaches it which of the answers it could give people would rather receive.

Two phases, three stages
You will see both counts used. Pretraining is one stage, post-training is usually two, so the whole pipeline is commonly described as three. The split that matters is the one between them: pretraining supplies the knowledge, and post-training supplies the manners.