Temperature
The predictability dial
Having scored the next token, the model still has to choose one. Temperature reshapes the odds: low temperature flattens the long tail and picks the favourite, high temperature gives outsiders a real chance.
Same odds, different pick
The model produces a set of odds. Something then has to choose from them — and how strictly it favours the front-runner is a setting you can sometimes control.
That setting is called temperature. It is the most misunderstood dial in this whole area, so it is worth being precise: it does not make the model smarter, more creative, or more accurate. It changes exactly one thing — how much the underlying odds are flattened before a choice is made.
The same distribution at seven settings at once. Move the dial and watch which one you are standing on.
Seven settings of the same distribution, drawn at once. The scores never change — only how flat they get before one is picked.
Like a weighted dice you can reshape
The model hands you a set of weighted dice: this face is 40% likely, this one 25%, this one 3%. Temperature does not change the model's opinion about which face should be favoured. It re-weights the dice. Turn it down and the 40% face comes up every time. Turn it up and the faces even out until the 3% option starts winning regularly.