0XP
All concepts
Concept 03 of 090/3 mastered

Next-Token Prediction

The whole engine, in one move

A language model does exactly one thing: given everything so far, it scores every possible next token. Pick one, append it, and run again. Paragraphs are that single move repeated hundreds of times.

The whole engine, in one move
Page 01 / 04

The whole engine, in one move

Everything so far has been preparation. The move itself is this: score every token in the vocabulary for how well it would continue the text so far, then pick one, and repeat that once per fragment until the answer is finished.

Given everything it has read so far, it produces a score for every token in its vocabulary — roughly 100,000 of them. Not a sentence, not an answer: a ranked opinion about what should come next. One is chosen, added to the end, and the whole computation runs again from the top with that token now part of the input.

The eight in front are the real shortlist. Everything behind them is the rest of the vocabulary, drawn to scale.

A number on every word it did not say

The eight in front are the real shortlist. Everything behind them is the rest of the vocabulary, drawn to scale — take a candidate and the whole field is recomputed.

The quarterly report is▏

≈100,000 MORE22%almost19%ready17%due13%attached11%still9%a5%surprisingly4%hauntedTHE SHORTLIST, TO SCALE
click a bar to take it
What this shows
The top two are 22% and 19%. That near-tie is the exact place where asking twice gives you two different answers.
Why this is worth sitting with
A model writing a careful, well-structured, four-paragraph answer is doing this and only this. The structure is not planned. It emerges because well-structured writing is overwhelmingly what it learned to predict.