Hallucination & Grounding
Why it invents things so convincingly
The model always produces the most plausible-sounding continuation. When it has the facts, plausible and true line up. When it does not, it still produces something plausible, in the same confident voice.
Why it invents things, so convincingly
A model does not know the difference between recalling something and producing something that sounds like the sort of thing it would recall. From the inside those feel identical, because mechanically they are identical.
Ask for a citation and the machinery does what it always does, which is produce the most plausible next fragments. A plausible citation has a plausible author, a plausible journal, a plausible year and a plausible title, so every piece of it is individually reasonable while the paper itself has never existed.
Like recounting a film you half-remember
You are fairly sure of the plot. You confidently name an actor. Some of it is right, some is another film entirely, and crucially you cannot feel the join. You are not lying. You have no internal marker separating the parts you actually remember from the parts your brain reconstructed to make the story hold together.