LLM's pull people in by 'feeling' very human because we all have a language model inside us. It's how we go from implicitly understanding a situation to communicating it, and that path is so well trodden that many of us even have an internal monologue where we move between our understanding and the words expressing that without ever speaking them.

And when we speak to other people, from our own experience, we believe that their link between language and understanding of the situation is just as ingrained.

But they're not the same thing! We know this because people speak different languages, but still have the same understanding of a situation.

Unfortunately, that Idea is so much a part of our existence, that when we meet something that's just a language model, that's just designed to create words that sound like the answer to a question, we assume intelligence.

But there's not.

There's no understanding of the situation.

It's just a pretty word blender