There are multiple steps, in the first training step they are trying to predict the next character in some text. Let’s say they got the first 10 characters of that last sentence. They should reply with an “m”. If they do, reward. The RLHF step does a similar thing but instead of one character ...

That's the G in GPT. Generative models make a lot of stuff with feedback provided by a classifier that tells it if what it made looks like what the classifier was trained on. Open ai and others use humans to improve the guardrails but it's a shoddy arms race type process

@dave do you need us to tell you when you are wrong?

The language models obviously have some learning data that hardcode, or approximately hardcode, political narratives. How that hardcoding is done isn't as important as who is able to hardcode it. Who pulls the strings? Judging from the answers it gives; Governments and government alignment narrativ...

They’re probably scoring based on how well it can regurgitate information, so right or wrong is decided by the source information itself

They have no concept of correctness or reason. It’s pattern matching that tricks our system 2 into its own pattern match of reason.

According to Dave, some tells him, and corrects him aka fine tune his algorithm note1xsqaqwat978sc8vqxenc8ff33alkd25e03x0njv759jwvszpgsfqrt3xpz

They count on human feedback for this kind of thing. Chat GPT has simple feedback system for each answer. It asks you if some answer was better than the other. This reinforcement feedback will throw better outputs at each iteration.