karnage on nostr

Open in Damus

The thing that I think all try to not talk about is: What if the LLM capability plateaued (or about to)? There is a limit to how much an LLM can probabilistically predict based on the training, and ...

Karnage @karnage 1787488368

Synthetic data. Data generated by LLM output. As long as that output keeps getting steered by the users, you have an endless data conveyor belt.

1👍1

This is what they do with driving models. They generate environments and drive their vision models in them. I don’t think it’s possible to generate new things from the conveyor belt, but probably get better at steering context. Also to a limit.