waxwing
· 9h
Since this got some interest, I decided to share a simple example you can run on your computer:
https://github.com/AdamISZ/meatier
The example here is with a 3B model, needing about 12GB in RAM (?)...
This is very interesting indeed. I need to keep wrapping my head around this, but I have a doubt. How large does the initial prompt have to be? AFAIK, this initial prompt that both users need to use in order to initialize the LLM state so they can decode would work as a kind of “symmetric key,” right? If this is true, I was just wondering how secure or what is the shape of that initial prompt is, and whether it is susceptible to brute forcing in some way. Because for someone to participate in a conversation, they just need the LLM model and the prompt, and then they can exchange encoded messages out of band, right? Please correct me if I’m wrong on any of the points; I’m still trying to understand this :)