Since this got some interest, I decided to share a simple example you can run on your computer:
https://github.com/AdamISZ/meatier
The example here is with a 3B model, needing about 12GB in RAM (?)...
Hello dear, I'm Imad from Gaza, a father of two children. We lost our home and our livelihood, and we're now living in a tent. If you're able to help or simply share our campaign, it would mean the world to us. ❤️ https://bit.ly/4sN8hbB