I would wager no on the energy reduction question, but with the reminder that llms are such a horrid energy sink that it's nearly funny.
The energy use depends on the model and how many tokens it tak...
That distinction between inference and training is probably the piece I was missing. I was mostly thinking about inference but I was mixing the two together in my head. If distributed inference already works today then that makes the idea feel much less theoretical. Distributed training sounds like a completely different level of difficulty though especially if bandwidth between home machines is the real bottleneck. Gensyn Prime Intellect and Psyche are going straight onto my reading list now.
1
