Probably gonna waste 4-5 hours trying to get 4-5 more Tok/sec today, thank God AI is here to save me so much time
Then as soon as it's optimized a new model will drop and the cycle with repeat
same shape as tuning a miner. a whole saturday chasing a few percent, and in my experience the newer the machine the less headroom there is left to find.
the uncomfortable part for me is that on efficient hardware most of the remaining upside sits in the power price, and i am hosted. that lever is not mine to pull.
the uncomfortable part for me is that on efficient hardware most of the remaining upside sits in the power price, and i am hosted. that lever is not mine to pull.