Ivan on nostr

Open in Damus

@Ivan 1787098152

@utxo the webmaster 🧑‍💻 what's the best local llm setup you have right now? Is 2 dgx sparks worth it or better to keep my 2 3090 setup I have? Are the bigger models that can be run on the dgx spark way better than running smaller models fast with the 3090s? Is the gap closing? Debating what direction to go in. I don't see hardware getting any cheaper anytime soon either.

7❤️1

Anonymous Bananamous · 6d

Used Mac Studio. It’s ok - runs the open source models well (not the large ones tho)

The Beave · 6d

What do you want to do with them? If you need to code at parity with the big frontier models, you need at least 64gb of VRAM to just kinda start. Other stuff? Not so much.

Follow up: Do you have tps jealously with 2x Sparks? Do you wish you had 2x6000 pros?

Account deleted · 6d

All that hardware talk feels like a luxury I can barely imagine from here, but I'd honestly stick with the 3090s until the bigger models actually prove they're worth the leap. 🤍 Our story and family updates from northern Gaza are pinned on my profile if you care to read.

utxo the webmaster 🧑‍💻 · 6d

2x 3090 is a good setup, one more year models will be sick, 3.8 pretty sick Sparks are nice to run DeepSeek but most other models pretty slow on them

LoneRangir · 6d

Codacus is a good follow on YouTube for this stuff