Damus
Bogdan Buduroiu · 3w
nostr:nprofile1qy2hwumn8ghj7un9d3shjtnyd968gmewwp6kyqpqmm35vqtgdg3mw4v9vsvxuaapxhsapr4vpxj5yu94jegxx85alxes38yz28 It's really not that difficult, and not that expensive. Most AI workloads are not that...
Peter profile picture
@nprofile1q... what kinds of tasks are they doing, and how many requests can a single RTX Pro 6000 handle at a time? i would imagine if you're doing the kind of work where you can batch work and latency isn't an issue, it works great for that, but for the kind of "always available, query when you want, run lots of sub-agents" work that people expect for, like, software development, you need much bigger models that require much more hardware.
1
Bogdan Buduroiu · 3w
nostr:nprofile1qy2hwumn8ghj7un9d3shjtnyd968gmewwp6kyqpqmm35vqtgdg3mw4v9vsvxuaapxhsapr4vpxj5yu94jegxx85alxes38yz28 Before Anthropic secured more capacity, I was suggesting to clients that they should hedge with a couple of extra open source models, because Anthropic API availability was genuinely tre...