I thought about this myself and decided it's not worth it yet, and I'll wait another year or two before getting dedicated hardware

Ha! I scanned through all the responses looking for actual recommendations and didn’t find much. When I went down the rabbit hole recently, it seemed like a cluster of 2 or 4 loaded out M3 Ultra Mac Studio’s with the RDMA over thunderbolt support was the ticket.