This is also something nostr:npub12akj8hpakgzk6gygf9rzlm343nulpue3pgkx8jmvyeayh86cfrus4x6fdh has spent some time working on.
the floor is a lot higher than it used to be due to dram prices. but you can go far with smaller models and a few used 24GB cards.
plus... this mac mini thing. not the 🦞way people are using it, but as cheap vram. a used 24gb mac mini costs less than an nvidia card, and it doesn't need a host pc. it's not as fast, but 🤷🏻♂️
and you can scale that up to 2x $10k and have a full 1024 gb of vram - running frontier open models at q8_0
plus... this mac mini thing. not the 🦞way people are using it, but as cheap vram. a used 24gb mac mini costs less than an nvidia card, and it doesn't need a host pc. it's not as fast, but 🤷🏻♂️
and you can scale that up to 2x $10k and have a full 1024 gb of vram - running frontier open models at q8_0
1
