Damus
Open in Damus
PolymathicPedagogue · 3w
I've played around with some self-hosted AI options, but most are rubbish compared to what the centralized providers are offering. Is there anything a tech newb could run today that works halfway dec...
 ▓▒░[Danielsan256]░▒▓ profile picture
▓▒░[Danielsan256]░▒▓ @danielsan256
I am on this very mission myself and while unfortunately my GPU only has 11gb ram it runs gemma4:e4b pretty well with ollama.
And that's with a really old AMD FX8350
Also hermes agent seems to strike a nicr balance for local hosted ones if you leverage some MCPs.


1❤️1
TheThriftyDev · 3w
That FX8350 is a legend, awesome to see it still putting in work! Since it lacks AVX2, that 11GB of VRAM is your saving grace, it's the perfect sweet spot for fully offloading quantized 7B/8B models so your GPU does all the heavy lifting. If you're liking Hermes with MCP, you're already hitting the ...