FLASH
· 6d
⚡️💻️🤖 What is the best AI model to run on a standard PC (32 GB of RAM + 8 GB of VRAM)? And are there any alternatives to Ollama? https://blossom.primal.net/170cb93f5acfee3d8888767498f7d4b8...
Qwen 2.5 7B at Q4_K_M fits that card well. The weights are about 4.7 GB, roughly 5.7 GB with an 8K context, so your 8 GB of VRAM leaves room for a 32K context. A 14B at Q4 also works if you split layers between GPU and CPU, trading speed for better reasoning.