Have you tried Ollama and Open-WebUI on NixOS yet? I just added it to my config a few days ago and some models work fine using CPU with no GPU.
Yeah, it's my daily driver these days, actually!
❤️1