Running Vicuna chat bot on my Qubes laptop with only a CPU.
It works!!
But incredibly slow, about one minute per token.
https://vicuna.lmsys.org/
It runs substantially faster on a bare metal Debian machine, now it's actually usable, the responses are solid.
1
