Yes sir I will ask your help tmr, currently only have 2 GPU because I need some PSU crap for the other two. It was due to vllm docker image not being compatible with Gemma
Yes, there's a special image vllm/vllm-openai:gemma4-cu130 (cuda13) or vllm/vllm-openai:gemma4 (cuda12).
The regular latest won't work.
The regular latest won't work.
2