That’s great to know, thanks!

The Saylor Moonboyz are fine with audits by the big consulting firms and banks. They’re all fiat maxis.

Here’s one from the NIH:

https://ntp.niehs.nih.gov/research/assessments/noncancer/completed/fluoride

Running the 4 bit quant from ggml-org version (although I think I got similar performance from unsloth). Specifically gemma4:26b-q4_k_m.

A good bit of the performance comes from the unified memory because the apple GPU itself is weak.

Here’s a screenshot of it doing OCR on its model card for example with reasoning disabled and it finished in 1.9s at 78.1 t/s

Haven’t ran qwen in a minute but it’s surprising you’re not getting higher throughput for gemma4 on your 3090s.

For what it’s worth if you use llama.cpp and disable reasoning you should see faster time to first byte at the cost of a slight degradation in quality. Haven’t used vllm so can’t comment there. I get 70-75 tok/s on a macbook m3 for comparison and only have 40 GPU cores

Not only were the prices lower back then but the quality of ingredients was higher. It’s hard to even compare anymore.