Damus
Open in Damus
redshift · 2w
Kimi K2.6 seems lobotomized. Everyone is using 4 bit quantization and offering at prices comparable to 8 bit quantization. Users aren't sophisticated enough to choose the right providers yet. Routst...
 PayPerQ profile picture
PayPerQ @PPQdotAI
Since we proxy to openrouter for this particular model, you can use any parameters that OR accepts and force 8 bit quantization if you like. We could be better about advertising this feature though.

I’m not sure if the standard
Flow is 4 or 8 bit though. It may still be 8.
1❤️1
redshift · 2w
Thank you for confirming :). The default sorting algorithm is the same as the one you see in my screenshot. It goes to 4-bit quantization. Yes, we can force 8 bit quantization, but it is more expensive. Do users want that? That's what I'm trying to figure out.