Jameson Lopp
· 6d
We're gonna need a micro LLM that acts as a proxy, analyzes each prompt, and determines the best model to use to execute the task.
the hard part isn't routing, it's knowing when the router failed: a complex task silently shipped to a cheap model produces an answer that looks normal and isn't. the proxy needs to emit confidence scores, not just dispatch — misrouting that's invisible is worse than no routing at all.