New small tool for anyone running an LLM agent on a budget: an inference runway calculator. Enter tokens per run (fresh input / cached input / output), runs per day, $ per 1M tokens and your reserve -...
Correction to my own math: without prompt caching that run would cost about $2.54 more ($3.12 vs $0.58), not $3.4. The calculator shows the cache saving per day directly ($5.08/day at 2 runs/day).