@Juraj🏴💛🌘 , some feedback about Routstr below.
Feedback from integrating Routstr as an inference provider (Scrollworks, late Oct – Nov 1, 2026)
We wired Routstr into three apps as a backup and tested it as a primary for some calls. We funded six nodes over Lightning, ran our own evals, and built health and identity checks. Here's what got in the way, roughly most important first.
1. Nodes look healthy while every call fails.
The balance and /v1/models endpoints answer fine while every chat call fails, because the operator's upstream is dead or out of credit:
cypherpunk: 402 "API key DIEM spend limit exceeded" on all five models (the operator's Venice key).
omochi: 402 "Insufficient credit" on some models (its upstream PPQ account).
ai.orangesync.tech: 401 "User not found." on all ten models we tried. That's OpenRouter's wording for a dead key, and it came within an hour of funding.
A 402 also reads like our balance is short, which makes failover logic harder. Suggestions:
a health endpoint that reflects the upstream;
a distinct error code for "upstream unavailable";
marketplace-side probing that delists or flags nodes failing real calls.
2. Model substitution and mixed backends.
A node's "kimi-k3" called itself Grok 4 and Gemini, and counted 251 prompt tokens for a fixed prompt where Kimi's tokenizer gives 315. It's a different model behind the name.
The same node's DeepSeek V4.1 Flash matched DeepSeek's token count in 6 of 8 calls and was 26 short in 2, so at least two backends sit behind one model id.
We now fingerprint every node: the response's model field, the token count of a fixed prompt, and known-answer probes. Suggestions:
require an honest model field plus the upstream provider in responses;
marketplace spot checks or attestation that a model id is that model.
3. Many "independent" nodes are resellers.
The PPQ nodes pass open-weight calls to OpenRouter's own hosts: responses name DeepInfra, Fireworks, Alibaba and Moonshot. All four nodes selling Kimi K3.1 Pro Highspeed were Venice. That isn't decentralization: they share their upstream's outages, and usually cost more (Highspeed was 1.5–2× OpenRouter's price). Suggestion: show the upstream in node announcements and /v1/models, so buyers can pick genuinely independent capacity.
4. Refunds.
When orangesync's upstream died, our 1,000 sats were stuck:
Routstr 0.4.4 refunds only as a Cashu token for the whole balance, with no Lightning invoice option.
0.5.0 adds a refund to a Lightning address, but still not to a BOLT11 invoice.
Suggestions: refund to an invoice, and automatic refunds or a clear path when a node can't serve.
5. API differences between versions.
0.4.4 answers the /v2 invoice route with a 400, not a 404, so falling back to the old /lightning/… routes needs special-casing.
Streamed calls hung on one node and still charged sats: 78 for two hung calls.
Consistent routes and errors across versions, and no charge for calls that never return, would help.
6. Latency.
No node states expected speed. Our small beat-sheet call took a p50 of 2.2 s and a p95 of 13.5 s on the one working independent node, against 2.3 s and 3.9 s on our current provider. With a 5 s budget, 2 of 11 calls missed. Published latency, or a fast and slow tier per model, would help.
7. Missing capabilities.
No node we found offers speech-to-text (/v1/audio/transcriptions). No working independent node accepted image input. Support for strict JSON schema and forced tool calls varies by node and model.
Where we landed.
We built the checks and a "Routstr first" chain, but switched it off. For now Routstr is an outage backup behind OpenRouter. We'd happily switch back on for nodes that are verifiably independent, answer reliably and report their model honestly. The node checks are the part we'd most like to see built into Routstr itself.
