@nprofile1q... I set a reminder to update this thread when I have it released. Anyways, I'm also ideologically committed to the idea that the insane capex in the AI space right now can only be stopped by eating into API margins via these kinds of cost reduction exercises, where people realise they can do more with orders of magnitude cheaper models.

@nprofile1q... Before Anthropic secured more capacity, I was suggesting to clients that they should hedge with a couple of extra open source models, because Anthropic API availability was genuinely trending towards barely 2-9s. It's better now, but outages are still common, and their availability is 99.5% on the API.

Re: kind of work, I'm in the process of writing a blog post about this, but most "agentic" work, especially with RLMs, is very "reactive", meaning that you spend a lot of time hot-potato-ing tasks on an event queue:

- user message arrives at queue, hit LLM, LLM wants to run a tool, LLM idle while tool executes, tool execution finishing prompts LLM again
- LLM triggers durable workflow, workflow runs over seconds, minutes, hours, hits callback to LLM, LLM meanwhile idle

All of this time spent idle is free capacity. I presume most APIs already handle this kind of batching and scheduling (if you've ever experienced Claude or Z.ai stuttering, I think this is a symptom of this batching and scheduling).

Numbers wise, I obviously can't divulge company info, but the order of magnitude is somewhere around 10^5 conversations monthly. Horizontally scaling is a no-brainer, because adding another GPU is a drop in the bucket compared to previous token costs on API.

@nprofile1q... It's really not that difficult, and not that expensive. Most AI workloads are not that complex, and modern techniques to improve long-context performance make even on-device models capable of solving many rudimentary office tasks.

My consultancy does exactly this, there's at least 3 examples I can refer to with clients reducing their AI bills by 90% after switching to a Gemma or Qwen model running on a single RTX Pro 6000.

(Pictured is the performance improvement on a couple of long-context tasks, i.e. OOLONG, OOLONG-Pairs, and the step-change performance improvement after an 8B param model is given a Python REPL loop and ability to recursively call itself)

Source: https://arxiv.org/abs/2512.24601

@nprofile1q...

> Capitalism is doomed

Yes.

> but we have already exceeded the global growth limits of our planet

Also yes.

> The cleverer part of the rich know this, which is why they're switching from capitalism to feudalism.

Three yes-es.

I don't think they're switching, it's a natural progression of insane wealth hoarding.

Also, when Thiel says "AI will solve the climate crisis", he means: "6 billion people will die, robotics will become good enough to replace them, and the 2 billion remaining people will be able to enjoy the same or more level of consumption and move to areas unaffected by climate change"

@nprofile1q...

@nprofile1q... this makes a big assumption, namely that consumers are still needed.

If you look at the IOU circlejerk in AI right now, it's clear as day that capitalism has transcended consumer capitalism.

I think that's infinitely more scary.

@nprofile1q...

Bitcoin is finite, you can't print Bitcoin. When you need to stimulate your economy in a recession by investing in industry, please enlighten me about the bitcoin-based mechanisms that would allow you to run a budget deficit, or to devalue your currency to be internationally competitive in terms of trade?

@nprofile1q...

@nprofile1q...

If everything was Bitcoin, you'd have endless austerity.

Money not being a commodity is beautiful, actually.

@nprofile1q...

@nprofile1q... of course, but we're not doing the No True Scotsman here.

The main idea is to stop buying a full laptop every 2-3 years. If you can upgrade your RAM only and keep the existing mainboard, that's one less mainboard that needs to be manufactured, less rare earths that need to be mined, less carbon produced, that's absolutely amazing, and shows that degrowth IS possible.

The door is cracked a tiny bit to finally get right to repair and repairable tech, people are feeling the pinch of cost of living that might make them look for repairing rather than dumping e-waste.

I think the Omarchy and Rails issue is inconsequential at a time we need a Rainbow Coalition for open computing. I think infighting and bullying Framework over this is, frankly, dumb, and won't actually change anything related to western bigotry.

@nprofile1q...

> We don't have to accept one at the cost of the other!

Certainly, but there's give or take 50 million people in the market for a PC this quarter, worldwide, given previous Q1-Q2 laptop sales as an estimation.

Let's say 90% are priced out of Framework, that leaves 5 mil.

Let's say of those 5, 3 million are ok with buying second hand (doubtful, but let's be conservative).

The remaining 2 million, should they a) buy the forced labour and PLA civil-military fusion Lenovos, the strip-mining the DRC and funding Trump Apple, or the oh-we-sponsored-some-dudes-conference-and-promoted-his-oss-arch-theming Framework.

I'm genuinely asking, not using it as clever retort.

@nprofile1q... No, my priorities are to kneecap the e-waste and strip mining industries, not to moralise online. I know this is wild to say on here, but sponsoring a Ruby conference and Omarchy is orders of magnitude less consequential than the child labour, and environmental damage Western countries export to the Global South.

For Framework to be effective and cheap, they need scale and normies to switch. If that's through Omarchy, good. If 1 more person can not buy an XPS this year, I'm happy. If Framework and Omarchy gets 1 more person to not buy a license from genocidaire-enabling Microsoft, I'm happy. I think this is orders of magnitude a better outcome, regardless of DHH.

You lot probably buy second hand laptops and such, but most normies don't, and that e-waste from their new XPS probably ends up in a landfill in MY country [1].

Framework manufactures in Taiwan, and while their partner doesn't have a stellar labour record, it's not Foxconn suicide nets or South Asian sweat shops, or by proxy supporting an actual ethno-nationalist, irredentist state (China).

This Framework witch-hunt is again, mostly upper middle class, liberal, Imperial-core pearl clutching that doesn't want political power, it just wants to endlessly critique power.

[1] https://www.rferl.org/a/romania-garbage-asia-european-union/31429822.html