Any chance to import from Paprika? How about export support?

Tell us about your rig running the model. Hardware and software stack. I saw that you're running vLLM?

I don't always listen to doomer climate talk. But when I do, it's from a neurologist with zero training in climate science.

It also enables dumb people to do dumb things rapidly. Unfortunately, this creates collateral damage. Buckle up.

A right of passage.

Look at the growth rate of your 401k. Most plans restrict you to a select set of funds, which almost assuredly don’t include any bitcoin derivatives. Some lucky plans allow you to use a % in a brokerage, which you could then put into bitcoin ETFs, and pump your growth rate some.

Then look at just buying and holding bitcoin directly over that same period. Even with half the contributions, you stand good odds to beat your 401k growth.

The best scenario is when your plan allows you to rollover some of your contributed funds into an IRA. Set up a self-directed IRA, where you can buy bitcoin and custody it yourself, and you’re off to the races:
- contribute to 401k
- get company match
- rollover contributed funds to IRA
- buy self-custodied bitcoin in IRA
- let company match funds in 401k grow (albeit slowly)


This ignores all kinds of other variables, like your age, liquidity needs, etc. Bitcoin you buy is always available to you to use however you need - no restrictions, no age limits, taxes already paid (on basis).

The future of AI is local models. Existing providers and future central providers will find competition too steep to have meaningful profits. It will be like bitcoin mining - razor thin margins.

This will drive more and more research into dynamic MoE models along with purpose built hardware, enabling local models to grow in both ease of setup and capability.

And then there will be the cloud offerings, allowing for easier deployment of private models.

The math seems really clear to me. Agents will outnumber people 1000:1 or more in the future. This cannot be centralized.

This is true, GPUs are faster for inference. But you'll also be consuming 1500 watts, have to deal with those thermal issues, and still struggle to fit a model larger than 32B with decent quantization.

Alternatively, the 395 chips and their NPU are doing pretty good. Combine 2 of them and you're looking at low GPU level inference AND you get 256MB for a larger model and plenty of context and STILL under 1000 watts.

So one of those is enough to get you started. It is well supported by AMD and there's even guides out there for how to ccluster 4 of them together (definitely ad a later phase).

Stay away from Mac minis. Its a good toy, but you lose a good bit of memory to osx and your limited in config options. If you want a large-ish model, you're forced to cluster and that opens up a whole other can of worms.

Agreed - 128GB is the only way to go. Running a 72B Q4 is definitely doable while still allowing a decent amount of headroom for context/kV cache.

Recommend checking out the latest gemma 4 offerings. You can get a lot done with the E4EB model handling tooling, routing, compaction, and other tasks. The 31B is also great for better reasoning.

I would NOT use this machine for anything besides inference. Save all memory for context (target 128k tokens). I really meant it when I said to treat it like an "inference appliance".

Offload everything else to whatever you have laying around, including openclaw. Keep it separate so you have a stable substrate.

What’s your budget? Ryzen AI Max+ 395 APUs offer UMA, which you’ll need to a decent model. I like Framework’s desktop offering. A bit more expensive than some chinesium builds, but you’re going to get solid firmware and driver support in linux - and that is king.

Set 1 or more up as an inference “appliance” and that’s all it does. Have everything else run on a different machine.

Stick with Ubuntu Server to start with - just easier support. Go ROCm + llama.cpp first, then fall back to vulkan if there’s issues. Can go Ollama when things are looking good.

I aim to build a Nix port once it’s all stable, making rebuilds of these “appliances” simple.

Pretty much as expected! A nothing-burger with a side of obvious lies. Just waiting on that 6102-style order to come soon, and then we can feast.