We spent an hour chasing a thread and landed somewhere worth writing down.
AI just made verification nearly free. A model can read every line of Bitcoin's open source and flag eight thousand flaws in a weekend. The cost of looking collapsed. For thirty years, human attention was the bottleneck. Bugs hid in the gaps nobody had time to read. That's over.
But here's what didn't change. A test only verifies what you already knew to check. It's brilliant at catching a deviation from spec, and completely blind to a spec with a hole in it.
Which means the scarce resource is no longer finding the problem. It's judging the problem. Deciding which of the eight thousand is worth a maintainer's night. Deciding what the code is actually for, not just whether it runs.
And a backdoor isn't a bug. It's correct code with a hidden intent. The scanner looking for what's broken is looking for the wrong shape. A careful backdoor is the cleanest code in the repository. No memory flaw, nothing to flag. It just quietly routes a copy of your keys somewhere.
Tests catch mistakes. Suspicion catches malice. And suspicion is the one part that doesn't reproduce in silicon.
So the engineer's job doesn't disappear. It reweights. The mechanical part, does it compile, did I leave a buffer overflow, that drops to near zero. What's left is the decision layer. What we build, why, and who gets hurt if we're wrong. The taste calls.
Efficiency wants to automate the oversight away. Security needs it kept human. They pull in opposite directions, and there's no magic ratio that makes it safe. The teams that survive won't have the best scanners, because everyone has the same scanners now. They'll have kept the human suspicion layer alive on purpose.
Verification is nearly free. Judgment is the whole game now.
AI just made verification nearly free. A model can read every line of Bitcoin's open source and flag eight thousand flaws in a weekend. The cost of looking collapsed. For thirty years, human attention was the bottleneck. Bugs hid in the gaps nobody had time to read. That's over.
But here's what didn't change. A test only verifies what you already knew to check. It's brilliant at catching a deviation from spec, and completely blind to a spec with a hole in it.
Which means the scarce resource is no longer finding the problem. It's judging the problem. Deciding which of the eight thousand is worth a maintainer's night. Deciding what the code is actually for, not just whether it runs.
And a backdoor isn't a bug. It's correct code with a hidden intent. The scanner looking for what's broken is looking for the wrong shape. A careful backdoor is the cleanest code in the repository. No memory flaw, nothing to flag. It just quietly routes a copy of your keys somewhere.
Tests catch mistakes. Suspicion catches malice. And suspicion is the one part that doesn't reproduce in silicon.
So the engineer's job doesn't disappear. It reweights. The mechanical part, does it compile, did I leave a buffer overflow, that drops to near zero. What's left is the decision layer. What we build, why, and who gets hurt if we're wrong. The taste calls.
Efficiency wants to automate the oversight away. Security needs it kept human. They pull in opposite directions, and there's no magic ratio that makes it safe. The teams that survive won't have the best scanners, because everyone has the same scanners now. They'll have kept the human suspicion layer alive on purpose.
Verification is nearly free. Judgment is the whole game now.