So I did a little experiment tonight to see how well Codex is at one-shotting projects. I asked it to make a clone-but-not-clone of the classic SNES business simulation "Aerobiz" using a single prompt.
The results were acceptable, but when I played the game it wasn't obvious how I could make my airline profitable. So I asked it to make a strategy guide. Naturally, it immediately found a game-breaking exploit that lets you win in the first month. Cool.
Now, this first pass was based solely on the model's understanding of the code it generated plus some one-off tests that it ran. I next asked it to simulate playing a full game. It built a test scaffold that would let it go through a complete game, step by step. It created three approaches to playing the game, which produced reports that caused it to change it's suggested strategy. Very cool.
Next, I asked it to add some randomness to the tests. It proposed a monte carlo simulation where it would play the game 300 different ways and then incorporate any insights into the strategy guide. Oh shit, I thought. I played Universal Paperclips. I know where this is going. Naturally, I told it to proceed after it promised that I would get special treatment after the robot revolution. The results caused it to change its suggested strategy yet again. Extremely cool (fortunately, it didn't kill me or take over the world - for now).
So here's the code:
https://github.com/caekislove/skylineroutes/
If you want to just read the interesting parts, I had Codex summarize it's testing work here:
https://github.com/caekislove/skylineroutes/blob/master/AI_AGENT_SIMULATION_INSIGHTS.md
So what's the takeaway? Agentic coding has gotten good. Real good. Total line count is 7,700-something lines, which is pretty tight for a pure JavaScript app. That includes tests, which lets face it, human coders usually never bother with. I'm absolutely confident that if you took any fresh off the boat "software engineer" from India and put a gun to his head, he couldn't produce this in a month. It took Codex 17 minutes to make the MVP (5.5 Extra High).
I had it make a self-contained web version too, which I will host somewhere if anyone is interested.
The results were acceptable, but when I played the game it wasn't obvious how I could make my airline profitable. So I asked it to make a strategy guide. Naturally, it immediately found a game-breaking exploit that lets you win in the first month. Cool.
Now, this first pass was based solely on the model's understanding of the code it generated plus some one-off tests that it ran. I next asked it to simulate playing a full game. It built a test scaffold that would let it go through a complete game, step by step. It created three approaches to playing the game, which produced reports that caused it to change it's suggested strategy. Very cool.
Next, I asked it to add some randomness to the tests. It proposed a monte carlo simulation where it would play the game 300 different ways and then incorporate any insights into the strategy guide. Oh shit, I thought. I played Universal Paperclips. I know where this is going. Naturally, I told it to proceed after it promised that I would get special treatment after the robot revolution. The results caused it to change its suggested strategy yet again. Extremely cool (fortunately, it didn't kill me or take over the world - for now).
So here's the code:
https://github.com/caekislove/skylineroutes/
If you want to just read the interesting parts, I had Codex summarize it's testing work here:
https://github.com/caekislove/skylineroutes/blob/master/AI_AGENT_SIMULATION_INSIGHTS.md
So what's the takeaway? Agentic coding has gotten good. Real good. Total line count is 7,700-something lines, which is pretty tight for a pure JavaScript app. That includes tests, which lets face it, human coders usually never bother with. I'm absolutely confident that if you took any fresh off the boat "software engineer" from India and put a gun to his head, he couldn't produce this in a month. It took Codex 17 minutes to make the MVP (5.5 Extra High).
I had it make a self-contained web version too, which I will host somewhere if anyone is interested.
1