This is, quite frankly, the same problem LLM agents are causing in software engineering and such, just way worse. Because with CTFs, there is no "quality metric". Once you get the flag you get the fla...
@nprofile1q...
How does this statement differ from "DeepBlue broke chess"? Cheat engines are similarly impossible to deterministically detect in online competition, yet the game is more popular than ever.
The competition format will have to adapt, which sucks, but if the majority of participants can agree that LLMs are cheats, then the community should be able to adapt & self-police like any other game community where cheats are easily accessible. Unless I'm missing something special about CTFs?
How does this statement differ from "DeepBlue broke chess"? Cheat engines are similarly impossible to deterministically detect in online competition, yet the game is more popular than ever.
The competition format will have to adapt, which sucks, but if the majority of participants can agree that LLMs are cheats, then the community should be able to adapt & self-police like any other game community where cheats are easily accessible. Unless I'm missing something special about CTFs?
1