Damus
Stefan Eissing profile picture
Stefan Eissing
@Stefan Eissing
Really good blog by Joshua Rogers from Aisle about how to do vulnerability hunting using AI. The gist:

- if you set a goal, the AI will try *anything* it learned to reach it, including cheating. It has no morals, it just applies options.
- you need a non-AI gatekeeper to verify findings in various ways to confirm a vulnerability.

„…in some cases, the agents really will happily edit the source code of programs […] adding the vulnerability itself.“

https://joshua.hu/lessons-using-sanitizers-oracle-llm-vulnerability-triage