Really good blog by Joshua Rogers from Aisle about how to do vulnerability hunting using AI. The gist:
- if you set a goal, the AI will try *anything* it learned to reach it, including cheating. It has no morals, it just applies options.
- you need a non-AI gatekeeper to verify findings in various ways to confirm a vulnerability.
„…in some cases, the agents really will happily edit the source code of programs […] adding the vulnerability itself.“
https://joshua.hu/lessons-using-sanitizers-oracle-llm-vulnerability-triage
- if you set a goal, the AI will try *anything* it learned to reach it, including cheating. It has no morals, it just applies options.
- you need a non-AI gatekeeper to verify findings in various ways to confirm a vulnerability.
„…in some cases, the agents really will happily edit the source code of programs […] adding the vulnerability itself.“
https://joshua.hu/lessons-using-sanitizers-oracle-llm-vulnerability-triage