Paul CantrellThis is good (from @shriramk): https://mastodon.social/@shriramk/110040524796761802
The skill of recognizing and diagnosing broken code only becomes •more• important in the face of LLM code generators.
Any experienced programmer worth their salt will tell you that •producing• code — learning syntax, finding examples, combining them, adding behaviors, adding complexity — is the •easy• part of programming.
The hard part: “How can it break? How will it surprise us? How will it change? Does it •really• accomplish our goal? What •is• our goal? Are we all even imagining the same goal? Do we understand each other? Will the next person to work on this understand it? Should we even build this?”
A thought exercise:
Which of the problems in the post above does AI code generation make easier? faster?
Which does it not help?
Which might it exacerbate?
Hans Cathcart@inthehands A machine learning algorithm to help think of ways an app could break or be abused could be quite useful, I'd imagine.
@hanscath Indeed. I can imagine carefully hand written invariants / assertions as input to a sort of ML fuzzer. Or imagine a model checker like Alloy running on ML-generated input: “Based on how you described your intent in words, could this situation that your code allows be a breakage?”
arclight@inthehands @hanscath I learned of KLEE from https://www.amazon.com/Embedded-Software-Development-Safety-Critical-Systems/dp/0367338858 (the cheaper 1st ed. is also great). The maze-solving example linked off KLEE's website is really cool, especially when it ferrets out really subtle logic bugs that "cheat" to solve the maze.
@inthehands I'm reminded of the time when I watched AlphaGo play Lee Sedol, and the commentators were trying to figure out what AlphaGo was doing. AlphaGo had started a strategy no human had ever thought of. So either white hats, or more likely black hats, will use this tech for penetration testing.
@hanscath
That seems very likely. And in general, ML seems very well suited to tasks where the reward of one successful attempt outweighs the cost 100,000 failed attempts. Automated spearphising is another example.
That seems very likely. And in general, ML seems very well suited to tasks where the reward of one successful attempt outweighs the cost 100,000 failed attempts. Automated spearphising is another example.