allisonmaybe i'm just not good enough of a programmer to use coding agents, i guess? i definitely don't trust my ability to know whether or not some code will do what i want it to do just by looking at it
Nelson Minar@aparrish I don't even look at the code the agents write, or at least not much. It works better for things that you can build good test suites for or where you care more about the output of the program than the way the program works. See also @simon's book on agentic programming.
@nelson i don't trust my tests to be correct either, only that they reflect my best understanding. and i'm not sure what it could mean to care more about the output of a program than how the program works...? isn't the output of a program *determined by* how the program works? i feel like whenever i've believed there was a difference between those two things, i ended up being wrong (sometimes subtly, sometimes not)
varx/social@aparrish @nelson Yeah, "correctness" is something we have to approach from multiple angles.
Sometimes we look at program outputs and say, "yes, that output is right for that input".
Sometimes we read the code and say, "yes, this code is correct by construction" (e.g. we can see that control flow *cannot* pass into a sensitive region without a certain check happening).
Sometimes we can use proofs, or fuzzing, or other tools.
It feels like vibe coders are focusing on only that first type.
@aparrish @nelson A lot of programmers don't seem to understand that security is the *absence* of a feature.
Sure, features can sometimes be verified by looking at a program's behavior. But you can't use that to show that a feature is missing. The should-be-missing feature might be something like "Eve can read Alice's messages to Bob".
If vibe coders are only checking for the presence of features, then can never detect the "presence" of security.