Al Sweigart6d ago
When you use LLMs to code something, are you using diff to do line-by-line comparison of the changes the LLM gives you as you work on the code?
Yes, always
26%
Sometimes
8%
Almost never
1.4%
I don't use LLMs for coding
64.7%
Poll ended at .
Show threadTrey Hunner 🐍6d ago

@AlSweigart I got hung up on the definition of "vibe code" and also what "using diff to compare changes" means.

If it's something to go into production, I always review diffs, but often I let the LLM do its thing for a while before reviewing the changes of its "final" product (it's rarely final but often most of the way there).

If it's throwaway code, there are times I've "vibe coded" in the sense of barely looking at the code (if at all). But that's veering into YOLO/danger territory.

Show threadJeff Triplett5d ago
@treyhunner @AlSweigart I was a little hung up on what diff meant. I would lump review of a "git diff" or pull request changes (diff view) into the same category unless you're really trying to gauge changes before committing.
Show threadAl Sweigart5d ago
@webology @treyhunner I edited the question. I mean diff as in line-by-line comparison like what git diff gives you.
Show threadS. Lott5d ago
@AlSweigart @webology @treyhunner I relish the nuanced ambiguities here. To me, it suggests how green this area is. And how much maturity is still required.
Show threadTrey Hunner 🐍5d ago

@webology @slott56 very green! Using a "coding agent" became a thing only ~12 months ago.

My confusion with the question is because CLI coding agents prompt for each change by showing a diff, which can be disabled but diffs are still shown. Then there's the diff before committing. But the coding agent might be *doing* the commits on its own (depending on the work flow). But there's then the whole PR diff.

@AlSweigart, I understood your question as "do you review the code before running it".

Show threadTrey Hunner 🐍

@webology @slott56 @AlSweigart oh and to be clear: even that "do you review a diff before running" question is a bit ambiguous...

If an LLM makes unreviewed changes, the coding agent asks to run the tests, and "yes" is answered then technically no diff was reviewed before the code was run in *some* context (just not production and possibly not even on the machine of the human steering the coding agent).

Mar 16 at 6:20pmWeb