Mastodawn

Seems painfully obvious that, whatever you think about #genai code, anyone using it is heading for a code-review logjam. Assuming that the org requires code review; if yours doesn’t, nothing I can say will help you. Anyhow, Rishi Baldawa writes smart stuff about the problem and possible ways forward, in ˚The Reviewer Isn't the Bottleneck”: https://rishi.baldawa.com/posts/review-isnt-the-bottleneck/

[My prediction: A lot of orgs will *not* do smart things about this and will suffer disastrous consequences in the near future.]

The Reviewer Isn't the Bottleneck

AI tools are flooding PR queues and the instinct everywhere is to call review the bottleneck. I think that’s the wrong question. The reviewer is the last sync point before production changes. The goal shouldn’t be how to remove the gate, but how to make it cheaper to operate.

Rishi Baldawa

Show thread

degenerating degenerate Mar 10

@timbray FWIW the LLM knows a lot of best practices and bad-things-you're-not-supposed-to-do, that it doesn't pay attention to well when coding. It likes to do the minimum that meets what was asked with bad results for maintainability.

But it has seen, does know, what 'good code' would look like, so the exact same LLM that did the coding, can very usefully itself assess, critique and fix what it just emitted.

Show thread

Violet Madder Mar 10

@hopeless @timbray

If what it emitted needs fixing, why did it emit it in the first place instead of getting it right the first time? How many times do you pull the lever on this self-regurgitating slot machine before you proudly announce it came out right this time? Do we roll dice, or pull some floaty balls out of a globe or what?

It doesn't KNOW what good code looks like. It imitates whatever is associated with markers that indicate "good code", spewing similar patterns based on guesstimating what had positive feedback, in general averaged across who knows what criteria that's processed and judged some particular way, with occasional direct human input attempting to correct its course and it all sounds like pachinko balls falling.

Show thread

degenerating degenerate

@[email protected] @timbray

While it's engaged writing code, same as a human, its context is full of all the apis and surrounding code and language rules, and task definition from the human, it needs to do that, it literally lacks the ability to give attention to things that are not directly needed to do what's in front of it at that time.

In another context though, it's not coding and has the attention free to use its existing knowledge to do the critique, and after setting the task, the rewrite.