Eric SchultzMar 19

RE: https://mastodon.green/@gerrymcgovern/116246272007151195

I've seen what vibe coding results in. But I've seen what heavily AI generated but directed by a talented engineer results in.

It results in bad code. It MIGHT run but even if it does it will be horrible to read and horrible to maintain. Once you generate anything larger than the contents of a small function, you're creating mountains of technical debt.

Caveat: it might be possible to generate multiple unit tests at once and not create a mess. That said, you need to be incredibly careful to not overlook appropriate abstractions. If you do, you're creating a maintenance headache.

Show threadDaniel V.Mar 19

@wwahammy At work, I had to do code review on some claude code output for writing some e2e tests in Playwright.

I looked at the PR initially, realized it was all AI generated, saw how unbelievably repetitive it was, and just left a comment suggesting we think very carefully before merging it because it looked like automated tech debt.

3 rounds of review of code from a guy whom I have never had to give much push back on basically ever cuz he had good principles, and it was finally in a state where it would be “acceptable”

The crux of it is yeah it cranked out like 40 tests!

30 of them should have just been definitions that were passed through into a single test method with light abstraction and basically just feeding am array of test variables into.

If 1 thing updated, ALL of the tests would have to be individually updated vs just updating the 1 method.

I also had to nudge to make changes so that it will run through every test case and not fail the whole suite if just the 1 test failed.

And this is just within an SDET type setting!

One of the senior devs has been doing an unfortunate amount of vibe coding and the weird issues that keep coming up have been really bizarre. I catch a lot of them but there is so much that I am certain I won’t be able to catch in advance due to systemic issues in the org’s testing capabilities. It is a really stressful time to be in QA rn and I need this bubble to pop and the phantasms of GenAI to fall from people’s eyes so we can build shit people actually want.

All this to say: You are completely right and sitting on the other side of vibe code and told to validate it has been so very stressful.

Show threadPhilip Mallegol-HansenMar 19

@dvandal @wwahammy Definitely seeing some of the same in my work.

It seems like (Not surprisingly) lowering the cost (Time, cognitive, whatever) of generating more lines of code, is that more of the people I work with are perfectly content to ship “brute force” code instead of spending 5 minutes considering any abstraction that would lower line count.

Show threadPhilip Mallegol-HansenMar 19

@dvandal @wwahammy Just today, I was reviewing a code change where a coworker was changing ~50 instances of a magic number from value x to value y.

I had the gall to suggest we might want to abstract the value into a constant or something, and just change it once.

And this is from seasoned engineers, not someone fresh out of school.

I don’t look forward to fixing this shit for the next decade.

Show threadEric Schultz

@philip @dvandal it really is exhausting. I had to try to convert a vibe coded prototype into a deployable prototype a few months ago on a short timeline and it was one of the worst experiences of my career. It just wasn't possible. We got lucky and figured out a workaround that barely worked. Folks seem to have realized this was a mistake after the fact but being the strongest AI skeptic I was baffled as to why they expected anything else.

Fortunately, I haven't had much of that but if that kind of thing becomes my career... I don't think I can do it

Mar 19 at 4:53amWeb
Show threadDaniel V.Mar 19

@wwahammy @philip right now at my job, we have some PMs that are vibe coding prototypes and passing it over to developers to integrate into the product.

They are AI features being integrated with no QA done on them beyond the developer saying “yeah this is working alright.”

When you look at the code for the feature compared to previous code, it is so abundantly clear that even the implementation is Claude code.

I need the bubble to pop yesterday, cuz the brashness of it is just insane to me.
But hey I guess the customer base who seem to be all in on AI get to be the ones to grab tools to drive customers away from TV even faster than than it was already decaying. 🤷