Mastodawn

RE: https://mastodon.green/@gerrymcgovern/116246272007151195

I've seen what vibe coding results in. But I've seen what heavily AI generated but directed by a talented engineer results in.

It results in bad code. It MIGHT run but even if it does it will be horrible to read and horrible to maintain. Once you generate anything larger than the contents of a small function, you're creating mountains of technical debt.

Caveat: it might be possible to generate multiple unit tests at once and not create a mess. That said, you need to be incredibly careful to not overlook appropriate abstractions. If you do, you're creating a maintenance headache.

Show thread

Daniel V.Mar 19

@wwahammy At work, I had to do code review on some claude code output for writing some e2e tests in Playwright.

I looked at the PR initially, realized it was all AI generated, saw how unbelievably repetitive it was, and just left a comment suggesting we think very carefully before merging it because it looked like automated tech debt.

3 rounds of review of code from a guy whom I have never had to give much push back on basically ever cuz he had good principles, and it was finally in a state where it would be “acceptable”

The crux of it is yeah it cranked out like 40 tests!

30 of them should have just been definitions that were passed through into a single test method with light abstraction and basically just feeding am array of test variables into.

If 1 thing updated, ALL of the tests would have to be individually updated vs just updating the 1 method.

I also had to nudge to make changes so that it will run through every test case and not fail the whole suite if just the 1 test failed.

And this is just within an SDET type setting!

One of the senior devs has been doing an unfortunate amount of vibe coding and the weird issues that keep coming up have been really bizarre. I catch a lot of them but there is so much that I am certain I won’t be able to catch in advance due to systemic issues in the org’s testing capabilities. It is a really stressful time to be in QA rn and I need this bubble to pop and the phantasms of GenAI to fall from people’s eyes so we can build shit people actually want.

All this to say: You are completely right and sitting on the other side of vibe code and told to validate it has been so very stressful.

Show thread

Philip Mallegol-Hansen

@dvandal @wwahammy Definitely seeing some of the same in my work.

It seems like (Not surprisingly) lowering the cost (Time, cognitive, whatever) of generating more lines of code, is that more of the people I work with are perfectly content to ship “brute force” code instead of spending 5 minutes considering any abstraction that would lower line count.

Show thread

Philip Mallegol-Hansen Mar 19

@dvandal @wwahammy Just today, I was reviewing a code change where a coworker was changing ~50 instances of a magic number from value x to value y.

I had the gall to suggest we might want to abstract the value into a constant or something, and just change it once.

And this is from seasoned engineers, not someone fresh out of school.

I don’t look forward to fixing this shit for the next decade.

Show thread

Daniel V.Mar 19

@philip @wwahammy architecture, design, patterns, DRY, etc. are all where the work of software development is.
Coding isn’t the hard part, it is just the vehicle and tooling to “build the building”.

Mediocre and difficult to maintain code like this can only last so long as the tokens are subsidized with stupid money, and it will (arguably has) single handedly delay technological progress as a whole until stupid money gets smart.

Show thread

Philip Mallegol-Hansen Mar 19

@dvandal @wwahammy The "funny" part of course is that even if you're a diehard LLM fan, you should want the same things I want:

If your code is reasonably encapsulated, your LLM can fit the whole problem in the context window because the details of some implementation doesn't matter, you understand the interface!

But no... instead we have this.

Show thread

Daniel V.Mar 19

@philip @wwahammy Altman and Amodei off to the side: “But if it is encapsulated, then it means less tokens and less tokens means less better!”

Meanwhile they rot away developer’s ability to code and make them dependent on a toolset they have to rent to commit code worse than an intern would write.

As an aside sorta, as this shit is starting to sunset, we are seeing the discussion switch to tokens as something exchangeable as a thing to be exchanged for some supposed value. Pivot to AI had a video that touched on it, and it is just crypto all the way down.

https://youtu.be/KLERkM2TJbw

The money will continue to be invested until the still born ‘AGI’ god is declared born, and then it is a rush to not be left holding the bag.

AI companies try to pay staff in AI tokens, not money

YouTube

Show thread

Eric Schultz Mar 19

@philip @dvandal it really is exhausting. I had to try to convert a vibe coded prototype into a deployable prototype a few months ago on a short timeline and it was one of the worst experiences of my career. It just wasn't possible. We got lucky and figured out a workaround that barely worked. Folks seem to have realized this was a mistake after the fact but being the strongest AI skeptic I was baffled as to why they expected anything else.

Fortunately, I haven't had much of that but if that kind of thing becomes my career... I don't think I can do it

Show thread

Daniel V.Mar 19

@wwahammy @philip right now at my job, we have some PMs that are vibe coding prototypes and passing it over to developers to integrate into the product.

They are AI features being integrated with no QA done on them beyond the developer saying “yeah this is working alright.”

When you look at the code for the feature compared to previous code, it is so abundantly clear that even the implementation is Claude code.

I need the bubble to pop yesterday, cuz the brashness of it is just insane to me.
But hey I guess the customer base who seem to be all in on AI get to be the ones to grab tools to drive customers away from TV even faster than than it was already decaying. 🤷