Mastodawn

Zendadaist Jul 17

jonny (good kind)

I am loving watching the vibecoding bubble bursting and reforming in real time. It turns out it is ludicrously expensive to have humongous billion parameter models generate a whole library of code in a loop hundreds of times per hour rather than, you know, "writing code." Cursor's pricing model finally started to try and operate at less of a loss and people revolted since they could no longer autogenerate code for 8 hours solid, and now Amazon is trying to capture the runoff with a first-hit-free cursor replacement.

Very funny how, to me, FOSS is about creating a credible bulwark of power so that we aren't at the mercy of information giants, but for vibe coders it's like a "how can I get locked in and get got as deeply as possible" speedrun.

jonny (good kind)Jul 16

I don't think I've seen something funnier in programming than people sharing screen recordings of their terminal as their multi-agent setup is generating and printing dozens of 10k line diffs per second in order to fix a bug. Especially funny is when each of those changes goes through like four different models with four different prompts (they love giving agents job titles like "investigator" and "consultant" and stuff and pretend they are doing different things) that they imagine are actually having a "conversation" but if you browse the chat logs the different roles just use different quantities of emojis but make the same dogshit changes

Adam Kent Jul 16

@jonny Pulling the handle of that slot machine 1000 times hoping to get lucky.

verifiedlesbian

@jonny It’s gotten so absurd I no longer feel secondhand embarassment

@jonny Basically coding using a genetic algorithm on one's source.

If I were to do that but without all the bells and whistles I would be laughed out of the room.

jonny (good kind)Jul 17

@aerique it's even better though - it's coding using genetic algorithm where the loss function is over a statistical approximation of what a natural language human evaluation of what code like yours might look like.

it's much worse than a random walk with no loss function, it's a very opinionated loss function that is totally meaningless

jonny (good kind)Jul 17

@aerique AND it's a genetic algorithm where the next generation is not random variation in some parameter space around the previous generation, it can be literally anything in an unknowable probability cloud up to and including deleting the entire project and itself

patrislav ♾️ #RIPNatenom Jul 17

But can also be nudged to open up specific backdoors and vulnerabilities („LLM injection“)

@jonny @aerique

interzoneboy Jul 16

@jonny on good days it makes me chuckle. On bad days it makes me feel sick.

patrislav ♾️ #RIPNatenom Jul 17

@interzoneboy I'm a bit concerned when this pile of sh** starts creeping up in apps and services I use myself, or worse, starts to affect critical infrastructure.

@jonny I'm surprised this study found vibe coding only 19% slower than actual development. I guess they used real devs who fixed the mistakes before they were too baked in. https://arstechnica.com/ai/2025/07/study-finds-ai-tools-made-open-source-software-developers-19-percent-slower/

Study finds AI tools made open source software developers 19 percent slower

Coders spent more time prompting and reviewing AI generations than they saved on coding.

Ars Technica

jonny (good kind)Jul 16

@kacey
There's a bit of a distinction between "LLM assisted programming" where it might generate some code in a constrained area vs pure "vibe coding" which is like "don't look at the code ever, just regenerate huge sections of the code until they work." One is a bit more defensible to me than the other, if I'm ignoring the macro societal level consequences of these tools.

@jonny that's fair! I forget the exact settings I was in (had to turn it all off for an interview) but one of my IDEs recently landed in a predictive text mode where it only predicted a word or two ahead, which was right more often than not -- it was really quite pleasant. I should probably figure out how to turn that limited functionality back on.

jonny (good kind)Jul 16

@kacey
AST- and context-aware line completion is dope. That's about all the ML i need

Jeremy Kahn Jul 17

Cosign exactly this.

Integrating with linters and linkers (in opposite directions) is another useful "sense" for an engaged coder

jonny (good kind)Jul 17

@trochee
@kacey
What u mean linkers and opposite directions? Am baby

David Chisnall (*Now with 50% more sarcasm!*)Jul 17

@jonny @kacey I was really confused by the fact Copilot didn’t seem to talk to the LSP server. There’s a thing right there in my IDE that can tell you if there are problems with code, but it will still happily generate invalid code and present it to me as something I might want. Even a simple ‘reject the thing if the LSP server says it won’t even compile’ check would have made it less annoying.

Still not useful though. I turned it off primarily because I spent more time debugging things that looked plausibly correct but had subtle and difficult to spot errors than I saved in typing. After that, I noticed how much time I had previously spent waiting for autocompletions that I’d often discard. Most of the time I could copy and paste and edit as fast Copilot could generate trivial things. For non-trivial things, my cycle went from start, pause, see the completion is nonsense, write it myself to just write it myself, which was faster.

If I didn’t very carefully pay attention to my own process, Copilot felt faster. It just wasn’t.

jonny (good kind)Jul 17

@david_chisnall @kacey that's the "obviously it should work this way but actually it does not work anything like that" dissonance of the LLM - the LLM does not do anything like something that actually parses the code in the same way that it does not do anything like comprehending language. the LLM does not and cannot be "AST-aware" in the way that LSPs are, the best they can be is "AST-conditioned" in the sense that they generate output until it passes some check by the LSP which is nightmarishly expensive with no termination condition. the RAG myth of LLMs has an enormous "category difference between indeterminate statistical process without computational form and strict computational process" barrier to bridge before it is anything close to real.

David Chisnall (*Now with 50% more sarcasm!*)Jul 17

I’m not suggesting it should be AST aware. The LSP server is able to take a stream of characters, tokenise it, build an AST, and then report errors. I’m suggesting that the LLM integration in the editor should:

Get a token stream from the LLM.

Feed it to the LSP server as an edit to the current file.

If it has no errors, present it to the user and stop.

Pass error messages to the LLM.

Go to step 1.

This doesn’t require the LLM to be any more than a thing that generates streams of tokens and receives streams of tokens. The problem is that it may take a few hundred ms or more for the LSP server to parse the new text. If it has errors, then the LLM needs to regenerate code.

This flow will be slower. It will make you more productive because the LSP server is far faster than you at spotting nonsense output. But it will make you feel slower because now you’re waiting doing nothing, rather than doing pointless work.

And that would be a disaster for people selling these things because they tap into exactly the same ‘people are terrible at estimating their own productivity’ problem that people selling management buzzwords have used for decades to sell things that make people feel more productive for a few months while ignoring that they may have a zero to-negative effect on actual measured productivity.

jonny (good kind)Jul 17

@david_chisnall @kacey yeah the thing about that loop is it is "nightmarishly expensive with no termination condition." because the error messages are similarly not parsed like code by the LLM. the idea that the loop can work despite the category error is what keeps the vibe coding bubble inflated.

David Chisnall (*Now with 50% more sarcasm!*)Jul 17

Coding LLMs are fairly good at ‘I got this compiler error, how do I fix it?’ queries (in the syntactic sense. Not necessarily a correct fix) because they have a lot of training data from StackOverflow and similar places that are exactly this shape: someone can’t figure out what an error means and asks. Feeding that back often gives syntactically correct code in the next response.

The lack of termination isn’t really a problem for the assistant (not full vibe coding) case because if it fails after n attempts you just give up. The extra latency is the killer because it makes it clear that it isn’t actually a productivity win.

@david_chisnall @jonny @kacey don't forget "the thing you want is apparently impossible now, but wasn't impossible a year ago."

I asked Claude Code to make a tiny crud app for me as a side thing while I did other real work, and got it there surprisingly well, but when I wanted to use Instagram's OAuth even massive googling only hinted at what I assume to be the reality: you actually can't "sign in with Instagram" anymore unless you abuse Meta's TOS, which is a relatively recent change.

Ignas Kiela Jul 17

@jonny @david_chisnall @kacey FWIW you could make models that are AST-aware - after all, the model only outputs the probability weights for choosing between tokens - and if your tokens were syntax elements, and the LLM driver eliminated ones that were invalid, you could get way better results. But nobody does that, for some reason.

verified_blobcat

@kacey @jonny I think we had that with one iteration of the Github Copilot extension for PHPStorm. It was much more "beefy autocomplete" than dangling the chat UI.

I found it annoying because it tended to overwhelm the IDE's own (usually more accurate) hints but it occasionally it had a reasonably competent take, that it immediately removed when I typed three more characters, and it didn't feel like jumping OUT OF programming to play Coach The Excessively Positive Enthusiastic Moronbot.

vivi 💫Jul 16

@jonny @kacey Yeah, and this study was done for experienced programmers who knew the codebase

@jonny vibe coding and the vibe is this https://www.youtube.com/watch?v=NE1-dKc6R_I

I have done nothing but teleport bread for three days.

Valve.I DO NOT TAKE CREDIT FOR THIS VIDEO.All rights reserved to Valve.©

YouTube

@jonny
"first-hit-free cursor replacement"

Omg this..

I really don't think lots of devs (thinking mainly of my webdev friends) even give a shit about getting locked in. They're already 'locked in' to so many bloated frameworks..

David Nash Jul 17

@dills_world @jonny Part of the reason I stopped doing web dev as a day job was because it had gone from "ok, we'll choose whatever tools make sense for the task at hand" to "ok, so it's going on the web, so we're going to use React and Webpack and a zillion extra node.js doohickeys".

I still have a personal web site using boring (but fast) old-school server-side rendering of almost all the things. It gives me way less grief.

CyberFrog Jul 17

@jonny honestly the people doing vibe coding aren't even thinking that deeply about the issue, they just see "oh cheap code generator" and immediately jump to "how can this benefit me?" skipping all fundamental analysis of the situation they're creating for themselves

which ironically perfectly explains the vibe coding mindset anyway

@jonny @drahardja I have a client that produces a SaaS product that wanted to have a viber rewrite completely and for it to include all sorts of “AI” features that just wouldn’t work, I’ve been telling him that it’s not going to happen… he was amusingly quiet this week.

Chris Ford Jul 17

@jonny I saw the study that said AI slowed down experienced coders, doesn't surprise me all that much. It's a LOT of years since I've coded as a full time job, but I can imagine an amount of time reworking the AI's suggestions to your style and approach getting in the way.

As someone who now only codes occasionally when I need a small something or want to do a bit of an experiment I find GitHub Copilot really useful to get me started on things. Some of its suggestions are dumb, but it gives me a starting point I can improve on. Speeds me up heaps, but can see it would slow some people down.

People with almost no coding skills full on vibe coding for production use cases? Sure it might work sometimes, but who's going to maintain it, who's going to make sure it's secure, scalable, reliable etc?

@jonny Could there be psychology at work here? People who may always wanted to do some programming but were formed by society to think of themselves that they would never be able to do this.

Upon seeing this cool new tech that finally enables them to do all the cool stuff they always wanted they just dive deep into it.

jonny (good kind)Jul 17

@Isurandil
Very in favor of breaking down imposter syndrome that makes people think they can't "be a programmer" for whatever reason. Vibe coding, Alas, does not provide a great gateway for learning to be a programmer, but to be a slot machine junkie

@jonny Vibe coding is bad, but it seems there are people who cannot recognize this.