Mastodawn

Very experienced software developers can be incredibly deluded about the quality of vibe coded output esp. for large scale feature work - see the SBCL discussion on fibers https://sourceforge.net/p/sbcl/mailman/sbcl-devel/thread/CACxje5-Nwfoe8REix2B8KGksLi31OSW0A32LQbfCccrpM1R1%3DA%40mail.gmail.com/#msg59302483 - worth reading closely. A proposal/implementation that took a couple days to vibe now has no specific timeline to address all the issues raised by reviewers.

Thread: [Sbcl-devel] SBCL Fibers proposal | Steel Bank Common Lisp

💙🩷💜Ⓑⓡⓔⓣⓣ🐡🍉🐧Mar 15

@[email protected] the exact thread that led to my decision to drop SBCL and Lisp, I'm using Chicken Scheme now instead and am happy. (Chicken does accept vibe coded contributions in theory, I have not seen any commits though).

swannodette Mar 15

@brettm I don't see any cause to drop SBCL in that thread? But ok.

starscape 6d ago

@swannodette @brettm If anything, the response from the maintainers was exactly what I'd like to see. They were extremely professional and engaged genuinely with the proposal while noting its flaws and poking holes. They are clearly not blindly accepting slop and are holding LLM-generated material to the same standard (if not higher) than human-generated.

💙🩷💜Ⓑⓡⓔⓣⓣ🐡🍉🐧6d ago

@[email protected] @[email protected] ok but how about the planet-destroying llms that wrote the code, no concern for the hundreds of millions of extra people that will die due to encouraging all those data centres to be built and add and extra 20-30% draw on the world's power grids? Can't get much more professional than throwing humanity under the bus, I suppose!

💙🩷💜Ⓑⓡⓔⓣⓣ🐡🍉🐧Mar 15

@[email protected] ps tbh SBCL looks like a joke for not telling that submitter to fuck off from their project

swannodette Mar 15

@brettm I maintain projects and I wouldn't interact w/ people that way even if I disagree strongly w/ something.

💙🩷💜Ⓑⓡⓔⓣⓣ🐡🍉🐧Mar 15

@[email protected] others may choose different words 🙂

but the point I mean to make is the original submitter is clearly way out of their depth, and rather than choose some simple basic improvement that they are capable of making, chose a grandiose major subsystem like threading, and got an llm (which also does not know what it is doing) to cobble together something plausable-looking.

Whether you want llm submissions in a project or not, the _absence_ of that whole submission being shot down in no uncertain terms is the telling part.

swannodette Mar 15

@brettm by not shooting it down shows that the project maintainers take new contributions seriously. The submitter was not an inexperienced Lisp hacker. The SBCL folks seem to have their heads on straight far as I can tell wrt. civilized discourse.

💙🩷💜Ⓑⓡⓔⓣⓣ🐡🍉🐧Mar 15

@[email protected] be serious mate, don't treat me like an idiot from behind your facade of civility, if not inexperienced then why submit slop with such glaring flaws

swannodette Mar 15

@brettm I don’t know SBCL internals, but my reading of the discussion is that there are many non-obvious issues (not glaring) and the context to see them isn’t in the head of any particular contributor. This issue is what I was trying to get at.

@swannodette The irony that I click the link and the ad on the page is for a Slop generator.

Craig Brozefsky Mar 15

@swannodette Christophe’s lengthy review resonates with my experience reviewing design docs and code that was largely llm generated by a skilled colleague.

The meta comments he makes about the fatigue of reading due to the error rate, confident language, and structure transcends subject matter, in my experience. The knowledge that a stochastic machine is a intimate interlocutor between you and the other participant is tiring. You lose signals about where the submitter has spent attention and time, and where machine extrapolation has convinced them upon initial review.

I have seen this also in professional work, reviewing PRs from senior devs who have been , to be blunt, degraded by this type of LLM interaction.

Our approach at work has been to hold the submitter personally responsible for the quality of work and communication, and ask for transparency in the use of the tools, while making it clear that they are accountable for the quality and also the clarity of communication around it.

Software is easier to fix than relationships, so I appreciate the way SBCL handled this. I caution others against conflating this proposal with the slop PRs curl and others get which overwhelm them.

@craigbro @swannodette I appreciate this kind of comment/pushback because while I'm staunchly opposed to LLMs on social grounds, I do see that Chrstophe and Stas both still deeply care about maintaining a quality compiler. Anthony Green, the one driving the implementation of fibers is not a nobody but it definitely seems like Claude is in the driving seat based on the commits of their branch. They do seem to be taking it seriously though.

Still its hard to tell whether or not one should be staunchly opposed to LLMs even in cases like this where the project is still in quality hands. Do we ignore the social chaos of LLMs even when they're used 'appropriately' (and is that something that can be done?). Genuine questions I'm unsure of.

G. Wozniak Mar 16

@zyd @craigbro @swannodette For me, the answer is yes, they should be rejected because of the source. If you could do an ethical LLM (still unsure about that) then I would probably be fine with them when used responsibly (meaning "not spamming with submissions").

When Claude and friends are the source these should be rejected outright. The social cost is too high.

@gwozniak @craigbro @swannodette Just in case I wasn't clear, and to make sure I understand your own position: what I meant is, if a given project starts accepting LLM contributions like these, and the maintainers response is to critically evaluate them, perhaps with extra scrutiny like what is happening in this list thread, should you be staunchly opposed to even using this project now that it is incorporating LLM contributions?

I'm always going to be opposed to LLMs at all times no matter what, especially when it comes to using them: that's my own hardline. What I'm extremely unclear about is whether this hardline then should be extended to software I use, such as SBCL. It's a case where the maintainers themselves are not vibecoding and have talked about how they themselves do not use LLMs but they will accept such patches.

Even when 'software quality is still in command' so to speak, should projects like SBCL still be rejected (in use, promotion/recommendation etc) on the basis of the social ramifications of LLMs?

Craig Brozefsky 6d ago

@zyd @swannodette You can answer that question for yourself, even deciding not to use any software that accepts LLM contributions based on your own principals. Some people don't use firmware blobs -- even tho it essentially limits them to old hardware with relatively limited capabilities. That has social and ecological impacts too, but it's normalized in our social circles and we sympathize with those who "need" to use hardware like that.

It appears to me that what is hard is not your decision but how you relate to others who make different decisions, and come to terms with the impact your decision has on your use of the shared social labor as this technology in it's current monopolist, extractive, and exploitative form, is normalized.

@swannodette Damn.
I'm impressed with Christophe's patience through all this. Looks like this will turn out to be an excellent practical demonstration that you can't simply outsource your thinking to a text-blender, and there's no shortcut to becoming a SME.

Brian Campbell Mar 15

@swannodette OK, I loved this little exchange.

"> I pseudo-believe because an AI told me that Franz didn't have any
> coroutines but that Clozure CL (CCL), it's descendant, does:

"Mother of Lynn Conway, CCL isn't even a descendant of Franz:"

I probably haven't touched Common Lisp in 20 years, I was always more of a Scheme person, but even I could tell off the top of my head that this was bullshit.

And it amazes me that the person had enough skepticism and took the time to write "I pseudo-believe...", but didn't take the time to do one single search to confirm. It's amazing how these tools cause people to just turn off their brains, even when they know they are doing it.

Simon Brooke 6d ago

@swannodette interesting read, and has relevance to the scheduling of my own (very different) #Lisp implementation. The problem of reintegrating results from tasks completed on multiple nodes, while still being able to keep those nodes busy with useful tasks and not waiting idly for other nodes to complete the prior task, is one I haven't really addressed yet.