Mastodawn

tautologico Nov 24

Re: last boost https://chaos.social/@dpk/115589097803252590

That OCaml PR is textbook open source in the era of vibe coding...

It's got everything:

- PR submitted without the author acknowledging they didn't write it and don't understand it.
- Copyright laundering.
- "I just wanted to get it done!" versus maintainers who know they have to live with code contributions for years.
- Zero-effort pasting LLM output as reply to real people's thoughtful questions. (At least the author acknowledged what they were doing that time.)
- It doesn't matter that it's hard to review because "AI has a very deep understanding of how this code works."
- "Beats me. AI decided to do so and I didn't question it."

If this is our new world then it's going to turbocharge maintainer burnout.

(If you don't want to read a quite long often depressing thread, would still recommend reading this well reasoned comment by one of the maintainers:
https://github.com/ocaml/ocaml/pull/14369#issuecomment-3556593972 )

Daphne Preston-Kendal (@[email protected])

Fucking. Hell. https://github.com/ocaml/ocaml/pull/14369

chaos.social

Stewart Russell Nov 22

@projectgus holy shit! he just keeps digging! I lost it when he got to the "Here's the AI-generated copyright review", which added words, but no value.

I liked gasche's super-thoughtful comments, esp. the one with "The fact that you were able to generate large amount of code that passes test is interesting, but that's only 20% of the work, the other 80% are to get the feature discussed, reviewed and integrated" — https://github.com/ocaml/ocaml/pull/14369#issuecomment-3556593972

DWARF support for macOS and Linux by joelreymont · Pull Request #14369 · ocaml/ocaml

DWARF v5 Debugging Support for OCaml Native Compiler This PR adds DWARF v5 debug information to the OCaml native compiler, allowing proper source-level debugging in GDB and LLDB. What's Impleme...

GitHub

@scruss @projectgus

Wow.

"AI has a very deep understanding of how this code works. Please challenge me on this."

This is like saying: "I shat on your carpet and it smells really good. Please challenge me on this."

No idea why they have not just blocked the user.🤷

@HaraldKi @scruss @projectgus "I'm right. Debate me."

(Or, pretty much, the original context of the "change my mind" meme.)

abrasive Nov 22

@projectgus holy crap what a shitshow! thanks for boosting this~

This whole "revolution" is highlighting to me a lot of work that our culture labels an "overhead", implying a waste of time, and how much value is actually in that work.

The canonical example in software is developing and using a mental model of how software works, while lines of code are really a side-effect rather than the main show.

abrasive Nov 22

@projectgus But it's equally true in other fields! One I'm heavily exposed to is doctors' notes - there's this big push to do auto transcription, whatever, but then LLM summarisation for the doctor's records. Talking to the doctors I know well, they spend enormous amounts of time and energy writing notes after interacting with patients - viewed by many as overhead again, but in their view it's a critical part of, essentially, validating (or invalidating) their understanding of what's going on!

@abrasive 100%, it's honestly scary. And a real indictment of many people's relationship with thinking.

I'm surprised pro-AI people don't address this more, because I don't think LLMs are totally useless[*] but the consequences of using them excessively and uncritically don't help convince people of that, either. I guess excessive use does help line go up.

[*] I do think their benefits aren't worth the significant societal and energy demand costs, but yeah...

@projectgus @abrasive I've been railroaded into GitHub CoPilot at work. It's honestly a really good tool at what it does well.

Without some photonics or quantum breakthrough to massively reduce the resource consumption though, it's a dead end. Even with that it should be limited to local models where costs are realized closer to the consumption instead of hidden behind a curtain.

Morgan ⚧️Nov 24

@projectgus @abrasive "I guess excessive use does help line go up." And that's it, that's the whole story. "AI" is being pushed into everything, like crude oil, for basically no reason other than line go up. Once again, we're* digging our* own graves for line go up.
*For those of us who refuse to use LLMs, or anyone who has somehow managed to not use oil, other people are digging the graves for us but we sure will be included in the burial

abrasive Nov 22

@projectgus The doctor one is particularly troubling because LLMs generate *likely* text and so of course will miss the things that make each patient different & thus needing of help... And sure the docs review them but the hardest thing to review is something that's 95% correct 95% of the time?

I find so many mistakes in my work in the write-up, because it forces me to linearise my fuzzy fishbowl of thought... and what I swore I *knew* turns out to have been no such thing

@abrasive Totally. I write a lot and I realised a long time ago that writing is often thinking! Outsourcing the writing often means outsourcing the thinking, even if I "review" it later.

Regarding doctor notes, I'm reminded of this paragraph from https://taranis.ie/llms-are-a-failure-a-new-ai-winter-is-coming/

---

Transformers must never be used for certain applications – their failure rate is unacceptable for anything that might directly or indirectly harm (or even significantly inconvenience) a human. This means that they should never be used in medicine, for evaluation in school or college, for law enforcement, for tax assessment, or a myriad of other similar cases. It is difficult to spot errors even when you are an expert, so nonexpert users have no chance whatsoever.

LLMs are a failure. A new AI winter is coming.

Though LLMs had a lot of promise, this has not been demonstrated in practice. The technology is essentially a failure, and a new AI winter is coming.

Taranis

Jay Grant 🏳️‍⚧️Nov 22

@projectgus literally guffawing at the "AI Copyright Analysis" from the plagiarism machine

deepfryed Nov 22

@projectgus We're breeding Eloi, who are we fattening them for I wonder.

@deepfryed Does The Time Machine end with them being fed to Nvidia investors? Or am I remembering it wrong? 🙃

deepfryed Nov 22

@projectgus The uber rich need new sources of protein on Mars, can't farm cattle.

0x4d6165 (Julie or Mae)Nov 22

@projectgus Christ. Imagine being this smug and confident in something you had no hand in writing. The way the vibe coder literally just threw AI replies to questions in the thread and then complained about being treated unfairly is just...wow. No fucking shame

kessler herd immunity Nov 22

@projectgus these always give off a heavy waft of "person who decided they want to be a developer and does not want to do a single thing developers actually do." like they just want a github profile and a blog and some pr's to well recognized projects and that's it

@projectgus Maybe it's time for me to investigate using OCaml, on account of the developers having the patience of saints. I would have shut the whole thread down (and reached through the internet to punch the guy) after the "Here's an AI generated summary" but they wanted to keep the conversation alive.

At least that hole got deep enough that it is probably going to go viral and other project maintainers will recognize the PR submitter's name and know to immediately deny-and-block.

@derek Yes, I try to be patient with contributors but they are next level! It's heartening, despite being a bit painful to read.

I AM BANKSY ☕ / 🗑‍🔥Nov 22

@projectgus
I like the bit where the guy asks the LLM if there are copyright issues with the LLM-generated code.

"Here's the AI-written copyright analysis..."

@projectgus Props to that maintainer for not insulting that person and their ancestors on a dozen generations. Or downright closing the PR without a word. I know I wouldn't have the patience for this bullshit.

Christiano Haesbaert Nov 22

@projectgus whoa gasche is way too nice for this world. I can feel his pain from a mile.

skrlet13 Nov 22

@projectgus
"Accept this PR I barely was bothered to even read. I want free accolades of being accepted on a important project"

I would tell that person to fork and maintain his own version thi. It would be at least some food for thought for him and his arrogance would be contained.

Very Human Robot Nov 22

@projectgus
Delightful:
"you had us review and discuss the implementation of a feature that was in fact already available in the too"

So yeah, AI code generation tools make dangerous fools dangerouser...

"AI has a very deep understanding of how this code works."

🤦🤦‍♂️🤦‍♀️🤦🤦‍♂️🤦‍♀️🤦🤦‍♂️🤦‍♀️🤦🤦‍♂️🤦‍♀️

Altomare Nov 24

@projectgus I had a feeling of déjà vu, and on a hunch I had to confirm something. Yup, that's someone who was into borkchain and the likes. At least their previous toy didn't interfere that much with useful things...

the roamer Nov 24

Heaven help us.

Indeed, a textbook case.

A troubling combination of Dunning-Kruger, general tone-deafness, narcissistic delusion, and specific AI cultism. There are examples of that poisonous mix in every field, this is perhaps the purest case. I found reading the Github thread very illuminating. Thank you for the link!

#StopTheAICorruption

@projectgus Read this thread this morning, it was a wild ride. Especially asking the LLM to explain the copyright issues.

✧✦Catherine✦✧Nov 24

@projectgus this is why Amaranth has a blanket "no AI" contribution policy: you could argue about its social-scale problems but my view is that it fades in comparison to "if I have to deal with slop being submitted I will simply choose to not take submissions from an open set of people at all"

"no AI PRs" is a compromise, with the other option being "the only way to get your code in is to be from the same guild"

Amᵃᵖanda | map data witch Nov 24

@whitequark oh ““no AI” is the price for “open PR system”” is a *great* way to view it!

@amapanda @whitequark agreed!

@whitequark @projectgus There's also "only guild members may submit LLM-generated code" but that may be stretching the definition of "open" (and there may not be any guild members who would submit LLM-generated code anyway).

@projectgus just ban those people from contributing tbh, or close their PR if they dont engage with the code themselves

"I just wanted to get it done!"

then fork it and use it yourself