404 MediaA story about an AI-generated article contained fabricated, AI-generated quotes.
Stuart Longland (VK4MSL)@404mediaco Interesting, @benjedwards is listed as one of the authors there. Wonder how the slop crept into that one?
berserk du soleil@stuartl @404mediaco @benjedwards "I'm a tech journalist who specializes in AI" yep seems like it pal!
@aetataureate @404mediaco @benjedwards
Well, we don't know how that AI slop got in there at this time. I don't know how things work internally at Condé Naste / Ars Technica (or indeed, any media organisation).
That said, a fabricated quote is a fabricated quote, no matter how it was fabricated. I can't trust what Ars Technica publish going forward from now on, I've unfollowed !@arstechnica and placed the account on mute.
@stuartl @404mediaco @benjedwards @arstechnica I'm a journalist and I cover AI sometimes as part of my beat. I would be surprised if Kyle Orland asked AI to scrape quotes for him. Benj Edwards is a freelancer producing multiple stories many days, most of which are press release or social media type rewrites: not unethical, but not the same as reporting. I do not discount Ars overall (you may, to each their own!), but this level of judgment failure would be catastrophic for an AI beat writer.
@stuartl @404mediaco @benjedwards @arstechnica another possibility, but this would surprise me, is that one of the two asked an intern or someone to pull quotes from the blog for them. That would be silly -- a blog is already not a live source so it's not adding novelty or beefiness imo. This is why it reads more to me like an author was asked to report with quotes and chose to shortcut. That's also silly, because you don't need to read far into a single website to get what you need. Alas.
@stuartl @404mediaco @benjedwards @arstechnica if you have any expertise at all on the material you're covering as an AI beat writer, you would never ask any of them to do this work for you. It's a notable failure point of LLMs in general, we all know of AI hallucinations, even people who advocate some or certain AI usage know it is a weakness. (I do not use and have never used an LLM, for my work or anything else.)
Zero@aetataureate if you read 404 article, it was Ben, he claims to have been sick and was a miss, but he was not the sole writer, and editing missed it as well.
@zeroarg yeah, i was seeing that on bluesky in the meantime. i think @404mediaco's phrasing of his statement is not sound to be honest.
(eta: following just information for you:) at web publications, it's not (as far as i know) standard practice to fact check colleagues' work on daily news type stories. i'm not saying that's good or even -- ars will have to watch this guy like a hawk if he stays on. but it's standard at this point, just as information for people.