Tony BarkJimmy Wales Says Wikipedia Could Use AI. Editors Call It the 'Antithesis of Wikipedia'
brucethemooseWhale’s quote isn’t nearly as bad as the byline makes it out to be:
Wales explains that the article was originally rejected several years ago, then someone tried to improve it, resubmitted it, and got the same exact template rejection again.
“It’s a form letter response that might as well be ‘Computer says no’ (that article’s worth a read if you don’t know the expression),” Wales said. “It wasn’t a computer who says no, but a human using AFCH, a helper script […] In order to try to help, I personally felt at a loss. I am not sure what the rejection referred to specifically. So I fed the page to ChatGPT to ask for advice. And I got what seems to me to be pretty good. And so I’m wondering if we might start to think about how a tool like AFCH might be improved so that instead of a generic template, a new editor gets actual advice. It would be better, obviously, if we had lovingly crafted human responses to every situation like this, but we all know that the volunteers who are dealing with a high volume of various situations can’t reasonably have time to do it. The templates are helpful - an AI-written note could be even more helpful.”
That being said, it still wreaks of “CEO Speak.” And trying to find a place to shove AI in.
More NLP could absolutely be useful to Wikipedia, especially for flagging spam and malicious edits for human editors to review. This is an excellent task for dirt cheap, small and open models, where an error rate isn’t super important. And it’s a huge existing problem that needs solving.
…Using an expensive, proprietary API to give error prone yet “pretty good” sounding suggestions to new editors is not.
This is the problem. Not natural language processing itself, but the seemingly contagious compulsion among executives to find some place to shove it when the technical extend of their knowledge is typing something into ChatGPT.
FaceDeerThat being said, it still wreaks of “CEO Speak.” And trying to find a place to shove AI in.
I don't see how this is "shoved in." Wales identified a situation where Wikipedia's existing non-AI process doesn't work well and then realized that adding AI assistance could improve it.
Neither did Wales. Hence, the next part of the article:
For example, the response suggested the article cite a source that isn’t included in the draft article, and rely on Harvard Business School press releases for other citations, despite Wikipedia policies explicitly defining press releases as non-independent sources that cannot help prove notability, a basic requirement for Wikipedia articles.
Editors also found that the ChatGPT-generated response Wales shared “has no idea what the difference between” some of these basic Wikipedia policies, like notability (WP:N), verifiability (WP:V), and properly representing minority and more widely held views on subjects in an article (WP:WEIGHT).
“Something to take into consideration is how newcomers will interpret those answers. If they believe the LLM advice accurately reflects our policies, and it is wrong/inaccurate even 5% of the time, they will learn a skewed version of our policies and might reproduce the unhelpful advice on other pages,” one editor said.
It doesn’t mean the original process isn’t problematic, or can’t be helpfully augmented with some kind of LLM-generated supplement. But this is like a poster child of a troublesome AI implementation: where a general purpose LLM needs understanding of context it isn’t presented (but the reader assumes it has), where hallucinations have knock-on effects, and where even the founder/CEO of Wikipedia seemingly missed them.
Don’t mistake me for being blanket anti-AI, clearly it’s a tool Wikipedia can use. But the scope has to be narrow, and the problem specific.
Adding AI assistance to any review process only ever worsens it, because instead of having to review one thing, now the reviewer has to review two things, one of which is defo hallucinated but it’s hard to justify the “why”, and the reviewer is also paid far less in exchange and has his entire worker class threatened.
I don't see how this fits into the actual case being discussed here.
The situation currently is that a newbie editor whose article is deleted gets presented with a simple "your article was deleted" message. The proposition is to have an AI flesh that out with a "possibly for the following reasons:" Explanation. How is that worse?
All that stuff about paying less and threatening the worker class is irrelevant. This is Wikipedia, its editors and administrators are all unpaid volunteers.