Prof. Emily M. Bender(she/her)There's a lot that's alarming in this article, but perhaps the most alarming part is the NYC spokesperson assering that the problem can be fixed via upgrades:
>>
https://www.thecity.nyc/2024/03/29/ai-chat-false-information-small-business/
It seems to bear repeating: chatbots based on large language models are designed to *make shit up*. This isn't a fixable bug. It's a fundamental mismatch between tech and task.
Also, it's worth noting that RAG (retrieval augmented generation) doesn't fix the problem. See those nice links into NYC web pages? Not stopping the system from *making shit up*. (Second column is chatbot response, third is journalist's report on the actual facts.)
>>
Qybat@emilymbender A tangential point: RAG and similar technologies are in demand because it's becoming impossible to find desired information through conventional search, due to the resurgence of SEO methods, the heavy dependence of most websites on advertising, and the optimisation of websites to this aim. Search for a simple question now and you may have to wade through pages of fluff and padding (perhaps AI generated) to make sure you see enough ads to earn your answer.