Science cannot live by LLM alone, part 12,386.
As an experiment, Almira Osmanovic Thunström published fake papers about a fake eye disease that credited Professor Maria Bohm of Starfleet Academy. They were said to be funded by the Professor Sideshow Bob Foundation in partnership with the University of the Fellowship of the Ring.
Despite such obvious trickery, within a few weeks, chatbots were citing the studies as real:
https://www.nature.com/articles/d41586-026-01100-y
@mattsheffield Science cannot live by LLMs alone ... it also requires lazy researchers and sloppy reviewers:
“[T]he fake papers were then cited in peer-reviewed literature. Osmanovic Thunström says this suggests that some researchers are relying on AI-generated references without reading the underlying papers.”*
@mattsheffield
* Being a reviewer myself, I am highly aware of the vastly increasing number of submissions written with the aid of AI and lacking proper verification - and also of colleagues who do not cross-check citations when they review articles.
However, to me this is just another symptom of a well-known problem in academia that existed long before AI: Scientists take shortcuts because they perceive publishing as an annoying side effect - rather than an essential part - of research.
@SleepyCat Sadly all of this is true, and there seems to be no simple solution to the dilemmas attached as long as there is money to be made with scientific publishing.
However, there is one benefit to reviewing: It gives you free access to the latest research - in a raw, sort of “unfiltered” form before it gets smoothed out by the publication process.
Matthew Sheffield
kernpanik 🐾 🕊️ ☮ 🖖
SleepyCat