Earlier I posted about using ChatGPT's propensity to fabricate citations entirely as a short-term strategy for detecting journal submissions and classroom assignments that had been written by machine.
I've been playing with the system for the last couple of hours, and as best as I can tell, ChatGPT now does a much better job than it did when first released at only citing papers that actually exist.
They're not perfect—for example, DOIs can be wrong and some are fabricated—but most are not.
If I'm not just imagining things, it raises an interesting question.
While this constitutes an "improvement" in the technology in some manner of speaking, it's unclear whether this is a desirable development. It strikes me as a step that makes detecting more difficult without confering any significant epistemic improvement.
In other words, if the system has really been adjusted to avoid citing fake papers, this constitutes a deliberate choice to create more persuasive bullshit.
Also to be clear, the system is most definitely bullshitting if not outright lying.
Take, for example, this methods section ChatGPT just generated in response to my prompt "Write an scientific review paper with references about whether sunscreen ingredients are carcinogenic."
Unlike many factual claims about background, this is straight up false.
The system decidedly DID NOT conduct a comprehensive literature search on specifically those sources using specifically these terms.
i call that confabulating and obfuscating
Carl T. Bergstrom
John Bonomi
Jon Awbrey