Earlier I posted about using ChatGPT's propensity to fabricate citations entirely as a short-term strategy for detecting journal submissions and classroom assignments that had been written by machine.
I've been playing with the system for the last couple of hours, and as best as I can tell, ChatGPT now does a much better job than it did when first released at only citing papers that actually exist.
They're not perfect—for example, DOIs can be wrong and some are fabricated—but most are not.
If I'm not just imagining things, it raises an interesting question.
While this constitutes an "improvement" in the technology in some manner of speaking, it's unclear whether this is a desirable development. It strikes me as a step that makes detecting more difficult without confering any significant epistemic improvement.
In other words, if the system has really been adjusted to avoid citing fake papers, this constitutes a deliberate choice to create more persuasive bullshit.
Also to be clear, the system is most definitely bullshitting if not outright lying.
Take, for example, this methods section ChatGPT just generated in response to my prompt "Write an scientific review paper with references about whether sunscreen ingredients are carcinogenic."
Unlike many factual claims about background, this is straight up false.
The system decidedly DID NOT conduct a comprehensive literature search on specifically those sources using specifically these terms.
@ct_bergstrom
That is very disappointing. Because doing literature search regarding a special question should be something #AI die really help and shine.
Without adhering to facts #ChatGPT is just a fancy expensive Lore Ipsum generator 🤬
Carl T. Bergstrom
Claudius Link