Earlier I posted about using ChatGPT's propensity to fabricate citations entirely as a short-term strategy for detecting journal submissions and classroom assignments that had been written by machine.
I've been playing with the system for the last couple of hours, and as best as I can tell, ChatGPT now does a much better job than it did when first released at only citing papers that actually exist.
They're not perfect—for example, DOIs can be wrong and some are fabricated—but most are not.
If I'm not just imagining things, it raises an interesting question.
While this constitutes an "improvement" in the technology in some manner of speaking, it's unclear whether this is a desirable development. It strikes me as a step that makes detecting more difficult without confering any significant epistemic improvement.
In other words, if the system has really been adjusted to avoid citing fake papers, this constitutes a deliberate choice to create more persuasive bullshit.
@ct_bergstrom Both "bullshitting" and "Lying" imply consciousness and agency - in other words that it knows what it's doing. It doesn't. All it can do is predict very accurately what sort of words fit with what sort of other words. The fact that this fools humans into thinking what it produces is plausible says more about us that about it.
I think we need to be very careful about the language we use to describe what ChatGPT does or we will attribute to it much more power than it actually has.
Carl T. Bergstrom
rob