Björn BrembsAug 7, 2025

The PNAS paper from Monday got a lot of attention

https://doi.org/10.1073/pnas.2420092122

One particularly attention-grabbing point was the growth of paper mill papers, i.e., the red line.
The area under the black curve is the entire scholarly literature. Judging from reproducibility projects, I have added the # of articles that are likely to be irreproducible (yellow).
Sure, paper mills can some day be a problem. But compared to irreproducibility, it's a really minor problem:

https://bjoern.brembs.net/2024/02/how-reliable-is-the-scholarly-literature/

Show threadBjörn BrembsAug 7, 2025

P.S.:
Yes, the Y-axis should be labelled "per year"

I wonder what makes the red curve so interesting and attention-grabbing for people that they completely forget the yellow curve? Is an AI generated unreliable paper somehow worse than a human-generated unreliable paper?

Show threadKonrad Hinsen
@brembs I suppose that one reason is the obvious intention behind paper mill products, as opposed to irreproducibility, which often is due to bad practices applied in good faith (often by ignorance).
Aug 7, 2025 at 10:20amWeb
Show threadBjörn BrembsAug 7, 2025

@khinsen

True, but the large majority of at least the retracted papers is for fraud.
Not sure how that can be generalized, of course, but at least it seems this is not as bnig of a difference as it may seem.

But perhaps lots of people assume the incidence of fraud is low? I think i can vaguely remember recent studies that also seem to contradtict that notion...

Show threadKonrad HinsenAug 7, 2025

@brembs There's always the benefit of doubt!

I suspect that the incidence of doubt, and its contribution to total irreproducibility, depends on the discipline and on the year of publication. Most work I have personally tried in vain to reproduce were from before 2000 and in physics. Probably no fraud there.

Show threadNicolas P. RougierAug 7, 2025
@khinsen Do you have the PDF by any chance?
Show threadKonrad HinsenAug 7, 2025

@rougier There's a download link in this post (first paragraph) on Reese Richardson's blog:

https://reeserichardson.blog/2025/08/04/a-do-or-die-moment-for-the-scientific-enterprise/

A do-or-die moment for the scientific enterprise

Reflecting on our paper “The entities enabling scientific fraud at scale are large, resilient, and growing rapidly”

Reese Richardson
Show threadNicolas P. RougierAug 7, 2025
@khinsen Thanks.