Mastodawn

Björn Brembs Aug 7, 2025

The PNAS paper from Monday got a lot of attention

https://doi.org/10.1073/pnas.2420092122

One particularly attention-grabbing point was the growth of paper mill papers, i.e., the red line.
The area under the black curve is the entire scholarly literature. Judging from reproducibility projects, I have added the # of articles that are likely to be irreproducible (yellow).
Sure, paper mills can some day be a problem. But compared to irreproducibility, it's a really minor problem:

https://bjoern.brembs.net/2024/02/how-reliable-is-the-scholarly-literature/

P.S.:
Yes, the Y-axis should be labelled "per year"

I wonder what makes the red curve so interesting and attention-grabbing for people that they completely forget the yellow curve? Is an AI generated unreliable paper somehow worse than a human-generated unreliable paper?

Björn Brembs Aug 7, 2025

Maybe the paper mills are getting so much attention in the legacy journals because they are a convenient distraction from the fact the yellow curve shows: the by a huge margin largest fraction of unreliable literature is published in traditonal journals - the ones that now heap so much attention on the paper mills.

The corporate publishers also get a twofer: it's convenient to lump the newer publishers (often misleadingly called "predatory") in there - the legacy publishers' "competition".

red Aug 7, 2025

@brembs I think it may be due to a log scale. This makes the red curve look like a bigger problem than it is and makes its growth look faster.

Björn Brembs Aug 7, 2025

Yup, I think this is a really good guess!

RenkeSiems Aug 7, 2025

@brembs
1. Escalating paper mills mean escalating irreproducible research. Technology progress (AI etc.) causes a flood of papers the legacy publishers won't sell, what a pity.
2. We have seen examples in the past that that legacy publishers are affected from these broker networks, too. So studies like this one in PNAS trigger the blame game.

Konrad Hinsen Aug 7, 2025

@brembs I suppose that one reason is the obvious intention behind paper mill products, as opposed to irreproducibility, which often is due to bad practices applied in good faith (often by ignorance).

Björn Brembs Aug 7, 2025

True, but the large majority of at least the retracted papers is for fraud.
Not sure how that can be generalized, of course, but at least it seems this is not as bnig of a difference as it may seem.

But perhaps lots of people assume the incidence of fraud is low? I think i can vaguely remember recent studies that also seem to contradtict that notion...

Konrad Hinsen Aug 7, 2025

@brembs There's always the benefit of doubt!

I suspect that the incidence of doubt, and its contribution to total irreproducibility, depends on the discipline and on the year of publication. Most work I have personally tried in vain to reproduce were from before 2000 and in physics. Probably no fraud there.

Nicolas P. Rougier Aug 7, 2025

@khinsen Do you have the PDF by any chance?

Konrad Hinsen Aug 7, 2025

@rougier There's a download link in this post (first paragraph) on Reese Richardson's blog:

https://reeserichardson.blog/2025/08/04/a-do-or-die-moment-for-the-scientific-enterprise/

A do-or-die moment for the scientific enterprise

Reflecting on our paper “The entities enabling scientific fraud at scale are large, resilient, and growing rapidly”

Reese Richardson

Nicolas P. Rougier Aug 7, 2025

@khinsen Thanks.

Redish Lab Aug 7, 2025

@brembs I wonder if it is because the yellow line is a constant relative to the black line, which means maybe we can live with it. Science has been doing well over those years (despite there being some irreproducible papers*). But exponential growth signals a problem that will soon get out of hand. Can we stop it before it gets to be a pandemic?

* As Stuart Fierstein says in his wonderful book Failure: what proportion of initial results should we expect to be irreproducible in a robust and healthy scientific enterprise? That's not a trivial question.

Björn Brembs Aug 7, 2025

@adredish
I probably should have emphasized more that the yellow line is not in the original figure. I added it as a very rough guesstimate from reproducibility studies. My implied assumption that the rate was constant is very likely a gross oversimplification.

The original authors do not seem concerned at all and neither were the commentators I've seen.

Ulrike Hahn Aug 7, 2025

@brembs @adredish I don't understand the yellow line, Bjoern- isn't it reasonable to assume that paper mill articles have higher irreproducibility rates than other articles, in which case that yellow line should be moving toward the black line?

Björn Brembs Aug 7, 2025

@UlrikeHahn @adredish

Excellent catch! I created the yellow line simply by copying the black one and positioning it at roughly 50%.

That being said, if paper mill papers, say, plagiarize figures, their replicability should be the same as the source material. Not sure what the fraction of totally made up experimental papers is, though.

Either way, assuming a constant 50% is of course a gross oversimplification! But it was just meant as an illustration to give a better sense of proportion.

Björn Brembs Aug 7, 2025

@UlrikeHahn @adredish

My linked post has more numbers...

Konrad Hinsen Aug 7, 2025

@brembs There's another reason for caring more about paper mills: the slope of the curve.

Paper mills are another level of fraud because there are now institutions that actively work towards increasing fraud, as opposed to the uncoordinated individual frauders of the past.

Assuming they are successful and grow, they will gain influence at the institutional level, and their kind of fraud will end up being normalized.

Björn Brembs Aug 7, 2025

Good point - but isn't this what you'd expect for anything to do with computers when you start from zero?

Konrad Hinsen Aug 10, 2025

@brembs Yes, I'd expect it to happen, but it's worrying nevertheless!