The numbers from this story are based on a laughable "fake paper detector" that literally consists of the following ONLY. Do the authors:

1) use private (non-institutional) email addresses and/or have a hospital affiliation,

and

2) have no international coauthors.

That's it.

If these criteria are met, the paper is deemed a "potential red-flag fake publication" and counted toward that 30% tally.

Spin notwithstanding, the technical details within preprint itself make it abundantly clear that the method doesn't work.

In a "juiced" test set with as many fake papers as real ones, the indicators that they use have a sensitivity of 86% and a false alarm rate of 44%.

Yes, they flag 44% of the known real papers as fake.

That's not a detector, it's a coinflip.

This should be a profound embarrassment to everyone involved with the preprint and Science story alike.

https://www.medrxiv.org/content/10.1101/2023.05.06.23289563v1.full.pdf

To test their indicator, the authors conjecture that a valid indicator should meet three criteria based on a questionnaire sent to authors:

(i) Authors of fake publications are reluctant to provide critical information as revealed by their response – or non-response – to the questionnaire by the editor,

(ii) the number of fake publications increases steadily over time, and

(iii) journals with a low to medium impact factor are most affected.

There's a huge problem here.

If non US/EU authors are more likely to use non-institutional email addresses, the detector will pick up these authors disproportionally.

And indeed we expect non US/EU authors to

i) have lower response rates, due to language issues not to mention (deserved, apparently!) distrust

ii) make up an increasing fraction of publication share

iii) publish at higher rates in low and mid impact journals

So all three test hypotheses fail to distinguish between fake papers and non US/EU authorship.