Mastodawn

Andrew Mar 26, 2024

I have a preprint out estimating how many scholarly papers are written using chatGPT etc? I estimate upwards of 60k articles (>1% of global output) published in 2023. https://arxiv.org/abs/2403.16887

How can we identify this? Simple: there are certain words that LLMs love, and they suddenly start showing up *a lot* last year. Twice as many papers call something "intricate", big rises for "commendable" and "meticulous".

#bibliometrics #scholcomm #chatgpt

ChatGPT "contamination": estimating the prevalence of LLMs in the scholarly literature

The use of ChatGPT and similar Large Language Model (LLM) tools in scholarly communication and academic publishing has been widely discussed since they became easily accessible to a general audience in late 2022. This study uses keywords known to be disproportionately present in LLM-generated text to provide an overall estimate for the prevalence of LLM-assisted writing in the scholarly literature. For the publishing year 2023, it is found that several of those keywords show a distinctive and disproportionate increase in their prevalence, individually and in combination. It is estimated that at least 60,000 papers (slightly over 1% of all articles) were LLM-assisted, though this number could be extended and refined by analysis of other characteristics of the papers or by identification of further indicative keywords.

arXiv.org

Andrew Mar 26, 2024

I looked at 24 words that were identified as distinctively LLMish (interestingly, almost all positive) and checked their presence in full text of papers - four showed very strong increases, six medium, and two relatively weak but still noticeable. Looking at the number of these published each year let us estimate the size of the "excess" in 2023. Very simple & straightforward, but striking results.

#bibliometrics #scholcomm #chatgpt

Andrew Mar 26, 2024

Can we say any one of those papers specifically was written with ChatGPT by looking for those words? No - this is just a high level survey. It's the totals that give it away.

Can we say what fraction of those were "ChatGPT generated" rather than just copyedited/assisted? No - but my suspicions are very much raised.

Isn't this all a very simplistic analysis? Yes - I just wanted to get it out in the world sooner rather than later. Hence a fast preprint.

#bibliometrics #scholcomm #chatgpt

Andrew Mar 26, 2024

Is it getting worse? You bet. Difficult to be confident for 2024 papers but I'd wildly guess rates have tripled so far. And it's *March*.

Is this a bad thing? You tell me. If it's a tell for LLM-generated papers, I think we can all agree "yes". If it's just widespread copyediting, a bit more ambiguous. But even if the content is OK, will very widespread chatGPT-ification of papers start stylistically messing up later LLMs built on them? Maybe...

#bibliometrics #scholcomm #chatgpt

Is there more we could look at here? Definitely. Test for different tells - the list here was geared to distinctive words *on peer reviews*, which have a different expected style to papers. Test for frequency of those terms (not just "shows up once"). Figure out where they're coming from (there seems to be subject variance etc).

Glad I've got something out there for now, though.

#bibliometrics #scholcomm #chatgpt

Andrew Mar 28, 2024

huh, this is neat! someone did an AI-detector-tool based analysis looking at preprint platforms, and released it on exactly the same day as mine. Shows evidence for differential effects by discipline & country. https://www.biorxiv.org/content/10.1101/2024.03.25.586710v1

#bibliometrics #scholcomm #chatgpt

Andrew Apr 3, 2024

More on LLMs and peer reviews: https://www.404media.co/chatgpt-looms-over-the-peer-review-crisis/

(Back to work tomorrow, & to revising the paper. I feel it's going to be a race to keep up.)

#bibliometrics #scholcomm #chatgpt

ChatGPT Looms Over the Peer-Review Crisis

Research shows that academics might be using generative AI tools to cut corners on a fundamental pillar of the scientific process.

404 Media

Andrew Apr 10, 2024

my work on LLMs has made it to Nature! (Not a sentence I ever expected to write.) https://www.nature.com/articles/d41586-024-01051-2

#bibliometrics #scholcomm #chatgpt

Is ChatGPT corrupting peer review? Telltale words hint at AI use

A study of review reports identifies dozens of adjectives that could indicate text written with the help of chatbots.

Michael Meckler Mar 28, 2024

@generalising One point in your research is something I have noticed anecdotally, namely that some of the most obvious examples of ChatGPT use in scientific papers involved authors for whom English is not a first language. I suspect that these authors are using ChatGPT as a way of creating idiomatic English text, something that Google Translate does not always provide.

#ChatGPT #GoogleTranslate #linguistics

Andrew Mar 28, 2024

@michaelmeckler I wonder if part of the issue is that these words are not "wrong" but they are (in context) tonally "awkward" - a thing that is harder to spot and edit out for a second language speaker, if they're not looking for it?

(eg in my case I could look at a auto-translated French text and say "yeah, that sounds like what I was trying to get across", but probably not "hmm, that sounds subtly off")

guenther Mar 30, 2024

@michaelmeckler @generalising

maybe this isn't obvious to people who speak English as a first language, but writing something in another language and then automatically translating it does not sound like a reasonable way to publish any kind of long-form writing.

Joe ❌👑May 10, 2024

@guenther @michaelmeckler @generalising It appears that some are writing in English and then asking an LLM to "improve" what they wrote, particularly if they aren't fluent.

New-Cleckit Dominie Mar 26, 2024

@generalising Interesting! I also suspect we'll see a tendency for humans to imitate the style of LLMs, as LLMese becomes a widely used, computer-endorsed, and thus relatively prestigious dialect. (I suspect I'm starting to see this among students already.)

It's good news at least for "outwith", though. I take that as some compensation for the war that spellcheckers have been waging on the word for years.

Andrew Mar 26, 2024

@ncdominie yes, I think this definitely seems plausible - but goodness knows what it will mean for all the people selling tools to detect LLM written student essays!

Can't decide what I think about "outwith". Good to see it being used, but a little disappointed it's not going to be a distinctive sign of human authorship any more...

New-Cleckit Dominie Mar 26, 2024

@generalising We shall just have to increase our use of other Scottish shibboleths and stay one step ahead of the bots.

(I'm going to start using "furth" and "anent" more, and that's just the polite ones.)

Andrew Mar 26, 2024

@ncdominie or we could just accept the inevitable triumph of the Leal Leid Makars?

Ditol Mar 27, 2024

@generalising @ncdominie LLMs might destroy the world, I don't like it but fine. But what does "outwith"mean?

New-Cleckit Dominie Mar 27, 2024

@ditol @generalising Antonym of "within"; English used to use "without" in that sense but has lost it in recent centuries.

Ditol Mar 27, 2024

@ncdominie @generalising Ah, thank you!

Steffen Christensen Mar 26, 2024

@generalising @ncdominie "pivotal", "notable", and "intricate" are going off the *hook* in 2024!

Mark Stosberg Mar 26, 2024

@ncdominie @generalising LLMese will happen. And this one way to describe what Grammarly is selling.

Miranda Gray Mar 27, 2024

@ncdominie @generalising This is a depressing truth you have just revealed to me.

@generalising I wonder if there is a correlation between having these "LLM markers" and the first author being from a non English speaking country. I know a lot of people who use LLM powered tools for translation, and in such cases the Markers would show up, even if the content is totally original.

Just a passing thought, but seems to be a very interesting study, congrats!

Gerard Ritsema van Eck Mar 26, 2024

@Jey_snow @generalising
As an author from a non English speaking country: absolutely. Not just translation though, the stuff I write in English I will often run through quillbot for fluency, or ChatGPT to summarise. Helps tremendously, very meticulous and intricate.

(Also my whole academic career is built around tech law and privacy so very aware of how shady these LLMs can be)

Andrew Mar 26, 2024

@Jey_snow thanks - yes, I think that's very likely! Dimensions doesn't let me easily test for author affiliation location, but I think you'd be safe placing a small bet on it...