Mastodawn

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪

I try to be optimistic about academia, but then there are evenings like today...

I am reviewing a textbook on GenAI and second language teaching for a renowned commercial publisher. Although I now priorise reviewing for scholar-led, non-commercial publishers, the topic is of personal interest to me and I accepted this invitation because I wanted to ensure that critical AI literacies were well represented in this new textbook (and decent financial compensation was offered, which I admit was also a motivation). This evening, I started reading the preface (which was short but promising) and the introduction. After a couple of pages of pleasant reading, I came across a statement that I thought was odd so I decided to follow up on the reference that was used to support this statement. It turned out that the DOI in the reference section returned a 404. And that the entire reference does not exist! This made me suspicious so I went through all references in Ch. 1 and found three additional hallucinated references and several more incorrect DOIs! I'm not reading any further – will now stop working and attempt to write a polite review tomorrow.

#AppliedLinguistics #SLA #GenAI #academia

Hemmets Spårbössa Jun 14

@ElenLeFoll oof. i hope they retract that piece of trash without wasting any more time on it

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪Jun 14

@Stoori Oh, it's not published yet. I am peer-reviewing it. So I thoroughly hope it won't get published at all (although it wouldn't surprise me if it still did because who am I to stop the march of progress... etc. etc.).

Hemmets Spårbössa Jun 14

@ElenLeFoll related, i've been very appalled to hear from my colleagues at the uni that in their opinion genAI tools can help in second language learning. i wonder if their opinions are based on randomly generated articles similar to what you were about to review.

@frueheneuzeit Jun 14

@ElenLeFoll oh, please temper the politeness. This ist willful tampering with the scholarly record and gross misconduct.

LC Wanderlust Jun 14

@stefan_hessbrueggen @ElenLeFoll Politeness falls on deaf ears in these cases.
Blame and shame is the appropriate academic response here u. u

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪Jun 14

@lcwander @stefan_hessbrueggen I understand the sentiment because this makes me really despair, too, but I am not a fan of naming and shaming. Especially as here there are multiple authors and this may well be the “work” of just one person. I trust my colleagues and don’t systematically check the integrity of their references. The other authors may also have trusted their colleague in good faith.

Ajay Iyer Jun 14

@ElenLeFoll @lcwander @stefan_hessbrueggen I agree with @lcwander on this. Since the work is supposed to be a collective effort of all the authors, the responsibility (and liability) should be on the same collective.

LC Wanderlust Jun 14

@ajayiyer @ElenLeFoll @stefan_hessbrueggen plus, any decent publisher would have at least some automatic reference/doi checker, so they are wasting your time twice by having you do it for them.
And then they wonder why no one wants to do peer reviews for free...

P.S. there is also a personal risk for you that if this book gets published, then you may be in a public list of reviewers.

@frueheneuzeit Jun 14

@lcwander it's still not a good idea to break the confidentiality of the peer review process. And frankly speaking we lack the data to assess the situation meaningfully (e. g. does the publisher have an AI policy?) And we are not entitled to them imho. @ajayiyer @ElenLeFoll

LC Wanderlust Jun 14

@stefan_hessbrueggen @ajayiyer @ElenLeFoll I am not asking for that, I agree blind peer review is the best we have.
With "blame and shame" I mean a comment to the authors of the like: "There are plenty of other jobs where your laziness will not be equally damaging to science and society. Please consider them."

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪Jun 15

@lcwander @stefan_hessbrueggen @ajayiyer Sorry I misunderstood what you meant. I don't know if this is intentional, but the names of the authors of the book are actually in the running header of the document I have been sent for reviewing which makes this review process single-blind, I guess. Ironically, the document is a password-protected PDF with an annoyingly large "Do not put manuscript into any AI tools" watermark meaning that I cannot highlight or copy + paste anything from the PDF thus rendering the reviewing process considerably more cumbersome than necessary...

LC Wanderlust Jun 15

@ElenLeFoll @stefan_hessbrueggen @ajayiyer I see why you prefer noncommercial publishers. They are really antagonising their reviewers...

@ElenLeFoll It may be time to deploy tools (if they exist—and if they don’t, we need some) that auto check citation URLs and flag errors. Does something like that exist in, for example, website development? It would save reviewers (and copy-editors) so much time. @lcwander @stefan_hessbrueggen

Filambulle Jun 14

@feisty_lemming @ElenLeFoll @lcwander @stefan_hessbrueggen
There are a few. My librarian colleagues have tested them in November 2025 and found none they could really recommend. Some were still better for some tasks or area of research, but the amount of false positive (and false negative), the lack of transparency in criteria and/or analysis was underwhelming. Of course it was six month ago, so maybe some are better now.

@Filambulle Ah. That’s too bad. @ElenLeFoll @lcwander @stefan_hessbrueggen

Mark Dingemanse Jun 14

@ElenLeFoll glad you found out early. Solidarity. For quality journals undisclosed use can be straight reject. Even Arxiv nowadays bans authors for the kind of misconduct you found, as I'm sure you have seen

Here's hoping the publisher values (your) expertise!

Tatjana Scheffler Jun 14

@ElenLeFoll ugh. I second others who recommend being clear and not *too* polite. Sorry that was such a bad experience

Tom Hayward Jun 14

@tschfflr @ElenLeFoll

I agree.

I don't know how it works for textbooks, but this is one of those places where (if it were a paper), I would write at least as many comments to the editor as I would to the authors.
In any just world, it should become a desk-reject once they read the sentences that you wrote above.

Neil Moffatt Jun 14

@ElenLeFoll AI poisoning in far too many ways.

It really has great value in selected areas. But the presumption of a universal value is clearly deeply flawed.

Dr Susi Arnott Jun 14

@ElenLeFoll Thank you for doing this. Good luck staying sane; you are appreciated

Iris Young (he/they/she) (PhD)Jun 14

@ElenLeFoll there's precedent now for considering nonexistent references to be data fabrication. I'd suggest dropping those words in the review.

Jonne Arjoranta 📚Jun 15

@iris @ElenLeFoll I was dealing with a similar case recently and found this useful:

https://www.tandfonline.com/doi/full/10.1080/08989621.2026.2645390

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪Jun 15

@jaranta @iris Thank you!

Polly Kraisus Jun 14

@ElenLeFoll ugh... So demoralising, and so disrespectful to make people waste their time like this.
Hang in there and keep up your great work 💚

Kenneth P Mitton Jun 14

@ElenLeFoll Ah ha. Standard for text written by AI models. I let students discover that they can ask AI to write their report and provide a complete bibliography of references and then I let them know what percentage of the references are not real. Unfortunately I have shifted a large part of a report's mark to proper citation and references. So I just mark their report accordingly and tell them they can complain about their bad mark to their AI bot. :)

Joanie with the Good Hair 😷Jun 14

@ElenLeFoll Well, at least we know for certain that irony is dead..

😩

Olga Lovick (she/her)Jun 14

@ElenLeFoll I would aim for clarity over politeness. This is not acceptable and shouldn’t be treated as such

@ElenLeFoll ugggggggh. Doesn't sound deserving of a polite review, but a caustic one.