You might have read that arxiv is banning people for a year if they post LLM-generated papers and cheered it on. But most of the discussion about this doesn't correctly explain the policy and it is not a good thing.
First up, the policy is that "incontrovertible evidence" of using LLMs and not checking the output is what's at stake. An example given is a hallucinated reference.
Second, the ban will apply to all coauthors of the paper, not just the person submitting.
Third, it's not just a 1 year ban, it's followed by a permanent ban on submitting papers that have not been peer reviewed in a "reputable" journal or conference. Given that arxiv is a preprint server and not a repository for published papers, that makes the ban effectively permanent.
(EDIT: They've subsequently clarified that this isn't a permanent ban but will be lifted after 1 or 3 (the clarification isn't clear) peer reviewed papers. This is still problematic, but much better. The rest of this post left as I originally wrote it.)
So imagine: you are a masters student working on a project with a few other people, and your role is relatively minor. The project leads to a paper and you get your name on it, hurray. The lead author handles the submission and doesn't ask for your permission to send the final version because you're only a masters student. Your supervisor explains that this is how things are done and nothing to worry about. What you didn't know is that someone else on this paper at the last minute made some edits to the grammar of the paper using an LLM because none of you are native English speakers, and the LLM inserted a hallucinated reference. Arxiv picks up on this and you are now permanently banned from using arxiv as a preprint server. Further, every time you try to collaborate with someone else to write a paper and they want to put it in arxiv you have to explain that you can't, and that this means that by collaborating with you, they also can't put it on arxiv. Since arxiv is one of the main channels for distributing papers in your field, soon enough people stop asking you to work with them and your career is effectively over. Because someone else didn't notice that an overenthusiastic grammar checker inserted a fake reference and you weren't in a position of enough power at the time to insist on checking the final version.
Ok that's a long story, but I don't think this is a fanciful situation. Stuff like this happens all the time. It's easy to say - and I've seen a lot of people saying times like this - that everyone should take responsibility for reading the paper, or that it's the responsibility of supervisors to make sure this doesn't happen. But in the world we actually inhabit, power imbalances exist: the masters student can't make the supervisor wait until they read the paper because they're worried about their project grade. Bad supervisors are out there, and it's not fair to punish their students.
This policy will lead to terrible consequences for a lot of innocent people who should not reasonably be held responsible because they weren't in a position of power. I suspect it won't lead to very bad consequences for big name researchers who will just get on the phone to someone at arxiv or one of arxiv's funders and get the decision reversed in their case.
I understand the anger towards LLMs and tech companies, and I share it. I understand the anger towards the people cynically generating whole papers using them, polluting the scientific literature and making all our lives more difficult, and I share it too. But that doesn't mean we should jump to implement extreme and poorly thought out policies that will hurt a lot of people who haven't done anything wrong.
Finally, as an advocate of open science and publishing reform, this is really disappointing from arxiv. By saying that peer reviewed papers in "reputable" journals are ok, they've defined themselves (arxiv) as second class citizens in the world of publishing. This shows such limited ambition, and actively hurts the cause of making the world better by getting rid of the parasitic and harmful publishing industry.
#academicchatter #arxiv
Hendrik Weimer
Derek van Vliet
Raul Krauthausen
Dan Goodman
Erika
Obsidian Urbex Photography
jonny (nonvenomous)
Don Trueten 
Christian Schwägerl
Anne C.A. Baanen