Widely covered MIT paper saying AI boosts worker productivity is, in fact, complete bullshit it turns out.

https://www.wsj.com/tech/ai/mit-says-it-no-longer-stands-behind-students-ai-research-paper-11434092?st=sF3Wvo&reflink=desktopwebshare_permalink

@GossiTheDog Odds on that an AI was responsible for some of the writing and most of the data.

@GossiTheDog

An MIT spokesperson went on to say that they have no confidence in the veracity or reliability of journalistic institutions that repeat claims made in a student paper that has not undergone peer review.

@david_chisnall @GossiTheDog you would be surprised the amount of times I see research papers referring to arxiv articles (not necessarily reviewed, some are pre-studies, bogus AI crap or plain flawed experiments). The sad part is that we also assume publications are the truth once published, when in reality things change. Core to science is reproduction of results, independently. And that does not happen for 99% of them, it’s time consuming and funding is time limited. So there is no incentive to do so. Asking and pursuing answers to tough questions are deemed unproductive, a career is measured by how many papers you published, and that opens a whole new can of worms as “publish or perish”). This, and much more led me to leave academia and not look back.
@GossiTheDog This is total speculation, but I wouldn't put it past the AI techbros to pay some students to write some BS papers. 🤷 It's enough to have some papers out there you can cherry pick. They don't have to be valid, good, or peer reviewed. The revocation process for academic papers is even less effective than the one for TLS certs 😂 Most people outside academia just see that someone cites a paper and assumes the facts are good. If you are lucky someone checks wheter the paper actually exist.

@gilgwath @GossiTheDog

^this

even the dogforsaken antivax movement of today exists precisely because a *doctor was paid by VC pharmabros to write a BS paper*. Which has since been thoroughly debunked *and* eventually retracted, but as you say (and as we're constantly grimly reminded) the damage has been done.

there's no reason to expect this bunch of them bros to be any better, evidence seems to suggest they're even worse.

@maybenot @gilgwath @GossiTheDog I thought Wakefield did it of his own volition, his angle was that he would later market his own vaccines as safe and rake money in.

The scum is probably responsible for more deaths than Putin, Assad, George W Bush and Agathe Habyarimana combined.

@fazalmajid @maybenot @gilgwath @GossiTheDog AFAIK he had a deal with a company to produce those single purpose vaccines
@gilgwath @GossiTheDog Why pay them when you have college techbros drooling at the opportunity to simp for the industry for free?
@dalias @GossiTheDog I assumed that suiciding your academic credibility would be something people wouldn't consider without ample compensation, even if they didn't intend to go into classic academia. I projected myself ascribing value to my own integrity onto other people. My mistake 😞
@gilgwath @GossiTheDog
Pay? I thought that's what interns were for! /s
@gilgwath
And they're already paying thousands of low wage laborers to write content for them anyway, in the name of RLHF (human feedback to correct AI mistakes, in near real time).
@GossiTheDog
@GossiTheDog It is like putting lipstick on AI slop.

@GossiTheDog

#ALT4you

screenshot from the article. It reads:
MIT didn't name the student in its statement Friday, but it did name the paper. That paper, by Aidan Toner-Rodgers, was covered by The Wall Street Journal and other media outlets.
In a press release, MIT sait it "has no confidence in the provenance, reliability or validity of the data and has no confidence in the veracity of the research contained in the paper."
The university said the author of the paper is no longer at MIT.

@GossiTheDog

FWIW, here's my take.

0: "AI" means three things nowadays: neural nets, machine learning, and LLM stuff. They are different things.
1: There was a paper in Science last year in which Materials Science types were doing some seriously kewl work on systems with 5 different metals using "machine learning" (gradient descent search in high dimensional spaces). And calling it AI.
2: The Econ. grad student didn't understand this and thought they were doing LLM stuff. Oops.

@djl @GossiTheDog "AI" does not mean gradient descent. If you're doing gradient descent in high dimensions for some scientific purpose, awesome, call it that! Calling it "AI" tells us that you're willing to be an advertisement for scammers and pillagers for the sake of hyping your research.
@GossiTheDog One can only hope this paper doesn’t become the basis of a larger non-sense movement like anti-vax.
@lasombra_br @GossiTheDog Too late, already has. The biz world is just agog over the potential wage savings aka increased profits they’ll reap by using AI.
@GossiTheDog I wonder if the author left the US and is now living in the UK. Possibly working as Andrew Wakefield's "houseboy". Apparently he brought his own sarong.
@GossiTheDog
Did they fired the „AI" who wrote that paper…
@GossiTheDog
Was this paper written with a worker productivity boost?
@GossiTheDog and who is surprised?
Silicon Valley is nothing but a hustle now, not for serious people.
@GossiTheDog I'm sure no-one is astounded.
@GossiTheDog So that's curtains for another of the very few examples every hypester constantly pull out (because there's so few of them) when trying to argue how useful #AI can be... squire, fetch my surprised face 😑
@GossiTheDog depends, I find it has helped vastly. Depends on how you use it.

@GossiTheDog

"The author of the paper is no longer at MIT."

What are the odds that has something to do with academic dishonestly?

@GossiTheDog every time I am at a dead spot when programming and try to use AI, it fails completely

Assuring an accurate research record | MIT Economics
https://economics.mit.edu/news/assuring-accurate-research-record

MIT asks arXiv to withdraw preprint of paper on AI and scientific discovery
https://news.ycombinator.com/item?id=44006426

Assuring an accurate research record | MIT Economics

@GossiTheDog With a name like that, I am picturing a cross between an 80s AI chat persona and a Xerox copy machine.

AI-Dan, Toner-Rodgers.