Taylorism is a management philosophy based on using scientific optimization to maximize labor productivity and economic efficiency.
Here's the result of making the false Taylorist assumption that the output of scientific research is scientific papers—the more, faster, and cheaper, the better.
Papers are not the output of scientific research in the way that cars are the output of automobile manufacturing.
Papers are merely a vehicle through which a portion of the output of research is shared.
We confuse the two at our peril.
The entire idea of outsourcing the scientific ecosystem to LLMs — as described below — is a concept error that I can scarcely begin to get my head around.
sakana.ai/ai-scientist/
"While there are still occasional flaws in the papers produced by this first version..."
Meanwhile the authors note that the output itself fails to meet standards of scientific rigor, but treat this as a minor wrinkle, not a fundamental barrier imposed by using the wrong tool for the wrong job.
Scientists: We need to slow down the publication race and produce higher quality papers at a slower rate to make the literature manageable again.
Engineers: We hear you. Now every lab in the world will be able to produce hundreds of medium-quality papers (with a few mistakes in each) every week.
I do appreciate the authors' candor in detailing failure modes.
A system that makes difficult-to-catch mistakes in implementation, fails to compare quantitative data appropriately, and fabricates entire results—maybe I have high standards but I don't see this as writing "medium-quality" papers.
Here's the weird Taylorism again. The system produces work at the level of an early trainee requiring substantive supervision. This is not good ROI for producing papers.
The primary output of time invested in trainee research is the development of independent scientists—not the research papers.
In the end, how one judges this paper probably comes down to how one assesses the claim that is always used to justify this kind of work.
The authors "believe" that future versions will be
greatly improved.
Given what I know of fundamental limits to what LLMs can do, I see no reason to agree.
When I fail to do something, I either don't publish or very occasionally I publish describing that failure. When I do so, I don't pretend it was a success and promise that it'll magically get better.
@ct_bergstrom
Is the AI scientist is being categorised, by the language used, as a responsible entity?
Is there an explicit statement that a human has been given authority over, and responsibility for, the actions of the AI scientist.
EDIT: I answered my own question.
Neither "responsibility" nor "responsible" are found in the paper.
smh
arxiv.org/abs/2408.06292
I appreciate your full analysis, but honestly, if I were reading the paper, I would have stopped once I realized the name was always set in small caps with some funky spacing. It reeks of marketing.
@ct_bergstrom
> 'The authors "believe" that future versions will be greatly improved.'
I'm wondering if there's a name for this effect - where a machine does something resembling a human activity, and people ascribe it further human qualities - then assume it will advance in ability in a similar manner to a human.
See also: self-driving cars. Shuffling mindlessly around a car park? Great! It'll soon be driving perfectly.
Or not.
@FeralRobots @Spring @coprolite9000 @ct_bergstrom
And then there is the problem of an AI that has defeated the top-level human players is somehow still beatable using a strategy that only an intermediate level player would consider deploying
@coprolite9000 @ct_bergstrom
In #Robotics, it's (reasonably) well-known that people will ascribe human cognitive abilities and intent to absolutely anything.
Roboticists are particularly bad about this (see linked paper below).
I think the optimism that it will get better is not just universal but required for publication and grant approval.
Link: https://www.sciencedirect.com/science/article/abs/pii/S0921889013001863
In the 1960s, without realizing it, AI researchers were hard at work finding the features, rules, and representations needed for turning rationalist philosophy into a research program, and by so doing AI researchers condemned their enterprise to failure. About the same time, a logician, Yehoshua Bar-Hillel, pointed out that AI optimism was based on what he called the “first step fallacy”. First step thinking has the idea of a successful last step built in. Limited early success, however, is not a valid basis for predicting the ultimate success of one’s project. Climbing a hill should not give one any assurance that if he keeps going he will reach the sky. Perhaps one may have overlooked some serious problem lying ahead. There is, in fact, no reason to think that we are making progress towards AI or, indeed, that AI is even possible, in which case claiming incremental progress towards it would make no sense. In current excited waiting for the singularity, religion and technology converge. Hard headed materialists desperately yearn for a world where our bodies no longer have to grow old and die. They will be transformed into information, like Google digitizes old books, and we will achieve the promise of eternal life. As an existential philosopher, however, I suggest that we may have to overcome the desperate desire to digitalize our bodies so as to achieve immortality, and, instead, face up to and maybe even enjoy our embodied finitude.
This proposal reveals a blatant lack of understanding of how LLMs work.
Just a reminder that LLMs have never provided actual answers to any question asked of them or any actual prompt set forth. They have, however, provided *answer-shaped responses*, and we as humans are seriously lacking when it comes to telling one from the other. Folks, a banana-shaped piece of wood is not edible, and I am embarrassed for our species that you cannot tell the difference.
@ct_bergstrom and the judges will be the already-strained academics doing unpaid peer review in an already-strained system. The big commercial publishers will be able to afford the detection tools and learn to use them effectively, maybe. The little journals will be swamped.
Science fiction short story publisher Clarkesworld almost went under from the burden of fraudulent LLM-output submissions. https://neil-clarke.com/a-concerning-trend/
Is academic publishing ready?
I know this is a rhetorical question
But the answer is "no", in case anyone reading wasn't sure
@ct_bergstrom
@vincib @ct_bergstrom Maybe but the difference is that humans face consequences.
(Billionaires are not humans, btw)
@ct_bergstrom While I personally agree with you that
"The primary output of time invested in trainee research is the development of independent scientists"
I think that you'll probably find that this is a minority view, both amongst those doing the training and those funding it, at least in some fields.
I tend to find that even the most progressive funders and supervisors believe that "PhD students are the backbone of the research workforce".
@ct_bergstrom Thanks a lot for making explicit many aspects of what is - to say the least - worrying about this work.
A similar aspect to the point you make about "developing independent scientists", and related to many points that @emilymbender is frequently making, stems from the perspective of human competence: doing research is how individual researchers learn *themselves* about the domain, doing research etc. There is no way of letting someone else do A + then being able to do A yourself.
As a society, then I'd argue that we want _people_ who can do research. 1) Because of their singularly human way of experiencing the world (as opposed to e.g., how bats or AIs ... etc. experience the world) and 2) Because humans are also different we'd like to have diversity in human researchers, not just the single human researcher who can still do it.
@ct_bergstrom Maybe I'm reading too much into this, but this makes me worried about these folks' opinions of others' work. Like do they think the quality of papers in general is so bad that what sounds rather objectively bad ranks as "medium-quality"?
I grant I'm not an academic or in the habit of reading scholarly papers, and I recognize the quality of papers in general could be poor
@ct_bergstrom I came into this thread at this toot and thought #TheAIScientist was the average person working in #AI
In the authors' defense, research in AI is not scientifically rigorous and makes a great deal of unsubstantiated claims (such as by using heat maps to infer meaning into what parts of a computer vision model are responding to)
Perhaps The AI Scientist has simply learned from the people training it?
@ct_bergstrom There shouldn't really be "medium-quality papers" anyway: If your research is worthwhile, methodologically sound and rigorous, it's world class science, period.
If it fails to meet any of the standard, it's rubbish. In the world of h-indices and publish or perish, we are - intentionally or not - reducing "research" to cargo-culting.
I deeply despise every part of it.
Carl T. Bergstrom
skua
jz.tusk
Coprolite9000
Bornach
Ed W8EMV 
Fabian Transchel
Jon Spring
FeralRobots
Robotistry
Dianne Patterson
Gemma ⭐️🔰🇺🇸 🇵🇭 🎐
Jonas
feliz
arceuthobium
🇳𝗮ꜟ𝖼𝘩
EssKah
Joan Combs Durso
Jeremy Kahn
Erik Jonker
Marc Vaudel
Bill Seitz
Bill Hooker
Cassandrich
Benjamin Sonntag-King 🐙
Ian Sudbery
Viktoria Pammer-Schindler
jadonn
A land fit for all our futures
Alastair Temple
🔮 oracle of dylphi :crumb_dancing: 🇬🇾
crab
refraction 