Papers are not the output of scientific research in the way that cars are the output of automobile manufacturing.

Papers are merely a vehicle through which a portion of the output of research is shared.

We confuse the two at our peril.
‪
The entire idea of outsourcing the scientific ecosystem to LLMs — as described below — is a concept error that I can scarcely begin to get my head around.

sakana.ai/ai-scientist/

"While there are still occasional flaws in the papers produced by this first version..."

Meanwhile the authors note that the output itself fails to meet standards of scientific rigor, but treat this as a minor wrinkle, not a fundamental barrier imposed by using the wrong tool for the wrong job.

Scientists: We need to slow down the publication race and produce higher quality papers at a slower rate to make the literature manageable again.

Engineers: We hear you. Now every lab in the world will be able to produce hundreds of medium-quality papers (with a few mistakes in each) every week.

I do appreciate the authors' candor in detailing failure modes.

A system that makes difficult-to-catch mistakes in implementation, fails to compare quantitative data appropriately, and fabricates entire results—maybe I have high standards but I don't see this as writing "medium-quality" papers.