The rise and fall of peer review

Why the greatest scientific experiment in history failed, and why that's a great thing

Experimental History

@gvwilson I liked it!

»We go to conferences where people give talks about unvetted projects, and we do not turn to each other and say, “So interesting! I can’t wait for it to be peer reviewed so I can find out if it’s true.”«

I esp. like how it applies to software development and yet doesn't. Tools and methods are definitely shared and flow through conferences.

But it's also true that technology is built through peer review, for instance protocols, architecture.. and *code review*.

@gvwilson

Mind blown.

But confirmed what I suspected all along

@gvwilson

I think this mischaracterizes the purpose of peer review and, more substantially, sets aside things related to Goodhart's law, shifting incentives in commercial academic publication, and changes in technology. But at the level of "something is rotten in the state of Denmark" I'll concur.

@gvwilson
I've read this carefully, and, I disagree a lot. A million things have changed in the way science works since the 1950's; to blame peer-reviewing appears arbitrary. Bibliometry-based evaluation, journal esteem hierarchy, merchandizing of scientific knowledge, oligopoly of big publishers are much more plausible causes of our systemic problems.
@gvwilson
Everybody is aware about what peer-reviewing is: two persons having, more or less professionally, validated a scientific document. No more, no less.
And yes, posts on social networks often do more buzz that peer-reviewed papers, so what?
@gvwilson
This is a weird combination of some very good points and some of the dumbest things I've ever seen written about science.