Reading « #AI reviewers are here — we are not ready » – Good reading!
🤔 It makes me think:
AI reviewers: Yet another example of stupid collective human behaviour?
Instead of rethinking in depth what peer-review means, we pile up technical patches* that fix each other when the bug is by design.
• Peer-review = a mean for a end
• End = build a trust
• Trust = Be convinced that’s “good“
• good is more than correct!
• And good is about a judgment
So we’re back on “Lecture on Ethics” by Wittgenstein. 🤩
*patches: becoming more and more resource hungry…
Next patch: conquer another planet?
« #AI reviewers are here — we are not ready » – by Giorgio F. Gilestro
🤔 It makes me think that as humans we’re individually super smart to invent mechanical tools for (most of the time) compensating our collective dumb failures.
Guess what? For easing this LLM review, I bet the journals’ll ask to submit the paper formatted by another LLM. Somehow a Long Looping Mediocre (LLM) process…
« #AI reviewers are here — we are not ready » – by Giorgio F. Gilestro
Researchers understand, for instance, which experiments are feasible within realistic budget and time constraints, and which protocols are robust or temperamental. But an LLM can only reflect the literature, and published claims can be exaggerated. Moreover, specialists know when older approaches in their field have been, or should be, superseded by techniques that are just starting to appear in an LLM’s training data set.
🤔 This sounds the same criticism as Wittgenstein’s book, right?
« #AI reviewers are here — we are not ready » – by Giorgio F. Gilestro
The noise in human peer reviews is crucial because it arises from the variability in human experience and practical knowledge […]
🤔 Human experience + practical knowledge = implicit knowledge?
Somehow, a “knowledge“ that you acquired but you cannot explicitely describe.
It reminds me Lecture on Ethics by Ludwig Wittgenstein…
« #AI reviewers are here — we are not ready » – by Giorgio F. Gilestro
#ArtificialIntelligence promises rapid and polite feedback on papers — but we must first review the reviewer.
Published in some Nature stuff
https://archive.is/VuiRP (no paywall)
#OpenScience #Sciecne
2/7
« #AI reviewers are here — we are not ready » – by Giorgio F. Gilestro
#ArtificialIntelligence promises rapid and polite feedback on papers — but we must first review the reviewer.
https://www.nature.com/articles/d41586-025-03909-5 (paywall)
#OpenScience #Sciecne
1/7
How to Boil an Egg? Scientists Claim to Have Cracked the Recipe.
Their new method takes 32 minutes.
https://www.nytimes.com/2025/02/06/science/boiled-egg-science-recipe.html
Simon Tournier
Global Museum
anubis2814
Boing Boing