"While economists pride themselves on the robustness of their seminars, what actually matters is publication in just five journals. The editors have immense power. Peer review is closed and anonymous. Virtually nothing is ever retracted. Post-publication peer review is minimal. Instead, my experience suggests that there is a culture of not publicly criticizing anything that has been published. If you do, you are viewed as too aggressive, possibly due to some kind of personality defect. Meanwhile, the original authors can use their right to reply for deflection and ad hominem attacks. The fear of upsetting one’s superiors is palpable.
The machines will perhaps bring about some changes. They are massively useful for replication. Without asking a robot to explain it to me repeatedly as if I were a five-year-old, I never would have understood the SCM or Borusyak et al.’s critique of Autor et al.; I never would have done all the coding my replications required; I never would have been able to search for errors in German-language source materials. The cost of doing replication studies has dropped dramatically, even if the institutions and the culture of economics are still hostile to them. Perhaps the system will catch up.
Ultimately, however, human experts are still needed to determine the truth. "
https://thepoorrichworld.substack.com/p/reflections-on-my-adventures-in-replication
#Economics #SocialSciences #Replication #PeerReview #AcademicPublishing