@sharoz A big problem is most reviewers won't be able to invest such time, because they have to do a bunch of other things and indeed get no time/money/etc for doing review work.

Given such constraints, one may have asked the authors themselves to prove what they said is correct. This should include describing the design/data collection such that it could actually be replicated.

In any case it shouldn't have been so hard to obtain what was needed for the review (privacy concerns aside)

@NKSchuurman Yeah, a point I want to get across in that bulleted list is that the review took only about an hour in total. The major time drain was the ridiculous back and forth with editors and authors. A reviewer doesn't have to check *everything*, but they should be able to check *anything*.

Also, you might be interested in this http://steveharoz.com/blog/2022/reviewing-tip-the-10-minute-data-check/

@NKSchuurman

Yeah, an hour is REALLY short. It was only possible because I spotted and verified a flaw so devastating that I didn't even feel the need to read the whole paper.

Does this link work?
http://steveharoz.com/blog/2022/reviewing-tip-the-10-minute-data-check/

@sharoz It does now! These are generally sensible things to check/useful tips!

I tend to focus mainly on reviewing what my specific expertise is about (and mostly I review methods papers, which is of course a somewhat different setup for reviewing).