Mastodawn

RE: https://neuromatch.social/@neuralreckoning/116286691385862197

Science would be so much better if we did review (of grants and papers) constructively and collaboratively, instead of only using them to produce binary accept/reject decisions. To do that, we have to separate review processes from decisions. One idea for grants 👇

Show thread

Stephen Royle 5d ago

@neuralreckoning I can see this working for some fields, in the way that pre-registration works for research that can be planned and executed linearly. IME wet lab experimental science is more messy and convoluted. A back-and-forth with a committee could hone a perfect plan on paper, and even with consensus that this is the best approach, failure is a likely outcome (for unforeseen reasons). To quote Mike Tyson "everyone has a plan until they get punched in the mouth”.

Show thread

Dan Goodman 5d ago

@steveroyle I don't see it as honing a plan with a committee. In my view it's more like having a mandatory independent critical opinion. It's much easier to find issues with someone else's ideas than your own. The end result doesn't have to be as detailed as a preregistered experiment, and indeed I don't find pre-registration compelling at all for precisely the reason you state.

Show thread

Stephen Royle 5d ago

@neuralreckoning OK. But currently, I write a proposal and my colleagues give me feedback (this won't work, have you thought of this). Then I submit it and the reviewers give me more feedback to hone the plan. There's no hiding details beyond not having space to describe everything. And I think an applicant needs to say how they would tackle the question to know that it is tractable. So, there's independent critical opinion already in the current process.

Show thread

Dan Goodman 5d ago

@steveroyle yes but the independent critical opinion is only destructive (get the grant rejected) not constructive. And I would guess that when you give your draft proposal to colleagues you're asking them to help you get it funded not how to do the research better, so both you and your colleagues aren't engaging in the type of constructive critical process I'm imagining.

Show thread

Dan Goodman 5d ago

@steveroyle even if this isn't consciously what you're doing, I think the fact that everyone understands the context of the grant writing activity, it's implicitly what you all know you're doing.

Show thread

Stephen Royle

@neuralreckoning yes I agree that the game is grantspersonship. It's also true that external reviewers’ critique is viewed negatively by panels, even when it’s constructive. I don't think critique is "only destructive” but I can see that decoupling the two (decision and criticism) has benefits. For sure, I've had proposals go unfunded where I’m sure the idea was sound but it didn't fly for some reason. Would be great to start from the agreement that it's worth pursuing the idea and build up.