Dan Goodman

Idea for a different way of doing project grants. Make decisions about which projects to fund based only on the questions asked, and then require successful grantees to go through a multi round post decision review process to sharpen up the methods and make sure they can answer the question.

One of the main reasons I find grant proposals and reviews unsatisfactory is the incentives are there for applicants to hide methodological issues which limits the collaborative value of the review. In this version, once you get the funding you want to bare all to get the best feedback to improve.

#metaresearch #academicchatter #science

Mar 24 at 11:14pmWeb
Show threadBjörn Brembs3d ago

@neuralreckoning

I can imagine this could be so constructive! Great idea!

Show threadDan Goodman3d ago
@brembs I would actually be really enthusiastic to do this, unlike standard grant review which fills me with existential dread.
Show threadBjörn Brembs3d ago

@neuralreckoning

Me too!

In the meetings after the decision, you'd have only 'winners'. Great atmosphere!

Show threadRedish Lab2d ago

@neuralreckoning but what happens if they ignore the feedback? They already have the funding.

Also, does anyone actually do the project proposed? I don't think I've ever done the actual grant proposed project because our observations change our understanding (and techniques work differently than advertised / expected / did last year .... which opens up surprising questions about what changed that we didn't think mattered...)

Show threadDan Goodman2d ago
@adredish doesn't matter for the same reasons.
Show threadRedish Lab2d ago

@neuralreckoning But then you're back to the fundamental problem with a "questions only" grant review - which is that you are not judging whether the experiments proposed are doable in that lab or whether they are even possible in the first place.

Over the years as the NIH grants have gotten shorter and much more "question focused" than "method/lab focused" is that people propose things that they have no track record of doing (and we all know how complex some of these techniques are to do right) or that are fully impossible given current technologies in the first place.

As you note, there is real incentive to hide methodological details. A questions-only focus leads to funding impossible projects.

Show threadRedish Lab2d ago

@neuralreckoning

Personally, I think that funding ideas rather than people/labs is the fundamental categorical mistake being made in the science funding world today.

Show threadDan Goodman2d ago
@adredish I agree with this completely. My thought was that my proposed change might be more palatable to existing funders because it's a relatively small change to what they already do.
Show threadBjörn Brembs2d ago

@adredish @neuralreckoning

Good point!
One way to do this would be a two stage application. Is there a better way? Is a one-step, 'traditional' process that always checks all methods better thna a two-stage procedure?