1/

#OpenAccess book [1] by @deevybee "As we shall see, demonstrating that an intervention has an impact is much harder than it appears at first sight"

https://mastodon.social/@deevybee/110118670777140484

"Much of the attention of methodologists has focused on how to recognize and control for unwanted factors that can affect outcomes of interest. But psychology is also important: it tells us that own human biases can be just as important in leading us astray"

#Statistics #CognitiveBias #uncertainty #Complexity

2/
"#CitationBias is often unintentional, but is a consequence of the way humans think. Bishop (2020) [2] described a particular cognitive process, confirmation bias, which makes it much easier to attend to and remember things that are aligned with our prior expectations. #ConfirmationBias is a natural tendency that in everyday life that often serves a useful purpose in reducing our #CognitiveLoad, but which is incompatible with objective scientific thinking" [1]

#science #research

3/
"It’s not generally possible to avoid all #ConflictOfInterest, but the important thing is to recognize experimenter #bias as the rule rather than the exception, identify possible threats [...] to study validity, take stringent steps to counteract these, and report openly"

"being a good scientist often conflicts with our natural human tendencies" [1]

"Faulty reasoning results in shoddy #science, even when the intentions are good. Researchers need to become more aware of these #pitfalls" [3]

4/

#References

[1] Bishop, D.V.M., Thompson, P.A., 2023. Evaluating what works. Bookdown. https://purl.org/INRMM-MiD/z-DB2FTMIG

[2] Bishop, D.V.M., 2020. The psychology of experimental psychologists: overcoming cognitive constraints to improve research - The 47th Sir Frederic Bartlett Lecture. Quarterly Journal of Experimental Psychology 73, 1–19. https://doi.org/10.1177/1747021819886519

[3] Bishop, D.V.M., 2020. How scientists can stop fooling themselves over statistics. Nature 584, 9–9. https://doi.org/10.1038/d41586-020-02275-8

#DOI

Evaluating What Works

Introduction to methods for evaluating effectiveness of non-medical interventions

5/

Vast #CognitiveBias spectrum: from data/methods to #science vs. society/policy: e.g. [4]

"Converging evidence from the behavioural and brain sciences suggests that the human moral judgement system is not well equipped to identify #ClimateChange — a complex, large-scale and unintentionally caused phenomenon — as an important moral imperative. As climate change fails to generate strong moral intuitions, it does not motivate an urgent need for action in the way that other moral imperatives do"

6/
"Why climate change doesn’t register as a moral imperative

Certain features of #ClimateChange and the ways in which it is communicated to the public interact with the human moral judgement system to decrease individual perceptions of the issue as a moral imperative.
[...] we identify six primary challenges that prevent climate change from activating the human moral alarm system" and "strategies that communicators could use to increase recognition of climate change as a moral imperative" [4]

8/
#Science is not perfect but its distinctive self-correction ability is key, and it's not automatic. It's a process to foster, not an intrinsic property (as we all are subject to #CognitiveBias, believing that we'll fix this in science, once and for all, is quite an obvious #catch22 paradox - hint: "believing").

Research once honestly believed good may be revisioned later. However, sometimes a thesis/paradigm/school fights to survive beyond good faith, "against" #ScienceSelfCorrection

@dderigo
In case it's of interest, see my 2008 essay on the role of #OpenAccess in facilitating scientific #SelfCorrection.
https://dash.harvard.edu/handle/1/4391168
Open access and the self-correction of knowledge

@petersuber
> my 2008 essay on the role of Open Access in facilitating scientific Self Correction.

I went to the page you linked to read this, but the link under "published version" appears to be broken. I've used the linked form to report the problem to the site admin, but I thought you might also want to know.

FYI that page is also trying you load JavaScript from two third-party domains; cloudflare.com and openrev.orv

@dderigo

@strypey @petersuber below a #PURL link to an archived copy (@internetarchive)

I found key this comment on #ScienceSelfCorrection
"it's precisely because individuals find it difficult to correct themselves, or precisely because they benefit from the perspectives of others, that we should employ means of correction that harness public scrutiny and #OpenAccess"

#Reference

Suber, P., 2008. Open access and the self-correction of knowledge. SPARC Open Access Newsletter 122. https://purl.org/INRMM-MiD/z-SCVLRJHP

Open access and the self-correction of knowledge