“Statisticism refers to an overemphasis on abstract statistical principles at the expense of context-specific nuance and caveats (e.g., Boring, 1919; Brower, 1949; Proulx & Morey, 2021). Statisticism may help to explain the unthinking statistical ritualism that has been noted by some commentators (Gigerenzer, 2004, 2018; Proulx & Morey, 2021).” #statistics #stats #MultipleTesting #Statisticism
Attached: 1 image New article from me: “Redundant multiple testing corrections: The fallacy of using family-based error rates to make inferences about individual hypotheses” Preprint: https://doi.org/10.48550/arXiv.2401.11507
Multiple Testing:
New article discusses the “use and misuse of corrections for multiple testing.”
“In general, avoid corrections for multiple testing if statistical claims are to be made for each individual test...”
New paper provides a history of “voodoo science,” which discusses the controversy surrounding Vul et al.’s (2009) controversial article “Puzzlingly High Correlations in FMRI Studies of Emotion, Personality, and Social Cognition.”
Five quotes follow: 🧵👉
🔓 https://doi.org/10.3390/socsci12010015
#MetaScience
#Neuroscience
#Neuroimaging
#MetaResearch
#PsychMethods
#ReplicationCrisis
#PhilosophyOfScience
#PhilSci
#Fmri
#VoodooCorrelations
#UseNovelty
#MultipleTesting
Since the 1990s, functional magnetic resonance imaging (fMRI) techniques have continued to advance, which has led researchers and non specialists alike to regard this technique as infallible. However, at the end of 2008, a scientific controversy and the related media coverage called functional neuroimaging practices into question and cast doubt on the capacity of fMRI studies to produce reliable results. The purpose of this article is to retrace the history of this contemporary controversy and its treatment in the media. Then, the study stands at the intersection of the history of science, the epistemology of statistics, and the epistemology of science. Arguments involving actors (researchers, the media) and the chronology of events are presented. Finally, the article reveals that three groups fought through different arguments (false positives, statistical power, sample size, etc.), reaffirming the current scientific norms that separate the true from the false. Replication, forming this boundary, takes the place of the most persuasive argument. This is how the voodoo controversy joined the replication crisis.
Useful recommendations when to use corrections for #MultipleTesting and when not:
Rubin, M. (2021). When to adjust alpha during multiple testing: A consideration of disjunction, conjunction, and individual testing. Synthese, https://doi.org/10.1007/s11229-021-03276-4
From the abstract:
"It is argued that alpha adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis."
Claudio Agostinelli
bs2
Daniel E. Weeks
Mark Rubin
Daniel Heck