#statstab #456 Bestiary of Questionable Research Practices in Psychology

Happy new year! ✨

Thoughts: Starting the year with a resource to make you more cynical in 2026.

#QRPs #phacking #metascience #research #ethics

https://journals.sagepub.com/doi/10.1177/25152459251348431

#statstab #442 Understanding uncertainty in school league tables

Thoughts: Rankings are virtually meaningless, especially neighbour values.

#ranking #comparison #uncertainty #QRPs #education #academia

https://research-information.bris.ac.uk/ws/portalfiles/portal/40324984/leckie2011fs.pdf

#statstab #396 If researchers find Cohen’s d = 8, no they didn’t

Thoughts: Sometimes an effect is so impressive that its unbelievable.

#effectsize #cohend #QRPs #sesoi

https://mmmdata.io/posts/2025/07/if-researchers-find-cohens-d-8-no-they-didnt/

#statstab #381 In psychology everything mediates everything

Thoughts: Another great explanation of the pitfalls of mediation analysis. Mediation is not a simple procedure and require strong theory.

#mediation #QRPs #correlation #regression #PROCESS

https://steamtraen.blogspot.com/2020/04/in-psychology-everything-mediates.html?m=1

In psychology everything mediates everything

In the past couple of years I have reviewed half a dozen manuscripts with abstracts that go something like this: <Construct X> is know...

#statstab #369 Don't Trust Internal Meta-Analysis

Thoughts: I don't agree with all the assumptions, but interesting to know how bad these can be.

#metaanalysis #QRPs #minimeta #datacolada #falsepositive #replication

https://datacolada.org/73

[73] Don't Trust Internal Meta-Analysis - Data Colada

Researchers have increasingly been using internal meta-analysis to summarize the evidence from multiple studies within the same paper. Much of the time, this involves computing the average effect size across the studies, and assessing whether that effect size is significantly different from zero. At first glance, internal meta-analysis seems like a wonderful idea. It increases...

Data Colada

#statstab #366 Type M error might explain Weisburd’s Paradox

Thoughts: Learn about type M error while you learn about the issues in criminology!

#replication #typeM #typeS #QRPs #paradox #power #effectsize #errorrate

https://sites.stat.columbia.edu/gelman/research/published/weisburd_28.05.2017.pdf

#statstab #363 p-checker The one-for-all p-value analyzer

Thoughts: Easy way to check for publication bias using some current tools.

#shiny #pvalue #phacking #QRPs #zcurve #bias #pcurve #rindex

https://shinyapps.org/apps/p-checker/

Experience Statistics

#statstab #358 What are some of the problems with stepwise regression?

Thoughts: Model selection is not an easy task, but maybe don't naively try step wise reg.

#stepwise #regression #QRPs #issues #phacking #modelselection #bias

https://www.stata.com/support/faqs/statistics/stepwise-regression-problems/

Stata | FAQ: Problems with stepwise regression

What are some of the problems with stepwise regression?

#statstab #353 The Abuse of Power; The Pervasive Fallacy of Power Calculations for Data Analysis

Thoughts: An seminal paper on "post hoc" power calculations.

#power #QRPs #NHST #posthoc #samplesize #effectsize

https://www.tandfonline.com/doi/abs/10.1198/000313001300339897

#statstab #344 Questionable Research Practices when Using Confirmatory Factor Analysis

Thoughts: CFAs might be better than EFAs, but they are still complicated and easy to abuse.

#factoranalysis #CFA #QRPs #fitindex #guide

https://dr.lib.iastate.edu/bitstreams/4efe4918-7cad-4db1-b3cc-44825cbb6e08/download

DSpace