Matt WillemsenMar 16, 2023
‘Spell-checker for statistics’ reduces errors in the psychology literature - A free-to-use tool designed to detect statistical errors is significantly reducing the number of mistakes creeping into papers, suggests a study published this January.
First described in a 2015 publication2, statcheck is an online tool that can identify errors in the P values — a controversial but popular statistical measure — reported in scientific papers. https://www.nature.com/articles/d41586-023-00788-6 #statistics #ErrorReduction ##statcheck #psychology
‘Spell-checker for statistics’ reduces errors in the psychology literature

Developed to detect statistical errors, statcheck reduces mistakes in reported P values by up to 4.5-fold.

Show threadDr. Michèle NuijtenJan 31, 2023

🔢 You can find the preregistration, data, and R code here: https://osf.io/q84jn/

🤖 Interested in #statcheck? Check out the latest version on http://statcheck.io.

Estimating the effectivity of statcheck in peer review to reduce statistical reporting inconsistencies

We compare the prevalence of statistical reporting inconsistencies between journals that implemented statcheck in their peer review process and matched control journals, before and after statcheck implementation. Hosted on the Open Science Framework

OSF
Show threadDr. Michèle NuijtenJan 31, 2023

These results provide initial evidence that using #statcheck in peer review may be a successful intervention to decrease statistical reporting inconsistencies. 🤖✅

but... >>

Show threadDr. Michèle NuijtenJan 31, 2023

We compared statistical inconsistencies in 2 journals that implemented #statcheck in peer review and 2 matched controls, before and after statcheck implementation:

- Psych Science (🤖) vs. Journal of Exp Psych: General

- Journal of Exp Soc Psych (🤖) vs. Journal of Pers & Soc Psy

Show threadDr. Michèle NuijtenJan 31, 2023
Several journals started implementing #statcheck in their peer review process. Their hope is that requiring a “clean” statcheck report will significantly reduce statistical reporting inconsistencies in the articles they publish. BUT! This remains an empirical question.
Dr. Michèle NuijtenJan 31, 2023

🚨 preprint alert 🚨

Initial evidence that using #statcheck in peer review may reduce errors.

Results from a preregistered observational study of 7000+ psychology articles.

/w @JelteWicherts

🧵

https://psyarxiv.com/bxau9

Dr. Michèle NuijtenJan 24, 2023

✨ #statcheck v1.4.0 now on CRAN! ✨

🔍 better stats finding
⚙️ completely refactored internals
🧹 cleaner variable names
🐞 squashed some bugs

List of all updates: https://github.com/MicheleNuijten/statcheck/blob/master/NEWS.md

Check it out in #R or visit the web app at http://statcheck.io

statcheck/NEWS.md at master · MicheleNuijten/statcheck

A spellchecker for statistics. Contribute to MicheleNuijten/statcheck development by creating an account on GitHub.

GitHub
Dr. Michèle NuijtenJan 23, 2023

Thank you @hadleywickham , for making CRAN pkg uploading so much less painful 🙏

(keep an eye out for #statcheck news! 👀 )

Dr. Michèle NuijtenJan 6, 2023

#introductions

Hi Mastodon! 🐘

I'm Michèle, an assistant professor in psychological methods from the Netherlands. I’m looking forward to talk and read about my research interests here on Mastodon (but I will definitely get distracted by funny animals and DIY videos). Topics you’ll find on my timeline:

🔓 #openscience

🤖 #statcheck- a spellchecker for statistics

🔬 #metascience to improve psychological research

Looking forward to finding my way here!

Show threadPhil McAleerNov 19, 2022
@srossit not sure if this is quite what you mean sorry but maybe #StatCheck by Sacha Epskamp, Michèle Nuitjen and Sean Rife: http://statcheck.io/ You can run a word file, pdf or webpage through the online shiny app I think or there is an #rstats package version on #cran - https://cran.r-project.org/web/packages/statcheck/index.html
statcheck // web

validation of reported statistics in scientific papers