#statstab #467 Replication, statistical consistency, and publication bias

Thoughts: Is the replication of a a finding the "gold standard" for scientific discovery? Maybe not.

#replication #metascience #metapsychology #statistics #bias #QRPs #nhst

https://doi.org/10.1016/j.jmp.2013.02.003

🎧 New #OpenScience Bites #podcast episode on #reproducibility!

Felipe Romero, #philosopher of #science at @facultyofphilosophygroningen, shares why he thinks reproducibility is undervalued, how reproducibility has advanced in recent years, and what needs to change for reproducibility to become part of everyday #scientific practice.

🔗 https://www.rug.nl/research/openscience/podcast/#Romero

#academicChatter #AcademicMastodon #philosophy #PhilosophyOfScience #MetaScience

A new #Science review of 34,000 world-class performers shows a surprising pattern: most top scientists, athletes, musicians, and chess players were not child prodigies. About 90% of elite adults were different people than the early “stars.”

📄 https://doi.org/10.1126/science.adt7790

Future champions often developed slower, explored multiple disciplines, and avoided early specialization. Early over-training may help kids win, but it rarely creates world leaders. 🤔

#Education #Creativity #Learning #Metascience

RE: https://fediscience.org/@UlrikeHahn/115887508756592856

Check out this paper on different conceptions of #reasoning and how it impacts the debate about whether #AI models can do it. 👇

#computerScience #psychology #philosophy #tech #publicDiscourse #academia #philSci #metascience #conceptualAnalysis #interdisciplinary

A new experiment asked 158 researchers in 71 teams to analyse the same #data and answer the same question: does immigration affect support for welfare policies? The results were all over the place - from strongly negative to strongly positive:

 https://doi.org/10.1126/sciadv.adz7173

The key finding: researchers’ political views mattered. Pro-immigration teams were more likely to find positive effects, while anti-immigration teams found negative ones.

#Ideology #OpenScience #Replication #Metascience

New (at least for me) term alert!

TORTURED PHRASE: An established scientific concept paraphrased into a nonsensical sequence of words. "Mean square error" becomes "mean square blunder". "Breast cancer" becomes "Bosom peril". Sign of fraudulent research. Can be a product of AI technologies.

https://thebulletin.org/2022/01/bosom-peril-is-not-breast-cancer-how-weird-computer-generated-phrases-help-researchers-find-scientific-publishing-fraud/

#metascience #researchfraud #academicethics #AI

"Bosom peril" is not "breast cancer": How weird computer-generated phrases help researchers find scientific publishing fraud

Many fraudulent scientific papers contain "tortured phrases," standard scientific terms paraphrased into gibberish. Breast cancer is a scary disease, but what is bosom peril? These phrases are likely popping up because scientists, to boost their article count, are running texts through paraphrasing software, which helps them avoid plagiarism accusations. And new AI technologies may be crafting some article text from whole cloth.

Bulletin of the Atomic Scientists

NEW PREPRINT

Applicants as reviewers: A mixed methods evaluation of the risks, benefits, and potential of distributed peer review for grant funding allocations

https://osf.io/preprints/metaarxiv/t2p56_v1

#Metascience

OSF

If a statistical test always produces the same result and if its results are often (usually?) described fallaciously, should the test be abandoned?

If so, we may need to abandon p-curve analysis:

https://replicationindex.com/2026/01/03/what-a-decade-of-p-curve-analyses-tells-us-about-false-positive-psychology/

#stats #psychology #philSci #metascience #sciComm

What a Decade of P-Curve Tells Us About False-Positive Psychology - Replicability-Index

The File-Drawer Problem A single study is rarely enough to provide sufficient evidence for a theoretically derived hypothesis. To make sense of inconsistent results across multiple studies, psychologists began to conduct meta-analysis. The key contribution of meta-analyses is that pooling evidence from multiple studies reduces sampling error and allows for more precise estimation of effect

Replicability-Index - Improving the replicability of empirical research

#statstab #456 Bestiary of Questionable Research Practices in Psychology

Happy new year! ✨

Thoughts: Starting the year with a resource to make you more cynical in 2026.

#QRPs #phacking #metascience #research #ethics

https://journals.sagepub.com/doi/10.1177/25152459251348431

RE: https://social.tchncs.de/@Kudusch/115843194087173145

Deep down, I know I shouldn't be surprised or shocked by these results and yet I am...

#metascience #OpenScience