Lukas RöselerMar 2

It's #LoveReplicationsWeek!
I'll be posting impressions from the talks all week here.

>> You can still sign up for talks since we are sending out invites on the morning for new registrations.
https://forrt.org/LoveReplicationsWeek/

>> Slides will be shared in our Zenodo Community: https://zenodo.org/communities/lrw/records

Love Replications Week

Show threadLukas Röseler
Here we go!
@ElenLeFoll
and Mara van der Ploeg kick off the #LoveReplicationsWeek with their talk about the what, why, and how of reproducibility in the language sciences.
Mar 2 at 9:24amWeb
Show threadLukas RöselerMar 2
Elen presents examples of reproductions from her stats class. Here is a wonderful example of a perfect reproduction by one of Elen's student Beatrice. Tthe numbers did not perfectly match and there was a plot for which she had to recreate the code and find out what the error bars represented.
Show threadLukas RöselerMar 2
Here is another interesting case where the authors computed means of means, which was not clear from the report and let to the student get larger estimate when computing means from the non-aggregated data.
Show threadLukas RöselerMar 2
Some of Elen's students publish their reproductions as case studies in her book. This way, they don't only learn about research methods but also apply their knowledge and make a valuable and visible scientific contribution: https://elenlefoll.github.io/RstatsTextbook/B_CaseStudies/Poppy/ThrowVerbs.html
Show threadLukas RöselerMar 2
Mara is now presenting the Tromso Repository of Language and Linguistics or short - TROLLing, which is a free and curated data repo.
Show threadLukas RöselerMar 2
It is not only free but "curated" means, that the team helps you making your data adhere to FAIR principles, visible, peer-review-friendly, and all of that happens fast.
Show threadLukas RöselerMar 2
Next up: Lenka Fiala from the Institute for Replication is taking us on a Reproducibility Study Speedrun.
Show threadLukas RöselerMar 2
She is actually showing us a reproduction study of an article published in the last four weeks!
Show threadLukas RöselerMar 2
With over 100 reproduction studies every year, she knows all sorts of tips, here are some of them:
- Data and code can sometimes be found on the journal website and sometimes in the authors' institutional repo
Show threadLukas RöselerMar 2
It looks like this is a simple case: The authors have provided a readme, code, and data. She has seen lots of replication packages where researchers shared sensitive data that should not appear in replication packages.
Show threadLukas RöselerMar 2
The longest part: Debugging. Sometimes, there are additional requirements that are not part of the readme (folder structure, paths, variable renaming, ...).
Show threadLukas RöselerMar 2
- If nothing works: Send what you have to the authors and maybe they can help you out.
- If everything works: You decide if there is a meaningful difference between original and reproduction/replication. Are there implications for the study's conclusions?
Show threadLukas RöselerMar 2

- Team up with others who are familiar with the software or with the method
- Having multiple experienced programmers can be very helpful
- If you are looking for a team, participate in the Institute for Replication
s #replicationgames

Slides: https://doi.org/10.5281/zenodo.18823102

Slides for Reproducibility Speedrun

Zenodo
Show threadLukas RöselerMar 2
Today's final talk will be by Cassie Short from IGOR and @GermanRepro on multiverse analyses.
Show threadLukas RöselerMar 2
During analyses, we face multiple decisions. The common approach is to make the decisions and report a single pipeline, leaving us aware of outcomes that we would have got with a different pipeline.
Show threadLukas RöselerMar 2
Multiverse analysis/specification curve/vibration of effects/snseitivity analysis/manyverse analysis uses every sensible pipeline and runs them. The result is usually a specification curve. The bottom part shows the values for the included factors.
Show threadLukas RöselerMar 2
Here are a few helpful resources for conducting multiverse analysis:
- A multi-disciplinary guide to multiverse analysis: osf.io/4yzeh_v1
- an R packaage: https://mverseanalysis.github.io/mverse/index.html
Tidy Multiverse Analysis Made Simple

Extends multiverse package (Sarma A., Kale A., Moon M., Taback N., Chevalier F., Hullman J., Kay M., 2021) <doi:10.31219/osf.io/yfbwm>, which allows users perform to create explorable multiverse analysis in R. This extension provides an additional level of abstraction to the multiverse package with the aim of creating user friendly syntax to researchers, educators, and students in statistics. The mverse syntax is designed to allow piping and takes hints from the tidyverse grammar. The package allows users to define and inspect multiverse analysis using familiar syntax in R.

Show threadLukas RöselerMar 2
The Systematic Multiverse Analysis Registration Tool (SMART) helps you decide, register, document, and share your analysis. For "defensible multiverse" (1.0) and data-driven multiverse analysis (2.0): https://www.apps.meta-rep.lmu.de/SMART/
Show threadLukas RöselerMar 2
That's a wrap for today. See you tomorrow for the continuation of #LoveReplicationsWeek!