Daniel LakensPilot studies are your friend. They will inform you about how participants interpret your instructions and questions, and identify where you need to adjust your analysis plan. It's very difficult to write a good preregistration without a pilot study.
Because pilot studies are often not discussed in a publication, you only learn how essential they are by seeing others do them. Sometimes students are surprised when I say 'let's just collect some data and see what happens'. They think every study they do needs to be perfect.
I need to explain that whenever we do something new, I don't know yet how to design a 'perfect' (or just pretty good) study. Thinking carefully is important, but reality too often immediately falsifies my expectations. Pilot studies are essential to learn how to study something.
Steve Lindsay@lakens Strong agree!
PepijnVemer@lakens Sounds like what my old boss always said when presented with new data. "First do a KZK regression".
The "Kip-Zonder-Kop" OLS regression (Chicken-without-a-head, running around blind).
If anything: it will show you some correlations, or links. If it's not there, you won't find it with any better methods. If there is a correlation, trust the statistical process of eliminating artifacts.
Rich Stein (he/him)@PepijnVemer @lakens
Lots of hypotheses don't really take shape until you use this "throw a lot of stuff" at the wall methodology.
Lots of hypotheses don't really take shape until you use this "throw a lot of stuff" at the wall methodology.
@lakens Also, good practice to summarize pilot work in intro to paper and make details available.
Guyinchicago@lakens Agree. We recently wrote a proposal, did some preliminary research and then totally rewrote it.
Peter Moleman@lakens But ain't it wrong to report them as preliminary results....?