Michael FrankNov 16, 2022

Developmentalists: have you been wondering if online experiments with young children are comparable to in-lab studies? Here's our new meta-analysis suggesting that overall they are!

https://psyarxiv.com/qc6fw - work by Aaron Chuey, @[email protected] and @[email protected]

Show threadjepustoNov 17, 2022
@mcxfrank looks really interesting. Have y’all looked at how size of method effects (online - lab) might vary by magnitude of primary effect?
Show threadMichael FrankNov 17, 2022
@jepusto great point. We haven’t… for better or worse all the effects we looked at were positive. Some had magnitudes that weren’t obviously interpretable and wouldn’t have been included in a standard MA (eg a single measure of some phenomenon) but we converted them to ES based on comparison with norms so that we could compare the difference in this meta regression… that might complicate things…
Show threadjepusto
@mcxfrank yeah that seems like it would complicate things (but perhaps could look at the primary ES for a subset of studies where this was less of an issue?). The reason I ask is that there’s evidence that ES heterogeneity increases with average ES magnitude, so I wonder whether these sort of method/platform could be more apparent with stronger primary effects.
Nov 17, 2022 at 4:54pmWeb