But we think that, based on decades of past memory research, we controlled the main variables that determine memorability, making large unmeasured confounders unlikely. Future work could certainly improve on ours via greater understanding and better estimation of relevant confounders and an even more tightly controlled design. We think that clarifying that the goal of a lot of cog neuro work on memory is a CAUSAL question will help further this discussion of what the best design is.

3) Causal inference is, of course, always based on assumptions. The power of this approach comes from the study design. We gave every subject, to the extent possible, the exact same experience during the study period and later recall test (every pair of words was presented in the same order). There are still small differences, e.g., breaks between runs were slightly different and subjects came into the lab at different times of year which may have differentially influenced item memorability.

2) If one of the variables we adjust for was a collider, that would mean that a particular subject’s fMRI signals and their later recall status (or their causal ancestors) both cause the collider. All of the variables we adjust for are derived from OTHER subjects’ behavior (via the IRT model with the test subject held out). Given that subjects did the experiment on their own, we think it’s unlikely that there are any colliders present.

1) The current adjustment set was already sufficient to explain the SME (the relationship between the fMRI signals and memory) in our data. So any unobserved confounders would be unlikely to change that main conclusion although perhaps could explain the JOL relationship with memory.

@kordinglab Thanks so much for the question! There’s kind of three questions here related to our selection on observables approach in this study: 1) should we be concerned about unobserved confounders in this study? 2) should we be concerned about collider bias with this approach? 3) is this a reasonable approach in general? I’ll try to respond to each separately:

Thank you so much @NicoleCRust, very curious to hear your thoughts on this approach!

We are very much looking forward to hearing everyone's thoughts and feedback!

Finally, I just want to thank my coauthors Shannon Tubridy and Lila Davachi and my advisor
@todd_gureckis for helping me get this paper to the finish line and putting up with me which I dove into a causal inference rabbit hole! 9/9

We hope that this work contributes to a growing literature, from scholars like
@kordinglab,
@NicoleCRust and many others, on increased clarity and better methods for causal inference in cognitive neuroscience. 8/9

Obtaining these indicators is still a complicated scientific task and we argue in the paper that they are a potentially useful concept for both cognitive theory and translational work. 7/9