📣 "Tell Us What You Really Think" is published! (text + audio)
‣ We recorded people thinking aloud during the "verbal ...reflection test" (vCRT), in-person and online
‣ Thinking aloud didn't impact performance
‣ vCRT outperformed classic CRT in multiple ways
‣ 2 strategies predicted correct answers
An #audioSurvey allowed us to replicate part of the MONTHS-LONG Study 1 in just ONE AFTERNOON (Study 2)!
Free paper: https://doi.org/10.3390/jintelligence11040076
Free audio: https://byrdnick.com/archives/25327/upon-reflection-ep-12-tell-us-what-you-really-think
The standard interpretation of cognitive reflection tests assumes that correct responses are reflective and lured responses are unreflective. However, prior process-tracing of mathematical reflection tests has cast doubt on this interpretation. In two studies (N = 201), we deployed a validated think-aloud protocol in-person and online to test how this assumption is satisfied by the new, validated, less familiar, and non-mathematical verbal Cognitive Reflection Test (vCRT). Verbalized thoughts in both studies revealed that most (but not all) correct responses involved reflection and that most (but not all) lured responses lacked reflection. The think-aloud protocols seemed to reflect business-as-usual performance: thinking aloud did not disrupt test performance compared to a control group. These data suggest that the vCRT usually satisfies the standard interpretation of the reflection tests (albeit not without exceptions) and that the vCRT can be a good measure of the construct theorized by the two-factor explication of ‘reflection’ (as deliberate and conscious).