a lot of research on the efficacy and cognitive effects of generative language models is just, uh, bad science https://www.404media.co/nature-retracts-paper-on-the-benefits-of-chatgpt-in-education/

from what we've seen (we try to dig into these things with our friends when we have time), that's true kind of regardless of whether the authors think they're making a statement for or against the things. lots of statistical sloppiness and conclusions that the methodology can't support

'Nature' Retracts Paper on the Benefits of ChatGPT in Education

“What educators, parents and policy officials really needed was high quality data and evidence to help guide them. What they have had to deal with instead is some substandard research.”

404 Media

@mos_8502 oh, we're in full agreement. it's really important. making stuff up to reach a correct conclusion is still a wrong path.

the world already has plenty of social patterns that have grown around the practice of doing right things for wrong reasons. if the world is ever to heal properly, we need to rip that out from the root. there is no alternative.

@mos_8502 because... if people and our institutions are in the habit of doing things for wrong reasons

then there is no process difference between doing right things and doing wrong things, and the latter can happen more easily