Taggart 1d ago
I am begging AI researchers trying to study human impact to get very rapidly better at methodology so I don't constantly read halfway through these papers only to find some ridiculous experiment design that will throw the conclusions into the air.
Show threadIan Campbell 🏴

@mttaggart I do so love how AI "research" is comfortable with AI interpreting the results of any data collection.

such a circle jerk.

Apr 6 at 4:59pmWeb
Show threadIan Campbell 🏴1d ago
@mttaggart "we studied the effect of AI on Y. then because there's not hundreds of years of history in how to analyze results, we had an LLM do it...SCIENCE!"
Show threadTaggart 1d ago
@neurovagrant That's kind of a separate issue from what I'm describing. Good faith behavioral researchers just aren't thinking through what they're asking of participants, or what the design means. But I agree that practice sucks.