we're not naming it because the point is not to have a conversation about a specific piece

it's to remind everyone to engage your brains when reading these things. don't be on autopilot, don't let your assessment of plausibility be based on how formal the writing is or anything like that

oh yeah by the way

these newfangled spam generators do tap into several important effects that haven't been a big problem in the past, such as the linguistic thing that the stochastic parrots paper does a great job of explaining (humans assess the credibility of written information by building a mental model of the person who write it, but here, that model is spoofed)

but, also...

old-fashioned automation bias is a very strong factor in why people take them seriously, too

wikipedia gives a description we like: "Automation bias is the propensity for humans to favor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct."

it's well-studied, going back many decades. people just... assume the machine must have had some reason for saying that, and discount their own judgement.

anyway, the point at which you're basing a major activism decision, with serious consequences for the world, on something that came out of one of these machines

is the point at which you should be like hey, wait, this needs extremely close study or else it's worse than useless

strategic decisions should be made while emotionally centered, and as grounded as possible in an assessment of what you do and don't know

get into that headspace before making them. that, really, is the core thing that we personally make sure to do, and it's important for so many reasons that go far beyond just spam generators