The confluence of this technology with the information ecosystem that we described in our paper from a couple of years could be an epistemic catastrophe.

https://www.pnas.org/doi/10.1073/pnas.2025764118

I’m coming to think that releasing these tools was a reckless act with the potential to generate negative externalities we have barely started to imagine.

The threat isn’t rogue superintelligence. It’s bullshit at unprecedented scale, reflected back upon itself and iteratively amplified.

I’m certainly not saying that the deployment of these systems will suddenly make it impossible to find and build upon trusted and vetted sources of information. Those aren’t going to magically disappear.

My bigger fear is that rather than making a blunder in tying their infotech empires, two automated bullshit generation. Microsoft and Google have correctly anticipated demand. My fear is that people might want what Bing and Bard are selling.

On the other hand, if I dial up the cynicism, just a little bit more, maybe it doesn’t matter much. One view is that by spewing bullshit into the information ecosystem, generative AI is poisoning the well from which it drew life and ensuring that future generations of such technologies will produce garbage.

Another is that the training set was never a pure wellspring. It was already the town cesspool—and even massive quantities of additional bullshit will barely be noticed.

@ct_bergstrom If bullshit is what’s produced by someone who doesn’t care whether they’re telling the truth or not (Harry Frankfurter),

then perhaps bullshit generators are what’s produced by someone who doesn’t care whether they cause harm or not.