Tom MorrisDec 5, 2022

Just tried ChatGPT. I asked it a series of specific Qs about areas I've studied in detail.

On all Qs, it gave answers that are plausible sounding but wrong. Not obviously wrong: wrong in subtle ways that need deep domain knowledge to grasp.

The ways humans will be practically misled by this kind of tech if trusted with, say, doling out medical, legal or business advice is horrific.

Letting this tech loose on the world will further destroy search engines that are already riddled with SEO BS.

Show threadTom MorrisDec 5, 2022

These kind of technologies are a natural response to content-as-commodity—SEO, content marketing, the YouTube algorithm, influencer culture etc.

It doesn't matter that the content is bullshit, X units of content are needed, humans are expensive, let's have a machine that churns out content. Doesn't have to be true, doesn't need to have passed through an actual brain or have had any connection with reality.

"Thanks, I hate it."

Show threadMoto Dec 5, 2022

@tommorris Great. So, to review: We humans have invented machines capable of dispensing ersatz “information”. And this ersatz is *least* distinguishable as such, by the people *most* in need of authentic information.

This is a particularly worrisome development amid the ongoing Disinformation Pandemic.

We need better tools for tracing the origins of the information we consume.

Show threadCassandrichDec 5, 2022
@Cmdrmoto @tommorris We really need to be notarizing all existing human knowledge bases so that at least "existed before 2022" can be evaluated mechanically and used as a shortcut for knowing you're only dealing with human-generated or very-low-convincingness bullshit.
Show threadMoto Dec 5, 2022
@dalias @tommorris Agreed that “attested by” signatures on useful information repositories - where those attestations could be cryptographically verified as belonging to a decentralized identifier - would be invaluable.
Show threadCassandrich
@Cmdrmoto @tommorris That's different and also valuable but I'm talking just about cryptographic proof of existence-at-time through a notary Merkle tree.
Dec 5, 2022 at 3:41pmWeb
Show threadMoto Dec 5, 2022
@dalias @tommorris True, a “pre-singularity” time stamp could help defend against this form of noise. It’s less certain to help me pick out high-quality information, but it’s at least a start
Show threadMoto Dec 5, 2022
@dalias @tommorris and yes. I am using Singularity to describe this event. I consider the invention of a perfect bullshit machine to be more than I was ready for.
Show threadMoto Dec 5, 2022

@dalias @tommorris I’ve been fretting about infohazards since the 1990s. I honestly believed I had thought of every plausible disinformation scenario, and understood the landscape.

But “computer that can effectively bullshit most humans on most topics” was *not* on my Bingo card.

Show threadMoto Dec 5, 2022

@dalias @tommorris so uhh … @internetarchive ?

Is this a valid feature request?