OpenAI made a system for being wrong much more efficiently, delighting politicians, propagandists and bullshitters, and scaring the living crap out of everyone with any commitment to the general idea that there are things which are true in a broadly coherent reality that's shared between human beings.

(Also, these are the same people who are worried about existential risk, in as much as that risk involves weird sci-fi like Roko's basilisk but not the actual risk of stuff like climate change.)

@tommorris 100% agree. But now, we are using this fallacy to cement it in systems that print out logic. It's fine as long as humans are involved, because it will inevitably be subjected to conflict, and thus the possibility for change.

We need legislation for this, now. We haven't even sufficiently legislated the current and the previous ear in surveillance capitalism, and now this is happening. Truly grim.

@tommorris Great. So, to review: We humans have invented machines capable of dispensing ersatz “information”. And this ersatz is *least* distinguishable as such, by the people *most* in need of authentic information.

This is a particularly worrisome development amid the ongoing Disinformation Pandemic.

We need better tools for tracing the origins of the information we consume.

@tommorris this is profound and very much true, as it is happening right now: one of top PR 'coaches' for indie musicians 'CyberPR', who I actually had some respect for, is now selling latest webinar that is coaching indie diy artists how to promote themselves online using AI tools to create content (texts and images) for their (semi-automated) stream of posts for social media.

😫

@myemuisemo One of the questions I gave it was a question about English law. The answer it gave was roughly "since the law changed in year N, all agreements of type X are covered by rule Y" when the actual answer is agreements of type X are covered by Y unless there's already a provision in the text of the agreement, in which case that text still binds the agreement.

It's close but the difference between those is practically very significant.

@myemuisemo @tommorris

> For giggles, I tried it out on a specific finance question I'm working on, and it said it can't do that analysis

I heard that it can be manipulated to still answer such questions, either by starting a new thread, or by asking it to roleplay an expert in that field

@tommorris I think current ml models only shine in areas where imprecise results can be mitigated or results are tested separately, like chaotic system controllers, or proteine folding.

One could argue with software the programmer + compiler are testing the results, but there will be much complacancy and overtrust (AI writes implementation + test, the test succeeds, the code compiles, the programmer is happy, the implementation is still wrong).