If you want to understand the whole OpenAI thing, this article is super helpful because it explains that OpenAI is not a real research organization or tech company like the ones we know: It's a cult. It's basically Scientology 2.0.

Quote:
"The consequences are what you might expect when a crowd of bright but rather naive (and occasionally creepy) computer science and adjacent people try to re-invent theology from first principles, to model what human-created gods might do, and how they ought be constrained. They include the following, non-comprehensive list: all sorts of strange mental exercises, postulated superhuman entities benign and malign and how to think about them; the jumbling of parts from fan-fiction, computer science, home-brewed philosophy and ARGs to create grotesque and interesting intellectual chimeras; Nick Bostrom, and a crew of very well funded philosophers; Effective Altruism, whose fancier adherents often prefer not to acknowledge the approach’s somewhat disreputable origins."

https://www.programmablemutter.com/p/look-at-scientology-to-understand

What OpenAI shares with Scientology

Strange beliefs, fights over money and bad science fiction

Programmable Mutter
@tante what is meant there with "the [Effective Altruism] approach’s somewhat disreputable origins"? The funding by Sam Bankman-Fried? The "business man in a suit in front of a drowning child" debate? Or something else? #effectiveAltruism

@maybit EA's history is full of shady people, eugenics, racism etc.

If you want to read a good criticism of the thoughts underpinning the ideology check out https://www.theintrinsicperspective.com/p/why-i-am-not-an-effective-altruist

Why I am not an effective altruist

Morality is not a market

The Intrinsic Perspective
@tante @maybit First thought: The Environment Agency? What's so bad about the Environment Agency?
Second thought: Oh you mean EA games. Yeah those gamer types sometimes have dodgy politics.
@tante @maybit He should do a follow up entitled "Why I am not a utilitarian" in which he'd discuss effective altruism for the first two thirds of the text.
@phrees that's not totally fair: EA is utilitarianism on steroids so discussing the flaws of that ideology is essential when discussing what was built on top of it.

@tante To be totally fair we have to consider EA proponents views about whether EA is utilitarianism at all.

https://80000hours.org/2020/08/misconceptions-effective-altruism/#misconception-4-effective-altruism-is-just-utilitarianism

Misconceptions about effective altruism

The core idea of effective altruism is not about any specific way of doing good. Rather, it’s about the idea that some ways of contributing to the common good are far more effective than what’s typical.

80,000 Hours
@tante To be fair, Mastodon has all of those things, too
@tante I'm reminded of alchemy and how it was shrouded in mysticism, belief structures, personal networks and goals that were all based on extrapolating from small demonstrations and effects nobody understood the potentials and limits of
@tante
Judging just the quality of the rhetoric displayed in the article's title leads one straight to the superlative.

@tante

very funny how Roko's Basilisk has been continuing to torture us every other year for more than a decade now

@kami_kadse @tante We should act now to avoid the future where predictions of Roko's Basilisk torture us.

@tante

Tagging myself as a reminder to read this tonight :)

@kingu

@tante
I figured it was something like that.
@tante This article is both too hard on the AI boosters and too kind. The technology known as AI has some technical uses that are good but are probably not world changing. Of course, the tech bros want everyone to think that they are world changing so they can hook some investors. It is very analogous to block chain/crypto foolishness.

It's the churchiness of these AI guys that's the creepiest thing about the movement. They're like the idol-makers in Isaiah 44:9-20. So far the work of their hands isn't convincingly godlike, but they need to believe it is.

More about Sam Altman and his god at the URL below.

https://futurism.com/sam-altman-imply-openai-building-god

Sam Altman Seems to Imply That OpenAI Is Building God

In a recent interview, Altman described AGI as a "magic intelligence in the sky," which sounds an awful lot like a God.

Futurism

@tante what is the argument here?

The alternative version of OpenAI is an for-profit entity that makes the most powerful AI in the world without any limits.

Not saying the article is correct or incorrect, but we do live in a world where no other entity is within a year of OpenAI when it comes to technology.

I for one hope they do think a little about safety, and keep the greedy executives at Microsoft at an arms length.

Though it is a good point, I just don't like the alternative.