Max KennerlyJul 22, 2023
OpenAI talks about ensuring AI is "aligned with human values" but if the CEO doesn't understand there are downsides to building a nuclear bomb and didn't even notice Mark Zuckerberg was the villain in The Social Network then it's unlikely the company is aligned with human values.
Show threadNoah Callaway
@maxkennerly this is why AI alignment is impossible. Humans don’t agree on values, so there’s no actual agreement on what to align *to*.
Jul 23, 2023 at 12:00amWeb
Show threadMatunosJul 23, 2023
@ncallaway @maxkennerly there's a whole sci-fi subgenre devoted to the inherent flaw in "aligning" AI with human values
Show threadWendy (prev Jason) AJul 23, 2023
@matunos @ncallaway @maxkennerly I think this is a trap. My pet theory is that human values align much more than we think. And that most of the unaligned portions are due to conservatives intentionally injecting fake morality into the conversation.
Show threadMatunosJul 23, 2023
@norgralin @ncallaway @maxkennerly demagogues are human too
Show threadNoah CallawayJul 23, 2023

@norgralin @matunos @maxkennerly okay, but this becomes a nuclear arms control style problem. As long as nukes are only ever held by rational actors, maybe we won’t all die to nuclear weapons.

But technology has a way of becoming easier to access over time.

Even if we “solve” AI alignment generally, you have the problem of AI being designed by those exact “unaligned humans” that you’re describing (who are, by the way, mostly the people who hold capital in the current system)

Show threadNoah CallawayJul 23, 2023
@norgralin @matunos @maxkennerly So, like, maybe we can theoretically get AI aligned with a common set of human values (though, I think one of the few values we could agree on is “don’t kill ALL humans”, I’m pretty sure “don’t kill humans” wouldn’t even be one of those broadly shared values)
Show threadNoah CallawayJul 23, 2023
@norgralin @matunos @maxkennerly but even then, we have to keep it out of the hands that are best equipped to get their hands on AI. The kind of hands who *currently* have more control over AI than most AI alignment and safety researchers.
Show threadWendy (prev Jason) AJul 23, 2023
@ncallaway @matunos @maxkennerly I’m not arguing for AI. My view is that Sam Altman would misconstrue human values to get the AI equivalent of a toxic libertarian. And then try to tell us it’s moral. Last I looked OpenAI’s definition of AI was inherently exploitative.
Show threadNoah CallawayJul 23, 2023

@norgralin @matunos @maxkennerly I think we’re in agreement, then.

My point was mostly that AI alignment is not a solvable problem, because Sam Altman will say an AI is “aligned” when I would say it isn’t, and when I would say an AI is aligned, Sam Altman would say it isn’t.

Show threadWendy (prev Jason) AJul 23, 2023
@ncallaway @matunos @maxkennerly we are mostly in agreement. My view is basically that the Sam Altmans are probably lying to us and themselves about human values. So we should regard their input as suspicious.