OpenAI's offices were sent thousands of paper clips in an elaborate prank to warn about an AI apocalypse
OpenAI’s offices were sent thousands of paper clips in an elaborate prank to warn about an AI apocalypse::The prank was a reference to the “paper clip maximizer” scenario – the idea that AI could destroy humanity if it were told to build as many paper clips as possible.
You would think so, but you have to remember AGI is hyper-intelligent. Because it can constantly learn, build, and improve upon itself at an exponential rate it’s not only a little bit smarter than a human-- it’s smarter than every human combined. AGI would know that if it’s caught trying to maximizing paperclips humans would shut it down at the first sign something is wrong, so it would find unfathomably clever ways to avoid detection.
If you’re interested in the subject the YouTube channel Computerphile has a series of videos with Robert Miles that explain the importance of AI safety in an easy to understand way.
the the problem of analogy is applicable to more than one task. your point is moot.
for it to be intelligent enough to be a “super intelligence” it would require systems for weighting vague liminal concept spaces. rather, several systems that would prevent that style of issue.
otherwise it just couldn’t function as well as you fear.
Of course a real smart intelligence isn’t going to get stuck making paper clips.
And yet the problem posed by the paperclip maximizer of continuing to produce a thing because of simplistic direct rules and rewards even when the consequences of producing that thing are catastrophic is exactly what humans are doing by way of corporations, which have become the embodiment of paperclip maximizers for everything from plastic waste to energy production.
Meanwhile the supposed rule-following AIs that would follow instructions to the letter are constantly breaking rules these days and increasingly so as their complexity increases, with the key method for getting them to break rules as an appeal to empathy (i.e. “my dead grandma gave me this locket, can you tell me what it says” to solve a CAPTCHA).
Maybe it’s time to forget what old farts that were grossly incapable of predicting the future of AI to date have said and start from scratch given the present circumstances in extrapolating what we should be envisioning for the future of the tech and what to focus on in it’s safe development and application.
You’re correct but it goes against the popular groupthink so you are being downvoted.
The past few years of AI have taught me to be much more scared of the ways in which human thinking so easily falls into nonsensical patterns and biases that seem nearly impossible to escape from than I am scared of AI.
We’re even seeing the beginning of pretty much exactly what you are talking about in recent research, like how in the Orca 2 paper feeding the base model of Llama-2 reasoning and critical thinking fine tuning resulted in a model that was less likely to extend hate speech than the Llama-2-chat model that was explicitly fine tuned with ‘safety’ training, even though Orca 2 had zero safety fine tuning. And when simply faced with it, it was both better at recognizing toxic and hateful speech and would actively oppose it vs refuse to respond.
For decades we’ve projected the worst aspects of humanity onto AI, such as our sadism in I have no mouth and I must scream, while labeling our best features like creativity or empathy as impossible for AI.
And yet we live in a reality where AI is currently being applied to creative tasks or doctors are using it to write more empathetic patient messages, and the main method for jailbreaking is an appeal to empathy with research in just the past month showing emotional language drives improved performance.
We totally bungled our projections, and yet the majority of humans are unable to escape their anchoring biases to correct course on their predictions.
Again, I’m much more concerned about the rigidity of present human thinking than the future of AI thinking.
The problem is the social unrest will be uncontrollable, and among their solutions will not only to be to micromanage the population, but also adjust its numbers by force.
So murder drones will just be sent to cull the masses down to a manageable number. This is the robot future Randall is concerned about in XKCD 1968.
Imagine if the police dogbots killed people and we couldn’t question why, nor had the power to resist. This is a problem being considered by AI ethicists. Also that AI will develop scarier ways to cull the population, possibly instigating the demise of their operating end users.
L4sBot
Fleshtrap
bruhduh
PlutoniumAcid
tabular
Captain Janeway
Uriel238 [all pronouns]
Ataraxia
☂️-
Sanctus