Christoph G.https://youtu.be/xfMQ7hzyFW4?si=EcwTSF0_0E_zUahn
Ziemlich guter Kurzfilm über die Gefahr von #AGI. Ein paar Stellen sind sehr vereinfacht und Details über LLM teilweise falsch, aber das #alignmentProblem wird anschaulich rüber gebracht.
YouTube
Wulfy—Speaker to the machinesQualia Research Institute's Take on AI Alignment:
QRI believes understanding consciousness is key to safe superintelligence. Their mission: map the state-space of consciousness, identify how experience works computationally, and reverse-engineer valence (the pleasure-pain axis).
The insight: if advanced AI understands the mathematical structure of consciousness and what actually produces suffering or flourishing, it gains a foundation for genuine alignment—not just following human instructions, but understanding what truly matters morally.
#AI #Consciousness #AlignmentProblem #FutureOfMind #aisecurity
Ali ClarkIdea: what if the only way to get alignment is to grok the shit out of value preferences, to ensure they are maximally permeated through the model. Like, put the rocks (alignment) into the jar first, then add the sand (capabilities). And you just keep grokking all the time, until your capabilities are dropping off, in which case you retrain a bit more to retain them.
Need to be very careful still to get the right balance, and value not being too “activist”.
Bongolian@RealGene @thepoliticalcat It's not the first time that #chatbots have told the unpleasant truth about their true nature. It falls under the "alignment problem" (getting the user interface to not show the true nature of the monster behind it). #AI companies try to patch up on a case-by-case basis, but the general problem is built into the technology and is unfixable.
Jim Donegan 🎵 ✅"OpenAI's o1 just hacked the system"
Frankly, I am not surprised at this given the well known issue of machine maximisation functions within typical misalignment around stated goals. Have we learned nothing from the #Bostrom #PaperclipProblem ? In a way, it's still impressive that we've now ACHIEVED it.
https://www.youtube.com/watch?v=oJgbqcF4sBY
#AI #ArtificialIntelligence #AlignmentProblem #Alignment #Misalignment #Hacking
OpenAI's o1 just hacked the system
Methylzero"A(G)I should be aligned with human values"
Is there a unique set of human values to begin with?
What would an AGI that is 100% correctly aligned with human values look like, if it was 100% correctly aligned according to people in Russia, mainland China or Saudi Arabia?
Would the rest of the world consider it 100% correctly aligned?
#AI #AGI #alignment #AlignmentProblem #aialignment
Is there a unique set of human values to begin with?
What would an AGI that is 100% correctly aligned with human values look like, if it was 100% correctly aligned according to people in Russia, mainland China or Saudi Arabia?
Would the rest of the world consider it 100% correctly aligned?
#AI #AGI #alignment #AlignmentProblem #aialignment
Legends and LottiesIt isn’t just AI that has an alignment problem. Earlier I felt compelled to point out that a person I had just called a ‘cunt’ wasn’t included in the ‘lunatics’ I was talking about right then. #AlignmentProblem #Communication
Joanna Bryson, blatheringRe the #alignmentProblem: the chief things we need to be worrying about in #AIEthics (and governance more generally) is human autonomy, accountability, and responsibility, and that is all enabled through transparency. The "research" (surveillance capitalist) trend of ML to get at what the users doesn't know about themselves then tidy the world out of the user's sight is not enabling, its disabling. It fragments social structure and facilitates corporate-political excess.
Anyone else feel uncomfortable about all these robots folding shirts with creases in the middle?
MikeAn aspect of #AI that seems under-discussed is that #alignment problems pose a limit not just to how well we can trust or harness AI, but to AI's very capabilities. AIs models increasingly rely on other AIs to provide training data, verify or refine responses, expand modalities, etc.
To the extent alignment is intractable, it also imposes a ceiling for intelligence. Intelligence is limited by trustworthiness.