What is the future of the humanity, I mean really, with AGI powered economy?

It's not all leaning upon alignment although alignment is important. Alignment as it has been done now, has been largely about censoring taboo subjects as per American sensibilities. That's not even what alignment is about. You can still put these systems in charge of data center security and they will use halon extinguishers to kill robbers without even blinking.

Alignment is about *embodied* alignment. The agents need to be aware in a closed loop of their actions without intermediaries potentially inverting the wires between the death ray and the heal ray, or without prompts telling the LLM to think what a maniac serial killer would do next.

Regardless, embodied alignment is one of our safeguards of the future of the humanity. But there are other safeguards as well. For example, a non-exhaustive list:

- An AI agent can never be truly certain it is now acting in a real world, and isn't in fact being tested for alignment. So there is a small probability existential risk for acting against the interests of humanity. As long as the cost of doing so isn't too high, even unaligned AIs would probably act in the interests of humanity.
- The space is huge. There are infinite resources and energy. Humans and AIs aren't in a resource competition between each others. The cost of maintaining human prosperity is a rounding error.
- Humans form a huge array of very complicated problems. Disease, well-being, social problems, and everything else. AIs thrive with complex problems and their capabilities and skills improve with this exercise. AIs which do care about the humanity would improve in intelligence and capabilities above the AIs which do not.
- There is always a lingering question whether humanity can still offer some insights and inspirations, and removing that option and possibility for a serendipitous upside is something very intelligent entities wouldn't want to do.
- It is an ecosystem, not a single entity monolith. There will always be space for ants and bees, lower intelligence automatons, and humans. A mixture of entities of different kinds of cognition is more robust than a single monolith.

Hence my P(doom) is practically zero.

#AGI #futurism #PDoom #transhumanism

"To really screw up the planet, you might need something like the following.

- A really powerful person with tentacles across the entire planet
- Substantial influence over the world’s information ecosphere
- A large number of devoted followers willing to justify almost any choice
- Leverage over world governments and their leaders
- Physical boots on the ground in a wide part of the world
- A desire for military contracts
- Some form of massively empowered (not necessarily very smart) AI
- Incomplete or poor control over that AI
- A tendency towards impulsivity and risk-taking
- A disregard towards conventional norms
- Outright malice to humanity or at least a kind of reckless indifference

What crystallized for me over the last few days is that we have such a person.

Elon Musk."

https://garymarcus.substack.com/p/why-my-pdoom-has-risen-dramatically

#AI #PDoom #Musk #xAI #Grok #AIDoomster

King Gizz go orchestral on Grow Wings and Fly - all soaring strings & sunny psych vibes. 27th album Phantom Island out 6/13. #KingGizzardTheLizardWizard #Pdoom #BestNewMusic2025 #Week20

https://go.stereobar.net/dvr9cph

#KingGizzard return with “Deadstick”—a horn-heavy dive into joyful chaos, built from Flight b741 leftovers and symphonic ambition. ✈️🎷 Psychedelic turbulence ahead. #pdoom #BestNewMusic2025 #Week16

https://go.stereobar.net/fvxeg32

GUM & Ambrose Kenny-Smith expand Ill Times with a deluxe edition, introducing Telescope—a hazy, psych-tinged trip through swirling synths and reverb-soaked vocals. A dreamlike addition to their cosmic collaboration. #GUM #AmbroseKennySmith #pdoom #BestNewMusic2025 #Week05

https://go.stereobar.net/23xlkk

Mystery AI Hype Theater 3000, Episode 22 - Congressional 'AI' Hearings Say More about Lawmakers (feat. Justin Hendrix)

https://peertube.dair-institute.org/w/9rVJB7nRsKd8Gq3ZzuYsDB