Jascha Sohl-Dickstein

1.2K Followers
140 Following
27 Posts
Senior staff research scientist, Google Brain. Inventor of diffusion models. Machine learning $\otimes$ physics $\otimes$ neuroscience.
Bloghttps://sohl-dickstein.github.io/
Websitehttps://sohldickstein.com

Paper here:
https://arxiv.org/abs/2311.02462

(Imagen is a Level 3 "Expert" Narrow AI)

Levels of AGI for Operationalizing Progress on the Path to AGI

We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI performance, generality, and autonomy, providing a common language to compare models, assess risks, and measure progress along the path to AGI. To develop our framework, we analyze existing definitions of AGI, and distill six principles that a useful ontology for AGI should satisfy. With these principles in mind, we propose "Levels of AGI" based on depth (performance) and breadth (generality) of capabilities, and reflect on how current systems fit into this ontology. We discuss the challenging requirements for future benchmarks that quantify the behavior and capabilities of AGI models against these levels. Finally, we discuss how these levels of AGI interact with deployment considerations such as autonomy and risk, and emphasize the importance of carefully selecting Human-AI Interaction paradigms for responsible and safe deployment of highly capable AI systems.

arXiv.org

These Levels of AGI provide a rough framework to quantify the performance, generality, and autonomy of AGI models and their precursors. We hope they help compare models, assess risk, and measure progress along the path to AGI.

(AlphaGo is a Level 4 "Virtuouso" Narrow AI)

Levels of AGI: Operationalizing Progress on the Path to AGI

Levels of Autonomous Driving are extremely useful, for communicating capabilities, setting regulation, and defining goals in self driving.

We propose analogous Levels of *AGI*.

(ChatGPT is a Level 1 "Emerging" AGI)

AI can enable awesome (as in inspiring of awe) good in the world. We have amazing leverage as the people building it. We should use that leverage carefully.
A theme is that we should worry about a *diversity* of risks. If we recognize e.g. only specific present harms, or only AGI misalignment risk, we will find our efforts overwhelmed by other types of AI-enabled disruption, and we won't be able to fix the problem we care about. And if we don't believe in a specific risk -- that's ok, there are plenty of others we can focus on!
My top fears include targeted manipulation of humans, autonomous weapons, massive job loss, AI-enabled surveillance and subjugation, widespread failure of societal mechanisms, extreme concentration of power, and loss of human control.
AI has the power to change the world in both wonderful and terrible ways. If we exercise care, the wonderful outcomes will be much more likely than the terrible ones. Towards that end, here is a brain dump of my thoughts about how AI might go wrong.
https://sohl-dickstein.github.io/2023/09/10/diversity-ai-risk.html
Brain dump on the diversity of AI risk

This blog is intended to be a place to share ideas and results that are too weird, incomplete, or off-topic to turn into an academic paper, but that I think may be important. Let me know what you think! Contact links to the left.

Jascha’s blog

See the blog post for more details -- including discussion of many ways these results are inconclusive and could be improved!
https://sohl-dickstein.github.io/2023/03/09/coherence.html

PS -- This is a Mastodon exclusive! I may repost on Twitter sometime tomorrow.

Living creatures, human organizations, and machine learning models are all judged to become more of a hot mess (less coherent) as they grow more intelligent. This suggests that AI failing to pursue a consistent goal is more likely than AI pursuing a misaligned goal.
Most work on AI misalignment risk is based on an assumption that more intelligent AI will also be more coherent. This is an assumption we can test! I collected subjective judgements of intelligence and coherence from colleagues in machine learning and neuroscience.