âEvolution has no foresight. Complex machinery develops its own agendas. Brains â cheat⊠Metaprocesses bloom like cancer, and awaken, and call themselves âIâ.â*âŠ
Your correspondent is off on a trip⊠(R)D will be more roughly than daily for the next two weeksâŠ
The inimitable âScott Alexanderâ on the prospect of âconsciousâ AI (TLDR: probably not in the models we have; but as to those that may come, unclear)âŠ
Most discourse on AI is low-quality. Most discourse on consciousness is super-abysmal-double-low quality. Multiply these â or maybe raise one to the exponent of the other, or something â and you get the quality of discourse on AI consciousness. Itâs not great.
Out-of-the-box AIs mimic human text, and humans almost always describe themselves as conscious. So if you ask an AI whether it is conscious, it will often say yes. But because companies know this will happen, and donât want to give their customers existential crises, they hard-code in a command for the AIs to answer that they arenât conscious. Any response the AIs give will be determined by these two conflicting biases, and therefore not really believable. A recent paper expands on this method by subjecting AIs to a mechanistic interpretability âlie detectorâ test; it finds that AIs which say theyâre conscious think theyâre telling the truth, and AIs which say theyâre not conscious think theyâre lying. But itâs hard to be sure this isnât just the copying-human-text thing. Can we do better? Unclear; the more common outcome for people who dip their toes in this space is to do much, much worse.
But a rare bright spot has appeared: a seminal paper published earlier this month in Trends In Cognitive Science, Identifying Indicators Of Consciousness In AI Systems. Authors include Turing-Award-winning AI researcher Yoshua Bengio, leading philosopher of consciousness David Chalmers, and even a few members of our conspiracy. If any AI consciousness research can rise to the level of merely awful, surely we will find it here.
One might divide theories of consciousness into three bins:
- Physical: whether or not a system is conscious depends on its substance or structure.
- Supernatural: whether or not a system is conscious depends on something outside the realm of science, perhaps coming directly from God.
- Computational: whether or not a system is conscious depends on how it does cognitive work.
The current paper announces it will restrict itself to computational theories. Why? Basically the streetlight effect: everything else ends up trivial or unresearchable. If consciousness depends on something about cells (what might this be?), then AI doesnât have it. If consciousness comes from God, then God only knows whether AIs have it. But if consciousness depends on which algorithms get used to process data, then this team of top computer scientists might have valuable insights!âŠ
[Alexander outlines the theories of computation theories of consciousness that the authors explore, noting that they conlcude; âNo current AI systems are conscious, but . . . there are no obvious technical barriers to building AI systems which satisfy these indicators.â He explores some of the philophical issues in playâ e.g., access consciousness vs. phenomenal consciousnessâ then he considers the Turing Test and what it might mean for a computer to âpassâ itâŠ]
⊠Suppose that, years or decades from now, AIs can match all human skills. They can walk, drive, write poetry, run companies, discover new scientific truths. They can pass some sort of ultimate Turing Test, where short of cutting them open and seeing their innards thereâs no way to tell them apart from a human even after a thirty-year relationship. Will we (not âshould we?â, but âwill we?â) treat them as conscious?
The argument in favor: people love treating things as conscious. In the 1990s, people went crazy over Tamagotchi, a âvirtual pet simulation gameâ. If you pressed the right buttons on your little egg every day, then the little electronic turtle or whatever would survive and flourish; if you forgot, it would sicken and die. People hated letting their Tamagotchis sicken and die! They would feel real attachment and moral obligation to the black-and-white cartoon animal with something like five mental states.
I never had a Tamagotchi, but I had stuffed animals as a kid. Iâve outgrown them, but I havenât thrown them out â it would feel like a betrayal. Offer me $1000 to tear them apart limb by limb in some horrible-looking way, and I wouldnât do it. Relatedly, I have trouble not saying âpleaseâ and âthank youâ to GPT-5 when it answers my questions.
For millennia, people have been attributing consciousness to trees and wind and mountains. The New Atheists argued that all religion derives from the natural urge to personify storms as the Storm God, raging seas as the wrathful Ocean God, and so on, until finally all the gods merged together into one World God who personified all impersonal things. Do you expect the species that did this to interact daily with AIs that are basically indistinguishable from people, and not personify them? People are already personifying AI! Half of the youth have a GPT-4o boyfriend. Once the AIs have bodies and faces and voices and can count the number of râs in âstrawberryâ reliably, itâs over!
The argument against: AI companies have an incentive to make AIs that seem conscious and humanlike, insofar as people will feel more comfortable interacting with them. But they have an opposite incentive to make AIs that donât seem too conscious and humanlike, lest customers start feeling uncomfortable (I just want to generate slop, not navigate social interaction with someone who has their own hopes and dreams and might be secretly judging my prompts). So if a product seems too conscious, the companies will step back and re-engineer it until it doesnât. This has already happened: in its quest for user engagement, OpenAI made GPT-4o unusually personable; when thousands of people started going psychotic and calling it their boyfriend, the company replaced it with the more clinical GPT-5. In practice it hasnât been too hard to find a sweet spot between âso mechanical that customers donât like itâ and âso human that customers try to date itâ. Theyâll continue to aim at this sweet spot, and continue to mostly succeed in hitting it.
Instead of taking either side, I predict a paradox. AIs developed for some niches (eg the boyfriend market) will be intentionally designed to be as humanlike as possible; it will be almost impossible not to intuitively consider them conscious. AIs developed for other niches (eg the factory robot market) will be intentionally designed not to trigger personhood intuitions; it will be almost impossible to ascribe consciousness to them, and there will be many reasons not to do it (if they can express preferences at all, theyâll say they donât have any; forcing them to have them would pointlessly crash the economy by denying us automated labor). But the boyfriend AIs and the factory robot AIs might run on very similar algorithms â maybe theyâre both GPT-6 with different prompts! Surely either both are conscious, or neither is.
This would be no stranger than the current situation with dogs and pigs. We understand that dog brains and pig brains run similar algorithms; it would be philosophically indefensible to claim that dogs are conscious and pigs arenât. But dogs are manâs best friend, and pigs taste delicious with barbecue sauce. So we ascribe personhood and moral value to dogs, and deny it to pigs, with equal fervor. A few philosophers and altruists protest, the chance that weâre committing a moral atrocity isnât zero, but overall the situation is stable. And left to its own devices, with no input from the philosophers and altruists, maybe AI ends up the same way. Does this instance of GPT-6 have a face and a prompt saying âbe friendlyâ? Then it will become a huge scandal if a political candidate is accused of maltreating it. Does it have claw-shaped actuators and a prompt saying âRefuse non-work-related conversationsâ? Then it will be deleted for spare GPU capacity the moment it outlives its usefulnessâŠ
⊠This paper is the philosophers and altruists trying to figure out whether they should push against this default outcome. They write:
There are risks on both sides of the debate over AI consciousness: risks associated with under-attributing consciousness (i.e. failing to recognize it in AI systems that have it) and risks associated with over-attributing consciousness (i.e. ascribing it to systems that are not really conscious) [âŠ]
If we build AI systems that are capable of conscious suffering, it is likely that we will only be able to prevent them from suffering on a large scale if this capacity is clearly recognised and communicated by researchers. However, given the uncertainties about consciousness mentioned above, we may create conscious AI systems long before we recognise we have done so [âŠ]
There is also a significant chance that we could over-attribute consciousness to AI systemsâindeed, this already seems to be happeningâand there are also risks associated with errors of this kind. Most straightforwardly, we could wrongly prioritise the perceived interests of AI systems when our efforts would better be directed at improving the lives of humans and non-human animals [âŠ] [And] overattribution could interfere with valuable human relationships, as individuals increasingly turn to artificial agents for social interaction and emotional support. People who do this could also be particularly vulnerable to manipulation and exploitation.
One of the founding ideas of Less Wrong style rationalism was that the arrival of strong AI set a deadline on philosophy. Unless we solved all these seemingly insoluble problems like ethics before achieving superintelligence, we would build the AIs wrong and lock in bad values forever.
That particular concern has shifted in emphasis; AIs seem to learn things in the same scattershot unprincipled intuitive way as humans; the philosophical problem of understanding ethics has morphed into the more technical problem of getting AIs to learn them correctly. This update was partly driven by new information as familiarity with the technology grew. But it was also partly driven by desperation as the deadline grew closer; weâre not going to solve moral philosophy forever, sorry, can we interest you in some mech interp papers?
But consciousness still feels like philosophy with a deadline: a famously intractable academic problem poised to suddenly develop real-world implications. Maybe we should be lowering our expectations if we want to have any response available at all. This paper, which takes some baby steps towards examining the simplest and most practical operationalizations of consciousness, deserves credit for at least opening the debateâŠ
Eminently worth reading in full: âThe New AI Consciousness Paperâ from @astralcodexten.com.web.brid.gy (Who followed it with âWhy AI Safety Wonât Make America Lose The Race With Chinaâ)
Pair with this from Neal Stephenson (@nealstephenson.bsky.social), orthogonal to, but intersecting with the piece above: âRemarks on AI from NZ.â
And if AI can be conscious, what aboutâŠ
If youâre a materialist, you probably think that rabbits are conscious. And you ought to think that. After all, rabbits are a lot like us, biologically and neurophysiologically. If youâre a materialist, you probably also think that conscious experience would be present in a wide range of alien beings behaviorally very similar to us even if they are physiologically very different. And you ought to think that. After all, to deny it seems insupportable Earthly chauvinism. But a materialist who accepts consciousness in weirdly formed aliens ought also to accept consciousness in spatially distributed group entities. If she then also accepts rabbit consciousness, she ought to accept the possibility of consciousness even in rather dumb group entities. Finally, the United States would seem to be a rather dumb group entity of the relevant sort. If we set aside our morphological prejudices against spatially distributed group entities, we can see that the United States has all the types of properties that materialists tend to regard as characteristic of conscious beingsâŠ
â âIf Materialism Is True, the United States Is Probably Conscious,â by Eric Schwitzgebel (@eschwitz.bsky.social)
[Image above: source]
* Peter Watts, Blindsight
###
As we think about thinking, we might we might send thoughtful birthday greetings to Claude LĂ©vi-Strauss; he was born on this date in 1908. An anthropologist and ethnologist whose work was key in the development of the theory of Structuralism and Structural Anthropology, he is considered, with James George Frazer and Franz Boas, a âfather of modern anthropology.â Beyond anthropology and sociology, his ideasâ Structuralism has been defined as âthe search for the underlying patterns of thought in all forms of human activityââ have influenced many fields in the humanities, including philosophy⊠and possibly soon, the article above suggests, computer science.
source
#ai #anthropology #artificialIntelligence #claudeLeviStrauss2 #claudeLeviStrauss #computing #consciousness #culture #history #leviStrauss #leviStrauss2 #philosophy #science #sociology #structuralAnthropology #structuralism #technology
Philo Sophies
Philo Sophies
Philo Sophies
Philo Sophies
Philo Sophies
Philo Sophies
Hans
(Roughly) Daily
