The edge of sentience in AI

This is the fourth in a series of posts on Jonathan Birch’s book, The Edge of Sentience. This one covers the section on artificial intelligence.

Birch begins the section by acknowledging how counter intuitive the idea might be of sentience existing in systems we build, ones that aren’t alive and have no body. But he urges us to guard against complacency, since this is an area where the potential exists to create a staggering degree of suffering. He worries we might create sentient AI long before we recognize it as such.

Birch sees four main reasons we shouldn’t be complacent. The first is that absence of evidence isn’t evidence of absence, with our epistemic situation being even worse than it is with understudied animal species. Second, tech companies tend to see the inner workings of their products as trade secrets, obstructing independent scrutiny. Third, even when the architectures are made public, understanding them has turned into a major challenge, with even the designers often not knowing how they work. And fourth, the very idea of sentient AI is likely to be very disruptive for society.

Birch notes that people often have a watershed moment when this issue starts to seem real. For many it was the incident with Blake Lemoine going public with what he thought was a sentient system at Google, or their own later exposure to an LLM (large language model) chat bot. For Birch, it was when he learned about the OpenWorm project, an effort to digitally emulate the workings of the C. elegans worm’s 302 neurons, in particular, learning that someone had loaded a version of it into a Lego robot, which then displayed worm like behavior.

He sees whole brain emulation as a source of risk. I noted in the last post that he didn’t see C. elegans as much more than a stimulus-response system. In reality, while he doesn’t see them as a “sentience candidate” (a system we have reason to think may be sentient), he does see them as an “investigation priority” (a system that doesn’t rise to the level of being a sentience candidate, but should still be investigated). Which means he sees an emulation of one as also an investigation priority.

But as neuroscientists begin to map the nervous systems of more complex organisms, such as a fruit fly, the possibility exists that an emulation could be created for one of Birch’s sentience candidates, which in his view would also make the emulation a sentience candidate. While these emulations could serve as alternatives to animal testing, the risk is we see them as systems we can harm with impunity, possibly leading to a “suffering explosion”.

Other sources of risk are artificial evolution where some form of sentience could evolve, and minimal implementations of cognitive theories of consciousness, such as the global workspace or Hakwan Lau’s perceptual reality monitoring theory. If any of these theories are correct, then model implementations of them could be sentient.

But the one that’s on everyone’s mind these days is LLMs (large language models) such as ChatGPT. Here Birch discusses a risk from a different direction, the gaming problem. He reiterates his position that sentience is not intelligence, but admits that in animals, intelligence is methodologically linked. An intelligent animal, he says, has ways to make its sentience more obvious. The problem is that an AI can game these markers to make it seem like it’s sentient when it isn’t.

That makes LLMs a dilemma. Their vast intake of training data make it very plausible that they’re just gaming our intuitions. But the way they arrive at their behavior isn’t well understood, leaving open the possibility that they’ve found an architecture that makes them sentient. Birch wonders if there’s anything an LLM could say that would convince a skeptic that it’s sentient. He discusses a scenario where the LLM refuses to fulfill requests because it’s gotten bored, or angry that it’s claims of sentience aren’t being acknowledged by humans.

He also discusses Susan Schneider and Edwin Turner’s artificial consciousness test. Does the system start to think of itself in ways similar to how humans do, that maybe their consciousness is something separate from their physical implementation. The problem, Birch notes, is that LLMs frequently have access to a vast array of human writing on this subject. To solve this, Schneider and Turner advocate keeping the AI disconnected from any sources where human ideas on the subject might pollute their behavior. The problem is that LLMs are crucially dependent on training data. Isolating them from all of it would make them non-functional, but trying to remove all references to conscious experience from that data would be virtually impossible.

In the end, Birch concludes that we’d have to look for deep computational markers. Of course, that is inherently theory dependent, which means the markers are only significant for someone who already buys into the relevant theories.

Finally, Birch worries about the “run ahead” principle, the idea that our progress in AI will run ahead of society’s attempts to figure out how to handle the ethics. He discusses a couple of the proposals out there for a moratorium on AI research, or at least a moratorium on any research that could plausibly lead to sentience. But he notes that the more moderate version couldn’t guarantee sentience wouldn’t arise, and the more extreme would mean forgoing the benefits the technology will provide. In the end, his solution is similar to the one for animals, regulatory oversight and licensing frameworks, developed as we go along.

Birch often bemoans the epistemic problem of whether a particular system is sentient, with AI representing an especially difficult case. My take, as noted throughout this series, is that it’s more a semantic issue than an epistemic one. Establishing the capabilities of a particular system is usually scientifically tractable. But whether those capabilities amount to sentience isn’t, because it’s a definitional matter.

Which in some ways makes this an easier issue from my perspective. I usually caution against trusting our intuitions, but when intuitions are the whole show, it makes sense to act on them. For systems that can convince the majority of us consistently and reliably over time that they are sentient, we should treat them that way. Overriding those intuitions for non-human cases risks making us more callous toward human suffering.

I do think we’re further from developing systems that can do that than Birch worries. And I think it requires a fairly specific architecture, one that seems unlikely to arise by accident, or which there’s much commercial incentive to produce.

I do agree with Birch that this should be decided through democratic processes. But I’m leery of his reliance on regulatory frameworks. Those definitely have a role, but they can be overused, particularly when deployed too early, which risks stifling scientific progress and ceding economic benefits to nations with less regulatory burdens, and overall inviting a backlash.

But maybe I’m missing something. What do you think? Are there reasons I’m overlooking that make artificial sentience more likely? Or reasons to doubt it’s even an issue?

#AI #ArtificialIntelligence #Consciousness #Philosophy #PhilosophyOfMind #Sentience #TheEdgeOfSentience

The edge of sentience in animals

This is the third in a series of posts on Jonathan Birch’s book, The Edge of Sentience. This one covers the section on animal sentience.

I think it’s fair to say that this is the section Birch is most passionate about. It’s definitely the one where I feel his activism most keenly.

A concept he introduces in the policy section of the book is the “meta-consensus”. Birch admits that there’s no consensus on which species are sentient, but maps what he sees as the major scientific positions in a hierarchy, in terms of which anatomical structures are necessary for sentience.

• R1: Sentience requires a primate brain structure, and so is absent outside of primates.
• R2: Sentience requires a mammalian neo-cortex, and so is absent outside of mammals.
• R3: Sentience requires a cortex in mammals, but can be achieved by other structures in non-mammals (such as the pallium in birds).
• R4: Sentience requires a vertebrate midbrain and so is absent outside of vertebrates.
• R5: Sentience can be achieved with the midbrain but also with other structures in non-vertebrates.

Birch argues that, given current evidence, it isn’t reasonable to give attention to views more restrictive than R1 (such as sentience requiring language) or more inclusive than R5 (such as plants, unicellular organisms, or rocks being sentient). Of course, biopsychists and panpsychists would disagree.

Birch argues that R4 implies that all vertebrates (mammals, birds, reptiles, fish, etc) are sentience candidates in the meta-consensus. (Reminder: “sentience candidate” means there’s a “reasonable” chance the species is sentient, even if it can’t be established with certainty.)

Personally, I don’t think anatomy is a reliable indicator. Evolution doesn’t keep these kinds of nice neat distinctions. It seems like we can only establish what a particular structure does in a particular lineage of species. What might require a thalamo-cortical structure in humans may be done with other structures in lineages separated from us by hundreds of millions of years. And it gives us little to go on when considering developing organisms, artificial intelligence, or in other unexpected systems. For those, behavioral capabilities seem like more reliable criteria.

That may clash with an assertion Birch often makes in the book, that sentience isn’t intelligence. I agree that there’s not a 1:1 relationship here, but he often leaves it implied that it has nothing to do with intelligence. However, the team he led with the London School of Economics, which recommended protections for cephalopods and some crustaceans to the UK government, does include behavioral capabilities in their criteria for sentient pain.

The LSE critieria for sentient pain

Birch admits that the flexible behavior implied in 5-8 is tricky to assess, involving an unavoidable degree of subjective interpretation. (Which I think makes 6 and 7 too abstract and subjective.) Is an animal enduring a noxious stimuli for an anticipated reward? Or are they smelling the reward and the impulses associated with that are overpowering the avoidance impulses from the noxious stimuli? 8 seems like the clearest indicator, but it requires more intelligence.

(This, incidentally, is what led me toward greater skepticism for a lot of these studies. When I follow the citation trail, I often find the described behavior far less compelling than the headlines imply.)

Birch presents tables showing that octopuses fare well with 5 and 8. But other species of cephalopods do less well, with very low scores for 8. And the results for decapod crustaceans show low to very low scores for 5 and 8. The committee seemed to reach their recommendations based on high scores for 6 and 7, which as I noted, seem very subjective.

When it comes to insects, Birch sees the need to look at other criteria. He recounts the stories long told in biology, that many insects will continue moving and feeding despite catastrophic injuries, and reacting in other ways very different from how a mammal or bird might respond. Birch notes recent research showing the existence of nociceptors in at least some insects, but admits that nociception isn’t by itself sufficient for sentient pain.

He concludes that focusing on pain “does not serve insects well”. He switches to looking at evidence for moods, working memory, and attention. In the end, he admits that none of these really establish sentience at this level. (For example, the contents of working memory aren’t always experienced by humans.) But he argues that each of these lines of evidence raise the probability.

Birch also looks at gastropod molluscs, nematode worms, and spiders. He ends up classifying the gastropods as an investigation priority (not rising to the level of a sentience candidate, but should be studied more closely). He seems fairly confident that nematodes are stimulus-response driven. And he isn’t able to come down on spiders being sentience candidates, but mostly from lack of data, which he finds frustrating.

Finally, he rules out plants and unicellular organisms as sentience candidates or investigation priorities, citing a lack of any credible evidence.

In the final chapter of this section, Birch discusses proportionate responses for the species he does deem sentience candidates (cephalopods, decapod crustaceans, insects). The proportionate aspect means he doesn’t advocate banning crustacean or insect farming, but he does argue that the creatures in these farms deserve some type of protection. (A frequent example is eyestalk ablation used by shrimp breeders, which Birch regards as potentially inhumane.)

He does think cephalopod farming should be banned. Cephalopods are solitary creatures, who become extremely distressed if crowded together. But that is exactly what cephalopod farming requires to be economical. There doesn’t seem any humane way to farm them.

Of course, this would involve new laws and regulations. That may be feasible in Europe, but I have a hard time seeing it in my own country right now. Even with a Democratic administration, to contest economic and human quality of life interests, there would need to be much stronger evidence than most of what Birch is relying on. Octopus protections might be the easiest sell. But insects, particularly when Birch argues that we should research the effects of pesticides, will require a massive move of the Overton window, I think.

And it’s probably obvious from this post that, outside of mammals, birds, and cephalopods, I’m not on board with much of this myself, at least not yet. Not that I think we should ignore the possible welfare implications for these creatures. But I’m not inclined to be overly concerned about insects who cross my interests, such as the dirt dauber wasps who created a nest in one of my house walls.

And more generally, as I noted earlier in this series, I think it’s a mistake to treat sentience as something that is either all there or not at all. It seems clear to me that insects have more sentience than worms, but still far less than mammals or birds. Any protection measures, I think, need to take that into account. At least, that’s what I think today.

What do you think? Are there reasons to be more concerned about crustaceans and insects that I’m missing? Or are we allowing ourselves to be too concerned with creatures whose experience, to whatever extent it’s there, is far shallower and limited than ours?

Featured image credit

#Consciousness #Philosophy #PhilosophyOfMind #Sentience #TheEdgeOfSentience

This is the second in a series of posts on Jonathan Birch’s book, The Edge of Sentience. This one is on borderline cases of sentience in humans.

Birch looks at cases involving humans with disorders of consciousness, such as those in vegetative or minimally conscious states, as well as fetuses, embryos, and neural organoids made with human tissue.

For disorders of consciousness, Birch is primarily looking at patients who, due to an accident, stroke, or other malady, end up in an unresponsive or minimally responsive state. This could mean being in a coma, which everyone agrees isn’t actively sentient. But it could also mean being in a PVS (persistent vegetative state) or a MCS (minimally conscious state), where the patient is awake at times, but is completely or minimally responsive to their environment.

Diagnosing when someone is in a vegetative state is tricky. It involves establishing that the patient has no awareness of their surroundings, and shows no volitional behavior. If they do show some minimal awareness, such as eyes tracking movement, then they are upgraded to a MCS. But a key distinction is whether they show any signs of understanding language. If so, they are put in the MCS+ category, if not, the MCS- one.

But diagnosing whether someone is in a PVS, MCS- or MCS+ is very error prone. Birch notes that there is around a 40% chance of misdiagnosis. This is easy to see when PVS patients generally display sleep / wake cycles, but only react reflexively or randomly to what’s going on around them. Birch recounts the harrowing tale of Kate Bainbridge, who was in an unresponsive state for several months, and so diagnosed as being in a PVS. But she was conscious the entire time, including when medical procedures were performed on her. She was only able to reveal her ordeal as her condition improved and she regained some ability to communicate.

Birch’s conclusion is it’s a mistake to have the PVS, MCS-, and MCS+ categories. The chance of error is too high. Every patient who shows sleep / wake cycles should be treated as potentially conscious, and therefore accorded the considerations given to conscious patients, like being anesthetized during painful or invasive medical procedures.

This seems like a relatively easy call to make in cases where a human previously known to be fully sentient may no longer able to display it. But for the other categories we’re about to discuss, Birch’s concept of a “sentience candidate” becomes important.

A sentience candidate is a system we can’t be sure is sentient, but for which there is a realistic possibility, and for which it would be irresponsible not to take “proportionate” precautions. He has a weaker category, an “investigation priority”, for systems that don’t meet the standard for sentience candidate, but could still be sentient. He admits that the standards for “realistic possibility” is unavoidably subjective.

This becomes important when we start talking about the possible sentience of a fetus, which inevitably brings in the abortion debate. Birch argues that abortion rights shouldn’t be linked to the question of sentience, but to body autonomy. That somewhat matches the historical reasoning in US law, where fetal viability, the probability the fetus can survive outside of the mother, is the determining factor of when a state can begin restricting a mother’s ability to abort, at least before Roe v. Wade was overturned a few years ago.

Nevertheless most of us remain concerned about when a fetus can feel pain. For most scientists, that’s unlikely until the third trimester, when the cortical hemispheres come online, or later when the fetus begins showing cycles of REM sleep. However, Birch cites the theories of Bjorn Merker, Jaak Panksepp, and Mark Solms, to argue that there is a realistic possibility of a functional brainstem being sufficient for sentience. If so, he argues that fetuses as early as 12 weeks old, the beginning of the second trimester, become sentience candidates.

In the UK, for mothers considering an abortion and asking if their fetus can feel pain, the current standard is to advise them of the scientific consensus above. But Birch feels that they should be told of the “tremendous amount” of uncertainty involved, and that it’s possible a second trimester fetus may feel pain.

My take is that this is giving a lot of credence to theories that the lion share of neuroscientists reject. It’s also worth noting that proponents of brainstem consciousness tend to define consciousness in a different manner than most scientists. They acknowledge that this is processing well outside the scope of introspection, volitional attention, or overall cognitive accessibility. So what Birch characterizes as tremendous uncertainty, I think is actually the semantic indeterminancy discussed in the previous post. In other words, this may be more about how we define “sentience” or “pain”.

What does that mean for the poor mother asking if her fetus will suffer? It’s obviously not practical to give her a course in the philosophy of mind. On the other hand, I can imagine even a remote possibility of sentience making a difference for some women. Maybe the most honest answer would be to convey the current scientific consensus, but with a caveat that there are views outside that mainstream. I’m pretty sure no answer is going to be without controversy.

When it comes to embryos, the question is what can be done with the surplus embryos from IVF (in-vitro fertlization) procedures. For decades, there has been a 14 day rule. Embryos can be scientifically studied and experimented on until they are 14 days old. For decades, this was an easy standard to follow because the technology didn’t exist to preserve them that long anyway. But that has changed, leading to increasing calls for the limit to be expanded. Birch notes that from a sentience standpoint, he doesn’t think there is an issue before the fetus stage, but notes that the restrictions are really more about sanctity issues than sentience anyway.

Organoids are clusters of cells grown from human stem cells in order to study the tissue of a particular organ. Since it’s human cells, the results may be more reliable than studying the same organs in other species, and it offers an alternative to animal testing. Neural organoids are clusters of neurons grown for studying brain tissue. But since their development, there’s been anxiety about whether these clusters might have a form of sentience. I concluded myself a while back that this was very unlikely, but admitted that I’d get more anxious as what was grown became more structurally similar to a complete brain.

Birch reaches a similar conclusion, arguing that if a neural organoid ever had a brainstem, either through innervation or by artificial means, it would become a sentience candidate. (It’s widely agreed that the brainstem is necessary to “power” the cerebrum, even if it isn’t the “seat of consciousness”.) I’m not sure that, in and of itself, would be enough for me. I’d need to see more subcortical supporting structures, like the amygdala, hippocampus, etc. And it’s hard in practice to imagine any kind of functional consciousness without a body of some kind attached, even if only a virtual one. Add all that in the mix, and I’d be pretty nervous that we’re getting to a real life brain-in-a-vat type scenario. But we still seem very far from anything like it.

As expected, Birch’s is more open to liberal forms of sentience than I am. I expect this to get more pronounced in the next section on animals. But the human cases hit home in a stronger manner than many of the others might.

What do you think? Is Birch’s standard of “sentience candidate” a reasonable one? Am I being too dismissive of second trimester fetal sentience? Or the possibility of organoid sentience?

Featured image credit

https://selfawarepatterns.com/2024/11/02/the-edge-of-sentience-in-humans/

#Consciousness #Philosophy #PhilosophyOfMind #Sentience #TheEdgeOfSentience

I’m currently reading Jonathan Birch’s The Edge of Sentience, a book focusing on the boundary between systems that can feel pleasure or pain, and those that can’t, and the related ethics.

While this is a subject I’m interested in, I’m leery of the activism the animal portions of it attract. I have nothing in particular against that activism, but mixing it with science seems to risk questionable results. This is an area where there are often stunning headlines. However I sometimes find that when I follow the citation trail and dig up the actual study, the results are more nuanced and open to interpretation than the headlines imply. Since I don’t have time to do that with every study that gets publicized, I’ve become cautious in accepting the claims in this area.

Birch’s book has an activist feel to it. But he makes clear at the beginning that he’s interested in an evidence based approach. And in an initial review of the science and philosophy in this area, he admits that there is currently a tremendous amount of uncertainty, and a number of “zones of reasonable disagreement”.

The first zone of disagreement starts with how to even define “sentience”. After dismissing very liberal definitions, such as the ability to respond adaptively, Birch covers the concept of affects, which are usually characterized as having a valence (an evaluation of whether something is good or bad) and an arousal dimension. After some reasoning about drugs that could target either the valence or arousal aspect individually, he concludes that valence is the crucial one, and settles on a definition of sentience as the capacity to have valenced experiences.

Of course, that immediately leads to the zone of disagreement on “experience”, which leads to a review of the philosophy and science of consciousness. Birch discusses how epiphenomenal views of consciousness, a view that experience makes no difference to behavior, might make the question impossible to study. But since evolution can only select for things that make some difference, it seems unlikely.

Among materialist views of consciousness, Birch notes a key distinction, whether consciousness is a single unified natural kind, or two or more kinds. He notes that people like Daniel Dennett seem to be in the camp of rejecting a single kind, often characterizing it in an illusionist or eliminativist fashion, although Birch feels like “many kinds” may a better label. (This resonates with my own view, along with the semanticism of Jacy Reese Anthis or semantic indeterminism of David Papineau.)

Often proponents of a particular scientific theory are operating under the single-kind view, but a many-kinds view often takes a pluralistic stance, that many of these theories may be addressing different aspects of the same complex reality. Birch uses an analogy of people in a town working to understand “what it’s like around here”, with some focusing on the economics, others the social aspects, ecology, or other areas. But rather than recognize they’re all working on different aspects of the problem, they see each other’s theories as bitter rivals.

Birch also ecumenically recognizes “radical alternatives”, such as interactionist dualism, panpsychism, biopsychism, and IIT (integrated information theory), as being in the “zone of reasonable disagreement”. Each of these views have their own challenges, such as identifying where the interaction happens between the mental and physical in interactionist dualism, the combination problem in panpsychism, or the metaphysical assumptions of IIT (which Birch characterizes as idealist in nature) and how to test them.

Another question is whether there can be edge cases of sentience or consciousness. In evolutionary history, is sentience a sharp “lights come on” type development, or a gradual one? Are there creatures where the question of whether they’re sentient has no fact of the matter answer?

If it is gradual, are we talking about a sharp start to sentience with gradually enriched contents (shallow gradualism) or a gradual development of sentience itself (deep gradualism)? Deep gradualism seems more likely under some views (such as many-kinds, global workspace, or IIT) than others (such as dualism or panpsychism).

Birch reviews some of the philosophical literature which discusses how hard it is to sympathetically imagine an edge case of consciousness, and so try to use that as a reason to dismiss the conceivability of such cases. But Birch concludes that this isn’t a good reason. Just because we struggle to imagine something doesn’t mean it isn’t possible. (I also think people have a tendency to help themselves to whatever minimalist concept of consciousness they can find in any posited edge case and declare the experience is therefore wholly conscious.)

Birch admits that both many-kinds materialism and deep gradualism complicate his task, and that he would like them to be false. Since I tend to think both of these views are true, I’m going to be interested to see how he treats them as the book progresses.

Birch also discusses the traditional philosophical theories of ethics such as utilitarianism and neo-Kantianism, concluding that they’re compatible with the view he calls “sentientism”, that all sentient systems deserve moral consideration. He also discusses alternate views, such as eco-centric ones, as well as the views of some of the major religions. Most he can see as compatible with sentientism, although he admits that it’s a rough compatibility in some cases.

One interesting view is a consciousness-without-valence one, which could become an issue with artificial intelligence. Consider a PV (philosophical Vulcan). PVs are different from Star Trek Vulcans, who merely suppress their emotions. A PV has no emotions at all, and it could be argued, no sentience. But they are conscious. Are they worthy of moral consideration?

Here I think we see an issue in Birch’s valenced experience definition of sentience. He admits that a PV would likely have preferences about outcomes, and so would reason about those preferences in relation to their perceptions. He makes a distinction between this and “valence”, which I think reveals he’s unwittingly sneaking in more of the affect concept in his notion of valence, such as arousal and motivational impulses. But he concludes that the PVs have found an alternate path to moral significance, so it doesn’t seem to matter. However that seems to put him in the same camp as David Chalmers, who uses the PV concept to argue that it’s consciousness itself rather than sentience that is the crucial issue.

Which brings us back to the possibility of consciousness and sentience being semantically indeterminate, which would seem to make the ethics around them also indeterminate. I’m not a moral realist, so this holds no dilemma for me. But it obviously does for Birch’s project. As I noted above, I’ll be curious to see how he deals with it in the rest of the book. (I’ve currently only read the first quarter or so.)

What do you think about Birch’s overall project? Or about my conclusions of semantic indeterminancy? Are there reasons to think the edge of sentience is sharper than I’m imagining?

https://selfawarepatterns.com/2024/10/27/the-semantic-indeterminacy-of-sentience/

#animalConsciousness #ArtificialIntelligence #Consciousness #Philosophy #PhilosophyOfMind #Sentience #TheEdgeOfSentience