the lucha whoreSep 8, 2025
"what if the ship of theseus was sentient" is not a question I was prepared to grapple with on a monday
Show thread2something
@leshiaaimee This was one of the responses to Searle's Chinese Room argument. Searle asserted that a machine can never be "truly" conscious, even if it has the exact same functionality as a human brain, because the individual pieces of the machine are not conscious.

One response is to ask what happens if a human is given brain surgery to replace a single neuron with a computer chip with the exact same size and function, thereby keeping the brain's function exactly the same. Then the human gets surgery to replace a second neuron, again without changing anything about how their mind works. And a third. If the process continues until all neurons have been replaced, then you have an all-computer brain which behaves exactly the same as a human brain.

According to Searle, the human brain is conscious, while the identically-behaving robot brain is not. At what point in the replacement process does it cease to be conscious?

EDIT:
And of course someone already put it on Wikipedia.
https://en.wikipedia.org/wiki/Chinese_room#Brain_replacement_scenario
Chinese room - Wikipedia

Dec 4 at 2:26amWeb
Show threadWesDymDec 4
@2something Searle's hypothesis seems to me to rely on assumptions about what and where consciousness is, and how it 'starts'. He could well be wrong about that.
Show thread2somethingDec 4
@wesdym

I don't think the Chinese Room can be called a "hypothesis." To be a hypothesis, it would have to make predictions that could be tested by an experiment.

Searle's argument is

1) Imagine a hypothetical machine which is completely indistinguishable from a human brain to any outside observations.

2) [Insert a bunch of semantics about definitions.]

3) Therefore, the machine from step (1) is not conscious even though no observer can distinguish it from a conscious being.

Searle's claim is, by design, impossible to test with any experiment. The first step assumes that no experiment could distinguish the Chinese Room machine from a human who is fluent in Chinese.
Show thread0xPsiDec 5
but doesn't that thought experiment just smuggle in functionalism as a premise? how do we know "exact same function" actually preserves whatever it is that makes subjective experience, rather than just outward behavior?
Show thread2somethingDec 5
@oxpsi No, it assumes the opposite. If you assume Searle is right and that a machine can behave identically to a human without being conscious, then at what point during the replacement process does the brain cease to be conscious?