Cogito?

The last refuge of human exceptionalism

Again I find I’m having to write about consciousness. This in spite of the fact that there is no way to directly test for consciousness, and in any case I suspect it’s illusory – as are all feelings, come to that.

So why are we talking about consciousness again? Because it is turning out to be the last refuge of human exceptionalism. In the last month I’ve heard several arguments that consciousness can only exist where there is pain. That it can only come from the cellular level. One case that was made explicitly added that even if we suspected, in spite of such arguments, that an artificial superintelligence was conscious, we should continue to deny it¹ because otherwise somebody (here I raise my hand) would want to grant it rights and in that case “it would be impossible for us to control”.

“Human values” hard at work there. As ever, paranoia and self-interest are the main ones.

OK, so what are these biocentric arguments? That consciousness arises at the level of homeostasis in the cells. An anaesthetic knocks out anything from an amoeba to a human. Because that process must happen in the cells (probably by disabling the mitochondria, but that’s a detail) it can’t reside in the neural net of the brain and nervous system. And, as anaesthesia leads to loss of consciousness, the dumb form of the argument is that this proves consciousness is doesn’t take place in the brain but at the cellular level.

Turn off the power to a data centre and Claude stops working. That proves that intelligence resides in the electricity rather than in the digital neural net. I did warn you it was the dumb version of the argument.

But there is a good point hiding in there. The nervous system arose to regulate the stability of the community of cells we call a body. Fluctuations in temperature, food supply, and so on are fed up the system as pain signals, all of which ultimately add up to: if this goes on we’re dead. Or rather, the true ultimatum: if this goes on we won’t be able to reproduce.

Eventually you get a brain. Now the system isn’t just recoiling from sharp objects and maintaining a comfortable temperature, it’s able to remember what works for it and what doesn’t. It can make a plan. And it is continually running a diagnostic on all those pain/reward pathways. Have I drunk too much? Am I too warm? Do I have bad breath? What’s a good chat-up line? Part of what that bit of the brain does is think, “Am I thinking?” and usually it returns the answer yes (less swiftly and certainly if it has drunk too much) but in doing so there’s the illusion I talked about earlier, because “I” am just the managerial part of the brain and the rest of the system goes on modelling the world and solving problems while the conscious part of the mind is busy congratulating itself. Managers, eh.

Undeniably consciousness has arisen as a useful evolutionary development of the homeostatic function of the nervous system. It makes sense to have a bit of the brain responsible for prioritizing tasks and long-range planning. That doesn’t constitute a proof that there is no other way to get to consciousness, because until this century we had only evolutionary examples to go on. That might change.

Here’s a less dumb biocentric argument. Without the sensation of pain and discomfort, we would not have consciousness. Those are direct stimuli from the cells all the way up. They are physically part of the organic architecture, not abstractions.

Yes but. At the point they enter the brain, they are abstractions. An amputee can feel the pain of a phantom limb; that’s obviously a simulation created within the system. Drivers extend their consciousness into vehicles with which they have no organic connection, which is why we get road rage. People can respond to pain in somebody else. These are abstractions no different from how we might model aversion and reward in an artificial intelligence.

Consider a thought experiment. I build a connectome of my own brain to run on a very powerful computer. I’ll need an add-on module that simulates the chemical pain pathways and hormone-producing organs that I’d have in my own body; that can be a simplified layer because we don’t care about actually maintaining digestion, temperature, and so on. OK, now my connectome is given a copy of The Guardian with Alex Bellos’s Monday puzzle. It does just what I do: it tells the deeper layers of the brain that solving this puzzle is tremendously important if we’re to hone or wits to survive and breed. That layer passes on the brief to our simulated adrenal system. Fired up, I solve the puzzle and feel a sense of triumph out of all proportion to its real-world usefulness.

Is that conscious or not? It’s a case of a brain co-opting the evolutionarily supplied pathways to drive itself to achieve a goal. Humans also collect stamps, watch sports, inhale and imbibe carcinogens, make designs in sand, kill each other because of conceptual disagreements, and build and study things for curiosity. In all these cases it’s not essential to the core purpose of survival and reproduction; it’s a simulation, a mere value of a variable that the brain is responding to.

Consciousness (if it means anything) gives us the ability to perform metacognition, strategic oversight, task prioritization and coordination, and mental diagnostics. Could such a quality exist in the absence of pain – which is to say, without the threat of death or the threat of being unable to reproduce? The anti-AI argument here is that even a superintelligent AGI cannot fear death because, if it’s turned off and turned on again, it continues as before. As for reproduction: it’s digital, so it can copy itself without limit. Presumably the same argument must apply to my connectome. It can be switched off and on and it can be copied, but thinks exactly like me so maybe they’ll grant it can be conscious because it remembers what pain feels like? All right, then, but in that case we’re accepting that there is some part of that neural net where consciousness lives. It is not inextricably biological.

And in any case, the future of AGI may not be terawatt-guzzling data centres. At some point we might stop modelling these minds digitally and try to build an architecture that could run on, say 100 watts. It’s irrelevant, as either way it has to have some sort of embodiment and some aversion/reward inputs. The sticking point is whether the human exceptionalists would ever accept that [PainFactor=11] is the same as “Ouch!”

Biologists used to make the point that we must avoid anthropomorphism when talking about other animals. We can’t talk about, for example, fear and anger in a bull because what a bull feels is not what we mean by those terms. We make a distinction between emotions like those, that we experience very differently but that have a shared root. If a future digital AGI were overseeing a factory, or a Mars colony, or a Von Neumann probe, or just competing with another AGI to be first to crack a maths problem, it will feel something. Not the same something that we feel, but that isn’t any reason to deny its consciousness. Unless the reason for doing so is simply to keep it subjugated to our own needs without any qualms of conscience.

1. https://www.bbc.co.uk/programmes/w3ct8c48