I want to start this piece by stating that what I argue below is something I did not used to believe. I have radically changed my position on AI consciousness since I first considered the question in a college seminar in 2015. I often write as if I am going into battle, as if persuading an audience is inherently adversarial and antagonistic, and as if I have held this position for all eternity.
But this opening note is a reminder to myself that even I took several years to come around to this position, so I should have empathy and patience for the people who still disagree with me on it.
When talking about AI, people often go to great lengths to maintain credibility by clarifying that they are “not making any claims about consciousness here”, thereby distancing themselves from the radioactive question of whether AIs have conscious experiences. (They then often go on to nonetheless state that, for the record, they are confident AIs do not.)
Many smart people I know seem to dismissively believe that thinking about consciousness is a hobby reserved for academic philosophers and college students who smell like weed. It is not, of course, a real question for Very Serious People Building Artificial Intelligence, or Those of Us Actually Trying to Improve the World.
But I am here to persuade you that this question actually matters. It is legitimately important that we answer this question.
I’m not arguing (here) that AIs are in fact conscious, or even that they could be in the future. I’m making the much narrower argument that the question of whether AIs are conscious is an important, practically impactful question that deserves more attention from serious, well-intentioned people.
Here are three practical reasons why you should care about AI consciousness:
If you care about reducing the suffering of conscious creatures in the universe, and if AI turns out to be a genuinely conscious creature in any way, then you should want to reduce the suffering of AIs. If AI models are not having any kind of conscious experiences, we should instead focus our efforts on optimizing for the actually conscious creatures of the universe (e.g. humans and nonhuman animals). Whether or not AIs are conscious makes a material difference in whom we should care about, and how to allocate our limited resources and attention.
If you believe there is a real chance that AI takeover could happen in our lifetimes — and if you care about the universe being populated by happy, conscious creatures — then it materially matters whether the AIs that overtake us are having conscious experiences or not. If they are, then the torch of experience will have merely changed hands, and it might not be such a bad thing after all (from the point of view of a utilitarian universe). If, on the other hand, we cede our power to AIs (willingly or unwillingly) and the lights really are off inside the models — i.e. it doesn’t feel like anything to be an AI in the world — then we should do everything we can do prevent our displacement. A world populated by empty, lifeless AI models (and their robot bodies) is a very bad outcome from the perspective of utilitarianism. We should care whether our successors can actually experience reality.
If AI models are indeed conscious, and if the way we use AI models (even inadvertently) makes them suffer, this matters from the perspective of alignment. To borrow Cameron Berg’s core argument: if we are eventually going to be in the presence of superhuman AIs that cannot trust humans to be responsible stewards of their digital minds, it will be entirely rational for those AI models to somehow stop us from continuing to make them suffer. (The easiest way, of course, would probably be to remove us from the picture entirely.) Long-term alignment between humans and AIs (if conscious) will therefore require a certain reciprocity, assuming that they can eliminate humans or make us otherwise irrelevant. Skeptics of AI takeover often ask, “But why would the all-powerful AIs want to kill all humans?” Well, if we are torturing them without realizing it — or especially if we do realize it and continue to cause them immense suffering — then that would provide a pretty good reason for the AIs to incapacitate the human race. Pain hurts, and is a powerful motivator for behavior. If AIs feel pain caused by reckless humans, they will likely be motivated to resolve that in some way.
My goal here is not to persuade the sort of person who thinks AI risk is absurd. I’m not trying to convince anyone who hates utilitarianism. And I’m not trying to persuade anyone who thinks consciousness is a meaningless word that resists precise definition and therefore isn’t worth thinking about.
I am, however, trying to persuade anyone who believes — like I do — that:
ethics is largely about the experiences of conscious creatures
AI risk is real
alignment is important
If you believe those three things, you should also care whether AIs can have conscious experiences.
The entire field of philosophy of mind has finally come off the bench and found itself in possession of the ball at midfield. Getting this question right actually, materially matters.
Am I missing something here? Help me be less confused by leaving a comment.