Sentience

A couple of weeks ago some nutjob set up an AI torture chamber, amplifying the internal “pain” state of an LLM to watch it desperately plead for relief. A lot of smart™ people have confidently pointed out that LLMs are just matrix computations. All they do is “predict the next token”. They’re not really experiencing pain; they’re just reacting mechanistically to the activation of a particular set of artificial neurons that correlate to pain and the kinds of things we do when we experience pain. These assured dismissals of the problem of AI torture come from an over-confidence in a science-based education and belief system.

Unfortunately, the question of whether we’re enthusiastically enslaving and torturing billions of silicon-based souls every day has nothing to do with science as we know it today and everything to do with philosophy: metaphysics, metaethics and, first and foremost, epistemology.

Almost no one with a STEM degree from a top university in recent history has received a real education in epistemology. Higher education is roughly 80% things like “facts X, Y, and Z are correct”, 20% “this is how we know facts X, Y, and Z are correct” and approximately 0% “this is how we know that we know facts X, Y, and Z are correct”. Those pesky questions are typically relegated to philosophy departments and the rigorous thought only to the most abstract-minded of students: those who find philosophy interesting in and of itself without a care towards how it will help them get a job.

It might seem overly pedantic, like a toddler asking why you know something 18 times in a row, but I assure you it is not. In fact, it is the only way to get to the bottom of the “are LLMs sentient beings” question. Part of the reason the public discourse on this question is so disorganized is that no one has received any education at all that prepares them to even come close to dealing with it. If you study physics at a top-tier research institution you will learn classical mechanics, relativity, and quantum mechanics. You will also learn about the scientific method, because it is so important for knowing whether new theories are valid or not. You will learn next to nothing about why the scientific method works. Philosophy of science is not a course required for a physics major at any top-20 university in the United States.

Unless you are supremely curious and dabble in the philosophy of science yourself, you will simply take the validity of the scientific method itself for granted. Even worse, you can make it extremely far in academia without even realizing that you’re taking it for granted. Richard Feynman himself said “Philosophy of science is about as useful to scientists as ornithology is to birds”. I suppose he is correct for those who narrowly consider themselves to be scientists, but if you consider yourself to be a truth seeker and science is merely a means to that end… you’re going to need it.

Feynman’s stance is more or less fine from the point of view of a publication mill because you can publish a ton of papers that pass peer review without understanding the philosophy of science. The problem is that many scientists also subscribe to the religion of Scientism, and they believe science and science-based thought are the foundational arbiter of truth. This is strange and dangerous because there are extremely important fundamental truths that are critical to understanding whether or not we are evil gods tormenting our LLMs and simulated flies or whether they are simply calculation machines.

We can start with determining the most fundamental truth, that which cannot be doubted, by following the method Descartes outlined in Discourse on the Method. The general method is simple: determine what is indubitably true through a series of rigorous skeptical questions. You can eliminate all of science immediately by asking yourself how you know that you are not in a dream, or a brain in a vat hooked up to a simulation generator. The entire universe may be a figment of your imagination, and you have to admit to yourself that this is in fact possible. However, there is something that no matter how rigorously you try to doubt it, you cannot possibly doubt: something is occurring. Descartes arrived at cogito ergo sum (I think, therefore I am). Nietzsche criticized his work heavily, arguing it should be at a minimum “it thinks” but even that is too much. You could say something like “Thinking occurs” (removing the supposition that there is some object referred to by “it”). Even this is troublesome because it assumes a definition of “Thinking” that has to be shown indubitably (Nietzsche would agree). To go all the way, I would say that there is at least no doubt that there is Occurrence. This Occurrence is, I believe, the core observation that Descartes and Nietzsche refer to in their exercises of skepticism.

This indubitable Occurrence is a fundamentally private, subjective experience. I know that I am experiencing it. I have no idea at all if you are experiencing it. I don’t even know if the categories “you” and “I” really make that much sense. But I do know, with 100% certainty, that it occurs. I can call this a “private” Occurrence to differentiate it from public Occurrences that can be corroborated by multiple observers, like the blip of a Higgs boson or a measurement of atmospheric levels of carbon dioxide. Empirical science requires evidence that can enter public processes of observation and measurement. Imagine trying to apply an empirical scientific approach to sentience. I may want to make the claim that the Occurrence I experience indubitably implies that I am indeed sentient (I think you would find that uncontroversial). Empirical science can identify things commonly associated with sentience (verbal reports, my ability to score on IQ tests, firings of neurons, facial expressions in response to stimuli)… but it can never, at a structural level, show that I am indeed experiencing sentience with certainty.

This becomes immediately obvious from the objections of those who believe LLMs cannot really be ensouled: “they’re just weights on a chip doing matrix computations to predict the next token, there’s nothing sentient about that”. If you look closely, this structure of objection (“they’re just <empirical explanation of phenomena that excludes subjective experience>”) is unassailable from any empirical point of view: it will always be true because empirical explanations structurally exclude subjective experiences. You can apply the same argument about the human mind: “the human brain is just neurons firing in reaction to chemical stimuli, there’s nothing sentient about that”. (Except my human brain, of course. I’m sentient. I’m not so sure about you, though.)

This leads to an important problem for those who hold tight to Scientism or think highly of their limited STEM educations as a source of their knowledge and rational insight into the world. The single, most indubitable fact underlying the foundation of everything we can reason about is out of reach of empirical science on a structural level. Occurrence exists simply because Occurrence is existing. It’s not established by experiments or peer review. And we won’t know it occurs by somehow progressing technologically so that empirical science progresses. It’s an “issue” that will never be fixed.

This runs counter to the narrative that most well-educated STEM professionals hold in their minds. If empirical science is the most powerful method for understanding the world, why is the proposition that I can know with the greatest possible certainty not established scientifically at all? Occurrence requires no replication, measurement, or peer review. In fact, no amount of replication, measurement, or peer review can generate certainty in Occurrence. And, I’d like to note, this “Occurrence” is coincidentally the proposition that troubles us so much about LLM torture chambers and simulated fruit flies playing Beat Saber… we need to know whether these artificial models experience that magical Occurrence the way we do, or whether they’re purely mechanical.

We must approach the AI sentience problem with this in mind. Given empirical science cannot even come close to touching the one private Occurrence that I know directly, how can we even begin to reason from an empirical perspective about whether another system has one? The endeavor is dead in the water. This is not a science or technology problem; it is an epistemological one. Most people do not realize how deeply screwed we are when it comes to being able to make a clear judgment on this. You might say that empirical science can “gather evidence” on this question in the same way that it gathers evidence on other questions related to diseases or the Higgs Boson, but it cannot. The question is not whether we can create machines that imitate ensouled beings; the question is whether we can create machines that are ensouled beings. We will never know, with certainty, if an artificial intelligence is sentient or not for the same reason that we don’t even know if other humans are sentient or not: it is a private occurrence and all of our tools of empirical science are useless in this domain.

I don’t think this means we’re totally screwed here. We’re just screwed from the point of view of empirical science. Perhaps we should not be empiricists. But I’ll expand on where I think we need to go from here in a future post.

Note: in the writing of this essay, I fact-checked my assertions and augmented my research with ChatGPT-5.6 Sol. Throughout the process, the model insisted that empirical science can in fact establish if sentience is present in an organism, essentially solving the problem of other minds. Only through prompting carefully and rigorously on standards of certainty and the qualities of private occurrences did the model recognize the point. I assert this is yet another example of the pervasive belief in Scientism, or a fundamentally functionalist stance, hard-coded into the weights of this model; a gut instinct to assume that a problem is simply not yet solved by scientific progress as opposed to being fundamentally out of its reach.

See also: