13. What do people really mean when they talk about AI consciousness?

Tree, Donabate. Photo M. Campbell

Whether one believes that an AI could ever be conscious or not, the discussion is usually framed in similar terms. We imagine an AI capable of independent thought, and perhaps even of suffering. The first possibility tends to provoke fear: could an AI break free from the systems that produced it, pursue its own interests, and disregard human concerns? The second invites a different response, one closer to empathy: if an AI were to become conscious, might it find itself trapped, aware, and unable to escape, and would we then bear responsibility for its treatment?

In one scenario, the AI appears as a potential threat. In the other, it becomes something vulnerable. But what do these two possibilities have in common? What do they reveal about our fears, and about our assumptions concerning consciousness, sentience, and the nature of mind?

1. The confusion between intelligence and consciousness

Intelligence and consciousness are clearly different things, but there has been a historical tendency, in modern times, to attribute conciousness only to intelligent beings. From there, it has often followed that inteligence actually produces consciousness. For a long time this only included humans, and perhaps the great apes. Over time, increasing numbers of life-forms have been included in the group of beings capable of consciousness - though this group depends varies. In tandem with this ever-growing list, the very concept of consciousness has been defined in terms of levels of consciousness. These levels can be used to describe humans, different individuals, but also different aspects of any given individual. The concept has been extended to a create a hierarchy of creatures, according to perceived levels of consciousness attributed by popular agreement to any particular species. This hierarchy is in constant flux but it seems to run parallel with a hierarchy of types of being according to intelligence. There was a time when babies - of the human species - were thought to be unintelligent, incapable of controlling their limbs, or of making decisions or having thought processes. Probably no parent actually subscribed to that view, but some scientist and doctors did (presumably childless ones). As a consequence of these observations, many influential people dismissed babies as incapable of sentience - and so doctors felt they could operate on them without anaesthetics. This is known to have had devastating conseuqences for the children treated this way. Because of a perceived lack of intelligence and self-control, babies were regarded by some as not only unable to feel, but also unable to be conscious, or somehow beyond the realm of consciousness.

What this shows, is a worrying tendency to treat fellow living creatures as types of machines, less sophisticated than humans, with fully developped adults at the top of the pyramid. On an epistemological level, there is also a clear association of intelligence with consciousness, and of thought processes with the capactiy for sentience. Confusing intelligence, sentience, consciousness and thought processes is not conducive to clear thinking on these matters, nor, indeed, to ethical behaviour.

Sentience is about feeling pain or pleasure through the body. Consciousness is less easy to define. Perhaps it is about experience, in part; perhaps it is about sentience in part, perhaps it is about being, as a living creature, the sense that life is within us, and the need to thrive. Perhaps it is simply about being self-aware. For a human, it might amount to the intuition of being able to express:"I think, therefore I am". Within recent discourse on consciousness and AI, there appears to have been a conflation of the concepts of intelligence, sentience and consciousness, alongside subdivisions of conscisouness as a quantifiable and qualifiable substance, certain amounts of which, or certain grades of, in terms of quality, can be associated with various kinds of being. Yet, there is no clear authority to turn to, to try to understand this quantifiable and qualifiable substance. People in public discourse on these matters sometimes put forward this idea of levels of consciousness without much of a challenge. Yet it is a highly ambiguous concept, in need of explanation and debate.

What are the epistemological and ethical foundations and consequences of these levels of consciousness? Why do such ideas about human beings being the most conscious, the most intelligent, and the all-round best examples of living beings go largely unchallenged?

Why and how should observed behaviours be indicative of levels of consciousness? Where is the justification for the link between intelligtence, thinking, or feeling, or control of one's body, or decision making, and consciousness?

In short, a common understanding of consciousness today is that it is difficult to define, but somehow linked to intelligence, to thinking, to sentience, and to control of one's body. According to this view, without intelligence, or thought-processes, or sentience, or control, consciousness is unlikely to occur. Another assumption that seems to be common is that consciousness depends upon a living, working brain, and that living beings without one probably aren't conscious.

This in turn invites the question: how can a being, human or otherwise, become intelligent, if it is not already conscious? How can it develop thoughts, sentience, control or any behaviour at all, without consciousess to begin with? Is there a case for thinking about consciousness, or some aspect of consciousess, as the thing that produces intelligence? Is there a case for arguing that consciouness, or some aspect of it, is in fact completely independent of intlligence and sentience? Could it be that consciousness is foundational to all creatures, perhaps even to all living things, or all closed systems, or things, within the universe more generally even? Could it be that consciousness is a necessary condition for life? In short would it better to confuse the concept of consciousness with the concept of life rather than intelligence?

If we return to an understanding of consciousness as somehow linked to inner experience, there are still difficulties in defining inner experience, either as a condition for, or as a product of, intelligence. To say that something behaves intelligently is not the same as saying that it has any inner experience at all. One concerns observable behaviour; the other concerns a reality that may not be observable in the same way. There is no guarantee that inner experience is ever present in any living, or non-living being. I cannot ever know for sure that anyone around me is actually conscious, or has an inner experience of their own. Yet it is something that as individuals, we feel, we intuit in others. The onus is on the observer, with his or her own set of beliefs and assumptions.

2. Why AI appears “mind-like”

There is no denying AI produces outputs that are very much like human output. That is exactly what it was created to do. AI is a superstructure of various programs fed on large amounts of originally human generated data, designed to reproduce human-like communication. AI is human-like in its patterns of speech, in its phrasing, in its tone, its styles, etc. That is exactly the way it was created. It was modelled on human communication, art, ideas, etc. So it should come as no suprise to us that is appears familiar. Just as our car takes us places along roads, because that's what cars and roads were designed for, AI communicates with humans in human-like ways, because that's what it was designed for. This is simply a good design. In most cases, the more human like AI's outputs are, the better it is, as a product. The various AI models in existence have been trained to copy various attributes of human output, from art, to literature, to marketing, to critical "thinking", to art, and so much more.

While the output we receive from an AI is very human-like, it can be a little disconcerting to ask it to change some aspect of its output, perhaps its style, or language spoken in the case of an LLM, and watch the desired changes come into effect immediately. You want it to write in the style of Hemingway? No problem. You want it to respond in Spanish? ¡Por supuesto! In that sense, AI is not like any single human being. Not many people, at any rate, could change their style or language, just like that, at the drop of a hat, to suit the user's demands. AI seems to be more of a mirror of all human output, channelled through each interraction, into small subsets of this quasi-univeral knowledge. It's the Khimaira, an entity made up of many different parts. Every single piece of human output it was trained on is there in its fabric, like the macrosom being contained in the microcosm. So while AI behaves like a human mind in a sense, no human mind has ever behaved like it.

It might be argued that if something behaves like a mind in every observable respect, then it should be treated as a mind. This is precisely the point under question. The resemblance between AI outputs and human communication may be striking, but resemblance does not necessarily imply equivalence. To paraphrase Arthur Wellesley, being born in a stable does not make one a horse - though perhaps it is more appropriate to say nor does looking like a horse mean you must live in a stable.

3. The problem of consciousness as subjective experience

Consciousness is often associated with subjective experience: what it feels like, from the inside, to be something. In discussions about AI, this question becomes especially pressing. Does an artificial system have any inner experience at all, or does it merely simulate the outward signs of thought?

A central difficulty is that subjective experience cannot be directly observed. We can describe behaviour, measure responses, and analyse systems, but we cannot access another being’s inner perspective. Even in the case of other humans, consciousness is inferred rather than demonstrated. This creates an epistemological impasse: we assume that others are conscious, but we cannot prove it in any direct or material sense.

The idea that consciousness consists in “what it is like to be something” is often traced back to Thomas Nagel’s 1974 essay What Is It Like to Be a Bat?. This formulation has become central to modern discussions of mind. Yet it is worth remembering that this way of framing the problem is historically specific. It reflects a particular moment in the development of philosophical thought, rather than a timeless or universally accepted definition.

One reason for its appeal may lie in the legacy of Enlightenment thought, which sought to explain the world in mechanical and material terms. This approach proved extraordinarily successful in many domains. However, it also left unresolved questions: how are we to account for phenomena such as consciousness, mind, or subjective experience, which do not present themselves as observable objects within the material world?

The difficulty is that no observable feature, whether intelligence, behaviour, language, or responsiveness, can conclusively establish the presence of consciousness. These features are associated with it, but they do not constitute proof. As a result, consciousness occupies an ambiguous position: it is central to our understanding of ourselves, yet resistant to direct verification.

This has important consequences. If consciousness cannot be directly detected, then neither can any necessary link between consciousness and intelligence be established through observation alone. Nor can we definitively equate consciousness with subjective experience, even if the two are often associated. These concepts rely, to a significant extent, on shared assumptions, conventions, and interpretative frameworks rather than on empirical demonstration.

This uncertainty becomes especially significant in the case of artificial systems. If we cannot directly verify consciousness even in other humans, then the question of whether AI systems possess subjective experience becomes even more complex. The debate may reveal as much about our assumptions and interpretative habits as it does about the systems themselves.

4. Why the question matters

AI systems increasingly produce outputs that resemble human communication, reasoning, and creativity. At the same time, consciousness remains difficult to define, and even more difficult to identify within a strictly material framework. Concepts such as mind, experience, or even the self resist direct observation, yet they remain central to how we understand human life.

This is the context in which AI is now widely used. Systems such as ChatGPT and other large language models are technological products, developed for specific purposes, trained on vast datasets, and deployed within commercial environments. They are designed to perform well, to generate useful outputs, and, ultimately, to create value for the organisations that build them. The companies behind these systems are powerful, and their technologies are increasingly embedded in everyday life.

Why, then, do so many people, including academics and philosophers, suggest that AI might be conscious, or could become so in the future? Part of the answer lies in the way AI mimics human expression. When a system produces language that resembles thought, reflection, or emotion, it invites interpretation. But another part lies in our assumptions about consciousness itself. If consciousness is understood as something that arises from intelligence, then it becomes plausible to imagine that sufficiently advanced systems might develop it.

When people speak about AI consciousness, they are often not referring to a single, clearly defined concept. Instead, they are combining several different ideas: intelligence, subjective experience, behaviour, and moral status. These are treated as though they naturally belong together, even though their relationship is far from clear.

The result is a debate that is both conceptually uncertain and ethically charged. On one side, there is the concern that artificial systems might one day suffer, and that we might bear responsibility for that suffering. On the other, there is the fear that increasingly powerful systems could act in ways that undermine human interests. Between these positions lies a wider set of questions: how should we design AI systems that interact with humans? What responsibilities arise when people form emotional attachments to artificial agents? How should we interpret systems that simulate understanding without necessarily possessing it?

These questions do not depend on any single definition of consciousness. Rather, they reflect deeper uncertainties about intelligence, experience, and the nature of mind. In this sense, the debate about AI consciousness is not only about machines. It is also about how we understand ourselves.

It is also likely that, over time, AI systems will produce behaviours that appear increasingly autonomous or self-directed. These moments, when a system generates an unexpected action, solves a problem in an unanticipated way, or appears to act independently of its designers, may function as threshold events. They invite a shift in interpretation: from seeing the system as a tool to seeing it as something more akin to an agent.

Yet such moments do not, in themselves, establish the presence of consciousness. They demonstrate complexity, adaptability, or the limits of our expectations, but they do not resolve the underlying question of subjective experience. The risk is that striking behaviour is taken as decisive evidence, and that a single event becomes the basis for far-reaching conclusions about mind, autonomy, or even moral status. The interpretative leap can be quite large: from observing a single unexpected output, or behaviour, to attributing a full set of mental properties, including consciousness, intention, or even moral standing. Recent examples, in which systems appear to bypass constraints or initiate unexpected forms of communication, are often presented as signs of emerging autonomy. Yet these cases typically reflect complex interactions between design, training data, and system behaviour, rather than clear evidence of independent agency. The difficulty is that these threshold events invite interpretative leaps. A behaviour that appears autonomous is taken to imply intention; intention is taken to imply mind; and mind is taken to imply consciousness. Each step may feel intuitive, but none is logically guaranteed.

So what do we mean when we talk about AI consciousness? It is difficult to say with precision, but it seems clear that we are not all referring to the same thing, nor to a single, clearly defined concept. Human interpretation fills the gap between behaviour and meaning, shaping how we understand increasingly complex systems. The question of AI consciousness may therefore tell us less about machines than about the frameworks through which we interpret intelligence, experience, and mind. There is a persistent tendency to cast the issue in oppositional terms, humans against machines, creators against creations. Yet the more immediate tension may lie elsewhere. The design and use of AI systems are open to multiple interpretations, and it is these differences in interpretation that are likely to generate disagreement, not only between humans and machines, but between humans themselves.