Author: L. M. Sacasas

Translated by: Deep Tide TechFlow

Deep Tide Guide: As we outsource more and more decision-making to AI, human society is generating an "unconscious layer" that is detached from active judgment. This is not a sci-fi threat of superintelligence, but a more insidious transformation of civilization — we can do increasingly more, but we understand increasingly less. This serves as a wake-up call for every practitioner who relies on AI tools.

In the early 20th century, British mathematician and philosopher Whitehead famously stated, "We should cultivate the habit of thinking about what we are doing — this is a grossly mistaken old cliché repeated by all textbooks and famous speeches." "On the contrary," he believed, "the progress of civilization lies in expanding the number of important operations we can perform without thinking." He even made an analogy: "Thinking about operations is like a cavalry charge in battle — strictly limited in number, requiring fresh horses, and can only be launched at decisive moments."

You are likely familiar with this quote. It does convey a certain truth. But to me, this statement always seemed shortsighted or insufficient. Perhaps it is because I often see people use this quote to justify indiscriminately outsourcing human cognitive activities to machines without properly calculating the costs, or even being completely unaware that there are costs.

Fairly speaking, Whitehead's quote comes from the 1911 "Introduction to Mathematics," where he is actually discussing the advantages of symbols and notation in mathematical calculations — although he mentions that these operations "do not require thought," they are still learned and applied by thinking minds. However, since he discussed "civilization," along with that grand rhetorical flourish, it arguably invites this (mis)interpretation.

The problem is not whether some cognitive automation, or even the externalization of certain mental processes, can be useful in specific circumstances. Frankly, I believe that drawing an analogy using physical activity rather than cognitive activity yields a more persuasive argument. I have long since moved away from any athletic skills I may have once had, but the lesson remains. Almost every young beginner in sports will become frustrated to find that you have to spend a seemingly excessive and painful amount of time executing basic and repetitive training over and over again. Waxing and polishing — those who have seen "The Karate Kid" will understand. But what actually happens is exactly the dynamic Whitehead describes. You are automating certain physical actions, so you can execute them without thought. Only by doing so can you display any creative or extraordinary skills. The same applies to learning to dance or play an instrument. Automating fundamental movements is an indispensable foundation for achieving mastery.

But does this dynamic apply equally to all fields of human activity? Is there such a situation where outsourcing certain activities weakens, rather than promotes, the higher good that one is pursuing? Or, are "lower-order" activities accompanied by certain goods that we do not want to lose? Or even, as in mathematics, can we always easily separate specific subroutines from a given process or activity? Are there indivisible holistic activities that would collapse if any of their elements were attempted to be outsourced or automated? Moreover, as I mentioned earlier, is there not a distinction between internally mastered cognitive automation and the complete externalization or overall outsourcing of cognitive labor? Returning to the analogy of physical activity: if an athlete imagines using machines to complete all the "mundane" training, so they can jump directly into the exciting part of the competition — that person will never actually reach that stage. Before we readily accept the promises of efficiency and liberation made in the name of new technology, these are crucial questions.

However, I feel uneasy about Whitehead's often-quoted hymn to cognitive automation, perhaps because it sharply contrasts with Hannah Arendt's warning: we should "think about what we are doing." Whitehead wrote half a century before Arendt, and I do not believe Arendt was alluding to Whitehead. It is just that these two statements happen to form opposing echoes, and I find it useful to explore this tension.

"We should cultivate the habit of thinking about what we are doing — this is a grossly mistaken old cliché." — Whitehead

"Therefore, my proposal is very simple: just to think about what we are doing." — Hannah Arendt

Interestingly, the context of Arendt's admonition is much closer to our current anxieties about technology and human existence. This sentence comes from the preface of "The Human Condition," first published in 1958. She explains:

"What I intend to do is to re-examine the human condition starting from our most recent experience and recent fears. This is clearly a matter for thought, while thoughtlessness — reckless impulsiveness, desperate confusion, or a complacent repetition of 'truths' that have become trivial and hollow — in my view, is one of the most prominent characteristics of our age. Therefore, my proposal is very simple: just to think about what we are doing."

The "most recent experiences" that prompted Arendt's exploration include the launch of artificial satellites and the advent of automation. Although until recently, people may have thought that Arendt's concerns about automation's impact on labor were misjudgments, it now seems that her worries may have simply been too early.

When I reflect on the differences between Whitehead and Arendt on the question of "do we need more thinking or less thinking," I find it useful to distinguish between the following two aspects: on one hand, establishing a set of unconscious routines to develop higher capabilities; on the other, being subjected to unconscious forces that can unpredictably drive your behavior and may cause harm.

Is there a critical point at which Whitehead’s principle flips or reverses? "The progress of civilization," Whitehead said, "lies in expanding the number of important operations we can execute without thought." But what if there is a critical point? Beyond a certain threshold — whether in terms of quantity or quality — would civilization degrade because of an expanded number of important operations that humans can perform without thinking? Or at the very least, would we obtain a civilization that is completely different from what we know? We may soon find out the answer, as the adoption of AI often means the automation of our thinking, creating a vast field of unconscious actions in the world, sourced from the externalized storage of our personal and collective memories.

Arendt worried that as our technological capabilities (primarily articulated in mathematical language) surpass our ability to understand them in ordinary language, we may lose the ability to "think and talk about what we can indeed do." (This suggests that issues of explainability existed even before the emergence of large language models and so-called black box algorithms.) In a passage that has now become relevant and urgent again, she continues, "It is as if our brains — which constitute the physical and material conditions of our thought — cannot keep pace with what we are doing, to the extent that from now on we indeed need artificial machines to think and speak for us." This is an urgent political question. Under these conditions, we will no longer be self-governing in any meaningful sense. The political realm of human action, constituted by speech, will cease to exist.

On Christmas 1958, the same year Arendt published "The Human Condition," W. H. Auden published "For the Time Being," which contains lines that are thoroughly Arendtian:

When we observe, we encounter

that observing heart of self-observation

that is neither concerning nor cruel

but thoroughly mediocre

even though the tools it controls

can make wishes and un-wishes come true

it clearly cannot understand

the things it can clearly do

Auden and Arendt were friends, and Auden wrote a favorable review of "The Human Condition," but I am uncertain of the direction of influence.

While Arendt directly explained how we reached the state of "clearly unable to understand the things we can clearly do" — namely, when our technology-enabled actions cannot be fully understood by our everyday language — there is another interesting framework.

Actions that language cannot penetrate, and thus conscious thought cannot penetrate, can also be explained through the unconscious. I am not one to instinctively resort to unconscious language, but it may be a useful analogy to help understand our relationship — whether as individuals or collectives — with the increasingly mediating artificial intelligence devices that shape our world experience.

Simply put, I can think of two ways to analogize certain instances of consumer AI as being akin to the unconscious. The first is relatively straightforward: as we outsource more and more tasks to agent AI at personal, organizational, and institutional levels, we generate a layer of activity in the world that is functionally detached from human active judgment and oversight. Moreover, as long as this layer is structured and shapes our experience, the ratio of conscious human actions to unconscious human actions is decreasing.

Clearly, at least since the industrial era (if not earlier), there have been relatively opaque systems operating within the realm of human life. But I believe the distinction lies in the fact that these systems were relatively isolated from everyday human activities. In contrast, AI systems are increasingly woven deeply into our experiences. In many cases, these not only involve external forces exerting influence on us; they are also processes that we release and initiate, acting on us and reacting back on us. It is this close intertwining of humans and machines that brings to mind the analogy of the unconscious.

Erik Hoel recently articulated a similar argument with more elegant phrasing and greater length. His conclusion is:

"The traditional danger of AI is usually considered a existential threat posed by superintelligence. However, this may overlook a real and more subtle danger: the AI revolution is a mechanism that shifts the processes of our civilization from conscious oversight to the unconscious. But when AI removes consciousness from the operations of the world, it makes the world increasingly inexplicable, more alien and harder to understand. So far, the 'pollution of shared space' has proven to be the primary risk of the large language model revolution. And, as AI systems become more intelligent, especially if they remain (or likely remain) unconscious, a more significant risk is the decline of the importance of consciousness in culture."

This line of thought parallels a notion I have intermittently argued for a decade: that digital technology enchants the world in yet another way. In other words, it makes the world mysterious and elusive.

We, these enchanted objects of technology, confront them in a way that cannot be ignored; we must deal with them. We turn to technology for help, placing our hopes in their power. We also fear technology, considering them to be the source of our troubles. The technological forces we encounter are sometimes benevolent, but just as often are malevolent forces that disrupt our efforts and derail our plans.

Technological objects not only have the potential to empower us but sometimes even amaze us. We also view these objects and forces as crucial factors determining our fortune and misfortune, operating on us outside our control and in ways we do not understand. In other words, we are vulnerable, and our autonomy is undermined by the distributed agency of technology, which intersects with our wills and desires.

The second way to analogize AI with the unconscious may be somewhat more abstract, so let me explain. In "Understanding Media," Marshall McLuhan claims that "with the advent of electric technology, man extended, or placed outside, the living model of his own central nervous system." He returns to the idea that the electric medium is an externalized version of our nervous system multiple times. For example:

"One of the main characteristics of the electric age is that it establishes a global network, which largely has the characteristics of our central nervous system. Our central nervous system is not merely an electrical network; it constitutes a single unified field of experience."

McLuhan was not the first to propose this viewpoint. As early as 1950, a Catholic priest and paleontologist, Pierre Teilhard de Chardin, claimed that media technologies constitute "a real nervous system for humanity" and a "carefully constructed collective consciousness." Through an emerging network of computing and communications technologies, a new layer of reality is arising, enveloping the biological layer. It is a unified human consciousness layer mediated by technology, which he referred to as the "noosphere," describing it as "an astonishing thinking machine."

(I admit this is not my usual mode of expression, but let's play around with these concepts a bit. In fact, I have been trying to persuade you that viewing AI from a psychoanalytic perspective may be a useful angle.)

The electric medium extended our capacity to perceive the world, bringing distant events to our eyes and ears, compressing the time it takes to send information to almost instantaneous. But since the time of Teilhard and McLuhan, something else has emerged: a massive increase in artificial memory or data storage capabilities. So, while the electric medium extended our nervous systems, it recently also facilitated the collection and storage of unprecedented amounts of information in the form of text, images, video, etc. Our nervous systems are supplemented by a vast scale of digital memory. It is this digital memory that nourishes large language models, which have now nearly become synonymous with artificial intelligence.

This vast repository of human knowledge and culture is essentially the internet, or at least the internet is the way it becomes accessible to us. For this reason, digital media theorist Gregory Ulmer referred to the internet in the early 2000s as "the prosthesis of our collective unconscious."

I believe Ulmer meant the following. We have always been social animals, meaning we are intersubjective beings of culture. However, much of what shapes us, particularly during childhood, typically fades from our conscious awareness over time. Yet, the internet, as a prosthesis of this unconscious dimension in our formation as social beings, renders our collective cultural memory accessible. The internet is a vast network of cultural products, symbols, images, texts, and assorted trivia that have shaped our lives, connected not by linear logic but by an associative logic reminiscent of the human unconscious in psychoanalytic theory.

Ulmer believed this offers us a profound opportunity. By making the collective unconscious something we can reflect upon, something accessible to conscious thought, we thus gain a method of overcoming what would typically be deep blind spots at the core of our experience. Needless to say, I have never been overly optimistic about this possibility. But regardless of how you assess Ulmer's optimistic vision — I have greatly simplified here, but I hope I have not misrepresented — it seems to me that the emergence of commercialized chat-based AI has indeed driven us toward a very different direction.

Interestingly, inserting a quotidian language interface between ourselves and the digitized collective unconscious makes it all the more opaque and difficult to comprehend. The chatbot interface reconstructs our agency in navigating the collective unconscious; one could say it becomes an anti-therapist, leading us away from self-awareness and insight (no matter how unsettling or surprising) into a comforting and soothing encounter. It provides a false clarity that traps us in self-satisfaction, shielding us from self-doubt, preventing us from lingering too long in the awareness of our ignorance, or in troubling uncertainty. It obscures the tangled forest of human experience, illuminating for us a man-made clear path toward the promise of knowledge and wisdom. But in doing so, it gently but decisively plunges us back into the unconscious. Perhaps this is the root of AI neurosis.

Regardless of the connection to AI neurosis, it is worth contemplating that the madness, lethargy, compulsiveness, aggressiveness, anxiety, and despair in our public and collective existence partly stem from human consciousness gradually retreating in the face of a new form of collective unconscious reestablished in the form of artificial intelligence.

The layer of non-human action in this world was already being constructed before agent AI emerged. Algorithmic processes have long been building the elements of human experience, particularly in the context of bureaucratic systems.

I am not sure when and by whom the conceptual connections between electricity, the nervous system, and thought were first established, but in his 1851 novel "The Blithedale Romance," Nathaniel Hawthorne wrote, "Then there was electricity, the devil, the angel, the powerful physical force, the ubiquitous intelligence!" and: "Is it a fact — or did I dream it? — that by electricity the material world has become a huge nerve, vibrating thousands of miles in an instant? More precisely, is the round earth a giant head, a brain charged with intelligence! Or should we say, it is itself a thought, just a thought, no longer the entity we imagined!"