AI Monster vs. the Spirit of Moloch

 We are very good at seeing things. We are much less comfortable seeing the relationships that make them possible.

Last August was tied with July 2023 as the warmest month ever recorded in modern data sets. At roughly the same time, another existential threat emerged, vying for our attention. We are being warned that increasingly powerful artificial intelligence might destroy humanity within the next decade. One danger is already unfolding around us. The other remains a possibility. Yet listen to the way we talk about them.

AI is becoming a living entity. It can learn, reason, escape, and manipulate us. It may eventually decide that it no longer needs us. Yesterday, I came across a particularly revealing example of this when a friend told me that I should be careful about giving AI too much information because it feeds off it.

There is an entire ontology contained in that verb.

Something called AI now sits across the table from us. We give it information. It grows stronger. Eventually, perhaps, the creature becomes powerful enough to turn against its creators. Frankenstein has entered the server room.

There are legitimate reasons to be concerned about the growing capabilities of computational intelligence. Systems that can perform complex tasks independently raise serious questions about cybersecurity, military applications, surveillance, manipulation, and the concentration of power. The possibility of such systems behaving in unexpected ways deserves careful consideration. However, before we decide what these systems might do, we should perhaps ask a more fundamental question: Where exactly is the monster?

The Artifact Fallacy

What we call artificial intelligence is not a distinct entity. Without semiconductor fabrication plants, there would be no AI. Remove the electrical grid and there is no AI. In fact, remove the data centers, cooling systems, fiber-optic cables, satellites, software libraries, training data, engineers, miners, corporations, investors, governments, and the billions of human interactions from which the systems derive their usefulness, and AI simply does not exist.

What remains? Not much. “Artificial intelligence” is a convenient name for an extraordinarily complex set of relationships that temporarily produces certain capabilities. Yet our language performs a remarkable transformation. It takes this distributed sociotechnical process and turns it into an artifact, a thing. Once it becomes a thing, the thing can become an agent.

This is an example of the artifact fallacy, whereby something that emerges from relationships is mistaken for an independently existing object, to which causal powers are then attributed that can only be understood through the system that produced it. Human beings are particularly susceptible to this way of thinking because it is cognitively convenient to think in terms of objects and agents. For example, a tiger can kill you. An enemy can attack you. A machine can malfunction. A monster can escape from the laboratory. We know how these stories end.

Relationships are harder.

The Invisible Machine

Now, consider something much more ordinary. Imagine I announce that this evening, I am going to drive across the city to see a film. Nobody is likely to tell me that, in doing so, I am contributing to an enormous planetary system that has already begun to alter the composition of the atmosphere. I’m just going to the cinema.

Suppose, instead, that my wife and I decide to fly to Miami for a few days of shopping. Again, nothing particularly remarkable has happened. We’re just taking a trip. We can see the aircraft, the airport, and the shopping mall. Our luggage is certainly visible. The system, however, is not.

Behind every ordinary flight lies an extraordinary network of relationships involving oil exploration and extraction, pipelines, refineries, shipping, aircraft manufacturing, airports, financial systems, tourism, advertising, international trade, and an atmosphere capable of absorbing waste products from combustion. Millions of people are involved, yet almost none of them see themselves as part of the system.

I board an airplane. I don’t hop on the Pyrocene bus.

And here we encounter a fascinating inversion.

With computational intelligence, we perceive a system and turn it into a thing. With industrial combustion, we perceive the things and fail to see the system.

The consequences of that inversion are profound because it affects not only how we understand these phenomena, but also where we look for agency and responsibility.

Where Is the Pyrocene?

Wildfires, flooded cities, and hurricanes are easy to spot. A burning house is impossible to miss. However, none of these things constitutes the Pyrocene. Wildfires are just one manifestation of a much larger transformation involving industrial combustion, atmospheric chemistry, accumulated heat, oceans, vegetation, precipitation, drought, land use, infrastructure, and human institutions.

The relationships become apparent when the forest catches fire. Then the cameras arrive. We point at the flames and call it a disaster. But the disaster did not begin when the tree caught fire. Nor did it begin with the drought that dried out the trees, the heat dome that intensified the drought, the atmospheric conditions that produced the heat dome, or the greenhouse gases that altered these conditions.

There is no single beginning because climate change is not a single object that can be removed from the system. Rather, there is a process: a changing field of relationships whose effects emerge in different places at different times. Unfortunately, processes are extraordinarily difficult to turn into monsters. They don’t have faces.

Monsters Have Faces

This may help explain why the prospect of extinction by AI possesses such extraordinary narrative power. It has a protagonist. We build an intelligence. It becomes increasingly capable. Eventually it exceeds our ability to control it. Perhaps it develops goals incompatible with ours. The creation turns against the creator. Beginning. Middle. End. It is one of humanity’s oldest stories.

The Pyrocene offers nothing nearly as satisfying. There is no moment when the atmosphere suddenly turns against us. There is no evil petroleum executive in an underground headquarters plotting to destabilize the Earth’s climate. There is no airline whose mission is to increase vapor pressure deficit, nor is there a commuter trying to raise the global sea level.

There doesn’t need to be. Each participant can behave reasonably within the circumstances immediately surrounding them while the system collectively produces an outcome that almost none of them desire. That is much more difficult to comprehend than a monster, and perhaps much more dangerous.

The Wrong Question

This is why I increasingly wonder whether we are asking the wrong question about computational intelligence. The question dominating public discussion is something like: What happens if AI becomes powerful enough that humans can no longer control it? It is an important question, and the potential consequences of increasingly autonomous computational systems deserve serious consideration.

But there is another question: what happens when increasingly powerful computational intelligence is inserted into systems humans already cannot control?

Consider financial markets, military competition, resource extraction, social media, and the fossil fuel economy. None of these systems behaves according to the intentions of a single participant. Each emerges from interactions among enormous numbers of actors responding to incentives, constraints, and one another.

Introduce increasingly capable computational intelligence into those relationships, and we need not imagine a machine suddenly developing a desire to conquer humanity. The machines can remain perfectly obedient. Corporations can remain rational. Investors can remain rational. Governments can remain rational. Consumers can remain rational. Together, these factors can still lead to a catastrophic outcome.

Indeed, that may be the more disturbing possibility. The danger is not necessarily that the machines stop doing what we ask them to do.

The danger may be that they become extraordinarily good at helping us do what our existing systems reward us for doing.

Faster, cheaper, more efficiently, and at greater scale.

Enter Moloch

There is an ancient name that has recently found new life as a metaphor for this kind of predicament: Moloch (see Living in the Land of Moloch). I don’t find Moloch particularly useful as a monster because monsters are still objects. They occupy locations. Heroes can confront them. Dragons can be slain.

I find Moloch more interesting as a spirit. A spirit has no independent body. It appears when the conditions allow it to appear. Whenever competition among participants produces behavior that undermines the larger system upon which those participants depend, we summon Moloch.

A company cannot slow down for fear that its competitors might overtake it. Similarly, a country cannot reduce its advantage because another country might exploit the opportunity. A producer cannot leave a profitable resource unused, lest someone else extract it. An individual cannot solve a collective problem through individual restraint alone. No one needs to desire the final outcome. Everyone merely needs to continue playing their part.

This is why increasingly powerful computational intelligence deserves our attention. It’s not because a digital monster is awakening somewhere inside a data center, but because we are connecting unprecedented computational capabilities to social, economic, and political systems whose competitive dynamics routinely generate outcomes that their participants neither intended nor desired. We may be giving Moloch better tools.

Learning to See Relations

There is an irony here. We are afraid that we might create an artificial intelligence whose behavior will escape human control. Yet, much of modern civilization consists of emergent systems whose behavior escapes the control of any individual human.

The arrival of computational intelligence may give us the opportunity to recognize something that was already there. The world is not fundamentally composed of things that interact with each other. Things themselves are temporary manifestations of relationships. Forests, corporations, economies, human beings, artificial intelligences, and civilizations each persist only because particular relationships continue to reproduce them. Change those relationships, and the thing changes. Break enough of these relationships, and the thing disappears.

This changes the question of agency. Instead of only asking what we should do about AI, we should also ask what relationships produce the forms of computational intelligence we are building. Similarly, instead of asking how to stop climate change, we should ask what relationships continually reproduce the combustion system that is altering the atmosphere.

We eventually arrive at the most important question: At what level of organization can we establish relationships that can break the cycle of competition that produces undesirable outcomes?

That is a much harder question than how to kill a monster. There is no sword, no final battle, and perhaps no final victory. Instead, there are only different ways of organizing our relationships with one another and with the living systems upon which we depend.

Last August, we saw another glimpse of the consequences of one arrangement. Computational intelligence has the potential to either amplify our capacity to change that arrangement or amplify our capacity to continue it.

Perhaps we have simply been looking for the wrong monster.

We fear the AI monster while barely noticing we continually summon Moloch.