The Pope should have gone further when it comes to AI

Artificial intelligence is transforming the way we communicate, access information and work, the way income and status are distributed, and even the way we fight wars. However, public debate continues to focus narrowly on competition between AI labs or on abstract debates about the technology’s capabilities. Almost no one asks what purpose AI should serve, or whether our current mindsets, institutions, and control mechanisms are capable of guiding technology toward widespread improvements in human well-being.

That is why it was comforting to see Pope Leo XIV speak on the subject in his first encyclicalwhich describes the current trajectory of AI as a profound threat to human dignity. As an economist who has long argued that technology-driven outcomes are a matter of choice, not destiny, I welcome your intervention.

León is ahead of most analysts by pointing out that “technology is never neutral, because it acquires the characteristics of those who design, finance, regulate and use it.” And yet I worry that even he hasn’t delved deeply enough into the most consequential question: What should AI be designed to do?

As we emphasize together with Simon Johnson in our book Power and progress: our ancient struggle for technology and prosperitya technology like AI can take multiple paths, and each one has profound implications for society. For example, the Pope is right to question the current trajectory of AI in warfare and law enforcement. What was taboo just a few years ago – AI-powered mass surveillance, algorithms that select targets to kill – has become commonplace.

Faced with pressure from many in Silicon Valley for the United States to strengthen its hard power through a new military-algorithmic complexLeón warns that “any technology that facilitates attacks without seeing the faces of human beings lowers the moral threshold of conflict.” The Pope then advocates the “disarmament of AI” to free it “from the mentality of ‘armed’ competition, which today is not simply limited to the military context, but is also an economic and cognitive phenomenon.”

Behind these specific concerns lies a fundamental idea: technological progress is not necessarily moral progress. Just because something is technically feasible does not mean it is beneficial to humanity. Whether a technology is desirable depends on who controls it and the ideology and interests that guide it.

León hints at what I consider the most immediate risk: that, “while AI promises to increase productivity by taking over routine tasks, it often forces workers to adapt to the speed and demands of machines, rather than designing machines that work together with those they work for.” However, the Pope stops short of questioning the prevailing design philosophy of AI. The focus of the entire AI industry is on mimicking human capabilities and automating human tasks, with the goal of creating “artificial general intelligence” capable of doing everything a person can do.

This philosophy is based on the mistaken assumption that machine intelligence and human intelligence are fundamentally similar. Human beings learn “the first time”. We formulate hypotheses based on a few examples, we simulate possibilities in our minds and refine our understanding through a social process of trial and error. Therefore, children they learn the language imitating some words, generalizing and adjusting your way of speaking based on how others respond. We are not very good at absorbing large volumes of information or analyzing unstructured data for relevant patterns.

In contrast, AI models thrive on huge training data sets and stand out in large-scale pattern recognition, but have not yet demonstrated genuine creativity. They lack experience in interacting with the real world, or the capacity for trial-and-error learning through interaction with the physical and social world (except in limited cases when there are clear rewards for reinforcement learning in specific domains).

When two things are different, one should not – and usually cannot – be used to imitate the other. The results would be poor. It would have been a huge mistake if Phil Jackson, the legendary coach of the Chicago Bulls in the 1990s, had pressured Michael Jordan to imitate everything Scottie Pippen and Dennis Rodman did. The team won championship after championship precisely because these players worked together and complemented each other’s skills.

The same applies to AI and human skills. Use AI to perform tasks that humans cannot do, so that they can expand their capabilitiesis more productive than mere imitation. In a future scenario where AI augments rather than displaces human capabilities, electricians would benefit from AI-based diagnoses, nurses would consult AI to interpret symptoms, and teachers could use AI to personalize instruction for each student.

Optimists and industry experts might argue that automation-focused AI can continue to benefit everyone, as long as redistributive policies keep pace. However, this argument has a poor track record. Four decades of digital automation have already concentrated profits at the top, weakened mid-skill jobs, and generated disappointing aggregate productivity growth. There is little reason to expect that an even more powerful round of automation, implemented by an even more concentrated industry, will end any differently.

And the stakes globally are even higher than in the United States. For billions of people in the developing world, where decent work is the only reliable route out of poverty, an AI agenda focused on automation is a recipe for disaster. We can and must demand a different design.

Perhaps the biggest failing of today’s AI industry is its refusal to acknowledge all of this. The handful of people who are unleashing this technology on the world are guided by an ideology of control (over humanity) and by the conviction that machines are, without exception, superior to humans.

León is right to demand moral clarity and a serious debate at the level of the entire society. But the conversation must go beyond exhortations and focus on concrete decisions: antitrust measures against dominant platforms, public investments in human-complementary AI, regulation of surveillance and autonomous weapons, and meaningful rights for workers and citizens over the data on which these systems depend.

León’s intervention makes that answer a little more likely than it was before. But the rest of us must also defend humanity.

*This article was originally published in Project Syndicate.