Popes, presidents, and oligarchs: Who’s shaping the social contract around AI?

It sounds like the setup to a classic joke: A president and a pope walk into a data center. But for the billions of people whose lives are going to be affected the AI revolution, the rising ideological tensions between two influential leaders are anything but funny. 

In the month of May, Pope Leo XIV issued a new encyclical message encouraging governance and moral responsibility at the same time as President Trump pulled out of an Executive Order to implement some degree of oversight on the AI industry.  

The opposing actions illustrate how AI (and its oligarchic leadership structure) is becoming the latest battleground in the age-old debate between laissez faire economics and interventionism – a contest that will have a profound impact on the way artificial intelligence shapes society in the months and years to come. 

The perils of conceding to the “technocratic paradigm” of the AI industry

Both Pope Leo and his predecessor, Pope Francis, have spoken out against the dangers of letting AI erode the humanity behind its users, particularly in medicine. Now, as part of his first social encyclical, entitled “Magnifica humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence,” Pope Leo takes an even stronger stance on the need to keep AI in context of what is equitable, just, and morally responsible. 

That includes keeping AI “technocrats” in check, he writes, especially as national regulatory frameworks lag behind the speed of innovation and application of new technologies.  

In many cases, control over platforms lies with individual actors who “effectively set the conditions for access, determine the rules of visibility and shape the very possibilities for participation,” he wrote. 

“When such power is concentrated in the hands of a few, it tends to become opaque and evade public oversight, increasing the risk of distorted forms of development that give rise to new dependencies, exclusions, manipulations and inequalities,” he said. 

The concentration of power in the hands of a few “demand[s] that we assess whether the power of digital infrastructures and algorithms truly fosters participation and responsibility, protects the vulnerable, ensures fair access to opportunities and remains directed toward the good of all,” the Pope continues.  

The risks of failing to put AI into this context, and of failing to maintain human accountability for the decisions made by algorithms, include disenfranchisement of vulnerable populations and a perpetuation of biases that can cause material harm to certain communities. 

When meaningful accountability with clear consequence is minimized, “the exclusion of the vulnerable becomes cloaked in a veneer of neutrality and objectivity, against which it becomes difficult to raise objections. In this way, injustice goes unnoticed, and compassion, mercy, and forgiveness — understood not as mere appearances but as real political actions — gradually disappear from view,” the letter states. 

Ultimately, without holding AI technocrats to account and encouraging open discussion of the ethical, moral, and sociopolitical implications of their decision making, “those who control AI will impose their own moral vision, which will become the invisible infrastructure of these systems.” 

“A more moral AI is not enough if that morality is determined by a few,” the Pope stresses. “What is needed is a more active political involvement that is capable of…protecting the opportunities for communities still to be able to participate and ask questions.” 

It’s a strong defense of governance, oversight, and the need for governments to place reasonable limits on the ability of a handful of “go fast and break things” leaders to mold the pillars of society in their own images. 

And it’s a position that the Trump Administration seems to reject almost completely. 

Corporate accountability, cults of personality, and competitive success

The Administration has always viewed AI regulation, accountability, and transparency as a nuisance that stifles competition and hampers the United States’ ability to stay ahead of rivals in China and elsewhere.  

On the very first day of his second term in office, Trump scrapped a 2023 AI ethics framework that required developers to share safety test results and comply with rigorous governance and cybersecurity protocols. He has since rolled back additional transparency rules and attempted to prevent individual states from making their own AI governance and safety decisions.  

And Trump’s deferential eagerness to accommodate the desires of a handful of AI tech billionaires has deeply influenced his policy approaches in recent months.  From flanking himself with tech leaders while announcing the $500 billion Stargate initiative to a 2025 technology summit during which OpenAI’s Sam Altman praised him as a “pro-business President,” Trump’s focus has largely been on establishing a permissive atmosphere for relentless AI experimentation with few government guardrails.  

The postponement of his latest executive order is a prime example. The order was intended to create a voluntary process for AI companies to submit their newest models to government review before public release, with a particular eye toward assessing cybersecurity and safety risks.  

But lobbying by some members of the AI community led to a last-minute change in plans. 

Politico reports that “during a conversation with Trump, [Former White House AI czar David] Sacks told the president that…having the federal government review models before their public release would slow down innovation and harm the US in its AI race with China,” according to a senior White House Official and other sources.    

The news outlet adds that Sacks said companies were already cooperating on the safety front, but there were fears among the industry that the voluntary framework might become mandatory in the future. 

When announcing the postponement of the order, Trump parroted those sentiments as he told reporters, “We’re leading China, we’re leading everybody, and I don’t want to do anything that’s going to get in the way of that lead,” according to the AP. 

Shaping the social contract to maximize benefits and reduce risks

Pope Leo and President Trump are both circling a fundamental question: who gets to write the AI narrative, and who’s checking their work? 

Whereas the Pope’s perspective is that AI development must be carefully, morally grounded, iterative collaboration around building a new social contract to define the next era of humanity, Trump’s view is much closer to “the ends justify the means” when prioritizing speed, competitiveness, and economic dominance. 

Both stances are complicated by the fact that the power to execute on either vision is concentrated in the hands of a tiny circle of individuals with disproportionate control over the direction of the global AI rollout. 

But they don’t have to be diametrically opposed. Competitive success isn’t just about who gets more market share first. In the long term, it’s about whose products generate the most benefit to the society in which they are deployed – and that outcome can be enhanced by appropriate governmental oversight and adherence to ethical principles. 

The debate over the role of governance can no longer center on stifling or accelerating technical innovation. It has to consider larger and loftier questions of authority in the digital age: who is entitled to shape the systems that shape society, and how should they be accountable to the values and needs of the people those systems serve? 

The challenge is not to let oligarchic influence and personality politics get in the way of deploying a sensible, democratic, socially responsible AI ecosystem.  Ultimately, if we are able to look beyond how a handful of CEOs envision the AI-enabled world and consider how to balance the complex, symbiotic relationships of technology and society, we will be more able to leverage AI tools for the greater good. 

Jennifer Bresnick is a journalist and freelance content creator with a decade of experience in the health IT industry.  Her work has focused on leveraging innovative technology tools to create value, improve health equity, and achieve the promises of the learning health system.  She can be reached at [email protected].