A self-driving taxi has no passengers, so it parks itself in a lot to reduce congestion and air pollution. After being hailed, the taxi heads out to pick up its passenger – and tragically strikes a pedestrian in a crosswalk on its way.

Who or what deserves praise for the car’s actions to reduce congestion and air pollution? And who or what deserves blame for the pedestrian’s injuries?

One possibility is the self-driving taxi’s designer or developer. But in many cases, they wouldn’t have been able to predict the taxi’s exact behavior. In fact, people typically want artificial intelligence to discover some new or unexpected idea or plan. If we know exactly what the system should do, then we don’t need to bother with AI.

Alternatively, perhaps the taxi itself should be praised and blamed. However, these kinds of AI systems are essentially deterministic: Their behavior is dictated by their code and the incoming sensor data, even if observers might struggle to predict that behavior. It seems odd to morally judge a machine that had no choice.

According to many modern philosophers, rational agents can be morally responsible for their actions, even if their actions were completely predetermined – whether by neuroscience or by code. But most agree that the moral agent must have certain capabilities that self-driving taxis almost certainly lack, such as the ability to shape its own values. AI systems fall in an uncomfortable middle ground between moral agents and nonmoral tools.

As a society, we face a conundrum: it seems that no one, or no one thing, is morally responsible for the AI’s actions – what philosophers call a responsibility gap. Present-day theories of moral responsibility simply do not seem appropriate for understanding situations involving autonomous or semi-autonomous AI systems.

If current theories will not work, then perhaps we should look to the past – to centuries-old ideas with surprising resonance today.

God and man

A similar question perplexed Christian theologians in the 13th and 14th centuries, from Thomas Aquinas to Duns Scotus to William of Ockham. How can people be responsible for their actions, and the results, if an omniscient God designed them – and presumably knew what they would do?

Medieval philosophers held that someone’s decisions result from their will, operating on the products of their intellect. Broadly speaking, they understood human intellect as a set of mental capabilities that enable rational thought and learning.

Intellect is the rational, logical part of people’s minds or souls. When two people are presented with identical situations and they both arrive at the same “rational conclusion” about how to handle things, they’re using intellect. Intellect is like computer code in this way.

But the intellect doesn’t always provide a unique answer. Often, the intellect provides only possibilities, and the will selects among them, whether consciously or unconsciously. Will is the act of freely choosing from among the possibilities.

As a simple example, on a rainy day, intellect dictates that I should grab an umbrella from my closet, but not which one. Will is choosing the red umbrella instead of the blue one.

For these medieval thinkers, moral responsibility depended on what the will and the intellect each contribute. If the intellect determines that there is only one possible action, then I could not do otherwise, and so I am not morally responsible. One might even conclude that God is morally responsible, since my intellect comes from God – though the medieval theologians were very cautious about attributing responsibility to God.

On the other hand, if intellect places absolutely no constraints on my actions, then I am fully morally responsible, since will is doing all of the work. Of course, most actions involve contributions from both intellect and will – it’s usually not an either/or.

In addition, other people often constrain us: from parents and teachers to judges and monarchs, especially in the medieval philosophers’ days – making it even more complicated to attribute moral responsibility.

Man and AI

Clearly, the relationship between AI developers and their creations is not exactly the same as between God and humans. But as professors of philosophy and computing, we see intriguing parallels. These older ideas might help us today think through how an AI system and its designers might share moral responsibility.

AI developers are not omniscient gods, but they do provide the “intellect” of the AI system by selecting and implementing its learning methods and response capabilities. From the designer’s perspective, this “intellect” constrains the AI’s behavior but almost never determines its behavior completely.

Most modern AI systems are designed to learn from data and can dynamically respond to their environments. The AI will thus seem to have a “will” that chooses how to respond, within the constraints of its “intellect.”

Users, managers, regulators and other parties can further constrain AI systems – analogous to how human authorities such as monarchs constrain people in the medieval philosophers’ framework.

Who’s responsible?

These thousand-year-old ideas map surprisingly well to the structure of moral problems involving AI systems. So let’s return to our opening questions: Who or what is responsible for the benefits and harms of the self-driving taxi?

The details matter. For example, if the taxi developer explicitly writes down how the taxi should behave around crosswalks, then its actions would be entirely due to its “intellect” – and so the developers would be responsible.

However, let’s say the taxi encountered situations it was not explicitly programmed for – such as if the crosswalk was painted in an unusual way, or if the taxi learned something different from data in its environment than what the developer had in mind. In cases like these, the taxi’s actions would be primarily due to its “will,” because the taxi selected an unexpected option – and so the taxi is responsible.

If the taxi is morally responsible, then what? Is the taxi company liable? Should the taxi’s code be updated? Even the two of us do not agree about the full answer. But we think that a better understanding of moral responsibility is an important first step.

Medieval ideas are not only about medieval objects. These theologians can help ethicists today better understand the present-day challenge of AI systems – though we have only scratched the surface.

(C) THE CONVERSATION

BIO:

David Danks is Professor of Data Science, Philosophy, & Policy, and affiliate faculty in Computer Science & Engineering, at University of California, San Diego. His research interests range widely across philosophy, cognitive science, and machine learning, including their intersection. Danks has examined the ethical, psychological, and policy issues around AI and robotics across multiple sectors, including transportation, healthcare, privacy, and security. He has also done significant research in computational cognitive science and developed multiple novel causal discovery algorithms for complex types of observational and experimental data. Danks is the recipient of a James S. McDonnell Foundation Scholar Award, as well as an Andrew Carnegie Fellowship. He currently serves on multiple advisory boards, including the National AI Advisory Committee. Before moving to UCSD in 2021, Danks was the L.L. Thurstone Professor of Philosophy & Psychology at Carnegie Mellon University. He received a Ph.D. in Philosophy from UCSD, and an A.B. in Philosophy from Princeton University.

Dr. Mike Kirby received his B.Sc. in Applied Mathematics and Computer and Information Sciences from The Florida State University in 1997. He received his M.Sc. in Applied Mathematics from Brown University in 1999 and received a M.Sc. in Computer Science from Brown University in 2001. In August 2002 he completed his Ph.D. in Applied Mathematics at Brown University.

Dr. Kirby has been with the university since 2002, moving up the ranks from assistant professor to associate professor in 2008 to full professor in 2014 within the School of Computing. He currently is the Acting Director of the Scientific Computing and Imaging (SCI) Institute and the founding Executive Director of the Utah Informatics Initiative (UI2). He also currently serves as the Director and Program Manager of the Army Research Laboratory (ARL) Multi-Scale Multidisciplinary Modeling of Electronic Materials (MSME) Collaborative Research Alliance (CRA) which is led out of Utah. His research interests include scientific and data computing, visualization, uncertainty quantification, high-performance computing and computational engineering and science applications.