CASE STUDY // Business Ethics 3

Ethical dilemma of who survives self-driving car accident Publication info: The New Zealand Herald ; Auckland, New Zealand [Auckland, New Zealand]03 Jan

2019: B.3.

ProQuest document link

FULL TEXT Imagine this scenario —the brakes fail on a self-driving car as it hurtles toward a busy crossing.

A homeless person and a criminal are crossing in front of the car. Two cats are in the opposing lane.

Should the car swerve to mow down the cats or hit two people?

It’s a relatively straightforward ethical dilemma, as moral quandaries go. And people overwhelmingly prefer to save

human lives over animals, according to a new ethics study that asked people how a self-driving car should respond

when faced with a variety of extreme trade-offs —dilemmas to which more than two million people responded.

But what if the choice is between two elderly people and a pregnant woman? An athletic person or someone who is

obese?

The study identified a few preferences that were strongest. People opt to save people over pets, to spare the many

over the few, and to save children and pregnant women over older people. But it also found other preferences for

sparing women over men, athletes over obese people, and higher status people, such as executives, instead of

homeless people or criminals. There were also cultural differences in the degree, for example, that people would

prefer to save younger people over the elderly in a cluster of mostly Asian countries.

“We don’t suggest that [policymakers] should cater to the public’s preferences. They just need to be aware of it, to

expect a possible reaction when something happens. If, in an accident, a kid does not get special treatment, there

might be some public reaction,” said Edmond Awad, a computer scientist at the Massachusetts Institute of

Technology Media Lab who led the work.

The thought experiments posed by the researcher’s Moral Machine website went viral, with their pictorial quiz

taken by several million people in 233 countries or territories.

Outside researchers said the results were interesting, but cautioned that the results could be overinterpreted. In a

randomised survey, researchers try to ensure a sample is unbiased and representative of the overall population,

but in this case the voluntary study was taken by a population that was predominantly younger men. The

scenarios are also distilled, extreme and far more black and white than the ones in the real world.

“The big worry I have is that people reading this are going to think this study is telling us how to implement a

decision process for a self-driving car,” said Benjamin Kuipers, a computer scientist at University of Michigan, who

was not involved in the work.

Kuipers added that these thought experiments may frame some of the decisions car makers and programmers

make about autonomous vehicle design in a misleading way. There’s a moral choice, he argued, that precedes the

conundrum of whether to crash into a barrier and kill three passengers or to run over a pregnant woman pushing a

stroller.

“Building these cars, the process is not really about saying, ‘if I’m faced with this dilemma, who am I going to kill.’

It’s saying, ‘if we can imagine a situation where this dilemma could occur, what prior decision should I have made

to avoid this?” he said.

The complexity of the world can be captured by the example of a criminal versus a dog. While many said they

would save the canine over its human counterpart, this overlooks the nuanced reasons why a person might be

PDF GENERATED BY PROQUEST.COM Page 1 of 3

driven to a life of crime.

Nicholas Evans, a philosopher at the University of Massachusetts, pointed out that while the researchers

described their three strongest principles as the ones that were universal, the cut-off between those and the

weaker ones that weren’t deemed universal was arbitrary. They categorised the preference to spare young people

over elderly people, for example, as a global moral preference, but not the preference to spare those who are

following walk signals versus those who are jaywalking, or to save people of higher social status.

Evans is working on a project that he said has been influenced by the approach taken by the MIT team. He says he

plans to use more nuanced crash scenarios, where real-world transportation data can provide a probability of

surviving a T-bone highway crash on the passenger side, for example, to assess the safety implications of self-

driving cars.

“We want to create a mathematical model for some of these moral dilemmas and utilise the best moral theories

that philosophy has to offer, to show what the result of choosing an autonomous vehicle to behave in a certain

way is,” Evans said.

Iyad Rahwan, a computer scientist at MIT who oversaw the work, said that a public poll shouldn’t be the

foundation of artificial intelligence ethics. But he said that regulating AI will be different from traditional products,

because the machines will have autonomy and the ability to adapt, making it more important to understand how

people perceive AI and what they expect of technology.

“We should take public opinion with a grain of salt,” Rahwan said. “I think it’s informative.”

—Washington Post and NZ Herald DETAILS

Subject: Preferences; Researchers; Older people; Ethics; Homeless people; Autonomous

vehicles; Traffic accidents &safety

Location: Massachusetts

Company / organization: Name: Massachusetts Institute of Technology; NAICS: 611310; Name: University of

Michigan; NAICS: 611310

Publication title: The New Zealand Herald; Auckland, New Zealand

First page: B.3

Publication year: 2019

Publication date: Jan 3, 2019

Section: Business

Publisher: New Zealand Media and Entertainment, NZME

Place of publication: Auckland, New Zealand

Country of publication: New Zealand, Auckland, New Zealand

Publication subject: General Interest Periodicals--New Zealand

PDF GENERATED BY PROQUEST.COM Page 2 of 3

Database copyright  2020 ProQuest LLC. All rights reserved. Terms and Conditions Contact ProQuest

ISSN: 11700777

Source type: Newspapers

Language of publication: English

Document type: News

ProQuest document ID: 2162490832

Document URL: https://search.proquest.com/docview/2162490832?accountid=142908

Copyright: Copyright New Zealand Media and Entertainment, NZME Jan 3, 2019

Last updated: 2019-07-01

Database: ProQuest Central

PDF GENERATED BY PROQUEST.COM Page 3 of 3