Fill worksheet

19 JANUARY 2018 • VOL 359 ISSUE 6373 2 6 3S C I E N C E sciencemag.org

Are algorithms good judges? People are as good as machines in predicting rearrest

CRIMINAL JUSTICE

E very day, judges across the United

States face an important decision:

Should they jail a defendant whose

innocence or guilt has not yet been

determined, or should they release

that person back into the community,

where he or she might commit a crime?

Increasingly, courts are turning to comp-

uter-based tools to help make those deci-

sions, lured by the promise of complex

algorithms that use an array of factors to

spit out risk scores. But a new study sug-

gests that at least one widely used algo-

rithm produces risk assessments that are no

better than those reached by people given

just a few key pieces of infor-

mation about a defendant.

The result, published this

week in Science Advances,

challenges widespread as-

sumptions that algorithms

are better at calculating risk

than humans, researchers

say. “A fancy model isn’t nec-

essarily a better model,” says

David Robinson, a legal scholar at George-

town University in Washington, D.C.

Julia Dressel, a computer science major

at Dartmouth College, got interested in risk

scoring algorithms after reading a 2016 se-

ries by investigative reporters at ProPublica

about one popular system, the Correctional

Offender Management Profiling for Alter-

native Sanctions (COMPAS), which is used

in at least five states and can cost up to

$22,000 a year. After examining COMPAS

scores for 10,000 defendants awaiting trial

in Broward County in Florida, as well as

their arrest records over the next 2 years,

ProPublica concluded that the tool dispro-

portionately classified black offenders as

being at high risk of rearrest.

That finding concerned civil rights advo-

cates, but Dressel went on to ask another

question: Are humans or machines better

at assessing the risk of rearrest? To find

out, she randomly selected 1000 of the de-

fendants and recorded seven pieces of in-

formation about each, including their age,

sex, and number of previous arrests. She

then recruited 400 people using the online

crowdsourcing service Amazon Mechanical

Turk. Each received profiles of 50 defen-

dants and was asked to predict whether

they would be rearrested within 2 years, the

same standard COMPAS uses. The human

judges were right about 63% to 67% of the

time, compared with about 65% of the time

for COMPAS.

Dressel was surprised. So was Megan

Stevenson, an economist and legal scholar

at George Mason University in Arlington,

Virginia, who calls the study the first to run

a “horse race” between human and algo-

rithm. She always assumed that algorithms

were somewhat better, so the results left her

“quite shocked.”

In a second experiment, Dressel and

her adviser, Dartmouth computer scientist

Hany Farid, explored whether a simpler

algorithm could beat COMPAS’s, which is

proprietary but described

in technical literature. They

created their own, ultimately

settling on just two factors:

age and number of prior

convictions. Plugging that

information into a simple for-

mula yielded predictions that

were about 67% accurate,

roughly matching COMPAS.

Robinson says that result reflects some-

thing that has long been known in crimi-

nology: If you’re young, you’re risky.

Mathematician Tim Brennan, who cre-

ated COMPAS in 1998 when working at

Northpointe (now Equivant) in Canton,

Ohio, says that far from undercutting COM-

PAS, the new study validates his approach.

Seventy percent accuracy, he says, has long

been considered the “speed limit” of such

prediction systems, and the fact that hu-

mans did no better is encouraging.

But humans are no better than machines

at eliminating bias, notes mathematician

Cathy O’Neil, founder of the risk consult-

ing and auditing firm O’Neil Risk Consult-

ing & Algorithmic Auditing in New York

City. Dressel’s study, for example, found

that people were just as likely as COM-

PAS to overstate rearrest risks for black

defendants and understate risks for white

defendants. That’s troubling, given that

similar algorithms are increasingly influ-

encing not just court decisions, but also

loan approvals, teacher evaluations, and

even whether child abuse charges are in-

vestigated by the state. “People get awed by

mathematical sophistication,” O’Neil says,

“but it’s mostly a distraction.” j

By Catherine Matacic

department since 2014, wrote in an email to

Science. “While some of Jaeger’s behavior as

a junior faculty member was inappropriate,

he clearly wasn’t a sexual predator.”

The complainants, who refused to be

interviewed by White’s firm because of

the ongoing litigation, strongly disagreed.

“The thrust of their report is to admit that

many bad things happened at UR—but

miraculously … no legal liability attaches

to the university,” the plantiffs’ lawyer, Ann

Olivarius of McAllister Olivarius in Maid-

enhead, U.K., said at an 11 January news

conference. “In fact, there is substantial

case law the report ignores that strongly

supports the idea that the university is ab-

solutely liable for the hostile environment

created by Jaeger’s actions.”

The report does not evaluate the ef-

fect of Jaeger’s behavior in the aggregate,

which the court might do, notes Alexandra

Tracy-Ramirez, a lawyer at HopkinsWay

in Phoenix. The plaintiffs also dispute de-

tails of White’s account. On 13 January,

they wrote to the faculty Senate Executive

Committee, saying the report suppresses

and misrepresents evidence. “In court, our

audio recordings … will establish blatant

factual inaccuracies in Ms. White’s narra-

tive,” they add. Their suit names the uni-

versity, Seligman, and Provost Robert Clark.

In addition to retaliation and defamation,

it charges that the school allowed a hostile

environment for three plaintiffs. University

attorneys are expected to file their first re-

sponse in early February.

The BCS department now faces the task

of healing and rebuilding. Of the nine

plaintiffs, seven have left or will soon leave

the university. Two, Piantadosi and Celeste

Kidd, assistant professors who are married

to each other, are searching for positions.

The BCS website lists 40 remaining faculty.

Recruiting students may be difficult,

given an open letter signed late last year

by 454 professors in cognitive neurosci-

ence and other disciplines. The signatories

vowed not to encourage students to attend

the university because it had “abrogated” its

duty to students “by supporting the preda-

tor and intimidating the victims.”

Few dispute that the department has

been damaged by the case. “It was known

as one of the top cognitive neuroscience

departments in the country,” says cogni-

tive neuroscientist Timothy Verstynen of

Carnegie Mellon University in Pittsburgh,

Pennsylvania, who signed the letter. “They

really did take a hit by losing some strong

key faculty.” The Board of Trustees said last

week it will need time to digest the lengthy

report before taking action. It also faces the

task of replacing Seligman, whom it called a

“brilliant, transformative leader.” j

“A fancy model isn’t necessarily a better model.” David Robinson,

Georgetown University

DA_0119NewsInDepth.indd 263 1/17/18 11:46 AM

Published by AAAS

o n Ju

n e 2

6 , 2

0 1 8

h ttp

://scie n ce

.scie n ce

m a g .o

rg /

D o w

n lo

a d e d fro

m

Are algorithms good judges? Catherine Matacic

DOI: 10.1126/science.359.6373.263 (6373), 263.359Science

ARTICLE TOOLS http://science.sciencemag.org/content/359/6373/263

CONTENT RELATED http://advances.sciencemag.org/content/advances/4/1/eaao5580.full

PERMISSIONS http://www.sciencemag.org/help/reprints-and-permissions

Terms of ServiceUse of this article is subject to the

is a registered trademark of AAAS.Science licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. The title Science, 1200 New York Avenue NW, Washington, DC 20005. 2017 © The Authors, some rights reserved; exclusive

(print ISSN 0036-8075; online ISSN 1095-9203) is published by the American Association for the Advancement ofScience

o n Ju

n e 2

6 , 2

0 1 8

h ttp

://scie n ce

.scie n ce

m a g .o

rg /

D o w

n lo

a d e d fro

m