week-5 Essay

Chapter:-1

1.1 The Pace of Change

· In a way not seen since Gutenberg’s printing press that ended the Dark Ages and ignited the Renaissance, the microchip is an epochal technology with unimaginably far-reaching economic, social, and political consequences.

· —Michael Rothschild 1

In 1804, Meriwether Lewis and William Clark set out on a two-and-a-half-year voyage to explore what is now the western United States. Many more years passed before their journals were published. Later explorers did not know that Lewis and Clark had been there before them. Stephen Ambrose points out in his book about the Lewis and Clark expedition, Undaunted Courage, that information, people, and goods moved no faster than a horse—and this limitation had not changed in thousands of years. 2  In 1997, millions of people went to the World Wide Web to watch a robot cart called Sojourner roll across the surface of Mars. We chat with people thousands of miles away, and instantly view Web pages from around the world. We can tweet from airplanes flying more than 500 miles per hour.

Telephones, automobiles, airplanes, radio, household electrical appliances, and many other marvels we take for granted were invented in the late 19th and early 20th centuries. They led to profound changes in how we work and play, how we get information, how we communicate, and how we organize our family lives. Our entry into space was one of the most dramatic feats of technology in the 20th century. Sputnik, the first man-made satellite, launched in 1957. Neil Armstrong walked on the moon in 1969. We still do not have personal spacecraft, vacation trips to the moon, or a large amount of commercial or research activity in space. Space tourism for the very rich is in an early stage. The moon landing has had little direct effect on our daily lives. But computer systems in cars can now apply the brakes if a pedestrian is in the car’s path. Some cars park themselves, and experimental cars drive themselves on city streets. Computer programs beat human experts at chess and Jeopardy!, and our smartphones answer our questions. Surgeons perform surgery with robotic instruments miles from the patient. Roughly five billion people use cellphones; U.S. texters send more than a trillion texts in a year; Facebook has more than 800 million members; Twitter users tweet hundreds of thousands of times a day; and these numbers will be out of date when you read them. A day without using an appliance or device containing a microchip is as rare as a day without turning on an electric light.

The first electronic computers were built in the 1940s. Scientists at Bell Laboratories invented the transistor—a basic component of microprocessors—in 1947. The first hard-disk drive, made by IBM in 1956, weighed more than a ton and stored only five megabytes of data, less than the amount of space we use for one photo. Now, we can walk around with 150 hours of video in a pocket. A disk with a terabyte (one thousand gigabytes, or one trillion bytes) of storage—enough for 250 hours of high definition video—is inexpensive. There are hundreds of billions of gigabytes of information on the Internet. The 1991 space shuttle had a 1-megahertz *  computer onboard. Ten years later, some luxury automobiles had 100-megahertz computers. Speeds of several gigahertz are now common. When I started my career as a computer science professor, personal computers had not yet been invented. Computers were large machines in air-conditioned rooms; we typed computer programs onto punched cards. If we wanted to do research, we went to a library, where the library catalog filled racks of trays containing 3 × 5 index cards. Social-networking sites were neighborhood pizza places and bars. The point is not that I am old; it is the speed and magnitude of the changes. The way you use computer systems and mobile devices, personally and professionally, will change substantially in two years, in five, and in ten, and almost unrecognizably over the course of your career. The ubiquity of computers, the rapid pace of change, and their myriad applications and impacts on daily life characterize the last few decades of the 20th century and the beginning of the 21st.

It is not just the technology that changes so fast. Social impacts and controversies morph constantly. With PCs and floppy disks came computer viruses and the beginnings of a huge challenge to the concept of copyright. With email came spam. With increased storage and speed came databases with details about our personal and financial lives. With the Web, browsers, and search engines came easy access by children to pornography, more threats to privacy, and more challenges to copyright. Online commerce brought bargains to consumers, opportunities to entrepreneurs, and identity theft and scams. Cellphones have had so many impacts that we discuss them in more detail later in this chapter and in  Chapter 2 . With hindsight, it might seem odd that people worried so much about antisocial, anticommunity effects of computers and the early Internet. Now, with the popularity of social networking, texting, and sharing video, photos, and information, the Net is a very social place. In 2008, “experts” worried the Internet would collapse within two years because of the demands of online video. It did not. Privacy threats of concern several years ago seem minor compared to new ones. People worried about how intimidating computers and the Internet were; now toddlers operate apps on tablets and phones. Concerns about technology “haves” and “have-nots” (the “digital divide”) waned as Internet access and cellphones spread throughout the United States and around the world, shrinking the digital divide far faster than long-standing global divides in, say, education and access to fresh water.

* This is a measure of processing speed. One megahertz is 1 million cycles per second; 1 gigahertz is 1 billion cycles per second. “Hertz” is named for the 19th-century physicist Heinrich Rudolf Hertz.

Discussions of social issues related to computers often focus on problems, and indeed, throughout this book we examine problems created or intensified by computer technologies. Recognizing the benefits is important too. It is necessary for forming a reasonable, balanced view of the impact and value of the technology. Analyzing and evaluating the impact of new technologies can be difficult. Some of the changes are obvious. Some are more subtle. Even when benefits are obvious, the costs and side effects might not be, and vice versa. Both the technological advances brought about by computer technology and the extraordinary pace of development have dramatic, sometimes unsettling, impacts on people’s lives. To some, this is frightening and disruptive. They see the changes as dehumanizing, reducing the quality of life, or as threats to the status quo and their well-being. Others see challenging and exciting opportunities. To them, the development of the technology is a thrilling and inspiring example of human progress.

When we speak of computers in this book, we include mobile devices such as smartphones and tablets, desktop computers and mainframes, embedded chips that control machines (from sewing machines to oil refineries), entertainment systems (such as video recorders and game machines), and the “Net,” or “cyberspace.” Cyberspace is built of computers (e.g., Web servers), communication devices (wired and wireless), and storage media, but its real meaning is the vast web of communications and information that includes the Internet and more.

In the next section, we look at some phenomena, often unplanned and spontaneous, that computer and communication technology made possible. They have deeply changed how we interact with other people, what we can accomplish, and how others can intrude into our relationships and activities. In the rest of the chapter, we introduce themes that show up often, and we present an introduction to some ethical theories that can help guide our thinking about controversies throughout the rest of the book. The next seven chapters look at ethical, social, and legal issues primarily from the perspective of any person who lives and works in a modern computerized society and is interested in the impact of the technology. The final chapter takes the perspective of someone who works as a computer professional who designs or programs computer systems or as a professional in any area who must make decisions and/or set policy about the use of computer systems. It explores the ethical responsibilities of the professional. The Software Engineering Code of Ethics and Professional Practice and the ACM Code of Ethics and Professional Conduct, in Appendix A, provide guidelines for professionals.

1.2 Change and Unexpected Developments

No one would design a bridge or a large building today without using computers, but the Brooklyn Bridge, built more than 130 years ago—long before computers, is both a work of art and a marvelous feat of engineering. The builders of the Statue of Liberty, the Pyramids, the Roman aqueducts, magnificent cathedrals, and countless other complex structures did not wait for computers. People communicated by letters and telephone before text messages, email, and Twitter. People socialized in person before social-networking sites. Yet we can identify several phenomena resulting from computer and communication technology that are far different from what preceded them (in degree, if not entirely in kind), several areas where the impacts are dramatic, and many that were unanticipated. In this section, we consider a brief sampling of such phenomena. Some are quite recent. Some are routine parts of our lives now. The point is to remind us that a generation ago they did not exist. They illustrate the amazingly varied uses people find for new tools and technologies.

· It is precisely this unique human capacity to transcend the present, to live one’s life by purposes stretching into the future—to live not at the mercy of the world, but as a builder and designer of that world—that is the distinction between human and animal behavior, or between the human being and the machine.

· —Betty Friedan 3

1.2.1 Connections: Cellphones, Social Networking, and More

The Web, social networking, cellphones, and other electronic devices keep us connected to other people and to information all day, virtually everywhere. We look at a few connectivity applications, focusing on fast changes and unanticipated uses and side effects (good and bad). The discussion suggests issues we study throughout the book.

Cellphones

In the 1990s, relatively few people had cellphones. Business people and sales people who often worked outside their office carried them. High-tech workers and gadget enthusiasts liked them. Others bought the phones so they could make emergency calls if their cars broke down. We were used to being out of touch when away from home or office. We planned ahead and arranged our activities so that we did not need a phone when one was not available. Within a short time, however, cell service improved and prices dropped. Cellphone makers and service providers developed new features and services, adding cameras, video, Web connections, and location detection. Apple introduced the iPhone in 2007, and phones got “smart.” People quickly developed hundreds of thousands of applications and embraced the term app. Consumers downloaded 10 billion apps from Apple’s App Store. Within very few years, people all over the world used phones, rather than PCs or laptops, as their connection to the Internet. Millions, then hundreds of millions, then billions of people started carrying mobile phones. In 2011, there were approximately five billion cellphone subscriptions worldwide—an astoundingly fast spread of a new technology. Writers describe the dramatic changes with observations such as, “A Masai warrior with a smartphone and Google has access to more information than the President did 15 years ago” and “More folks have access to a cellphone than to a toilet.” 4

Cellphones became a common tool for conversations, messaging, taking pictures, downloading music, checking email, playing games, banking, managing investments, finding a restaurant, tracking friends, watching videos. Smartphones serve as electronic wallets and identification cards at store terminals or security checkpoints. Phones monitor security cameras at home or control home appliances from a distance. Professional people use smartphone apps for a myriad of business tasks. Smartphones with motion detectors remind obese teenagers to get moving. An app analyzes blood glucose levels for diabetics and reminds them when to exercise, take medication, or eat something. Military personnel on the front lines can use specialized apps to download satellite surveillance video and maps. More unanticipated uses include location tracking, sexting, life-saving medical apps, and malicious data-stealing apps. People use cellphones to organize flash mobs for street dances and pillow fights—or for attacking pedestrians and looting stores. Terrorists use cellphones to set off bombs. Apps designed for poor countries inform people when water is available and help perform medical imaging.

These examples suggest the number and variety of unanticipated applications of this one, relatively new “connection” device. The examples also suggest problems. We discuss privacy invasion by data theft and location tracking in  Chapter 2 . In  Chapter 3 , we consider whether phone service should be shut down during riots. Is the security of smartphones sufficient for banking and electronic wallets? (What if you lose your phone?) Do people realize that when they synch their phone with other devices, their files become vulnerable at the level of the weakest security?

As a side effect of cellphone use and the sophistication of smartphones, researchers are learning an enormous amount about our behavior. Laws protect the privacy of the content of our conversations, but smartphones log calls and messages and contain devices that detect location, motion, direction, light levels, and other phones nearby. Most owners carry their phones all day. Researchers analyze this trove of sensor data. (Yes, much of it can be stored.) Analysis of the data generates valuable information about traffic congestion, commuting patterns, and the spread of disease. In an example of the latter, by studying movement and communication patterns of MIT students, researchers could detect who had the flu, sometimes before the students knew it themselves. Researchers also can determine which people influence the decisions of others. Advertisers and politicians crave such information. Perhaps the eeriest result is that reseachers who analyzed time and location data from millions of calls said that, with enough data, a mathematical model could predict where someone would be at a particular future time with more than 90% accuracy. Who will have access to that information? 5

* At least one company sells a working pen that records high-resolution video.

Rudeness is an issue with cellphones. People use them in inappropriate places, disturbing others. The fact that so many people carry small cameras everywhere (mostly in phones, but also hidden in other small objects such as pens * ) affects our privacy in public and nonpublic places. 6  How well do people armed with cellphone cameras distinguish news events and evidence of crimes from voyeurism, their own rudeness, and stalking?

Talking on a phone while driving a car increases the risk of an accident. Some states prohibit use of handheld phones while driving (and a lot of drivers ignore the ban). Researchers developed an app that uses motion detection by smartphones to deduce that a phone is in a moving car and block incoming calls. A more sophisticated version locates the phone well enough to block only the driver’s phone, not that of a passenger.

Here is an example of a subtle behavioral change. When people began carrying cellphones and could call for help, more headed out in the wilderness or went rock climbing without appropriate preparation. In many areas of life, people take more risk when technology increases safety. This is not unreasonable if the added risk and increased safety are in balance. When rescue calls surged, some rescue services began billing for the true cost of a rescue—one way to remind people to properly weigh the risk.

Kill switches

Soon after Amazon began selling electronic books for its Kindle ebook readers, the company discovered that a publisher was selling books in Amazon’s online store that it did not have the legal rights to sell in the United States. Amazon deleted the books from its store and from the Kindles of people who had bought them; it refunded their payments. A reasonable and appropriate response? Not to many customers and media observers. Customers were outraged that Amazon deleted books from their Kindles. People were startled to learn that Amazon could do so. *  The response was so strong that Amazon announced that it would not remove books from customer Kindles again. Few realized at that time that Apple’s iPhones already had a kill switch—a way for Apple to remotely delete apps from phones. In 2011, when a software developer discovered malicious code in an app for Android phones, Google quickly removed the app from its store and from more than 250,000 phones. Although this was a good example of the purpose of a kill switch and a beneficial use, the fact that Google could do it disturbed people. One of the troubling side effects of our connectivity is that outsiders can reach into our devices and delete our stuff.

Perhaps this extended reach should not have been a surprise. In many businesses, the IT department has access to all desktop computers and can install—or delete—software. Software on personal computers and other electronic devices communicates with businesses and organizations regularly, without our direct command, to check for updates of software, news, and our friends’ activities. When we enable updates of software, a company remotely deletes old versions.

* Ironically, one of the books Amazon removed was George Orwell’s 1984—a novel about a totalitarian government that regularly sent documents down a “memory hole” to destroy them.

Now, the operating systems for smartphones, tablets, and some computers (e.g.,Windows) have kill switches. The companies do not disclose much information about them. The main purpose is security—to remove malicious software that the company discovers in an app after users have downloaded it. Indeed, companies such as Google and Apple that provide popular app stores see it as a serious responsibility to protect users from malicious apps. Some companies tell us about their removal capability in their terms of use agreements, but such agreements can run to thousands of words and have vague, general statements. Few people read them.

What are some potential uses and risks? Kill switches could remove content that infringes copyrights. They could remove content that a company or government deems offensive. What if malicious hackers found a way to operate the kill switches on our devices? Governments in many countries have extensive censorship laws and require that communications services provide government access to communications. Governments, in free and unfree countries, pressure businesses to act as the government prefers. For more than 2000 years, governments and religious and social organizations have burned books that displeased them. What pressures might governments put on companies to use the kill switches? Will the impact of electronic kill switches be more devastating than attempts to prohibit printed material? Or will companies use them carefully for improved security? Our new tools are remarkably powerful and remarkably vulnerable.

Social networking

· While all this razzle-dazzle connects us electronically, it disconnects us from each other, having us “interfacing” more with computers and TV screens than looking in the face of our fellow human beings. Is this progress?

· —Jim Hightower, radio commentator, 1995 7

Facebook, one of the first of the social networking sites, started at Harvard as an online version of the hardcopy student directories available at many colleges. At first, the sites were wildly popular with young people, while older people did not understand the appeal or worried about safety and privacy. Adults quickly discovered benefits of personal and business social networking. Social networks are enormously popular with hundreds of millions of people because of the ease with which they can share so many aspects of their lives and activities with family, friends, co-workers, and the public.

As with so many other digital phenomena, people found unanticipated uses of social networking, some good, some bad. Friends and ex-boyfriends and ex-girlfriends post pranks and embarrassing material. Stalkers and bullies stalk and bully. Politicians, advertisers, businesses, and organizations seek donations, volunteers, customers, and connections. Protesters organize demonstrations and revolutions. Jurors tweet about court cases during trials (causing mistrials, overturned convictions, and jail time for offending jurors). Social networking brought us more threats to privacy and a steady stream of updates on the trivial details of people’s lives. Gradually, social network companies developed sophisticated privacy controls and feedback systems to reduce problems, though they certainly have not eliminated them. Overall, to most people, the benefits outweigh the problems, and social networking has become the new way of communicating.

 Privacy issues for socialnetworks:  Section 2.3.2

In a phenomenon called “crowd funding,” social networks, Twitter, and other platforms make it easy to raise money in small amounts from a large number of people for charities, political causes, artistic projects, and investment in start-up companies.

How do social networking sites affect people and relationships? People can have hundreds of friends and contacts, but have they traded quality of in-person relationships for quantity of superficial digital relationships? Does the time spent online reduce the time spent on physical activity and staying healthy? It is still too early for definitive answers, but it appears that the many critics who anticipated a serious problem of social isolation were mistaken. Researchers find that people use social networks mostly to keep in touch with friends and family and that the easy, frequent contact enhances relationships, empathy, and a sense of community. On the other hand, young people who spend a lot of time on a social network do poorly in school and have behavioral problems. (Are these people who would have problems in any case? Does the access to the networks exacerbate preexisting emotional problems?)

Just as researchers study social phenomena using the masses of data that smartphone systems collect, they also mine the masses of data in social networks. For example, social scientists and computer scientists analyze billions of connections to find patterns that could help identify terrorist groups. 8

A person you follow in social media might not be a person at all. A socialbot is an artificial intelligence program that simulates a human being in social media. Researchers tricked Twitter users into building relationships with artificial tweeting personalities, some of which gained large followings. Political activists launched socialbots to influence voters and legislators. The U.S. military raised concerns about automated disinformation campaigns by enemies. Advertising bots are likely to be common. When the Internet was new, someone commented (and many repeated) that “on the Internet, no one knows you’re a dog.” It meant that we could develop relationships with others based on common interests without knowing or caring about age, race, nationality, gender, or physical attractiveness. Some of those others might not even be people, and we might not know it. Should we be comfortable with that?

More about artificial intelligence:  Section 1.2.3

Communication and the Web

Email and the Web are so much a part of our culture now that we might forget how new and extraordinary they are. Email was first used mostly by computer scientists. In the 1980s, messages were short and contained only text. As more people and businesses connected to computer networks, use of email expanded to science researchers, then to businesses, then to millions of other people. Limits on length disappeared, and we began attaching digitized photos and documents. People worldwide still send several billion emails daily (not counting spam), although texting, tweeting, and other social media have replaced email as the favored communication method in many contexts. 9

High-energy physicists established the World Wide Web in Europe in 1990 to share their work with colleagues and researchers in other countries. In the mid- and late 1990s, with the development of Web browsers and search engines, the Web became an environment for ordinary users and for electronic commerce. Today there are billions of Web pages. The Web has grown from an idea to a huge library and news source, a huge shopping mall, an entertainment center, and a multimedia, global forum in less than one generation.

The Web gives us access to information and access to audiences unimaginable a generation ago. It empowers ordinary people to make better decisions about everything from selecting a bicycle to selecting medical treatments. It empowers us to do things that we used to rely on experts to do for us. Software tools, many available for free, help us analyze the healthiness of our diet or plan a budget. We can find references and forms for legal processes. We can read frank reviews of cameras, clothing, cars, books, and other products written by other buyers, not marketing departments. We can select our entertainment and watch it when we want to. We can fight back against powerful institutions by shaming them with videos that go viral *  (see, for example, “United Breaks Guitars” on YouTube) or by posting legal documents intended to intimidate us (see, for example, chillingeffects.org). Businesses and organizations use “viral marketing”—that is, relying on large numbers of people to view and spread marketing messages in clever videos. We can start our own Web-based television network without the huge investment and government license requirements of broadcast television networks. A college student with a good idea and some well-implemented software can start a business that quickly grows to be worth millions or billions of dollars; several have. The openness of the Internet enables “innovation without permission,” in the words of Vinton Cerf, one of the key people who has worked on Internet development since it began. 10

* “Going viral” describes the phenomenon where something posted in cyberspace catches the attention of people who view, copy, and spread it (or links to it) to millions more people.

Blogs (a word made up from “Web log”) and videos are two examples of the many new forms of creativity that flourish because Web technology and special software make them so easy and inexpensive. They began as outlets for amateurs and now are significant sources of news and entertainment. They have created new paths for jobs—with news media, publishers, and advertising and entertainment companies. Of course, some amateur blogs and videos are dull, silly, and poorly written or made, but many are gems, and people find them. People blog on current events, celebrity gossip, hobbies, books, movies, dieting, law, economics, technology, political candidates, Internet issues, and virtually any other topic. They provide varied, sometimes quirky perspectives. The independence of bloggers attracts readers; it suggests a genuine connection with what ordinary people are thinking and doing, not filtered through major news companies or governments. Businesses were quick to recognize the value of blogs, and many provide their own as part of their public relations and marketing programs. Inexpensive video cameras and video-manipulation tools have powered a burst of short amateur videos—often humorous, sometimes worthless, and sometimes quite serious. We can see a soldier’s view of war, someone’s encounter with aggressive whales, an arrest by police. Video sites also made it easy to post and trade professional videos, infringing copyrights owned by entertainment companies and individuals. We explore copyright issues in  Chapter 4 .

“I’ve got pressure”

When asked by a young man to speak more quietly on his cellphone, a Hong Kong bus rider berated the man for nearly six minutes with angry insults and obscenities. In the past, a few other riders might have described the incident to friends, then soon forgotten it. But in this instance, another rider captured the scene on his cellphone. The video soon appeared on the Internet, and millions of people saw it. People provided subtitles in different languages, set the video to music, used clips as mobile-phone ringtones, and produced t-shirts with pictures and quotes. “I’ve got pressure” and other phrases from the rant slipped into conversations.

This incident reminds us that anything we do in a public place can be captured and preserved on video. But more, it illustrates how the Internet facilitates and encourages creativity and the quick creation and distribution of culture artifacts and entertainment, with the contribution of ideas, modifications, variations, improvements, and new works from thousands of people.

The Web connects students and teachers. At first, universities offered online courses within their area, benefitting people who work full-time, who have varying work schedules that conflict with normal class schedules, who have small children at home, or who cannot travel easily because of disabilities. Gradually a potential to revolutionize advanced education became clear. *  More than 100 million people have viewed the thousands of free lessons on sciences, economics, and other subjects at the online Khan Academy. When two artificial intelligence experts offered a Stanford University graduate course for free online, they expected 500–1000 students to sign up. They got 160,000 people from around the world, and more than 20,000 completed the course, which included automatically graded homework assignments and exams. 11

* For elementary education, it appears that regular classes and in-person teachers still have the advantage.

The impact of the connections provided by the Web and cellphones is more dramatic in remote or less developed areas of the world, many of which do not have landline telephones. Mountains and thick jungle, with no roads, separate villagers in one town in Malaysia from the next, but the villagers order supplies, check the market price of rice to get a good deal when selling their crop, and email family photos to distant relatives. Farmers in Africa get weather forecasts and instruction in improved farming methods. An Inuit man operates an Internet service provider for a village in the Northwest Territories of Canada, where temperatures drop to −40°F. Villagers in Nepal sell handicrafts worldwide via a website based in Seattle. Sales have boomed, more villagers have regular work, dying local arts are reviving, and some villagers can now afford to send their children to school.

Telemedicine

Telemedicine, or long-distance medicine, refers to remote performance of medical exams, analyses, and procedures using specialized equipment and computer networks. On long airplane flights, telemedicine can help treat a sick passenger and ascertain whether the plane needs to make an emergency landing. Prisons use telemedicine to reduce the risk of escape by dangerous criminals. Some small-town hospitals use video systems to consult with specialists at large medical centers—eliminating the expense, time, and possible health risk of transporting the patient to the medical center. A variety of health-monitoring devices send their readings from a patient’s home to a nurse over the Internet. This technology eliminates the expense, time, and inconvenience of more frequent visits, while enabling more regular monitoring of patients and helping to catch dangerous conditions early.

Telemedicine goes well beyond transmission of information. Surgeons in New York used video, robotic devices, and high-speed communication links to remotely remove a gall bladder from a patient in France. Such systems can save lives in emergencies and bring a high level of surgical skills to small communities that have no surgeons.

The Web abounds with examples of collaborative projects, some organized, such as Wikipedia *  (the online encyclopedia written by volunteers), some spontaneous. Scientists collaborate on research with scientists in other countries much more easily and more often than they could without the Internet. Informal communities of programmers, scattered around the world, create and maintain free software. Informal, decentralized groups of people help investigate online auction fraud, a murder, stolen research, and other crimes. People who have never met collaborate on creating entertainment.

* A wiki is a website, supported by special software, that allows people to add content and edit content that others provide. Wikis are tools for collaborative projects within a business or organization or among the public.

Some collaborative projects can have dangerous results. To reduce the flow of illegal immigrants, a governor of Texas proposed setting up night-vision webcams along the Mexican border that volunteers would monitor on the Internet. Will the people monitoring a border webcam go out and attack those they see coming across the border? What training or selection process is appropriate for volunteers who monitor these security cameras? In China, a man posted the online name of another man he believed was having an affair with his wife. Thousands of people participated in tracking down the man’s real name and address and encouraging public action against him. Thousands of Twitterers in Saudi Arabia called for the execution of a young writer who they believed insulted the Prophet Muhammad. Mobs and individuals emotionally involved in a political, religious, or moral cause do not always pause for the details of due process. They do not carefully determine whether they identified the correct person, whether the person is guilty of a crime, and what the appropriate punishment is. On the other hand, police departments in cities in several countries effectively use instant messaging to alert residents who help find crime suspects or stolen cars in their neighborhoods. Enlisting volunteers is a useful new collaborative tool for crime fighting and possibly antiterrorism programs. How can we guide the efforts of thousands of individuals toward useful ends while protecting against mistakes, instant vigilantism, and other abuses?

1.2.2 E-commerce and Free Stuff

In the 1990s, the idea of commercial websites horrified Web users. The Web, they believed, was for research, information, and online communities. A few brick-and-mortar businesses and a few young entrepreneurs recognized the potential and benefits of online commerce. Among the earliest traditional businesses on the Web, United Parcel Service and Federal Express let customers check the status of packages they sent. This was both a novelty and a helpful service. Amazon.com, founded in 1994, started selling books on the Web and became one of the most popular, reliable, and user-friendly commercial sites. Many, many Web-based businesses followed Amazon, creating new business models—such as eBay with its online auctions. Traditional businesses established websites. Online sales in the United States now total hundreds of billions of dollars a year. The Web changed from a mostly academic community to a world market in little more than a decade.

Some of the benefits of e-commerce are fairly obvious: we can consider more products and sellers, some far away, in less time and without burning gasoline. Some benefits are less obvious or were not obvious before they appeared. Auction sites gave people access to customers they could not have found efficiently before. The lower overhead and the ease of comparison shopping on the Web brought down prices of a variety of products. Consumers save 10–40%, for example, by buying contact lenses online, according to a Progressive Policy Institute report. Consumers who do price-comparison research on the Web before buying a new car typically save about $400. 12  Small businesses and individual artists sell on the Web without paying big fees to middlemen and distributors. The Web enabled a peer-to-peer economy with websites where ordinary people sell or trade their skills, make small loans, and trade their homes for vacations.

Growth of commerce on the Web required solutions to several problems. One was trust. People were reluctant to give their credit card numbers on the Web to companies they had not dealt with or even heard of before. Enter PayPal, a company built on the idea of having a trusted intermediary handle payments. Encryption and secure servers also made payments safer. *  The Better Business Bureau established a website where we can find out if consumers have complained about a company. Auction sites implemented rating and comment systems to help buyers and sellers determine whom to trust. Email confirmations of orders, consumer-friendly return policies, and easy packaging for returns all contributed to consumer comfort and more online sales. The University of Michigan’s National Quality Research Center found that e-commerce businesses had a higher customer-satisfaction rating than any other sector of the economy. As online sales increased, competition led traditional stores to adopt some of the practices of e-commerce, such as consumer-friendly return policies.

 Impacts of e-commerceon free speech:  Section 3.2.5

Free stuff

Libraries have provided free access to books, newspapers, and journals for generations, and radio and television provided free news and entertainment before the Internet. But there is so much more free stuff now—a truly astounding amount—conveniently available on the Web.

For our computers, we can get free email programs and email accounts, browsers, filters, firewalls, encryption software, word processors, spreadsheets, software for viewing documents, software to manipulate photos and video, home inventory software, antispam software, antivirus software, antispyware software, and software for many other specialized purposes. This is a small sampling of software available for free.

We can find free game-playing programs for old board games and card games such as chess and bridge, as well as for new games. Phone service via Skype is free. There are free dating services on the Web. Major music festivals offer their concerts for free on the Internet, a nice alternative to paying $30 to $500 for a ticket. Craigslist, the classified ad site, one of the most popular websites in the world, is free to people who place ads and people who read them. Major (expensive) universities such as Stanford, Yale, and MIT provide video of lectures, lecture notes, and exams for thousands of their courses on the Web for free. We can download whole books from Google, Project Gutenberg, and other sources for free. †  We can read news from all over the world for free. We can store our personal photographs, videos, and other files online for free. MySpace, Facebook, Twitter, LinkedIn, and YouTube are free; Google, Bing, and Yahoo are free. Specialized, scholarly encyclopedias (e.g., the Stanford Encyclopedia of Philosophy), Wikipedia, and hundreds of other references are free.

* The ease and security of payment on the Web had a pleasant side effect: Many people contribute more to charitable organizations. That had the unpleasant side effect of spawning scam charity sites.

† Books available for free downloading are in the public domain (that is, out of copyright).

We pay for libraries with taxes. Advertisers pay for broadcasting radio and television programs. On the Web, advertising pays for many, many free sites and services, but far from all. Wikipedia carries no advertising—donations pay for its hardware and band width. Craigslist charges fees of some businesses that post job announcements and brokers who post apartment listings in a few cities. That keeps the site free to everyone else and free of other paid ads. Businesses provide some free information and services for good public relations and as a marketing tool. (Some free programs and services do not have all the features of the paid versions.) Nonprofit organizations provide information as a public service; donations or grants fund them. One of the distinct and delightful features of the Internet is that individuals provide a huge amount of free stuff simply because it pleases them to do so. They are professionals or hobbyists or just ordinary people who enjoy sharing their expertise and enthusiasm. Generosity and public service flourish in the Web environment.

It is often obvious when we are viewing advertisements on websites or phones. Ads annoy some people, but they are not insidious, and their presence on a screen is not an unreasonable price to pay for free services. However, to earn ad revenue to fund multimillion-dollar services, many free sites collect information about our online activities and sell it to advertisers. This tracking is often not obvious; we consider it in  Chapter 2 .

1.2.3 Artificial Intelligence, Robotics, Sensors, and Motion

Artificial intelligence

Artificial intelligence (AI) is a branch of computer science that makes computers perform tasks we normally (or used to) think of as requiring human intelligence. It includes playing complex strategy games such as chess, language translation, making decisions based on large amounts of data (such as approving loan applications), and understanding speech (where the appropriateness of the response might be the measure of “understanding”). AI also includes tasks performed automatically by the human brain and nervous system—for example, vision (the capture and interpretation of images by cameras and software). Learning is a characteristic of many AI programs. That is, the output of the program improves over time as it “learns” by evaluating results of its decisions on the inputs it encounters. Many AI applications involve pattern recognition, that is, recognizing similarities among different things. Applications include reading handwriting (for automatic sorting of mail and input on tablet computers, for example), matching fingerprints, and matching faces in photos.

Early in the development of AI, researchers thought the hard problems for computers were tasks that required high intelligence and advanced training for humans, such as winning at chess and doing mathematical proofs. In 1997, IBM’s chess computer, Deep Blue, beat World Champion Garry Kasparov in a tournament. AI researchers realized that narrow, specialized skills were easier for computers than what a five-year-old does: recognize people, carry on a conversation, respond intelligently to the environment. In 2011, another specially designed computer system called Watson (also built by IBM) defeated human Jeopardy! champions by answering questions more quickly than the humans. Watson processes language (including puns, analogies, and so on) and general knowledge. It searches and analyzes 200 million pages of information in less than three seconds. Practical applications of the Watson technology include medical diagnosis and various business decision-making applications.

We briefly describe a few more examples of AI applications. They were astonishing advances not long ago.

When a man had a heart attack in a swimming pool in Germany, lifeguards did not see him sink to the bottom of the pool. An underwater surveillance system, using cameras and sophisticated software, detected him and alerted the lifeguards who rescued him. The software distinguishes a swimmer in distress from normal swimming, shadows, and reflections. It is now installed in many large pools in Europe and the United States. Just as AI software can distinguish a swimmer in trouble from other swimmers, AI software in video surveillance systems can distinguish suspicious behavior by a customer in a store that might indicate shoplifting or other crimes. Thus, without constant human monitoring, the AI-equipped video system can help prevent a crime, rather than simply identify the culprits afterwards.

Search engines use AI techniques to select search results. They figure out what the user meant if the search phrase contains typos, and they use context to determine the intended meaning of words that have multiple meanings. Automated websites that answer questions use AI to figure out what a question means and find answers.

Speech recognition, once a difficult research area, is now a common tool for hundreds of applications. Computer programs that teach foreign languages give instruction in correct pronunciation if they do not recognize what the user says. Millions of people who carry Apple smartphones can ask questions of Siri, Apple’s “intelligent” personal assistant. Siri interprets our questions and searches the Web for answers. Air traffic controllers train in a mockup tower whose “windows” are computer screens. The trainee directs simulated air traffic. The computer system responds when the trainee speaks to the simulated pilots. Such simulation allows more intensive training in a safe environment. If the trainee mistakenly directs two airplanes to land on the same runway at the same time, no one gets hurt.

People continue to debate the philosophical nature and social implications of artificial intelligence. What does it mean for a computer system to be intelligent? Alan Turing, who developed fundamental concepts underlying computer science before there were computers, proposed a test, now called the Turing Test, for human-level intelligence. Let a person converse (over a network) with the system on any topics the person chooses. If the computer convinces the person that it is human, the computer passes the test. Is that enough? Many technologists think so (assuming the actual test is well designed). But is the computer intelligent? Philosopher John Searle argues that computers are not and cannot be intelligent. They do not think; they manipulate symbols. They do so at very high speed, and they can store (or access) and manipulate a huge quantity of data, but they are not conscious. They do not understand; they simulate understanding. Searle uses the following example to illustrate the difference: Suppose you do not know the Chinese language. You are in a room with lots of boxes of Chinese symbols and a large instruction book written in English. People submit to you sequences of Chinese symbols. The instructions tell you how to manipulate the symbols you are given and the ones in the boxes to produce a new sequence of symbols to give back. You are very careful, and you do not get bored; you follow the instructions in the book exactly. Unknown to you, the sequences you receive are questions in Chinese. The sequences that you give back by following the instructions (just as a computer follows the instructions of a program) are the correct answers in Chinese. Everyone outside the room thinks you understand Chinese very well. Do you? Searle might say that although Watson won at Jeopardy!, Watson does not know it won. 13

Whether we characterize machines as intelligent, or use the word metaphorically, or say that machines simulate intelligence, advances in AI are continuing at a very fast pace. It took IBM several years and millions of dollars to build Watson. 14  Technologist Ray Kurzweil thinks personal computers will have the power of Watson within 10 years.

The goal of 17th- and 18th-century calculators was modest: to automate basic arithmetic operations. It shocked people at the time. That a mindless machine could perform tasks associated with human intellectual abilities was disconcerting. Centuries later, Garry Kasparov’s loss to a computer chess program generated worried articles about the value—or loss of value—of human intelligence. Watson generated more. So far, it seems that each new AI breakthrough is met with concern and fear at first. A few years later, we take it for granted.How will we react whenJeopardy! is oh, so trivial that anyone can do well at it? How will we react when we can go into a hospital for surgery performed entirely by a machine? Will it be scarier than riding in the first automatic elevators or airplanes? How will we react when we can have a conversation over the Net about any topic at all—and not know if we are conversing with a human or a machine? How will we react when chips implanted in our brains enhance our memory with gigabytes of data and a search engine? Will we still be human?

 Implications of human-level AI:  Section 7.4.3

Robots

Robots are mechanical devices that perform physical tasks traditionally done by humans or tasks that we think of as human-like activities. Robotic machines have been assembling products in factories for decades. They work faster and more accurately than people can. Computer software with artificial intelligence controls most robotic devices now. Robotic milking machines milk hundreds of thousands of cows at dairy farms while the farmhands sleep or do other chores. Some robots dance, and some make facial expressions to convey emotions. However, just as general intelligence is a hard problem for AI, general movement and functioning is a hard problem for robots. Most robotic devices are special-purpose devices with a relatively limited set of operations.

McDonald’s and other fast-food sellers use robotic food preparation systems to reduce costs and speed service. A robot pharmacist machine, connected to a patient database, plucks the appropriate medications from pharmacy shelves by reading bar codes, checks for drug interactions, and handles billing. One of its main goals is reduction of human error. Robots deliver medications and carry linens in hospitals. They navigate around obstacles and “push” elevator buttons with wireless signals. Physicians do complex and delicate surgery from a console with a 3-D monitor and joysticks that control robotic instruments. The software filters out a physician’s shaky movements. Robots work in environments that are hazardous to people. They inspect undersea structures and communication cables. They search for survivors in buildings collapsed by bombs or earthquakes. They explore volcanoes and other planets. They move or process nuclear and other hazardous wastes.

For several years, Sony sold a robot pet dog, Aibo. It walked (with a camera system providing vision). It responded to commands, and it learned. Several companies make robots with a more-or-less human shape. Honda’s Asimo, for example, walks up and down stairs. Various companies and researchers are developing robots with more general abilities. One goal is to develop robots that can act intelligently and perform a variety of operations to assist people. Robots (doglike or humanlike) can serve as companions to elderly people. Is an emotional connection with a machine dehumanizing, or is it an improvement over living alone or in a nursing home where the staff cannot provide regular companionship? Will knowing that Grandma has a robot companion ease the guilt of family members and lead them to visit less often? Will we come to view robot companions as positively as pets?

Smart sensors, motion, and control

How do robots walk, climb stairs, and dance? Tiny motion-sensing and gravity-sensing devices collect status data. Complex software interprets the data and determines the necessary motions, and then sends signals to motors. These devices—accelerometers,or mems (for microelectromechanical systems)—help robots, and Segway’s motorized scooters, stay upright.

A sharp price drop for mems triggered a burst of applications. 15  They provide image stabilization in digital cameras. They detect when a car has crashed, when someone has dropped a laptop, or when an elderly person has fallen. (In those applications, the system deploys an airbag, triggers a lock on the disk drive to reduce damage, or calls for help.) The Wii game console, whose controller detects the user’s motion, and motion detectors in smartphones brought motion-sensing applications to millions of consumers.

Tiny microprocessors with sensors and radio transmitters (sometimes called smart dust, though they are still larger than dust particles) are finding all sorts of applications. Some are in use; some are in development. We mention a few examples. These examples have many obvious benefits. What are some potential problems?

Oil refineries and fuel storage systems uses thousands of sensors to detect leaks and other malfunctions. Sandia National Laboratory developed a “chemical lab on a chip” that can detect emissions from automobiles, chemical leaks, dangerous gases in fires (reducing risk for firefighters), and many other hazards. Similar chips could detect chemical warfare agents.

Sensors detect temperature, acceleration, and stress in materials (such as airplane parts). Sensors distributed throughout buildings and bridges can detect structural problems, report on damage from earthquakes, and so on. These applications increase safety while reducing maintenance costs.

Sensors in agricultural fields report on moisture, acidity, and so on, helping farmers to avoid waste and to use no more fertilizer than needed. Sensors could detect molds or insects that might destroy crops. Sensors implanted in chickens monitor the birds’ body temperature. A computer automatically reduces the temperature in the chicken coop if the birds get too hot, thus reducing disease and death from overheating. Sensors in food products monitor temperature, humidity, and other factors to detect potential health problems while the food is in transit to stores.

What will be the impact of tiny flying sensor/computers that communicate wirelessly and which the military can deploy to monitor movement of equipment and people, or with which police or criminals can spy on us in our homes and public places?

A Microsoft researcher developed a system with which a user manipulates 3-D images with hand movements, without touching a screen or any controls. Designers of buildings, machines, clothing, and so on, could use it to examine designs before implementing them. Someone with dirty (or sterile) hands (e.g., mechanics, cooks, surgeons) could examinere ference materials while working. What other applications will people think of?

Sensors in baby clothes detect when a baby is sleeping face down, at risk for Sudden Infant Death Syndrome, and warn parents on their cellphone. A heart monitor in a firefighter’s shirt alerts supervisors if the firefighter is too stressed and needs a break. Trainers plan to use sensors in special clothing to better train athletes. What other applications will we find for wearware?

Already we implant or attach microprocessor-controlled devices in or on human bodies: heart pacemakers and defibrillators and devices that restore motion to paralyzed people (which we describe in  Section 1.2.4 ). These will likely see modifications that enhance performance for healthy people. At first it might be physical performance for athletes—for example, to help a competitive swimmer swim more smoothly. Then what? Biological sciences and computer sciences will combine in new ways.

1.2.4 Tools for Disabled People

One of the most heartwarming applications of computer technology is the restoration of abilities, productivity, and independence to people with physical disabilities.

Some computer-based devices assist disabled people in using ordinary computer applications that other people use, such as Web browsers and word processors. Some enable disabled people to control household and workplace appliances that most of us operate by hand. Some improve mobility. Some technologies that are primarily conveniences for most of us provide significantly more benefit for disabled people: consider that text messaging was very popular among deaf people before it was popular with the general population.

For people who are blind, computers equipped with speech synthesizers read aloud what a sighted person sees on the screen. They read information embedded in Web pagesthat sighted visitors do not need, for example, descriptions of images. Google offers search tools that rank websites based on how accessible they are for blind users. For materials not in electronic form, a scanner or camera, optical-character-recognition software, and a speech synthesizer combine to read aloud to a blind person. The first such readers were large machines. Now, handheld versions can read menus, bills, and receipts in restaurants, as well as magazines and mail at home. Where noise is a problem (or for a person both blind and deaf), a grid of buttons raised and lowered by the computer to form Braille characters can replace speech output. Braille printers provide hard copy. (Books have long been available in Braille or on tape, but the expense of production for a small market kept the selection limited.) Systems similar to navigation systems in cars help blind people walk around and find their way in unfamiliar neighborhoods.

Prosthetic devices, such as artificial arms and legs, have improved from heavy, “dumb” wood, to lighter materials with analog motors, and now to highly sensitive and flexible digitally controlled devices that enable amputees to participate in sports and fly airplanes. A person whose leg was amputated above the knee can walk, sit, and climb stairs with an artificial “smart” knee. Sensors attached to the natural leg measure pressure and motion more than a thousand times a second and transmit the data to a processor in the prosthetic leg. Artificial intelligence software recognizes and adapts to changes in speed and slope and the person’s walking style. The processor controls motors to bend and straighten the knee and support the body’s movement, replacing the normal complex interplay of nerves, muscles, tendons, and ligaments. Artificial arms use electrodes to pick up tiny electrical fields generated by contractions of muscles in the upper (natural) limb. Microprocessors control tiny motors that move the artificial limb, open and close fingers, and so on. For people with paralyzed legs or for others who cannot use an artificial leg, there are wheelchairs that climb stairs and support and transport a person in an upright position. In 2012, Exso Bionics sold its first exoskeleton, a device with sensors and tiny motors that straps to a person with paralyzed legs and enables the person to walk. 16

Various conditions—loss of limbs, quadriplegia (paralysis in both arms and legs, often resulting from an accident), and certain diseases—eliminate all or almost all use of the hands. Speech recognition systems are an extremely valuable tool for these people and for others. (Deaf people can use speech-recognition systems to “hear” another speaker as the computer displays the spoken words on a screen.) People who cannot use their hands can dictate documents to a word processor and give commands to a computer to control household appliances.

To restore control and motion to people paralyzed by spinal injuries, researchers are experimenting with chips that convert brain signals to controls for leg and arm muscles.Researchers in the United States and Europe are developing brain–computer interfaces so that severely handicapped people can operate a computer and control appliances with their thoughts. 17

The impact of all these devices on the morale of the user is immense. Think about a person with an active mind, personality, and sense of humor—but who cannot write, type,or speak. Imagine the difference when the person gains the ability to communicate—with family and friends, and with all the people and resources available on the Internet.

1.3 Themes

Several themes and approaches to analysis of issues appear through this book. I introduce a few here.

Old problems in a new context

· Cyberspace has many of the problems, annoyances, and controversies of noncyber life, among them crime, pornography, violent fiction and games, advertising, copyright infringement, gambling, and products that do not work right.

Throughout this book, I often draw analogies from other technologies and other aspects of life. Sometimes we can find a helpful perspective for analysis and even ideas for solutions to new problems by looking at older technologies and established legal and social principles. The emphasis on the fact that similar problems occur in other areas is not meant to excuse the new problems. It suggests, however, that the root is not always the new technology but can be human nature, ethics, politics, or other factors. We will often try to analyze how the technology changes the context and the impact of old problems.

Adapting to new technology

· Changes in technology usually require adaptive changes in laws, social institutions, business policies, and personal skills, attitudes, and behavior.

When cellphones first came with built-in cameras, privacy laws in Pennsylvania (and elsewhere) were not sufficient to convict a man who used his cellphone to take a photo up a woman’s skirt. (The man was found guilty of disorderly conduct.) A federal regulation requiring medical x-rays on film, rather than digital formats, was still in effect in 2011. During Japanese election campaigns in 2005, candidates were afraid to use email and blogs and to update their websites to communicate with voters, because a 1955 law that specifies the legal means of communicating with voters does not, of course, include these methods. It allows postcards and pamphlets.

We might naturally think some actions are criminal, and some should be legal, but legislators did not consider them when writing existing laws. The legal status of an action might be the opposite of what we expect, or it might be uncertain. Many new activities that new technology makes possible are so different from prior ways of doing things that we need a new set of “rules of the game.”

We have to relearn standards for deciding when to trust what we read. The major impact of computer technology on privacy means we have to think in new ways about how to protect ourselves. We have to decide when privacy is important and when we are willing to put it at risk for some other benefit.

Varied sources of solutions to problems

· Solutions for problems that result from new technology come from more or improved technology, the market, management policies, education and public awareness, volunteer efforts, and law.

The cycle of problems and solutions, more problems and more solutions, is a natural part of change and of life in general. Throughout this book, when we consider problems, we consider solutions from several categories. Technical solutions include hardware and software. “Hardware” might mean something other than part of a computer system; improved lighting near ATMs to reduce robberies is a hardware solution. Authentication technology helps reduce identity theft. Market mechanisms, such as competition and consumer demand, generate many improvements. We all must become educated about the risks of the high-tech tools we use and learn how to use them safely. Legal solutions include effective law enforcement, criminal penalties, lawsuits, legislation, and regulation. For example, there must be appropriate penalties for people who commit fraud online, and there must be appropriate liability laws for cases where system failures occur.

The global reach of the Net

· The ease of communication with distant countries has profound social, economic, and political effects—some beneficial, some not.

The Net makes information and opportunities more easily available to people isolated by geography or by political system. It makes crime fighting and law enforcement more difficult, because criminals can steal and disrupt services from outside the victim’s country. Laws in one country prohibiting certain content on the Web or certain kinds of Web services restrict people and businesses in other countries because the Web is accessible worldwide.

Trade-offs and controversy

· Increasing privacy and security often means reducing convenience. Protecting privacy makes law enforcement more difficult. Unpleasant, offensive, or inaccurate information accompanies our access to the Web’s vast amounts of useful information.

Some of the topics we discuss are not particularly controversial. We will sometimes address an issue more as a problem-solving exercise than as a controversy. We will look at the impact of electronic technology in a particular area, observe some problems that result, and describe solutions. On the other hand, many of the issues are controversial: leaking confidential information on the Internet, proper policies for privacy protection, how strict copyright law should be, offshoring of jobs, the impact of computers on quality of life.

We consider various viewpoints and arguments. Even if you have a strong position on one side of a controversy, it is important to know the arguments on the other side, for several reasons. Knowing that there are reasonable arguments for a different point of view, even if you do not think they are strong enough to win overall, helps make a debate more civilized. We see that the people on the other side are not necessarily evil, stupid, or ignorant; they may just put more weight on different factors. To convince others of your own viewpoint, you must counter the strongest arguments of the other side, so, of course, you first must know and understand them. Finally, you might change your own mind after considering arguments you had not thought of before.

Perfection is a direction, not an option.

In general, when evaluating new technologies and applications, we should not compare them to some ideal of perfect service or zero side effects and zero risk. That is impossible to achieve in most aspects of life. Instead, we should compare them to the alternatives and weigh the problems against the benefits. The ideal shows us the direction to go as we endeavor to seek improvements and solutions to problems.

Another reason that we cannot expect perfection is that we all have different ideas of what perfection is.

This does not excuse sloppiness. It is possible to meet extremely high standards.

Differences between personal choices, business policies, and law

· The criteria for making personal choices, for making policies for businesses and organizations, and for writing laws are fundamentally different.

We can make a personal choice—for example, about what social networks to join, what apps to put on our phones, or what ebooks to buy—according to our individual values and situation. A business bases its policies on many factors, including the manager’s perception of consumer preferences, what competitors are doing, responsibilities to stockholders, the ethics of the business owners or managers, and relevant laws.

Laws are fundamentally different from personal choices and organizational policies because they impose decisions by force on people who did not make them. Arguments for passing a law should be qualitatively different from reasons for adopting a personal or organizational policy. It might seem odd at first, but arguments on the merits of the proposal—for example, that it is a good idea, or is efficient, or is good for business, or is helpful to consumers—are not good arguments for a law. We can use these arguments to try to convince a person or organization to adopt a particular policy voluntarily. Arguments for a law must show why the decision should be enforced against someone who does not agree that it is a good idea. It is better to base laws on the notion of rights rather than on personal views about their benefits or how we want people to behave.

1.4 Ethics

· Honesty is the best policy.

· —English proverb, pre-1600

1.4.1 What Is Ethics, Anyway?

Sometimes, we discuss issues and problems related to computer technology from a somewhat detached perspective. We see how a new technology can create new risks and how social and legal institutions continually adapt. But technology is not an immutable force, outside of human control. People make decisions about what technologies and products to develop and how to use them. People make decisions about when a product is safe to release. People make decisions about access to and use of personal information. People make laws and set rules and standards.

Should you download movies from unauthorized websites? Should you talk on your cellphone while driving on a freeway? Should you hire foreign programmers who work at low salaries? Should you warn potential customers that the smartphone app you sell needs to copy their contact list? Should you fire an employee who is criticizing your business in social media? What information should you allow advertisers and other trackers to collect from visitors to the website you run? Someone sent you the contents of a friend’s (a teacher’s, a city council candidate’s) email account; should you post it on the Web? In these examples, you are confronting practical and legal issues—and ethical ones. In each case you can restate the problem as a question in the form “Is it right to . . . ?”Is it right to make a significant change in your company’s privacy policy without giving customers or members advance notice?

In this section, we introduce several ethical theories. We discuss some distinctions (e.g., between ethics and law) that are important to understand when tackling ethical issues.

Ethics is the study of what it means to “do the right thing.” It is a complex subject that has occupied philosophers for thousands of years. This presentation is necessarily simplified.

Ethical theory assumes that people are rational and make free choices. Neither of these conditions is always and absolutely true. People act emotionally, and they make mistakes. A person is not making a free choice when someone else is pointing a gun at him. Some argue that a person is not making a free choice in a situation where she might lose a job. However, free choice and use of rational judgment are capacities and characteristics of human beings, and they are reasonably assumed as the basis of ethical theory. We take the view that the individual is, in most circumstances, responsible for his or her actions.

Ethical rules are rules to follow in our interactions with other people and in our actions that affect other people. Most ethical theories attempt to achieve the same goal: to enhance human dignity, peace, happiness, and well-being. Ethical rules apply to all of us and are intended to achieve good results for people in general, and for situations in general—not just for ourselves, not just for one situation. A set of rules that does this well respects the fact that we are each unique and have our own values and goals, that we have judgment and will, and that we act according to our judgment to achieve our goals. The rules should clarify our obligations and responsibilities—and our areas of choice and personal preference. *

We could view ethical rules as fundamental and universal, like laws of science. Or we could view them as rules we make up, like the rules of baseball, to provide a framework in which to interact with other people in a peaceful, productive way. The titles of two books illustrate these different viewpoints. One is Ethics: Discovering Right and Wrong; the other is Ethics: Inventing Right and Wrong. 18  We do not have to decide which view is correct to find good ethical rules. In either case, our tools include reason, introspection, and knowledge of human nature, values, and behavior.

Behaving ethically, in a personal or professional sphere, is usually not a burden. Most of the time we are honest, we keep our promises, we do not steal, we do our jobs. This should not be surprising. If ethical rules are good ones, they work for people. That is, they make our lives better. Behaving ethically is usually practical. Honesty makes interactions among people work more smoothly and reliably, for example. We might lose friends if we often lie or break promises. Social institutions encourage us to do right: We might land in jail if caught stealing. We might lose our jobs if we do them carelessly. In a professional context, doing good ethically often corresponds closely with doing a good job in the sense of professional quality and competence. Doing good ethically often corresponds closely with good business in the sense that ethically developed products and ethical policies are more likely to please consumers. Sometimes, however, it is difficult to do the right thing. It takes courage in situations where we could suffer negative consequences. Courage is often associated with heroic acts, where one risks one’s life to save someone in a dangerous situation—the kind of act that makes news. Most of us do not have those opportunities to display courage, but we do have many opportunities in day-to-day life.

* Not all ethical theories fit this description. Ethical relativism and some types of ethical egoism do not. In this book, however, we assume these goals and requirements for ethical theories.

1.4.2 A Variety of Ethical Views  19

Although there is much agreement about general ethical rules, there are many different theories about how to establish a firm justification for the rules and how to decide what is ethical in specific cases. We give very brief descriptions of a few approaches to ethics. Some ethicists *  make a distinction between ethical theories that view certain acts as good or bad because of some intrinsic aspect of the action and ethical theories that view acts as good or bad because of their consequences. They call these deontological (or nonconsequentialist) and consequentialist theories, respectively. The distinction is perhaps emphasized more than necessary. If the criteria that deontologists use to determine the intrinsic goodness or badness of an act do not consider its consequences for people—at least for most people, most of the time—their criteria would seem to have little ethical merit.

Deontological theories

Deontologists tend to emphasize duty and absolute rules, to be followed whether they lead to good or ill consequences in particular cases. One example is: Do not lie. An act is ethical if it complies with ethical rules and you chose it for that reason.

Immanuel Kant, the philosopher often presented as the prime example of a deontologist, contributed many important ideas to ethical theory. We mention three of them here. One is the principle of universality: We should follow rules of behavior that we can universally apply to everyone. This principle is so fundamental to ethical theory that we already accepted it in our explanation of ethics.

Second, deontologists argue that logic or reason determines rules of ethical behavior, that actions are intrinsically good because they follow from logic. Kant believed that rationality is the standard for what is good. We can reason about what makes sense and act accordingly, or we can act irrationally, which is evil. The view that something is evil because it is illogical might seem unconvincing, but Kant’s instruction to “Respect the reason in you”—that is, to use your reason, rationality, and judgment, rather than emotions, when making a decision in an ethical context—is a wise one.

Third, Kant stated a principle about interacting with other people: One must never treat people as merely means to ends, but rather as ends in themselves.

Kant took an extreme position on the absolutism of ethical rules. He argued, for instance, that it is always wrong to lie. For example, if a person is looking for someone he intends to murder, and he asks you where the intended victim is, it is wrong for you to lie to protect the victim. Most people would agree that there are cases in which even very good, universal rules should be broken—because of the consequences.

* Ethicists are philosophers (and others) who study ethics.

Utilitarianism

Utilitarianism is the main example of a consequentialist theory. Its guiding principle, as expressed by John Stuart Mill,20 is to increase happiness, or “utility.” A person’s utility is what satisfies the person’s needs and values. An action might decrease utility for some people and increase it for others. We should consider the consequences—the benefits and damages to all affected people—and “calculate” the change in aggregate utility. An act is right if it tends to increase aggregate utility and wrong if it tends to decrease it.

Utilitarianism is a very influential theory, and it has many variations. As stated above, the utilitarian principle applies to individual actions. For each action, we consider the impact on utility and judge the action by its net impact. This is sometimes called“act utilitarianism.” One variant of utilitarianism, called “rule utilitarianism,” applies the utility principle not to individual actions but to general ethical rules. Thus, a rule utilitarian might argue that the rule “Do not lie” will increase total utility, and for that reason it is a good rule. Rule utilitarians do not do a utility calculation for each instance where they consider lying. Generally, a utilitarian would be more comfortable than a deontologist breaking a rule in circumstances where doing so would have good consequences.

There are numerous problems with act utilitarianism. It might be difficult or impossible to determine all the consequences of an act. If we can do so, do we increase whatwe believe will, or should, contribute to the happiness of the people affected, or what they choose themselves? How do we know what they would choose? How do we quantify happiness in order to make comparisons among many people? Should some people’s utility carry more weight than others’? Should we weigh a thief’s gain of utility equal to the victim’s loss? Is a dollar worth the same to a person who worked for it and a person who received it as a gift? Or to a rich person and a poor person? How can we measure the utility of freedom?

A more fundamental (and ethical) objection to act utilitarianism is that it does not recognize or respect individual rights. It has no absolute prohibitions and so could allow actions that many people consider always wrong. For example, if there is a convincing case that killing one innocent person (perhaps to distribute his or her organs to several people who will die without transplants) or taking all of a person’s property and redistributing it to other community members would maximize utility in a community, utilitarianism could justify these acts. A person has no protected domain of freedom.

Rule utilitarianism suffers far less than does act utilitarianism from these problems. Recognizing that widespread killing and stealing decrease the security and happiness of all, a rule utilitarian can derive rules against these acts. We can state these particular rules in terms of rights to life and property.

Natural rights

Suppose we wish to treat people as ends rather than merely means and we wish to increase people’s happiness. These goals are somewhat vague and open to many interpretations in specific circumstances. One approach we might follow is to let people make their own decisions. That is, we try to define a sphere of freedom in which people can act freely according to their own judgment, without coercive interference by others, even others (including us) who think they are doing what is best for the people involved or for humanity in general. This approach views ethical behavior as acting in such a way that respects a set of fundamental rights of others, including the rights to life, liberty, and property.

These rights are sometimes called natural rights because, in the opinion of some philosophers, they come from nature or we can derive them from the nature of humanity. John Locke 21  argued that we each have an exclusive right to ourselves, our labor, and to what we produce with our labor. Thus, he argued for a natural right to property that we create or obtain by mixing our labor with natural resources. He saw protection of private property as a moral rule. If there is no protection for property, then the person who invents a new tool would be loathe to show it to others or use it in their view, as they might take it. Clearing land and planting food would be pointless, as one could not be present at all times to prevent others from picking all the crop. Thus, a right of private property increases overall wealth (utility) as well; the toolmaker or farmer has more to give or trade to others.

Respect for the rights to life, liberty, and property implies ethical rules against killing, stealing, deception, and coercion.

Those who emphasize natural rights tend to emphasize the ethical character of theprocess by which people interact, seeing acts generally as likely to be ethical if they involve voluntary interactions and freely made exchanges where the parties are not coerced or deceived. This contrasts with other ethical standards or approaches that tend to focus onthe result or state achieved by the interaction, for example, seeing an action as likely to be unethical if it leaves some people poor.

Negative and positive rights, or liberties and claim rights

When people speak of rights, they are often speaking about two quite different kinds of rights. In philosophy books, these rights are usually called negative and positive rights, but the terms liberties and claim rights are more descriptive of the distinction. 22

Negative rights, or liberties, are rights to act without interference. The only obligation they impose on others is not to prevent you from acting. They include the right to life (in the sense that no one may kill you), the right to be free from assault, the right to use your property, the right to use your labor, skills, and mind to create goods and services and to trade with other people in voluntary exchanges. The rights to “life, liberty, and the pursuit of happiness” described in the U.S. Declaration of Independence are liberties, or negative rights. Freedom of speech and religion, as guaranteed in the First Amendment of the U.S. Constitution, are negative rights: the government may not interfere with you, jail you, or kill you because of what you say or what your religious beliefs are. The right to work, as a liberty, or negative right, means that no one may prohibit you from working or, for example, punish you for working without getting a government permit. The (negative)right to access the Internet is so obvious in free countries that we do not even think of it. Authoritarian governments restrict or deny it.

Claim rights, or positive rights, impose an obligation on some people to provide certain things for others. A positive right to a job means that someone must hire you regardless of whether they voluntarily choose to, or that it is right, or obligatory, for the government to set up job programs for people who are out of work. A positive right to life means that some people are obligated to pay for food or medical care for others who cannot pay for them. When we interpret freedom of speech as a claim right, or positive right, it means that we may require owners of shopping malls, radio stations, and online services to provide space or time for content they do not wish to include. Access to the Internet, as a claim right, could require such things as taxes to provide subsidized access for poor people or foreign aid to provide access in poor countries. The last example suggests the following question: How far does the obligation to provide a positive right extend? Also, when thinking about what might be a positive, or claim, right, it is helpful to consider whether something should be a claim right if it depends on achieving a certain level of technology. For example, if access to the Internet is a positive right now, was it a positive right in the 1800s?

Here is a more fundamental problem: negative rights and positive rights often conflict. Some people think that liberties are almost worthless by themselves and that society must devise social and legal mechanisms to ensure that everyone has their claim rights, or positive rights, satisfied, even if that means diminishing the liberties of some. Other people think that there can be no (or very few) positive rights, because it is impossible to enforce claim rights for some people without violating the liberties of others. They see the protection of liberties, or negative rights, as ethically essential.

This is one of the reasons for disagreement on issues such as some privacy protection regulations, for example. Although we will not solve the disagreement about which kind of right is more important, we can sometimes clarify the issues in a debate by clarifying which kind of right we are discussing.

Golden rules

The Bible and Confucius tell us to treat others as we would want them to treat us. This is a valuable ethical guideline. It suggests a reciprocity, or a role reversal. We should not take the rule too literally however; we need to apply it at the appropriate level. It tells us to consider an ethical choice we are making from the perspective of the people it affects. No matter how much you enjoy fast driving on winding roads, it might not be kind to roar around those corners with a passenger who gets carsick easily. No matter how much you like your friends to share photos of you partying, it might not be good to share a photo of friend who prefers privacy. We want people to recognize us as individuals and to respect our choices. Thus, we should respect theirs.

Contributing to society

We are focusing on how to make ethical decisions. Some ethical theories take a wider goal: how to live a virtuous life. That is beyond the scope of this book, but some of the ideas relate to ethical choices. Aristotle says that one lives a virtuous life by doing virtuous acts. This leaves us with a question: What is a virtuous act? Most people would agree that helping to serve meals at a homeless shelter is a virtuous act. The view that this type of activity (doing unpaid charitable work) is the only or the main kind of virtuous act is common but is too limited. Suppose a nurse is choosing between spending one evening a week taking a course to learn new nursing skills or spending one evening a week helping at the homeless shelter. Or a programmer at a bank is choosing between a course on new computer security techniques and helping at the homeless shelter. There is nothing wrong with either choice. Is either one more virtuous than the other? The first choice increases the person’s professional status and possibly the person’s salary; you could see it as a selfish choice. The second choice is charitable work, helping unfortunate people. But the analysis should not stop there. A professional person, well trained and up-to-date in his or her profession, often can do far more to help a large number of people than the same person can accomplish performing low-skill tasks outside the person’s professional area. The fact that the person is paid for his or her work is not significant in evaluating its contribution. Doing one’s work (whether it is collecting garbage or performing brain surgery) honestly, responsibly, ethically, creatively, and well is a virtuous activity.

· His philanthropy was in his work.

· —Mike Godwin, writing about Apple co-founder Steve Jobs 23

Social contracts and a theory of political justice  24

Many topics we consider in this book go beyond individual ethical choices. They are social and legal policies. Thus we introduce (again, quite briefly) philosophical ideas about forming social and political systems.

The early foundations of social contract theory, the idea that people willingly submit to a common law in order to live in a civil society, are in the writings of Socrates and Plato but were not fully formed until the 1600s. Thomas Hobbes developed ideas of social contract theory in his bookLeviathan (1651). Hobbes describes a starting point called the State of Nature, a dismal place where each man acts according to his own interests, no one is safe from physical harm, and there is no ability to ensure the satisfaction of one’s needs. Hobbes believed that man is rational and will seek a better situation, even at the cost of giving up some independence in favor of common law and accepting some authority to enforce this “social contract.” John Locke thought people could enforce moral rules, such as the rights to life, liberty and property, in a state of nature but that it was better to delegate this function to a government instituted by an implicit social contract.

The modern philosopher John Rawls 25  took social contract theory further, developing provisions of the “contract” based on his view of justice as fairness. I will criticize parts of his work, but some of his points provide useful ethical guidelines. Rawls sought to establish principles for proper political power in a society with people of varying religions, viewpoints, lifestyles, and so on. Rawls, like other social contract theorists, said that reasonable people, recognizing that a legal (or political) structure is necessary for social order, will want to cooperate on terms that all accept, and they will abide by the rules of society, even those they do not like. He argued that political power is proper only if we would expect all citizens to reasonably endorse its basic, or constitutional, principles. Tolerance is essential because deep questions are difficult, we answer them differently based on our life experiences, and people of good will can disagree. Thus, a proper political system protects basic civil liberties such as freedom of speech and free choice of occupation. It will not impose the views of some on the others.

To this point, Rawls’ foundation is consistent with an emphasis on liberties (negative rights). Rawls distinguishes his system of justice by adding a strong requirement for claim rights (positive rights): a just and fair political system will ensure that all citizens have sufficient means to make effective use of their freedoms. *  To Rawls, government financing of election campaigns is an essential feature of the system. This is a very specific political policy; people hotly debate its fairness and practical consequences. Rawls has made a leap that appears inconsistent with his emphasis that people of good will disagree on important issues and that a proper political system does not impose the views of one group on another.

In Rawls’ view, an action or a social or political structure is not ethical if it has the effect of leaving the least-advantaged people worse than they were before (or would be in some alternative system). Thus, in a sense, Rawls gives far more weight (indeed, infinite weight) to the utility of the least-advantaged people than to anyone else. This is odd as an absolute rule, and its fairness is not obvious. His emphasis on concern for the least well off, however, is a reminder to consider impacts on such people; a loss or harm to them can be more devastating than to someone in a better position.

* The meaning of fairness is not obvious. In various contexts and to different people, it can mean being judged on one’s merits rather than irrelevant factors, getting an equal share, or getting what one deserves.

Rawls proposed a conceptual formulation termed the “veil of ignorance” for deriving the proper principles or policies of a just social or political system. By extension, we can use it as a tool for considering ethical and social issues in this book. We imagine that each person behind the veil of ignorance does not know his or her gender, age, race, talents, wealth, and so on, in the real world. Behind the veil of ignorance, we choose policies that would be fair for all, protecting the most vulnerable and least-advantaged members of society. Many writers use this tool to derive what they conclude to be the correct ethical positions on social policy issues. I find that sometimes when I go behind the veil of ignorance, I come to a different conclusion than the author. The tool is useful, like the principles of the ethical theories we described earlier, but, like them, it is not absolute. Even ignoring our status in society, people of good will come to different conclusionsbecause of their knowledge of human behavior and economics and their understanding of how the world works. *

We illustrate with a policy example. The Children’s Online Privacy Protection Act (COPPA) is a privacy law intended to protect a vulnerable population by requiring that websites get parental permission before collecting personal information from children under 13. After COPPA passed, because of the expense of complying with its requirements and the potential liability, some companies deleted online profiles of all children under 13, some canceled their free email and home pages for kids, and some banned children under 13 entirely. The New York Times does not allow children under 13 to register to use its website. Facebook’s terms of use prohibit children under 13 from joining, but Consumer Reports estimates that more than seven million children under 13 have ignored the rule and joined. 26  The fiction that there are no members under 13 implies there is no need to provide mechanisms to protect them. Economists would have predicted these effects. We might have come up with COPPA behind a veil of ignorance, but it is not clear how well it actually helps and protects children. More knowledge helps us make better decisions and design better policies and laws.

No simple answers

We cannot solve ethical problems by applying a formula or an algorithm. Human behavior and real human situations are complex. There are often trade-offs to consider. Ethical theories do not provide clear, incontrovertibly correct positions on most issues. We can use the approaches we described to support opposite sides of many an issue. For example, consider Kant’s imperative that one must never treat people as merely means to ends, but rather as ends in themselves. We could argue that an employer who pays an employee a very low wage, say, a wage too low to support a family, is wrongly treating the employee as merely a means for the employer to make money. But we could also argue that expecting the employer to pay more than he or she considers reasonable is treating the employer merely as a means of providing income for the employee. Similarly, it is easy for two utilitarians to come to different conclusions on a particular issue by measuring happiness or utility differently. A small set of basic natural rights might provide no guidance form any situations in which you must make ethical decisions—however, if we try to define rights to cover more situations, there will be fierce disagreement about just what those rights should be.

* Rawls specifies that we assume people behind the veil of ignorance have knowledge of accepted economic principles, but in fact many philosophers and ordinary people do not—and of course, people will disagree about what is accepted.

Although ethical theories do not completely settle difficult, controversial issues, they help to identify important principles or guidelines. They remind us of things to consider, and they can help clarify reasoning and values. There is much merit in Kant’s principle of universalism and his emphasis on treating people as intrinsically valuable “ends.” “Do not lie, manipulate, or deceive” is a good ethical principle. There is much merit in utilitarianism’s consideration of consequences and its standard of increasing achievement of people’s happiness. There is much merit in the natural rights approach of setting minimal rules in a rights framework to guarantee people a sphere in which they can act according to their own values and judgment. The Golden Rule reminds us to consider the perspective of the people our actions affect. Rawls reminds us that it is especially important to consider the impact of our choices on the least-advantaged people.

Do organizations have ethics?

Some philosophers argue that it is meaningless to speak of a business or organization as having ethics. Individual people make all decisions and take all actions. Those people must have ethical responsibility for everything they do. Others argue that an organization that acts with intention and a formal decision structure, such as a business, is a moral entity. 27  However, viewing a business as a moral entity does not diminish the responsibility of the individual people. Ultimately, it is individuals who are making decisions and taking actions. We can hold both the individuals and the company or organization responsible for their acts. *

Whether one accepts or rejects the idea that a business can have ethical rights and responsibilities, it is clear that organizational structure and policies lead to a pattern of actions and decisions that have ethical content. Businesses have a “corporate culture,” or a “personality,”or simply a reputation for treating employees and customers in respectful and honest—or careless and deceptive—ways. People in management positions shape the culture or ethics of a business or organization. Thus, decisions by managers have an impact beyond the particular product, contract, or action a decision involves. A manager who is dishonest with customers or who cuts corners on testing, for example, is setting an example that encourages other employees to be dishonest and careless. A manager’s ethical responsibility includes his or her contribution to the company’s ethical personality.

*Regardless of whether or not we view businesses and organizations as moral agents, they are legal entities and can be held legally responsible for their acts.

1.4.3 Some Important Distinctions

A number of important distinctions affect our ethical judgments, but they are often not clearly expressed or understood. In this section, we identify a few of these. Just being aware of these distinctions can help clarify issues in some ethical debates.

Right, wrong, and okay

In situations with ethical dilemmas, there are often many options that are ethically acceptable, with no specific one ethically required. Thus, it is misleading to divide all acts into two categories, ethically right and ethically wrong. Rather, it is better to think of acts as either ethically obligatory, ethically prohibited, or ethically acceptable. Many actions might be virtuous and desirable but not obligatory.

Distinguishing wrong and harm

Carelessly and needlessly causing harm is wrong, but it is important to remember that harm alone is not a sufficient criterion to determine that an act is unethical. Many ethical, even admirable acts can make other people worse off. For example, you may accept a job offer knowing someone else wanted the job and needed it more than you do. You may reduce the income of other people by producing a better product that consumers prefer. If your product is really good, you might put a competitor out of business completely and cause many people to lose their jobs. Yet there is nothing wrong with doing honest, productive work.

Declining to give something (say, $100) to someone is not the same ethically as taking the thing away from the person. Both actions leave the person less well off by $100 than they would be otherwise. But if we took that simplistic view of harm, the harm would be essentially the same. To identify harm as wrong, we must identify what the person is due, what his or her rights are, and what our rights and obligations are.

On the other hand, there can be wrong when there is no (obvious or immediate) harm. Some hackers argue that breaking into computer systems is not wrong, because they do no harm. Aside from the fact that the hacker might do unintended harm, one can argue that hacking is a violation of property rights: a person has no right to enter your property without your permission, independent of how much harm is done in any particular instance.

Separating goals from constraints

Economist Milton Friedman wrote that the goal or responsibility of a business is to make a profit for its shareholders. This statement appalled some ethicists, as they believe it justifies, or is used to justify, irresponsible and unethical actions. It seems to me that arguments on this point miss the distinction between goals, on the one hand, and constraints on actions that may be taken to achieve the goals, on the other hand—or the distinction between ends and means. Our personal goals might include financial success and finding an attractive mate. Working hard, investing wisely, and being an interesting and decent person can achieve these goals. Stealing and lying might achieve them too. Stealing and lying are ethically unacceptable. Ethics tells us what actions are acceptable or unacceptable in our attempts to achieve the goals. There is nothing unethical about a business having the goal of maximizing profits. The ethical character of the company depends on whether the actions taken to achieve the goal are consistent with ethical constraints. 28

Personal preference and ethics

Most of us have strong feelings about a lot of issues. It might be difficult to draw a line between what we consider ethically right or wrong and what we personally approve or disapprove of.

Suppose you get a job offer from a company whose products you do not like. You might decline the job and say you are doing so on ethical grounds. Are you? Can you convincingly argue that anyone who takes the job is acting unethically? Most likely you cannot, and that is not what you actually think. You do not want to work for a company you do not like. This is a personal preference. There is nothing ethically wrong with declining the job, of course. The company’s freedom to produce its products does not impose an ethical obligation on you to assist it.

When discussing political or social issues, people frequently argue that their position is right in a moral or ethical sense or that an opponent’s position is morally wrong or unethical. People tend to want to be on the “moral high ground.” People feel the stigma of an accusation that their view is ethically wrong. Thus, arguments based on ethics can be, and often are, used to intimidate people with different views. It is a good idea to try to distinguish between actions we find distasteful, rude, or ill-advised and actions that we can argue convincingly are ethically wrong.

Law and ethics

What is the connection between law and ethics? Sometimes very little. Is it ethical to prohibit marijuana use by terminally ill people? Is it ethical for the government or a state university to give preference in contracts, hiring, or admissions to people in specific ethnic groups? Is it ethical for a bank loan officer to carry customer records on a laptop to work at the beach? The current law, whatever it happens to be at a particular time, does not answer these questions. In addition, history provides numerous examples of laws most of us consider profoundly wrong by ethical standards; slavery is perhaps the most obvious example. Ethics precedes law in the sense that ethical principles help determine whether or not we should pass specific laws.

Some laws enforce ethical rules (e.g., against murder and theft). By definition, we are ethically obligated to obey such laws—not because they are laws, but because the laws implement the obligations and prohibitions of ethical rules.

Another category of laws establishes conventions for business or other activities. Commercial law, such as the Uniform Commercial Code, defines rules for economictrans actions and contracts. Such rules provide a framework in which we can interact smoothly and confidently with strangers. They include provisions for how to interpret a contract if a court must resolve a dispute. These laws are extremely important to any society and they should be consistent with ethics. Beyond basic ethical considerations, however, details could depend on historic conventions, practicality, and other non ethical criteria. In the United States, drivers must drive on the right side of the road; in England,drivers must drive on the left side. There is obviously nothing intrinsically right or wrong about either choice. However, once the convention is established, it is wrong to drive on the wrong side of the road because it needlessly endangers other people.

Unfortunately, many laws fall into a category that is not intended to implement ethical rules—or even be consistent with them. The political process is subject to pressure from special interest groups of all sorts who seek to pass laws that favor their groups or businesses. Examples include the laws (promoted by the television networks) that delayed the introduction of cable television and, later, laws (promoted by some cable television companies) to restrict satellite dishes. When margarine was first introduced, the dairy industry successfully lobbied for laws against coloring margarine yellow to look more like butter. After opposing re-sale auctions of event tickets for years, Ticketmaster accepted this popular online sales paradigm—and lobbied for laws restricting competitors. 29  Many prominent people in the financial industry reported receiving a large number of fundraising letters from members of Congress—in the week that Congress took up new regulations for their industry. Many political, religious, or ideological organizations promote laws to require (or prohibit) certain kinds of behavior that the group considers desirable (or objectionable). Examples include prohibitions on teaching foreign languages in schools (in the early 20th century),30 prohibitions on gambling or alcohol, requirements for recycling, and requirements that stores close on Sundays. At an extreme, in some countries, this category includes restrictions on the practice of certain religions. Some politicians or political parties pass laws, no matter how public-spirited they sound, purely to give themselves and their friends or donors advantages.

Copyright law has elements of all three categories we have described. It defines a property right, violation of which is a form of theft. Because of the intangible nature of intellectual property, some of the rules about what constitutes copyright infringement are more like the second category, pragmatic rules devised to be workable. Powerful groups (e.g., the publishing, music, and movie industries) lobby for specific rules to benefit themselves. This is why some violations of copyright law are clearly unethical (if one accepts the concept of intellectual property), yet others seem to be entirely acceptable, sometimes even noble.

Legislators and their staffs draft some laws in haste, and they make little sense. Some laws and regulations have hundreds or thousands of pages and are full of specific detail that make many ethical choices illegal. When members of Congress debate whether pizza is a vegetable,31 they are not debating an ethical issue.

Do we have an ethical obligation to obey a law just because it is a law? Some argue that we do: as members of society, we must accept the rules that the legislative process has created so long as they are not clearly and utterly ethically wrong. Others argue that, whereas this might often be a good policy, it is not an ethical obligation. Legislators are just a group of people, subject to errors and political influences; there is no reason to feel an ethical obligation to do something just because they say so. Indeed, some believe all laws that regulate personal behavior or voluntary economic transactions to be violations of the liberty and autonomy of the people forced to obey and, hence, to be ethically wrong.

Is it always ethically right to do something that is legal? No. Laws must be uniform and stated in a way that clearly indicates what actions are punishable. Ethical situations are complex and variable; the people involved might know the relevant factors, but it might not be possible to prove them in court. There are widely accepted ethical rules that would be difficult and probably unwise to enforce absolutely with laws—for example: Do not lie. New law lags behind new technology for good reasons. It takes time to recognize new problems associated with the technology, consider possible solutions, think and debate about the consequences and fairness of various proposals, and so on. A good law will set minimal standards that can apply to all situations, leaving a large range of voluntary choices. Ethics fills the gap between the time when technology creates new problems and the time when legislatures pass reasonable laws. Ethics fills the gap between general legal standards that apply to all cases and the particular choices made in a specific case.

While it is not ethically obligatory to obey all laws, that is not an excuse to ignore laws, nor is a law (or lack of a law) an excuse to ignore ethics.

Chapter:-4

4.1 Principles, Laws, and Cases

4.1.1 What Is Intellectual Property?

Have you made a video set to a popular song and put it on the Web? Have you recorded a televised movie to watch later in the week? Have you downloaded music from the Web without paying for it? Have you watched a streaming video of a live sports event? Do you know which of these actions are legal and which are illegal, and why? Is it legal for a search engine to copy videos and books in order to display excerpts? How should intellectual property owners respond to new technologies that make it easy to copy and distribute their property without permission? How do copyright owners abuse copyright? If you are developing software for an online retail site, can you implement one-click shopping without permission of a patent holder? Will enforcement of strict notions of copyright and patent smother the creativity enabled by modern technology? We begin our exploration of these and other issues about intellectual property by explaining the concept of intellectual property and reviewing principles of intellectual property laws.

Copyright is a legal concept that defines rights to certain kinds of intellectual property. Copyright protects creative works such as books, articles, plays, songs (both music and lyrics), works of art, movies, software, and videos. Facts, ideas, concepts, processes, and methods of operation are not copyrightable. Patent, another legal concept that defines rights to intellectual property, protects inventions, including some software-based inventions.

In addition to copyright and patents, various laws protect other forms of intellectual property. They include trademarks and trade secrets. This chapter concentrates more on copyright than other forms of intellectual property because digital technology and the Internet affect copyright so strongly. Patent issues for software and Web technologies are quite important and controversial. We discuss them in Section 4.5.

The key to understanding intellectual property protection is to understand that the thing protected is the intangible creative work—not its particular physical form. When we buy a novel in book form, we are buying a physical collection of paper and ink. When we buy a novel as an ebook, we are buying certain rights to an electronic-book file. We are not buying the intellectual property—that is, the plot, the organization of ideas, the presentation, the characters, and the events that form the abstraction that is the intangible “book,” or the “work.” The owner of a physical book may give away, lend, or resell the one physical book he or she bought but may not make copies (with some exceptions). The legal right to make copies belongs to the owner of the intangible “book”—that is, the owner of the copyright. The principle is similar for software, music, movies, and so on. The buyer of a software package is buying only a copy of it or a license to use the software. When we buy a movie on disc or via streaming video, we are buying the right to watch it, but not the right to play it in a public venue or charge a fee.

Why does intellectual property have legal protection? The value of a book or a song or a computer program is much more than the cost of printing it, putting it on disk, or uploading it to the Web. The value of a painting is higher than the cost of the canvas and paint used to create it. The value of intellectual and artistic works comes from the creativity, ideas, research, skills, labor, and other nonmaterial efforts and attributes their creators provide. Our property rights to the physical property we create or buy include the rights to use it, to prevent others from using it, and to set the (asking) price for selling it. We would be reluctant to make the effort to buy or produce physical things if anyone else could just take them away. If anyone could copy a novel, a computer program, or a movie for the small price of the copying, the creator of the work would receive very little income from the creative effort and would lose some of the incentive for producing it. Protection of intellectual property has both individual and social benefits: it protects the right of artists, authors, and inventors to compensation for what they create, and, by so doing, it encourages production of valuable, intangible, easily copied, creative work.

The author of a particular piece of intellectual property, or his or her employer (e.g., a newspaper or a software company), may hold the copyright or may transfer it to a publisher, a music recording company, a movie studio, or some other entity. Copyrights last for a limited time—for example, the lifetime of the author plus 70 years. After that, the work is in the public domain; anyone may freely copy and use it. Congress has extended the time period for copyright control more than a dozen times. The extensions are controversial, as they hold more material out of the public domain for a long time. For example, the movie industry lobbied for and obtained an extension of its copyright protection period from 75 years to 95 years when the first Mickey Mouse cartoon was about to enter public domain.

U.S. copyright law (Title 17 of the U.S. Code 1 ) gives the copyright holder the following exclusive rights, with some very important exceptions that we will describe:

· • To make copies of the work

· • To produce derivative works, such as translations into other languages or movies based on books

· • To distribute copies

· • To perform the work in public (e.g., music, plays)

· • To display the work in public (e.g., artwork, movies, computer games, video on a website)

Restaurants, bars, shopping centers, and karaoke venues pay fees for the copyrighted music they play. *  Moviemakers pay for the right to base a movie on a book, even if they make significant changes to the story.

Making a copy of a copyrighted work or a patented invention does not deprive the owner or anyone else of the work’s use. Intellectual property differs from physical property in this way. Thus, taking intellectual property by copying is different from theft of physical property, and copyright law does not prohibit all unauthorized copying, distribution, and so on. A very important exception is the “fair use” doctrine, which we discuss in Section 4.1.4. Uses of copyrighted material that the copyright owner has not authorized and that one of the exceptions in the law does not permit are infringements of the copyright and are subject to civil and/or criminal penalties.

Most of the discussions in this chapter are within a context that accepts the legitimacy of intellectual property protection but revolve around its extent, how new technology challenges it, and how it can or should evolve. Some people reject the whole notion of intellectual property as property, and hence, copyrights and patents. They see these mechanisms as providing government-granted monopolies, violating freedom of speech, and limiting productive efforts. This issue is independent of computer technology, so we do not cover it in depth in this book. However, the discussion of free software, in Section 4.4, overlaps arguments about the legitimacy of copyright in general.

4.1.2 Challenges of New Technologies

· Copyright law will disintegrate.

· —Nicholas Negroponte 2

· New technologies have been disrupting existing equilibria for centuries, yet balanced solutions have been found before.

· —Pamela Samuelson 3

Previous technologies raised challenges to intellectual property protection. For example, photocopiers made copying of printed material easy. However, such earlier technologies were not nearly as serious a challenge as digital technology. A complete photocopy of a book is bulky, sometimes of lower print quality, awkward to read, and more expensive than a paperback. Computers and communications technologies made high-quality copying and high-quantity distribution extremely easy and cheap. Technological factors include the following:

*Not all do, of course, but it is the accepted, and legal, practice.

· • Storage of all sorts of information (text, sound, graphics, video) in standard digitized formats; the ease of copying digitized material and the fact that each copy is a “perfect” copy

· • High-volume, relatively inexpensive digital storage media, including hard disks for servers and small portable media such as DVDs, memory sticks, and flash drives

· • Compression formats that make music and movie files small enough to download, copy, and store

· • Search engines, which make it easy to find material, and the Web itself

· • Peer-to-peer technology, which permits easy transfer of files over the Internet by large numbers of strangers without a centralized system or service; and later, file-hosting services that enable storage and sharing of large files (e.g., movies)

· • Broadband (high speed) Internet connections that make transfer of huge files quick and enable streaming video

· • Miniaturization of cameras and other equipment that enable audience members to record and transmit movies and sports events; and, before that, scanners, which simplify converting printed text, photos, and artwork to digitized electronic form

· • Software tools for manipulating video and sound, enabling and encouraging non-professionals to create new works using the works of others

In the past, it was generally businesses (newspapers, publishers, entertainment companies) and professionals (photographers, writers) who owned copyrights, and it was generally businesses (legal and illegal) that could afford the necessary copying and production equipment to infringe copyrights. Individuals rarely had to deal with copyright law. Digital technology and the Internet empowered us all to be publishers, and thus to become copyright owners (for our blogs and photos, for example), and they empowered us all to copy, and thus to infringe copyrights.

The first category of intellectual property to face significant threats from digital media was computer software itself. Copying software used to be common practice. As one writer said, it was “once considered a standard and acceptable practice (if it were considered at all).” 4  People gave copies to friends on floppy disks, and businesses copied business software. People traded warez(unauthorized copies of software) on computer bulletin boards. Software publishers began using the term “software piracy” for high-volume, unauthorized copying of software. Pirated software included (and still includes) word processing programs, spreadsheet programs, operating systems, utilities, games, and just about any consumer software sold. Some, such as new versions of popular games, often appear on unauthorized sites or for sale in other countries before their official release. The software industry estimates the value of pirated software in billions of dollars.

In the early 1990s, one could find on the Internet and download unauthorized copies of popular humor columns (copied from newspapers), lyrics of popular songs, and some images (e.g., Walt Disney Company characters, Playboy pinups, and myriad Star Trek items). Music files were too large to transfer conveniently. Tools for listening to music on computers were unavailable or awkward to use; devices for recording or copying digital music were expensive. Technology improved and prices fell. (CD recorders sold for about $1000 when first introduced, and for $99 within about three years.)

The audio data compression format MP3, introduced in the mid-1990s, reduced the size of audio files by a factor of about 10–12. People could download an MP3 song from the Internet in a few minutes. Hundreds of MP3 sites appeared, making thousands of songs available. MP3 has no mechanism for preventing unlimited or unauthorized copying. Many songwriters, singers, and bands willingly made their music available, but most trading of MP3 files on the Net was unauthorized.

In the 2000s, more new technology (e.g., sophisticated file-sharing schemes, inexpensive video cameras, video editing tools, and video-sharing sites) enabled members of the public to provide entertainment for each other—and to post and share professional videos owned by others. Copying music and movies became easy, fast, cheap, and ubiquitous. The scope of the term “piracy” expanded to include high-volume, unauthorized copying of any form of intellectual property. It can mean individuals posting unauthorized files to legitimate file-sharing sites; underground groups trading unauthorized copies; or highly profitable, multimillion-dollar businesses (mostly outside the United States) that encourage members to upload and share files, knowing that most of the files are unauthorized copies.

The content industries claim that about one-quarter of Internet traffic worldwide consists of copyright-infringing material. 5  The entertainment industry, like the software industry, estimates that people copy, trade, and sell billions of dollars of its intellectual property without authorization each year. The dollar amounts from industry sources might be inflated, *  but the amount of unauthorized copying and distribution of music, video, and other forms of intellectual property is huge. Entertainment companies and other content providers are losing significant income and potential income that they could earn from their intellectual property. As we seek solutions to this problem, though, we should recognize that “the problem” looks different from different perspectives. What does it mean to solve the problems of technology’s impact on intellectual property rights? What are the problems for which we seek solutions?

To consumers, who get movies and music online, the problem is to get them cheaply and conveniently. To writers, singers, artists, actors—and to the people who work in production, marketing, and management—the problem is to ensure that they are paid for the time and effort they put in to create the intangible intellectual-property products we enjoy. To the entertainment industry, to publishers and software companies, the problem is to protect their investment and expected, or hoped-for, revenues. To the millions who post amateur works using the works of others, the problem is to continue to create without unreasonably burdensome requirements and threats of lawsuits. To scholars and various advocates, the problem is how to protect intellectual property, but also to protect fair use, reasonable public access, and the opportunity to use new technologies to the fullest to provide new services and creative work. We explore problems and solutions from several perspectives in this chapter.

*Some figures seem to assume that everyone who downloads a movie or song for free illegally would buy it at full price if it were not available for free.

The two quotations at the beginning of this section date from 1995, when the significant threat to copyright from digital media became clear. Users and observers of digital media and of the Internet debated whether copyright would survive the enormously increased ease of copying and the habits and expectations that developed about sharing information and entertainment online. Some argued that copyright would survive, mostly because of firm enforcement of copyright law. Others said the ease of copying would win out; most content would be free or almost free. These positions seem more compatible today than they did at first. Enforcement has been fierce, but much legal content is free or cheap due to improved technology and the many services that provide free content sponsored by advertising.

4.1.3 A Bit of History

A brief history of copyright law will provide background and help illustrate how new technologies require changes or clarifications in law. 6

The first U.S. copyright law, passed in 1790, covered books, maps, and charts. It protected them for 14 years. Congress later extended the law to cover photography, sound recordings, and movies. The definition of an unauthorized copy in the Copyright Act of 1909 specified that it had to be in a form that could be seen and read visually. Even with the technologies of the early 20th century, this requirement was a problem. A court applied it in a case about copying a song onto a perforated piano-music roll. (Automatic pianos played such rolls.) A person could not read the music visually from the piano roll, so the copy was not judged a violation of the song’s copyright, even though it violated the spirit and purpose of copyright. 7  In the 1970s, a company sued for protection of its chess-playing program, implemented on a read-only-memory (ROM) chip in its handheld computer chess game. Another company sold a game with the identical program; they likely copied the ROM. But because the ROM could not be read visually, a court held that the copy did not infringe the program’s copyright. 8  Again, this did not well serve the purpose of copyright. The decision did not protect the creative work of the programmers. They received no compensation from a competitor’s sales of their work.

In 1976 and 1980, Congress revised copyright law to cover software. “Literary works” protected by copyright include computer databases that exhibit creativity or originality 9  and computer programs that exhibit “authorship,” that is, contain original expression of ideas. Recognizing that technology was changing rapidly, the revised law specifies that copyright applies to appropriate literary works “regardless of the nature of the material objects … in which they are embodied.” A copy could be in violation of a copyright if the original can be “perceived, reproduced, or otherwise communicated by or from the copy, directly or indirectly.”

One significant goal in the development of copyright law, illustrated by the examples above, has been devising good definitions to extend the scope of protection to new technologies. As copying technologies improved, another problem arose: a lot of people will break a law if it is easy to do so and the penalties are weak. In the 1960s, growth in illegal sales of unauthorized copies of recorded music (e.g., on tape) accompanied the growth of the music industry. In 1982, high-volume copying of records and movies became a felony. In 1992, making a small number of copies of copyrighted work “willfully and for purposes of commercial advantage or private gain” became a felony. In response to the growing phenomenon of sharing files for free on the Internet, the No Electronic Theft Act of 1997 made it a criminal offense to willfully infringe copyright (for works with total value of more than $1000 within a six-month period) even if there is no commercial advantage or private gain. The penalties can be severe. After huge growth in sales of unauthorized copies of movies, Congress made it a felony offense to record a movie in a movie theater—one of the ways copies get to those who reproduce and sell them illegally. Critics of these laws argue that the small offenses covered do not merit the severe penalties.

Why did copyright laws get more restrictive and punishing? Generally, creators and publishers of copyrighted works, including print publishers, movie companies, music publishers, sound recording companies (record labels), and the software industry support stronger copyright protection. Congress often delegates the drafting of laws in complex areas to the industries involved. For most of the 20th century, the intellectual property industries drafted laws heavily weighted toward protecting their assets. On the other side, librarians and academic and scientific organizations generally opposed strict rules reducing the public’s access to information. Most people were unaware of or indifferent to copyright issues. But digital media, and especially the growth of the Web, focused attention on issues about how much control copyright owners should have. In the 1990s, cybercitizens and organizations such as the Electronic Frontier Foundation joined librarians and others to fight what they view as overly restrictive copyright law. The content industries continue to be powerful lobbyists for their point of view. Web service companies and organizations (such as Google, Facebook, and Wikipedia) add some balance to the lobbying and public debate.

4.1.4 The Fair Use Doctrine

Copyright law and court decisions attempt to define the rights of authors and publishers consistent with two goals: promoting production of useful work and encouraging the use and flow of information. The fair use doctrine allows uses of copyrighted material that contribute to the creation of new work (such as quoting part of a work in a review) and uses that are not likely to deprive authors or publishers of income for their work. Fair uses do not require the permission of the copyright holder. The notion of fair use (for literary and artistic works) grew from judicial decisions. In 1976, U.S. copyright law explicitly included it. It applies to software also. The 1976 copyright law predated the widespread use of personal computers. The software issues addressed pertained mainly to large business systems, and the law did not address issues related to the Web at all. Thus, it did not take into account many situations where questions of fair use now arise.

The law identifies possible fair uses, such as “criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research.” 10  It lists four factors to consider in determining whether a particular use is a “fair use”:

· 1. The purpose and nature of the use, including whether it is for commercial purposes or nonprofit educational purposes. (Commercial use is less likely to be fair use.)

· 2. The nature of the copyrighted work. (Use of creative work, such as a novel, is less likely than use of factual work to be fair use.)

· 3. The amount and significance of the portion used.

· 4. The effect of the use on the potential market for or value of the copyrighted work. (Uses that reduce sales of the original work are less likely to be considered fair.) No single factor alone determines whether a particular use is a fair use, but the last one generally gets more weight than the others.

Court decisions about copyright must be consistent with the First Amendment. For example, courts interpret the fair use principle broadly to protect creation of parodies of other works. In many situations, it is not obvious whether a use is a fair use. Courts interpret and apply the guidelines in specific cases. Law scholars say that results of fair use cases are often notoriously difficult to predict. The uncertainty itself can chill free speech. Fear of an expensive legal case can reduce creation of valuable new work that makes fair use of other works.

4.1.5 Ethical Arguments About Copying

There is intrinsic “fuzziness” about the ethics of copying. Many people who get their music, movies, or software from unauthorized sources realize they get “something for nothing.” They benefit from the creativity and effort of others without paying for it. To most people, that seems wrong. On the other hand, much copying does not seem wrong. We explore some of the reasons and distinctions.

Copying or distributing a song or computer program does not decrease the use and enjoyment any other person gets from his or her copy. This fundamental distinction between intellectual property and physical property is a key reason why copying is ethical in far more circumstances than taking physical property. However, most people who create intellectual property in entertainment, software, and so on, are doing so to earn income, not for the benefit of using their product themselves. If movie theaters and websites could show, or stream, copies of movies without paying for them, far fewer people and companies would invest money, time, energy, and creative effort in making movies. If search engines could scan any book and offer free downloads without an agreement with the publisher, publishers would probably not sell enough copies to cover costs; they would stop publishing. The value of intellectual property is not just the direct use and enjoyment one gets from a copy. Its value is also as a product offered to consumers to earn money. That is an aspect of the property that one can steal from the copyright holder. When people widely copy intellectual property without permission, they diminish the value of the work as an asset to the owner. That is why a lot of copying is wrong.

Supporters of unauthorized file-sharing services and people who advocate loose restrictions on copying intellectual property argue that permitting copying for, say, trying out a song or computer program before buying it benefits the copyright owner because it encourages sales. Such uses seem ethical, and indeed, since a lot of the “wrong” in unauthorized copying stems from depriving owners of income from their product, the fourth of the fair use guidelines considers the impact on the market for the product. However, we should be careful not to go too far in usurping a copyright holder’s decisions. Many businesses give free samples and low-priced introductory offers to encourage sales, but that is a business decision. The investors and employees of the business take the risk for such choices. A business normally makes its own decisions about how it markets its product, not consumers who want free samples, nor even the courts.

People who copy for personal use or distribute works of others without charge usually do not profit financially. Personal use is, reasonably, more likely to be fair use (both ethically and legally) than is commercial use, but is personal use always fair? Is financial gain always relevant? In some contexts, a profit motive, or financial gain, is a factor in concluding that an activity is wrong. In other contexts, it is irrelevant. Vandals do not profit financially from their action, but vandalism is unethical (and a crime) because it destroys—or reduces the value of—someone’s property. A profit motive is not a significant factor in determining where to protect freedom of speech. Freedom of speech is an important social, ethical, and legal principle for book, magazine, newspaper, and website publishers, most of whom are in business to make a profit. Many kinds of abusive or threatening speech are unrelated to financial gain but are unethical.

Here are some arguments people make in support of personal copying or posting content on the Web without authorization (in situations that are not clearly fair use) and some counterpoints to consider. The responses below do not mean that unauthorized copying or use of someone else’s work is always wrong—in many cases it is not. These are brief suggestions for analyzing the arguments.

· • I cannot afford to buy the software or movie or pay the royalty for use of a song in my video. There are many things we cannot afford. Not being able to afford something does not justify taking it.

· • The company is a large, wealthy corporation. The size and success of the company do not justify taking from it. Programmers, writers, and performing artists lose income too when copying is common.

· • I wouldn’t buy it at the retail price (or pay the required fee) anyway. The company is not really losing a sale or losing revenue. The person is taking something of value without paying for it, even if the value to that person is less than the price the copyright owner would charge. There are times when we get things of value without paying. Our neighborhood looks better when our neighbors paint their houses. People do us favors. It can be easy to ignore a crucial distinction: Who makes the decision?

· • Making a copy for a friend is just an act of generosity. Philosopher Helen Nissenbaum argued that someone who copies software for a friend has a countervailing claim against the programmer’s right to prohibit making the copy: the “freedom to pursue the virtue of generosity.” 11  Surely we have a liberty (i.e., a negative right) to be generous, and we can exercise it by making or buying a gift for a friend. It is less clear that we have a claim right (a positive right) to be generous. Is copying the software an act of generosity on our part or an act that compels involuntary generosity from the copyright owner?

· • This violation is insignificant compared to the billions of dollars lost to piracy by dishonest people making big profits. Yes, large-scale commercial piracy is worse. That does not imply that individual copying is ethical. And, if the practice is widespread, the losses become significant.

· • Everyone does it. You would be foolish not to. The number of people doing something does not determine whether it is right. A large number of people in one peer group could share similar incentives and experience (or lack thereof) that affect their point of view.

· • I want to use a song or video clip in my video, but I have no idea how to get permission.This is a better argument than many others. Technology has outrun the business mechanisms for easily making agreements. The “transaction costs,” as economists call them, are so high that a strict requirement for obtaining permission slows development and distribution of new intellectual property.

· • I’m posting this video (or segment of a TV program) as a public service. If the public service is entertainment (a gift to the public), the observations above about copying as a form of generosity are relevant here. If the public service is to express an idea or make some statement about an important issue, the posting might be analogous to creating a review or a parody. In some cases, these might be reasonable fair uses with social value. Simply posting a complete program, or a substantial portion of one, is probably not a fair use.

Laws are not always good guides for ethical decisions, but the fair use guidelines do a respectable job of identifying criteria to help distinguish fair and unfair copying. Because of the complexity of the issues, there will always be uncertainty in the application of the guidelines, both ethically and legally. The guidelines might need expansion and clarification to cover new media, but they give us a good framework that corresponds to sensible ethical criteria.

4.1.6 Significant Legal Cases

The fair use doctrine is important for different contexts. First, it helps us figure out under what circumstances we as consumers can legally copy music, movies, software, and so on. Second, developers of new software, recording devices, game players, and other products often must copy some or all of another company’s software as part of the process of developing the new product. The new product might compete with the other company’s product. Is such copying a fair use? We look at cases that cover these contexts. Some of the cases also involve the degree of legal responsibility a company has for copyright violations by users of its products or services. This point is important for many Web-based services, some that implicitly or explicitly encourage unauthorized uses of the works of others.

Sony vs. Universal City Studios (1984)

The Sony case was the first case about private, noncommercial copying of copyrighted work that the Supreme Court decided. 12  It concerns videotape recording machines, but it is cited in Web-based entertainment cases and in cases about new kinds of digital recording devices.

Two movie studios sued Sony for contributing to copyright infringement because some customers used its Betamax video cassette recording machines to record movies shown on television. Thus, this case raised the important issue of whether copyright owners can sue makers of copying equipment because some buyers use the equipment to infringe copyrights. First, we focus on the other issue the Supreme Court decided in the Sony case: whether recording a movie for personal use was a copyright infringement or a fair use. People copied the entire movie. Movies are creative, not factual, works. Thus, factors (2) and (3) of the fair use guidelines argue against the taping. The purpose of recording the movie was to view it at a later time. Normally the consumer reused the tape after viewing the movie, making it an “ephemeral copy.” The copy was for a private, noncommercial purpose, and the movie studios could not demonstrate that they suffered any harm. The Court interpreted factor (2), the nature of the copyrighted work, to include not simply whether it was creative or factual, but also the fact that the studios receive a large fee for broadcasting movies on television, and the fee depends on having a large audience of people who view the movies for free. So factors (1), (2), and (4) argue for fair use. The Court ruled that recording a movie for viewing at a later time was a fair use.

The fact that people copied the entire work did not necessitate a ruling against fair use, although many examples of fair use apply only to small excerpts. The fact that the copying was a private, noncommercial use was significant. The Court said that private, noncommercial uses should be presumed fair unless there is realistic likelihood of economic harm to the copyright holder.

On the issue of the legitimacy of the Betamax machine, the Court said makers of a device with substantial legal uses should not be penalized because some people use it to infringe copyright. This is a very important principle.

Reverse engineering: game machines

In the Sony case, the Supreme Court’s decision said that noncommercial copying of an entire movie can be fair use. In several cases involving game machines, the courts ruled that copying an entire computer program for a commercial use was fair, largely because the purpose was to create a new product, not to sell copies of another company’s product. The first case is Sega Enterprises, Ltd. v. Accolade, Inc. Accolade made videogames to run on Sega machines. To make their games run properly, Accolade needed to figure out how part of Sega’s game-machine software worked. Accolade copied Sega’s program and decompiled it (i.e., translated it from machine code to a form in which they could read and understand it). This is reverse engineering. Sega sued; Accolade won. Accolade was making new games. The court viewed Accolade’s activities as fitting the purpose of fair use—that is, to encourage production of new creative work. The fact that Accolade was a commercial entity was not critical. Although Accolade’s games might reduce the market for Sega’s games, that was fair competition. Accolade was not selling copies of Sega’s games. 13  In Atari Games v. Nintendo, the court also ruled that making copies of a program for reverse engineering (to learn how it works so that a company can make a compatible product) was not copyright infringement. It is a fair “research” use.

The court applied similar arguments in deciding in favor of Connectix Corporation in a suit by Sony Computer Entertainment, Inc. Connectix copied Sony’s PlayStation BIOS (the basic input–output system) and reverse engineered it to develop software that emulates the PlayStation console. Game players could then buy the Connectix program and play PlayStation games on their computers without buying the PlayStation console. Connectix’s program did not contain any of Sony’s code, and it was a new product, different from the PlayStation console. The copying of the BIOS was fair use. 14

These decisions show how courts interpret fair use for situations not imagined when the guidelines were written. Reverse engineering is an essential process for creating new products that must interact with other companies’ hardware and software.

Sharing music: the Napster case

· When Big Steel and the auto industry were under pressure during the ’70s from low-cost imports, their first instinct was not to change their outmoded manufacturing plants but to beseech the courts to bar the outlanders. The record industry has taken a similar tack.

· —Karl Taro Greenfeld 15

Napster opened on the Web in 1999 as a service allowing users to copy songs in MP3 files from the hard disks of other users. It was wildly popular and had more than 50 million users little more than a year later. Almost 100 million MP3 files were available on the service. Webnoize found that almost 75% of college students it surveyed used Napster. It was well known that Napster users copied and distributed most of the songs they traded without authorization. Eighteen record companies sued for copyright infringement and asked for thousands of dollars in damages for each song traded on Napster. The record companies won. 16

The Napster case is important for many reasons. The fact that so many people participated in an activity that courts decided was illegal is an indication of how new technology challenges existing law and attitudes about what is acceptable. Many people thought the success of Napster meant the end of copyright. Instead the court decision showed that the legal system can still have a powerful impact. The arguments in the case apply to many other sites and services on the Internet.

The issues in the lawsuit against Napster were the following:

· • Was the copying and distribution of music by Napster users legal under the fair use guidelines?

· • If not, was Napster responsible for the actions of its users?

Napster argued that the sharing of songs by its users was a legal fair use. Let’s review the fair use guidelines and how they apply.

Copying songs via Napster does not fit any of the general categories of purposes covered by fair use (e.g., education, research, news), but neither does copying movies on tapes. The Sony v. Universal City Studios case showed that the Supreme Court is willing to include entertainment as a possible fair use purpose.

Napster argued that sharing songs via its service was fair use because people were making copies for personal, not commercial, use. Copyright experts said “personal” meant very limited use—say, within a household—not trading with thousands of strangers.

Songs (lyrics and music) are creative material. Users copied complete songs. Thus, fair use guidelines (2) and (3) argue against fair use, but, as the Sony case indicated, they do not necessarily outweigh other factors.

The final, and perhaps most important, point is the impact on the market for the songs—that is, the impact on the income of the artists and music companies that hold the copyrights. Napster argued that it did not hurt record industry sales; users sampled music on Napster and bought the CDs they liked. The music industry claimed Napster severely hurt sales. Survey and sales data did not unequivocally support either side. Sales data showed sales rising significantly during most years in the 1990s, and dropping or rising only slightly in 2000. For example, music sales in the United States (the largest market) dropped 1.5% in 2000. Sales of singles were down 46%. 17  We do not know if Napster was the only reason for the declines, but it is reasonable to conclude that the huge volume of copying on Napster had a negative impact on sales and that the impact would grow.

Many legal observers thought the large-scale copying by Napster users was illegal copyright infringement, not fair use, and that is how the court ruled.

But was Napster responsible for copyright infringement by its users? Napster did not keep copies of songs on its computers. It provided lists of available songs and lists of users logged on at any time. Users transferred songs from each other’s hard disks using peer-to-peer software downloaded from Napster. Napster argued that it was similar to a search engine and that a new law, the Digital Millennium Copyright Act (which we discuss at length in Sections 4.2.2 and 4.2.3), protected it from responsibility for copyright violations by its users. The record companies argued that the law requires companies to make an effort to prevent copyright violations and that Napster did not take sufficient steps to eliminate unauthorized songs or users who committed violations.

Napster cited the Sony Betamax case, in which the court said the maker of devices with substantial legitimate uses is not liable for users of the device who infringe copyrights, even if the maker knows some will. Napster had substantial legitimate uses in promoting new bands and artists who were willing to let users copy their songs. The recording industry argued that Napster was not a device or new technology, and it was not asking to ban a technology or shut Napster down. The record companies objected to how Napster used widely available technology to aid copyright infringement. It wanted Napster to stop listing songs without permission of the copyright owners.

Sony’s relationship with a customer ended when the customer bought the Betamax machine. Napster interacted with its members in providing access to songs they copied. The court said Napster was liable because it had the right and ability to supervise its system, including the copyright-infringing activities, and had a financial interest in those activities. Napster was a business. Although it did not charge for copying songs, it expected the free copying to attract users so that it would make money in other ways.

The court ruled in 2001 that Napster “knowingly encourages and assists in the infringement of copyrights.” 18  Napster faced civil suits that could have required payments of billions of dollars in damages. After some ineffective attempts to remove unauthorized songs from its song lists, Napster shut down. Another company bought the “Napster” name and set up a legal streaming music subscription service.

What consumers want from the entertainment industry

Why was Napster so popular? When I asked my college students (while the illegal version of Napster was thriving in 2000), many shouted out “It’s free!” That’s the obvious reason, but it was not the only one. My students quickly generated a list of other desirable features of Napster. They could get individual songs with-out having to buy a whole CD to get the ones they wanted. They could sample songs to see if they really wanted them. Through Napster, they had access to a huge “inventory,” not limited to one particular store or music label. They could get songs that were not commercially available. They liked the convenience of getting their music online. They could download and play a song from anywhere; they did not need to have a physical CD with them. The Napster site provided information about singers and musicians. Users could chat online with other users while they downloaded songs in the background. Thus, Napster used a variety of then-new technologies to provide flexibility, convenience, and services, in addition to free music.

The record companies did not embrace the new technologies. They expected their customers to continue to buy CDs from stores or order on the Web and wait a few days for shipping. They were used to the old paradigm of getting paid by each customer for each copy and were reluctant to allow or accept distribution of songs in file formats that people could easily copy.

When people began to post video clips from television shows and movies about five years later, content owners reacted like the record companies. They tried to stop the phenomenon rather than seek new business models to make it work.

More file sharing: MGM v. Grokster

About the time of the Napster decision, numerous companies and Web sites (Gnutella, Morpheus, Kazaa, and others) sprang up to provide a new kind of peer-to-peer file-sharing service. These systems enabled copying of files among users on the Internet without a central service, like Napster, to sue when users infringe copyrights. Within months of Gnutella’s appearance, more than a million files were available. Many were unauthorized MP3 music files and unauthorized software. In MGM v. Grokster, the music and movie industry sued Grokster and StreamCast Networks (the owner of Morpheus). Although the companies did not provide a central service or list of music files available on the disks of users (as did Napster), they provided the software for sharing files. Technologists and supporters of file sharing argued that peer-to-peer file-transfer programs had potential for many productive, legal uses. (They were correct.) However, the Supreme Court ruled unanimously that intellectual property owners could sue the companies for encouraging copyright infringement. (At about the same time, an Australian court made a similar ruling against Kazaa.)

The Napster and Grokster decisions made it clear that businesses that encourage copyright infringement and provide tools to do so as a fundamental part of their business model cannot operate legally in the United States. Many file-sharing companies settled suits with the entertainment industry, paying millions of dollars. Many shut down. Critics of the decisions worried that they threatened development of new peer-to-peer technology and applications.

Plagiarism and copyright

Plagiarism is the use of someone else’s work (usually written work), representing it as one’s own. Among students, it typically means copying paragraphs (with perhaps small changes) from websites, books, or magazines and incorporating them, without attribution, into a paper the student submits for a class assignment. It also includes buying a term paper and submitting it as one’s own work. Novelists, nonfiction writers, and journalists sometimes plagiarize sections or complete works from other authors. Plagiarism was a problem before there were computers, but word processors and the Web have made it easier by making so much information available and making copying as easy as cut and paste.

Most often, the author of the plagiarized material does not know of or authorize its use, so plagiarism often includes copyright infringement. If the material is in the public domain or if someone agrees to write a paper for another, it is not copyright infringement, but it still might be plagiarism.

Plagiarism is dishonest. It misappropriates someone else’s work without permission (usually) and without credit. In academia, it is a lie to the instructor, a false claim to have done an assignment oneself. In journalism or publishing, it is a lie to the employer or publisher and to the public. Plagiarism violates school rules and is considered a serious breach of professional ethics.

Thousands of high schools and colleges submit student term papers and essays to a service, turnitin.com, that checks them for plagiarism. Turnitin compares the student work to its database of millions of student papers and to material on the Web and in journal archives. The service builds its database of student papers by adding those submitted for checking. Several students sued the company for infringing their copyrights by adding their papers to the database. A federal appeals court ruled that turnitin.com’s storage of student term papers is a fair use. Turnitin copied the entire paper and is a commercial entity. However, the facts that it provides a service very different from writing a term paper and that its service does not reduce the market for a student’s paper weighed more strongly. 19

Social conventions can influence the determination of what is plagiarism. For example, the public and book publishers generally know that ghostwriters write books for politicians and celebrities even when only the politician’s or celebrity’s name appears as the author. Few call this practice plagiarism.

“Look and feel”

Does copyright apply to user interfaces? The term “look and feel” of a program refers to features such as pull-down menus, windows, icons, and finger movements and the specific ways one uses them to select or initiate actions. Two programs that have similar user interfaces are sometimes called “workalike” programs. The internal structure and programming could be entirely different. One program might be faster or have other advantages. Should the look and feel of a program be copyrightable? Does a workalike program infringe the copyright of the earlier program it resembles?

In the 1980s and 1990s, some companies won copyright infringement suits against others whose software had similar look and feel. An appeals court, reversing one such case, ruled that menu commands are “a method of operation,” explicitly excluded from copyright protection. They are, the court said, like the controls of a car. 20  The trend of court decisions has been against copyright protection for “look and feel.” Courts ruled that features such as overlapping windows, pull-down menus, and common operations like cut and paste are outside the scope of copyright.

The main argument in favor of protecting a user interface is that it is a major creative effort. Thus, the usual arguments for copyright and patent apply (e.g., rewarding and encouraging innovation). On the other hand, standard user interfaces increase productivity of users and programmers. We do not have to learn new interfaces for each program or device. Programmers do not have to “reinvent the wheel”—that is, design a new interface just to be different. They can concentrate on developing the truly new aspects of their programs. The value of similar interfaces for browsers, smartphones, and so on, is now well recognized and taken for granted. *

4.2 Responses to Copyright Infringement

4.2.1 Defensive and Aggressive Responses From the ContentIndustries

The entertainment industry employs numerous approaches in its efforts to prevent unauthorized use of its products. Its methods include technology to detect and thwart copying, education about copyright law and reasons to protect intellectual property, lawsuits (both reasonable and abusive), lobbying for expansions of copyright law (both reasonable and not), lobbying to restrict or prohibit technologies that aid copyright infringement, and new business models to provide digital content to the public in convenient forms.

Ideas from the software industry

A variety of techniques for protecting software developed early, with varying success. For example, software companies encoded an expiration date in free sample versions of software; the software destroyed itself after that date. Some expensive business software included a hardware dongle, a device that the purchaser has to plug into a port on the computer so that the software will run, thus ensuring that the software runs on only one machine at a time. Diskettes containing consumer software had “copy protection” to ensure that you could not copy it (or that a copy would not run). Some software requires activation or registration with a special serial number. Some of these systems were “cracked”—that is, programmers found ways to thwart the protection mechanisms. Many companies dropped these techniques, largely because consumers dislike the inconvenience that accompanies them. Some of these early access controls later developed into the more sophisticated digital rights management schemes for entertainment and ebooks that we discuss later in this section.

*Several companies have patents on the screen technologies that enable touch commands.

International piracy

Some countries traditionally have not recognized or protected intellectual property, including copy-rights, patents, and trademarks. Counterfeiting of brand name products, from blue jeans to expensive watches and medicines, is common in some parts of the world. Ignoring foreign copyrights on books and other printed material has long been common practice in many countries as well. Thus, software, music, and movie piracy in these countries are variants of an old phenomenon. Websites that sell or share games, software, and entertainment files without authorization thrive in many countries.

The Business Software Alliance (BSA), a software industry organization, estimates that piracy accounts for 42% of personal computer software in use worldwide. The regions with the highest rates are Central and Eastern Europe and Latin America. 21  (Obviously, it is difficult to get accurate figures for illegal activities. BSA makes estimates by considering the number of computers sold, the expected average number of software packages on each computer, and the number of software packages sold.)

Many countries with high piracy rates do not have a significant software industry. Thus, they do not have domestic programmers and software companies to lobby for protection of software. The lack of a domestic software industry may be an effect, as well as a contributing cause, of weak legal protection for software. It is difficult for such an industry to develop when it cannot recover its investment in software development. The fact that the major software companies are from other countries, and rich ones, may make both the people and the governments less inclined to take action to reduce unauthorized sales. In the United States, with its many legitimate sellers of entertainment and software, customers are likely to know when they are buying illegal products or sharing unauthorized files. In countries where it is common to purchase food unpackaged in outdoor markets, customers might not think there is anything unusual (or wrong) about the way unauthorized vendors sell software and music. It could be easier for a consumer to find a street vendor selling, say, a U.S. movie on DVD, than to find an authorized dealer. Another reason for piracy in some countries is that the economies are weak and the people are poor. (Some U.S. movie companies sell DVDs in China at relatively low prices to attract customers away from the illegal market.) Thus, culture, politics, economic development, low incomes, and lax enforcement of intellectual property laws are all contributing factors.

The BSA calculated that the software piracy rate in China was 98% in 1994. The U.S. government has repeatedly pressured China’s government to improve intellectual property protection, and China has repeatedly announced programs to do so, but with relatively little impact. As China’s economy has grown, its government has made more effective efforts to reduce illegal production, sale, and use of intellectual property. Recognition that poor intellectual property protection hindered its own content industries contributed to increased copyright protection in China. For example, under pressure from a Chinese company that represents U.S. music companies and owns rights to thousands of Chinese songs, China’s major search engine removed thousands of links to sites that offered pirated songs. Court decisions against infringement of foreign copyrights and jail sentences for offenders increased. In China, personal computer manufacturers used to sell their machines bare, without an operating system. This practice encouraged people to buy cheap, unauthorized copies. In 2006, the Chinese government required that all PCs be sold with an authorized operating system preinstalled. Also, according to the BSA, the Chinese government significantly reduced the use of unauthorized software by its own government agencies. The BSA reports that the software piracy rate in China dropped to 78% in 2010. (A Chinese study, based on surveys, reported a 45% rate for 2010.) 22  For comparison, the BSA gives a rate of 20% for the United States.

Decoys

Some music companies adopted a clever tactic to discourage unauthorized file sharing: They put a large number of damaged music files, called “decoys,” on file-sharing sites. The decoys might, for example, fail to download properly or be full of scratchy noises. The idea was that people would become frustrated and stop using the file-sharing sites if a large percentage of the songs they tried to download would not play properly. Movie companies adopted the tactic too, scattering many fake copies of new movies on the Internet.

Software industry organizations, dubbed “software police,” were active in business offices before they began policing cyberspace. In most cases, violations of copyright law were so clear that the business or organization agreed to big fines rather than go to trial. Software copying by businesses decreased, due in part to better understanding of the ethical issues involved and in part to fear of fines and exposure in a business climate that gradually came to view large-scale copyright violation as not acceptable.

Law enforcement agencies raided swap meets, warehouses, and other sites and prosecuted sellers of pirated software (and, later, music CDs and movie DVDs). Courts handed out severe penalties for organized, large-scale efforts. For example, the owner of iBackup received a prison sentence of more than seven years and was ordered to pay restitution of more than $5 million after pleading guilty to illegally copying and selling software worth more than $20 million. Similarly, a man who repeatedly recorded new movies on his camera in movie theaters and made pirate copies to sell received a sentence of seven years in jail.

Banning, suing, and taxing

Via both lawsuits and lobbying, the intellectual property industries have delayed, restricted, or prevented services, devices, technologies, and software that make copying easy and that people are likely to use widely in ways that infringe copyrights, although they also have many legal uses. The technology for consumer CD-recording devices for music was available in 1988, but lawsuits filed by record companies delayed its introduction. A group of companies, including a television network and the Walt Disney Company, sued the makers of digital video recording machines that store TV programs and can skip commercials. The movie and record industries delayed introduction of DVD players by threatening to sue companies that make them if consumers could copy movies on the devices. The Recording Industry Association of America (RIAA) sued in 1998 and obtained a restraining order to stop Diamond Multimedia Systems from shipping its Rio machine, a portable device to play MP3 music files. Diamond eventually won, partly because the court interpreted the Rio as a player, not a recorder, that allowed people to play their music at different locations—just as the Sony decision (Section 4.1.6) said people could watch TV shows at different times. 23  Some observers believe that Apple’s iPod would not have been possible if the RIAA’s lawsuit against the Rio had succeeded.

As new companies introduced a variety of new products and services to deliver entertainment in flexible and convenient ways, the costs of fighting industry lawsuits effectively shut some of them down—with no trial to decide whether their products were legal.

The entertainment industry pushed hard for laws and industry agreements to require that makers of personal computers and digital recorders and players build copy-protection mechanisms into their products. It pressured device makers to design their systems so that files in unprotected formats do not play well—or at all. Such requirements could reduce illegal copying, of course. However, they interfere with use and sharing of homemade works. They complicate sharing of material in the public domain. They restrict legal copying for personal use and other fair uses. Laws requiring or prohibiting specific features violate the freedom of manufacturers to develop and sell products they consider appropriate.

Software and entertainment companies targeted Internet service providers, threatening legal action against those whose subscribers operate file-sharing services or trade unauthorized files via peer-to-peer software, pressuring them to cancel accounts of alleged offenders. The entertainment industry sued or took other legal action against thousands of people for downloading or sharing unauthorized music files. Letters to college students threatened fines of thousands of dollars. Eventually, recognizing that the lawsuits angered customers and were not particularly effective in stopping copying and sharing, the industry cut back on the policy of mass lawsuits. Instead, the industry made agreements under which ISPs warn customers who transfer music or movies illegally and may close the accounts of customers who ignore the warnings.

As an alternative to banning devices that increase the likelihood of copyright infringement, several governments, including most in the European Union, tax digital media and equipment to pay copyright holders for losses expected from unauthorized copying. They introduced special taxes on photocopiers and magnetic tape in the 1960s and later added taxes on personal computers, printers, scanners, blank DVDs, recorders, iPods, and cell-phones. Advocates of these taxes argue that makers of copying equipment are responsible for losses their equipment causes for intellectual-property owners and that the tax schemes are a reasonable compromise in a situation where it is difficult to catch each infringer.

Analogies and perspective

Should we ban or restrict software, a technology, a device, or research because it has the potential for illegal use, or should we ban only the illegal uses? This question addresses a principle covering much more than copyright infringement. In  Chapter 2 , we described the FBI’s and NSA’s pressure for banning telephone technology that is difficult to tap and encryption schemes that were difficult for them to crack. Law enforcement agencies advocate banning anonymous Web browsing and email, because they can hide criminal activity. The issue of banning or restricting tools that have criminal uses arises in numerous areas unrelated to computer technology. Some U.S. cities prohibit sale of spray paint to minors, because they might paint graffiti on walls. Of course, they might paint a table. Some cities ban chewing gum, because some people discard the gum on the street, making a mess. Many countries prohibit ordinary people from owning guns to protect their homes or businesses, because some people misuse guns. Laws ban drug paraphernalia, because people might use it with illegal drugs. Some of these laws make prevention of specific crimes easier. For example, it might be hard to find the person who painted graffiti, but it is easy to reduce the sale of spray paint by threatening shop owners with fines.

In a free society, which should win: the freedom of people to develop and use tools for legal purposes, or the prevention of potential crimes? Those who reject the policy of banning a tool that has both legitimate and illegal uses argue its absurdity by taking it to its extreme: Should we ban matches because arsonists use them? Others argue that we should look at each application individually, considering the risks of harm. Proponents and lobbyists for bans on tools usually rank the damage they could cause (in general or to the interests of their clients) more highly than the loss of freedom and convenience to those who would use the tool honestly and productively. We can rarely predict all the creative and innovative (legal) uses of a new technology. Bans, delays, and expensive restrictions often cost all of society the unforeseen benefits. The technologies listed in Section 4.1.2 as causes of problems for intellectual-property protection are the foundation of incredible benefits that we enjoy.

Critics argue that the taxes make equipment more expensive, penalize equipment makers unfairly, charge honest users unfairly, and politicize the difficult job of fairly distributing the money collected.

Digital rights management

Digital rights management technologies (DRM) are a collection of techniques that control access to and uses of intellectual property in digital formats. DRM includes hardware and software schemes using encryption and other tools. DRM implementations embedded in text files, music, movies, ebooks, and so on, can prevent saving, printing, making more than a specified number of copies, distributing a file, extracting excerpts, or fast-forwarding over commercials.

 More about encryption: Section 2.5.1

There are many criticisms of digital rights management. DRM prevents fair uses as well as infringing uses. It can prevent extraction of small excerpts for review or for a fair use in a new work, for example. You cannot play or view protected works on old or incompatible machines and operating systems (e.g., Linux). We have long had the right to lend, resell, rent out, or give away a physical book, record, or CD that we owned. (These activities do not require making a copy.) If we could not lend or give a book to a friend, the friend might buy a copy, providing income to the copyright owner. But in 1908, the Supreme Court established the principle that the copyright owner has the right only to the “first sale” of a copy. 24  Publishers, especially of textbooks, which resell often, lobbied for legislation requiring a royalty to the publisher on each resale; they were unsuccessful. DRM enables the content seller to prevent lending, selling, renting, or giving away a purchased copy.

DRM differs in a fundamental way from the banning, suing, and taxing we described earlier. Companies that use DRM on their products are not interfering with other people or businesses. They are offering their own products in a particular way. It is a way that has disadvantages to the public, but surely a publisher should be free to offer its products in whatever form it chooses. If the car model we want to buy comes only in black, white, or green, we cannot demand that the company provide one in orange. But we can buy one and paint it orange. Can we do the equivalent with intellectual property wrapped in DRM? In the next section, we will see that a law says we often cannot.

4.2.2 The Digital Millennium Copyright Act: Anticircumvention

Congress passed the Digital Millennium Copyright Act (DMCA) in 1998. This very important law has two significant parts. The “anticircumvention” provisions prohibit circumventing technological access controls and copy-prevention systems implemented by copyright owners in intellectual property. The “safe harbor” provisions protect websites from lawsuits for copyright infringement by users of the site. We discuss the anticurcumvention provisions in this section and safe harbor in the next one.

Circumventing access controls

Programmers and researchers regularly find ways to crack or thwart (or “circumvent”) DRM, sometimes to infringe copyright on a large scale and sometimes for a variety of legal purposes. The “anticircumvention” provisions of the DMCA prohibit making, distributing, or using tools (devices, software, or services) to circumvent DRM systems used by copyright holders. (There are exceptions. We mention some later.) These provisions are extremely controversial. The law provides for heavy penalties and fines for violators. The ideal purpose of the DMCA is to reduce piracy and other illegal uses of intellectual property. However, it criminalizes actions that do not infringe any copyrights. It outlaws devices and software that have legitimate purposes, which court decisions protected before the DMCA. Content companies use the law in ways that threaten fair use, freedom of speech, research, competition, reverse engineering, and innovation. We give some examples. 25

The first major legal cases based on the DMCA involved the Content Scrambling System, or CSS, a protection scheme for movies. Three programmers, including 15-year-old Jon Johansen of Norway, * wrote and distributed a program, called DeCSS, that defeated the scrambling. 26  DeCSS could be used to create numerous unauthorized copies. But DeCSS also enables users of the Linux operating system to view (legally purchased) DVDs on their computers. It enables the legal owner of a DVD to view the disk anywhere in the world. (Some movie companies use incompatible codes in Europe and the United States.) Several Hollywood studios sued people who posted DeCSS on their websites. Attorneys in a prominent case argued that people could use DeCSS for fair uses, that banning it violated freedom of speech, and that programmers need to discuss computer code and techniques. None of these arguments mattered much. The judge ruled that DeCSS was illegal under the DMCA and ordered its removal from the Web. 27  Soon after the decision, descriptions of DeCSS appeared on the Web in haiku, bar code, short movies, a song, a computer game, and art. 28  Most of these publications of the code were protests of the judge’s decision. They demonstrate how difficult it is to distinguish between expression of an opinion, which the First Amendment strongly protects, and computer code, a form of speech the judge said the government could more easily regulate. †  Jon Johansen was tried in Norway under a Norwegian law. The Norwegian court ruled that it was not illegal to break DVD security to view legally purchased DVDs and that the prosecutors had not proved Mr. Johansen used the program to illegally copy movies. In the United States, the movie industry continued to win cases.

A team of researchers responded to a challenge by the Secure Digital Music Initiative (SDMI), an industry consortium, to test its digital watermarking schemes (a form of copyright protection) for music files. The researchers quickly found ways to thwart several of the techniques and planned to present a paper on the flaws in the protection schemes at a conference. The head of the research group, Princeton University computer science professor Edward Felten, said SDMI threatened lawsuits based on the DMCA. He decided not to present the paper. 29  The DMCA has exceptions for actions necessary for encryption research and computer security, but the scope of the exceptions is limited and unclear. This case showed that the DMCA and the industry’s threats of lawsuits have a chilling effect on publication of research. Software engineering journals worried about liability for some research papers they might publish. A major book publishing company decided not to publish a planned book on security vulnerabilities in popular game consoles. A computer science professional organization argued that fear of prosecution under the DMCA could cause researchers and conferences to leave the United States, eroding its leadership in the field. Felten and other researchers sued for a court ruling that the anticircumvention provisions of the DMCA (when applied to software and research) violate the First Amendment. The case ended after the recording industry and the government issued statements that lawsuits under the DMCA against scientists and researchers studying access control technologies were not appropriate. 30

*The others chose to remain anonymous.

† Recall that encryption export rules (discussed in Chapter 2), like the DMCA, restricted publication of research and software, but eventually a judge ruled that software is a form of speech.

 Russian programmer arrested for violating the DMCA: Section 5.4.1

We saw in several cases in Section 4.1.6 that courts ruled, before the DMCA, that copying for reverse engineering to produce new products was a fair use. Now, people and companies avoid reverse engineering because the legality under the DMCA remains murky. New, innovative products that might have come to market, but do not because of the DMCA, are invisible.

Smartphones, tablets, game machines, and other devices have mechanisms to prevent installation of software or use of services that the maker of the device does not supply or approve. Cracking such mechanisms is sometimes called jailbreaking, unlocking, or rooting. *  Originally, for example, Apple allowed only AT&T service contracts for iPhones; George Hotz figured out how to circumvent this limitation, as well as limitations on Sony game machines. Jailbreaking certain devices also lets users disable the feature that allows remote deletion of an app from the user’s device. These uses do not infringe copyright. However, Apple 31  threatened DMCA lawsuits against a website that hosted discussion of reverse engineering iPods so that they could work with software other than iTunes. Other companies threatened suits for similar dicussions for other devices.

Exemptions

The Library of Congress decides on exemptions to the DMCA’s anticircumvention provisions. 32  It now allows circumvention of CSS for fair use purposes. It allows an exemption for research on security vulnerabilities in access controls on CDs, but not as yet for research on such vulnerabilities for, say, video games. It ruled in 2010 that it is legal to alter phones to install third-party software (e.g., apps) or to use an alternate service provider. But the rule does not allow the same actions, for similar purposes, on other devices. 33  There is an exemption for circumventing access controls on ebooks to allow use of text-to-speech software (a useful function for blind people). However, the circumvention is legal only if all existing editions of the book have access controls that prevent enabling a read-aloud function or a screen reader.

As these examples illustrate, the exemptions the Library of Congress grants are very narrow. Many allow only a small action that does not infringe copyright and was legal before the DMCA. The exemptions come after years of threats, legal expenses, and delays in innovating new products or using lawfully purchased products. This is a very poor way to structure a law.

*I am using the terms informally, not with technical definitions.

4.2.3 The Digital Millennium Copyright Act: Safe Harbor

The “safe harbor” provisions of the DMCA protect websites from lawsuits and criminal charges for copyright infringement when users post infringing material. The site operators must make a good-faith attempt to keep infringing material off their sites. They must remove such material when asked to do so by the copyright owners (often publishers and music and movie companies). They can lose the protection if they profit from the infringing material. Like the safe harbor provisions of the Telecommunications Act of 1996 (Section 3.1.1) for other kinds of illegal content, this was a welcome protection for website owners and the public. It recognized that websites with user content have tremendous social value, but operators could not review everything members post. The safe harbor provisions of the DMCA, along with technological advances in the next several years, encouraged the development of thousands of websites that host user-generated content, including blogs, photos, videos, recipes, reviews, and the myriad other creative works we share on the Web. Holding the sites legally liable for copyright-infringing material a user might post could have severely restricted this phenomenon.

On the other hand, such sites include a huge amount of copyrighted material, from short clips from movies, TV shows, and concerts to entire movies and other shows. Copyright owners request removal of their content (and links to their content) by sending so-called takedown notices. Entertainment companies began sending floods of takedown notices. Infringing material appears and reappears faster than content owners can find it and request its removal. * 34  The entertainment industry and other content companies are unhappy that they have to bear the responsibility and expense of continually searching sites for material that infringes their copyrights and sending the takedown notices. They question the applicability of the safe harbor provision to large commercial websites such as YouTube. The companies argue that the large advertising revenue these sites take in depends in part on the unauthorized content. The safe harbor provision requiring the takedown notices might have been appropriate for legitimate websites of the 1990s whose business plans did not depend on users posting huge amounts of content. Today’s sites, they argue, are similar to the peer-to-peer music sites (like Grokster) that made their money on the intellectual property of others without permission. They argue that the sites should have the responsibility of filtering out copyright-infringing material. The burden should not be on the copyright holders. Supporters of the safe harbor provisions fear that weakening safe harbor protection would threaten the many websites that host user-generated material. Viacom sued YouTube and asked for $1 billion in damages. 35  Viacom complained that it found 100,000 of its videos on YouTube. YouTube responded that it complied with the law. It regularly removes video clips when Viacom informs the company that the clips infringe Viacom copyrights. Video-sharing site Veoh won a similar case against a lawsuit by Universal Music both at the trial level and on appeal in 2011.

*Google, for example, takes down millions of links to copyright-infringing sites each year.

However, Veoh declared bankruptcy; it cited the huge legal costs. The Viacom case against YouTube, filed in 2007 and still in the courts, could clarify the extent of efforts a site must make to keep out infringing material. *  In the meantime, technology has helped reduce the burden. Much of the detection and removal of infringing material is now automated. The content industries, large video sites, and social-networking sites use sophisticated tools to search through user-generated content for copyrighted material posted without authorization.

Although the safe harbor provision was a generally positive and important move, the takedown process has some weaknesses for websites and the public, as well as for the copyright holders. The takedown requirement of the DMCA is clearly open to abuse that threatens free speech and fair competition. Copyright holders are likely to interpret fair use principles narrowly and send takedown notices for material that might be fair use. A study of takedown notices found for about 30% of the notices there is significant question whether the material actually does infringe copyright. The fair use provisions protect much of it—for example, quotations from a book in an unfavorable book review. In one incident, Wendy Seltzer, a law professor, posted a video clip from a football game. YouTube removed it after the National Football League sent a takedown notice, then reposted it when Seltzer claimed it was an educational fair use (demonstrating issues about copyright—the clip included the NFL’s copyright notice), then took it down again after the NFL sent another takedown notice. More than half of the notices businesses send to Google demanding removal of links to allegedly infringing Web pages come from competitors of the targeted sites. 36 How can search engine companies and websites evaluate all the takedown notices they receive? How should they respond when they receive a notice that they believe is intended to silence critics or undermine competitors? It is often not obvious how a court will interpret the fair use guidelines. Website operators are likely to protect themselves by complying with requests from large content companies with large legal staffs.

The entertainment industry and other content companies lobby to curtail the safe harbor provisions of the DMCA. 37  They argue that they need more legal tools to shut down pirate sites outside the United States. As in other situations where it is difficult to find or stop the people who are doing what the government wants to stop, the content companies would put more burden of enforcement (and penalties) on legitimate companies. For example, they advocate requiring ISPs to block access to designated infringing sites and requiring payment companies (e.g., Paypal and credit card companies) to stop processing payments to such sites. There is strong debate about how new stringent requirements would affect YouTube, search engines, Flickr, Twitter, and so on, as well as many small sites that do not have the staffs and expertise to comply. Critics of such requirements warn that the standards the industy uses to identify infringing sites are too vague and broad, that ISP blocking can open security vulnerabilities, and that blocking access and funding, once begun, tends to expand to other purposes and threatens freedom of speech. Piracy continues to be a major headache and cost for the creators and owners of intellectual property. The challenge continues to be finding effective ways to reduce it without burdening legitimate activities and businesses or thwarting innovation and development of new services.

*The DMCA is a U.S. law. Lawsuits in Europe have had varying results. YouTube won a case in Spain but lost one in Germany.

· Washington regulating the Internet is akin to a gorilla playing a Stradivarius.

· —–L. Gordon Crovitz, Information Age columnist for the Wall Street Journal 38

4.2.4 Evolving Business Models

· The more we attempt to provide government protection to the old ways of doing business, the less motivation we provide to the entertainment industry to adapt and benefit from new technology.

· —Les Vadasz, former vice president of Intel 39

The success of Apple’s iTunes, which has sold more than ten billion songs and tens of millions of videos, shows that companies can sell digital entertainment successfully, from the point of view of the customers and the rights holders. After the Supreme Court decision in MGM v. Grokster(Section 4.1.6), people who wanted to operate legitimate businesses providing music realized that they had to make agreements with, and payments to, music companies. The entertainment industry initially viewed new distribution technologies, such as peer-to-peer file-sharing, podcasting, and streaming content, as threats—as the movie industry did with video cassette recorders in the 1980s, before it got the idea that it could earn billions by renting and selling movies on cassettes. It seemed to take a long time, but many entertainment companies came to realize that people who share music files are people who like music; they are potential customers. The industry began to explore new business and marketing models. Music subscription services now thrive, with millions of songs available and hundreds of thousands of subscribers. They operate under agreements with the music companies. Similarly, many companies offer (authorized) movie download services.

For years, the music industry fought against distribution of music in (unprotected) MP3 format. Steve Jobs and some people in the entertainment industry argued that DRM was ineffective against piracy. Between 2007 and 2009, a major shift occurred in music sales. EMI Group, Universal Music Group, and Sony (some of the largest music companies in the world) began selling music without DRM. Apple eliminated DRM from its iTunes store for music. The debate about DRM continues within the movie and book industries. Some see DRM as essential to protect against piracy. They fear the industries will suffer severe economic losses if they do not include access controls on digital content. Others point out that pirated movies circulate unprotected. Controls and restrictions on legally sold content encourage irritated consumers to seek out illegal, unprotected copies even though they are willing to pay.

Some entertainment companies and Web content-sharing sites negotiate contracts for the site to pay a share of its ad revenue to the entertainment companies. YouTube and Warner Music Group, for example, worked out such an arrangement for Warner music videos. Sharing sites can use filtering software that examines files as people upload them, looking for digital “fingerprints” of the entertainment company’s properties. Depending on agreements between the companies, the site can block a post entirely or pay the entertainment company for its appearance on the site. This is a creative way to allow users to post entertainment company material or include such material in their (usually noncommercial) creations without the overhead and legal liability for getting permissions. It makes sense that the Web companies that benefit from the advertising and have the assets and expertise to develop and use the sophisticated filtering tools make the payments, instead of individual users.

Safe harbor in the cloud?

Cloud services store a customer’s files on online servers so that they are synched among the customer’s devices and available from anywhere. Some cloud services enable sharing for small organizations or businesses. Cloud storage raises copyright issues. Is copying legally purchased files to and from the cloud a fair use? Will the companies operating the cloud services have any responsibility for unauthorized content their customers store and share on their servers? Unlike on public sites such as YouTube, an individual’s content stored in the cloud is not visible to other people or to movie and music companies. The copying is personal from the perspective of the user, but the cloud service operator stores and provides the content to the individual conveniently as part of its business. Since copyright holders do not see what is stored, they do not have the option of sending takedown notices. If cloud services adopt a system of filtering or checking for content that infringes copyright, how will they manage it to protect fair use and the privacy of the user?

What does not work

Some attempts at new business models do not work. Zediva, a small startup in 2011, bought DVDs. It rented them to customers, but it did not send the physical DVD. Instead, it streamed the movie to the renter. Zediva argued that if it could rent the physical DVD without authorization from the studios, as do services such as Netflix under the first sale doctrine (Section 4.2.1), then it should be legal to rent it digitally over the Internet, streaming a movie from one DVD to only one renter at a time. The movie studios argued that streaming a movie is a public performance, which requires authorization. A court agreed and Zediva shut down. 40  Does this interpretation of the law make sense? Should Zediva’s variant on streaming be legal?

Some business models appear intended to get around copyright law while helping people distribute illegally copied video. How far can they go? The Pirate Bay case (in Sweden, 2009) addressed the issue of whether the site violated Swedish copyright law by helping users find and download unauthorized copyrighted material (music, movies, computer games) even though the site itself did not host the material. Four organizers of the Pirate Bay were convicted of contributory copyright infringement. The Motion Picture Association of America has sued several other sites that do not host infringing videos but provide links to sites that do. It has won some of the suits. Do these sites differ in any fundamental way from the original Napster and Grokster? Should merely listing or linking to sites with unauthorized files be illegal?

Cyberlockers are services that provide storage of large files on the Web. Members transfer hundreds of thousands of files daily on popular sites. As on Napster more than a decade ago, singers and musicians store files on cyberlockers for free downloads to promote their work. The term cyberlocker, however, often refers to services that either intentionally encourage sharing files (e.g., movies) without authorization or structure their business in ways that make copyright infringement on a huge scale easy. The entertainment industry cites Megaupload, a cyberlocker that did more than $100 million in business (e.g., from membership fees), as an example of this form of piracy. Megaupload operated from Hong Kong and New Zealand, with servers in several countries, including the Netherlands. It had 180 million registered users. It claimed that its terms of use prohibited copyright infringement and that it took down infringing material when notified to do so. Determining whether a particular business illegally contributes to copyright infringement depends on consideration of the factors that are required for safe harbor protection and how seriously the business complies. The U.S. government shut Megaupload in 2012 (by legally seizing its domain names), and police in New Zealand arrested its founder and several employees. Other cyberlocker businesses modified some of their practices to protect themselves from legal action. 41

4.3 Search Engines and Online Libraries

Copying is essential to many of the operations and services of search engines. In response to search queries, search engines display copies of text excerpts from websites and copies from images or video. In order to respond to user queries quickly, they copy and cache *  Web pages and sometimes display these copies to users. Search engine companies copy entire books so that they can search them and display segments in response to user queries. Besides their own copying, search engines provide links to sites that might contain copyright infringing material. Individuals and companies have sued Google for almost every search service it provides (Web text, news, books, images, and video). Should search engines need authorization for the copying essential to search services? Should they be paying fees to copyright owners? As always, uncertainties about the legal status of industry practices can delay innovation. Google boldly introduces new services amid complaints of copyright infringement, but fear of lawsuits has deterred smaller companies that cannot estimate business costs in advance if they do not know their liability. We consider arguments related to a few of the contested practices.

*Caching, generally in computer science, means storing data in specialized memory, frequently updated, to optimize transfer of the data to other parts of a system that use it.

Tools for authorized sharing

Many authors and artists, including those who sell their work on the Web, are willing to share—to a degree. How can they easily—without a publishing company’s staff of lawyers and without the overhead of explicit authorization—indicate what they are willing to let others do with their work? From the user perspective, how does someone who wants to copy, say, a photo from someone else’s website determine if he or she must get permission or pay a fee? Many people are willing to respect the preferences of an author or artist, but it is often not easy to determine what those preferences are.

Creative Commons, 42  a nonprofit organization, developed a spectrum of licensing agreements inspired by the GNU General Public License for software (Section 4.4). The licenses, which the author or artist announces to viewers by a choice of clickable icons, explicitly permit a selection of actions normally requiring authorization from the copyright owner. They provide a large degree of flexibility. For example, one can allow or disallow copying for commercial uses, require a specified credit line with any use, allow copies or display of the entire work only if there are no changes, allow use of pieces of the work in new works, or put the entire work in the public domain. Like so much on the Web, the use of the licenses and associated software is free. The photo site Flickr is one of the largest users of Creative Commons licensing. Anyone who stores photos on Flickr can indicate what uses he or she permits.

Easy-to-use schemes like this eliminate confusion and expensive overhead. They facilitate and encourage sharing while protecting the wishes of intellectual property owners.

The search engine practice of displaying copies of excerpts from Web pages seems easily to fit under the fair use guidelines. The excerpts are short. Displaying them helps people find the website with the excerpted document—usually an advantage to the site. In most cases, the site from which the search engine copies the excerpt is public, available to anyone who wants to read its content. Web search services are a hugely valuable innovation and tool for the socially valuable goal of making information easily available. In Kelly v. Arriba Soft, an appeals court ruled that copying images from Web pages, converting them to thumbnail images (small, low-resolution copies), and displaying the thumbnails to search engine users did not infringe copyrights. In Field v. Google, an author sued Google for copying and caching a story he had posted on his website. Caching involves copying entire Web pages. The court ruled that caching Web pages is a fair use. In dismissing a similar suit that challenged both caching and the practice of displaying excerpts from a website, a court compared Google to an ISP that makes copies of Web pages to display them to users. For ISPs, automatically and temporarily storing data to transmit to users does not infringe copyright. 43

There are, however, some reasonable arguments on the other side. Most major operators of search engines are businesses. They earn significant revenue from advertising. Thus, the copying accomplishes a commercial purpose. The display of short excerpts can reduce income to copyright holders in some situations. A group of Belgian newspapers claimed they lose revenue from subscription fees when Google displays headlines, photos, and excerpts from their news archives. They won a lawsuit against Google (in a Belgian court) in 2007. In response to similar lawsuits and disputes with other news services, Google negotiated licensing agreements to copy and display headlines, excerpts, and photos.

Trademarked search terms

The practice of selling search terms raises intellectual property issues for trademarks. Businesses pay search engine companies to display the business’s ads when a user enters specific search terms. What if a business “buys” the name of another company or the names of some of its products? Users searching for one company will see its competitor’s ads. A company that makes software for learning foreign languages sued a competitor and Google over this issue. The case (Rosetta Stone Ltd. v. Google Inc.), filed in 2009, is still in the courts.

Books online

Project Gutenberg began converting books in the public domain into digital formats in the 1970s. Volunteers typed the entire text of the books—inexpensive scanners were not yet available. The University of California agreed to let Microsoft scan millions of books in its collection that are in the public domain. Google’s project of scanning books in major university libraries differs in that Google scans books covered by copyright. Google provides entire books for download, but only those that are in the public domain. For books still under copyright protection, Google Book Search provides short excerpts from the books. Does Google’s project infringe copyrights? How does the impact on the market for books differ from the impact of people browsing books in a library? How does it compare to providing excerpts from newspaper articles, as in the Belgian case we described above?

Publishers and authors filed several lawsuits against Google for copying their books. The court so far has rejected several versions of long, complex settlement agreements that Google and the publishers devised in The Author’s Guild et al v. Google, Inc.. The agreements cover, among other things, sharing of proceeds from sales of out-of-print works, setting prices, and how much of a book Google could display as fair use, without payment. The main reason for the judge’s rejection of the agreement in 2011 is that it would give Google significant rights to use books in the future in new ways, not related to the actions that the original lawsuit covered and without approval of copyright owners. It would also release Google from liability for some future actions. In effect, it rewards Google “for engaging in wholesale copying of copyrighted works without permission.” 44

A French publisher, La Martiniere, won a suit against Google (in France) for scanning its books and putting extracts online without permission. Google and La Martiniere made an agreement to split revenue from digital sales of books.

Similar legal and ethical issues arise again each time technology makes copying and searching of more complex content (movies, for example) possible, especially for content produced explicitly to earn revenue (again, movies, for example). We see that search engine companies sometimes negotiate contracts with major intellectual property owners for displaying excerpts from and/or providing links to content such as images, news archives, television programs, books, and so on. Such contracts recognize that the search companies benefit from the use of another company’s intellectual property, that some uses threaten the revenue of the copyright holders, and hence that, for both legal and ethical reasons, a search engine company might need permission to copy and display intellectual property for certain purposes.

4.4 Free Software

In  Chapter 1 , we talked about all the free stuff on the Web. Individuals post information and create useful websites. Large groups of volunteers, who do not know each other, collaborate on projects such as Wikipedia. Experts share their knowledge and contribute their work. This creation of valuable information “products” is decentralized. It has little or no “management” in the business sense. It flows from incentives other than profits and market pricing. This phenomenon, which some call “peer production,” has a predecessor: the free software movement, begun in the 1970s. 45

4.4.1 What Is Free Software?

Free software is an idea, an ethic, advocated and supported by a large loose-knit group of computer programmers who allow and encourage people to copy, use, and modify their software. The free in free software means freedom, not necessarily lack of cost, though often there is no charge. Free software enthusiasts advocate allowing unrestricted copying of programs and making the source code (the human-readable form of a program) available to everyone. Software distributed or made public in source code is open source, and the open source movement is closely related to the free software movement. (Commercial software, often called proprietary software, is normally sold in object code, the code run by the computer, but not intelligible to people. The source code is kept secret.)

Richard Stallman is the best-known founder and advocate of the free software movement. Stallman began the GNU project in the 1970s (though the GNU name dates from 1983). It began with a UNIX-like operating system, a sophisticated text editor, and many compilers and utilities. GNU now has hundreds of programs freely available and popular among computer professionals and skilled amateur programmers. *  In addition, thousands of software packages are available as free software, including audio and video manipulation packages, games, educational software, and various science and business applications. 46

Free software has many advantages. With freely distributed software, more people can use and benefit from a program. With source code available, any of thousands of programmers can find and fix bugs quickly. Users and programmers can adapt and improve programs. Programmers can use existing programs to create new and better ones. Stallman compares software to a recipe. We can all decide to add a little garlic or take out some salt without paying a royalty to the person who developed the recipe.

To enforce the openness and sharing of free software within the current legal frame-work that provides copyright protection, the GNU project developed the concept of copyleft. 47  Under copyleft, the developer copyrights the program and releases it under an agreement that allows people to use, modify, and distribute it, or any program developed from it, but only if they apply the same agreement to the new work. In other words, no one may develop a new program from a copylefted program and add restrictions that limit its use and free distribution. The widely used GNU General Public License (GPL) implements copyleft. Courts uphold copyright protection for open source software. A federal court said a person who distributes open source software can sue for an injunction against someone who uses the software for commercial products without following the open source licensing agreement. 48

For a long time, technically savvy programmers and hobbyists were the principal users of free software. Commercial software companies were hostile to the idea. That view changed gradually, then more dramatically, with the Linux operating system. †  Linus Torvalds wrote the Linux kernel in 1991. Torvalds distributed it for free on the Internet, and a global network of free software enthusiasts continue development. At first, Linux was difficult to use, not well suited as a consumer or business product. Businesses referred to it as “cult software.” Two early users were the company that did the special effects for the movie Titanic and the NASA Goddard Space Flight Center. Gradually, small companies began selling a version of Linux along with manuals and technical support, and, eventually, major computer companies, including IBM, Oracle, Hewlett-Packard, and Silicon Graphics, used, supported, and marketed it. Large businesses such as Royal Dutch/Shell and Home Depot adopted Linux. Several movie studios adopted Linux for their special effects and digital animations. Dell sold PCs with Linux installed. Other examples of popular free software include Firefox, the Web browser provided by Mozilla, and Apache, the most widely used program to run websites. Google’s mobile operating system, Android, which is Linux-based, has elements of free and open source software.

*“GNU” is an acronym for “GNU’s Not UNIX.” (Programmers like recursive acronyms.)

† Technically, Linux is the kernel, or core part, of the operating system. It is a variant of the earlier UNIX operating system. Other parts are from the GNU project, but the whole operating system is often referred to as Linux.

Major companies began to appreciate the benefits of open source. Several now make source code for their own products public, allowing free use in noncommercial applications. Sun Microsystems licensed the Java programming language under GPL. *  Adopting the view of the free software movement, companies expected that programmers would trust the software more if they could see how it operates. Programmers might be more likely to use it and to improve it. IBM placed full-page ads in major newspapers announcing that it “embraced Linux and the open-source movement as a pillar of e-business.” 49  IBM donates hundreds of its patents to the open source community. Free software became a competitor for Microsoft, and so those who are critical of Microsoft’s products and influence see it as a considerable social benefit.

Critics (and some supporters) of free software point out some of its weaknesses. Much free software is not easy for ordinary consumers to use. Often, there is no technical support number to call for help. (Programmers and users share information about problems and fixes on very active websites.) Because anyone can modify free software, there are many versions and few standards, creating a difficult and confusing environment for nontechnical consumers and businesses. Many businesses want to deal with a specific vendor from whom they can request enhancements and assistance. They are uncomfortable with the loose structure of the free software movement. Some of these weaknesses faded as businesses learned how to work with a new paradigm. New businesses developed to support and enhance free software (like Red Hat for Linux), and more established businesses embraced the movement.

The spirit behind free software and open source spread to other forms of creative work. For example, the Berkeley Art Museum provides digital artworks online with their source files and allows people to download and modifiy the art.

4.4.2 Should All Software Be Free?

Some people in the free software movement do not believe that copyright should protect software at all. They argue that all software should be open source, free software. Thus, here we consider not the question “Is free software a good thing?” but “Should free software be the only thing?” When considering this question, we must take care to clarify the context of the question. Are we looking at it from the point of view of a programmer or business deciding how to release software? Are we developing our personal opinion about what would be good for society? Or are we advocating that we change the legal structure to eliminate copyright for software, to eliminate proprietary software? We will focus on the last two: Would it be good if all software were free software? And should we change the legal structure to require it?

Free software is undoubtedly valuable, but does it provide sufficient incentives to produce the huge quantity of consumer software available now? How are free software developers paid? Programmers donate their work because they believe in the sharing ethic.

*Oracle acquired Sun in 2010.

They enjoy doing what they do. Stallman believes that many good programmers would work like artists for low pay out of commitment to their craft. Contributions, some from computer manufacturers, support some free software efforts. Stallman has suggested government grants to universities as another way of funding software.

Would such funding methods for free software be sufficient? Most programmers work for a salary, even if they write free software on their own time. Would the extra services for which a business could charge bring in enough revenue to support all software development? Would the free software paradigm support the kinds of consumer software sold in millions of copies? What other funding methods could developers use?

A supporter of free software used the analogy of listener-supported radio and television. It is a good analogy for free software, but not one for eliminating proprietary software, because most communities have one listener-supported station and numerous proprietary ones.

Stallman believes that proprietary software—particularly, the aspect that prohibits people from making copies and changes in programs without the software publisher’s approval—is ethically wrong. He argues that copying a program does not deprive the programmer, or anyone else, of use of the program. (We saw some counterarguments to this viewpoint in Section 4.1.5.) He emphasizes the distinction between physical property and intellectual property. He also points out that the primary purpose of copyright, as stated in the U.S. Constitution, is to promote progress in arts and sciences, not to compensate writers. 50

For those who oppose copyright and proprietary software completely, the concept of copyleft and the GNU Public License provide an excellent device for protecting the freedom of free software within the current legal framework. For those who believe there are important roles for both free and proprietary software, they are an excellent device with which the two paradigms can coexist.

4.5 Patents for Inventions in Software  *

· Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.

· —U.S. Patent Law (Title 35 U.S. Code, Section 101)

*Patent law is extremely complex. I use some terms informally, not in their precise legal meanings. The aim here is to present an overview of the controversies, not a legal analysis.

· A smartphone might involve as many as 250,000 (largely questionable) patent claims.

· —David Drummond, Chief Legal Officer of Google 51

Google, Apple, and Microsoft paid billions of dollars to buy thousands of wireless and smartphone patents. It is generally recognized that the companies do not buy the patents because they need them for products they are developing. They buy patents so that they can sue other companies for patent infringement when the other companies sue them for patent infringement. Google explicitly said it bid (billions of dollars) on thousands of Nortel patents to “create a disincentive for others to sue Google” and to protect continued innovation in Android and other projects. 52  It is common for news articles to refer to “arsenals of patents” and to explicitly call patents “weapons.” The large-scale defensive (and offensive) accumulation of patents is a symptom of problems with patents for innovations implemented in software and patents for business methods. (Business methods, in our context, include innovations such as one-click shopping, recommending products based on a customer’s history, privacy controls, pop-up ads, and marketing to smartphones.) Fierce controversies rage over such patents. One controversy is over whether there should be patents for business methods and software-based inventions at all. There is also controversy about many specific patents and about the criteria for granting software-related patents. Billions of dollars and future technology development depend on how these controversies are resolved.

How does—and how should—patent law apply to innovations implemented in software? We will consider both aspects of this question. First, we review the murky state of patent law in this area.

4.5.1 Patent Decisions, Confusion, and Consequences

Patents protect inventions by giving the inventor a monopoly for a specified period of time. * Patents differ from copyrights in that they protect the invention, not just a particular expression or implementation of it. Anyone else who wants to use the patented invention or process must get the authorization of the patent holder, even if the other person independently came up with the same idea or invention. One device or machine might involve many patents. Since 1895, for example, thousands of patents (with some estimates over 100,000), have been issued covering various aspects of the automobile. Laws of nature and mathematical formulas cannot be patented. Nor are patents to be granted for an invention or method that is obvious (so that anyone working in the field would have used the same method) or if it was in use by others before the filing of the patent application.

*Under current law, the period is 20 years from the time of application.

A patent holder can build and sell the patented device or devices using the patented element. Also, the patent holder may license others to do so for a license fee, or royalty. Businesses routinely pay license fees to use patented inventions in their products.

The U.S. Patent and Trademark Office (which I will simply call the Patent Office) evaluates patent applications and decides whether to grant them. In the early days of computing technology, the Patent Office did not issue patents for software. In 1968, it declared computer programs not patentable. In 1981, the Supreme Court said that while software itself is not patentable because it is abstract, a machine or process that includes software, and in which the sole new aspect is the innovation implemented in the software, could be eligible for a patent. In the following decades, the Patent Office issued thousands of patents, and the Federal Circuit court (which handles patent appeals) approved many, interpreting Supreme Court guidelines loosely. Patents now cover encryption algorithms, data-compression algorithms, one-click shopping and other e-commerce techniques, copy-protection schemes, news feeds, location-based services for smartphones, delivery of email to cellphones, and so on. The Patent Office has a backlog of more than 600,000 patent applications. It grants an estimated 40,000 software patents each year. With hundreds of thousands of companies producing software, there are simply not enough patent attorneys to review the patents and determine if a new software product would violate an existing patent. 53

Courts have made several attempts to clarify the conditions for innovations based in software to be patentable, often issuing decisions that reject prior criteria. Some decisions depended on whether software produced “a useful, concrete, and tangible result,” whether a business method “transforms a particular article into a different state or thing,” and whether the term “process” in patent law includes “methods.” If these phrases and terms do not seem to clarify the criteria, that is the point. A significant Supreme Court ruling in 2007 (KSR v. Teleflex) broadened the definition of “obvious” for rejecting patents. In 2010 (Bilski v. Kappos), the court reemphasized that a patent must not give control over an abstract idea or mathematical algorithm. The decision declared a previous standard for software patentability to be only a “useful and important clue,” not a determining factor, adding more fuzziness. 54  Justice Kennedy summed up the difficulties in making patent decisions and the court’s declining to make a clear, general decision about software patents:

· It is important to emphasize that the Court today is not commenting on the patentability of any particular invention, let alone holding that any of the abovementioned technologies from the Information Age should or should not receive patent protection. This Age puts the possibility of innovation in the hands of more people and raises new difficulties for the patent law. With ever more people trying to innovate and thus seeking patent protections for their inventions, the patent law faces a great challenge in striking the balance between protecting inventors and not granting monopolies over procedures that others would discover by independent, creative application of general principles. Nothing in this opinion should be read to take a position on where that balance ought to be struck. 55

We saw that application of the fair use criteria for determining copyright infringement leads to uncertain results. The situation for patents is far more confused and unsettled. Judgments in some patent cases are close to or above $1 billion. Uncertainty and lawsuits are expensive, and they delay innovation.

A few cases

Decisions about granting patents are complex, as are decisions about whether a device or method infringes a patent. Reasonable decisions require knowledge of details of the particular case, expertise in the area, and knowledge of history of related technology. Establishing that an invention is not obvious and is not in use is difficult in fast-developing fields such as Web and smartphone technologies, especially when the Patent Office staff must research and process a large number of patent applications. The Patent Office makes mistakes. Various organizations, including the Electronic Frontier Foundation, argue that many patented techniques are not particularly new or innovative. For example, Amazon.com generated a lot of criticism when it sued Barnesandnoble.com for violating its patent on one-click shopping. Many in the industry objected that the government should not have granted the patent in the first place. (The companies settled the suit without disclosing the terms.)

Paul Allen (co-founder of Microsoft) sued several companies (Google, Facebook, Apple, eBay, Netflix, AOL, and others) for violating four early patents related to now widely used e-commerce and Web-viewing features. A judge dismissed the suit in 2011 while the Patent Office reconsiders the patents. 56

Apple won a patent case against a maker of Android phones. It covers technology that allows a user to tap a touch screen to perform various tasks, such as calling a phone number that is in an email or text message. We can expect more lawsuits over software-related patents for smartphones.

Many Web users remember Amazon innovating the idea of recommending books to customers based on their previous purchases. But Amazon might not have originated the technique for doing so. IBM sued Amazon for violating several of its patents on e-commerce techniques. IBM had obtained a patent on electronic catalogues in 1994, before online retail was common. The patent covers a wide area, including targeted advertising and recommending specific products to a customer. Eventually, Amazon agreed to pay IBM a licensing fee. 57

Patent trolls

Some companies accumulate thousands of technology patents, including many of the type of software patents and business method patents we are discussing. The firms buy most or all of their patents from individuals or other companies. They do not make any products. They license the patents to others and collect fees. Intellectual Ventures (co-founded by former Microsoft executive Nathan Myhrvold) is an example. The firm has an estimated 30,000 patents. *  It says it has collected close to $2 billion in license fees. Some such companies make all or a significant part of their income by suing other companies for patent infringement (for hardware as well as software patents). Critics call these companies “patent trolls,” a pejorative term.

Some see the existence of patent-licensing firms as an indication of a serious flaw in the patent system. However, if the patents themselves are legitimate (still an open question for many), this business model is not unreasonable. Marketing and negotiating contracts for patent licenses are services that an inventor might have neither the skills for nor the desire to do. A person or company might be better at inventing and patenting new technologies than at implementing them in a successful business. In a highly specialized economy, the existence of firms that buy and license patents is not in itself a negative thing. There are many analogous services in other contexts. (For example, some farmers sell their crop well in advance of harvest to free themselves from risks of market fluctuation. Firms with expertise in economics and risk analysis are the buyers.) However, as many critics of the current state of software patent law observe, when companies collect patents mainly or only to bring lawsuits for patent infringement, the law does not seem to be serving the goal of encouraging innovation well.

4.5.2 To Patent or Not?

In favor of software patents

The main arguments for allowing patents for software-based inventions and certain business methods are similar to those for patents and copyright in general. They reward inventors for their creative work. By protecting rights to the work, they encourage inventors to disclose their inventions so that others can build upon them. They encourage innovation.

Before the digital age, inventions were physical devices and machines. A huge portion of the astounding number of innovative developments in computing and communication technology consists of techniques implemented in software. These inventions have contributed enormous value to all of us. We take many for granted now, but they were truly innovative. Someone thought them up and developed them. Patents help to reward those people ethically and fairly and to encourage more innovation. Patent protection is necessary to encourage the large investment often required to develop innovative systems and techniques.

Businesses routinely pay royalties and license fees for use of intellectual property. It is a cost of doing business, like paying for electric power, raw materials, and so on. Software-related patents fit into this well-established context.

Copyright covers some software, but it is not sufficient for all of it. Software is a broad and varied field. It can be analogous to writing or to invention. A particular computer game, for example, might be analogous to a literary work, like a novel, and copyright would be appropriate. On the other hand, the first spreadsheet program, VisiCalc, introduced in 1979, was a remarkable innovation that had enormous impact on ways of doing business planning and on the sales of computer software and hardware. Similarly, the first hypertext system, the first peer-to-peer system, and many of the innovations that make smartphones so useful have characteristics more like new inventions. Patent might be more appropriate for such innovations.

*Intellectual Ventures obtained some of its patents on inventions it developed.

Against software patents

Critics of software patents include those who oppose software patents in general as a matter of principle and those who conclude that the system developed so far has done a very poor job. Both see patents for software as stifling innovation, rather than encouraging it.

There are now so many software patents that it is difficult for software developers (individuals writing apps or large companies developing new technology) to know if their software infringes patents. Many software developers come up with the same techniques independently, but patent law does not allow them to use their own invention if someone else has patented it. The costs of lawyers to research patents and the risk of being sued discourage small companies from attempting to develop and market new innovations. Businesses cannot sensibly estimate costs of new products and services when lawsuits are so common and results so uncertain. Even large companies, as we indicated earlier, amass patents as defensive weapons for inevitable lawsuits.

If courts uphold patents for software techniques, common e-commerce and smart-phone features, and so on, then prices will go up and we will see more incompatible devices and inconsistent user interfaces. In Section 4.1.6, we reviewed earlier controversies about copyrighting user interfaces (the “look and feel” of software systems). The principle that evolved in those cases—that uniformity of interfaces is valuable and that the look and feel should not be copyrightable—suggests a similar principle for patentability of user interfaces for smartphones.

It is difficult to determine what is truly original and to distinguish a patentable innovation from one that preempts an abstract idea, mathematical formula, or fact of nature. (Indeed, many computer scientists see all algorithms as mathematical formulas.) The very fact that there are so many controversial software and business method patents argues against granting these kinds of patents. The Supreme Court has not been able to arrive at clear, consistent legal criteria. This legal confusion suggests that it might be better not to issue patents in these areas.

Evaluating the arguments

Some of the problems of software patents are problems of patents in general. That does not mean we should abandon them; most things have advantages and disadvantages. (It does suggest areas for possible improvement.) Lawsuits over patents for physical inventions are common. (The holder of the 1895 patent on an automobile sued Henry Ford.)

Intellectual property law is a subset of property rights law. For complex areas, it sometimes takes many years to work out reasonable principles. *  Software patent holders sue others who independently develop the same techniques, but all patents allow such suits. That is an unfair aspect of patents. Does it do significantly more damage for software-related inventions than for other inventions?

That there has been an enormous amount of innovation in the past decades is obvious. Looking at the same facts and trends, some see patents on software as essential to this innovation, whereas others see them as threatening it. While the patent system has some big flaws, it is likely one of the important factors that contributed to the centuries of innovation in the United States. Legal scholars and software industry commentators emphasize the need for clear rules so that companies can do their work without the threat of changing criteria and unforeseen lawsuits. So, is the idea of patenting software innovations fundamentally flawed, or is it that reasonable criteria have not yet developed? If the latter, is it better to stop granting such patents in the meantime, while better criteria develop, or is it better to continue to issue software patents?

Several Supreme Court justices stated in the Bilski case that, while certain patent criteria were useful for the industrial age, the information age and its new technologies need a new approach. We do not have a good new approach yet.

Chapter:- 7

7.1 Evaluating Information

· A little learning is a dang’rous thing; Drink deep, or taste not the Pierian spring; There shallow draughts intoxicate the brain, And drinking largely sobers us again.

· —Alexander Pope, 1709 1

7.1.1 The Need for Responsible Judgment

Expert information or the “wisdom of the crowd”?

· We can get the wrong answer to a question quicker than our fathers and mothers could find a pencil.

· —Robert McHenry 2

There is a daunting amount of information on the Web—and much of it is wrong. Quack medical cures abound. Distorted history, errors, outdated information, bad financial advice—it is all there. Marketers and public relations firms spread unlabeled advertisements through blogs, social media, and video sites. Search engines have largely replaced librarians for finding information, but search engines rank Web pages by popularity (at least partly) and give prominent display to content providers who pay them; librarians do not. Wikipedia, the biggest online encyclopedia, is immensely popular, but can we rely on its accuracy and objectivity when anyone can edit any article at any time? On social journalism sites, readers submit and vote on news stories. Is this a good way to get news? The nature of the Internet encourages people to post their immediate thoughts and reactions without taking time for contemplation or for checking facts. How do we know what is worth reading in contexts where there are no editors selecting the well-written and well-researched?

When we see a video of a currently popular performer singing with Elvis Presley, we know we are watching creative entertainment—digital magic at work. But the same technology can deceive. Video-manipulation tools (and increased bandwidth) provide the opportunity for “forging” people. A company developed an animation system that modifies video images of a real person to produce a new video in which the person is speaking whatever words the user of the system provides. Another system analyzes recordings of a person’s voice and synthesizes speech with the voice, inflections, and tones of that person. Combined, these systems will likely have many uses, including entertainment and advertising, but clearly people can also use them to mislead in highly unethical ways. 3  How do we know when someone is manipulating us?

Example: Wikipedia

To explore some issues of information quality, we consider Wikipedia. Wikipedia is a collaborative project among large numbers of strangers worldwide. It is huge, free, participatory, noncommercial, ad-free, and written by volunteers. The English edition has almost four million articles, more than 10 times as many as the long-respected Encyclopaedia Britannica, first published in 1768 and online since 1994. 4  Wikipedia is one of the Internet’s most-used reference sites. But are its entries true, honest, and reliable?

We expect encyclopedias to be accurate and objective. Traditionally, expert scholars selected by editorial boards write encyclopedias. Volunteers, not carefully selected scholars, write and continually edit and update Wikipedia articles. Anyone who chooses to participate can do so. People worry that the lack of editorial control means no accountability, no standards of quality, no way for the ordinary person to judge the value of the information. They argue that because hundreds of millions of people—anyone at all—can write or edit articles, accuracy and quality are impossible. Truth does not come from populist free-for-alls. Members of the staffs of political candidates have distorted the Wikipedia biographies of their candidates to make their bosses look better. Opponents and enemies regularly vandalize profiles of prominent people. The staff of a federal agency removed criticisms of the agency from its Wikipedia article. Discredited theories about historic events such as the terrorist attacks on September 11, 2001, and the assassination of John F. Kennedy reappear regularly. A lawyer reported that one party in a case edited Wikipedia entries to make information appear more favorable to that party. (Jurors are not supposed to consult online sources about a trial, but some do.) Removing false information, hoaxes, and the like requires constant effort, according to Wikipedia volunteers. The Encyclopaedia Britannica has had errors and oddities, but the nature of Wikipedia makes it prone to more. Anonymity of writers encourages dishonesty. Open, volunteer, instant-publishing systems cannot prevent errors and vandalism as easily as publishers of printed books or closed, proprietary online information sources.

In spite of the errors, sloppiness, bad writing, and intentional distortions, most of Wikipedia is, perhaps surprisingly, of high quality and extraordinary value. Why? What protects quality in large, open, volunteer projects? First, although anyone can write and edit Wikipedia articles, most people do not. Thousands write and edit regularly, not millions. Most are educated and have expertise in the subjects they write about. They correct articles promptly. (Wikipedia saves old versions, so it can restore an article someone has vandalized.) After well-publicized incidents of manipulation of articles, Wikipedia’s managers developed procedures and policies to reduce the liklihood of such incidents. For example, they lock articles on some controversial topics or people; the public cannot directly edit them.

We, as users, can (and must) learn to deal appropriately with side effects or weaknesses of new paradigms. Even though so much of Wikipedia is excellent and useful, we learn that someone might have wrecked the accuracy and objectivity of any individual article at any hour. We learn that articles on technology, basic science, history, and literature are more likely to be reliable that those on politics, controversial topics and people, and current events. We learn to use Wikipedia for background, but to check facts and alternative points of view. Should we judge Wikipedia (and, by extension, the mass of information on the Web) by the excellent material it provides or by the poor-quality material it includes?

· Written by fools for the reading of imbeciles.

· —An evaluation of newspapers, not websites, by a character in Joseph Conrad’s novel The Secret Agent (1907)

The “wisdom of the crowd”

People ask all sorts of questions on Yahoo! Answers (and other sites like it) about dating, make-up, food, college (“Are online college classes as good as classroom classes?”), and wide-ranging technical, social, economic, and political issues (“If we can produce enough food to feed everyone in the world, why don’t we?”) Of course, a lot of answers are ill-informed. The questioner designates the posted answer he or she deems the best. What qualifies the questioner, presumably a person who does not know the answer, to judge the worthiness of the replies? To what extent does the ease of posting a question reduce the likelihood that a person will seek out well-researched or expert information on the subject? There are obviously questions for which this kind of forum might not provide the best results. (An example might be: Is it safe to drink alcohol while using an acne medicine?) However, the first two sample questions I quoted above are likely to generate a lot of ideas and perspectives. Sometimes that is exactly what the questioner wants. Without the Web, if someone asked questions like those of only a few friends, the answers might be less varied and less useful.

Some health sites on the Web encourage the public to rate doctors, hospitals, and medical treatments. Are such ratings valuable or dangerous? Will they motivate doctors and hospitals to change their practices to achieve higher ratings at the expense of good medical care? Steve Case, co-founder of AOL and founder of a health site that emphasizes ratings by the public, argues that if millions of people participate, the results will be very useful. Others are extremely suspicious of “the wisdom of the crowd.” And there is always concern about manipulation. Websites have sprung up to buy and sell votes to get prominent display for articles on social media sites. What are the implications of such practices for sites where the public rates medical care? Will providers of new or questionable medical treatments generate fake favorable reviews and votes? Will responsible operators of sites that display material based on rankings or votes anticipate manipulation and protect against it?

Let’s pause briefly to put the problems of incorrect, distorted, and manipulated information in perspective. Quack medical cures and manipulative marketing are hardly new. Product promotions not labeled as advertising date back hundreds of years. Eighteenth-century opera stars paid people to attend performances and cheer for them or boo their rivals. “Hatchet jobs” in the form of news articles, books, ads, and campaign flyers have dishonestly attacked politicians long before the Web existed. There are plenty of poorly written and inaccurate books. Historical movies merge truth and fiction, some for dramatic purposes, some for ideological purposes. They leave us with a distorted idea of what really happened. Two hundred years ago, cities had many more newspapers than they do today. Most were opinionated and partisan. At supermarket counters, we can buy newspapers with stories as outlandish as any online. The New York Times is a prime example of a respected newspaper, staffed by trained journalists, with an editorial board in charge. Yet one of its reporters fabricated many stories. Numerous other incidents of plagiarism, fabrication, and insufficient fact-checking have embarrassed newspapers and television networks.

OK, the problems of unreliable information are not new. But they are problems, and the Web magnifies them. So we consider two questions: How good is the wisdom of the crowd? And how can we distinguish good sources of information on the Web?

Researchers find that crowds do, in fact, generate good answers to certain kinds of questions. When a large number of people respond, they produce a lot of answers, but the average, or median, or most common answer is often a good one. This works well when the people are isolated from each other and express independent opinions. Some researchers think a large (independent) group is likely to be more accurate than a committee of experts for a variety of questions such as estimating economic growth or how well a new product or movie will do. (A Canadian mining company, perhaps hoping for such a phenomenon, posted a large set of geological data on the Web and held a contest to choose areas to look for gold.) However, when people see the responses provided by others, some undesirable things happen. People modify their responses so that the set of responses becomes less diverse, and the best answer may no longer stand out. People become more confident from reinforcement even though accuracy has not improved. The wisdom of crowds depends on diversity and independence. In social networks (as well as in-person teams working on projects in businesses, organizations, and government agencies), peer pressure and dominant personalities can reduce the wisdom of the group. 5

How can we distinguish good sources of information on the Web? Search engines and other services at first ranked sites by the number of people who visit them. Some developed more sophisticated algorithms to consider the quality of information on sites where users provide content. (In response, some sites added editors and fact-checking to improve quality.) A variety of people and services review and rate sites and blogs. Critics of the quality of information on the Web and the lack of editorial control disdain such ratings as merely popularity contests, contending, for example, that the Internet gratifies the “mediocrity of the masses.” 6  For blogs, as for Wikipedia or health care sites, they argue that popularity, voting, and consensus do not determine truth. That is correct, but there is no magic formula that tells us what is true and reliable either on the Web or off the Web. That a large number of people visit a website does not guarantee quality, but it provides some information. (Why have newspapers long published “best seller” lists for books?) We can choose to read only blogs written by Nobel Prize winners and college professors, if we wish, or only those recommended by friends and others we trust. We can choose to read only product reviews written by professionals, or we can read reviews posted by the public and get an overview of different points of view.

Over time, the distinction between the online equivalents of responsible journalism and supermarket tabloids becomes clear. Good reputations develop, just as they have for decades offline. Many university libraries provide guides for evaluating websites and the information on them. (I list some at the end of this chapter.) One good step is to determine who sponsors the site. If you cannot determine the sponsor of a site, you can consider its information as reliable as the information on a flyer you might find under your car’s windshield wiper when you park in a busy parking lot. Ultimately, we must find sites, reviewers, ratings, editors, experts, and other sources we trust. Good judgment and skepticism are always useful.

· The only way to preserve the wisdom of the crowd is to protect the independence of the individual.

· —Jonah Lehrer 7

Vulnerable viewers

Since you are reading this book, you probably are a student, a reasonably well-educated person who is learning how to analyze arguments and make good judgments. You can develop skills to evaluate material you read on the Web. But what about people who have less education or ability? For example, what risks does bad information pose to children who find it on the Web? Some critics of the Web worry most about the impact of inaccurate information on such vulnerable people. The fears of some seem to edge toward a belief that we (or experts, or the government) should somehow prevent such information from appearing. The many strong arguments for freedom of speech in general are arguments against any centralized or legally mandated way of accomplishing this. What can we do to improve the quality of information? Basic social and legal forces help (to a degree): freedom of speech (to provide responses, corrections, alternative viewpoints, and so on), teachers and parents, competition, fraud and libel laws—and people who care, who volunteer to write, review, and correct online information. What else can we do to reduce access to dangerously wrong information by vulnerable people?

Narrowing the information stream

All the problems of junk and nonsense on the Web notwithstanding, the Web now gives us access to more high-quality, up-to-date information than libraries did in the past. Consider current events, politics, and controversial issues. We can read and listen to thousands of news sources on the Web from our own and other countries, getting different cultural and political perspectives on events. We can read the full text of government documents—bills, budgets, investigative reports, congressional testimony and debate— instead of relying on a few sentences quoted from an official news release or a sound bite from a biased spokesperson. We can search archives of millions of news articles from the past 200 years. We can follow websites, blogs, tweets, and social media news of conservatives, liberals, libertarians, tea party activists, environmentalists, evangelical Christians, animal rights activists, and so on, far more easily and cheaply than when we had to seek out and susbscribe to their print newsletters and magazines. But what do people actually do? Some get all their news and interpretation of events from a small number of sites that reflect a specific political point of view. Online tools make it easy: you just set up your bookmarks and feeds and never look anywhere else, except at other sites recommended by the ones you frequent. Some critics see the Web as significantly encouraging political narrowness and political extremes by making it easy for people to avoid seeing alternative opinions.

The phenomenon of using the information that is easy to get applies to other fields besides politics, of course. I hear sad complaints from librarians and experienced researchers: Too many students and professionals believe whatever they read in Wikipedia. Researchers “reinvent the wheel” (or apply for grants to do so) because they do not read relevant work in their field published in the past and available only in paper journals. Because there is so much on the Web, it is too easy to ignore what is not there. How serious is the problem of ignoring older, undigitized work? Is it a temporary problem that will go away when virtually all research is available electronically?

A researcher analyzed millions of academic articles published over 50 years and found that as journals moved online, authors tend to cite fewer articles, more recent ones, and articles from a narrower set. The speculation is that researchers using search engines to find articles related to their work select from among the ones that appear high in search results—the ones that are already cited frequently. Those articles might indeed be the most important, but this approach reinforces previous choices and can lead researchers to miss less popular but very relevant work. Researchers have far more (and easier) access to articles and journals online than they had in the stacks of libraries. However, as the author of the study says, searching online “puts researchers in touch with prevailing opinions, but this may accelerate consensus and narrow the range of findings and ideas built upon.”8 The effect of accelerating consensus and narrowing results is similar to what researchers saw with the wisdom of the crowd when crowd members were not independent, though the mechanism is different. Clearly, it is good for researchers to be aware of this phenomenon and to broaden their searches when appropriate. The number of scholarly papers published each year has grown enormously (to, very roughly, a million yearly). Is it the tendency to use search tools in a somewhat lazy way—or the sheer number of papers—that causes some valuable work to be missed?

Idiots and dunderheads

· A fool and his money are soon parted.

· —Old English proverb

New technologies can have the unintended side effect of diminishing older skills. Computing technology has reduced the use of cursive writing, for example. Microsoft made a conscious decision with the effect of diminishing language skills. The thesaurus in Microsoft Word 2000 (and some later versions) lists the verb “trick” as the only meaning for “fool.” It omits noun synonyms “clown,” “blockhead,” “idiot,” “ninny,” “dunderhead,” “ignoramus,” and others—all present in earlier versions. Standard references such as dictionaries and Roget’s Thesaurus contain some of these and more choices.

Microsoft said it eliminated words “that may have offensive uses.” 9 *  Was this a dunderheaded decision that dulls the language and reduces literacy? Do producers of widely used reference works have an ethical responsibility to report the substance of their field accurately, or a social responsibility to remove potentially offensive words from the language?

*

Microsoft restored some synonyms meaning a foolish person but continues to omit the more colorful and more offensive terms.

If we receive too much information that does not interest us, we stop reading it. To counter this problem, Facebook implemented algorithms to filter news feed updates from friends based on how recently a member communicated with them. But of course, sometimes we want to hear from those people we have not heard from in a long time. What better methods could Facebook use? And is this relevant to social issues beyond personal relationships? It is. Eli Pariser, president of (liberal) MoveOn.org, includes conservatives among his Facebook friends because he wants to be aware of views different from his own. Over time, he realized he was no longer receiving updates from them (because he did not communicate with them regularly). Although Facebook members can turn off the filtering of news feeds, most people are not aware of it. Pariser considered the problem of filtered information so disturbing that he wrote a book about it. 10  What lessons can we learn from Facebook’s filtering? It is not ideal to use Facebook as our main source of access to political discussion. Facebook’s choice of a default setting (filtering turned on) might not be best (but, then again, most people might prefer it). More fundamentally, the problem of determining what information is relevant and desired does not have an easy or obvious solution. We observed (in the box in  Section 3.2.4 ) that in some situations filtering out too much is better than filtering out too little, while in other situations the opposite is true. Any solution that a search engine or social media service adopts will not be perfect. However, when people do not know that they are seeing filtered information, they do not know to turn off the filter or to look elsewhere for more information. There should be a clear indication when filters are active and an explanation of what they do.

How else does the Web narrow information streams? In  Chapter 2  we saw that search engines personalize results for users based on their location, past searches, profile information, and other criteria. Given the huge amount of information on the Web, this fine tuning helps us find what we want quickly. It is very valuable. However, it does mean that when we are searching for something outside our usual context, including perhaps information on controversial subjects, we might have to make an effort to look a little harder.

Do the various aspects of the Internet that narrow our information stream significantly diminish access to different points of view on controversial social and political topics? Members of radical political groups (left and right) and cults got information and opinions from narrow sources well before the Web. Does the Web encourage or increase ideological isolation? Does it simply reflect the choices that some people have often made in the past? Does it make it more likely that we will see a variety of points of view? When we criticize aspects of the Web, it is helpful to look to human nature and the past for perspective. It is also helpful to look toward an ideal to suggest improvements.

Abdicating responsibility

· I have a spelling checker.

· It came with my PC.

· It plainly marks four my revue,

· Miss steaks aye can knot sea.

· Eye ran this poem threw it,

· I’m sure your pleased too no.

· It’s letter perfect in it’s weigh,

· My checker tolled me sew.

· —Jerrold H. Zar, “Candidate for a Pullet Surprise” 11

The tools and technologies we use encourage certain practices and consequences by making them easier. The spelling-checker verse above humorously illustrates the problem of doing what the tool makes easy and ignoring other important tasks. Software can check the spelling of all the words in a document faster than a person can find the first one by flipping through the pages of a printed dictionary. But a simple spell checker looks up each word only to discover whether it is in its dictionary. It does not check whether the writer uses the word properly. *

* Grammar checkers were rudimentary when the poem first circulated on the Internet. They would now catch some of the errors.

The convenience of using a computer system and abdication of responsibility to exercise judgment can encourage a mental laziness with serious consequences. A trucker in Britain got his truck stuck on a small farm road after ignoring a sign saying the road was not suitable for large vehicles. He was unquestioningly following the directions of a navigation system. A newspaper editor in Pakistan received a letter to the editor by email and inserted it into the newspaper without reading beyond the title. The letter was an attack on the prophet Muhammad. Angry Muslims set fires in the newspaper office. Several editors were arrested and charged with blasphemy, sometimes punishable by death. 12  Back when newspaper content was still typeset and copyedited, such an accident would have been unlikely.

Businesses make decisions about loan and insurance applications with the help of software that analyzes risks. School districts make decisions about the progress of students and the careers of administrators on the basis of computer-graded and -calibrated tests. They sometimes make bad decisions because of ignorance of the kinds of errors that limitations of the system can cause. Law enforcement agents arrested people when a check of an FBI database showed an arrest warrant for someone with a similar name. Do officers think that because the computer displayed the warrant, the system has decided that the person they are checking is the wanted person? Or does an officer understand that the system simply displays any close matches and that the responsibility for the arrest decision lies with the officer?

Sometimes reliance on a computer system rather than human judgment becomes “institutionalized” in the sense that an organization’s management and the legal system can exert strong pressure on individual professionals or employees to do what the computer says. In bureaucracies, a decision maker might feel that there is less personal risk (and less bother) in just accepting what the software produces rather than doing additional checking or making a decision the software does not support. Computer programs advise doctors on treatments for patients. It is critical to remember that, in complex fields, the computer systems might provide valuable information and ideas but might not be good enough to substitute for an experienced professional’s judgment. In some institutions, when something goes wrong, “I did what the program recommended” is a stronger defense (to superiors or against a lawsuit) than “I did what my professional judgment and experience recommended.” Such institutions are encouraging abdicatation of personal responsibility, with potentially harmful results.

7.1.2 Computer Models

· Likeness to truth is not the same thing as truth.

· —Peter L. Bernstein 13

Evaluating models

Computer-generated predictions based on mathematical models of subjects with important social impact frequently appear in the news.  Figure 7.1  shows a few examples. A mathematical model is a collection of data and equations describing, or simulating, characteristics and behavior of the thing studied. The models and simulations of interest to us here require so much data and/or computation that they must be run on computers. Researchers and engineers do extensive modeling to simulate both physical systems, such as the design for a new car or the flow of water in a river, and intangible systems, such as parts of the economy. Models allow us to simulate and investigate the possible effects of different designs, scenarios, and policies. They have obvious social and economic benefits: They help train operators of power plants, submarines, and airplanes. They enable us to consider alternatives and make better decisions, reducing waste, cost, and risk. They enable us to project trends and plan better for the future.

Figure 7.1 Some problems studied with computer models.

Although the models we consider are abstract (i.e., mathematical), the meaning of the word “model” here is similar to its meaning in “model airplane.” Models are simplifications. Model airplanes generally do not have an engine, and the wing flaps might not move. In a chemistry class, we could use sticks and balls to build models of molecules to help us understand their properties. The molecule models might not show the components of the individual atoms. Similarly, mathematical models do not include equations for every factor that could influence the outcome. They often include simplified equations because the correct ones are unknown or too complicated. For example, we use a constant known as the acceleration of gravity in a simple equation to determine when an object dropped from a high place will hit the ground. We ignore the effect of wind in the equation, but, on some days, wind could make a difference.

Physical models are usually not the same size as the real thing. Model planes are smaller; the molecule model is larger. In mathematical models, it is time rather than physical size that often differs from reality. Computations done on a computer to model a complex physical process in detail often take more time than the actual process takes. For models of long-range phenomena, such as population growth and climate change, the computation must take less time than the real phenomenon for the results to be useful.

Predictions from expensive computers and complex computer programs impress people, but models vary enormously in quality. Some are worthless. Others are very reliable. Politicians and special interest groups use model predictions to justify multibillion-dollar programs and laws with significant impact on the economy and the standard of living and choices of millions of people. It is important for both computer professionals and the general public to have some idea of what is in such computer programs, where their uncertainties and weaknesses might lie, and how to evaluate their claims. It is the professional and ethical responsibility of those who design and develop models for public issues to describe honestly and accurately the results, assumptions, and limitations of their models.

The following questions help us determine the accuracy and usefulness of a model.

· 1. How well do the modelers understand the underlying science or theory (be it physics, chemistry, economics, or whatever) of the system they are studying? How well understood are the relevant properties of the materials involved? How accurate and complete are the data?

· 2. Models necessarily involve assumptions and simplifications of reality. What are the assumptions and simplifications in the model?

· 3. How closely do the results or predictions of the model correspond with results from physical experiments or real experience?

Among three models developed to predict the change in health care costs that would result if the United States adopted a national health system, the predictions varied by hundreds of billions of dollars. Two of the models predicted large increases and one predicted a drastic decrease. 14  Why was there such a difference? There are both political and technical reasons why models might not be accurate. Political reasons, especially for this example, are probably obvious. In addition to technical reasons that the questions above suggest (incomplete knowledge of the system being modeled, incomplete or inaccurate data, and faulty assumptions or oversimplification), other reasons are that computing power could be inadequate for the number of computations needed to model the full complexity of the system, and the difficulty, if not impossibility, of numerically quantifying variables that represent human values and choices.

Figure 7.2 Factors in diaper life cycle modeling.

Are reusable (washable cloth) diapers better for the environment than disposable diapers? When environmentalists proposed bans and taxes on disposable diapers, this controversy consumed almost as much energy as diaper manufacturing. Several modelers developed computer models to study the question. We call this particular kind of model a life cycle analysis. It attempts to consider the resource use and environmental effects of all aspects of the product, including manufacture, use, and disposal. To illustrate how difficult such a study might be,  Figure 7.2  lists a few of the questions about which the modelers made assumptions. Depending on the assumptions, the conclusions differed. 15  It is worth noting also that the models focused on one quality—environmental impact. To make a personal decision, we might consider the results of such a model (if we think it reliable), and we might also consider other factors such as cost, aesthetics, convenience, comfort, and health risks.

The U.S. Army Corps of Engineers uses mathematical models to predict how long an artifically constructed or replenished beach will last before waves wash it away. Two geologists have explained weaknesses in these models. 16  Among other simplifying assumptions, the models assume that all waves have the same wavelength, that all waves come from the same direction, and that all grains of sand are the same size. A model uses only 6 of 49 parameters that might affect the amount of sand washed away. Even if these 6 are the most important (or if the model included all 49), the appropriate values for a particular beach are uncertain. Often, say the critics, the beaches do not last as long as the models predict, partly because the models do not accurately provide for relevant but irregular natural phenomena such as big storms.

Example: Modeling car crashes *

Car crash analysis programs use a technique called the finite-element method. They superimpose a grid on the frame of a car, dividing the car into a finite number of small pieces, or elements. The grid is entered into the program, along with data describing the specifications of the materials making up each element (e.g., density, strength, and elasticity). Suppose we are studying the effects on the structure of the car from a head-on collision. Engineers initialize data to represent a crash into a wall at a specified speed. The program computes the force, acceleration, and displacement at each grid point and the stress and strain within each element. It repeats these calculations to show what happens as time passes in small increments. These programs require intensive computation to simulate 40–100 milliseconds of real time from the impact.

* An earlier version of this section appeared in my chapter, “Social and Legal Issues,” in An Invitation to Computer Science by G. Michael Schneider and Judith L. Gersting, West Publishing Co., 1995. (Used with permission.)

A real crash test can cost several hundred thousand dollars. It includes building and testing a unique prototype for a new car design. The crash analysis programs allow engineers to consider alternatives—for example, to vary the thickness of steel for selected components, or change materials altogether—and discover the effect without building another prototype for each alternative. But how good are the programs?

How well is the physics of car crashes understood? How accurate and complete are the data? Force and acceleration are basic principles. The physics involved in these programs is straightforward. Engineers know the relevant properties of steel, plastics, aluminum, glass, and other materials in a car fairly well. However, although they understand the behavior of the materials when force is applied gradually, they know less about the behavior of some materials under abrupt acceleration, as in a high-speed impact, and their behavior near or at breaking point. There are good data on the density, elasticity, and other characteristics of materials used in the model.

What simplifications do the programs make? The grid pattern is the most obvious. A car is smooth, not made up of little blocks. Also, time is continuous. It does not pass in discrete steps. The accuracy of a simulation depends in part on how fine the grid is and how small the time intervals are. Current computer speeds allow updating the calculations on fine grids with small time intervals (e.g., one millionth of a second).

How do the computed results compare to actual crash tests on real cars? High-speed cameras record real crash tests. Engineers attach sensors to the car and mark reference points on the frame. They compare the values the sensors record with values the program computes. They physically measure the distortion or displacement of the reference points, then compare these measurements to the computed positions of the points. Starting with the results of the physical crash, the engineers use elementary physics to calculate backward and determine the deceleration and other forces acting on the car. They compare these to the values computed in the simulation. The conclusion? Crash analysis programs do an extremely good job. In part because of the confidence that has developed over time in the validity of the results, engineers use variations of the same crash analysis modeling programs in a large variety of other impact applications, including those in Figure 7.3 .

Engineers who work with the crash analysis programs do not believe that they will or should eliminate physical crash testing. The computer program is an implementation of theory. Results could be poor if something happens that the program designers simply did not consider. The crash analysis programs are excellent design tools that enable increases in safety with far less development cost. The physical crash test is confirmation.

Figure 7.3 Other uses of crash analysis programs.

Example: Modeling climate

The earth has Ice Ages and warm interglacial periods. We are now in an interglacial period that is more than 11,000 years old. Within such periods, climate varies over time and in different parts of the world. For example, the Northern Hemisphere experienced both a medieval warm period about a thousand years ago and a later colder period (roughly 1550–1850), sometimes called the Little Ice Age.

* CO2 concentration has increased by almost 40% since 1750. 17  The older data come from measurements of gases trapped in ice cores drilled in Antarctica and Greenland.

Climate change is a very complex phenomenon. Solar radiation warms the earth. Some of the heat is reflected back, and gases trap some in the atmosphere. The latter phenomenon is known as the greenhouse effect. Without it, the temperature on the earth would be too cold to support life. Water vapor is the main greenhouse gas, but there are several other significant greenhouse gases as well. Among those whose concentration has been increased by human activity (in particular, burning of fossil fuels), carbon dioxide (CO2) is most important. An upward trend in CO2 concentration began roughly 16,000 years ago. However, since the beginning of the Industrial Revolution, CO2concentration has been increasing at a faster rate. *  Between the period 1850–1899 and the period 2001–2005, average global temperature rose roughly 0.76°C. * 18  The increase has been steeper since roughly 1980. The global temperature increase raised concern about the threat of excess global warming, possibly caused by human-induced increase of CO2 and other greenhouse gases in the atmosphere. Global warming predictions are based on computer models of climate. We consider those models. Since 1990, the Intergovernmental Panel on Climate Change (IPCC), sponsored by the United Nations and the World Meteorological Organization, has published comprehensive reports on the science of climate change and the quality and projections of climate models roughly every five years. Much of the information in this section comes from those reports. 19

Climate models, like the car crash analysis models, calculate relevant variables for grid points and elements (grid boxes) for specified simulated time intervals. The grid circles the earth, rises through the atmosphere, and goes down into the ocean. The models contain information about the sun’s energy output; the orbit, inclination, and rotation of the earth; geography (a map of land masses); topography (mountains, etc.); clouds; sea and polar ice; soil and air moisture; and a large number of other factors. Equations simulate atmospheric pressure, temperature, wind speed and direction, moisture, precipitation, ocean currents, and so forth. Researchers use climate models to study several aspects of future climate. They try, for example, to determine the effect of doubling CO2 concentration in the atmosphere. (Current trends suggest the concentration will have doubled, from its approximate level at the beginning of the 20th century, by some time in the 21st century. † ) Models also project the likely increase in global temperature, sea level, and other climate characteristics in various scenarios with assumptions about population, industrial and economic activity, energy use, and so on, for the rest of this century. Another task for the models is to distinguish how much warming is caused by human activity and how much is natural. The IPCC has concluded that it is “extremely likely” that human activity has had a substantial warming effect on climate since 1750. 20

* With an error range of ±0.19.

† Other greenhouse gases are included too by converting their amount and effect to the equivalent number of units of CO2.

Climate models have improved over the few decades that scientists have been developing and working with them. The models used in the 1980s and 1990s were quite limited. Here is a brief sampling of simplifications, assumptions, and factors modelers did not fully understand: The models did not distinguish day and night. 21  They used a fairly coarse grid (with points roughly 500 kilometers apart). They did not include the El Niñno phenomenon. They made assumptions about methane (a greenhouse gas) that scientists later determined were incorrect. They did not include aerosols (small particles in the air) that have a cooling effect. Clouds are extremely important to climate, but many processes involved with the formation, effects, and dissipation of clouds were not particularly well understood. The IPCC summarized in 2001: “As has been the case since the first IPCC Assessment Report in 1990, probably the greatest uncertainty in future projections ofclimate arises from clouds and their interactions with radiation… Clouds represent a significant source of potential error in climate simulations.” 22  The extremely simplified representations of the oceans in these models was another very significant weakness. Computing power was insufficient to do the many calculations to simulate ocean behavior. When run on past data, some of the early climate models predicted temperature increases three to five times as high as what actually occurred over the previous century. The 1990 IPCC report predicted that temperature would increase 0.3°C per decade (with an error range of 0.2°–0.5°C). The actual temperature increase over the next two decades was lower than that. 23  Thus, it should not be surprising that there was much skepticism about the climate models and their projections.

Current models are more detailed and complex. Increased computer power allows the use of finer grids (with points spaced roughly 100–300 kilometers apart) and more experiments with the models. Increased data collection and basic science research have been improving the understanding of the behavior and interactions of climate system components. The models project that doubling the concentration of greenhouse gases in the atmosphere will cause a global temperature increase within the range 2°–4.5°C. The models project warming of 0.2°C per decade for the next few decades and a sea level rise of between 8 and 23 inches by the end of the 21st century.24

How well is the science understood? How accurate are the data? Climatologists know an enormous amount about climate. The models incorporate a huge amount of good science and data. But the amount not known is also large.

Much of the variation in model results comes from the still troublesome lack of full understanding of clouds. When the earth warms, water evaporates, and the additional water vapor in the atmosphere absorbs more thermal energy, warming the atmosphere farther. On the other hand, water vapor forms clouds, which reflect incoming solar radiation with a cooling effect. Thus, clouds have positive (destabilizing) and negative (stabilizing) feedback effects. The basic science of the mechanisms is fairly well understood, but not the complexity and magnitude of the feedbacks. 25

A related area of uncertainty has been the impact of variations in output from the sun. Recent research on the interactions between solar activity, cosmic rays (radiation from space), and cloud formation suggests that solar activity might have an impact on cloud formation, and thus on warming, that the climate models do not include. The research is at an early stage, the theory is controversial, and the magnitude of the impact is unknown. Experiments are continuing. 26

There is insufficient data on many phenomena for the period before satellites collected data. For example, the IPCC lists among “key uncertainties” insufficient data to draw conclusions about trends in thickness of Antarctic sea ice.

The temperature data sets that models use for temperature over the past century have been a source of some contention. They include many kinds of limitations (for example, few monitoring stations in the oceans and remote land areas) and errors. In 2011, the Berkeley Earth Surface Temperature project completed a multiyear effort analyzing temperature data from 15 different data sets. It reviewed algorithms and statistical methods used to develop the data sets, and it developed new statistical methods to try to overcome problems in the previous methods. It developed and published a new global surface temperature record and provided an uncertainty analysis. 27  Climate science researchers had much praise for the quality of the work in this project and for Berkeley Earth’s decision to openly publish its methodology.

What are the assumptions and simplifications in the models? Ideally, equations derived from the underlying science (generally, physics and chemistry) would model all the processes that affect climate. This is not possible, because it would require too much computation time and because all the underlying science is not known. Simplified equations, called parametrizations, represent many processes; they seem to give realistic results but are not derived from scientific theory. The specific parametrizations vary among the models, reflecting the choices of the modelers.

The IPCC acknowledges that the underlying complexity of the problem still hampers the accuracy of projections for future climate change. That is, even the extremely powerful computers of today are not sufficient to achieve an ideal level of resolution (grid size) and to include simulation of more processes that affect climate. 28

The model projections based on scenarios (rather than a specific increase in greenhouse gas concentration) include numerous assumptions about technological development, political control of emissions, population, economic development, energy use, and so on, throughout a century.

Science and fiction

Why do science fiction movies about global warming show the buildings of cities underwater? The entertainment industry exaggerates and dramatizes, of course. Why does an exhibit in a science museum show water up to the middle of the Statue of Liberty (about 200 feet above sea level)? A climate scientist once said: “[T]o capture the public’s imagination,” “we have to offer up scary scenarios, make simplified dramatic statements, and make little mention of any doubts we may have. … Each of us has to decide what the right balance is between being effective and being honest.” 29  Although he said he hoped climate scientists could be both effective and honest, there is clearly an ethical problem when we trade honesty for something else. Is it a good idea? A 20-inch rise in sea level would be a very significant problem, but one we can tackle. Tens or hundreds of feet of sea level rise would be an enormous disaster. Exaggeration might lead people to take constructive action. Or exaggeration might lead to overreaction and counterproductive, expensive actions, draining resources from effective approaches. If we hope to solve real potential problems (such as flooding in low-lying areas), we must first identify them accurately.

How well do predictions of the models correspond with actual experience? The models predict seasonal variations and other actual broad-scale phenomena. The general patterns of predictions by different models are similar. For example, they all predict warming, and they all predict that more of the warming would take place near the poles and in winter. Many models now do a good job predicting air temperature near the surface of the earth (that is, close to observed temperatures) for the recent past. The models do well enough that the IPCC expresses many of its projections as very likely or likely.

For more than a decade at the beginning of the 21st century, global temperature fluctuated but did not rise overall. The models did not indicate that this would happen. Scientists are devising and testing theories to account for it. The amount of water vapor in the stratosphere is a key suspect. 30 Models designed to project long-term trends might not predict short-term variations well. Thus, we do not know yet if the first decade of the century was a short-term variation or whether it will require revisions in the models.

7.2 The “Digital Divide”  31

The term digital divide refers to the fact that some groups of people (the “haves”) enjoy access to and regularly use the various forms of modern information technology, while others (the “have-nots”) do not. The focus of the discussion about “the digital divide” has shifted over time. In the 1990s, the focus was on access to computers and the Internet for poor people, people in rural areas, and certain demographic groups within the United States (and other developed countries). As more people acquired digital technology and Internet access, focus shifted to a divide among those who have broadband and those who do not. There is also more focus now on the digital divide between developed countries and poor countries.

7.2.1 Trends in Computer Access

Once upon a time, everyone in the world had equal access to personal computers and the Internet. They did not exist, and we all had none. Later, a small, elite minority enjoyed these new, expensive tools. As the technology began to spread and its value became clearer, people became more concerned about the gap in access. Poor children and children of some ethnic minorities had less access to computers both in schools and at home. In the early 1990s, only about 10% of Net users were women. By 1997, the gender gap had vanished,32 but other gaps remained. *  Black and Hispanic households were about half as likely as the general population to own a computer. Access in rural and remote regions lagged behind the cities.

* A gender gap remains among those who work in information technology fields. Only about one-quarter are women, and the percentage of women undergraduates interested in the computer science major dropped drastically between 2000 and 2009.

Cost is one factor that affects access by the general population. Ease of use is another. At first, personal computers and the Internet were difficult to use. Software innovations, such as point-and-click graphical user interfaces, Web browsers, and search engines made computer use significantly more comfortable for ordinary people. With lower prices, more useful applications, and ease of use, ownership and access spread quickly. The data I found about the extent of computer ownership and Web access differ in specific numbers, but all showed the same trends: In 1990, 22% of households in the United States owned a computer. In 2001, 84% of homes with children in middle and high school had Internet access. That was a significantly higher percentage of households overall, suggesting that families perceived access to be important for their children and allocated their spending accordingly. 33

Individuals, businesses, community organizations, foundations, and government programs contributed to the spread of computers and Internet access. Internet cafés sprang up from Alaska to Cairo in the 1990s when Net access from home was relatively uncommon. The federal government and local governments spent billions of dollars on technology for schools. By the end of the 1990s, most public libraries provided Internet access for the public for free. By 2000, 98% of high schools had Internet access. At about the same time, African Americans, people 65 and older, and Hispanics increased their use of the Internet significantly. Groups with low access in earlier years began to catch up. The gaps among Hispanic, black, and white people almost completely disappeared among those with the same education levels. 34  By 2011, there were more than 300 million cellphone subscriptions in the United States. Computing and new communication technology reached more households much more quickly than earlier technologies such as telephone, television, electricity, and automobiles.

Virtually all technological innovation is first available to the rich (or others willing to pay the initially high price). The early purchases finance improvements in design and production techniques that bring the price down. Prices of many consumer products follow this pattern. Telephones and televisions were originally luxuries of the rich. Now, almost everyone in developed countries has them. (By 2006, there were more televisions than people in the United States.) When first introduced in the 1980s, compact disk music players cost $1000. Now we play music and video on our phones. Computer prices plunged more dramatically than prices of most other products, even while the memory, speed, and variety of input/output devices and software increased enormously. *  The vast resources of the Internet are available for about what home telephone service used to cost. The phenomenon that new technologies and inventions first are expensive luxuries, then become cheaper and spread throughout the population, has led some observers to conclude that it is more accurate to think of people as “haves” and “have-laters” rather than “haves” and “have-nots.” 35

* For example, the cost of disk storage fell from hundreds of dollars per megabyte in the 1980s to about $100 a terabyte (one million times as much space) by 2012.

Access to broadband connection is a newer version of a digital divide. The same demographic groups that were the have-laters with respect to Internet access in the 1990s had less broadband access in the first decade of the 2000s, with similar disadvantages. Without broadband access, it is more difficult to find employment opportunities, access news and information, and make use of some online health information and tools. Children are less likely to graduate from high school. People without broadband are less likely to create Internet content. According to Connected Nation, in 2011 only 46% of low-income households with children and 37% of low-income minority households with children had broadband at home, compared to 66% of households nationally. (About 20% of all U.S. households had broadband in 2003.) The largest barrier to adopting broadband, according to people surveyed, was cost, followed by digital literacy and not believing that the Internet was relevant to their lives. In 2011, the FCC approved a program to extend broadband service to rural areas of the United States. Businesses and nonprofit organizations started a program to address cost and digital literacy. Under this initiative, major cable carriers offer broadband service at a low price to low-income customers. Best Buy, Microsoft, America’s Promise Alliance, and United Way, among many others, provide digital literacy training. 36

A related digital divide exists among content consumers and content producers on the Internet. Internet users create blogs, Web pages, videos, and product reviews. Being a content creator empowers a user to communicate his or her message to a large number of people. The Internet can be a strong agent for change for those who have the skills, education, and tools to create content. Content creators tend to be people who access the Internet frequently from multiple places using multiple gadgets. They also are more educated. 37  The content-production divide shows a gap among users based on socioeconomic status. How should we view the Internet content-production divide? Before the Internet, a very small percent of people wrote books and articles and produced movies and television shows. The vast majority of people were content consumers only. Is the current divide less of a social problem than the pre-Internet divide because so many more people can now create content, or is it more of a problem than before because it isolates a smaller part of the population that cannot?

7.2.2 The Global Divide and the Next Billion Users

Approximately two billion people worldwide use the Internet, a fivefold increase over roughly a decade. 38  From one perspective, that is an extraordinary accomplishment in a very short time. From another perspective, it means that about five billion people do not use the Internet. Lack of access to the Internet in much of the world has the same causes as lack of health care, education, and so on: poverty, isolation, poor economies, and politics.

Both nonprofit organizations and huge computer companies have ongoing projects to spread computer access to more people in developing countries. Some companies use the catchphrase “the next billion users” to describe the people their programs address. For the companies, these programs create good will and—if successful in improving the standard of living and economies of the target countries—a large future customer base. Companies have trained hundreds of thousands of teachers to use technology effectively in classrooms in China, India, and other countries.

One Laptop per Child is a nonprofit organization that supplies an inexpensive laptop computer specially designed for elementary school children in developing countries. The laptop works in extreme heat or cold, extremes of humidity, and dusty or rainy environments. The power requirements are very low. The program provided an important lesson: giving out computers and walking away will not close the digital divide. The success of the program in implementing the technology into school curricula depends on the presence of supporting social and technical infrastructures such as electricity, networks, tech support, parental support, teacher attitudes towards technology, and administrative school support. Purely financial resources can be less important than these factors. 39

Some people active in movements to shrink the “digital divide” emphasize the need to provide access in ways appropriate to the local culture. For example, one website argues that access can hurt the poor “by loosening the bonds of tradition.” In many countries, access “is one-way, entertainment-oriented, commercial.” Access might accelerate the exodus of untrained, unprepared young people from rural areas into cities. 40  How significant are these concerns? What can be done to alleviate them?

Only a few years ago, most people in the world had never made a telephone call. By the end of 2010, there were five billion cellphone subscriptions. * 41  Almost every time I have read about a program to bring Internet access or cellphones to rural, third-world adults over the past several years, the most immediate uses are similar. Farmers use the Internet to learn about better farming techniques and to get up-to-date pricing information for their crops. Fishermen use their cellphones to find a nearby village where they will get a good price for their catch. As the technology spreads, food production and economic well-being improve. Some see each new digital divide as a serious social problem. What is perhaps most surprising is how quickly most of these divides shrink and how much more quickly they shrink than did previous technological divides between rich and poor, men and women, black and white, or developed and undeveloped countries.

* That does not mean that five billion people had cellphones. In wealthier areas, some people have more than one.

7.3 Neo-Luddite Views of Computers, Technology, and Quality of Life

· The microchip is … made of silicon, or sand—a natural resource that is in great abundance and has virtually no monetary value. Yet the combination of a few grains of this sand and the infinite inventiveness of the human mind has led to the creation of a machine that will both create trillions of dollars of added wealth for the inhabitants of the earth in the next century and will do so with incomprehensibly vast savings in physical labor and natural resources.

· —Stephen Moore 42

· Quite apart from the environmental and medical evils associated with them being produced and used, there are two moral judgments against computers. One is that computerization enables the large forces of our civilization to operate more swiftly and efficiently in their pernicious goals of making money and producing things . . . And secondly, in the course of using these, these forces are destroying nature with more speed and efficiency than ever before.

· —Kirkpatrick Sale 43

7.3.1 Criticisms of Computing Technologies

The quotations above, both from 1995, illustrate the extreme divergence of views about the anticipated value of computer technology. Evaluations cover the spectrum from “miracle” to “catastrophe.” Although most of this book discusses problems that arise with the use of computers, the Internet, and other digital communications technologies, the implicit (and sometimes explicit) view has been that these technologies are a positive development bringing us many benefits. The potential for loss of freedom and privacy via government surveillance and the building of consumer dossiers is a serious danger. Computer crime is expensive, and changes in employment are disruptive. Our discussion of systems failures in the next chapter warns us that some potential applications can have horrifying risks. We might urgently try to prevent implementation of some applications and urgently advocate for increased protection from risks, yet not consider the threats and risks as reasons for condemning the technology as a whole. For the most part, we have looked at new risks and negative side effects as problems that occur in the natural process of change, either problems we need to solve or the price we pay for the benefits, part of a trade-off. Many people with quite different political views share this attitude, although they disagree about the significance of specific computer-related problems and about exactly how to solve them.

On the other hand, there are people who utterly reject the view that computing technology is a positive development with many important benefits. They see the benefits as few and overwhelmingly outweighed by the damage done. Neil Postman says that voting, shopping, banking, and getting information at home online is a “catastrophe.” There are fewer opportunities for people to be “co-present,” resulting in isolation from neighbors. Richard Sclove and Jeffrey Scheuer argue that electronic communication will erode family and community life to the point that people will mourn the loss of depth and meaning in their lives. 44  A comment made by one reviewer of this book illustrates the difference in perspective. He objected to the “gift of fire” analogy I use to suggest that computers can be very useful and also very dangerous. The reviewer thought “Pandora’s box” was more appropriate. Pandora’s box held “all the ills of mankind.” Kirkpatrick Sale, author of Rebels Against the Future, used to demonstrate his opinion of computers by smashing one with a sledgehammer at public appearances.

In England in 1811–1812, people burned factories and mills in efforts to stop the technologies and social changes that were eliminating their jobs. Many were weavers who had worked at home on small machines. They were called Luddites. *  For 200 years, the memory of the violent Luddite uprising has endured as the most dramatic symbol of opposition to the Industrial Revolution. The term “Luddite” has long been a derisive description for people who oppose technological progress. More recently, critics of technology have adopted it as an honorable term. Kirkpatrick Sale and many others who share his viewpoint call themselves neo-Luddites, or simply Luddites.

What do the neo-Luddites find so reprehensible about computers? Some of their criticisms are problems that also trouble people whose view of computing technology is generally positive, problems we discussed in earlier chapters. One of the differentiating characteristics of the neo-Luddites is that they focus on these problems, seeing no solutions or trade-offs, and conclude that computers are a terribly bad development for humankind. Among their specific criticisms are the following:

* The name Luddite comes from General Ned Ludd, the fictitious, symbolic leader of the movement.

· • Computers cause massive unemployment and de-skilling of jobs. “Sweatshop labor is involved in their manufacture.” 45

· • Computers “manufacture needs”; that is, we use them just because they are there, not because they satisfy real needs.

· • Computers cause social inequity.

· • Computers cause social disintegration; they are dehumanizing. They weaken communities and lead to isolation of people from each other.

· • Computers separate humans from nature and destroy the environment.

· • Computers benefit big business and big government most.

· • Use of computers in schools thwarts development of social skills, human values, and intellectual skills in children. They create an “ominous uniformity of knowledge” consistent with corporate values. 46

· • Computers do little or nothing to solve real human problems. For example, Neil Postman, in response to claims of the benefits of access to information, argues that “if families break up, children are mistreated, crime terrorizes a city, education is impotent, it does not happen because of inadequate information.” 47

Some of these criticisms might seem unfair. The conditions in computer factories hardly compare to conditions in the sweatshop factories of the early Industrial Revolution. In  Chapter 6 , we saw that computers eliminate some jobs, and that the pace of computerization causes disruptions, but the case that computers, and technology in general, cause massive unemployment is not convincing. Blaming computers for social inequity in the world ignores thousands of years of history. Postman is right that inadequate information is not the source of most social problems. A computer in the classroom does not replace good parents in the home. But should this be a criticism of computers and information systems? Access to information and communication can assist in solving problems and is not likely to hurt. The main problem for ordinary people, Postman says, is how to find meaning in life. We need answers to questions like “Why are we here?” and “How are we supposed to behave?” 48  Is it a valid criticism of computing technology that it does not solve fundamental social and philosophical problems that have engaged us for thousands of years?

To the neo-Luddites, the view that computers are fundamentally malevolent is part of a wider view that almost all of technology is malevolent. To the modern-day Luddites, computer technology is just the latest, but in many ways the worst, stage in the decline of what was good in human society. Computers are worse than earlier technologies because of their enormous speed and flexibility. Computers increase the negative trends that technology causes. Thus, if one points out that a particular problem blamed on computers already existed because of an earlier technology, Luddites consider the distinction to be a minor one.

The depth of the antipathy to technology in the Luddite view is perhaps made clearer by attitudes toward common devices most of us use daily. For example, Sale has said, “I find talking on the phone a physical pain, as well as a mental anguish.” Sven Birkerts, another critic of computers, says that if he lived in 1900, he would probably have opposed the telephone. *  Speaking of the invention of the printing press, Sale laments that “literacy . . . destroys orality.” He regards not only computers but civilization as a catastrophe. Some of us see modern medicine as a life-saving and life-enhancing boon to humanity; some Luddites point out that it gave us the population explosion and extended senility. 49

* Critics of telephones complained that they replaced true human interaction with disembodied, remote voices. They actually expanded and deepened social relationships for isolated people—for example, women in general (farm wives, in particular) and the elderly. 50

Having read and listened to the arguments of technology enthusiasts and technology critics, I find it striking that different people look at the same history, the same society, the same products and services, the same jobs—and come to diametrically opposed conclusions about what they see. There is a fundamental difference between the world views of supporters and opponents of technology. It is more than the difference between seeing a glass as half full or half empty. The difference seems to be one of contrasting views about what should be in the glass. Supporters of technology see an upward trend in quality of life, beginning with people living at the mercy of nature with an empty glass that technology has been gradually filling. Neo-Luddites view the glass as originally full when people lived in small communities with little impact on nature; they see technology as draining the glass.

The neo-Luddite view is associated with a particular view of the appropriate way of life for human beings. For example, Sale’s first point, in the quotation at the beginning of this section, makes the moral judgment that making money and producing things is pernicious. His introductory remark and his second point barely hint at the unusually high valuation he places on not disturbing nature (unusually high even in the contemporary context, where there is much awareness of the importance of protecting the environment). We explore these views further.

7.3.2 Views of Economics, Nature, and Human Needs

Luddites generally have a negative view of capitalism, business, markets, consumer products, factories, and modern forms of work. They see the profit-seeking goals of businesses as in fundamental conflict with the well-being of workers and the natural environment. They see work in factories, large offices, and business in general as dehumanizing, dreary, and bad for the health of the workers. Hence, for example, the Luddite criticisms of the clock. Neil Postman describes the invention of the clock as “the technology of greatest use to men who wished to devote themselves to the accumulation of money.” 51

Choice of words, making subtle differences in a statement, sometimes illustrate the difference in perspective between Luddites and non-Luddites. What is the purpose of technology? To the Luddites, it is to eliminate jobs to reduce the costs of production. To proponents of technology, it is to reduce the effort needed to produce goods and services. The two statements say nearly the same thing, but the first suggests massive unemployment, profits for capitalists, and a poorer life for most workers. The second suggests improvements in wealth and the standard of living.

The Luddite view combines a negative attitude toward business with a high estimation of the power of corporations to manipulate and control workers and consumers. For example, Richard Sclove describes telecommuting as being “imposed by business.” (Interestingly, one of the common criticisms of the Industrial Revolution was that working in factories instead of at home weakened families and local community.)

Luddites make particularly strong criticisms of automobiles, of cities, and of most technologies involved in communications and transportation. Thus, it is worth noting that most of us get both personal and social benefits from them. Cities are centers of culture, wealth production, education, and job opportunities. 52  Modern transportation and communication reduce the price of products and increase their variety and availability. For example, we can eat fresh fruits and vegetables all year. We can look up menus and movie schedules on our smartphone to find what we want. We can shop worldwide on the Web. We can commute a long distance to take a better job without having to sell our house and move. If we move to a new city for college or a job, modern conveniences such as airplanes, telephones, and the Internet make the separations less unpleasant. We can visit more often in person. We can share greetings and activities with friends and family members via social media. Luddites and other critics of technology do not value these advantages highly. In some cases, in their point of view, the advantages are merely ameliorating other problems technology causes. For example, Postman quotes Sigmund Freud’s comment, “If there had been no railway to conquer distances, my child would never have left his native town and I should need no telephone to hear his voice.” 53

Does the technology create the need for itself?

A common criticism of capitalism is that it survives by convincing us to buy products we do not need. Luddites argue, similarly, that technology causes production of things we do not need. This contrasts with the market-oriented view that sees consumer choices as determining which products, services, and businesses succeed or fail (in the absence of government favoritism, subsidies, and restrictions). We examine the issue of created needs.

Sale argued that small, portable computers do not “meet any known or expressed need,” but companies produced them simply because miniaturization of computing components made it possible. People have bought many millions of laptops, tablet computers, and cellphones. The number of uses is phenomenal. So, does a mobile computer meet a need? It depends on what we mean by “need.” Do we need to do homework in the backyard or listen to music on an iPod? Does an architect or contractor need a laptop at a construction site? Those who emphasize the value of individual action and choices argue that needs are relative to goals, and goals are held by individuals. Thus, should we ask whether “we,” as a society, need portable computers? Or should this be an individual decision with different responses? Many people demonstrate, by their purchases, that they want portable computers. Anyone who does not feel a desire or need for one does not have to buy one. The Luddites, who believe that advertising, work pressure, or other external forces manipulate buyers, reject this individual-oriented approach.

Environmental and anti-technology groups use computers and the Web. The editor of Wild Earth, who considers himself a neo-Luddite, said he “inclines toward the view that technology is inherently evil,” but he “disseminates this view via E-mail, computer, and laser printer.” 56  An interviewer reported that in 2007, after a long career attacking computers, Kirkpatrick Sale was using a laptop. The question is: Are Sale and the editor of Wild Earth using computer equipment because of an artificial need or because it is useful and helpful to them? Sale sees the use of computers as an uncomfortable compromise. The use of computers, he says, insidiously embeds into the user the values and thought processes of the society that makes the technology. 57

Wal-Mart and e-commerce versus downtown and community  54

Does electronic commerce force changes on communities that no one wants? Richard Sclove and Jeffrey Scheuer think so. 55  They use the analogy of a Wal-Mart store draining business from downtown shops, resulting in the decline of the downtown community, a “result that no consumers wanted or intended.” They generalize from the Wal-Mart scenario to cyberspace. As we conduct more economic transactions electronically, we lose more local stores, local professional and social services, and convivial public spaces like the downtowns of small towns. Consumers are “compelled” to use electronic services, “like it or not.” Other strong critics of technology share the underlying point of view of Sclove and Scheuer, so it is worth examining their argument.

The Wal-Mart analogy is a good one. The scenario is useful for illustrating and clarifying some issues about the impact of e-commerce on communities. Suppose, say Sclove and Scheuer, that a new Wal-Mart store has opened just outside of town and about half the town residents begin to do about a third of their shopping there, while the others continue to do all their shopping downtown. Everyone shops downtown, and everyone wants the downtown stores to remain. But downtown stores have lost about 16.5% of their sales, and many will not survive. Sclove and Scheuer describe this as an “involuntary transformation” that no consumer wanted or intended. It occurs, they say, because of a “perverse market dynamic.” The changes, however, are not involuntary or perverse. The core of the problem with Sclove’s and Scheuer’s interpretation is their failure to make two important distinctions: the distinction between wanting something and the willingness to pay for it, and the distinction between something being coerced or involuntary, on the one hand, and being unwanted, unintended, or unexpected on the other.

Consider a simpler situation for a moment. Suppose we poll the adult residents of a small town with a population of, say, 3000 and ask if they would like to have a fine French restaurant in town. Almost everyone says yes. Will a French restaurant open in the town? Probably not. Almost everyone wants it, yet there is not enough potential business for it to survive. There is a market dynamic at work, but it is not perverse. The fact that consumers want a particular service, store, or product is irrelevant if not enough people are willing to pay the prices that make the business viable. In Sclove’s and Scheuer’s Wal-Mart scenario, the downtown stores could stay in business if the people were willing to pay higher prices to make up for the 16.5% of revenue lost to Wal-Mart. But we know that if the stores raise prices, they will almost certainly lose even more customers. The town residents are not willing to pay what it costs to keep the downtown stores in business. You might object: The townspeople did not have to pay the higher prices before. Why now? Because now the people who shop at Wal-Mart—or online—have another choice. Whatever price advantage or convenience lured them, they were not getting that benefit before. Again, a market dynamic is at work, but not a perverse one: competition.

The second issue about the Wal-Mart/e-commerce scenario is whether the change is an “involuntary” transformation. Sclove and Scheuer say that, as local businesses decline, people will be compelled to use electronic services, like it or not. Is this accurate? No more so than Wal-Mart shoppers or cyberspace enthusiasts were compelled to shop downtown (or from other offline stores), like it or not, before they had the new option. The new status quo is no more involuntary than the previous one. Although no one wants to see the downtown decline, the actions that could lead to that result are all voluntary. When a new store opens (online or offline), no one is forced to shop there. The impact on the downtown stores might not have been obvious to all the townspeople at the beginning (although now it is common enough that they might anticipate it), but an unexpected or unintended result is not the same as a coerced result. In a free society, individuals make millions of decisions based on their knowledge and preferences. This decentralized, individualized decision making produces a constantly changing pattern of stores, services, and investments (not to mention social and cultural patterns). No one can predict exactly what the result will be, and no one intends a particular picture of the economy or society, but (apart from government subsidies, prohibitions, and regulations) the actions of the consumers and merchants are voluntary. No one person can expect to have exactly the mix of shopping options (or other community characteristics) that he or she wants. If the result flows from the myriad decisions that consumers and producers make, it is not coerced. It is the process, not the result, that tells us whether an outside force is coercing people.

Do we need cellphones?

Hundreds of thousands of people have heart attacks in the United States each year. Treatment received in the first few minutes can be critical to their survival. A fire department in California helped develop a smartphone app that alerts people trained in CPR if they are near the location where a person is having a heart attack, perhaps in the same office building, shopping center, or neighborhood. The app provides the location of the victim and the locations of any nearby emergency defibrillator devices, so a trained person can get to the scene quickly and has the tools to save a life.

The argument that capitalists or technologies manipulate people to buy things they do not really want, like the argument that use of computers has an insidiously corrupting effect on computer users, displays a low view of the judgment and autonomy of ordinary people. It is one thing to differ with another person’s values and choices. It is another to conclude that, because of the difference, the other person is weak and incapable of making his or her own decisions. The Luddite view of the appropriate way of life puts little value on modern comforts and conveniences or on the availability of a large variety of goods and services. Perhaps most people value these things more highly than the Luddites do. To get a clearer understanding of the Luddite view of a proper life style, we consider some of their comments on the relationship of humans and nature.

Nature and human life styles

Luddites argue that technology has made no improvement in life, or at best improvements of little importance. Sale’s list of benefits includes speed, ease, and mass access—all of which he disdains. Sale says that although individuals might feel their lives are better because of computers, the perceived benefits are “industrial virtues that may not be virtues in another morality.” He defines moral judgment as “the capacity to decide that a thing is right when it enhances the integrity, stability, and beauty of nature and is wrong when it does otherwise.” 58  Jerry Mander, founder of the Center for Deep Ecology and author of books critical of technology and globalization, points out that thousands of generations of humans got along without computers, suggesting that we could do just fine without them too. While some people evaluate trade-offs between negative side effects of pesticides and the benefits of reducing diseases or protecting food crops, Mander’s objections to technology lead him to the conclusion that there can be no “good” pesticide. While many people work on technological, legal, and educational approaches to reducing the gasoline usage of automobiles, Mander says there can be no “good” automobile. 59

What are the underlying premises behind these comments by Sale and Mander? We consider Sale’s comment on moral judgment first. Many debates about the environment set up a humans-versus-nature dichotomy. 60  This is not the true conflict. Nature, biodiversity, forests, a hospitable climate, clean air and water, open space away from cities— these are all important and valuable to humans. So is shelter from the rain, cold, and heat. So are life-saving medicines and medical techniques. Conflicts about the environment are not conflicts between humans and nature. They are conflicts between people with different views about how to meet human needs. In contrast to Sale’s statement, moral judgment, to many people, and for many centuries, has meant the capacity to choose that which enhances human life, reduces misery, and increases freedom and happiness. Sale’s comment chooses nature, not humanity, as the primary standard of moral value.

Whether an automobile (or computing device) is “good,” by a human-centered standard, depends on whether it meets our needs, how well it does so, at what cost (to the environment and society, as well as to our bank account), and how well it compares to alternatives. Critics of modern technologies point out their weaknesses but often ignore the weaknesses of alternatives—for example, the millions of acres once needed to grow feed for horses and the hundreds of tons of horse manure dropped on the streets of cities each day, a century ago. 61  Mander’s comment about automobiles again raises the issues of our standard of value and our need for a product or service. Candles, gas lamps, and kerosene lamps filled homes with fumes and soot. Do we need electricity? Do we need hot water on tap, movies, and symphony orchestras? Or do we need nothing more than food and shelter? Do we need an average life expectancy of more than 25 years? Do we want to merely exist—do we need even that?—or do we want long, happy, comfortable lives filled with time for love, interesting activities, and an opportunity to use our marvelously inventive brains?

· The Web is alive, and filled with life, nearly as complex and, well, natural as a primordial swamp.

· —John Perry Barlow 62

Accomplishments of technology

It is easy to miss the extreme changes in quality of life that have taken place over the past few centuries. We mention here a scattering of examples.

Technology and the Industrial Revolution have had a dramatic impact on life expectancy. A study in 1662 estimated that only 25% of people in London lived to age 26. Records from 18th-century French villages showed that the median age of death was lower than the median age of marriage. Until recent generations, parents had to endure the deaths of most of their children. Starvation was common. In the United States, life expectancy at birth increased from 47.3 years in 1900 to 77.9 in 2007. Worldwide average life expectancy increased from approximately 30 in 1900 to approximately 64 in 2006. Science and technology (along with other factors such as education) reduced or almost eliminated typhoid, smallpox, dysentery, plagues, and malaria in most of the world. Deaths at work, during travel, and by accidents declined dramatically. 66

Environmental impacts of computing technology

I had thought of including a section in this book on environmental impacts of computers, mobile devices, and the Internet. As I looked for data, I concluded that attempts to quantify environmental benefits and costs would be subject to the same weaknesses and criticisms of models that we discussed in  Section 7.1.2 . It is extremely difficult to measure impacts and to determine how to compare to impacts of technologies and activities that computing technology replaces. However, we can make some observations.

Production of computers is energy intensive and uses hazardous materials. Because of these materials, disposal is an issue, as it is for flourescent light bulbs. Running and cooling the millions of servers on the Internet in the United States accounts for about 2% of U.S. electric power usage,63more than the U.S. auto industry and less than the chemical industry. There are estimates that production of computers uses roughly twice as much energy as operating them.

On the other hand, digitally controlled machinery uses less power than older electromechanical controls. Digital sensors and controls for regulating lighting, heating, air conditioning, and farm irrigation (among many other examples) save resources by determining just what is needed and thus reduce waste. Microprocessors control hybrid cars, reducing gasoline use. Telecommuting, e-commerce, and online libraries and information sites significantly reduce the need for driving and flying and thus, the need for fuel. One fiber-optic cable, with about 150 pounds of silica, carries more messages than a ton of copper wire. 64

Digital storage of documents, data, photos, and so on, reduce the need for paper (and the amount of trash produced.) Specific examples suggest the reductions: A large insurance company reduced its use of paper by 100 million pages in a nine-month period by storing its manuals digitally instead of printing them. A computerized system for recording insurance claims replaced more than 30 million index cards. We use email and texting instead of sending letters and cards on paper. Electronic payments eliminate paper bills and checks. We read books, newspapers, magazines, and so on, on tablets, e-readers, and smartphones, reducing paper use. The decline in business for the U.S. Postal Service and printed newspapers, while population and economic activity grow, are indications of these reductions. But do we actually use less paper than we did before? I could not find clear data for total paper use. However, between 2001 and 2011, annual consumption of newsprint for daily newspapers in the United States dropped by an estimated 61%, and the number of pieces of first class mail dropped by about 24%. 65

We take, post, and share far more photos (billions per month) than we did when we made prints and slides. This is one example of a phenomenon that occurs in many fields: as a product or service becomes more efficient and cheaper, we use more of it. It seems that people have certain levels of cost that they are willing to accept. We do more and use more when the cost goes down. Perhaps we have increased our use of resources as we have increased our use of computing technology. Certainly, we shift resources from areas where there are savings to other uses, including improved medical technology, more music and video, easier access to education, and other products and services that bring us benefits.

In the early 2000s, Americans spent less than 10% of family income on food, compared to approximately 47% in 1901. Agronomist Norman Borlaug, who won a Nobel Peace Prize for his work in improving agricultural productivity, reported that when new forms of wheat and crop management were introduced in India, yields rose from 12.3 million tons in 1965 to 73.5 million tons in 1999. In about the same timeframe, U.S. production of its 17 most important crops increased from 252 million tons to 596 million tons, but used 25 million fewer acres. Nicholas Eberstadt, an expert on population, reported that food supplies and gross domestic product have been growing faster than population for decades in most areas of the world, in both developing and developed countries. 67

The benefits of telecommunications and information technology are enormous in developing countries. A report of a United Nations Conference on Trade and Development, for example, observes that developing economies can make productivity gains worth billions of dollars by encouraging the growth of electronic commerce. The report said that “it is because the internet revolution is relevant not just to the high-tech, information-intensive sectors but also to the whole organisation of economic life that . . . developing countries stand a better chance of sharing in its benefits earlier than in previous technological revolutions.” 68

Technology is certainly not the only factor in improving quality of life. Progress against disease, discomfort, and early death depends on the stability, freedom, and flexibility of political and economic systems as well. Measuring quality of life is subjective, and some find other measures more important than the few we cited above. But, for many people, these data suggest that technology has contributed much to human well-being.

7.4 Making Decisions About Technology

· No one voted for this technology or any of the various machines and processes that make it up.

· —Kirkpatrick Sale 69

7.4.1 Questions

We saw, in  Section 7.3 , that the determination of what are true needs depends on our choice of values. Throughout this book, we saw controversies about specific products, services, and applications of computer technology (for example, personalized advertising, anonymous Web surfing, and face recognition systems). How should we make decisions about the basic question of whether to use a whole technology, or major segments of it, at all? Who would make such decisions?

Most people in science, engineering, and business accept, almost without question, the view that people can choose to use a technology for good or ill. Some critics of technology disagree. They argue that technologies are not “neutral.” Neil Postman says, “Once a technology is admitted [to our culture], it plays out its hand; it does what it is designed to do.” 70  This view sees the technologies themselves as being in control.

In the view of some critics of computing technology, big corporations and governments make decisions about uses of the technology without sufficient input or control by ordinary people. Kirkpatrick Sale’s lament at the beginning of this section expresses this view: there was never a vote on whether we should have computers and the Internet. Some people argue that we should not use a new technology at all until we have studied it, figured out its consequences, and made a determination that the consequences are acceptable. The idea is that if the technology does not meet certain criteria, we would not permit its development and use.

This view leads to a few basic questions. Can a society choose to have certain specific desirable modern inventions while prohibiting others or prohibiting whole technologies? How well can we predict the consequences of a new technology or application? Who would make the decisions? We consider the first question here and the others in the next few sections.

How finely can we make decisions about acceptable and unacceptable technologies? In response to a criticism that the tribal life he extolled would have no pianos, no violins, no telescope, no Mozart, Sale replied, “[I]f your clan thought that the violin was a useful and nonharmful tool, you could choose to invent that.” 71  Perhaps critics of computing technology who recognize its value to disabled people would permit development of applications for them. The question is whether it is possible for a clan or society to choose to invent a violin or a book reader for blind people without the technological and economic base on which development of these products depends. That base includes the freedom to innovate, a large enough economy to get materials from distant sources, and a large number of potential applications that make the research, development, and production of the basic ingredients of these products economically feasible. It is unlikely that anyone would even think of developing a book reader for the blind if some of the components did not already exist in prior products (for example, perhaps, a photocopy machine).

Telemedicine: A bad application of technology?

In  Chapter 1 , we described long-distance medicine, or telemedicine, as a benefit of computer technology. Computer and communications networks make possible remote examination of patients and medical test results, and they make possible remotely controlled medical procedures. You should be able to think of potential privacy and safety problems with such systems. You might think of other objections as well. Should we ban telemedicine?

Several states passed laws prohibiting the practice of telemedicine by doctors who are not licensed in that state. The main argument they give for the laws is safety, or concern about out-of-state “quacks.” The laws will “keep out the charlatans and snake-oil salesmen,” according to one supporter. 72  Also, telemedicine could increase the influence of large, well-financed medical centers—to the detriment of local physicians in private practice. Large hospitals might become the “Wal-Marts of medicine,” says one writer. Telemedicine might make medical care even more impersonal than it is already.

Is concern for patients the real reason for the laws? The arguments about charlatans and quacks seem weak, considering that the laws target doctors who are licensed, but in another state. Many doctors who support the bans see telemedicine as a significant competitive threat. As the director of one state medical board put it, “They’re worried about protecting their turf.” 73  The laws restrict competition and protect established special interests—a risk of any mechanism designed to prohibit a new technology or product.

7.4.2 The Difficulty of Prediction

A brief look at the development of communications and computer technology suggests the difficulty of evaluating the consequences and future applications of a new technology. Early computers were developed to calculate ballistics trajectories for the military. The PC was originally a tool for doing computation and writing documents. No one but a few visionaries imagined most of their current uses. Each new technology finds new and unexpected uses. When physicists began developing the World Wide Web, who would have predicted online auctions, social networking, or sharing home video? Would anyone have predicted even a small fraction of the ways we use smartphones? Postman’s statement that a technology does “what it is designed to do” ignores human responsibility and choice, innovation, discoveries of new uses, unexpected consequences, and social action to encourage or discourage specific applications. Computer scientist Peter Denning takes a different view: “Although a technology does not drive human beings to adopt new practices, it shapes the space of possibilities in which they can act: people are drawn to technologies that expand the space of their actions and relationships.” 74  Denning says people adopt technologies that give them more choices. Note that he does not say more choices of consumer products, but more actions and relationships. Don Norman also suggests that society influences the role of a technology when he says, “The failure to predict the computer revolution was the failure to understand how society would modify the original notion of a computational device into a useful tool for everyday activities.” 75

Figure 7.4 Predictions. 76

How well can a government committee, a think tank, or a computer industry executive predict the consequences of a new technology? The history of technology is full of wildly wrong predictions—some overly optimistic, some overly pessimistic. Consider the quotations in  Figure 7.4 . Some scientists were skeptical of air travel, space travel, and even railroads. (They believed that passengers would not be able to breathe on high-speed trains.) The quotations in  Figure 7.4  reflect a lack of imagination about the myriad uses people would find for each new technology, about what the public would like, and about what they would pay for. They demonstrate humorously that many experts can be utterly wrong. We examine the prediction problem more seriously and in more depth by considering arguments made by computer scientist Joseph Weizenbaum in 1975 against the development of a particular computer technology: speech recognition systems. 77  We now have more than 35 years of hindsight. However, many inexpensive applications of speech recognition had already appeared by the early 1990s. Here are Weizenbaum’s objections, accompanied by comments from our perspective today.

· • “The problem is so enormous that only the largest possible computers will ever be able to manage it.” Speech recognition software runs on smartphones.

· • “. . . a speech-recognition machine is bound to be enormously expensive, . . . only governments and possibly a very few very large corporations will therefore be able to afford it.” Millions of people own smartphones and other devices that include speech recognition.

· • “What can it possibly be used for?” Much more than I will mention here. (Speech recognition technology is a multibillion-dollar industry.) We can search the Web from a cellphone by speaking what we want instead of typing. We can call a business, speak the name of the person we want to reach, and automatically connect to that person’s extension. Other customer-service applications include checking airline flight schedules, getting stock quotes and weather information, conducting banking transactions, and buying movie tickets on the phone by speaking naturally instead of pushing buttons. Recall some of the applications described in  Sections 1.2.3  and  1.2.4 : training systems (e.g., for air traffic controllers and for foreign languages) and tools that help disabled people use computers and control appliances in their homes. People who suffer from repetitive strain injury use speech recognition input instead of a keyboard. IBM advertised speech-input software for poets, so they can concentrate on poetry instead of typing. People with dyslexia use speech recognition software so they can write by dictation. Speech translation systems recognize speech and translate it into other languages. Full translation is still a difficult problem, but tourists, business people, social service workers, hotel reservations clerks, and many others use specialized versions. Voice-activated, hands-free operation of cellphones, car stereos, and other appliances in automobiles eliminates some of the safety hazard of using these devices while driving.

· • The military planned to control weapons by voice command, “a long step toward a fully automated battlefield.” Some argue that we should have the best possible weapons to defend ourselves. Others argue that, if wars are easier to fight, governments fight more of them. If countries fight wars with remotely controlled automated weapons and no humans on the battlefield, is that an improvement over wars in which people are slaughtered? What if only one side has the high-tech weapons? Would that cause more wars of aggression? Is there any technology that the military cannot or does not use? Should we decline to develop strong fabrics because the military can use them for uniforms? Clearly, military use of high-tech tools raises serious ethical and policy questions. Are these questions sufficient reason to abandon or condemn a technology?

· • Governments can use speech recognition to increase the efficiency and effectiveness of wiretapping. Abuses of wiretapping concerned Weizenbaum (e.g., tapping done by oppressive governments). He does not explicitly mention wiretapping of criminal suspects. One can argue that governments can use the same tool beneficially in legal wiretapping of suspected criminals and terrorists, but it is true that speech recognition, like many other technological tools, can be a danger in the hands of governments. Protection from such abuses depends in part on the recognition of the importance of strictly controlling government power and in part on the appropriate laws and enforcement mechanisms to do so.

Discussion of Weizenbaum’s objections is important for several reasons. (1) Although Weizenbaum was an expert in artificial intelligence, of which speech recognition is a subfield, he was mistaken in his expectations about the costs and benefits. (2) His objections about military and government use highlight the dilemma: Should we decline to develop technologies that people can misuse, or should we develop the tools because of their beneficial uses, and use other means, including our votes and our voices, to influence government and military policy? (3) Weizenbaum’s argument against development of a technology because of its expected cost is similar to arguments expressed by others about current and future computer applications and other technologies. For example, a common objection to some new medical technologies is that they are so expensive that only the rich will be able to afford them. This shortsighted view can result in the denial of benefits to the whole population. For many new inventions, prices are high at first but quickly come down. A computer chip developed to float on the retina of the eye and send visual signals to the brain has the potential to restore sight to some blind people. The initial cost was $500,000. Should we ban it because it would be available only to the very rich? The developer of the chip expected the cost to come down to $50 with mass production.

Weizenbaum was not trying to evaluate computer technology as a whole but was focusing on one specific application area. If we are to permit the government, or experts, or the people via a majority vote to prohibit development of certain technologies, it is essential at least that we be able to estimate the consequences—both risks and benefits—of the technology fairly accurately. We cannot do this. The experts cannot do it.

But what if a technology might threaten the survival of the human race? We consider such an example in the next section.

* I include some references at the end of the chapter.

7.4.3 Intelligent Machines and Superintelligent Humans—Or theEnd of the Human Race?

Prominent technologists such as Hans Moravec, Ray Kurzweil, and Vernor Vinge describe a not-very-distant future in which intelligence-enhancing devices, artificial intelligence, and intelligent robots change our society and our selves in profound ways. *  The more optimistic scenarios include human use of intelligent machines and services of many kinds. People might acquire advanced mental powers through brain implants and computer–brain interfaces. When someone has a stroke, doctors might remove the damaged part of a brain and replace it with a chip that performs the lost functions, perhaps with a large amount of extra memory or a chip to access the Web directly. Why wait for a stroke? Once the technology is available, healthy people will likely buy and install such implants. MIT robotics researcher Rodney Brooks, for example, suggests that by 2020 we might have wireless Internet interfaces that doctors can implant in our heads. He says people might be just as comfortable with them as they are now getting laser eye surgery at a mall. 78  Will such implants make someone less human than a heart transplant or pacemaker does? What social problems will intelligence enhancement cause in the next few decades? What philosophical and ethical problems arise when we combine human and machine intelligence in such intimate ways?

Going farther into the future, will we “download” our brains to long-lasting robot bodies? If we do, will we still be human?

The technological singularity

The term technological singularity refers to the point at which artificial intelligence or some combined human–machine intelligence advances so far that we cannot comprehend what lies on the other side. It is plausible, says computer scientist Vinge, that “we can, in the fairly near future, create or become creatures who surpass humans in every intellectual and creative dimension. Events beyond such a singular event are as unimaginable to us as opera is to a flatworm.” 79

Some technologists welcome the idea of the human race transforming into an unrecognizable race of superintelligent, genetically engineered creatures within this century. Others find it horrifying—and others unlikely. Some see potential threats to the survival of the human race. They see the possibility of the machines themselves achieving human-level intelligence, then rapidly improving themselves to a superhuman level. Once robots can improve their design and build better robots, will they “outcompete” humans? Will they replace the human race, just as various species of animals displace others? And will it happen soon, say within the next 20 years or so?

Two estimates support these scenarios. One is an estimate of the computing power of the human brain. The other is based on Moore’s Law, the observation that the computing power of new microprocessors doubles roughly every 18 to 24 months. If the progress of hardware power continues at this rate, then by roughly 2030 computer hardware will be about as powerful as a human brain, sufficiently powerful to support the computation requirements of intelligent robots.

Both those who think an extreme advance in machine intelligence or human–machine intelligence is likely in the near future and those who criticize these ideas provide several reasons why it might not happen. Here are some of them. First, hardware progress might slow down. Second, we might not be able to develop the necessary software in the next few decades, or at all. Developments in AI, particularly in the area of general intelligence, have been much slower than researchers expected when the field began. Third, the estimates of the “hardware” computing power of the human brain (the sophistication of the computing power of neurons) might be drastically too low. Finally, some philosophers argue that robots programmed with AI software cannot duplicate the full capability of the human mind.

Responding to the threats of intelligent machines

Whether the singularity occurs within a few decades, or later, or not at all, many in the relevant fields foresee general-purpose intelligent machines within your lifetime. By its definition, we cannot prepare for the aftermath of the singularity, but we can prepare for more gradual developments. Many of the issues we explored in previous chapters are relevant to enhanced intelligence. Will software bugs or other malfunctions kill thousands of people? Will hackers hack brains? Will a large division open up between the superintelligent and the merely humanly intelligent? We saw that protections for safety and privacy in computer systems are often weak because they were not designed in from the start. It is valuable to think about potential problems of superintelligent systems and intelligence enhancement for humans well before they confront us so that we can design the best protections.

Bill Joy is cofounder of Sun Microsystems and a key developer of Berkeley Unix and the Java programming language. In his article “Why the Future Doesn’t Need Us,” 80  Joy describes his worries about robotics, genetic engineering, and nanotechnology. He observes that these technologies will be more dangerous than technologies of the 20th century (such as nuclear weapons) because they will be self-replicating and will not require rare and expensive raw materials and huge factories or laboratories. Joy foresees profound threats, including possibly the extinction of the human race.

What protections do people who fear for the future of the human race recommend? Joy describes and criticizes some before suggesting his own. Space enthusiasts suggest creating colonies in space. Joy observes that it will not happen soon enough. If it does, it might save the human race but not the vast majority of humans on earth. If colonists take the current technologies with them, the threat goes too. A second solution is to develop protections that can stop the dangerous technologies from getting out of control. Futurist Virginia Postrel suggests “a portfolio of resilient responses.” 81  Joy argues that we could not develop “shields” in time, and if we could, they would necessarily be at least as dangerous as the technologies they are supposed to protect us against. Joy recommends “relinquishment,” by which he means we must “limit development of the technologies that are too dangerous, by limiting our pursuit of certain kinds of knowledge.” He cites, as earlier examples, treaties to limit development of certain kinds of weapons and the United States’s unilateral decision to abandon development of biological weapons. However, relinquishment has the same kinds of weaknesses Joy attributes to the approaches he rejects: they are “either undesirable or unachievable or both.” Enforcing relinquishment would be extraordinarily difficult, if not impossible. As Joy recognizes, intelligent robots and the other technologies that concern him have huge numbers of potentially beneficial applications, many of which will save lives and improve quality of life. At what point should governments stop pursuit of knowledge and development? Ethical professionals will refuse to participate in development of some AI applications, but they too face the difficult problem of where to draw the line. Suppose we develop the technology to a point where we get useful applications with legal and technological safety controls. How will we prevent visionary or insane scientists, hackers, teenagers, aggressive governments, or terrorists from circumventing the controls and going beyond the prohibited level? Joy sees a relinquishment verification program on an unprecedented scale, in cyberspace and in physical facilities, with privacy, civil liberties, business automony, and free markets seriously curtailed. Thus, relinquishment means not only that we might lose development of innovative, beneficial products and services. We would lose many basic liberties as well.

Although we can find flaws with all proposals to protect against the dangers of powerful technologies, that does not mean we should ignore the risks. We need to choose appropriate elements from the various proposals and develop the best protections we can.

· Prediction is difficult, especially about the future.  82

7.4.4 A Few Observations

We have presented arguments against the view that we should evaluate and perhaps ban new technologies at the start. Does this mean that no one should make decisions about whether it is good to develop a particular application of a new technology? No. The arguments and examples suggest two things: (1) that we limit the scope of decisions about development of new technology, perhaps to particular products, and (2) that we decentralize the decision-making process and make it noncoercive, to reduce the impact of mistakes, avoid manipulation by entrenched companies who fear competition, and prevent violations of liberty. We cannot often predict the decisions and the results of decisions made by individual engineers, researchers, programmers, entrepreneurs, venture capitalists, customers, and teenagers who tinker in their garages, but they have a valuable robustness. The fundamental problem is not what decision to make about a particular technology. Rather, it is to select a decision-making process that is most likely to produce what people want, to work well despite the difficulty of predicting consequences, to respect the diversity of personal opinions about what constitutes a desirable life style, and to be relatively free of political manipulation.

When we consider the most extreme potential developments, such as superintelligent robots, what level of certainty of dire consequences should we require before restricting the freedom to develop technologies and products that might have marvelous benefits?

Chapter:- 8

8.1 Failures and Errors in Computer Systems

8.1.1 An Overview

· “Navigation System Directs Car Into River”

· “Data Entry Typo Mutes Millions of U.S. Pagers”

· “Flaws Found in Software That Tracks Nuclear Materials”

· “Software Glitch Makes Scooter Wheels Suddenly Reverse Direction”

· “IRS Computer Sends Bill for $68 Billion in Penalties”

· “Robot Kills Worker”

· “California Junks $100 Million Child Support System”

· “Man Arrested Five Times Due to Faulty FBI Computer Data”

These headlines describe real incidents. Most computer applications, from consumer software to systems that control communications networks, are so complex that it is virtually impossible to produce programs with no errors. In the next few sections, we describe a variety of mistakes, problems, and failures—and some factors responsible for them. Some errors are minor. For example, a word processor might incorrectly hyphenate a word that does not fit at the end of a line. Some incidents are funny. Some are tragic. Some cost billions of dollars. Studying these failures and risks contributes to understanding their causes and helps prevent future failures.

Are computer systems too unreliable and too unsafe to use? Or, like many news stories, do the headlines and horror stories emphasize the bad news—the dramatic but unusual events? We hear reports of car crashes, but we do not hear that drivers completed 200,000 car trips safely in our city today. Although most car trips are safe, there is a good purpose for reporting crashes: It teaches us what the risks are (e.g., driving in heavy fog) and it reminds us to be responsible and careful drivers. Just as many factors cause car crashes (faulty design, sloppy manufacturing or servicing, bad road conditions, a careless or poorly trained driver, confusing road signs, and so on), computer glitches and system failures also have myriad causes, including faulty design, sloppy implementation, careless or insufficiently trained users, and poor user interfaces. Often, there is more than one factor. Because of the complexity of computer systems, it is essential to follow good procedures and professional practices for their development and use. Sometimes, no one does anything clearly wrong, but an accident occurs anyway. Occasionally, the irresponsibility of software developers and managers is comparable to driving while very drunk.

If the inherent complexity of computer systems means they will not be perfect, how can we distinguish between errors we should accept as trade-offs for the benefits of the system and errors that are due to inexcusable carelessness, incompetence, or dishonesty? How good is good enough? When should we, or the government, or a business decide that a computer system or application is too risky to use? Why do multimillion-dollar systems fail so miserably that the firms and agencies that pay for them abandon them before completion? We cannot answer these questions completely, but this chapter provides some background and discussion that can help us in forming conclusions. It should help us understand the problems from the perspective of several of the roles we play:

· • A computer user. Whether we use our own tablet computer or a sophisticated, specialized system at work, we should understand the limitations of computer systems and the need for proper training and responsible use.

· • A computer professional. If you are planning a career as computer professional (system designer, programmer, or quality assurance manager, for example), studying computer system failures should help you become a better professional. Understanding the source and consequences of failures is also valuable if you will be responsible for buying, developing, or managing a complex system for a hospital, airport, or business. The discussions of the examples in this chapter include many implicit and explicit lessons about how you can avoid similar problems.

· • An educated member of society. There are many personal decisions and social, legal, and political decisions that depend on our understanding of the risks of computer system failures. We could be on a jury. We could be an active member of an organization lobbying for legislation. We could be deciding whether or not to have surgery performed by a robot. Also, we can apply some of the problem-solving approaches and principles in this chapter to professional areas other than computer systems.

We can categorize computer errors and failures in several ways—for example, by the cause, by the seriousness of the effects, or by the application area. In any scheme to organize the discussion, there will be overlap in some categories and mixing of diverse examples in others. I use three categories: problems for individuals, usually in their roles as consumers; system failures that affect large numbers of people and/or cost large amounts of money; and problems in safety-critical applications that may injure or kill people. We will look at one safety-critical case in depth (in Section 8.2 ): the Therac-25. This computer-controlled radiation treatment machine had a large number of flaws that resulted in the deaths of several patients. In  Sections 8.3  and  8.4 , we try to make some sense of the jumble of examples.  Section 8.3  looks at underlying causes in more depth and describes professional practices and other approaches to preventing failures and handling them properly when they occur.  Section 8.4  puts the risks in perspective in various ways.

The incidents described here are a sampling of the many that occur. Robert Charette, an expert on software risk management, emphasizes that computer system errors and failures occur in all countries, in systems developed for businesses, governments, and nonprofit organizations (large and small) “without regard to status or reputation.” 1  In most cases, by mentioning specific companies or products, I do not mean to single those out as unusual offenders. One can find many similar stories in news reports, software engineering journals, and in The Risks-Forum Digest organized by Peter Neumann. 2  Neumann collects thousands of reports describing a wide range of computer-related problems.

8.1.2 Problems for Individuals

Billing errors

The first few errors we look at are relatively simple ones whose negative consequences were undone with relative ease.

· • A woman received a $6.3 million bill for electricity. The correct amount was $63. The cause was an input error made by someone using a new computer system.

· • The IRS is a constant source of major bloopers. When it modified its programs to avoid billing victims of a Midwest flood, the computer generated erroneous bills for almost 5000 people. One Illinois couple received a bill for a few thousand dollars in taxes—and $68 billion in penalties. In one year the IRS sent 3000 people bills for slightly more than $300 million. One woman received a tax bill for $40,000,001,541.13.

· • The auto insurance rate of a 101-year-old man suddenly tripled. Rates depend on age, but the program handled ages only up to 100. It mistakenly classified the man as a teenager.

· • Hundreds of Chicago cat owners received bills from the city for failure to register dachshunds, which they did not own. The city used two databases to try to find unlicensed pets. One database used DHC as the code for domestic house cat, and the other used the same code for dachshund.

Programmers and users could have avoided some of these errors. For example, programmers can include tests to determine whether a billing amount is outside some reasonable range or changed significantly from previous bills. In other words, because programs can contain errors, good systems have provisions for checking their results. If you have some programming experience, you know how easy it would be to include such tests and generate a list of cases for a person to review. These errors are perhaps more humorous than serious. Big mistakes are obvious. They usually get fixed quickly. They are worth studying, because the same kinds of design and programming errors can have more serious consequences in different applications. In the Therac-25 case ( Section 8.2 ), we will see that including tests for inconsistent or inappropriate input could have saved lives.

How close to perfection should we expect billing systems to be? A water-utility company sent a customer an incorrect bill for $22,000. A spokesman for the company pointed out that one incorrect bill out of 275,000 monthly bills is pretty good. It is better than a 99.999% accuracy rate. Is that reasonable? At some point, the expense of improving a system is not worth the gain, especially for applications where the impact of the error is small and errors can be detected (once they occur) and corrected at much lower cost than it would take to try to prevent them.

Inaccurate and misinterpreted data in databases

Credit bureau records incorrectly listed thousands of New England residents as not having paid their local property taxes. An input error appeared to be the cause of the problem. People were denied loans before someone identified the scope of the problem and the credit bureau corrected it. Like $40 billion tax bills, a systematic error affecting thousands of people is likely to get noticed. The relevant company or agency is likely to fix it quickly. More serious, perhaps, are all the errors in individual people’s records. In one case, a county agency used the wrong middle name in a report to a credit bureau about a father who did not make his child-support payments. Another man in the same county had the exact name reported. He could not get credit to buy a car or a house. A man applied for jobs at several retail stores. They all turned him down. Eventually he learned that the stores used a database to screen applicants, and it listed him as a shoplifter. A real shoplifter had given the police the innocent man’s identification from a lost wallet.

It is difficult to get accurate and meaningful error rates for major databases with information about millions of people. Also, we need to distinguish between a spelling error in someone’s address and an incorrect report that someone bounced several checks. The results of numerous surveys and studies vary considerably, but they indicate that a high percentage of credit records have serious errors.

 Errors affecting job applicants:  Section 6.3.1

Federal law requires states to maintain databases of people convicted of sex crimes against children and to release information about them to the public. A family was harassed, threatened, and physically attacked after their state posted an online list of addresses where sex offenders live. The state did not know the offender had moved away before the family moved in. A man murdered two men in Washington state after getting their addresses from the state’s sex offender database, and another man killed two men listed in Maine’s online registry. One of them was in the database because, as a teenager, he had sex with his girlfriend who was a few weeks below the age of consent. While technically not an error in the database, this case illustrates the need for careful thought about what a database includes and how it is presented to the public, especially if it involves a highly charged subject.

A high school excluded a 14-year-old boy from football and some classes without explanation. He eventually learned that school officials thought he had been using drugs while in junior high school. The two schools used different disciplinary codes in their computerized records. The boy had been guilty of chewing gum and being late. This case is very similar to the case of the dachshund/cat confusion described earlier—except that the consequences were more significant. Both cases illustrate the problems of relying on computer systems without taking the responsibility of learning enough about them to use them properly.

When errors occur in databases used by law enforcement agencies, the consequences can include arrests at gunpoint, strip searches, and time in jail with violent criminals. For example, two adults went to jail and a child to a juvenile home for 24 hours while police determined that they really had rented the rental car they were driving. The car rental company had listed the car as stolen. An adoption agency ran a routine check on an applicant and found a conviction for grand larceny. In fact, the applicant had taken part in a college prank—stealing a restaurant sign—years before. He had apologized and paid for the damage, and the charges had been dropped. The error could have caused the agency to deny the adoption. Police arrested a Michigan man for several crimes, including murders, committed in Los Angeles. Another man had assumed his identity. It is understandable that the FBI’s National Crime Information Center (NCIC) database showed the innocent man as wanted—someone using his name was committing crimes. However, the innocent man was arrested five times; the database was not corrected. The military imprisoned a man for five months because NCIC mistakenly reported that he was AWOL. *  A college professor returning from London spent two days in jail after a routine check with NCIC at Customs showed that he was a wanted fugitive. NCIC was wrong— for the third time—about this particular man. Police stopped and frisked an innocent driver because his license plate number incorrectly appeared as the license number of a man who had killed a state trooper. The computer record did not include a description of the car. (NCIC now includes digitized photographs and fingerprints to help reduce the number of incidents in which police detain an innocent person.) 3

After the terrorist attacks in 2001, the FBI gave a “watch list” to police departments and businesses such as car rental agencies, banks, casinos, and trucking and chemical firms. Recipients emailed the list to others, and eventually thousands of police departments and thousands of companies had copies. Many incorporated the list into their databases and systems that screened customers or job applicants. Although the list included people who were not suspects but whom the FBI wanted to question, some companies labeled the list “Suspected terrorists.” Many entries did not include date-of-birth, address, or other identifying information, making mistaken identifications likely. Some companies received the list by fax and typed misspelled names from blurred copies into their databases. The FBI stopped updating the list but did not tell the recipients; thus, many entries became obsolete. 4  Even if someone corrects an error in the original database, problems may not be over for the affected person. Copies of incorrect or mislabeled data remain in other systems.

Several factors contribute to the frequency and severity of the problems people suffer because of errors in databases and misinterpretation of their contents:

*AWOL means “absent without official leave.”

· • A large population (Many people have identical or similar names, and most of our interactions are with strangers.)

· • Automated processing without human common sense or the power to recognize special cases

· • Overconfidence in the accuracy of data stored on computers

· • Errors (some due to carelessness) in data entry

· • Failure to update information and correct errors

· • Lack of accountability for errors

The first factor is unlikely to change. It is the context in which we live. The second is partly a side effect of the speed and processing ability of computer technology, but we can reduce its negative impacts with better system specifications and training of users. The remaining factors in the list above are all within our control as individuals, professionals, and policy makers. We discuss them throughout this chapter.

· It is repugnant to the principles of a free society that a person should ever be taken into police custody because of a computer error precipitated by government carelessness. As automation increasingly invades modern life, the potential for Orwellian mischief grows. —Arizona Supreme Court 5

8.1.3 System Failures

Modern communications, power, medical, financial, retail, and transportation systems depend heavily on computer systems. They do not always function as planned. We give examples of failures, with some indications of the causes. For computer science students and others who might contract for or manage custom software, one aim is to see the serious impacts of the failures—and to see what you want to work hard to avoid. The lessons of adequate planning and testing, of having backup plans in case of failures, and of honesty in dealing with errors apply to large projects in other professions as well.

Millions of BlackBerry users did not get their email for nine hours after the company installed a faulty software update. Customers of AT&T lost telephone service for voice and data for hours because of a software error in a four-million-line program. A three-line change in a two-million-line telecommunications switching program caused a failure of telephone networks in several major cities. Although the program underwent 13 weeks of testing, it was not retested after the change—which contained a typo. American Express Company’s credit card verification system failed during the Christmas shopping season. Merchants had to call in for verification, overwhelming the call center. Log-ins overloaded Skype’s peer-to-peer network system when a huge number of people rebooted their computers after installing routine Windows updates. A majority of Skype’s Internet phone users could not log in for two days.

BlackBerry thumb and RSI

Millions of children play games on small electronic devices, and millions of adults answer email on portable electronic gadgets with mini keypads. In many professions, people type on a keyboard for hours each day. Most of the risks we describe in this chapter result from errors in software, poor system design, or inaccurate and misinterpreted information. Here, we look at physical phenomena known as BlackBerry thumb, gamer’s thumb, Nintendonitis, repetitive strain injury (RSI), and by a variety of other terms. Repetitive strain injury, the more formal term, covers a variety of injuries or pain in thumbs, fingers, wrists, and arms (and sometimes neck and shoulders). You may have seen computer programmers, prolific bloggers, or secretaries wearing wrist braces, called splints—a common sign of RSI. These injuries can make ordinary activities painful or impossible and can prevent people from working.

RSI is not a new disease. There are references to similar problems in the 18th and 19th centuries afflicting clerks and scribes (we used to call this “writer’s cramp”), women who milked cows, and others whose work required repetitive hand motions. RSI problems occur among gymnasts, sign language interpreters for the deaf, “pushup enthusiasts,” auto workers, seamstresses, musicians, carpenters, meat processors, and workers in bakery factories. (An article in the Journal of the American Medical Association listed 29 occupations with common RSI problems.) 6  Computer game players and smart-phone and keyboard users are among the newest significant group of RSI sufferers.

Thousands of people suffering from RSI sued keyboard makers and employers in the 1990s. They charged that the companies were at fault and should pay for medical costs and damages to the victims. Many of the suits resulted in dismissals or decisions for the defendants. The uncertainty of causation (defects in the devices or improper use) made it difficult to win such suits. Some judges and others compare the complaints to ordinary aches and pains from overexercising or overusing a normally safe tool or device. What would we think of an RSI lawsuit against the maker of a tennis racket or a violin?

Attention to proper ergonomic design of keyboards and workstations reduced RSI problems for keyboard users. We can now buy split, twisted, and otherwise nontraditionally shaped keyboards—each one implementing some manufacturer’s idea of what will be more comfortable and reduce strain. But modifying equipment alone does not solve the problem. RSI experts stress the importance of training in proper technique (including the importance of rest breaks, posture, and exercises). One can install free software that interrupts the user at regular intervals for rest breaks and software-guided exercises. Speech input devices might also reduce RSI caused by keyboard use. (But we might discover an increase in strain of the vocal cords.) Partly because of growing recognition of the RSI problem, and partly as protection against lawsuits, computer companies now provide information about proper use and arrangement of keyboards. Some game device makers package their product with reminders for users to take rest breaks.

Adult users of any tool or toy should learn proper techniques for its use. Young children need parental supervision or rules for electronic devices, as they might, for example, about wearing a helmet when riding a bicycle. Employers have a responsibility to provide training in proper and safe use of tools. Being aware of the potential for RSI might or might not encourage game players and tweeters to take breaks and rest their hands and fingers. Mothers and doctors tell us repeatedly that we should sit up straight, exercise often, and eat our vegetables. Many people do not follow this advice—but, once we have the information, we can choose what to do with it.

· Balance is very important for hand comfort. You’ll be surprised at how quick your wrist will ache if the knife is not balanced properly. —George McNeill, Executive Chef, Royal York Hotel, Toronto (on an advertisement for fine cutlery)

When a Galaxy IV satellite computer failed, many systems we take for granted stopped working. Pager service stopped for an estimated 85% of users in the United States, including hospitals and police departments. Airlines that got their weather information from the satellite had to delay flights. The gas stations of a major chain could not verify credit cards. Some services were quickly switched to other satellites or backup systems. It took days to restore others. 7

Every few years, the computer system of one of the world’s large stock exchanges or brokerages fails. An error in a software upgrade shut down trading on the Tokyo Stock Exchange. A glitch in an upgrade in the computer system at Charles Schwab Corporation crashed the system for more than two hours and caused intermittent problems for several days. Customers could not access their accounts or trade online. A computer malfunction froze the London Stock Exchange for almost eight hours—on the last day of the tax year, affecting many people’s tax bills. 8

A failure of Amtrak’s reservation and ticketing system during Thanksgiving weekend caused delays because agents had no printed schedules or fare lists. Virgin America airline switched to a new reservation system a month before Thanksgiving. Its website and check-in kiosks did not work properly for weeks. 9

The $125 million Mars Climate Orbiter disappeared when it should have gone into orbit around Mars. One team working on the navigation software used English-measure units while another team used metric units. The investigation of the loss emphasized that while the error itself was the immediate cause, the fundamental problem was the lack of procedures that would have detected the error. 10

An inventory management system caused severe losses for businesses that used it. The system had been developed and written for one computer and operating system, then modified and sold to run on another. The modified system sometimes did not accept purchase orders, causing an expensive backlog in orders. Printing invoices took minutes instead of seconds. The system gave incorrect information about inventory and prices. Several users claimed that although the company that sold the system received complaints of serious problems from many customers, the company told customers the problems they were having were unique. Eventually, the company agreed it “did not service customers well” and the program should have undergone more extensive testing. The sources of the problems included technical difficulties (converting software to a different system), poor management decisions (inadequate testing of the modified system on the new platform), and, according to the customers, dishonesty in promoting the system and responding to the problems.11

Voting systems

The U.S. presidential election of 2000 demonstrated some of the problems of old-fashioned election machines and paper or punch-card ballots. Vote counters found these ballots sometimes difficult to read or ambiguous. Recounting was a slow tedious process. Many people saw electronic systems as the solution. In 2002, Congress passed the Help America Vote Act and authorized $3.8 billion to improve voting systems. By the 2006 elections, only a very small percentage of Americans voted with paper ballots. The rush to electronic voting machines demonstrated that they too could have numerous faults. Here are some of the problems that occurred: Some electronic voting systems just crashed— voters were unable to vote. Machines in North Carolina failed to count more than 400 votes because of a technical problem. One county lost more than 4000 votes because the machine’s memory was full. A programming error generated 100,000 extra votes in one Texas county. A programming error caused some candidates to receive votes actually cast for other candidates.

Destroying careers and summer vacations  12

CTB/McGraw-Hill develops and scores standardized tests for schools. Millions of students take its tests each year. An error in CTB’s software caused it to report test results incorrectly—substantially lower than the correct scores—in several states. In New York City, school principals and superintendents lost their jobs because their schools appeared to be doing a poor job of teaching students to read. Educators endured personal and professional disgrace. One man said he applied for 30 other superintendent jobs in the state but did not get one. Parents were upset. Nearly 9000 students had to attend summer school because of the incorrect scores. Eventually, CTB corrected the error. New York City’s reading scores had actually risen five percentage points.

Why was the problem not detected sooner, soon enough to avoid firings and summer school? School testing officials in several states were skeptical of the scores showing sudden, unexpected drops. They questioned CTB, but CTB told them nothing was wrong. They said CTB did not tell them that other states experienced similar problems and also complained. When CTB discovered the software error, the company did not inform the schools for many weeks, even though the president of CTB met with school officials about the problem during those weeks.

What lessons can we learn from this case? Software errors happen, of course. People usually notice significant mistakes, and they did here. But the company did not take seriously enough the questions about the accuracy of the results and was reluctant to admit the possibility—and later the certainty—of errors. It is this behavior that must change. The damage from an error can be small if the error is found and corrected quickly.

CTB recommended that school districts not use scores on its standardized tests as the sole factor in deciding which students should attend summer school. But New York City did so. In a case with a similar lesson, Florida state officials relied on computer-generated lists of possible felons to prevent some people from voting, even though the database company supplying the lists said the state should do additional verification. 13  Relying solely on one factor or on data from one database is temptingly easy. It is a temptation that people responsible for critical decisions in many situations should resist.

Security against vote fraud and sabotage is a significant issue in elections. Programmers or hackers can intentionally rig software to give inaccurate results. Depending on the structure of the system, independent recounting may be difficult. Security researchers strongly criticized electronic voting machines. They said the machines had insecure encryption techniques (or none at all), insufficient security for installation of upgrades to software, and poor physical protection of the memory card on which the system stores votes. One research group demonstrated a system’s vulnerability to a virus that essentially took over the machine and manipulated the vote results. They found that voting system developers lacked sufficient security training. Programmers omitted basic procedures such as input validation and boundary checks. Researchers opened the access panel on a voting machine with a standard key that is easily available and used in office furniture, electronic equipment, and hotel minibars. There were certification standards for voting systems, but some flawed systems were certified; the standards were inadequate. 14  In some counties, election officials gave voting machines to high school students and other volunteers to store at home and deliver to polling places on election day.

Many of the failures that occurred result from causes we will see over and over: lack of sufficient planning and thought about security issues, insufficient testing, and insufficient training. (In this application, the task of training users is complex. Thousands of ordinary people volunteer as poll workers and manage and operate the machines on election day.) An underlying cause is haste. In projects like these, the desire of states to obtain federal grants encourages haste. The grants have short limits on how soon the states must spend the money,

Long before we voted on computers, Chicago and parts of Texas were infamous for vote fraud. In some cities, election officials found boxes full of uncounted paper ballots after an election was over. Reasonable accuracy and authenticity of vote counts are essential in a healthy democracy. Electronic systems have the potential for reducing some kinds of fraud and accidental loss of ballots, but they introduce a host of other problems that must be addressed. The first step is to recognize that developing them requires a high degree of professionalism and a high degree of security. In the near future, we will probably vote online. Sadly, it is likely that, at least at first, online voting systems will be highly vulnerable to fraud.

· Those who cast the votes decide nothing. Those who count the votes decide everything. —Attributed to Joseph Stalin (former Premier of the Soviet Union) 15

Stalled airports: Denver, Hong Kong, and Malaysia

Ten months after the $3.2 billion Denver International Airport airport was supposed to have opened, I flew over the huge airport. It covers 53 square miles, roughly twice the size of Manhattan. It was an eerie sight—nothing moved. There were no airplanes or people at the airport and no cars on the miles of wide highway leading to it. The opening was rescheduled at least four times. The delay cost more than $30 million per month in bond interest and operating costs. The computer-controlled baggage-handling system, which cost $193 million, caused most of the delay. 16

The plan for the baggage system was quite ambitious. Outbound luggage checked at ticket counters or curbside counters was to travel to any part of the airport in less than 10 minutes via an automated system of carts traveling at up to 19 miles per hour on 22 miles of underground tracks. Similarly, inbound luggage would go to terminals or transfer directly to connecting flights anywhere in the airport. Carts, bar-coded for their destinations, carried the bags. Laser scanners throughout the system tracked the 4000 carts and sent information about their locations to computers. The computers used a database of flights, gates, and routing information to control motors and switches to route the carts to their destinations.

The system did not work as planned. During tests over several months, carts crashed into each other at track intersections. The system misrouted, dumped, and flung luggage. Carts needed to move luggage went by mistake to waiting pens. Both the specific problems and the general underlying causes are instructive. Some of the specific problems:

· • Real-world problems. Some scanners got dirty or knocked out of alignment and could not detect carts going by. Faulty latches on the carts caused luggage to fall onto the tracks between stops.

· • Problems in other systems. The airport’s electrical system could not handle the power surges associated with the baggage system. The first full-scale test blew so many circuits that the test had to be halted.

· • Software errors. A software error caused the routing of carts to waiting pens when they were actually needed.

No one expects software and hardware of this complexity to work perfectly when first tested. In real-time systems, *  especially, there are numerous interactions and conditions that designers might not anticipate. Mangling a suitcase is not embarrassing if it occurs during an early test and if the problem is fixed. It is embarrassing if it occurs after the system is in operation or if it takes a year to fix. What led to the extraordinary delay in the Denver baggage system? There seem to have been two main causes:

*Real-time systems are systems that must detect and respond to or control activities of objects or people in the real world within time constraints.

· • The time allowed for development and testing of the system was insufficient. The only other baggage system of comparable size was at Frankfurt Airport in Germany. The company that built that system spent six years on development and two years testing and debugging. BAE Automated Systems, the company that built the Denver system, was asked to do it in two years. Some reports indicate that because of the electrical problems at the airport, there were only six weeks for testing.

· • Denver made significant changes in specifications after the project began. Originally, the automated system was to serve United Airlines, but Denver officials decided to expand it to include the entire airport, making the system 14 times as large as the automated baggage system BAE had installed for United at San Francisco International Airport.

As a PC Week reporter said, “The bottom-line lesson is that system designers must build in plenty of test and debugging time when scaling up proven technology into a much more complicated environment.” 17  Some observers criticized BAE for taking on the job when the company should have known that there was not enough time to complete it. Others blamed the city government for poor management, politically motivated decisions, and proceeding with a grandiose but unrealistic plan.

Opening day at the new airports in Hong Kong and Kuala Lumpur were disasters. The ambitious and complex computer systems at these airports were to manage everything: moving 20,000 pieces of luggage per hour and coordinating and scheduling crews, gate assignments for flights, and so on. Both systems failed spectacularly. At Hong Kong’s Chek Lap Kok airport, cleaning crews and fuel trucks, baggage, passengers, and cargo went to the wrong gates, sometimes far from where their airplanes were. Airplanes scheduled to take off were empty. At Kuala Lumpur, airport employees had to write boarding passes by hand and carry luggage. Flights, of course, were delayed; food cargo rotted in the tropical heat.

At both airports, the failures were blamed on people typing in incorrect information. In Hong Kong, it was perhaps a wrong gate or arrival time that was dutifully sent throughout the system. In Kuala Lumpur, mistakes by check-in agents unfamiliar with the system paralyzed it. “There’s nothing wrong with the system,” said a spokeman at the airport in Malaysia. A spokesman at Hong Kong made a similar statement. They are deeply mistaken. One incorrect gate number would not have caused the problems experienced at Hong Kong. Any system that has a large number of users and a lot of user input must be designed and tested to handle input mistakes. The “system” includes more than software and hardware. It includes the people who operate it. As in the case of the Denver airport, there were questions about whether political considerations, rather than the needs of the project, determined the scheduled time for the opening of the airports. 18

Abandoned systems

The flaws in some systems are so extreme that the systems end up in the trash after wasting millions, or even billions, of dollars. A large British food retailer spent more than $500 million on an automated supply management system; it did not work. The Ford Motor Company abandoned a $400 million purchasing system. The California and Washington state motor vehicle departments each spent more than $40 million on computer systems before abandoning them because they never worked properly. A consortium of hotels and a rental car business spent $125 million on a comprehensive travel-industry reservation system, then canceled the project because it did not work. The state of California spent more than $100 million to develop one of the largest and most expensive state computer systems in the country: a system for tracking parents who owe child support payments. After five years, the state abandoned the system. After spending $4 billion, the IRS abandoned a tax-system modernization plan; a Government Accountability Office report blamed mismanagement. The FBI spent $170 million to develop a database called the Virtual Case File system to manage evidence in investigations, then scrapped it because of many problems. A Department of Justice report blamed poorly defined and changing design requirements, lack of technical expertise, and poor management. (The FBI’s next major attempt at a paperless case-management system was scheduled for completion in 2009 but delayed at least until 2012.) 19  There are many more such examples.

Figure 8.1 Why abandonned systems failed.

Software expert Robert Charette estimates that from 5% to 15% of information technology projects are abandoned before or soon after delivery as “hopelessly inadequate.”  Figure 8.1  includes some reasons he cites. 20  Such large losses demand attention from computer professionals, information technology managers, business executives, and public officials who set budgets and schedules for large projects.

Legacy systems

After US Airways and America West merged, they combined their reservations systems. The self-service check-in kiosks failed. Long lines at ticket counters delayed thousands of passengers and flights. Merging different computer systems is extremely tricky, and problems are common. But this incident illustrates another factor. According to a vice president of US Airways, most airline systems date from the 1960s and 1970s. Designed for the mainframe computers of that era, they, in some cases, replaced reservations on 3 × 5 paper cards. These old systems “are very reliable, but very inflexible,” the airline executive said. 21  These are examples of “legacy systems”—out-of-date systems (hardware, software, or peripheral equipment) still in use, often with special interfaces, conversion software, and other adaptations to make them interact with more modern systems.

The problems of legacy systems are numerous. Old hardware fails and replacement parts are hard to find. Old software often runs on newer hardware, but it is still old software. Programmers no longer learn the old programming languages. Old programs often had little or no documentation, and the programmers who wrote the software or operated the systems have left the company, retired, or died. If there were good design documents and manuals, they probably no longer exist or cannot be found. Limited computer memory led to obscure and terse programming practices. A variable a programmer might now call “flight_number” would then have been simply “f.”

The major users of computers in the early days included banks, airlines, government agencies, and providers of infrastructure services such as power companies. The systems grew gradually. A complete redesign and development of a fully new, modern system would, of course, be expensive. It would require a major retraining project. The conversion to the new system, possibly requiring some downtime, could also be very disruptive. Thus, legacy systems persist.

We will continue to invent new programming languages, paradigms, and protocols— and we will later add on to the systems we develop as they age. Among the lessons legacy systems provide for computer professionals is the recognition that someone might be using your software 30 or 40 years from now. It is important to document, document, document your work. It is important to design for flexibility, expansion, and upgrades.

8.1.4 What Goes Wrong?

Computer systems fail for two general reasons: the job they are doing is inherently difficult, and sometimes the job is done poorly. Several factors combine to make the task difficult. Computer systems interact with the real world (including both machinery and unpredictable humans), include complex communications networks, have numerous features and interconnected subsystems, and are extremely large. Automobiles, passenger airplanes, and jet fighters contain millions of lines of computer code. 22  A smartphone has several millions of lines of code. Computer software is “nonlinear” in the sense that, whereas a small error in a mechanical system might cause a small degradation in performance, a single typo in a computer program can cause a dramatic difference in behavior.

The job can be done poorly at any of many stages, from system design and implementation to system management and use. (This characteristic is not unique to computer systems, of course. We can say the same about building a bridge, a house, a car, or any complex system.)  Figure 8.1  (in Section 8.1.3 ) summarized high-level, management-related causes of system failures.  Figure 8.2 lists more factors in computer errors and system failures. The examples we described illustrate most of them. We comment on a few.

Figure 8.2 Some factors in computer system errors and failures.

Overconfidence

Overconfidence, or an unrealistic or inadequate understanding of the risks in a complex system, is a core issue. When system developers and users appreciate the risks, they have more motivation to use the techniques that are available to build more reliable and safer systems and to be responsible users. How many people do not back up their files or contact lists until after their computers crash or they lose their phones?

Some safety-critical systems that failed had supposedly “fail-safe” computer controls. In some cases the logic of the program was fine, but the failure resulted from not considering how the system interacts with real users or real-world problems (such as loose wires, fallen leaves on train tracks, a cup of coffee spilled in an airplane cockpit, and so on).

Unrealistic estimates of reliability or safety can come from genuine lack of understanding, from carelessness, or from intentional misrepresentation. People without a high regard for honesty, or who work in an organization that lacks a culture of honesty and focus on safety, sometimes give in to business or political pressure to exaggerate safety, to hide flaws, to avoid unfavorable publicity, or to avoid the expense of corrections or lawsuits.

Reuse of software: the Ariane 5 rocket and “No Fly” lists

Less than 40 seconds after the first launch of France’s Ariane 5 rocket, the rocket veered off course and was destroyed as a safety precaution. The rocket and the satellites it was carrying cost approximately $500 million. A software error caused the failure. 23  The Ariane 5 used some software designed for the earlier, successful Ariane 4. The software included a module that ran for about a minute after initiation of a launch on the Ariane 4. It did not have to run after takeoff of the Ariane 5, but a decision was made to avoid introducing new errors by making changes in a module that operated well in Ariane 4. This module did calculations related to velocity. The Ariane 5 travels faster than the Ariane 4 after takeoff. The calculations produced numbers bigger than the program could handle (an “overflow” in technical jargon), causing the system to halt.

A woman named Jan Adams, and many other people with first initial J and last name Adams, were flagged as possible terrorists when they tried to board an airplane. The name “Joseph Adams” is on a “No Fly” list of suspected terrorists (and other people considered safety threats) that the Transportaton Security Agency had given to the airlines. To compare passenger names with those on the “No Fly” list, some airlines used old software and strategies designed to help ticket agents quickly locate a passenger’s reservation record (e.g., if the passenger calls in with a question or to make a change). The software searches quickly and “casts a wide net.” That is, it finds any possible match, which a sales agent can then verify. In the intended applications for the software, there is no inconvenience to anyone if the program presents the agent with a few potential matches of similar names. In the context of tagging people as possible terrorists, a person mistakenly “matched” will likely undergo questioning and extra luggage and body searches by security agents.

Do these examples tell us that we should not reuse software? One of the goals of programming paradigms such as object-oriented code is to make software elements that can be widely used, thus saving time and effort. Reuse of working software should also increase safety and reliability. After all, it has undergone field testing in a real, operational environment; we know it works. At least, we think it works. The critical point is that it works in a different environment. It is essential to reexamine the specifications and design of the software, consider implications and risks for the new environment, and retest the software for the new use.

8.2 Case Study: The Therac-25

8.2.1 Therac-25 Radiation Overdoses

The benefits of computing technology to health care are numerous and very impressive. They include improved diagnosis, monitoring of health conditions, development of new drugs, information systems that speed treatment and reduce errors, devices that save lives, and devices that increase the safety of surgeries. Yet one of the classic case studies of a deadly software failure is a medical device: a radiation treatment machine.

The Therac-25 was a software-controlled radiation-therapy machine used to treat people with cancer. Between 1985 and 1987, Therac-25 machines at four medical centers gave massive overdoses of radiation to six patients. In some cases, the operator repeated an overdose because the machine’s display indicated that it had given no dose. Medical personnel later estimated that some patients received more than 100 times the intended dose. These incidents caused severe and painful injuries and the deaths of three patients. Why is it important to study a case as old as this? To avoid repeating the errors. Medical physicists operating a different radiation-treatment machine in Panama in 2000 tried to circumvent a limitation in the software in an attempt to provide more shielding for patients. Their actions caused dosage miscalculations. Twenty-eight patients received overdoses of radiation, and several died. 24  It seems that dramatic lessons need repetition with each new generation.

What went wrong with the Therac-25?

Studies of the Therac-25 incidents showed that many factors contributed to the injuries and deaths. The factors include lapses in good safety design, insufficient testing, bugs in the software that controlled the machines, and an inadequate system of reporting and investigating the accidents. (Articles by computer scientists Nancy Leveson and Clark Turner and by Jonathan Jacky are the main sources for this discussion. 25 )

To understand the discussion of the problems, it will help to know a little about the machine. The Therac-25 is a dual-mode machine. That is, it can generate an electron beam or an x-ray photon beam. The type of beam needed depends on the tumor being treated. The machine’s linear accelerator produces a high-energy electron beam (25 million electron volts) that is dangerous. Patients must not be exposed to the raw beam. A computer monitors and controls movement of a turntable that holds three sets of devices. Depending on the intended treatment, the machine rotates a different set of devices in front of the beam to spread it and make it safe. It is essential that the proper protective device be in place when the electron beam is on. A third position of the turntable uses a light beam instead of the electron beam to help the operator position the beam precisely in the correct place on the patient’s body.

8.2.2 Software and Design Problems

Design flaws

The Therac-25 followed earlier machines called the Therac-6 and Therac-20. It differed from them in that it was fully computer controlled. The older machines had hardware safety interlock mechanisms, independent of the computer, that prevented the beam from firing in unsafe conditions. The design of the Therac-25 eliminated many of these hardware safety features. The Therac-25 reused some software from the Therac-20 and Therac-6. The developers apparently assumed the software functioned correctly. This assumption was wrong. When new operators used the Therac-20, there were frequent shutdowns and blown fuses, but no overdoses. The Therac-20 software had bugs, but the hardware safety mechanisms were doing their job. Either the manufacturers did not know of the problems with the Therac-20, or they completely missed the serious implications.

The Therac-25 malfunctioned frequently. One facility said there were sometimes 40 dose-rate malfunctions in a day, generally underdoses. Thus, operators became used to error messages appearing often, with no indication that there might be safety hazards.

There were a number of weaknesses in the design of the operator interface. The error messages that appeared on the display were simply error numbers or obscure messages (“Malfunction 54” or “H-tilt”). This was not unusual for early computer programs when computers had much less memory and mass storage than they have now. One had to look up each error number in a manual for more explanation. The operator’s manual for the Therac-25, however, did not include an explanation of the error messages. The maintenance manual did not explain them either. The machine distinguished between errors by the amount of effort needed to continue operation. For certain error conditions, the machine paused, and the operator could proceed (turn on the electron beam) by pressing one key. For other kinds of errors, the machine suspended operation and had to be completely reset. One would presume that the machine would allow one-key resumption only after minor, non-safety-related errors. Yet one-key resumption occurred in some of the accidents in which patients received multiple overdoses.

Atomic Energy of Canada, Ltd. (AECL), a Canadian government corporation, manufactured the Therac-25. Investigators studying the accidents found that AECL produced very little documentation concerning the software specifications or the testing plan during development of the program. Although AECL claimed that they tested the machine extensively, it appeared that the test plan was inadequate.

Bugs

Investigators were able to trace some of the overdoses to two specific software errors. Because many readers of this book are computer science students, I will describe the bugs. These descriptions illustrate the importance of using good programming techniques. Because some readers have little or no programming knowledge, I will simplify the descriptions.

After the operator entered treatment parameters at a control console, a software procedure called Set-Up Test performed a variety of checks to be sure the machine was in the correct position, and so on. If anything was not ready, this procedure scheduled itself to rerun the checks. (The system might simply have to wait for the turntable to move into place.) The Set-Up Test procedure can run several hundred times while setting up for one treatment. A flag variable indicated whether a specific device on the machine was in the correct position. A zero value meant the device was ready; a nonzero value meant it must be checked. To ensure that the device was checked, each time the Set-Up Test procedure ran, it incremented the variable to make it nonzero. The problem wasthat the flag variable was stored in one byte. After the 256th call to the routine, the flag overflowed and showed a value of zero. (If you are not familiar with programming, think of this as an automobile’s odometer rolling over to zero after reaching the highest number it can show.) If everything else happened to be ready at that point, the program did not check the device position, and the treatment could proceed. Investigators believe that in some of the accidents, this bug allowed the electron beam to be on when the turntable was positioned for use of the light beam, and there was no protective device in place to attenuate the beam.

Part of the tragedy in this case is that the error was such a simple one, with a simple correction. No good student programmer should have made this error. The solution is to set the flag variable to a fixed value, say 1, rather than incrementing it, to indicate that the device needs checking.

Other bugs caused the machine to ignore changes or corrections made by the operator at the console. When the operator typed in all the necessary information for a treatment, the program began moving various devices into place. This process could take several seconds. The software checked for editing of the input by the operator during this time and restarted the set-up if it detected editing. However, because of bugs in this section of the program, some parts of the program learned of the edited information while others did not. This led to machine settings that were incorrect and inconsistent with safe treatment. According to the later investigation by the Food and Drug Administration (FDA), there appeared to be no consistency checks in the program. The error was most likely to occur with an experienced operator who was quick at editing input.

In a real-time, multitasking system that controls physical machinery while an operator enters—and might modify—input, there are many complex factors that can contribute to subtle, intermittent, and hard-to-detect bugs. Programmers working on such systems must learn to be aware of the potential problems and to use good programming practices to avoid them.

8.2.3 Why So Many Incidents?

There were six known Therac-25 overdoses. You may wonder why hospitals and clinics continued to use the machine after the first one.

The Therac-25 had been in service for up to two years at some clinics. Medical facilities did not immediately pull it from service after the first few accidents because they did not know immediately that it caused the injuries. Medical staff members considered various other explanations. The staff at the site of the first incident said that one reason they were not certain of the source of the patient’s injuries was that they had never seen such a massive radiation overdose before. They questioned the manufacturer about the possibility of overdoses, but the company responded (after the first, third, and fourth accidents) that the machine could not have caused the patient injuries. According to the Leveson and Turner investigative report, they also told the facilities that there had been no similar cases of injuries.

After the second accident, AECL investigated and found several problems related to the turntable (not including any of the ones we described). They made some changes in the system and recommended operational changes. They declared that they had improved the safety of the machine by five orders of magnitude, although they told the FDA that they were not certain of the exact cause of the accident. That is, they did not know whether they had found the problem that caused the accident or just other problems. In making decisions about continued use of the machines, the hospitals and clinics had to consider the costs of removing the expensive machine from service (in lost income and loss of treatment for patients who needed it), the uncertainty about whether the machine was the cause of the injuries, and, later, when that was clear, the manufacturer’s assurances that they had solved the problem.

A Canadian government agency and some hospitals using the Therac-25 made recommendations for many more changes to enhance safety; they were not implemented. After the fifth accident, the FDA declared the machine defective and ordered AECL to inform users of the problems. The FDA and AECL spent about a year (during which the sixth accident occurred) negotiating about changes in the machine. The final plan included more than two dozen changes. They eventually installed the critical hardware safety interlocks, and most of the machines remained in use after that with no new incidents of overdoses. 26

Overconfidence

In the first overdose incident, when the patient told the machine operator that the machine had “burned” her, the operator told her that was impossible. This was one of many indications that the makers and some users of the Therac-25 were overconfident about the safety of the system. The most obvious and critical indication of overconfidence in the software was the decision to eliminate the hardware safety mechanisms. A safety analysis of the machine done by AECL years before the accidents suggests that they did not expect significant problems from software errors. In one case where a clinic added its own hardware safety features to the machine, AECL told them it was not necessary. (None of the accidents occurred at that facility.)

The hospitals using the machine assumed that it worked safely, an understandable assumption. Some of their actions, though, suggest overconfidence, or at least practices that they should have avoided. For example, operators ignored error messages because the machine produced so many of them. A camera in the treatment room and an intercom system enabled the operator to monitor the treatment and communicate with the patient. (The operator uses a console outside the shielded treatment room.) On the day of an accident at one facility, neither the video monitor nor the intercom was functioning. The operator did not see or hear the patient try to get up after an overdose. He received a second overdose before he reached the door and pounded on it. This facility had successfully treated more than 500 patients with the machine before this incident.

8.2.4 Observations and Perspective

From design decisions all the way to responding to the overdose accidents, the manufacturer of the Therac-25 did a poor job. The number and pattern of problems in this case, and the way they were handled, suggest serious irresponsibility. This case illustrates many of the things that a responsible, ethical software developer should not do. It illustrates the importance of following good procedures in software development. It is a stark reminder of the consequences of carelessness, cutting corners, unprofessional work, and attempts to avoid responsibility. It reminds us that a complex system can work correctly hundreds of times with a bug that shows up only in unusual circumstances—hence the importance of always following good safety procedures in operation of potentially dangerous equipment. This case also illustrates the importance of individual initiative and responsibility. Recall that some facilities installed hardware safety devices on their Therac-25 machines. They recognized the risks and took action to reduce them. The hospital physicist at one of the facilities where the Therac-25 overdosed patients spent many hours working with the machine to try to reproduce the conditions under which the overdoses occurred. With little support or information from the manufacturer, he was able to figure out the cause of some of the malfunctions.

To emphasize that safety requires more than bug-free code, we consider failures and accidents involving other radiation treatment systems. Three patients received overdoses in one day at a London hospital in 1966 when safety controls failed. Twenty-four patients received overdoses from a malfunctioning machine at a Spanish hospital in 1991; three patients died. Neither of these machines had computer controls. 27  Two news reporters reviewed more than 4000 cases of radiation overdoses reported to the U.S. government. Here are a few of the overdose incidents they describe. A technician started a treatment, then left the patient for 10–15 minutes to attend an office party. A technician failed to carefully check the prescribed treatment time. A technician failed to measure the radioactive drugs administered; she just used what looked like the right amount. In at least two cases, technicians confused microcuries and millicuries. *  The underlying problems were carelessness, lack of appreciation for the risk involved, poor training, and lack of sufficient penalty to encourage better practices. (In most cases, the medical facilities paid small fines or none at all.) 28

*A curie is a measure of radioactivity. A millicurie is one thousand times as much as a microcurie.

Most of the incidents we just described occurred in systems without computers. For some, a good computer system might have prevented the problem. Many could have occurred whether or not the treatment system was controlled by a computer. These examples remind us that individual and management responsibility, good training, and accountability are important no matter what technology we use.

8.3 Increasing Reliability and Safety

· Success actually requires avoiding many separate possible causes of failure. —Jared Diamond 29

8.3.1 Professional Techniques

The New York Stock Exchange installed a $2 billion system with hundreds of computers, 200 miles of fiber-optic cable, 8000 telephone circuits, and 300 data routers. The exchange managers prepared for spikes in trading by testing the system on triple and quadruple the normal trading volume. On one day, the exchange processed 76% more trades than the previous record. The system handled the sales without errors or delays. 30  We have been describing failures throughout this chapter. Many large, complex computer systems work extremely well. We rely on them daily. How can we design, build, and operate systems that are likely to function well?

To produce good systems, we must use good software engineering techniques at all stages of development, including specifications, design, implementation, documentation, and testing. There is a wide range between poor work and good work, as there is in virtually any field. Professionals, both programmers and managers, have the responsibility to study and use the professional techniques and tools that are available and to follow the procedures and guidelines established in the various relevant codes of ethics and professional practices. (The Software Engineering Code of Ethics and Professional Practice and the ACM Code of Ethics and Professional Conduct, in Appendix A, are two important sets of general guidelines for the latter.)

Management and communications

Management experts use the term high reliability organization (HRO) for an organization (business or government) that operates in difficult environments, often with complex technology, where failures can have extreme consequences (for example, air traffic control, nuclear power plants). 31 Researchers have identified characteristics of HROs that perform extremely well. These characteristics can improve software and computer systems in both critical and less critical applications. One characteristic is “preoccupation with failure.” That means always assuming something unexpected can go wrong—not just planning, designing, and programming for all problems the team can foresee, but always being aware that they might miss something. Preoccupation with failure includes being alert to cues that might indicate an error. It includes fully analyzing near failures (rather than assuming the system “worked” because it averted an actual failure) and looking for systemic reasons for an error or failure rather than focusing narrowly on the detail that was wrong. (For example, why did some programmers for the Mars Climate Orbitor assume measurements were in English units while others assumed metric?)

Another feature of successful organizations is loose structure. It should be easy for a designer or programmer to speak to people in other departments or higher up in the company without going through rigid channels that discourage communication. An atmosphere of open, honest communication within the organization and between a company and client are essential for learning of problems early and reducing the effort required to handle them.

There is much more to the field of organizational features that encourage success. It is well worthwhile for project managers, founders of start-up companies, and anyone in management to devote time to studying it.

Safety-critical applications

A subfield of computer science focuses on design and development of safety-critical software. Safety specialists emphasize that developers must “design in” safety from the start. There are techniques of hazard analysis that help system designers identify risks and protect against them. Software engineers who work on safety-critical applications should have special training. Software expert Nancy Leveson emphasizes that with good technical practices and good management, you can develop large systems right: “One lesson is that most accidents are not the result of unknown scientific principles but rather of a failure to apply well-known, standard engineering practices.” 32

To illustrate two important principles in safety-critical applications, I will use as examples accidents that destroyed two space shuttles, each killing the seven people onboard. Computer systems and software were not the cause, but these tragedies make the points well. Burning gases leaked from a rocket shortly after launch of the Challenger and destroyed it. The night before the scheduled launch, the engineers argued for a delay. They knew the cold weather posed a severe threat to the shuttle. We cannot prove absolutely that a system is safe, nor can we usually prove absolutely that it will fail and kill someone. An engineer reported that, in the case of the Challenger, “It was up to us to prove beyond a shadow of a doubt that it was not safe to [launch].” 33  For the ethical decision maker, the policy should be to suspend or delay use of the system in the absence of a convincing case for safety, rather than to proceed in the absence of a convincing case for disaster. In the second accident, a large piece of insulating foam dislodged and struck the wing of theColumbia space shuttle as it launched. NASA knew this happened, but pieces of foam had dislodged and struck the shuttle on other flights without causing a major problem. Thus NASA managers declined to pursue available options to observe and repair the damage. Columbia broke up when reentering the earth’s atmosphere at the end of its mission. This tragedy illustrates the danger of complacency. An organization focused on safety must explore ambiguous risks. Tragedies are less likely if the organization has established policies and procedures to evaluate such risks. 34

Specifications

Companies that do well expend extensive effort to learn the needs of the client and to understand how the client will use the system. Good software developers help clients better understand their own goals and requirements, which the clients might not be good at articulating. The long planning stage allows for discovering and modifying unrealistic goals. One company that developed a successful financial system that processes one trillion dollars in transactions per day spent several years developing specifications for the system, then only six months programming, followed by carefully designed, extensive testing.

User interfaces and human factors

If you are editing a document and you try to quit without saving your changes, what happens? Most programs will remind you that you have not saved your changes and give you a chance to do so. The designers of the programs know that people forget or sometimes click or type the wrong command. This is a simple and common example of considering human factors in designing software—one that has avoided personal calamities for millions of people.

Well-designed user interfaces can help avoid many problems. System designers and programmers need to learn from psychologists and human-factors experts who know principles and practices for doing a good job. *  User interfaces should provide clear instructions and error messages. They should be consistent. They should include appropriate checking of input to reduce major system failures caused by typos or other errors a person will likely make.

The crash of American Airlines Flight 965 near Cali, Colombia, illustrates the importance of consistency (and other aspects of good user interfaces). While approaching the airport, the pilot intended to lock the autopilot onto the beacon, called Rozo, that would lead the plane to the airport. The pilot typed “R,” and the computer system displayed six beacons beginning with “R.” Normally, the closest beacon is at the top of the list. The pilot selected it without checking carefully. The beacon at the top of the list was “Romeo” and was more than 100 miles away, near Bogota. The plane turned more than 90 degrees and headed for Romeo. In the dark, it crashed into a mountain, killing 159 people. 35

*See, for example, the books by Shneiderman, Tufte, Nielsen, and Norman in the list of references at the end of the chapter.

In the lawsuits that followed, juries attributed blame mostly to pilot error. The pilot chose the wrong beacon without checking and continued to descend at night after the plane made a large, unexpected turn. One jury assigned some of the responsibility to the companies that provided the computer system. While it is clear that the pilot could have and should have avoided the crash, it is also clear that the inconsistency in the display—not putting the nearest beacon at the top of the list—created the dangerous situation.

Crashing into mountains was a major cause of air travel fatalities. The Cali crash triggered the adoption of a ground proximity warning system (GPWS) to reduce such crashes. Older radar-based systems sometimes gave warning only 10 seconds before a potential impact. The GPWS contains a digital map of the world’s topography. It can give a pilot up to a minute of warning if a plane is too close to a mountain and automatically displays a map of nearby mountains. Dangerous peaks are shown in red. The GWPS is likely responsible for preventing crashes in several incidents in which pilots incorrectly set an altimeter, attempted to land with poor visibility, mistook building lights for airport lights, and so on. No commercial U.S. airliner has crashed into a mountain since the GPWS was implemented. 36

As an illustration of more principles that can help build better and safer systems, we consider several aspects of automated flight systems. An expert in this area emphasizes the following points:37

· • The user needs feedback to understand what the system is doing at any time. This is critical when a pilot must suddenly take over if the automation fails or if he or she must turn it off for any reason. One example is having the throttle move as a manually operated throttle would, even though movement is not necessary when the automated system is operating.

· • The system should behave as an experienced user expects. Pilots tend to reduce their rate of climb as they get close to their desired altitude. On the McDonnell Douglas MD-80, the automated system maintains a climb rate that is up to eight times as fast as pilots typically choose. Pilots, concerned that the plane might overshoot its target altitude, made adjustments, not realizing that their intervention turned off the automated function that caused the plane to level out when it reached the desired altitude. Thus, because the automation behaved in an unexpected way, the airplane climbed too high—exactly what the pilot was trying to prevent. (The incidence of the problem declined with more training.)

· • A workload that is too low can be dangerous. Clearly, an overworked operator is more likely to make mistakes. One of the goals of automation is to reduce the human workload. However, a workload that is too low can lead to boredom, inattention, or lack of awareness of the current status. That is a danger if the pilot must take over in a hurry.

Redundancy and self-checking

Redundancy and self-checking are two techniques important in systems on which lives and fortunes depend. Redundancy takes several forms. On aircraft, several computers can control an accuator on, say, a wing flap. If one computer fails, another can do the job. Software modules can check their own results—either against a standard or by computing the same thing in two different ways and then comparing to see if the two results match. A more complex form of redundancy, used, for example, in flight control systems in aircraft, aims to protect against consistently faulty assumptions or methods of one programming team. Three independent teams write modules for the same purpose, in three different programming languages. The modules run on three separate computers. A fourth unit examines the outputs of the three modules and chooses the result obtained by at least two out of three. Safety experts say that even when programmers work separately, they tend to make the same kinds of errors, especially if there is an error, ambiguity, or omission in the program specifications. 38  Thus, this type of “voting” redundancy, while valuable in many safety-critical applications, might not overcome problems in other areas of the software development process.

Testing

It is difficult to overemphasize the importance of adequate, well-designed testing of software. Testing is not arbitrary. There are principles and techniques for doing a good job. Many significant computer system failures in previously working systems occurred soon after installation of an update or upgrade. Even small changes need thorough testing. Unfortunately, many cost-conscious managers, programmers, and software developers see testing as a dispensable luxury, a step you can skimp on to meet a deadline or to save money. This is a common but foolish, risky, and often irresponsible attitude.

A practice called independent verification and validation (IV&V) can be very useful in finding errors in software systems. IV&V means that an independent company (that is, not the one that developed the program and not the customer) tests and validates the software. Testing and verification by an independent organization is not practical for all projects, but many software developers have their own testing teams that are independent of the programmers who develop a system. The IV&V team acts as “adversaries” and tries to find flaws. IV&V is helpful for two reasons. The people who designed and/or developed a system think the system works. They think they thought about potential problems and solved them. With the best of intentions, they tend to test for the problems they have already considered. Also, consciously or subconsciously, the people who created the system may be reluctant to find flaws in it. Their testing may be half-hearted. Independent testers bring different perspectives, and for them, success in finding flaws is not emotionally or professionally tied to responsibility for those flaws.

You might have used a beta version of a product or heard of beta testing. Beta testing is a near-final stage of testing. A selected set of customers (or members of the public) use a complete, presumably well-tested system in their “real-world” environment. Thus, this is testing by regular users, not software experts. Beta testing can detect software limitations and bugs that the designers, programmers, and testers missed. It can also uncover confusing aspects of user interfaces, the need for more rugged hardware, problems that occur when interfacing with other systems or when running a new program on older computers, and many other sorts of problems.

· We are what we repeatedly do. Excellence, therefore, is not an act, but a habit. —Will Durant, summarizing Aristotle’s view in his Nicomachean Ethics 39

8.3.2 Trust the Human or the Computer System?

How much control should computers have in a crisis? This question arises in many application areas. We address it in the context of aircraft systems.

Like antilock braking systems in automobiles that control braking to avoid skidding (and do a better job than human drivers), computer systems in airplanes control sudden sharp climbs to avoid stalling. Some airplanes automatically descend if they detect cabin depressurization and the pilot does not take action quickly.

The Traffic Collision Avoidance System (TCAS) detects a potential in-air collision of two airplanes and directs the pilots to avoid each other. The first version of the system had so many false alarms that it was unusable. In some incidents, the system directed pilots to fly toward each other rather than away, potentially causing a collision instead of avoiding one. TCAS was improved, however. It is a great advance in safety, according to the head of the Airline Pilots Association’s safety committee. 40  The TCAS systems functioned correctly when a Russian airplane carrying many children and a German cargo plane got too close to each other. The systems detected a potential collision and told the Russian pilot to climb and the German pilot to descend. Unfortunately, the Russian pilot followed an air traffic controller’s instruction to descend, and the planes collided. In this example, the computer’s instructions were better than the human’s. A few months after this tragedy, the pilot of a Lufthansa 747 ignored instructions from an air traffic controller and followed instructions from the computer system instead, avoiding a midair collision. U.S. and European pilots are now trained to follow TCAS instructions even if they conflict with instructions from an air traffic controller.

Pilots are trained to immediately turn off autopilot systems when TCAS signals a potential collision. They manually maneuver the plane to avoid the collision. That might change. Pilots of the Airbus 380, the world’s largest passenger airplane, are trained to allow its autopilot system to control the plane when a midair collision threatens. The aircraft maker says that pilots sometimes overreact to collision warnings and make extreme maneuvers that can injure passengers or cause a collision with other air traffic in the area. The policy is controversial among pilots. 41

Computers in some airplanes prevent certain actions even if the pilot tries them (for example, banking at a very steep angle). Some people object, arguing that the pilot should have ultimate control in case unusual action is needed in an emergency. Based on accident statistics, some airlines believe otherwise: that preventing pilots from doing something “stupid” can save more lives than letting them do something bold and heroic, but outside the program limitations, in the very rare cases where it might be necessary.

8.3.3 Law, Regulation, and Markets

Criminal and civil penalties

Legal remedies for faulty systems include suits against the company that developed or sold the system and criminal charges when fraud or criminal negligence occurs. Families of Therac-25 victims sued; they settled out of court. A bank won a large judgment against a software company for a faulty financial system that caused problems a user described as “catastrophic.” Several people have won large judgments against credit bureaus for incorrect data in credit reports that caused havoc in their lives.

Many contracts for business computer systems limit the amount the customer can recover to the actual amount spent on the computer system. Customers know when they sign the contract that there is generally no coverage for losses incurred because the system did not meet their needs for any reason. Courts uphold such contract limitations. If people and businesses cannot count on the legal system upholding the terms of a contract, contracts would be almost useless. Millions of business interactions that take place daily would become more risky and therefore more expensive. Because fraud and misrepresentation are not, of course, part of a contract, some companies that suffer large losses allege fraud and misrepresentation by the seller in an attempt to recover some of the losses, regardless of whether the allegations have firm grounding.

Well-designed liability laws and criminal laws—not so extreme that they discourage innovation, but clear and strong enough to provide incentives to produce good systems— are important legal tools for increasing reliability and safety of computer systems and accuracy of data in databases, as they are for protecting privacy and for protecting customers in other industries. After-the-fact penalties do not undo the injuries that occurred, but the prospect of paying for mistakes and sloppiness is incentive to be responsible and careful. Payments compensate the victim and provide some justice. An individual, business, or government that does not have to pay for its mistakes and irresponsible actions will make more of them. (In many contexts, the government does not permit lawsuits against it.)

Unfortunately, there are many flaws in liability law in the United States. People often win multimillion-dollar suits when there is no scientific evidence or sensible reason to hold the manufacturer or seller of a product responsible for accidents or other negative impacts. Abuse of the liability lawsuit system almost shut down the small-airplane manufacturing industry in the United States for years. The complexity of large computer systems make designing liability standards difficult, but this is a necessary task.

Regulation and safety-critical applications

Is there legislation or regulation that can prevent life-threatening computer failures? A law saying that a radiation machine should not overdose a patient would be silly. We know that it should not do that. We could ban the use of computer control for applications where an error could be fatal, but such a ban is ill advised. In many applications, the benefits of using computers are well worth the risks.

A widely accepted option is regulation, possibly including specific testing requirements and requirement for approval by a government agency before a new product can be sold. The FDA has regulated drugs and medical devices for decades. Companies must do extensive testing, provide huge quantities of documentation, and get government approval before they sell new drugs and some medical devices. Arguments in favor of such regulation, both for drugs and for safety-critical computer systems, include the following: Most potential customers and people who would be at risk (e.g., patients) do not have the expertise to judge the safety or reliability of a system. It is better to prevent use of a bad product than to rely on after-the-calamity remedies. It is too difficult and expensive for ordinary people to sue large companies successfully.

If the FDA had thoroughly examined the Therac-25 before it was put into operation, it might have found the flaws before any patients were injured. However, we should note some weaknesses and trade-offs in the regulatory approach. 42  The approval process is extremely expensive and time consuming. The multiyear delays in introducing a good product cost many lives. Political concerns affect the approval process. Competitors influence decisions. Also, there is an incentive for bureaucrats and regulators to be overcautious. Damage caused by an approved product results in bad publicity and possible firing for the regulator who approved it. Deaths or losses caused by the delay or failure to approve a good new product are usually not obvious and get little publicity.

Leveson and Turner, in their Therac-25 article, summarize some of these dilemmas:

· The issues involved in regulation of risky technology are complex. Overly strict standards can inhibit progress, require techniques behind the state of the art, and transfer responsibility from the manufacturer to the government. The fixing of responsibility requires a delicate balance. Someone must represent the public’s needs, which may be subsumed by a company’s desire for profits. On the other hand, standards can have the undesirable effect of limiting the safety efforts and investment of companies that feel their legal and moral responsibilities are fulfilled if they follow the standards. Some of the most effective standards and efforts for safety come from users. Manufacturers have more incentive to satisfy customers than to satisfy government agencies. 43

Professional licensing

Another controversial approach to improving software quality is mandatory licensing of software development professionals. Laws require licenses for hundreds of trades and professions. Licensing requirements typically include specific training, the passing of competency exams, ethical requirements, and continuing education. The desired effect is to protect the public from poor quality and unethical behavior. The history of mandatory licensing in many fields shows that the actual goals and the effects were and are not always very noble. In some trades (plumbing, for example), the licensing requirements were devised to keep black people out. Requirements for specific degrees and training programs, as opposed to learning on one’s own or on the job, tend to keep poorer people from qualifying for licenses. Economic analyses have shown that the effect of licensing is to reduce the number of practitioners in the field and keep prices and income for licensees higher than they would otherwise be—in many cases, without any improvement in quality. 44  Some see a requirement for a government-approved license as a fundamental violation of the freedom to work (that is, of the negative right, or liberty, to work, in the terms of  Section 1.4.2 ).

 Clashes between licensing laws and the Web:  Section 3.2.5

There are voluntary approaches to measuring or certifying qualifications of software personnel—for example, a diploma from a respected school and certification programs by professional organizations—particularly for advanced training in specialized areas.

Taking responsibility

In some cases of computer errors, businesses pay customers for problems or damages (without a lawsuit). For example, Intuit offered to pay interest and penalties that resulted from errors in flawed income-tax programs. When United Airlines mistakenly posted ticket prices on its website as low as about $25 for flights between the United States and Europe, it honored tickets purchased before it corrected the error. United, at first, charged the buyers the correct fare and probably had the legal right to do so, but the airline concluded that having angry customers would cost more than the tickets. We noted that business pressures can lead to cutting corners and releasing defective products. Business pressure can also be a cause for insistence on quality and maintaining good customer relations. Good business managers recognize the importance of customer satisfaction and the reputation of the business. Also, some businesses have an ethical policy of behaving responsibly and paying for mistakes, just as a person would pay for accidentally breaking a neighbor’s window with a misdirected softball.

Other market mechanisms besides consumer backlash encourage a quality job and provide ways to deal with the risk of failures. Insurance companies have an incentive to evaluate the systems they insure and require that certain standards are met. Some businesses pay a higher rate for “uninterrupted” satellite communications service. That is, the service company would switch their communications quickly to other satellites in case of a failure. Businesses that can withstand a few hours of interruption need not pay for that extra protection. Organizations whose communications are critical to public safety, such as police departments and hospitals, should take responsibility to ensure they have appropriate backup service, possibly paying extra for the higher level of service.

How can customers protect themselves from faulty software? How can a business avoid buying a seriously flawed program? For high-volume consumer and small-business software, one can consult the many websites that review new programs, or consult one’s social network. Specialized systems with a small market are more difficult to evaluate before purchase. We can check the seller’s reputation with the Better Business Bureau. We can consult previous customers and ask how well the seller did the job. Online user groups for specific software products are excellent sources of information for prospective and current customers. In the case of the Therac-25, the users eventually spread information among themselves. If the Web had existed at the time of the accidents, it is likely that the problems would have been identified sooner and that some of the accidents would not have happened.

8.4 Dependence, Risk, and Progress

8.4.1 Are We Too Dependent on Computers?

Many people who write about the social impacts of computers lament our dependence on computing technology. Because of their usefulness and flexibility, computers, cellphones, and similar devices are now virtually everywhere. Is this good? Or bad? Or neutral? The word “dependence” often has a negative connotation. “Dependence on computers” suggests a criticism of our use of the technology and its gadgets. Is that appropriate?

In Holland, no one discovered the body of a reclusive, elderly man who died in his apartment until six months after his death. Eventually someone noticed that he had a large accumulation of mail. This incident was described as a “particularly disturbing example of computer dependency.” Many of the man’s bills, including rent and utilities, were paid automatically. His pension check went automatically to his bank account. Thus, “all the relevant authorities assumed that he was still alive.” 45  But who expects the local gas company or other “relevant authorities” to discover a death? The problem here, clearly, was the lack of concerned family, friends, and neighbors. I happened to be present in a similar situation. An elderly, reclusive woman died in her home. Within two days, not six months, the mailman noticed that she had not taken in her mail. He informed a neighbor, and together they checked the house. It did not matter whether her utility bills were paid automatically.

On the other hand, many people and businesses are not prepared to do without the computer systems and electronic devices they use every day. Many drivers would be lost if their navigation system failed. A BlackBerry email blackout disrupted the work of bankers, technology workers, talent agents, and others who depend on constant communication—some who receive more than 500 emails per day. A physician commented that modern hospitals and clinics cannot function efficiently without medical information systems. Modern crime fighting depends on computers. Some military jets cannot fly without the assistance of computers. In several incidents, computer failures or other accidents knocked out communications services. Drivers could not buy gasoline with their credit cards. “Customers were really angry,” said a gas station manager. More than 1000 California state lottery terminals were down; people could not buy tickets or collect winnings. Asupermarket manager reported, “Customers are yelling and screaming because they can’t get their money, and they can’t use the ATM to pay for groceries.” 46

Is our “dependence” on electronic technology different from our dependence on electricity, which we use for lighting, entertainment, manufacturing, medical treatments—just about everything? Is our “dependence” on computers different from a farmer’s dependence on a plow? Modern surgery’s dependence on anesthesia?

Computers, smartphones, and plows are tools. We use tools because we are better off with them than without them. They reduce the need for hard physical labor and tedious routine mental labor. They help us be more productive, or safer, or more comfortable. When we have a good tool, we can forget (or no longer even learn) the older method of performing a task. If the tool breaks down, we are stuck. We cannot perform the task until someone fixes it. That can mean that no telephone calls get through for several hours. It might mean the loss of a large amount of money, and it can mean danger or death for some people. But the negative effects of a breakdown do not condemn the tool. To the contrary, for many applications (not all), the inconveniences or dangers of a breakdown are a reminder of the convenience, productivity, or safety the tool provides when it is working. The breakdown can remind us, for example, of the billions of communications, carrying voice, text, photos, and data, that are possible or more convenient or cheaper because of the technology.

Some misconceptions about dependence on computers come from a poor understanding of the role of risk, confusion of “dependence” with “use,” and blaming computers for failures where they were only innocent bystanders. On the other hand, abdication of responsibility that comes from overconfidence or ignorance is a serious problem. There are valid technical criticisms of dependence when a system design allows a failure in one component to cause a major breakdown. There are valid criticisms of dependence when businesses, government agencies, and organizations do not make plans for dealing with systems failures. The wise individual is grateful for ATMs and credit cards, but keeps a little extra cash at home in case they do not work. The driver with a navigation system might choose to keep a map in the car.

8.4.2 Risk and Progress

· Electricity lets us heat our homes, cook our food, and enjoy security and entertainment. It also can kill you if you’re not careful. —“Energy Notes” (Flyer sent with San Diego Gas & Electric utility bills)

We trust older technologies when we turn on a light or ride a bicycle. As the tools and technologies we use become more complex and more interconnected, the amount of damage that results from an individual disruption or failure increases, and we sometimes pay the costs in dramatic and tragic events. If a person out for a walk bumps into another person, neither is likely to be hurt. If both are driving cars at 60 miles per hour, they could be killed. If two jets collide, or one loses an engine, several hundred people could be killed. However, the death rate per mile traveled is lower for air travel than for cars.

Most new technologies were not very safe when first developed. If the death rate from commercial airline accidents in the United States were the same now as it was 50 years ago, 8,000 people would die in plane crashes each year. In some early polio vaccines, the virus was not totally inactivated. The vaccines caused polio in some children. We discover and solve problems. Scientists and engineers study disasters and learn how to prevent them and how to recover from them. A disastrous fire led to the development of fire hydrants—a way to get water to the fire from the water pipes under the street. Automobile engineers used to design the front of an automobile to be extremely rigid, to protect passengers in a crash. But people died and suffered serious injuries because the car frame transmitted the force of a crash to the people. The engineers learned it was better to build cars with “crumple zones” to absorb the force of impact. 47  Software engineering textbooks use the Cali crash, described in  Section 8.3.1 , as an example so that future software specialists will not repeat the mistakes in the plane’s computer system. We learn. Overall, computer systems and other technologies have made air travel safer. In the first decade of this century, there was roughly one fatal accident per four million commercial flights, down 60% from 10 years earlier. 48

The death rate from motor vehicle accidents in the United States declined almost 80% from 1965 to 2010 (from 5.30 per 100 million vehicle miles traveled to 1.13 per 100 million vehicle miles traveled). 49  Why? One significant factor is increased education about responsible use (i.e., the campaign against drunk driving). Devices that protect people when the system fails (seat belts and airbags) are another. Other systems help avoid accidents: Rear-view cameras help drivers avoid hitting a child when backing up. “Night vision” systems detect obstacles and project onto the windshield an image or diagram of objects in the car’s path. Electronic stability systems have sensors that detect a likely roll-over, before the driver is aware of the problem, and electronically slow the engine. As use of technology, automation, and computer systems has increased in virtually all work places, the risk of dying in an on-the-job accident dropped from 39 among 100,000 workers (in 1934) to 5 in 100,000 in 2008. 50

There are some important differences between computers and other technologies. Computers make decisions; electricity does not. The power and flexibility of computers encourages us to build more complex systems—where failures have more serious consequences. The pace of change in computer technology is much faster than that in other technologies. Software is not built from standard, trusted parts as is the case in many engineering fields. These differences affect the kind and scope of the risks we face. They need our attention as computer professionals, workers and planners in other fields, and as members of the public.

Observations

Throughout this chapter, we have made several points:

· 1. Many of the issues related to reliability and safety for computers systems have arisen before with other technologies.

· 2. There is a “learning curve” for new technologies. By studying failures, we can reduce their occurrence.

· 3. Much is known about how to design, develop, and use complex systems well and safely. Ethical professionals learn and follow these methods.

· 4. Perfection is not an option. The complexity of computer systems makes errors, oversights, and failures likely.

· 5. Comparing the risks of using computer technologies with the risks of using other methods, and weighing the risks against the benefits, give us important perspective.

This does not mean that we should excuse or ignore computer errors and failures because failures occur in other technologies. It does not mean we should tolerate carelessness or negligence because perfection is not possible. It does not mean we should excuse accidents as part of the learning process, and it does not mean we should excuse accidents because, on balance, the contribution of computer technology is positive.

The potential for serious disruption of normal activities and danger to people’s lives and health because of flaws in computer systems should always remind the computer professional of the importance of doing his or her job responsibly. Computer system developers and other professionals responsible for planning and choosing systems must assess risks carefully and honestly, include safety protections, and make appropriate plans for shutdown of a system when it fails, for backup systems where appropriate, and for recovery.

Knowing that one will be liable for the damages one causes is strong incentive to find improvements and increase safety. When evaluating a specific instance of a failure, we can look for those responsible and try to ensure that they bear the costs of the damage they caused. It is when evaluating a particular application area or when evaluating the technology as a whole that we should look at the balance between risks and benefits.