2 Assignments

7 Experts and Evidence

Lewis Vaughn

Chapter Objectives

7.1 Experts and Nonexperts

•Understand that an expert is someone who is more knowledgeable in a particular subject area or field than most others are.

•Know that if a claim conflicts with other claims we have good reason to accept, we have good grounds for doubting it.

•Understand that if a claim conflicts with our background information, we have good reason to doubt it.

•Remember that we are justified in doubting a claim when it comes from someone deemed to be an expert who in fact is not an expert. Relying on bogus expert opinion is known as the fallacious appeal to authority.

7.2 Judging Experts

● Recognize the two indicators for being considered an expert: (1) education and training from reputable institutions or programs in the relevant field and (2) experience in the field.

•Recognize two additional indicators of true expertise: (1) reputation among peers and (2) professional accomplishments. •Understand that if we have reason to believe that an expert is biased, we are not justified in accepting the expert’s opinion. •Know that even though qualified, unbiased, honest experts can be wrong, in general, genuine experts are more likely to be right about things in their fields than we are.

7.3 Experts and Personal Experience ● Know that it’s reasonable to accept the evidence provided by personal experience only if

there’s no good reason to doubt it. ● Be aware that our personal experience is susceptible to error: Under certain

circumstances, our senses, memory, and judgment can’t be trusted. 7.4 Innumeracy and Probability •Understand that we are not good at off-the-cuff judgments about the chances of something happening.

•Know that thinking that previous events can affect the probabilities in the random event at hand is a common error known as the gambler’s fallacy. Attitudes toward experts and expertise are changing. Many people, for example, seem to confidently believe the following: •If you read a book, do a Google search, and see what people are saying on social media, you will be an expert. •Experts have been wrong so often that they have no credibility. •Experts can’t be trusted, because they contradict my beliefs. •On the internet, there are no experts: Everyone’s opinion on any issue is equal to everyone else’s.

This chapter (combined with the preceding ones) shows that these beliefs are in fact false. The probability of becoming an instant expert is pretty low, and experts are not—and never have been—infallible, but neither are they clueless. Nonexperts can come to know a lot about some complex issues if they respect evidence, expertise, and critical thinking. And the insight and know-how of experts—when approached critically and used wisely—can help us live more intelligently and avoid mistakes. An expert is someone who is more knowledgeable in a particular subject area or field than most others are. Experts in professions and fields of knowledge provide us with reasons for believing a claim because, in their specialty areas, they are more likely to be right than we are. They are more likely to be right because (1) they have mastered particular skills or bodies of knowledge, and (2) they practice those skills or use that knowledge as their main occupation in life.1 Experts make mistakes, but in general they are much less likely to err than non-experts are. True experts are familiar with the established facts and existing data in their field, understand how to properly evaluate that information, and know how to apply it. Essentially, this means that they know how to assess the evidence and arguments for particular claims involving that information. They are true authorities on a specified subject. Someone who knows the lore of a field but can’t evaluate the reliability of a claim is no expert.

7.1 Experts and Nonexperts

In a complex world where we can never be knowledgeable in every field, we must rely on

experts—a perfectly legitimate state of affairs. But good critical thinkers are careful about expert

opinion, guiding their use of experts by some commonsense principles. Some of these principles

apply to critical thinking generally and some to assessments of opinion and evidence offered by

putative experts. The most basic principle is this:

If a claim conflicts with other claims we have good reason to accept, we have good grounds for

doubting it.

When two claims conflict (as when one says A, and another says not-A), they simply cannot

bothbe true; at least one of them has to be false. You are not justified in believing either one of

them until you resolve the conflict. Sometimes this job is easy. If, for example, the competing

claims are reports of personal observations, you can often decide between them by making

further observations. If your friend says that your dog is sleeping atop your car, and you say that

your dog is not sleeping atop your car (because you checked a short time ago), you can see who’s

right by simply looking at the roof of your car.

Many times, however, sorting out conflicting claims requires a deeper inquiry. You may need to

do some research to see what evidence exists for each of the claims. In the best-case scenario,

you may quickly discover that one of the claims is not credible because it comes from an

unreliable source.

Now suppose that you’re confronted with another type of conflict—this time between a claim

and your background information. Background information is that huge collection of very well

supported beliefs that we all rely on to inform our actions and choices. A great deal of this lore

consists of basic facts about everyday things, beliefs based on overwhelming evidence (including

our own reliable personal observations and the statements of excellent authorities), and strongly

justified claims that we would regard as “common sense” or “common knowledge.” Background

beliefs include obvious claims such as “The sun is hot,” “The Easter bunny is not real,” “Humans

are mortal,” “Fire burns,” and “George Washington lived in the 18th century.”

Suppose, then, that you’re asked to accept this unsupported claim:

Some babies can bench-press a 500-pound weight.

You are not likely to give much credence to this claim for the simple reason that it conflicts with

an enormous number of your background beliefs concerning human physiology, gravity,

weightlifting, and who knows what else.

Or how about this claim:

The U.S. president is entirely under the control of the chief justice of the United States.

This claim is not as outlandish as the previous one, but it, too, conflicts with our background

beliefs, specifically those having to do with the structure and workings of the U.S. government.

So we would have good reason to doubt this one also.

The principle exemplified here is this:

If a claim conflicts with our background information, we have good reason to doubt it.

Other things being equal, the more background information the claim conflicts with, the more

reason we have to doubt it. We would normally—and rightfully—assign a low probability to any

claim that conflicts with a great deal of our background information.

You would be entitled, for example, to have some doubt about the claim that Joan is late for

work if it conflicts with your background information that Joan has never been late for work in

the 10 years you’ve known her. But you are entitled to have very strong doubts about, and to

assign very low credibility to, the claim that Luis can turn a stone into gold just by touching it.

You could even reasonably dismiss the claim out of hand. Such a claim conflicts with too much

of what we know about the physical world.

It’s always possible, of course, that a conflicting claim is true and some of our background

information is unfounded. So many times it’s reasonable for us to examine a conflicting claim

more closely. If we find that it has no good reasons in its favor, that it is not credible, we may

reject it. If, on the other hand, we discover that there are strong reasons for accepting the new

claim, we may need to revise our background information. For example, we may be forced to

accept the claim about Luis’s golden touch (and to rethink some of our background information)

if it is backed by strong supporting evidence. Our background information would be in need of

some serious revision if Luis could produce this stone-to-gold transformation repeatedly under

scientifically controlled conditions that rule out error, fraud, and trickery.

We need to keep in mind that although our background information is generally trustworthy, it

is not infallible. What we assume is a strongly justified belief may be nothing more than

prejudice or dogma. We should therefore be willing to re-examine background beliefs that we

have doubts about—and to be open to reasonable doubts when they arise.

So it is not reasonable to accept a claim if there is good reason to doubt it. And sometimes, if the

claim is dubious enough, we may be justified in dismissing a claim out of hand. But what should

we believe about a claim that is not quite dubious enough to summarily discard yet not worthy of

complete acceptance? We should measure out our belief according to the strength of reasons.

That is,

We should proportion our belief to the evidence.

The more evidence a claim has in its favor, the stronger our belief in it should be. Weak evidence

for a claim warrants weak belief; strong evidence warrants strong belief. And the strength of our

beliefs should vary across this spectrum as the evidence dictates.

Implicit in all of the foregoing is a principle already mentioned, but it deserves to be repeated

because it’s so often ignored:

It’s not reasonable to believe a claim when there is no good reason for doing so.

The famous 20th-century philosopher Bertrand Russell tried hard to drive this idea home. As he

put it, “It is undesirable to believe a proposition when there is no ground whatever for supposing

it true.”2 Russell claimed that if the use of this principle became widespread, social life and

political systems would be transformed.

Now, when a claim runs counter to a consensus among experts, this principle holds:

If a claim conflicts with expert opinion, we have good reason to doubt it.

This tenet follows from our definition of experts. If they really are more likely to be right than

nonexperts about claims in their field, then any claim that conflicts with expert opinion is at

least initially dubious.

Here’s the companion principle:

When the experts disagree about a claim, we have good reason to doubt it.

If a claim is in dispute among experts, then nonexperts can have no good reason for accepting

(or rejecting) it. Throwing up your hands and arbitrarily deciding to believe or disbelieve the

claim is not a reasonable response. The claim must remain in doubt until the experts resolve the

conflict or you resolve the conflict yourself by becoming informed enough to competently decide

on the issues and evidence involved—a course that’s possible but usually not feasible for

nonexperts.

But when is a claim considered in dispute among experts? It’s in dispute when substantial

numbers of experts disagree with one another—but not when a mere handful of dissidents

disagree with almost all of the others. We cannot reasonably consider a claim in dispute when,

say, three experts disagree with five thousand of their fellows. It is disingenuous and misleading

to declare that an issue is undecided when a few experts disagree with the opinions of the

overwhelming majority—which was the case in 2019 when 97 percent of climate scientists

agreed that climate-warming trends are extremely likely to be caused by human activities.3

Sometimes we may have good reason to be suspicious of unsupported claims even when they are

purportedly derived from expert opinion. Our doubt is justified when a claim comes from

someone deemed to be an expert who in fact is not an expert. When we rely on such bogus

expert opinion, we make the mistake known as the fallacious appeal to authority.

The fallacious appeal to authority usually happens in one of two ways. First, we may find

ourselves disregarding this important rule of thumb: Just because someone is an expert in one

field, he or she is not necessarily an expert in another. The opinion of experts generally carries

more weight than our own—but only in their areas of expertise. Any opinions that they proffer

outside their fields are no more authoritative than those of nonexperts. Outside their fields, they

are not experts.

We needn’t look far for real-life examples of such skewed appeals to authority. Any day of the

week we may be urged to accept claims in one field based on the opinion of an expert from an

unrelated field. An electrical engineer or Nobel Prize–winning chemist may assert that herbs can

cure cancer. A radio talk show host with a degree in physiology may give advice in psychology. A

former astronaut may declare that archeological evidence shows that Noah’s ark now rests on a

mountain in Turkey. A botanist may say that the evidence for the existence of ESP is conclusive.

The point is not that these experts can’t be right, but that their expertise in a particular field

doesn’t give us reason to believe their pronouncements in another. There is no such thing as a

general expert, only experts in specific subject areas.

Second, we may fall into a fallacious appeal to authority by regarding a nonexpert as an expert.

We forget that a nonexpert—even one with prestige, status, or sex appeal—is still a nonexpert.

Movie stars, famous actors, YouTube celebs, renowned athletes, and well-known politicians

endorse products of all kinds in online, TV, and print advertising. But when they speak outside

their areas of expertise—when they back their claims by nothing more than their own

opinion—they give us no good reason for believing that the products are as advertised.

Advertisers, of course, know this, but they hope that we will buy the products anyway because of

the appeal or attractiveness of the celebrity endorsers.

Are Doctors Experts?

Yes and no. Physicians are certainly experts in the healing arts, in diagnosing and treating

disease and injury. They know and understand the relevant facts, and they have the wherewithal

to make good judgments regarding those facts. But are physicians experts in determining

whether a particular treatment is safe and effective? Contrary to what many believe, the answer

is, in general, no. Determining the safety and efficacy of treatments is a job for scientists (who

may also be physicians). Medical scientists conduct controlled studies to try to ascertain

whether treatment X can safely alleviate disease A—something that usually cannot be

determined by a doctor interacting with her patients in a clinical setting. Medical studies are

designed to control all kinds of extraneous variables that can skew the study results, the same

extraneous variables that are often present in the doctor’s office.

Critical thinkers should keep this distinction in mind because they will often hear people

assert that treatment Y works just because Dr. Wonderful says so.

Historically, the regarding of a nonexpert as an expert has probably been the most prevalent

form of the appeal to authority—with disastrous results. Political, religious, tribal, and cultural

leaders often have been designated as authorities not because they knew the facts and could

correctly judge the evidence, but because culture, tradition, or whim dictated that they be

regarded as authorities. When these “authorities” spoke, people listened and believed—then

went to war, persecuted unbelievers, or undertook countless other ill-conceived projects. If we

are to avoid this trap, we must look beyond mere labels and titles and ask, “Does this person

provide us with any good reasons or evidence?”

Section Query

SECTION QUERY

Have you ever fallen for a fallacious appeal to authority online—that is, have you ever accepted

the claims of a bogus expert? If so, why?

7.2 Judging Experts This question about good reasons, of course, is just another way of asking if someone is a true expert. How can we tell? To be considered an expert, someone must have shown that he or she has the knowledge, judgment, and competence required in a particular field. What are the indicators that someone has this essential kind of expertise? There are several that provide clues to someone’s ability but do not guarantee the possession of true expertise. In most professional fields, the following two indicators are considered minimal prerequisites for being considered an expert: 1. Education and training from reputable institutions or programs in the relevant field (usually evidenced by degrees or certificates). Teachers, airline pilots, plumbers, electricians, and many others are required to have credentials to show that they have met standards of knowledge and competence. 2. Experience in the field. Long experience (generally the more years the better) suggests that the expert is good enough to have outlasted others who are unsuited to, or unskilled in, the work. He or she has had chances to learn from mistakes and to handle challenges that less experienced practitioners may yet to encounter. But, unfortunately, people can have the requisite education and experience and still not know what they’re talking about in the field in question. Woe be to us, for in the real world there are well-trained, experienced auto mechanics who do terrible work—and tenured PhD’s whose professional judgment is shaky. Two additional indicators, though, are more revealing: 1. Reputation among peers (as reflected in the opinions of others in the same field, relevant prestigious awards, and positions of authority) 2. Professional accomplishments These two indicators are more helpful because they are very likely to be correlated with the intellectual qualities expected in true experts. People with excellent reputations among their professional peers and with significant accomplishments to their credit usually are true experts. As we’ve seen, we are often justified in believing an unsupported claim because it’s based on expert opinion. But if we have reason to doubt the opinion of the experts, then we are not justified in believing the claim based on that opinion. And chief among possible reasons for doubt (aside from conflicting expert opinion) is bias. When experts are biased, they are motivated by something other than the search for the truth—perhaps financial gain, loyalty to a cause, professional ambition, emotional needs, political outlook, sectarian dogma, personal

ideology, or some other judgment-distorting factor. Therefore, if we have reason to believe that an expert is biased, we are not justified in accepting the expert’s opinion. But how can we tell when experts are biased? There are no hard-and-fast rules here. In the more obvious cases, we often suspect bias when an expert is being paid by special-interest groups or companies to render an opinion, or when the expert expresses very strong belief in a claim even though there is no evidence to support it, or when the expert stands to gain financially from the actions or policies that he or she supports. It’s true that many experts can render unbiased opinions and do high-quality research even when they have a conflict of interest. Nevertheless, in such situations we have reasonable grounds to suspect bias—unless we have good reason to believe that the suspicion is unwarranted. These good reasons might include the fact that the expert’s previous opinions in similar circumstances have been reliable or that he or she has a solid reputation for always offering unbiased assessments. There are, of course, many other possible reasons to doubt the opinion of experts. Any blatant violation of the critical thinking principles discussed in this text, for example, would give us good reason to question an authority’s reliability. Among the more common tip-offs of dubious authority are these: • The expert is guilty of simple factual or formal errors. • The expert’s claims conflict with what you have good reason to believe. • The expert does not adequately support his or her assertions. • The expert’s writing contains logical contradictions or inconsistent statements. • The expert does not treat opposing views fairly. • The expert is strongly biased, dogmatic, dismissive, or intolerant. • The expert relies on information you know is out of date. • The expert cherry-picks data to support his or her claims. • Most other experts in the same field disagree. The amount of weight you give to any one of these factors—and the subsequent degree of doubt you attach to an expert’s opinion—will vary in each case. In general, a single minor error of fact or style does not justify dismissing an expert’s entire article that is otherwise excellent. But doubt is cumulative, and as reasons for doubt are added, you may rightfully decide that you are not justified in believing any part of an expert’s testimony, regardless of his or her credentials. Depending on your aims, you may decide to check the expert’s assertions against other sources or to consult an authority with much less evidential or rhetorical baggage. Keep in mind that there are certain kinds of issues that we probably don’t want experts to settle for us. Indeed, in most cases the experts cannot settle them for us. These issues usually involve

moral, social, or political questions. If we’re intellectually conscientious, we want to provide our own final answers to such questions, though we may draw heavily on the analyses and arguments provided by experts. We may study what the experts have to say and the conclusions they draw. But we want ultimately to come to our own conclusions. We prefer this approach in large part because the questions are so important and because the answers we give help define who we are. What’s more, the experts typically disagree on these issues. So even if we wanted the experts to settle one of these questions for us, they probably couldn’t. Do Nonexperts Know Best? Some people have a bias against experts—all experts. Their thoughts on the subject might run something like this: “It’s the uneducated ones, the simple seekers of knowledge who are the truly wise, for their thinking has not yet been corrupted by ivory-tower learning and highbrow theorizing that’s out of touch with the real world. Thus the wisdom of the nonexpert is to be preferred over the expert whenever possible.” This attitude is, oddly enough, sometimes embraced by very educated people. This nonexpertism is related to the appeal to ignorance discussed in Chapter 6. (A variation of the appeal to ignorance says that since there’s no evidence refuting a position, it must be true.) The problem is that both tacks, though psychologically compelling, are fallacious. A lack of good reasons—evidence or expert testimony—does not constitute proof of a claim. In addition, when we as nonexperts try to judge scientific and medical claims using only our personal experience, we are likely to reach conclusions that are wrong (as explained in the next section). The history of science shows that virtually all notable scientific discoveries have been made by true experts—men and women who were fully knowledgeable about their subject matter. There have been many more instances, however, of cocksure nonexperts who proposed theories, cures, and solutions to problems that turned out to be worthless. Here’s an obvious truth that’s easy to forget: Even qualified, unbiased, honest experts can be wrong. In fact, they are often wrong. Error is in the nature of expertise, especially in attempts at prediction. Experts were wrong when they made these predictions: • Fast-moving trains would kill passengers by asphyxiation. • Rockets will never be able to leave Earth’s atmosphere. • Flying cars will soon become common. • The Soviet Union is in no danger of collapsing. • We will find weapons of mass destruction (WMDs) in Iraq. • People will never want a computer in their homes. • The Internet will be a spectacular flop. • The iPhone will be a market failure. •

The American economy is basically healthy (it collapsed in 2008). • Donald Trump will not win the presidential election of 2016. • An AI computer program will not be able to beat a human player at the board game Go until around 2025 (it happened in 2016). But the mistakes of experts do not invalidate our earlier premise that, in general, genuine experts are more likely to be right about things in their fields than we are. When errors do occur, they usually happen because experts depart from investigating and explaining the facts and jump to trying to predict the facts. Prediction in any field is hard, and most experts aren’t very good at it, although some are better at it than others. Prediction is notoriously iffy in the social sciences (notably economics, history, and political science) and in public policy. The natural sciences have a much better track record. Tom Nichols reminds us, Predictive failure, however, does not retroactively strip experts of their claim to know more than laypeople. Laypeople should not jump to the assumption that a missed call by the experts therefore means all opinions are equally valid (or equally worthless). The polling expert Nate Silver, who made his reputation with remarkedly accurate forecasts in the 2008 and 2012 presidential elections, has since admitted that his predictions about Republican presidential nominee Donald Trump in 2016 were based on flawed assumptions. But Silver’s insights into the other races remain solid, even if the Trump phenomenon surprised him and others.4 Fallacious Appeal to (Questionable) Authority Why do so many people listen to the advice and endorsements of famous people who may be no more knowledgeable than the least informed among us? If you have ever fallen for the celebrity version of the fallacious appeal to authority, maybe the following quotes will do you good. They prove that some really famous people can say some really stupid things—and knowing that might help you think twice before getting stung by this fallacy. “I make Jessica Simpson look like a rock scientist.” (Tara Reid) “Whenever I watch TV and see those poor starving kids all over the world, I can’t help but cry. I mean I’d love to be skinny like that, but not with all those flies and death and stuff.” (Mariah Carey) “I get to go overseas places, like Canada.” (Britney Spears) “I’m not anorexic, I’m from Texas. Are there people from Texas that are anorexic? I've never heard of one, and that includes me.” (Jessica Simpson) “Circumcision is barbaric and stupid. Who are you to correct nature? Is it real that GOD requires a donation of foreskin? Babies are perfect.” (Russell Crowe) “I don’t think there is anything particularly wrong about hitting a woman. An open-handed slap is justified if all other alternatives fail and there has been plenty of warning. If a woman is a bitch, or hysterical, or bloody-minded continually, then I’d do it.” (Sean Connery) “The media is—really, the word, I think one of the greatest of all terms I’ve come up with—is fake.” (Donald Trump) “Uh, uh, Chuck Graham, state senator [who is wheelchair-bound], is here. Stand up, Chuck, let ’em see you. Oh, God love you. What am I talking about?” (Joe Biden)

“I think that gay marriage is something that should be between a man and a woman.” (Arnold Schwarzenegger) “So where’s the Cannes Film Festival being held this year?” (Christina Aguilera) Media Activity and Section Query

MEDIA ACTIVITY 7.1 [Please note: You must be using an online, browser-based eReader in order to view this content.]

MEDIA ACTIVITY 7.2 [Please note: You must be using an online, browser-based eReader in order to view this content.]

SECTION QUERY What should be your attitude toward the claims of an authority who seems knowledgeable but is strongly biased and dogmatic?

7.3 Experts and Personal Experience We accept a great many claims because they are based on personal experience—our own or someone else’s. Personal experience, broadly defined, arises from our senses, our memory, and our judgment involved in those faculties. In countless cases, our personal experience (or the personal experience of someone else, online or off) is our evidence (or part of the evidence) that something is or is not the case. You think the herbal tea cured your headache because the pain went away after you drank it. Or you’re sure your headache will go away because people calling themselves experts online say their experience proves that it will. You believe that Jack caused the traffic accident because you, or someone else, witnessed it. You believe that your friend can bend spoons with her mind because you saw her do it at a party. You’re sure that the other guy threw the first punch, not you, because that’s how you remember the incident. Or you share a grainy photo on Instagram and Facebook because the original poster says the picture shows the assailant running from the scene of a mass shooting. Or you vote to convict the alleged assailant because eyewitness testimony puts him at the scene of the crime with a gun in his hand. But can we trust personal experience to reveal the truth? The answer is a qualified yes. And here’s the qualification in the form of an important principle: It’s reasonable to accept the evidence provided by personal experience only if there’s no good reason to doubt it. In the simplest cases, if we have no good reason to doubt what our personal experience reveals to us, then we’re justified in believing it. This means that if our faculties are working properly and our use of them is unimpeded by anything in our environment, we’re entitled to accept what our personal experience tells us. If we seem to see a cat on the mat under good viewing conditions—that is, we have no reason to believe that our observations are impaired by, say,

poor lighting, cracked glasses, or too many beers—then we’re justified in believing that there’s a cat on the mat. But many cases—maybe most of them—are not simple, and there’s plenty of room for doubt. Identifying what causes something inside the human body is difficult at best. Tracking cause and effect or multiple factors in complex situations is never easy, even for careful observers. When deliberate deception is likely to be at play (as in the spoon-bending demonstration and in the online mountains of fake photos and phony testimony), discerning what’s real and what isn’t takes critical inquiry. The main problem is that our personal experience is susceptible to error in all sorts of ways. Under certain circumstances, our senses, memory, and judgment can’t be trusted. It’s easy enough to identify these circumstances in an abstract way. The harder job is (1) determining when they actually occur in real-life situations and (2) avoiding them or taking them into account. Here’s a rundown of the major difficulties. Impairment

This should be obvious: If our perceptual powers are somehow impaired or impeded, we have reason to doubt them. The unambiguous cases are those in which our senses are debilitated because we are ill, injured, tired, stressed out, excited, drugged, drunk, distracted, or disoriented. And just as clear are the situations that interfere with sensory input—when our environment is, say, too dark, too bright, too noisy, or too hazy. If any of these factors are in play, the risk of misperception is high, which gives us reason to doubt the trustworthiness of what we experience. Memories can be affected by many of the same factors that interfere with accurate perception. They are especially susceptible to distortion if they are formed during times of stress—which helps explain why the memories of people who witness crimes or alleged ghosts are so often unreliable. These situations are understandably stressful. The impairment of our faculties is complicated by the peculiar way they operate. Contrary to what many believe, they are not like recording devices that make exact mental copies of objects and events in the world. Research suggests that they are more like artists who use bits of sensory data or memory fragments to concoct creative representations of things, not exact replicas. Our perception and memory are constructive, which means that what we perceive and remember is to some degree fabricated by our minds. Here are some of the more blatant examples: You see a man standing in the shadows by the road—then discover when you get closer that the man is a tree stump. You anxiously await a phone call from Aunt Mary, and when the call comes and you hear the person’s voice, you’re sure it’s her—then realize that it’s some guy asking for a charitable donation. While in the shower you hear the phone ring—but no one is calling, and the ringing is something your mind is making up. The constructive workings of our minds help us solve problems and deal effectively with our environment. But they can also hinder us by manufacturing too much of our experiences using too little data. Unfortunately, the constructive tendency is most likely to lead us astray precisely when our powers of perception and memory are impaired or impeded. Competent investigators of alleged paranormal phenomena understand this and are rightfully skeptical of paranormal claims based on observations made under dubious conditions like those mentioned here. Under the right conditions, the mind is very good at showing us UFOs and midnight ghosts that aren’t

there. Likewise, juries are expected to be suspicious of the testimony of eyewitnesses who swear they plainly saw the dirty deed committed but were frightened, enraged, or a little tipsy at the time. Expectation

A tricky thing about perception is that we often perceive exactly what we expect to perceive—regardless of whether there’s anything there to detect. Ever watch the second hand on an electric clock move—then suddenly realize that the clock is not running at all? Ever been walking through a crowd looking for a friend and hear her call your name—then find out later that she was 10 blocks away at the time? Such experiences—the result again of the constructive tendencies of mind—are common examples of how expectation can distort your perceptions. Scientific research shows that expectation can have a more powerful effect on our experiences than most people think. In numerous studies, subjects who expected to see a flash of light, smell a certain odor, or feel an electric shock did indeed experience these things—even though the appropriate stimuli were never present. The mere suggestion that the stimuli would occur was enough to cause the subjects to perceive, or apparently perceive, things that did not exist. Our tendency to sometimes perceive things that are not really there is especially pronounced when the stimuli are vague or ambiguous. For example, we may perceive completely formless stimuli—clouds, smoke, “white noise,” garbled voices, random-patterned wallpaper, blurry photos, lights in the night sky, stains on the ceiling—yet think we observe very distinct images or sounds. In the formlessness we may see ghosts, faces, and words and hear songs, screams, or verbal warnings. We may see or hear exactly what we expect to see or hear. Or the mere suggestion of what we should perceive helps us perceive it. This phenomenon is a kind of illusion known as pareidolia. It’s the reason some people claim to hear Satanic messages when rock music is played backward, or to observe a giant stone face in fuzzy pictures of the surface of Mars, or to see the perfect likeness of Jesus in the skillet burns on a tortilla. Scientists are keenly aware of the possible distorting influence of expectancy, so they try to design experiments that minimize it. We, too, need to minimize it as much as possible. Our strong expectations are a signal that we should double-check our sensory information and be careful about the conclusions we draw from it. We should also be alert to when other people (such as alleged experts) are not as careful. Causal Confusions

In our lives, we are constantly challenged to sort out causes and effects. We know that mistaking the one for the other, or simply being confused by the causal commotion, can be disadvantageous at best and fatal at worst. Here are some of the more common mistakes. Misidentifying Relevant Factors

A key issue in any type of causal reasoning is whether the factors preceding an effect are truly relevant to that effect. It’s easy to find a preceding factor common to all occurrences of a phenomenon, but that factor may be irrelevant. Relevant factors include only those things that could possibly be causally connected to the occurrence of the phenomenon being studied. Eyewitness Testimony and Wrongful Convictions

Eyewitness testimony is unreliable. So says a raft of scientific evidence. Consider this report published in Scientific American: In 1984 Kirk Bloodworth was convicted of the rape and murder of a nine-year-old girl and sentenced to the gas chamber—an outcome that rested largely on the testimony of five eyewitnesses. After Bloodworth served nine years in prison, DNA testing proved him to be innocent. Such devastating mistakes by eyewitnesses are not rare, according to a report by the Innocence Project, an organization affiliated with the Benjamin N. Cardozo School of Law at Yeshiva University that uses DNA testing to exonerate those wrongfully convicted of crimes. Since the 1990s, when DNA testing was first introduced, Innocence Project researchers have reported that 73 percent of the 239 convictions overturned through DNA testing were based on eyewitness testimony. One third of these overturned cases rested on the testimony of two or more mistaken eyewitnesses. How could so many eyewitnesses be wrong? … The uncritical acceptance of eyewitness accounts may stem from a popular misconception of how memory works. Many people believe that human memory works like a video recorder: the mind records events and then, on cue, plays back an exact replica of them. On the contrary, psychologists have found that memories are reconstructed rather than played back each time we recall them. The act of remembering, says eminent memory researcher and psychologist Elizabeth F. Loftus of the University of California, Irvine, is “more akin to putting puzzle pieces together than retrieving a video recording.” Even questioning by a lawyer can alter the witness’s testimony because fragments of the memory may unknowingly be combined with information provided by the questioner, leading to inaccurate recall. Many researchers have created false memories in normal individuals; what is more, many of these subjects are certain that the memories are real.5 Your ability to identify relevant factors depends mostly on your background knowledge—what you know about the kinds of conditions that could produce the occurrences in which you’re interested. Lack of background knowledge might lead you to dismiss or ignore relevant factors or to assume that irrelevant factors must play a role. The only cure for this inadequacy is deeper study of the causal possibilities in question. Mishandling Multiple Factors

Most of the time, the biggest difficulty in evaluating causal connections is not that there are so few relevant factors to consider—but that there are so many. We are in trouble if we cannot narrow the possibilities to just one or a few, but at the same time, ordinary causal reasoning is frequently flawed because of the failure to consider all the relevant antecedent factors. Sometimes this kind of oversight happens because we simply don’t look hard enough for possible causes. At other times, we miss relevant factors because we don’t know enough about the causal processes involved. This again is a function of skimpy background knowledge. Either way, there is no countermeasure better than your own determination to dig out the whole truth. Being Misled by Coincidence

Sometimes ordinary events are paired in unusual or interesting ways: You think of Hawaii, then suddenly a Twitter ad announces low-cost fares to Maui; you receive some email just as your doorbell sounds and your phone rings; or you stand in the lobby of a hotel thinking of an old friend—then see her walk by. Plenty of interesting pairings can also show up in scientific

research. Scientists might find, for example, that men with the highest rates of heart disease may also have a higher daily intake of water. Or women with the lowest risk of breast cancer may own Toyotas. Such pairings are very probably just coincidence, merely interesting correlations of events. A problem arises, though, when we think that there nevertheless must be a causal connection involved. For several reasons, we may very much want a coincidence or correlation to be a cause-and-effect relationship, so we come to believe that the pairing is causal. Just as often we may mistake causes for coincidences because we’re impressed or excited about the conjunction of events. The pairing of events may seem “too much of a coincidence” to be coincidence, so we conclude that one event must have caused the other. You may be thinking about how nice it would be for your sister to call you from her home in Alaska—then the phone rings, and it’s her! You’re tempted to conclude that your wishing caused her to call. But such an event, though intriguing and seemingly improbable, is not really so extraordinary. Given the ordinary laws of statistics, incredible coincidences are common and must occur. Any event, even one that seems shockingly improbable, is actually very probable over the long haul. Given enough opportunities to occur, events like this surprising phone call are virtually certain to happen to someone. People are especially prone to “it can’t be just coincidence” thinking because, for several psychological reasons, they misjudge the probabilities involved. They may think, for example, that a phone call from someone at the moment they’re thinking of that person is incredible—but only because they’ve forgotten about all the times they’ve thought of that person and the phone didn’t ring. Such probability misjudgments are a major source of beliefs about the paranormal or supernatural. If you observe that the longer you boil eggs, the harder they get (and no other relevant factors complicate this relationship), you can safely conclude that this correlation between boiling and hardening is a causal connection. You have good evidence that the boiling causes the hardening. But most causal connections are not so easily established. In medical science, consistent correlations are highly prized because direct evidence of cause and effect is so hard to come by. Correlations are often indirect evidence of one thing causing another. In exploring the link between cigarette smoking and lung cancer, for example, researchers discovered first that people who smoke cigarettes are more likely to get lung cancer than those who don’t smoke. But later research also showed that the more cigarettes people smoke, the higher their risk of lung cancer. Medical scientists call such a correlation a dose-response relationship. The higher the dose of the element in question (smoking), the higher the response (the more cases of lung cancer). This dose-response relationship between cigarette smoking and lung cancer is, when combined with other data, strong evidence that smoking causes lung cancer. So the important lesson here is this: Correlation does not always mean that a causal relationship is present. A correlation could just be a coincidence. An increase in home PC sales is correlated with a rise in the incidence of AIDS in Africa, but this doesn’t mean that one is in any way causally linked with the other. Unfortunately, there is no foolproof way to distinguish coincidence from cause and effect. But this rule of thumb can help: Don’t assume that a causal connection exists unless you have good reason for doing so.

Usually, when a cause-effect connection is uncertain, only further evaluation or research can clear things up. Confusing Cause with Temporal Order

A particularly prevalent type of misjudgment about coincidences is the logical fallacy known as post hoc, ergo propter hoc (“after that, therefore because of that”). We believe that a cause must precede its effect. But just because one event precedes another that doesn’t mean that the earlier one caused the later. To think so is to be taken in by this fallacy. Outrageous examples of post hoc arguments include: “The rooster crowed, then the sun came up, so the rooster’s crowing caused sunrise!” and “Jasmine left her umbrella at home Monday, and this caused it to rain.” You can clearly see the error in such cases, but consider these arguments: Argument 1 After the training for police officers was enhanced, violent crime in the city decreased by 10 percent. So enhanced training caused the decline in violent crime. Argument 2 An hour after Julio drank the cola, his headache went away. The cola cured his headache. Argument 3 As soon as Smith took office and implemented policies that reflected his conservative theory of economics, the economy went into a downward slide characterized by slow growth and high unemployment. Therefore, the Smith policies caused the current economic doldrums. Argument 4 I wore my black shirt on Tuesday and got an F on a math quiz. I wore the same shirt the next day and flunked my psych exam. That shirt’s bad luck. The conclusion of argument 1 is based on nothing more than the fact that the enhanced training preceded the reduction in violent crime. But crime rates can decrease for many reasons, and the enhanced training may have had nothing to do with the decline in crime. For the argument to be strong, other considerations besides temporal order would have to apply—for example, that other possible causes or antecedent factors had been ruled out; that there was a close correlation between amount of training and decline in crime rates; or that in previous years (or in comparable cities) enhanced training was always followed by decreased violent crime (or no change in training was always followed by steady crime rates). Argument 2 is also purely post hoc. Such reasoning is extremely common and underlies almost all folk remedies and a great deal of quackery and bogus self-cures. You take a vitamin E capsule, and eight hours later your headache is gone. But was it really the vitamin E that did the trick? Or was it some other overlooked factor such as something you ate, the medication you took (or didn’t take), the nap you had, the change in environment (from, say, indoors to outdoors), the natural variation in bodily sensations experienced in any illness, or the stress reduction you felt when you had pleasant thoughts? Would your headache have gone away on its own anyway? Was it the placebo effect—the tendency for people to feel better when treated even when the treatment is fake or inactive? A chief function of controlled medical testing is to evaluate cause-and-effect relationships by systematically ruling out post hoc thinking and irrelevant factors.

Argument 3 is typical post hoc reasoning from the political sphere. Unless there are other good reasons for thinking that the economic policy is causally connected to specific economic events, the argument is weak and the conclusion unreliable. Argument 4 is 100 percent post hoc and undiluted superstition. There is no difference in kind between this argument and much of the notorious post hoc reasoning of centuries ago: “That girl gave me the evil eye. The next day I broke my leg. That proves she’s a witch, and the Elders of Salem should put her to death!” Confusing Cause and Effect

Sometimes we may realize that there’s a causal relationship between two factors—but we may not know which factor is the cause and which is the effect. We may be confused, in other words, about the answers to questions like these: Does your coffee drinking cause you to feel stressed out—or do your feelings of being stressed out cause you to drink coffee? Does participation in high school sports produce desirable virtues such as courage and self-reliance—or do the virtues of courage and self-reliance lead students to participate in high school sports? Does regular exercise make people healthy—or are healthy people naturally prone to regular exercise? Did the misbehaving kid cause the parents to yell at each other—or did the parents’ yelling cause the kid to misbehave? The Deadly Post Hoc Fallacy Despite a growing body of scientific research showing no connection between a measles vaccine and autism in young children, many people have insisted that the vaccine causes the disorder. Some parents of autistic children reasoned that since autism symptoms arose after the children were vaccinated, the vaccine was to blame. As evidence builds against a causal link, the reasoning looks more and more post hoc—and dangerous. Here’s a recent statement from the National Institutes of Health: Study after study has found no link between autism spectrum disorders (ASD) and the measles-mumps-rubella (MMR) vaccine—or any vaccine for that matter. Yet many parents still refuse or delay vaccinations for their young children based on misplaced fear of ASD, which can be traced back to a small 1998 study that’s since been debunked and retracted. Such decisions can have a major negative impact on public health. With vaccination rates in decline, we’ve recently seen the resurgence of measles and other potentially fatal childhood infectious diseases. Among the parents most likely to avoid getting their kids vaccinated are those who already have a child with ASD. So, it’s especially important and timely news that researchers have once again found no link between MMR vaccines and ASD—even among children known to be at greater risk for autism because an older sibling has the developmental brain disorder.6 As you can see, it’s not always a simple matter to discern what the nature of a causal link is. Again, we must rely on our rule of thumb: Don’t assume that a causal connection exists unless you have good reason for doing so. This tenet applies not only to our ordinary experience but to all states of affairs involving cause and effect, including scientific investigations.

In everyday life, sorting cause from effect is often easy because the situations we confront are frequently simple and familiar—as when we’re trying to discover what caused the kettle to boil over. But as we’ve seen, in many other common circumstances, things aren’t so simple. We often cannot be sure that we’ve identified all the relevant factors, or ruled out the influence of coincidence, or correctly distinguished cause and effect. Our rule of thumb, then, should be our guide in all the doubtful cases. Science faces all the same kinds of challenges in its pursuit of causal explanations. And despite its sophisticated methodology and investigative tools, it must expend a great deal of effort to pin down causal connections. Identifying the cause of a disease, for example, usually requires not one study or experiment, but many. The main reason is that uncovering relevant factors and excluding irrelevant or misleading factors is always tough. This is why we should apply our rule of thumb even to scientific research that purports to identify a causal link. Section Query

SECTION QUERY What is the most amazing coincidence you have (or someone you know has) ever experienced? How do you explain this event?

7.4 Innumeracy and Probability

When we make an off-the-cuff judgment about the chances of something happening (whether an event in the past or one in the future), we should be extra careful. Why? Because, generally, we humans are terrible at figuring probabilities. Here’s a classic example. Imagine that your classroom has 23 students present including yourself. What are the chances that at least two of the students have exactly the same birthday? (Not the same date of birth, but the same birthday out of the 365 possible ones.) The answer is neither 1 chance in 365 (1/365), nor 1 in 52 (1/52). It’s 1 chance in 2 (1/2, or 50-50)—a completely counterintuitive result. As we’ve seen, a common error is misjudging the probability of coincidences. Many of us often believe that an event is simply too improbable to be a mere coincidence, that something else surely must be going on—such as secret intervention by someone else. But amazing coincidences occur all the time. The probability that a particular strange event will occur—say, that an ice cube tossed out of an airplane will hit the roof of a barn—may be extremely low, maybe one in a billion. But that same event given enough opportunities to occur may be highly probable over the long haul. It may be unlikely in any given instance for you to flip a coin and get tails seven times in a row. But this “streak” is virtually certain to happen if you flip the coin enough times. What are the odds that someone will be thinking of a person she knew, or knew of, from the past 25 years then suddenly learn that the person is seriously ill or dead? Believe it or not, such a strange event is likely to occur several times a day. If we make the reasonable assumption that someone would recognize the names of a few thousand people (both famous and not so famous) from the past 25 years and that a person would learn of the illness or death of each of those few thousand people in the 25 years, then the chances of our eerie coincidence

happening to someone somewhere are pretty good. We could reasonably expect that each day several people would have this experience. Another error is to think that previous events can affect the probabilities in the random event at hand. This mistake is known as the gambler’s fallacy. Let’s say you toss an unbiased coin six times in a row. On the first toss, the odds are, of course, 1 in 2, or 50-50, that it will land tails. It lands tails. Astoundingly, on the other five tosses the coin also lands tails. That’s six tails in a row. So what are the odds that the coin will land tails on the seventh toss? Answer: 50-50. Each toss has exactly the same probability of landing tails (or heads): 50-50. The coin does not remember previous tosses. To think otherwise is to commit the gambler’s fallacy. You see it a lot in casinos, sporting events, and—alas—everyday decision making. The lesson here is not that we should mistrust all judgment about probabilities, but that we shouldn’t rely solely on our intuitive sense in evaluating them. Relying entirely on intuition, or “gut feeling,” in assessing probabilities is usually not a reason to trust the assessment, but to doubt it. If we require greater precision in judging probabilities, we’re in luck because mathematicians have worked out how to quantify and evaluate them. In the simplest case, calculating the probability of the occurrence of an event or outcome is a matter of division. For example, the probability of getting heads in the toss of an unbiased coin is one chance out of two—1/2 or 0.50. There is one toss and only two possible outcomes, heads or tails. Likewise, the probability of randomly drawing the jack of spades out of a standard deck of 52 cards is 1 chance in 52—1/52, or 0.192. And the probability of drawing one of the hearts out of the deck is 13 in 52, or 0.25 (because there are 13 cards in each suit). Hierarchy of Reliability Every source of information must be judged on its own merits and in the context of other available evidence. But it’s possible to provide a very general ranking of trustworthiness for the sources we rely on for most of our knowledge—a kind of hierarchy of reliability. Here is one such ranking, ranging from most reliable sources to the least. The hierarchy is a general guide, not a precision tool, for even highly ranked sources can sometimes be wrong, low-ranking sources can be right, and some sources may generally be better or worse than their category might suggest. Tier 1 • Articles and reports in peer-reviewed journals and conference proceedings • Books published by academic and highly credible publishers (e.g., Harvard University Press, Cambridge University Press, Oxford University Press, Columbia University Press, Basic Books, Blackwell, John Wiley, Routledge, Rowman and Littlefield) • Official government and university reports and web pages (.gov, .edu, NASA, FDA, NIH, GAO, etc.) Tier 2 • News agencies. (e.g., AP, UPI, Reuters, AFP) •

Highly rated newspapers, magazines, and websites* (e.g., the Economist, World News, Chicago Tribune, Politico, Commentary magazine, the New York Times, National Affairs magazine, National Journal, Washington Blade, the Washington Post, U.S. News and World Report, Barron’s, Fortune magazine) • Highly rated broadcast news organizations** (PBS NewsHour, NPR, ABC News, CBS News, NBC News, C-SPAN, Spectrum News NY1) • Major trade book publishers (e.g., Hachette Book Group, HarperCollins, Macmillan, Simon & Schuster, W. W. Norton, Penguin Random House) Tier 3 • Nongovernmental organizations (NGOs), advocacy groups, and political action committees (PACs) • TV, radio, and podcast shows • Advertising, marketing, publicity (paid sponsored content online—social media, search engines, promotional blogs, websites, etc.; print ads; TV and radio ads) But suppose we want to know the probability of getting a 10 by throwing two dice (two unbiased six-sided dice). Here we are talking about two events that are independent of each other—the event of the first die showing a 5, and the event of the second die showing a 5. The one event has no effect on the other. The probability of the first event occurring is 1 chance out of 6—1/6, and the other event has the same probability, 1/6. To determine the probability of both events happening in one throw of the dice, we find the mathematical product of the two: 1/6 × 1/6 = 1/36. Just as you would expect, the chances of these dual events happening (1/36) are much lower than that of just one of them happening (1/6). So to calculate the probability of two independent events happening together, we multiply the probability of the first event occurring by the probability of the second event occurring. Now let’s say the events in question are not independent of one another—each event can affect the other. Suppose we want to know the probability of drawing two hearts one after another from one standard (shuffled) deck of cards. Note that the deck will be light by one card after the first draw, thereby giving the second draw slightly different odds. The probability of drawing the first heart is 13/52 (13 hearts in the deck of 52), and the probability of drawing the next one is 12/51. To determine the probability of drawing two hearts in a row, we multiply: 13/52 × 12/51 = 1/17. So even when two events are affected by each other, to figure the odds of joint occurrence, we still multiply the probability of the first event occurring by the probability of the second event occurring. Sometimes we may want to know the chances of either one of two events happening. Here we are not looking merely for two events to occur jointly as in the previous examples. We are interested in the odds of either one happening when they are mutually exclusive (if one occurs, the other cannot). Say we want to know the probability of pulling either a diamond or a club from a 52-card deck in one draw. The odds of drawing a diamond is 1 chance out of 4 (1/4), and the

odds of drawing a club is also 1 in 4 (1/4). To figure the odds of drawing either one, we add the two probabilities: 1/4 + 1/4 = 1/2. Section Query

SECTION QUERY Suppose you toss an unbiased coin, and it lands tails three times in a row. What are the odds that your next toss will also be tails? * Rated high for factual reporting by MediaBiasFactCheck.com. ** Rated high for factual reporting by MediaBiasFactCheck.com.

Chapter 8

Science, Nonscience, and the Media

Lewis Vaughn

Chapter Objectives 8.1 What Science Is and Is Not • Define science and understand why it is neither ideology, motivated reasoning, nor technology. • Identify at least four signs of bogus science. 8.2 How Science Is Done • Know the five steps in the scientific method and understand how and why scientists derive test implications of a hypothesis. • Understand how a hypothesis is confirmed or disconfirmed and why this process cannot yield conclusive results. 8.3 Judging Scientific Theories • Explain the reasoning process known as inference to the best explanation and how science uses it to judge the worth of theories. • Understand the minimum requirement of consistency used in judging scientific theories. Define internal and external consistency. •

Explain how the criteria of adequacy are used by scientists to judge the merits of theories. Define testability, fruitfulness, scope, simplicity, conservatism, and ad hoc hypothesis. • Know why conspiracy theories fail as explanations of phenomena. 8.4 Telling Good Theories from Bad • Understand how to use the TEST formula to judge the relative worth of theories. • Explain why Copernicus’s theory of planetary motion is better than Ptolemy’s. • Explain why scientists think evolution is a better theory than creationism. • Understand how scientists know that climate change is happening. 8.5 How the Media Get Science Wrong • Understand that health news is often hyped, exaggerated, and false. • Learn to recognize when a study does not establish that a treatment is effective or that a cause-and-effect relationship exists. • Understand why cause and effect generally cannot be established by single studies, small studies, anecdotes, case studies, nonintervention studies, and animal studies. 8.6 Scientific Opinion Polls • Define random sampling, self-selecting sample, margin of error, and confidence level. • Understand why a poll cannot be trusted if the sample is not random or not representative, the questions are phrased improperly, or the sample is self-selected. Science and technology are making and unmaking the planet, shaping machines and minds, increasing our power and our peril, while we struggle to adapt. Science gives us knowledge—knowledge that some people embrace, some resent, some fear, and some deny. But do we understand this knowledge? More to the point, do we even understand science? This question is not, Do we know that Earth rotates around the sun once a year, or do we know how to use a quantum computer, or can we pass a course in biology or physics? The question has several parts: Do we understand how science is done, how to evaluate the knowledge it yields, and how to use that knowledge to make sense of our world, guide our lives, and plot our futures? The challenge of answering these questions is not merely theoretical. It’s pressing on us right now. Science writer Shawn Otto explains our predicament: Science and technology have come to affect every aspect of life on the planet. There is a phase change going on in the scientific revolution: a shifting from one state to another, as from a solid

to a liquid. There is a sudden, quantitative expansion of the number of scientists and engineers around the globe, coupled with a sudden qualitative expansion of their ability to collaborate with each other over the Internet. These two changes are dramatically speeding up the process of discovery and the convergence of knowledge across once-separate fields, a process Harvard entomologist Edward O. Wilson named consilience. We now have fields where economics merges with environmental science, electrical engineering with neuroscience and physics, computer science with biology and genetics, astronomy with biology, and many more. This consilience is shedding new light on long-held assumptions about the world we live in and the nature of life. Over the course of the next forty years, science is poised to create more knowledge than humans have created in all of recorded history, completely redefining our concepts about—and power over—life and the physical and mental worlds as we assume editing control over the genetic code and mastery in our understanding of the brain. One only has to recall the political battles fought over past scientific advances to see that we are in for a rocky ride. How that rush of new knowledge will impact life, how it will be applied through technology and law, and whether our societies and governments will be able to withstand the immense social and economic upheavals it will bring depends upon whether we can update our political process to accommodate it. Can we manage the next phase of the scientific revolution to our advantage, or will we become its unwilling victims?1 Into this maelstrom of change, whole armies of interested parties are marching, twisting science to their own uses, misinterpreting data, quashing evidence, cherry-picking results, contradicting established facts, pretending that scientific consensus doesn’t exist, and ignoring reality altogether. The soldiers in these anti-science brigades include politicians, legislators, U.S. presidents, PR firms, journalists, political parties, special interest groups, industry hacks, partisan bloggers, talk radio hosts, conspiracy theorists, policymakers, and countless others. How can we safely make our way through this storm? Let’s see what we can do in the following pages.

8.1 What Science Is and Is Not Science is the careful, systematic search for knowledge and understanding of reality through the formulation, testing, and evaluation of theories. It has proven itself through the centuries to be the most powerful tool we have for uncovering truths about the world and distinguishing between what is real and what is not. It has been successful in helping us acquire knowledge because it methodically guards against the common biases and prejudices of the human mind, resists partisan pressure and popular opinion, accepts nothing on faith, and demands hard evidence and solid reasons. Science embodies to a high degree what is essential to reliable knowing of empirical facts: systematic consideration of alternative solutions or theories, rigorous testing of them, and careful checking and rechecking of the conclusions. Some would say that science is reliable because it is self-correcting. Science does not grab hold of an explanation and never let go. Science is not dogma. Instead, it looks at alternative ways to explain a phenomenon, tests these alternatives, and opens up the conclusions to criticism from scientists everywhere. Eventually, after much testing and thinking, scientists may hit upon a

theory that does hold up under scrutiny. They are then justified in accepting that theory, even though no conclusion in science can be proved beyond doubt, and there is no guarantee that future research will not reveal a better theory. We will look more closely at the scientific method and the nature of scientific theories shortly, but we first need to say what science is not. Science Is Not Ideology

Some people say that science is not a way of finding out how the world works, but a worldview affirming how the world is, just as Catholicism or socialism affirms a view of things. To some, science is not only an ideology, but a most objectionable one—one that posits a universe that is entirely material, mechanistic, and deterministic. On this “scientific view,” the world—including us—is nothing more than bits of matter forming a big machine that turns and whirs in predetermined ways. This mechanistic notion is thought to demean humans and human endeavors by reducing us to the role of cogs and sprockets. But we can’t identify science with a specific worldview. At any given time, a particular worldview may predominate in the scientific community, but this fact doesn’t mean that the worldview is what science is all about. Predominant worldviews among scientists have changed over the centuries, but the general nature of science as a way of searching for truth has not. For example, the mechanistic view of the universe, so common among scientists in the 17th century, has now given way to other views. Discoveries in quantum mechanics (the study of subatomic particles) have shown that the old mechanistic perspective is incorrect. Science is, above all, a process. Over the centuries the process has yielded findings that have contradicted prevailing ideologies, but that doesn’t make science an ism. Scientific facts can become political. Because science, by its very nature, questions traditional beliefs and unchallenged assumptions about the world, it often provokes opposition to its findings, opposition that is political, even if the scientific facts themselves are just objective descriptions of reality. Thus we can honestly say that science is not partisan, but it may be political whether it likes it or not. Science Is Not Motivated Reasoning

Recall that motivated reasoning is reasoning for the purpose of supporting a predetermined conclusion, not to uncover the truth (Chapter 3). Many people are guilty of this backward “research,” amassing evidence that agrees with their preferred opinions and ignoring or dismissing evidence that supports contrary views. Science does not work this way. Scientific inquiry begins with a question to answer, formulates hypotheses to answer the question, and then tests those hypotheses to see if any of them are true. This approach commits scientists to following the evidence wherever it leads, even to conclusions that might conflict with their expectations or wishes. Scientific research is initiated precisely because the conclusions are unknown. Among scientists, in universities and research facilities throughout the world, motivated reasoning is considered an egregious violation of scientific practice and professional ethics. Researchers found guilty of flagrant motivated reasoning are censured by the community of scientists and disciplined by scientific organizations. Science Is Not Technology

Science is a way of searching for truth. Technology is not a search for truth; it’s the production of products—cell phones, GPS, social media, robots, self-driving cars, better mousetraps. Technology applies knowledge acquired through science to practical problems that science generally doesn’t care about, such as the creation of electronic gadgets. Technology seeks facts to use in producing stuff. Science tries to understand how the world works not by merely cataloging specific facts but by identifying general principles that both explain and predict phenomena. This nice distinction gets blurry sometimes when technologists do scientific research in order to build a better product or scientists create gadgets in order to do better scientific research. But in general, science pursues knowledge; technology makes things. Seven Warning Signs of Bogus Science What would a distinguished scientist tell trial judges who must try to discern whether scientific testimony by an expert is credible? Robert L. Park is that scientist (as well as an author and professor of physics), and he has identified the following clues “that a scientific claim lies well outside the bounds of rational scientific discourse.” He cautions, though, that “they are only warning signs—even a claim with several of the signs could be legitimate.” 1. The discoverer pitches the claim directly to the media. The integrity of science rests on the willingness of scientists to expose new ideas and findings to the scrutiny of other scientists. Thus, scientists expect their colleagues to reveal new findings to them initially. An attempt to bypass peer review by taking a new result directly to the media, and then to the public, suggests that the work is unlikely to stand up to close examination by other scientists. 2. The discoverer says that a powerful establishment is trying to suppress his or her work. The idea is that the establishment will presumably stop at nothing to suppress discoveries that might shift the balance of wealth and power in society. Often, the discoverer describes mainstream science as part of a larger conspiracy that includes industry and government. Claims that the oil companies are frustrating the invention of an automobile that runs on water, for instance, are a sure sign that the idea of such a car is baloney. 3. The scientific effect involved is always at the very limit of detection. Alas, there is never a clear photograph of a flying saucer, or the Loch Ness monster. All scientific measurements must contend with some level of background noise or statistical fluctuation. But if the signal-to-noise ratio cannot be improved, even in principle, the effect is probably not real and the work is not science. Thousands of published papers in parapsychology, for example, claim to report verified instances of telepathy, psychokinesis, or precognition. But those effects show up only in tortured analyses of statistics. The researchers can find no way to boost the signal, which suggests that it isn’t really there. 4. Evidence for a discovery is anecdotal. If modern science has learned anything in the past century, it is to distrust anecdotal evidence. Because anecdotes have a very strong emotional impact, they serve to keep superstitious beliefs alive in an age of science. The most important discovery of modern medicine is not vaccines or antibiotics; it is the randomized double-blind test, by means of which we know what works and what doesn’t. Contrary to the saying, the word “data” is not the plural of “anecdote.”

5. The discoverer says a belief is credible because it has endured for centuries. There is a persistent myth that hundreds or even thousands of years ago, long before anyone knew that blood circulates throughout the body or that germs cause disease, our ancestors possessed miraculous remedies that modern science cannot understand. Much of what is termed “alternative medicine” is part of that myth. 6. The discoverer has worked in isolation. The image of a lone genius who struggles in secrecy in an attic laboratory and ends up making a revolutionary breakthrough is a staple of Hollywood’s science-fiction films, but it is hard to find examples in real life. Scientific breakthroughs nowadays are almost always syntheses of the work of many scientists. 7. The discoverer must propose new laws of nature to explain an observation. A new law of nature, invoked to explain some extraordinary result, must not conflict with what is already known. If we must change existing laws of nature or propose new laws to account for an observation, it is almost certainly wrong.2 Section Query

SECTION QUERY Do you think science is an ideology—or a process for discovering facts about the world? Explain your answer.

8.2 How Science Is Done The scientific method cannot be identified with any particular set of experimental or observational procedures because there are many different methods to evaluate the worth of a hypothesis. In some sciences such as physics and biology, hypotheses can be assessed through controlled experimental tests. In other sciences such as astronomy and geology, hypotheses usually must be tested through observations. For example, an astronomical hypothesis may predict the existence of certain gases in a part of the Milky Way, and astronomers can use their telescopes to check whether those gases exist as predicted. The scientific method, however, does involve several steps, regardless of the specific procedures involved: 1. Identify the problem or pose a question. 2. Devise a hypothesis to explain the event or phenomenon. 3. Derive a test implication or prediction. 4. Perform the test. 5. Accept or reject the hypothesis. Scientific inquiry begins with a problem to solve or a question to answer. So in step 1 scientists may ask: What causes X? Why did Y happen? Does hormone therapy cause breast cancer? Does aspirin lower the risk of stroke? How is it possible for whales to navigate over long

distances? How did early hominids communicate with one another? Was the Big Bang an uncaused event? In step 2 scientists formulate a hypothesis that will constitute an answer to their question. In every case there are facts to explain, and the hypothesis is an explanation for them. The hypothesis guides the research, suggesting what kinds of observations or data would be relevant to the problem at hand. Without a hypothesis, scientists couldn’t tell which data are important and which are worthless. Where do hypotheses come from? One notion is that hypotheses are generated through induction—by collecting the data and drawing a generalization from them to get a hypothesis. But this can’t be the way that most hypotheses are formulated because they often contain concepts that aren’t in the data. Hypotheses or theories generally reach beyond the known data to posit the existence of things unknown. The construction of hypotheses is not usually based on any such mechanical procedure. In many ways, they are created just as works of art are created. Scientists dream them up. They, however, are guided in hypothesis creation by certain criteria—namely, the criteria of adequacy mentioned earlier. Remember, though, that scientists must consider not just their favorite hypothesis, but alternative hypotheses as well. The scientific method calls for consideration of competing explanations and for their examination or testing at some point in the process. Sometimes applying the criteria of adequacy can immediately eliminate some theories from the running, and sometimes theories must be tested along with the original hypothesis. In step 3 scientists derive implications, or consequences, of the hypothesis to test. As we’ve seen, sometimes we can test a theory directly, as when we simply check the lawnmower’s gas tank to confirm the theory that it won’t run because it’s out of gas. But often theories cannot be tested directly. How would we directly test, for example, the hypothesis that chemical X is causing leukemia in menopausal women? We can’t. So scientists test indirectly by first deriving a test implication from a hypothesis and then putting that implication to the test. Deriving such an observational consequence involves figuring out what a hypothesis implies or predicts. Scientists ask, “If this hypothesis were true, what consequences would follow? What phenomena or events would have to obtain?” The logic of hypothesis testing, then, works like this. When we derive a test implication, we know that if the hypothesis to be tested (H) is true, then there is a specific predicted consequence (C). If the consequence turns out to be false (it does not obtain as predicted), then the hypothesis is probably false, and we can reject it. The hypothesis, in other words, is disconfirmed. We can represent this outcome in a conditional, or hypothetical, argument: If H, then C. not-C. Therefore, not-H. This is, remember, an instance of modus tollens, a valid argument form. In this case, H would be false even if only one of several of its consequences (test implications) turned out to be false. On the other hand, we would get a very different situation if C turned out to be true: If H, then C. C. Therefore, H.

Notice that this is an instance of affirming the consequent, an invalid argument form. So just because C is true, that doesn’t necessarily mean that H is true. If a consequence turns out to be true, that doesn’t prove that the hypothesis is correct. In such a result, the hypothesis is confirmed and the test provides at least some evidence that the hypothesis is true. But the hypothesis isn’t then established. If other consequences for the hypothesis are tested, and all the results are again positive, then there is more evidence that the hypothesis is correct. As more and more consequences are tested, and they are shown to be true, we can have increasing confidence that the hypothesis is in fact true. As this evidence accumulates, the likelihood that the hypothesis is actually false decreases—and the probability that it’s true increases. In step 4 scientists carry out the testing. Usually this experimentation is not as simple as testing one implication and calling it quits. Scientists may test many consequences of several competing hypotheses. As the testing proceeds, some hypotheses are found wanting, and they’re dropped. If all goes well, eventually one hypothesis remains, with considerable evidence in its favor. Then step 5 can happen, as the hypothesis or hypotheses are accepted or rejected. Because scientists want to quickly eliminate unworthy hypotheses and zero in on the best one, they try to devise the most telling tests. This means that they are on the lookout for situations in which competing hypotheses have different test consequences. If hypothesis 1 says that C is true, and hypothesis 2 says that C is false, a test of C can then help eliminate one of the hypotheses from further consideration. Implicit in all this is the fact that no hypothesis can ever be conclusively confirmed. It’s always possible that we will someday find evidence that undermines or conflicts with the evidence we have now. Likewise, no hypothesis can ever be conclusively confuted. But our inability to conclusively confirm or confute a hypothesis does not mean that all hypotheses are equally acceptable. Maintaining a hypothesis in the face of mounting negative evidence is unreasonable, and so is refusing to accept a hypothesis despite accumulating confirming evidence. Through the use of carefully controlled experiments, scientists can often affirm or deny a hypothesis with a high degree of confidence. Let’s see how we might use the five-step procedure to test a fairly simple hypothesis. Suppose you hear reports that some terminal cancer patients have lived longer than expected because they received high doses of vitamin C. And say that the favored hypothesis among many observers is that the best explanation for the patients’ surviving longer is that vitamin C is an effective treatment against cancer. So you decide to test this hypothesis: High doses of vitamin C can increase the survival time of people with terminal cancer. (Years ago, this hypothesis was actually proposed and tested in three well-controlled clinical trials.3) An obvious alternative hypothesis is that vitamin C actually has no effect on the survival of terminal cancer patients and that any apparent benefits are due mainly to the placebo effect (the tendency for people to temporarily feel better after they’re treated, even if the treatment is a fake). The placebo effect could be leading observers to believe that people taking vitamin C are being cured of cancer and are thus living longer. Or the placebo effect could be making patients feel better, enabling them to take better care of themselves (by eating right or complying with standard medical treatment, for example), increasing survival time. Now, if your hypothesis is true, what would you expect to happen? That is, what test implication could you derive? If your hypothesis is true, you would expect that terminal cancer patients

given high doses of vitamin C would live longer than terminal cancer patients who didn’t receive the vitamin (or anything else). How would you conduct such a test? To begin with, you could prescribe vitamin C to a group of terminal cancer patients (called the experimental group) but not to another group of similar cancer patients (called the control group) and keep track of their survival times. Then you could compare the survival rates of the two groups. But many people who knowingly receive a treatment will report feeling better—even if the treatment is an inactive placebo. So any positive results you see in the treated group might be due not to vitamin C but to the placebo effect. To get around this problem, you would need to treat both groups, one with vitamin C and the other with a placebo. That way, if most of the people getting the vitamin C live longer than expected and fewer of those in the placebo group do, you can have slightly better reason for believing that vitamin C works as advertised. But even this study design is not good enough. It’s possible for the people conducting the experiment, the experimenters, to unknowingly bias the results. Through subtle behavioral cues, they can unconsciously inform the test subjects which treatments are real and which ones are placebos—and this, of course, would allow the placebo effect to have full rein. Also, if the experimenters know which treatment is the real one, they can unintentionally misinterpret or skew the study results in line with their own expectations. This problem can be solved by making the study double-blind. In double-blind experiments, neither the subjects nor the experimenters know who receives the real treatment and who the inactive one. A double-blind protocol for your vitamin study would ensure that none of the subjects would know who’s getting vitamin C, and neither would the experimenters. What if you have a double-blind setup but most of the subjects in the vitamin C group were sicker to begin with than those in the placebo control group? Obviously, this would bias the results, making the vitamin C treatment look less effective—even if it is effective. To avoid this skewing, you would need to randomly assign subjects to each group. This randomization helps ensure that each group is as much alike as possible to start. Finally, you would need to run some statistical tests to ensure that your results are not a fluke. Even in the most tightly controlled studies, it’s possible that the outcome is the result of random factors that cannot be controlled. Statisticians have standard methods for determining when experiment results are likely, or not likely, to be due to chance. Suppose you design your study well, conduct it, and the results are that the patients receiving the high doses of vitamin C did not live longer than the placebo group. In fact, all the subjects lived about the same length of time. Therefore, your hypothesis is disconfirmed. On the other hand, the alternative hypothesis—that vitamin C has no measurable effect on the survival of terminal cancer patients—is confirmed. Should you now reject the vitamin C theory? Not yet. Even apparently well-conducted studies can have hidden mistakes in them, or there can be factors that the experimenters fail to take into account. This is why scientists insist on study replication—the repeating of an experiment by different groups of scientists. If the study is replicated by other scientists, and the study results hold up, then you can be more confident that the results are solid. In such a case, you could safely reject the vitamin C hypothesis. (This is, in fact, what scientists did in the real-life studies of vitamin C and cancer survival.)

At this point, when evidence has been gathered that can bear on the truth of the hypothesis in question, good scientific judgment is crucial. It’s here that consideration of other competing hypotheses and the criteria of adequacy again come into play. At this stage, scientists need to decide whether to reject or accept a hypothesis—or modify it to improve it. Nonintervention (Population) Studies Not all medical hypotheses are tested by treating (or not treating) groups of patients and analyzing the results (as in the vitamin C example). Many are tested without such direct intervention in people’s lives. The former type of study is known as an intervention, or controlled, trial, while the latter is called, not surprisingly, a nonintervention study (also an observational or population study). The basic idea in a nonintervention study is to track the interplay of disease and related factors in a specified population, uncovering associations among these that might lead to better understanding or control of the disease process. A typical nonintervention study might go like this: For seven years scientists monitor the vitamin E intake (from food and supplements) and the incidence of heart disease of 90,000 women. Evaluation of this data shows that the women with the highest amounts of vitamin E in their diets have a 40 percent lower incidence of heart disease. That is, for reasons unknown, a lower risk of heart disease is associated with a higher intake of vitamin E in women. This study does not show that higher intakes of vitamin E cause less heart disease, only that there is a link between them. Perhaps some other factor merely associated with vitamin E is the true protector of hearts, or maybe women who take vitamin E are more likely to do other things (such as exercise) that lower their risk of heart disease. Generally, nonintervention studies cannot establish cause-and-effect relationships, though they may hint that a causal relationship is present. And sometimes multiple nonintervention studies yielding the same results can make a strong case for a causal connection. Intervention trials, however, can establish cause and effect. Nonintervention studies have led scientists to some of the most important findings in preventive health. It was a series of such studies done over decades, coupled with other kinds of scientific data, that revealed that cigarette smoking caused cancer. And it was such investigations that showed that high blood pressure, high cholesterol, overweight, and smoking are risk factors for heart disease. Note to critical thinkers: Very often the media misreport the results of nonintervention studies, reading cause and effect into a mere association. For example, if a single nonintervention study finds a link between chewing gum and better eyesight, a headline in a blog, or the morning paper, or a TV newscaster may proclaim, “Gum-chewing improves your eyesight!” Maybe, maybe not—but the study would not justify that conclusion. Section Query

SECTION QUERY Have you ever accepted health or medical claims in a news story that was based on a nonintervention study? If so, why? If not, why not?

8.3 Judging Scientific Theories

Theory testing is part of the broader effort to assess the merits of one theory against a field of alternatives. This broader effort—the central task of science—comes down to asking (and answering) one question: What is the best explanation (theory) for this phenomenon or state of affairs? The best explanation is the one most likely to be true. Science does its most important work, then, through the inductive form of reasoning known as inference to the best explanation (Chapter 2). Inference to the best explanation probably seems very familiar to you. That’s because you use it all the time—and need it all the time. Often when we try to understand something in the world, we construct explanations for why this something is the way it is, and we try to determine which of these is the best. Devising explanations helps increase our understanding by fitting our experiences and background knowledge into a coherent pattern. At every turn we are confronted with phenomena that we can fully understand only by explaining them. Sometimes we’re barely aware that we’re using inference to the best explanation. If we awaken and see that the streets outside are wet, we may immediately posit this explanation: It’s been raining. Without thinking much about it, we may also quickly consider whether a better explanation is that a street-sweeper machine has wet the street. Just as quickly we may dismiss this explanation because we see that the houses and cars are also wet. After reasoning in this fashion, we may decide to carry an umbrella that day. In science, where inference to the best explanation is an essential tool, usually the theories of interest are causal theories, in which events are the things to be explained and the proposed causes of the events are the explanations. Just as we do in everyday life, scientists often consider several competing theories for the same event or phenomenon. Then—through scientific testing and careful thinking—they systematically eliminate inadequate theories and eventually arrive at the one that’s rightly regarded as the best of the bunch. Using this form of inference, scientists discover planets, viruses, cures, subatomic particles, black holes—and many things that can’t even be directly observed. As you can see, the term theory as it’s used in science is not synonymous with conjecture or guess. A theory is an explanation, and if it is the best explanation to explain something, it is a fact. Thus we refer to the germ theory of disease, the heliocentric (sun-centered) theory of planetary motion, Einstein’s theory of relativity, the oxygen theory of combustion, the theory of gravitation—these are facts about the way the world is because they have been established scientifically. So to dismiss a scientific finding by calling it just a theory—as in “evolution is just a theory”—is to misunderstand the scientific meaning of the word. Theories and Consistency

Of course, it’s easy to make up theories to explain things we don’t understand. People do it all the time. The harder job is sorting out good theories from bad. The work can be demanding, but there are special criteria we can use to get the job done. Before we apply these criteria, though, we have to make sure that the theory in question meets the minimum requirement of consistency. A theory that does not meet this minimum requirement is worthless, so there is no need to use the special criteria to evaluate the theory. A theory that meets the requirement is eligible for further consideration. Here we are concerned with both internal and external consistency. A theory that is internally consistent is consistent with itself—it’s free of

contradictions. A theory that is externally consistent is consistent with the data it’s supposed to explain—it fully accounts for the phenomenon to be explained. If we show that a theory contains a contradiction, we have refuted it. A theory that implies that something both is and is not the case cannot possibly be true. By exposing an internal contradiction, Galileo once refuted Aristotle’s famous theory of motion, a venerable hypothesis that had stood tall for centuries. He showed that the theory implied that one falling object falls both faster and slower than another one. If a theory is externally inconsistent, we have reason to believe that it’s false. Suppose you leave your car parked on the street overnight and the next morning discover that (1) the windshield is broken, (2) there’s blood on the steering wheel, and (3) there’s a brick on the front seat. And let’s say that your friend Charlie offers this theory to explain these facts: Someone threw a brick through your windshield. What would you think about this theory? You would probably think that Charlie had not been paying attention. His theory accounts for the broken windshield and the brick—but not the blood on the steering wheel. You would likely toss his theory out and look for one that was complete. Like this one: A thief broke your windshield with a brick and then crawled through the broken window, cutting himself in the process. An adequate theory must fully account for the facts to be explained. Theories and Criteria

A simplified answer to the problem of theory choice is this: Just weigh the evidence for each theory, and the theory with the most evidence wins. As we will soon see, the amount or degree of evidence that a theory has is indeed a crucial factor—but it cannot be the sole criterion by which we assess explanations. Throughout the history of science, major theories—from the heliocentric theory of the solar system to Einstein’s general theory of relativity—have never been established by empirical evidence alone. The task of determining the best explanation has another complication. There could be no end to the number of theories that we could devise to explain the data at hand. In fact, we could come up with an infinite number of possible theories for any phenomenon simply by repeatedly adding one more element. For example, to explain why your cell phone died, you could propose the one-poltergeist theory (a single entity causing the trouble), a two-poltergeist theory, a three-poltergeist theory, and so on. Fortunately, despite these complications, we can use the criteria of adequacy to help us judge the merits of eligible theories and to arrive at a defensible judgment of which theory is best. The criteria of adequacy are the essential tools of science and have been used by scientists throughout history to uncover the best explanations for all sorts of events and states of affairs. Science, though, doesn’t own these criteria. They are as useful—and as used—among nonscientists as they are among men and women of science. Applying the criteria of adequacy to a set of theories constitutes the ultimate test of a theory’s value, for the best theory is the eligible theory that meets the criteria of adequacy better than any of its competitors. Here, eligible means that the theory has already met the minimum requirement for consistency. All of this implies that the evaluation of a particular theory is not complete until alternative, or competing, theories are considered. As we’ve seen, there is an indefinite number of theories that could be offered to explain a given set of data. The main challenge is to give a fair

assessment of the relevant theories in relation to each other. To fail to somehow address the alternatives is to overlook or deny relevant evidence, to risk biased conclusions, and to court error. Such failure is probably the most common error in the appraisal of theories. A theory judged by these criteria to be the best explanation for certain facts is worthy of our belief, and we may legitimately claim to know that such a theory is true. But the theory is not then necessarily or certainly true in the way that a sound deductive argument’s conclusion is necessarily or certainly true. Inference to the best explanation, like other forms of induction, cannot guarantee the truth of the best explanation. That is, it is not truth-preserving. The best theory we have may actually be false. Nevertheless, we would have excellent reasons for supposing our best theory to be a true theory. The criteria of adequacy are testability, fruitfulness, scope, simplicity, and conservatism. Let’s examine each one in detail. Testability

Most of the theories that we encounter every day and all the theories that scientists take seriously are testable—there is some way to determine whether the theories are true or false. If a theory is untestable—if there is no possible procedure for checking its truth—then it is of little or no help in increasing our understanding. Suppose someone says that an invisible, undetectable spirit is causing your headaches. What possible test could we perform to tell if the spirit actually exists? None. So the spirit theory is entirely empty. We can assign no weight to such a claim. Here’s another way to look at it. Theories are explanations, and explanations are designed to increase our understanding of the world. But an untestable theory does not—and cannot—explain anything. It is equivalent to saying that an unknown thing with unknown properties acts in an unknown way to cause a phenomenon—which is the same thing as offering no explanation at all. We often run into untestable theories in daily life, just as scientists sometimes encounter them in their work. Many practitioners of alternative medicine claim that health problems are caused by an imbalance in people’s chi, an unmeasurable form of mystical energy that is said to flow through everyone. Some people say that their misfortunes are caused by God or the Devil. Others believe that certain events in their lives happen (and are inevitable) because of fate. And parents may hear their young daughter say that she did not break the lamp, but her invisible friend did. Many theories throughout history have been untestable. Some of the more influential untestable theories include the theory of witches (some people called witches are controlled by the Devil), the moral fault theory of disease (immoral behavior causes illness), and the divine placement theory of fossils (God created geological fossils to give the false impression of an ancient Earth). But what does it mean for a theory to be testable or untestable? A theory is testable if it predicts something other than what it was introduced to explain. Suppose your electric clock stops each time you touch it. One theory to explain this event is that there is an electrical short in the clock’s wiring. Another theory is that an invisible, undetectable demon causes the clock to stop. The wiring theory predicts that if the wiring is repaired, the clock will no longer shut off when touched. So it is testable—there is something that the theory predicts other than the obvious

fact that the clock will stop when you touch it. But the demon theory makes no predictions about anything, except the obvious, the very fact that the theory was introduced to explain. It predicts that the clock will stop if you touch it, but we already know this. So our understanding is not increased, and the demon theory is untestable. Now, if the demon theory says that the demon can be detected with x-rays, then there is something the theory predicts other than the clock’s stopping when touched. You can x-ray the clock and examine the film for demon silhouettes. If the theory says that the demon can’t be seen but can be heard with sensitive sound equipment, then you have a prediction, something to look for other than clock stoppage. So other things being equal, testable theories are superior to untestable ones; they may be able to increase our understanding of a phenomenon. But an untestable theory is just an oddity. Fruitfulness

Imagine that we have two testable theories, theory 1 and theory 2, that attempt to explain the same phenomenon. Theory 1 and theory 2 seem comparable in most respects when measured against the criteria of adequacy. Theory 1, however, successfully predicts the existence of a previously unknown entity, say, a star in an uncharted part of the sky. What would you conclude about the relative worth of these two theories? If you thought carefully about the issue, you would probably conclude that theory 1 is the better theory—and you would be right. Other things being equal, theories that perform this way—that successfully predict previously unknown phenomena—are more credible than those that don’t. They are said to be fruitful, to yield new insights that can open up whole new areas of research and discovery. This fruitfulness suggests that the theories are more likely to be true. If a friend of yours is walking through a forest where she has never been before, yet she seems to be able to predict exactly what’s up ahead, you would probably conclude that she possessed some kind of accurate information about the forest, such as a map. Likewise, if a theory successfully predicts some surprising state of affairs, you are likely to think that the predictions are not just lucky guesses. All empirical theories are testable (they predict something beyond the thing to be explained). But fruitful theories are testable and then some. They not only predict something, they predict something that no one expected. The element of surprise is hard to ignore. Decades ago Einstein’s theory of relativity gained a great deal of credibility by successfully predicting a phenomenon that was extraordinary and entirely novel. The theory predicts that light traveling close to massive objects (such as stars) will appear to be bent because the space around such objects is curved. The curve in space causes a curve in nearby light rays. At the time, however, the prevailing opinion was that light always travels in straight lines—no bends, no curves, no breaks. In 1919 the physicist Sir Arthur Eddington devised a way to test this prediction. He managed to take two sets of photographs of exactly the same portion of the sky—when the sun was overhead (in daylight) and when it was not (at night). He was able to get a good photo of the sky during daylight because there was a total eclipse of the sun at the time. If light rays really were bent when they passed near massive objects, then stars whose light passes near the sun should appear to be shifted slightly from their true position (as seen at night). Eddington discovered that stars near the sun did appear to have moved and that the amount of their apparent movement was just what the theory predicted. This novel prediction

then demonstrated the fruitfulness of Einstein’s theory, provided a degree of confirmation for the theory, and opened up new areas of research. So the moral is that other things being equal, fruitful theories are superior to those that aren’t fruitful. Certainly many good theories make no novel predictions but are accepted nonetheless. The reason is usually that they excel in other criteria of adequacy. Scope

Suppose theory 1 and theory 2 are two equally plausible theories to explain phenomenon X. Theory 1 can explain X well, and so can theory 2. But theory 1 can explain or predict only X, whereas theory 2 can explain or predict X—as well as phenomena Y and Z. Which is the better theory? We must conclude that theory 2 is better because it explains more diverse phenomena. That is, it has more scope than the other theory. The more a theory explains or predicts, the more it extends our understanding. And the more a theory explains or predicts, the less likely it is to be false because it has more evidence in its favor. A major strength of Newton’s theory of gravity and motion, for example, was that it explained more than any previous theory. Then came Einstein’s theory of relativity. It could explain everything that Newton’s theory could explain plus many phenomena that Newton’s theory could not explain. This increased scope of Einstein’s theory helped convince scientists that it was the better theory. Here’s a more down-to-earth example. For decades psychologists have known about a phenomenon called constructive perception (discussed in Chapter 7). In constructive perception what we perceive (see, hear, feel, etc.) is determined in part by what we expect, know, or believe. Studies have shown that when people expect to perceive a certain stimulus (say, a flashing light, a certain color or shape, a shadow), they often do perceive it, even if there is no stimulus present. The phenomenon of constructive perception then can be used to explain many instances in which people seem to perceive something when it is not really there or when it is actually very different from the way people think it is. One kind of case that investigators sometimes explain as an instance of constructive perception is the UFO sighting. Many times people report seeing lights in the night sky that look to them like alien spacecraft, and they explain their perception by saying that the lights were caused by alien spacecraft. So we have two theories to explain the experience: constructive perception and UFOs from space. If these two theories differ only in the degree of scope provided by each one, however, we must conclude that the constructive-perception theory is better. (In reality, theories about incredible events usually differ on several criteria.) The constructive-perception theory can explain not only UFO sightings but all kinds of ordinary and extraordinary experiences—hallucinations, feelings of an unknown “presence,” misidentification of crime suspects, contradictory reports in car accidents, and more. The UFO theory, however, is (usually) designed to explain just one thing: an experience of seeing strange lights in the sky. Scope is often a crucial factor in a jury’s evaluation of theories put forth by both the prosecution and the defense. The prosecution will have a very powerful case against the defendant if the prosecutor’s theory (that the defendant did it) explains all the evidence and many other things while the defense theory (innocence) does not. The defendant would be in big trouble if the prosecutor’s theory explains the blood on the defendant’s shirt, the eyewitness accounts, the

defendant’s fingerprints on the wall, and the sudden change in his usual routine—and the innocence theory renders these facts downright mysterious. Other things being equal, then, the best theory is the one with the greatest scope. And if other things aren’t equal, a theory with superior scope doesn’t necessarily win the day because it may do poorly on the other criteria—or another theory might do better. Simplicity

Suppose you want to explain why your car didn’t start this morning, and you consider the theory that the car’s battery is dead (theory 1), along with four other theories: Theory 2: Each night, you are sabotaging your own car while you sleepwalk. Theory 3: Your 90-year-old uncle, who lives a thousand miles away from you, has secretly been going for joyrides in your car, damaging the engine. Theory 4: A poltergeist (a noisy, mischievous ghost) has damaged the car’s carburetor. Theory 5: Yesterday, you accidentally drove the car through an alternative space-time dimension, scrambling the electrical system. By now you probably suspect that these explanations are somehow unacceptable, and so they are. One important characteristic that they each lack is simplicity. Other things being equal, the best theory is the one that is the simplest—that is, the one that makes the fewest assumptions. The theory making the fewest assumptions is less likely to be false because there are fewer ways for it to go wrong. Another way to look at it is that since a simpler theory is based on fewer assumptions, less evidence is required to support it. Theories 4 and 5 lack simplicity because they each must assume the existence of an unknown entity (poltergeists and another dimension that scrambles electrical circuits). Such assumptions about the existence of unknown objects, forces, and dimensions are common in occult or paranormal theories. Theories 2 and 3 assume no new entities, but they do assume complex chains of events. This alone makes them less plausible than theory 1, the dead battery explanation. The criterion of simplicity has often been a major factor in the acceptance or rejection of important theories. For example, scientists eventually accepted Copernicus’s theory of planetary motion (heliocentric orbits) over Ptolemy’s (Earth-centered orbits) because the former was simpler (see the next section). In order to account for apparent irregularities in the movement of certain planets, Ptolemy’s theory had to assume that planets have extremely complex orbits (orbits within orbits). Copernicus’s theory, however, had no need for so much extra baggage. His theory could account for the observational data without so many orbits-within-orbits. Sometimes a theory’s lack of simplicity is the result of constructing ad hoc hypotheses. An ad hoc hypothesis is one that cannot be verified independently of the phenomenon it’s supposed to explain. If a theory is in trouble because it is not matching up with the observational data of the phenomenon, you might be able to rescue it by altering it—by positing additional entities or properties that can account for the data. Such tinkering is legitimate (scientists do it all the time) if there is an independent way of confirming the existence of these proposed entities and

properties. But if there is no way to verify their existence, the modifications are ad hoc hypotheses. Ad hoc hypotheses always make a theory less simple—and therefore less credible. Conservatism

What if a trusted friend told you that—believe it or not—some dogs lay eggs just as chickens do? Let’s assume that your friend is being perfectly serious and believes what she is saying. Would you accept this claim about egg-laying dogs? Not likely. But why not? Probably your main reason for rejecting such an extraordinary claim would be that it fails the criterion of conservatism, though you probably wouldn’t state it that way. (Note: This sense of “conservatism” has nothing to do with political parties.) This criterion says that other things being equal, the best theory is the one that fits best with our well-established beliefs—that is, with beliefs backed by excellent evidence or very good arguments. We would reject the canine-egg theory because, among other things, it conflicts with our well-founded beliefs about mammals, evolution, canine anatomy, and much more. Humans have an enormous amount of experience with dogs (scientific and otherwise), and none of it suggests that dogs can lay eggs. In fact, a great deal of what we know about dogs suggests that they cannot lay eggs. To accept the canine-egg theory despite its conflicting with a mountain of solid evidence would be irrational—and destructive of whatever understanding we had of the subject. Perhaps one day we may be shocked to learn that—contrary to all expectations and overwhelming evidence—dogs do lay eggs. But given that this belief is contrary to a massive amount of credible experience, we must assign a very low probability to it. What kind of beliefs fall into the category of “well-established” knowledge? For starters, we can count beliefs based on our own everyday observations that we have no good reasons to doubt (such as “it’s raining outside,” “the parking lot is empty,” and “the train is running late today”). We can include basic facts about the world drawn from excellent authority (“Earth is round,” “men have walked on the moon,” and “Cairo is the capital of Egypt”). And we can include a vast array of beliefs solidly supported by scientific evidence, facts recognized as such by most scientists (“cigarettes cause lung cancer,” “vaccines prevent disease,” “dinosaurs existed,” and “germs cause infection”). Many of our beliefs, however, cannot be regarded as well established. Among these, of course, are all those we have good reasons to doubt. But there is also a large assortment of beliefs that occupy the middle ground between those we doubt and those we have excellent reasons to believe. We may have some reasons in favor of these beliefs, but those reasons are not so strong that we can regard the beliefs as solid facts. We can only proportion our belief to the evidence and be open to the possibility that we may be wrong. Very often such claims reside in areas that are marked by controversy—politics, religion, ethics, economics, and more. Among these notions, we must walk cautiously, avoid dogmatism, and follow the evidence as best we can. We should not assume that the claims we have absorbed from our upbringing and culture are beyond question. That being said, there are good reasons for respecting the criterion of conservatism, properly understood. We are naturally reluctant to accept explanations that conflict with what we already know, and we should be. Accepting beliefs that fly in the face of our well-supported knowledge has several risks: 1.

The chances of the new belief being true are not good (because it has no evidence in its favor, while our well-established beliefs have plenty of evidence on their side). 2. The conflict of beliefs undermines our knowledge (because we cannot know something that is in doubt, and the conflict would be cause for doubt). 3. The conflict of beliefs lessens our understanding (because the new beliefs cannot be plausibly integrated into our other beliefs). So everything considered, the more conservative a theory is, the more plausible it is.4 Here’s another example. Let’s say that someone claims to have built a perpetual motion machine. This type of machine is supposed to function without ever stopping and without requiring any energy input from outside the machine; it is designed to continuously supply its own energy. Now, this is an intriguing idea—that we shouldn’t take too seriously. The problem is that the notion of a perpetual motion machine is not conservative at all. It conflicts with a very well-established belief—namely, one of the scientific laws of thermodynamics. The law of conservation of mass-energy says that mass-energy cannot be created or destroyed. A perpetual motion machine, though, would have to create energy out of nothing. Like any law of nature, however, the law of conservation of mass-energy is supported by a vast amount of empirical evidence. We must conclude, then, that it is extremely unlikely that anyone could escape the law of conservation of mass-energy through the use of any machine. (This fact, however, has not stopped countless optimistic inventers from claiming that they’ve invented such devices. When the devices are put to the test, they invariably fail to perform as advertised.) It’s possible, of course, that a new theory that conflicts with what we know could turn out to be right and a more conservative theory wrong. But we would need good reasons to show that the new theory was correct before we would be justified in tossing out the old theory and bringing in the new. Science looks for conservative theories, but it still sometimes embraces theories that are departures (sometimes radical departures) from the well-worn, accepted explanations. When this dramatic change happens, it’s frequently because other criteria of adequacy outweigh conservatism. We will look at other examples shortly, but before going further, we need to understand two crucial points about the nature of theory appraisal. First, there is no strict formula or protocol for applying the criteria of adequacy. In deductive arguments there are rules of inference that are precise and invariable. But inference to the best explanation is a different animal. There are no precise rules for applying the criteria, no way to quantify how a theory measures up according to each criterion, and no way to rank each criterion according to its importance. Sometimes we may assign more weight to the criterion of scope if the theory in question seems comparable to other theories in terms of all the remaining criteria. Other times we may weight simplicity more when considering theories that seem equally conservative or fruitful. The process of theory evaluation is not like solving a math problem—but more like diagnosing an illness or making a judicial decision. It is rational but not formulaic, and it depends on the dynamics of human judgment. The best we can do is follow some guidelines

for evaluating theories generally and for applying the criteria of adequacy. Fortunately, this kind of help is usually all we need. Second, despite the lack of formula in theory assessment, the process is far from subjective or arbitrary. There are many distinctions that we successfully make every day that are not quantifiable or formulaic—but they are still objective. We cannot say exactly when day turns into night or when a person with a full head of hair becomes bald or when a puddle in the rain becomes a pond, but our distinctions between night and day or baldness and hirsuteness or puddles and ponds are clearly objective. Of course, there are cases that are not so clear-cut that give rise to reasonable disagreement among reasonable people. But there are also many instances that are manifestly unambiguous. Pretending that these states of affairs are unclear would be irrational. It would simply be incorrect to believe that broad daylight is nighttime or that a puddle is a pond. What’s Wrong with Conspiracy Theories? Conspiracy theories try to explain events by positing the secret participation of numerous conspirators. The assassination of JFK, the terrorist attacks of 9/11, the death of Elvis Presley, the UFO crash at Roswell, the Great Recession of 2008, the NASA moon landings, the bloodline of Jesus Christ—all these and more have been the subject of countless conspiracy theories, both elaborate and provocative. Online they are everywhere. Some conspiracy theories, of course, have been found to be true after all. But most of them are implausible or absurd. The main problem with them is that they fail the criteria of adequacy, especially the criterion of simplicity. They would have us make numerous assumptions that raise more questions than they answer: How do the conspirators manage to keep their activities secret? How do they control all the players? Where is the evidence that all the parts of the conspiracy have come together just so? As we have seen, for any set of facts, it is shockingly easy to devise a theory that fits them, but this fit alone cannot establish the truth of the theory. That’s why we must apply the criteria of adequacy to sort out the plausible theories from the implausible. Very often the set of alleged facts that are supposed to back up the theory are not facts at all. They are unsupported assertions coughed up by a fevered social media and red-hot political conflicts. For those who eagerly believe them, conspiracy theories are what scientists call nonfalsifiable hypotheses: There is no possible evidence that believers would accept as counting against their beloved theory. Every piece of counterevidence is dismissed, ignored, chalked up to a government cover-up, or interpreted as actually confirming the theory. There is no escape from this prison of the mind except through critical thinking—through a fair weighing of the evidence, a careful consideration of alternative theories, and an unwillingness to believe without good reason. Section Query

SECTION QUERY Have you ever accepted a conspiracy theory? If so, how well did it pass the criterion of simplicity? To you, was the theory nonfalsifiable?

8.4 Telling Good Theories from Bad

Many (perhaps most) theories that you run into every day are easy to assess. They are clearly the best (or not the best) explanations for the facts at hand. The dog barked because someone approached the house. Your friend blushed because he was embarrassed. The senator resigned because of a scandal. In such cases, you may make inferences to the best explanation (using some or all of the criteria of adequacy) without any deep reflection. But at other times, you may need and want to be more deliberate, to think more carefully about which explanation is really best. In either case, it helps to have a set of guidelines that tells you how your inquiry should proceed if you’re to make cogent inferences. Here, then, is the TEST formula, four steps to finding the best explanation: Step 1. State the Theory and check for consistency. Step 2. Assess the Evidence for the theory. Step 3. Scrutinize alternative theories. Step 4. Test the theories with the criteria of adequacy. Step 1. State the theory and check for consistency. Before you can evaluate a theory, you must express it in a statement that’s as clear and specific as possible. Once you do this, you can check to see if the theory meets the minimum requirement for consistency. If it fails the consistency test, you can have no good grounds for believing that it’s correct. And, obviously, if the theory fails step 1, there’s no reason to go to step 2. Step 2. Assess the evidence for the theory. To critically evaluate any theory, you must understand any reasons in its favor—the empirical evidence or logical arguments that may support or undermine it. Essentially, this step involves an honest assessment of the empirical evidence relevant to the truth (or falsity) of the theory. To make this assessment, you must put to use what you already know about the credibility of sources, logical argument, and evidence from personal and scientific observations (topics covered in Chapters 2 and 7). In this step, you may discover that the evidence in favor of a theory is strong, weak, or nonexistent. You may find that there is good evidence that seems to count against the theory. Or you may learn that the phenomenon under investigation did not occur at all. Whatever the case, you must have the courage to face up to reality. You must be ready to admit that your favorite theory has little to recommend it. Step 3. Scrutinize alternative theories. Inference to the best explanation will not help us very much if we aren’t willing to consider alternative explanations. Simply examining the evidence relevant to an eligible theory is not enough. To get to the truth, we must abandon motivated reasoning. Theories can often appear stronger than they really are if we don’t bother to compare them with others. To take an outrageous example, consider this theory designed to explain the popularity and seeming omnipresence of an American icon: Mickey Mouse is not an animated character but a living, breathing creature that lives in Hollywood. The evidence for this explanation is the following: (1) Millions of people (mostly children) worldwide believe that Mickey is real; (2) Walt Disney (Mickey’s alleged creator) always talked about Mickey as if the mouse was real; (3)

millions of ads, books, movies, and TV shows portray Mickey as real; (4) it’s possible that through millions of years of Earth’s history a biological creature with Mickey’s physical characteristics could have evolved; and (5) some say that if enough people believe that Mickey is real, then—through psychic wish fulfillment or some other paranormal process—he will become real. Now, you don’t believe that Mickey is real (do you?), even in the face of reasons 1–5. But you might admit that the Mickey theory is starting to sound more plausible. And if you never hear any alternative explanations—and in motivated reasoning, you never hear any alternative explanations—you might eventually become a true believer. (Anthropologists can plausibly argue that various cultures have come to believe in many very unlikely phenomena and exotic deities in large part because of a lack of alternative explanations.) When you do consider an alternative explanation—for example, that Mickey is an imaginary character of brilliant animation marketed relentlessly to the world—the Mickey-is-real theory looks ridiculous. And once you consider the evidence for this alternative theory (for example, documentation that Walt Disney created Mickey with pen and ink and that countless marketing campaigns have been launched to promote his creation), the other explanation looks even sillier. Step 3 requires us to have an open mind, to think outside the box, to ask if there are other ways to explain the phenomenon in question and to consider the evidence for those theories. Specifically, in this step we must conscientiously look for competing theories and then apply both step 1 and step 2 to each one of them. This process may leave us with many or few eligible theories to examine. In any case, it’s sure to tell us something important about the strength or weakness of competing theories. Many times the criteria of adequacy can help us do a preliminary assessment of a theory’s plausibility without our surveying alternative theories. For example, a theory may do so poorly regarding a particular criterion that we can conclude that, whatever the merits of alternative explanations, the theory at hand is not very credible. Such a clear lack of credibility is often apparent when a theory is obviously neither simple nor conservative. Skipping step 3 is an extremely common error in the evaluation of explanations of all kinds. It is a supreme example of many types of errors discussed in earlier chapters—overlooking evidence, preferring available evidence, looking only for confirming evidence, and denying the evidence. Step 3 goes against our grain. The human tendency is to grab hold of a favorite theory—and to halt any further critical thinking right there. Our built-in bias is to seize on a theory immediately—because we find it comforting or because we just “know” it’s the right one—and then ignore or resist all other possibilities. The result is a greatly increased likelihood of error and delusion and a significantly decreased opportunity to achieve true understanding. Step 4. Test the theories with the criteria of adequacy. As we’ve seen, simply toting up the evidence for each of the competing theories and checking to see which one gets the highest score will not do. We need to measure the plausibility of the theories using the criteria of adequacy. The criteria can help us put any applicable evidence in perspective and allow us to make a judgment about theory plausibility even when there’s little or no evidence to consider.

By applying the criteria to all the competing theories, we can often accomplish several important feats. We may be able to eliminate some theories immediately, assign more weight to some than others, and distinguish between theories that at first glance seem equally strong. The best way to learn how to do step 4, as well as steps 1–3, is by example. Watch what happens when we assess the plausibility of the following theories—Copernican planetary motion, evolution, and climate change—using the TEST formula. Copernicus versus Ptolemy

Consider the historic clash between the geocentric (Earth-centered) and the heliocentric (sun-centered) theories of planetary motion. It’s difficult to imagine two rival theories that have more profoundly influenced how humanity views itself and its place in the universe. In the beginning was the geocentric view. Aristotle got things going by putting forth the theory that a spherical Earth was at the center of a spherical universe consisting of a series of concentric, transparent spheres. On one celestial sphere we see the sun, the moon, and the known planets. On the outermost sphere we behold the stars. All the heavenly bodies rotate in perfect circles around the stationary Earth. The heavenly bodies are pure, incorruptible, and unchanging; Earth is impure, corruptible, and transient. Then came the great astronomer and mathematician Ptolemy, who flourished in Alexandria between 127 and 148 ce. He discovered inconsistencies in the traditional geocentric system between the predicted and observed motions of the planets. He found, in other words, that Aristotle’s theory was not conservative, a crucial failing. So he fine-tuned the old view, adding little circular motions (called epicycles) along the planet orbits and many other minor adjustments. He also allowed for an odd asymmetry in which the center of planet orbits was not exactly the center of Earth—all this so the theory would match up to astronomical observations. By the time Ptolemy finished tinkering, he had posited 80 circles and epicycles—80 different planetary motions—to explain the movements of the sun, moon, and five known planets. The result was a system far more complex than Aristotle’s was. But the revised theory worked well enough for the times, and it agreed better than the earlier theory did with observational data. Despite the complications, learned people could use Ptolemy’s system to calculate the positions of the planets with enough accuracy to effectively manage calendars and astrological charts. So for 15 centuries, astronomers used Ptolemy’s unwieldy, complex theory to predict celestial events and locations. In the West, at least, Earth stood still in the center of everything as the rest of the universe circled around it. The chief virtue of the Ptolemaic system, then, was conservatism. It fit, mostly, with what astronomers knew about celestial goings-on. It was also testable, as any scientific theory should be. Its biggest failing was simplicity—or the lack thereof. The theory was propped up by numerous assumptions for the purpose of making the theory fit the data. Enter Nicolaus Copernicus (1473–1543). He was disturbed by the complexity of Ptolemy’s system. It was a far cry from the simple theory that Aristotle bequeathed to the West. Copernicus proposed a heliocentric theory in which Earth and the other planets orbit the sun, the true center of the universe. In doing so, he greatly simplified both the picture of the heavens and the calculations required to predict the positions of planets. Copernicus’s theory was simpler than Ptolemy’s on many counts, but one of the most impressive was retrograde motion, a phenomenon that had stumped astronomers for centuries.

From time to time, certain planets seem to reverse their customary direction of travel across the skies—to move backward! Ptolemy explained this retrograde motion by positing yet more epicycles, asserting that planets orbiting Earth will often orbit around a point on the larger orbital path. Seeing these orbits within orbits from Earth, an observer would naturally see the planets sometimes backing up. But the Copernican theory could easily explain retrograde motion without all those complicated epicycles. As the outer planets (Mars, Jupiter, Saturn) orbit the sun, so does Earth, one of the inner planets. The outer planets, though, move much slower than Earth does. On its own orbital track, Earth sometimes passes the outer planets as they lumber along on their orbital track, just as a train passes a slower train on a parallel track. When this happens, the planets appear to move backward, just as the slower train seems to reverse course when the faster train overtakes it. Copernicus’s theory, however, was not superior on every count. It explained a great many astronomical observations, but Ptolemy’s theory did too, so they were about even in scope. It had no big advantage in fruitfulness over the Ptolemaic system. It made no impressive predictions of unknown phenomena. Much more troubling, it seemed to conflict with some observational data. One test implication of the Copernican theory is the phenomenon known as parallax. Critics of the heliocentric view claimed that if the theory were true, then as Earth moved through its orbit, stars closest to it should seem to shift their position relative to stars farther away. There should, in other words, be parallax. But no one had observed parallax. Copernicus and his followers responded to this criticism by saying that stars were too far away for parallax to occur. As it turned out, they were right about this, but confirmation didn’t come until 1832 when parallax was observed with more powerful telescopes. Another test implication seemed to conflict with the heliocentric model. Copernicus reasoned that if the planets rotate around the sun, then they should show phases just as the moon shows phases due to the light of the sun falling on it at different times. But in Copernicus’s day, no one could see any such planetary phases. Fifty years later, though, Galileo used his new telescope to confirm that Venus had phases. Ultimately, scientists accepted the Copernican model over Ptolemy’s because of its simplicity—despite what seemed at the time like evidence against the theory. As Copernicus said, “I think it is easier to believe this [sun-centered view] than to confuse the issue by assuming a vast number of Spheres, which those who keep the Earth at the center must do.”5 Evolution versus Creationism

Few scientific theories have been more hotly debated among nonscientists than evolution and its rival, creationism (or creation science). Both theories purport to explain the origin and existence of biological life on Earth, and each claims to be a better explanation than the other. Can science decide this contest? Yes. Despite the complexity of the issues involved and the mixing of religious themes with the nonreligious, good science can figure out which theory is best. Remember that the best theory is the one that explains the phenomenon and measures up to the criteria of adequacy better than any of its competitors. There is no reason that the scientific approach cannot provide an answer here—even in this thorniest of thorny issues.

Neither the term “evolution” nor the concept began with Charles Darwin (1809–1882), the father of evolutionary theory. The word showed up in English as early as 1647. The ancient Greek philosopher Anaximander (c. 611–547 bce) was actually the first evolutionary theorist, inferring from some simple observations that humans must have evolved from an animal and that this evolution must have begun in the sea. But in his famous book On the Origin of Species (1859), Darwin distilled the theory of evolution into its most influential statement. Scientists have been fine-tuning the theory ever since, as new evidence and new insights pour in from many different fields, such as biochemistry and genetics. But the basic idea has not changed: Living organisms adapt to their environments through inherited characteristics; this adaptation results in changes in succeeding generations. Specifically, the offspring of organisms differ physically from their parents in various ways, and these differences can be passed on genetically to their offspring. If an offspring has an inherited trait (such as sharper vision or a larger brain) that increases its chances of surviving long enough to reproduce, the individual is more likely to survive and pass the trait on to the next generation. After several generations, this useful trait, or adaptation, spreads throughout a whole population of individuals, differentiating the population from its ancestors. Natural selection is the name that Darwin gave to this process. Creation science, on the other hand, maintains that (1) the universe and all life was created suddenly, out of nothing, only a few thousand years ago (6000 to 10,000 is the usual range); (2) natural selection could not have produced living things from a single organism; (3) species change very little over time; (4) man and apes have a separate ancestry; and (5) Earth’s geology can be explained by catastrophism, including a worldwide flood.6 The first thing we should ask about these two theories is whether they’re testable. The answer is yes. Recall that a theory is testable if it predicts or explains something other than what it was introduced to explain. On this criterion, evolution is surely testable. It explains, among other things, why bacteria develop resistance to antibiotics, why there are so many similarities between humans and other primates, why new infectious diseases emerge, why the chromosomes of closely related species are so similar, why the fossil record shows the peculiar progression of fossils that it does, and why the embryos of related species have such similar structure and appearance. Creationism is also testable. It, too, explains something other than what it was introduced to explain. It claims that Earth’s geology was changed in a worldwide flood, that the universe is only a few thousand years old, that all species were created at the same time, and that species change very little over time. Innumerable test implications have been derived from evolutionary theory, and innumerable experiments have been conducted, confirming the theory. For example, if evolution is true, then we would expect to see systematic change in the fossil record from simple creatures at the earlier levels to more complex individuals at the more recent levels. We would expect not to see a reversal of this configuration. And this sequence is exactly what scientists see time and time again. Creationism, however, has not fared as well. Its claims have not been borne out by evidence. In fact, they have consistently conflicted with well-established scientific findings.

This latter point means that creationism fails the criterion of conservatism—it conflicts with what we already know. For example, the scientific evidence shows that Earth is not 6000 to 10,000 years old—but billions of years old. According to the National Academy of Sciences, There are no valid scientific data or calculations to substantiate the belief that Earth was created just a few thousand years ago. [There is a] vast amount of evidence for the great age of the universe, our galaxy, the Solar system, and Earth from astronomy, astrophysics, nuclear physics, geology, geochemistry, and geophysics. Independent scientific methods consistently give an age for Earth and the Solar system of about 5 billion years, and an age for our galaxy and the universe that is two to three times greater.7 Creationism also fails the criterion of conservatism on the issue of a geology-transforming universal flood: Nor is there any evidence that the entire geological record, with its orderly succession of fossils, is the product of a single universal flood that occurred a few thousand years ago, lasted a little longer than a year, and covered the highest mountains to a depth of several meters. On the contrary, intertidal and terrestrial deposits demonstrate that at no recorded time in the past has the entire planet been under water. … The belief that Earth’s sediments, with their fossils, were deposited in an orderly sequence in a year’s time defies all geological observations and physical principles concerning sedimentation rates and possible quantities of suspended solid matter.8 Has either theory yielded any novel predictions? Evolution has. It has predicted, for example, that new species should still be evolving today; that the fossil record should show a movement from older, simpler organisms to younger, more complex ones; that proteins and chromosomes of related species should be similar; and that organisms should adapt to changing environments. These and many other novel predictions have been confirmed. Creationism has made some novel claims, as we saw earlier, but none of these have been supported by good evidence. Creationism is not a fruitful theory. The criterion of simplicity also draws a sharp contrast between the two theories. Simplicity is a measure of the number of assumptions that a theory makes. Both theories make assumptions, but creationism assumes much more. Creationism assumes the existence of a creator and unknown forces. Proponents of creationism readily admit that we do not know how the creator created nor what creative processes were used. In this contest of theories, the criterion of scope—the amount of diverse phenomena explained—is probably more telling than any of the others. Biological evolution explains a vast array of phenomena in many fields of science. In fact, a great deal of the content of numerous scientific fields—genetics, physiology, biochemistry, neurobiology, and more—would be deeply perplexing without the theory of evolution. As the eminent geneticist Theodosius Dobzhansky put it, “Nothing in biology makes sense except in the light of evolution.”10 Can We See Evolution? Critics of the theory of evolution often ask, “If evolution occurs, why can’t we see it?” Here’s how the National Academy of Sciences responds to this objection: Scientific conclusions are not limited to direct observation but often depend on inferences that are made by applying reason to observations. Even with the launch of Earth-orbiting spacecraft, scientists could not directly see the Earth going around the Sun. But they inferred from a wealth of independent measurements that the Sun is at the center of the solar system. Until the recent development of extremely powerful microscopes, scientists could not observe atoms, but the

behavior of physical objects left no doubt about the atomic nature of matter. Scientists hypothesized the existence of viruses for many years before microscopes became powerful enough to see them. Thus, for many areas of science, scientists have not directly observed the objects (such as genes and atoms) or the phenomena (such as the Earth going around the Sun) that are now well-established facts. Instead, they have confirmed them indirectly by observational and experimental evidence. Evolution is no different.… This contention that nobody has seen evolution occurring further ignores the overwhelming evidence that evolution has taken place and is continuing to occur. The annual changes in influenza viruses and the emergence of bacteria resistant to antibiotics are both products of evolutionary forces. Another example of ongoing evolution is the appearance of mosquitoes resistant to various insecticides, which has contributed to a resurgence of malaria in Africa and elsewhere. The transitional fossils that have been found in abundance since Darwin’s time reveal how species continually give rise to successor species that, over time, produce radically changed body forms and functions. It also is possible to directly observe many of the specific processes by which evolution occurs. Scientists regularly do experiments using microbes and other model systems that directly test evolutionary hypotheses.9 Virtually all scientists would agree—and go much further: It helps to explain the emergence of new infectious diseases, the development of antibiotic resistance in bacteria, the agricultural relationships among wild and domestic plants and animals, the composition of Earth’s atmosphere, the molecular machinery of the cell, the similarities between human beings and other primates, and countless other features of the biological and physical world.11 And Evolution provides a scientific explanation for why there are so many different kinds of organisms on Earth and how all organisms on this planet are part of an evolutionary lineage. It demonstrates why some organisms that look quite different are in fact related, while other organisms that may look similar are only distantly related. It accounts for the appearance of humans on Earth and reveals our species’ biological connections with other living things. It details how different groups of humans are related to each other and how we acquired many of our traits. It enables the development of effective new ways to protect ourselves against constantly evolving bacteria and viruses.12 Creationism, however, can explain none of this. And it provokes, not solves, innumerable mysteries: What caused the worldwide flood? Where did all that water come from? Where did it all go? Why does Earth seem so ancient (when it’s said to be so young)? How did the creator create the entire universe suddenly—out of nothing? Why does the fossil record seem to suggest evolution and not creation? So many questions are an indication of diminished scope and decreased understanding. Good scientists must be prepared to admit this much: If creationism meets the criteria of adequacy as well as evolution does, then creationism must be as good a theory as evolution. But creationism fails to measure up to the criteria of adequacy. On every count it shows itself to be inferior. Scientists, then, are justified in rejecting creationism in favor of evolution. And this is exactly what they do.

For some people, all this talk about science, criteria, and evidence is irrelevant. They reject evolution because they think it’s incompatible with religion. Many religious believers, religious scientists, and religious denominations, however, would strongly disagree. Science does prove that some religious beliefs are unfounded—like the notions that Earth is only a few thousand years old, that Earth underwent a worldwide flood, and that new species of living things cannot evolve over time. But millions of religious people—many Christians, Hindus, Muslims, Jews, and Buddhists—see no contradiction between evolution and their religious beliefs. Denominations that have accepted evolution include the Roman Catholic Church, the Presbyterian Church, the United Methodist Church, the United Church of Christ, the Episcopal Church, the Evangelical Lutheran Church of America, and others. Evolution and Intelligent Design A controversial view known as intelligent design (ID) is a common conceptual challenge to evolution, maintaining that biological life is much too complex to be fully explained by evolutionary processes. Some claim that life on Earth is best explained by the intervention of a supreme intelligence. Michael Behe, professor of biochemistry at Lehigh University, famously argues that some biological systems are so profoundly intricate—so “irreducibly complex”—that they could not have been produced by gradual evolutionary changes. Only an intelligent designer can account for such complexity. He says that an irreducibly complex system (for example, the eye) is composed of several interconnected, perfectly matched parts such that if even one part is missing, the system will not function. An eye can improve the survival prospects of organisms only if it functions, and proper functioning requires that each of its parts is there to do its job. According to evolution, the eye came about through slow, incremental changes. But, Behe asks, how can an unfinished, nonfunctioning eye improve survival? This shows, he argues, that the eye and all other irreducibly complex systems were created whole—not through evolution, but by some great intelligence. But most biologists deny that the development of irreducibly complex systems through natural selection is physically impossible. Behe thinks natural selection requires that a complex system be formed by gradual addition of components until a functioning model is achieved. But critics point out that the components can be present all along or arise at different times, performing tasks that improve various processes. Then, because of a change in the genome, the parts may be put to new uses, forming an irreducibly complex structure. For example, Evolutionary biologists also have demonstrated how complex biochemical mechanisms, such as the clotting of blood or the mammalian immune system, could have evolved from simpler precursor systems. With the clotting of blood, some of the components of the mammalian system were present in earlier organisms, as demonstrated by the organisms living today (such as fish, reptiles, and birds) that are descended from these mammalian precursors. Mammalian clotting systems have built on these earlier components. Existing systems also can acquire new functions. For example, a particular system might have one task in a cell and then become adapted through evolutionary processes for different use.13 From the fact that biologists generally do not know precisely how each step of such a process happens, it does not follow that the process is impossible or unknowable. Philip Kitcher, professor of philosophy at Columbia, thinks that the remedy for our ignorance of these matters is more and better research, not the presumption of an intelligent designer:

Even if intelligent designers were right in supposing that the phenomena they indicate couldn’t have evolved by natural selection, only a more explicit identification of the causal mechanism that was at work could justify the conclusion that that mechanism is intelligent.14 A letter signed by more than 15,000 Christian clergy members, compiled by the Clergy Letter Project, declares, We the undersigned, Christian clergy from many different traditions, believe that the timeless truths of the Bible and the discoveries of modern science may comfortably coexist. We believe that the theory of evolution is a foundational scientific truth, one that has stood up to rigorous scrutiny and upon which much of human knowledge and achievement rests. To reject this truth or to treat it as “one theory among others” is to deliberately embrace scientific ignorance and transmit such ignorance to our children. We believe that among God’s good gifts are human minds capable of critical thought and that the failure to fully employ this gift is a rejection of the will of our Creator.15 Some religious scientists hold similar views. For example, Francis Collins, director of the Human Genome Project and of the National Human Genome Research Institute at the National Institutes of Health, says, In my view, there is no conflict in being a rigorous scientist and a person who believes in a God who takes a personal interest in each one of us. Science’s domain is to explore nature. God’s domain is in the spiritual world, a realm not possible to explore with the tools and language of science. It must be examined with the heart, the mind, and the soul.16 Climate Change

The first thing to understand about the empirical question of climate change—whether it’s happening and, if so, why—is that it’s a scientific question. What people should do about it is a moral, political, or policy question. An elementary mistake in critical thinking comes from letting your views on the latter dictate your assessment of the former. A larger difficulty in sorting out the facts is that much of the media coverage does not do the science justice. Matthew C. Nisbet, professor of communication, public policy, and urban affairs at Northeastern University and editor-in-chief of the journal Environmental Communication, explains the problem like this: At the opinion-leading legacy print publications such as The Guardian or Washington Post, and at newer digital-native outlets such as HuffPost (formerly The Huffington Post) or Buzzfeed, the challenge in most instances is not the amount of coverage but how the risks and solutions to climate change are characterized. Studies conducted by social scientists in the United States and Europe using statistical techniques to rigorously evaluate hundreds of news stories show that journalists frequently gloss over the uncertainties and caveats inherent in a single study or line of climate change research, neglect to report on the varying predictions offered by different climate models, and fail to include in their reporting the careful language that the Intergovernmental Panel on Climate Change (IPCC) has developed to qualify the likelihood of various consequences of climate change. In coverage of major climate change-related events such as a new IPCC report or United Nations summit, journalists also tend to dramatize their significance by emphasizing the most calamitous future climate change scenarios, framing a new scientific report’s findings in terms of disastrous and fear-inducing risks, rather than emphasizing in the face of those risks opportunities to protect health or sustainably grow

economies. Reviewing available studies, the German journalism researcher Michael Bruggeman concludes that reporting too often “simplifies science and turns context-dependent and preliminary findings into established facts.”17 So to make headway in understanding climate change, we must proceed as we would when delving into any other politically inflamed, complex issue that science addresses. We must be especially alert to fake news and media bias, fallacious appeals to authority and unsupported claims, confirmation bias and evidence denial. And we must, of course, apply the criteria of adequacy to competing theories. For years, scientists throughout the world have been issuing unsettling reports and dire warnings about changes in Earth’s climate. The Climate Science Special Report (CSSR), one of many statements issued by scientific organizations worldwide, says, “Global annually averaged surface air temperature has increased by about 1.8 degrees Fahrenheit (1.0 degrees C) over the last 115 years (1901–2016). This period is now the warmest in the history of modern civilization.”18 Such a rise may seem small, but it can lead to huge, potentially cataclysmic effects worldwide. According to the Intergovernmental Panel on Climate Change (IPCC), a group of 1300 independent scientific experts worldwide, Each of the last three decades has been successively warmer at the Earth’s surface than any preceding decade since 1850. The period from 1983 to 2012 was likely the warmest 30-year period of the last 1400 years in the Northern Hemisphere, where such assessment is possible (medium confidence).19 Earth has always undergone natural eras of warming and cooling, but scientists say this current period of planet-heating is different. They assert that there is a greater than 95 percent probability that it is caused by human activity, and the changes are happening at an unprecedented rate. Climate scientists say the planet is heating up because of the greenhouse effect: The heat of solar radiation is being trapped in the lower atmosphere by so-called greenhouse gases. These include carbon dioxide (CO2), methane, and nitrous oxide, all of which have increased dramatically in the atmosphere since before the Industrial Revolution. Some of the increase comes from natural sources like volcanic eruptions, but most of it has been traced to human activity such as the burning of fossil fuels (coal, oil, and gas), as well as to deforestation and land use changes. The IPCC asserts,

FIG. 8.1 Despite ups and downs from year to year, global average surface temperature is rising. By the beginning of the 21st century, Earth’s temperature was roughly 0.5 degrees Celsius above the long-term (1951–1980) average. (NASA figure adapted from Goddard Institute for Space Studies Surface Temperature Analysis.) Anthropogenic [human-caused] greenhouse gas emissions have increased since the pre-industrial era, driven largely by economic and population growth, and are now higher than ever. This has led to atmospheric concentrations of carbon dioxide, methane and nitrous oxide that are unprecedented in at least the last 800,000 years. Their effects, together with those of other anthropogenic drivers, have been detected throughout the climate system and are extremely likely to have been the dominant cause of the observed warming since the mid-20th century.20

This global warming has already had powerful effects on the planet. These have been both positive and negative but, on balance, mostly negative. NASA declares in its report “Global Climate Change: Vital Signs of the Planet,” Glaciers have shrunk, ice on rivers and lakes is breaking up earlier, plant and animal ranges have shifted and trees are flowering sooner. Effects that scientists had predicted in the past would result from global climate change are now occurring: loss of sea ice, accelerated sea level rise and longer, more intense heat waves.21 According to the U.S. Global Change Research Program, there are many effects of climate change that are happening now in the United States and are likely to continue: Northeast. Heat waves, heavy downpours and sea level rise pose growing challenges to many aspects of life in the Northeast. Infrastructure, agriculture, fisheries and ecosystems will be increasingly compromised. Many states and cities are beginning to incorporate climate change into their planning. Northwest. Changes in the timing of streamflow reduce water supplies for competing demands. Sea level rise, erosion, inundation, risks to infrastructure and increasing ocean acidity pose major threats. Increasing wildfire, insect outbreaks and tree diseases are causing widespread tree die-off. Southeast. Sea level rise poses widespread and continuing threats to the region’s economy and environment. Extreme heat will affect health, energy, agriculture and more. Decreased water availability will have economic and environmental impacts. Midwest. Extreme heat, heavy downpours and flooding will affect infrastructure, health, agriculture, forestry, transportation, air and water quality, and more. Climate change will also exacerbate a range of risks to the Great Lakes. Southwest. Increased heat, drought and insect outbreaks, all linked to climate change, have increased wildfires. Declining water supplies, reduced agricultural yields, health impacts in cities due to heat, and flooding and erosion in coastal areas are additional concerns.22

FIG. 8.2 This graph, based on the comparison of atmospheric samples contained in ice cores and more recent direct measurements, provides evidence that atmospheric CO2 has increased since the Industrial Revolution. (Published by NASA. Credit: Luthi, D., et al., 2008; Etheridge, D.M., et al., 2010; Vostok ice core data/J.R. Petit et al.; NOAA Mauna Loa CO2 record.) Climate change and global warming are controversial mostly because of their political and policy implications. If the planet really is becoming overheated, and if the main cause of the warming is increased levels of greenhouse gases produced by the burning of fossil fuels, then it seems that the obvious solution is to implement major reductions in the burning of these fuels. But such reductions would disrupt or drastically alter the economic, commercial, and industrial systems that now run on the fuels—changes that many people strongly oppose. There is little doubt that climate change is happening, but there is considerable disagreement among many nonscientists about why it’s happening. So the question is, What is the best explanation of the climate-change data? Let’s consider a few of the more popular theories. Theory 1: Global warming is due to Earth’s natural cycles of warming and cooling over time and doesn’t have much to do with human activities. Evidence for this view comes from Earth’s climate history, which scientists have documented:

Earth has experienced climate change in the past without help from humanity. We know about past climates because of evidence left in tree rings, layers of ice in glaciers, ocean sediments, coral reefs, and layers of sedimentary rocks. For example, bubbles of air in glacial ice trap tiny samples of Earth’s atmosphere, giving scientists a history of greenhouse gases that stretches back more than 800,000 years. The chemical make-up of the ice provides clues to the average global temperature. Using this ancient evidence, scientists have built a record of Earth’s past climates, or “paleoclimates.” The paleoclimate record combined with global models shows past ice ages as well as periods even warmer than today.24 Is It Too Late to Prevent Climate Change? Here is NASA’s answer: Humans have caused major climate changes to happen already, and we have set in motion more changes still. Even if we stopped emitting greenhouse gases today, global warming would continue to happen for at least several more decades, if not centuries.… In the absence of major action to reduce emissions, global temperature is on track to rise by an average of 6 °C (10.8 °F), according to the latest estimates. Some scientists argue a “global disaster” is already unfolding at the poles of the planet; the Arctic, for example, may be ice-free at the end of the summer melt season within just a few years. Yet other experts are concerned about Earth passing one or more “tipping points”—abrupt, perhaps irreversible changes that tip our climate into a new state. But it may not be too late to avoid or limit some of the worst effects of climate change. Responding to climate change will involve a two-tier approach: 1) “mitigation”—reducing the flow of greenhouse gases into the atmosphere; and 2) “adaptation”—learning to live with, and adapt to, the climate change that has already been set in motion. The key question is: what will our emissions of carbon dioxide and other pollutants be in the years to come? Recycling and driving more fuel-efficient cars are examples of important behavioral change that will help, but they will not be enough. Because climate change is a truly global, complex problem with economic, social, political and moral ramifications, the solution will require both a globally-coordinated response (such as international policies and agreements between countries, a push to cleaner forms of energy) and local efforts on the city- and regional-level (for example, public transport upgrades, energy efficiency improvements, sustainable city planning, etc.). It’s up to us what happens next.23 But this support for theory 1 is weak because these warming/cooling cycles cannot explain Earth’s relatively recent surge in global temperatures: The paleoclimate record also reveals that the current climatic warming is occurring much more rapidly than past warming events. As the Earth moved out of ice ages over the past million years, the global temperature rose a total of 4 to 7 degrees Celsius over about 5,000 years. In the past century alone, the temperature has climbed 0.7 degrees Celsius, roughly ten times faster than the average rate of ice-age-recovery warming. Models predict that Earth will warm between 2 and 6 degrees Celsius in the next century. When global warming has happened at various times in the past two million years, it has taken the planet about 5,000 years to warm 5 degrees. The predicted rate of warming for the next century is at least 20 times faster. This rate of change is extremely unusual.25

Theory 2: Global warming is caused not by human activity but by surges in the sun’s energy. We know that the sun affects Earth’s climate system. Subtle changes in Earth’s orbit around the sun have caused the advances and retreats of the ice ages, and scientists have documented the sun’s natural 11-year cycle of decreases and increases in solar radiation. But these facts do not show that global warming is propelled by the sun and not human activity. NASA scientists explain why: The warming we’ve seen over the last few decades is too rapid to be linked to changes in Earth’s orbit, and too large to be caused by solar activity. One of the “smoking guns” that tells us the Sun is not causing global warming comes from looking at the amount of the Sun’s energy that hits the top of the atmosphere. Since 1978, scientists have been tracking this using sensors on satellites and what they tell us is that there has been no upward trend in the amount of the Sun’s energy reaching Earth. A second smoking gun is that if the Sun were responsible for global warming, we would expect to see warming throughout all layers of the atmosphere, from the surface all the way up to the upper atmosphere (stratosphere). But what we actually see is warming at the surface and cooling in the stratosphere. This is consistent with the warming being caused by a build-up of heat-trapping gases near the surface of the Earth, and not by the Sun getting “hotter.”26 Theory 3: The idea that human-caused global warming exists is a fraud perpetuated by a conspiracy of scientists. The evidence for this charge is nonexistent. In 2009, a controversy, now labeled Climategate, arose when servers at the University of East Anglia in the United Kingdom were hacked and emails between climate scientists were stolen and published on the internet. Many people alleged that the stolen emails showed that the scientists had suppressed or falsified climate data and that this revelation was proof that global warming was a conspiracy. Pennsylvania State University, the U.K. House of Commons Science and Technology Committee, the U.S. Department of Commerce Inspector General, and the National Science Foundation launched inquiries and concluded that no scientific misconduct was involved. But even if the allegations against the scientists were true, that alone would not prove that there is no human-made global warming nor that there is a conspiracy of scientists bent on promoting a lie. Conspiracy or no conspiracy, the case for human-caused warming is backed by a vast array of evidence gathered for decades by independent scientists throughout the world. In addition, theory 3 suffers from the same weaknesses that plague conspiracy theories generally: It fails criteria of adequacy. It fails the criterion of scope because it provides no evidence about who the conspirators are and how they were able to conspire together to fake or suppress evidence and to fool thousands of other scientists. So it doesn’t really explain anything. It fails the criterion of simplicity because it makes a host of assumptions—that there are conspiring scientists, that they can follow a global-warming agenda in their research without raising suspicions, that they can coordinate their activities, that they can keep their conspiracy secret, that the scientific community would not discover the deception, that they would risk their careers and reputation to falsify their findings, that their employers and institutions would not figure out what they were really up to, and so on. The theory is not conservative because it conflicts with human nature, with what we know about the way scientists and scientific organizations work, and with the behavior and values of scientists themselves. It also conflicts with what science has already firmly established—that human activity caused and is causing global warming.

Theory 4: Global warming is happening, and human activity is the cause. An enormous number of scientific studies and observations demonstrate that Earth is getting warmer. According to the National Academy of Sciences, Earth’s average surface air temperature has increased by about 0.8 °C (1.4 °F) since 1900, with much of this increase taking place since the mid-1970s. A wide range of other observations (such as reduced Arctic sea ice extent and increased ocean heat content) and indications from the natural world (such as poleward shifts of temperature-sensitive species of fish, mammals, insects, etc.) together provide incontrovertible evidence of planetary-scale warming. The clearest evidence for surface warming comes from widespread thermometer records. In some places, these records extend back to the late 19th century. Today, temperatures are monitored at many thousands of locations, over both the land and ocean surface. Indirect estimates of temperature change from such sources as tree rings and ice cores help to place recent temperature changes in the context of the past. In terms of the average surface temperature of Earth, these indirect estimates show that 1983 to 2012 was probably the warmest 30-year period in more than 800 years. A wide range of other observations provides a more comprehensive picture of warming throughout the climate system. For example, the lower atmosphere and the upper layers of the ocean have also warmed, snow and ice cover are decreasing in the Northern Hemisphere, the Greenland ice sheet is shrinking, and sea level is rising. These measurements are made with a variety of monitoring systems, which gives added confidence in the reality that Earth’s climate is warming.27 The evidence that this warming is human-caused is extensive, consistent, and multidimensional. From 1800 to 2012, the a major greenhouse gas carbon dioxide has increased in the atmosphere by about 40 percent—from about 280 parts per million by volume to about 380 parts per million. As carbon dioxide has increased, so has global surface temperature, and scientists have been able to link the carbon dioxide directly to human activities: Measurements of different forms of carbon (isotopes) reveal that this increase is due to human activities. Other greenhouse gases (notably methane and nitrous oxide) are also increasing as a consequence of human activities. The observed global surface temperature rise since 1900 is consistent with detailed calculations of the impacts of the observed increase in atmospheric CO2 (and other human-induced changes) on Earth’s energy balance. Different influences on climate have different signatures in climate records. These unique fingerprints are easier to see by probing beyond a single number (such as the average temperature of Earth’s surface), and looking instead at the geographical and seasonal patterns of climate change. The observed patterns of surface warming, temperature changes through the atmosphere, increases in ocean heat content, increases in atmospheric moisture, sea level rise, and increased melting of land and sea ice also match the patterns scientists expect to see due to rising levels of CO2 and other human-induced changes.28 Some people have claimed that this global warming must be due to natural (nonhuman) causes. But scientists have demonstrated that this is not the case: The expected changes in climate are based on our understanding of how greenhouse gases trap heat. Both this fundamental understanding of the physics of greenhouse gases and fingerprint studies show that natural causes alone are inadequate to explain the recent observed changes in climate. Natural causes include variations in the Sun’s output and in

Earth’s orbit around the Sun, volcanic eruptions, and internal fluctuations in the climate system (such as El Niño and La Niña). Calculations using climate models have been used to simulate what would have happened to global temperatures if only natural factors were influencing the climate system. These simulations yield little warming, or even a slight cooling, over the 20th century. Only when models include human influences on the composition of the atmosphere are the resulting temperature changes consistent with observed changes.29 Most actively publishing climate scientists—97 percent—affirm that global warming is happening now and that humans are causing it. Eighteen American scientific societies and 11 international science academies concur. A joint statement from the American scientific organizations says, Observations throughout the world make it clear that climate change is occurring, and rigorous scientific research demonstrates that the greenhouse gases emitted by human activities are the primary driver.30 Now let’s apply the criteria of adequacy to our four theories. Theory 1 (natural cycles of warming and cooling) is testable and simple, and it has some scope, since natural variations in Earth’s climate can explain ice ages and some warming periods. But it is not fruitful, because it has predicted no previously unknown entities or processes. Its worst fault, however, is that it fails the criterion of conservatism: It conflicts with established facts about the magnitude and pattern of global warming. Scientists understand the nature of Earth’s normal cycles of warming and cooling, but these climate variations do not match the scientific data regarding the planet’s abnormal rise in temperature. Theory 2 (solar energy changes) is testable and simple and has some scope, since it explains phenomena involved in Earth’s 11-year cycle of decreases and increases in solar radiation. But it isn’t fruitful since it has predicted no new phenomena. Worst of all, like theory 1, it falls short on the criterion of conservatism because solar radiation cycles don’t correspond to the documented patterns of global warming. Theory 3 (conspiracy of scientists) is the worst of the competing theories. As noted earlier, it is neither simple nor conservative, and it is without scope—it explains nothing. If the World Is Warming, Why Are Some Winters and Summers Still Very Cold? Snow, ice, and frigid weather may occur even while global warming is happening. The U.S. National Academy of Sciences and the United Kingdom’s Royal Society explain why: Global warming is a long-term trend, but that does not mean that every year will be warmer than the previous one. Day to day and year to year changes in weather patterns will continue to produce some unusually cold days and nights, and winters and summers, even as the climate warms. Climate change means not only changes in globally averaged surface temperature, but also changes in atmospheric circulation, in the size and patterns of natural climate variations, and in local weather. La Niña events shift weather patterns so that some regions are made wetter, and wet summers are generally cooler. Stronger winds from polar regions can contribute to an occasional colder winter. In a similar way, the persistence of one phase of an atmospheric circulation pattern known as the North Atlantic Oscillation has contributed to several recent cold winters in Europe, eastern North America, and northern Asia. Atmospheric and ocean circulation patterns will evolve as Earth warms and will influence storm tracks and many other aspects of the weather. Global warming tilts the odds in favour of more warm days and seasons and fewer cold days and seasons. For example, across the continental

United States in the 1960s there were more daily record low temperatures than record highs, but in the 2000s there were more than twice as many record highs as record lows. Another important example of tilting the odds is that over recent decades heatwaves have increased in frequency in large parts of Europe, Asia and Australia.31 Theory 4 is by far the best of the four theories. It is testable, simple, fruitful, and conservative. It has successfully predicted some previously unknown phenomena, including global sea level rise, melting of land and sea ice, retreating glaciers, and warming of the atmosphere. These successful predictions make the theory fruitful. And of course it is conservative because it is consistent with scientific laws, well-supported theories, and established facts. Section Query

SECTION QUERY 1. Do you think evolution can be compatible with religious beliefs? Why or why not? 2. Do you believe climate change is being caused by human activity? Why or why not?

8.5 How the Media Get Science Wrong Science sheds its light on the world and helps us distinguish between what is real and what is not. But the media are prisms that can sometimes distort that light. Too often the science news that floods our feeds is not just slightly inaccurate; it’s not even close to what the science actually says. The resulting headlines can range from silly to ludicrous to morbid: • Scientists Say Smelling Farts Might Prevent Cancer (Time Magazine) • Why Oreos May Be as Addictive as Cocaine (Forbes) • “Horns” are growing on young people’s skulls. Phone use is to blame, research suggests. (Washington Post) • More people have died from selfies than shark attacks this year. (Mashable) But false or misleading science news, whether amusing or not, can also alarm us unnecessarily and lead us to change our lives for no good reason. For example, • Bacon Gives Kids Cancer (U.K. Daily Mirror) • Asparagus link to breast cancer is discovered by scientists. (Evening Standard) • Junk Food in Pregnancy Leaves Children Fat for Life (U.K. Daily Mirror) • Sugar as Addictive as Cocaine, Heroin (New York Daily News) •

Eat Less Meat, Live Longer? (New York Times) And some science news can lead to harm by encouraging unproven or risky treatments and unsafe habits: • Vitamin D: Supplement Linked to Weight Loss in Overweight and Obese Children (Newsweek) • Ketamine Shows Promise as Treatment for Adolescents with Depression (Newsweek) • Does Echinacea Really Work Against Colds? (Time Magazine) We can add to this list a few stories highlighted by comedian John Oliver in his funny and right-on-target takedown of bad science reporting: • Drinking a glass of red wine is equivalent to an hour at the gym. • Sugar might make cancer grow. • Pregnant women who eat chocolate every day can improve blood flow to the placenta and benefit the growth and development of their baby. • Drinking champagne every week may delay dementia and Alzheimer’s disease. • Driving while dehydrated is just as dangerous as driving drunk.

FIG. 8.3 Screen grab from Fox News report on red wine and exercise. What’s going on here? Two things: hyping the science and misunderstanding the science. Hyping the Science

Health and medical studies are produced by universities, government and commercial labs, and independent health organizations. When a study is completed, the sponsoring organization announces it to the world by distributing press releases. The point of the releases is to enhance the organization’s prestige, gain publicity, or attract future funding. So those who write the releases are under pressure to highlight the most interesting, media-attracting aspects of the research, even when it is highly technical, complicated, and (to most nonscientists) yawn-inducing. The result is often just what you might expect: news releases that are as much hyperbole and distortion as fact.

FIG. 8.4 Screen grab from Today Show report on sugar and cancer. When news outlets get the releases, they pounce on them, looking for science news they can turn into eye-opening headlines or clickbait and stories that have immediate relevance to the readers. The better reporters from the more responsible news organizations will look beyond the press release for the facts. They will read the original study, talk to the scientists involved, see what other studies have been done, and ask other scientists in the field what they think of the new research. But other news outfits won’t go to all that trouble. They will report what’s in the press release, exaggerations and all, and might even add a sexy (and misleading) twist or two

of their own. So in too many cases, the original research—which may be solid but preliminary or exploratory—gets distorted twice.

FIG. 8.5 Screen grab from Channel 4 WBZ News report on chocolate and health of mothers and babies.

FIG. 8.6 Screen grab from KTVU report on champagne and dementia. A now infamous example of this double whammy of misrepresentation is the chocolate story mentioned earlier. The story was based on a preliminary study of the effect of high- and low-flavonoid chocolate on the risk of the pregnancy complication known as preeclampsia. The result: The researchers found no significant difference in the rate of preeclampsia between women who ate the two kinds of chocolate. In other words, chocolate didn’t help prevent preeclampsia. But the press release issued by a medical society put a more positive spin on the study with the headline “The benefits of chocolate during pregnancy.” Several news outlets then covered the story, reporting that chocolate decreases the risk of preeclampsia.32

FIG. 8.7 Screen grab from Fox News report on the dangers of driving while dehydrated. Often the hype in a science story needs no nudge at all from a press release. Consider the Washington Post report about horns growing on young people’s skulls due to phone use. The study in question was not about phone use, and the “horns” came from a bit of speculation at the end of the study.33 Steven Novella, science news critic and clinical neurologist at Yale University School of Medicine, puts the study in perspective: When I see headlines like that my first questions is always—what did the research actually show? What was the data? In this case the researchers were looking at X-rays of the skull, and particularly at the occipital protuberances. This is a pair of bumps at the back of the head where the posterior neck muscles insert. They found that the risk of having bony spurs or calcifications in the ligaments attaching to the skull (not horns) increased in men, with forward tilt of the head, and in younger subjects. That’s the data. Everything else is the authors’ speculation about what these results mean.34

FIG. 8.8 X-rays of bone spurs at the back of the skull. Misunderstanding the Science

Scientific studies drive science news reporting, and scientific studies are frequently misunderstood and misconstrued by nonscientists, whether they’re journalists or not. But nonscientists don’t need to be experts to make some reasonable judgments about the evidence that studies purportedly offer. Many times you can see, as well as any expert can, when a study does not establish that a treatment is effective or that factor A causes factor B. You can see this if you understand some basic facts about the nature and limitations of this kind of research. Recall that the gold standard in medical and health research is the double-blind, randomized, controlled clinical trial, a study configured to minimize bias and error at every step. When this kind of study is conscientiously designed and conducted, it can offer strong and clear support for claims about treatments and cause-and-effect relationships. For all the reasons enumerated earlier, it has both an experimental (treatment) group and a control group; the subjects in the

control group get a placebo or a different treatment; neither the subjects nor the experimenters know who receives the real treatment and who receives the placebo; and each group is as much alike as possible to start. If one of these elements is missing, we have good reason to doubt the results of the study. (Of course, even if all these elements are present, the study’s results can still be comprised by errors in collecting, analyzing, and interpreting the data.) Other kinds of research can help scientists understand human physiology and disease, but these studies alone also typically cannot establish cause and effect. These include the following: • Single studies. In most instances, a single study cannot prove very much. Research is exacting work, and many things can—and do—go wrong. The probability that researchers in any given study have reached false conclusions is high. That’s why scientists seek replication—the redoing of a study by different researchers to see if they get the same results. Usually it takes many studies to confirm conclusions. The media like to give the impression that a medical breakthrough has just popped out of a single study, but real breakthroughs almost never happen that way. All of the dubious headlines noted earlier were based on single studies. • Small studies. Studies with only a handful of subjects are very preliminary, usually designed just to probe a question and to see if larger clinical trials should be done. They are unlikely to yield strong evidence of cause-and-effect connections. The smaller the study, the greater the chance that some confounding factor will skew the results. The study about dehydration and drunk driving was done on just 12 men; the study on ketamine and depression included just 13 adolescents. The reaction of a conscientious scientist to the findings of a small study is likely to be something like “Gee, I wonder if that’s true” or “Let’s look into that,” not “Wow! A new cure!” • Anecdotes and case studies. Anecdotes are individual stories of personal experience (“Whiskey cured my warts” or “Ginseng boosted my IQ”). But for all the reasons discussed in Chapter 7, anecdotes are very weak evidence. Case studies (or case reports) are doctor’s observations of individual patients. These reports can give us important clues about the nature of an illness, but they cannot establish the cause of a disease or confirm the value of a treatment. A doctor’s attempt to draw firm conclusions about the effectiveness of a treatment is undermined by all the same confounding factors that controlled studies try to avoid—the placebo effect, overlooked causes, the variable nature of illness, and more. • Nonintervention studies. As we saw earlier, nonintervention studies don’t involve intervening in subjects’ lives. Their purpose is to search for associations between disease and health habits, body weight, diet, medical conditions, and countless other factors. Nonintervention studies (also known as case-control, cohort, and prospective studies) can involve thousands of subjects, examine scores of factors, and run for years. But despite their size, they cannot prove cause-and-effect relationships. They can only reveal correlations that merely suggest possible cause and effect. They can show that vitamin X supplements are consistently linked to liver disease, but they

can’t demonstrate that vitamin X supplements cause liver disease. Only controlled clinical trials can do that. Scientists know it is extremely easy to find correlations among all sorts of things in such studies. It is easy to examine dozens of factors and sift them until associations are found that are “statistically significant” but probably merely coincidental (a practice known as “p-hacking”). Through p-hacking, we might find links between cell phone use and pregnancy, shirt size and IQ, pet ownership and diabetes, law degrees and cancer, eating bananas and impotence. But probably none of these associations would be causal. The headline about eating meat is based on an observational study that has been accused of p-hacking.35 • Animal studies. Animal studies are invaluable preliminary research, providing clues to the possible effectiveness of drugs, the hazards of chemicals, and the nature of disease. But by themselves, they cannot show that a treatment works in humans or that a food or drug is safe for human consumption. Treatments proven effective in animals usually do not work as hoped in humans. These facts, however, have not stopped the media from insisting that a substance that worked wonders for a rat can do the same for a human being. Studies in mice and rats were the starting points for the headlines about red wine and gym workouts, asparagus and breast cancer, sugar and cancer, champagne and dementia, and Oreos and addiction. Is All Health News Wrong?

Considering the mountains of hyped and distorted science reporting that we encounter every day, we might be tempted to answer yes. But we don’t know that for sure. We do know, however, that a disconcertingly large proportion of health news on social media is misleading or untrue. In 2019, a coalition of scientists, clinicians, and science editors published a study of the scientific accuracy of health news articles. They focused on the top 100 most popular health articles—that is, those that had the highest number of social media “engagements” (shares, comments, and likes). These experts assessed the credibility of each article as Very High, High, Neutral, Low, or Very Low. They found that of the top 10 most popular articles (representing 6.6 million shares), only 3 of them got a high credibility rating. Three of them received a very low credibility rating, meaning that they contained major inaccuracies. Four of them got a medium credibility score: They contained no major inaccuracies but did give misleading information. Perhaps not surprisingly, the most inaccurate articles got the most attention on social media. As the researchers pointed out, This result is expected: sensational headlines (as exemplified by these 3 [very low rating] articles) are much more likely to attract social media engagements, as opposed to headlines in which a balanced tone is struck. Coupled with the fact that clickbait headlines have a tendency to be factually inaccurate (often involving exaggerations and logical fallacies), it is therefore not surprising that scientifically inaccurate stories tend to be more popular on social media than accurate stories. This also highlights a major concern in online credibility, as this means that the general public is more likely to come into contact with misleading information than accurate ones on social media.36 These were the top 10 articles ranked by popularity:37

1. Federal Study Finds Marijuana 100X Less Toxic Than Alcohol, Safer Than Tobacco 2. Video shows difference between healthy lungs and those of a smoker 3. Benefits of Walking: 8 Ways Walking Regularly Improves Your Health 4. Everything You Know About Obesity Is Wrong 5. World Health Organization Officially Declares Bacon is as Harmful as Cigarettes 6. Have Cold or Flu Symptoms? Here’s How to Tell the Difference 7. Stem Cell Treatment Could Be A Game-Changer for MS Patients 8. How Cycling In Old Age Can Keep Your Immune System Young 9. Is everything you think you know about depression wrong? 10. Cause of polycystic ovary syndrome discovered at last

FIG. 8.9 Credibility ratings for the 10 most popular health articles on social media in 2018.

FIG. 8.10 Credibility rating for each of the 100 most popular health articles on social media in 2018. Less than half of the 100 most popular articles achieved a high credibility rating. The rest had problematic content ranging from exaggerated to confused to false. The number of shares for these dubious articles accounted for almost half of the total shares. Expert Review of Article 9: “Is Everything You Think You Know about Depression Wrong?” This article, published in the Guardian and shared 469,000 times, was judged by experts as “not credible and potentially harmful.” According to their online report, This article discusses current scientific thinking behind clinical depression and raises the possibility that clinicians may be treating the condition using the wrong approach, suggesting that most cases of depression are the result of feeling a lack of fulfilment in one’s life, instead of a chemical imbalance in the brain. While there are some grains of truth in this, the article is highly misleading in many ways.38 One reviewer, whose comments are included on the site, concurs: This article is an excerpt from a provocative book written by a lay person who is clearly anti-psychiatry, so there is no pretense of providing evidence (except cherry-picking evidence which supports his views) or a balanced viewpoint. It is full of wild exaggerations, oversimplifications and inaccuracies. Just a few include: … • That all psychiatrists believe that depression is caused only by biochemical changes in the brain—NOT TRUE. All psychiatrists are taught a biopsychosocial model of illness, comprised of

biological factors, psychological factors and social factors all working in a complex intertwined relationship (with each factor amenable to evidence-based treatments).… • That all psychiatrists believe depression is caused by low serotonin and that antidepressant medications work by increasing serotonin—NOT TRUE. That theory was dispelled over 30 years ago and no one believes that there is a single cause for depression. The prevailing theory now is that antidepressant medications work by altering complex biochemical pathways that lead to formation of new brain cells and brain pathways, in effect to allow the brain to better compensate for stress. • That all psychiatrists only treat depression with medications—NOT TRUE. All psychiatrists are trained in psychotherapy and use supportive psychotherapy in management of depression. Many psychiatrists also deliver evidence-based psychotherapies (for example, cognitive behavioural therapy or interpersonal psychotherapy).39 Media Activity and Section Query

MEDIA ACTIVITY 8.1 [Please note: You must be using an online, browser-based eReader in order to view this content.]

MEDIA ACTIVITY 8.2 [Please note: You must be using an online, browser-based eReader in order to view this content.]

SECTION QUERY Find one news story that you think does not hype or exaggerate the scientific research it’s based on.

8.6 Scientific Opinion Polls Recall from Chapter 2 that in the kind of argument called enumerative induction, we reason from premises about individual members of a group to conclusions about the group as a whole. Enumerative inductions reach a high level of sophistication in the form of scientific opinion polls conducted by professional polling organizations. Opinion polls are used to arrive at generalizations about everything from the outcome of presidential elections to public sentiments about cloning babies to the consumer’s appetite for tacos. But as complex as they are, opinion polls are still essentially inductive arguments (or the basis of inductive arguments) and must be judged accordingly. So, as inductive arguments, opinion polls should (1) be strong and (2) have true premises. More precisely, any opinion poll worth believing must (1) use a large enough sample that accurately represents the target population in all the relevant population features and (2) generate accurate data (the results must correctly reflect what they purport to be about). A poll can fail to meet this

latter requirement through data-processing errors, botched polling interviews, poorly phrased questions, and the like. In national polling, samples need not be enormous to be accurate reflections of the larger target population. Modern sampling procedures used in national polls can produce representative samples that are surprisingly small. Polling organizations such as Gallup and Pew regularly conduct polls in which the target group is American adults (a population of more than 327 million in 2018), and the representative sample consists of only 1000 to 1500 individuals. How can a sample of 1000 be representative of almost 200 million people? This can be achieved by using random sampling. To ensure that a sample is truly representative of the target group, the sample must be selected randomly from the target group. In a simple random selection, every member of the target group has an equal chance of being selected for the sample. Imagine that you want to select a representative sample from, say, 1000 people at a football game, and you know very little about the characteristics of this target population. Your best bet for getting a representative sample of this group is to choose the sample members at random. Any nonrandom selection based on preconceived notions about what characteristics are representative will likely result in a biased sample. Selecting a sample in truly random fashion is easier said than done (humans have a difficult time selecting anything in a genuinely random way). Even a simple process such as your trying to arbitrarily pick names from a list of registered voters is not likely to be truly random. Your choices may be skewed, for example, by unconscious preferences for certain names or by boredom and fatigue. Researchers and pollsters use various techniques to help them get close to true randomization. They may, for instance, assign a number to each member of a population and then use a random-number generator to make the selections. One approach that definitely does not yield a random sample is allowing survey subjects to choose themselves. The result of this process is called a self-selecting sample—a type of sample that usually tells you very little about the target population. We would get a self-selecting sample if we publish a questionnaire on a website and ask readers to fill it out and send it in, or if during a TV or radio news broadcast we ask people to cast their vote on a particular issue by clicking options on a website or emailing their responses. In such cases, the sample is likely to be biased in favor of subjects who, for example, just happen to be especially opinionated or passionate; who may have strong views about the topic of the survey and are eager to spout off; or who may simply like to fill out questionnaires. Magazines, newspapers, talk shows, and news programs sometimes acknowledge the use of self-selecting samples by labeling the survey in question as “unscientific.” But whether or not that term is used, the media frequently tout the results of such distorted surveys as though the numbers actually proved something. How Survey Questions Go Wrong Many opinion polls are untrustworthy because of flaws in the way the questions are asked. The sample may be large enough and representative in all the right ways, but the poll is still dubious. Here are a few of the more common problems. Question Phrasing

Poll results can be dramatically skewed simply by the way the questions are worded. A poll might ask, for example, “Are you in favor of a woman’s right to kill her unborn child?” The question is ostensibly about a woman’s right to terminate a pregnancy through abortion and is

supposed to be a fair measure of attitudes on the question. But the wording of the question practically guarantees that a very large percentage of respondents will answer no. The controversial and emotionally charged characterization of abortion as the killing of an unborn child would likely persuade many respondents to avoid answering yes. More neutral wording of the question would probably elicit a very different set of responses. Another example: A 1995 poll of African Americans discovered that 95 percent of the sample group approved of a local school voucher program. To get this huge approval rating, the survey question was worded like this: “Do you think that parents in your area should or should not have the right to choose which local schools their children will attend?” Who would want to give up such a right? No wonder the question elicited an overwhelming number of “shoulds.” Such biased wording is often the result of pollster sloppiness. Many other times it’s a deliberate attempt to manipulate the poll results. The crucial test of polling questions is whether they’re likely to bias responses in one direction or another. Fair questions aren’t skewed this way—or are skewed as little as possible. Question Ordering

The order in which questions are asked in a poll can also affect the poll results. Pollsters know that if the economy is in bad shape and they ask people about the economic mess first and then ask them how they like the president, respondents are likely to give the president lower marks than if the order of the questions were reversed. Likewise, if you’re asked specific questions about crimes that have been committed in your hometown and then you’re asked if you feel safe from crime, you’re more likely to say no than if you’re asked the questions in reverse order. Restricted Choices

Opinion polls frequently condense broad spectrums of opinions on issues into a few convenient choices. Some of this condensation is necessary to make the polling process manageable. But some of it is both unnecessary and manipulative, seriously distorting the opinions of those polled. Daniel Goleman of the New York Times offers this example: “In one survey … people were asked if they felt ‘the courts deal too harshly or not harshly enough with criminals.’ When offered just the two options, 6 percent said ‘too harshly’ and 78 percent answered ‘not harshly enough.’ But when a third alternative was added—‘don’t have enough information about the courts to say’—29 percent took that option, and 60 percent answered ‘not harshly enough.’” So a well-conducted poll using a random sample of 1000 to 1500 people can reliably reflect the opinions of the whole adult population. Even so, if a second well-conducted poll is done in exactly the same way, the results will not be identical to those of the first poll. The reason is that every instance of sampling is only an approximation of the results that you would get if you polled every single individual in a target group. And, by chance, each attempt at sampling will yield slightly different results. If you dipped a bucket into a pond to get a one-gallon sample of water, each bucketful would be slightly different in its biological and chemical content—even if the pond’s content was very uniform. Such differences are referred to as the margin of error for a particular sampling or poll. Competently executed opinion polls will state their results along with a margin of error. A presidential poll, for example, might say that candidate X will receive 62 percent of the popular vote, plus or minus 3 points (a common margin of error for presidential polls). The usual way of

expressing this number is 62 percent ±3. This means that the percentage of people in the target population who will likely vote for candidate X is between 59 and 65 percent. Connected to the concept of margin of error is the notion of confidence level. In statistical theory, the confidence level is the probability that the sample will accurately represent the target group within the margin of error. A confidence level of 95 percent (the usual value) means that there is a 95 percent chance that the results from polling the sample (taking into account the margin of error) will accurately reflect the results that we would get if we polled the entire target population. So if our aforementioned presidential poll has a 95 percent confidence level, we know that there’s a 95 percent chance that the sampling results of 62 percent ±3 points will accurately reflect the situation in the whole target group. Of course, this confidence level also means that there’s a 5 percent chance that the poll’s results will not be accurate. Note that confidence level refers only to sampling error, the probability of the sample not accurately reflecting the true values in the target population. It doesn’t tell you anything about any other kinds of polling errors such as bias that can occur because of poorly worded questions or researchers who may consciously or unconsciously influence the kinds of answers received. Sample size, margin of error, and confidence level are all related in interesting ways. • Up to a point, the larger the sample, the smaller the margin of error because the larger the sample, the more representative it is likely to be. Generally, for national polls, a sample size of 600 yields a margin of error of ±5 points; a sample of 1000, ±4 points; and a sample of 1500, ±3 points. But increasing the sample size substantially to well beyond 1000 does not substantially decrease the margin of error. Boosting the sample from 1500 to 10,000, for example, pushes the margin of error down to only 1 percent. • The lower the confidence level, the smaller the sample size can be. If you’re willing to have less confidence in your polling results, then a smaller sample will do. If you can accept a confidence level of only 90 percent (a 10 percent chance of getting inaccurate results), then you don’t need a sample size of 1500 to poll the adult population. • The larger the margin of error, the higher the confidence level can be. With a large margin of error (±8, for example), you will naturally have more confidence that your survey results will fall within this wide range. This idea is the statistical equivalent of a point made earlier: You can have more confidence in your enumerative inductive argument if you qualify, or decrease the precision of, the conclusion. Mean, Median, and Mode If you read enough opinion polls, you will surely encounter one of these terms: mean, median, or mode. These concepts are invaluable in expressing statistical facts, but they can be confusing. Mean is simply an average. The mean of these four numbers—6, 7, 4, and 3—is 5 (6 + 7 + 4 + 3 = 20 divided by 4). The median is the middle point of a series of values, meaning that half the values are above the point and half the values are below the point. The median of these eleven values—3, 5, 7, 13, 14, 17, 21, 23, 24, 27, 30—is 17 (the number in the middle). The mode is the most common value. The mode in this series of values—7, 13, 13, 13, 14, 17, 21, 21, 27, 30, 30—is 13 (the most frequently appearing value).

The notions of mean, median, and mode are often manipulated to mislead people. For example, let’s say that the dictator of Little Island Nation (population 1000) proposes a big tax cut for everyone, declaring that the mean tax savings will be $5000 (the total tax cut divided by 1000 taxpayers). The Islanders begin to gleefully envision how they will spend their $5000. But then they learn that the mean figure has been skewed higher because of a few millionaires whose tax savings will be $100,000 or more. The tax savings for the vast majority of taxpayers is actually less than $500. The $5000 figure that the dictator tossed out is the true mean—but painfully misleading. To the Islanders, the median tax savings is much more revealing: The median is $400. The mode, the most common figure, is $300. When they get all the facts, the Islanders stage a revolt—the first one in history caused by a better understanding of statistics. An enumerative induction, like any other inductive argument, must be strong and have true premises for us to be justified in accepting the conclusion. A strong enumerative induction must be based on a sample that is both large enough and representative. An opinion poll, as a sophisticated enumerative induction, must use a sufficiently large and representative sample and ensure that the gathered data accurately reflect what’s being measured. Trustworthy Sources in Science and Medicine American Association for the Advancement of Science (AAAS) American Physical Society (APS) Centers for Disease Control and Prevention (CDC) Committee for Skeptical Inquiry HealthNewsReview.org Howtogeek James Randi Educational Foundation (JREF) Johns Hopkins Medicine Live Science Mayo Clinic MIT Technology Review NASA National Academy of Sciences (NAS) National Center for Science Education (NCSE) National Geographic National Institutes of Health National Oceanic and Atmospheric Administration (NOAA) Nature New Scientist Public Library of Science (PLOS) PubMed Quackwatch SciCheck Science Science Based Medicine Science Daily Science.gov Scientific American

Skeptic Skeptical Inquirer Skeptical Raptor Understanding Science Union of Concerned Scientists Section Query

SECTION QUERY Do you see any disadvantages in trusting the results of an opinion poll based on a self-selecting sample?