PSY Discussion 2
Shivani21999
CaseStudyinEvaluatingEvidenceHomeopathy.htmlSome posts in the past have mentioned the alternative medicine practice known as homeopathy. This topic is such a great example of how difficult it can be to find reliable information on the web that I thought it would be worth discussing in some detail. Some questions this example can help us explore are: What sources of information are trustworthy? What counts as “research” or as “evidence”? When I encounter claims that “research” supports two opposite conclusions, how can I decide what to believe?
Homeopathy is a practice based on the belief that very tiny doses of a substance that causes a disease can be used to treat that disease. A quick web search for “homeopathic arsenic,” for example, produced offers to sell me “30c” diluted arsenic tablets to treat stomach ailments. Arsenic is a deadly poison which produces (among other things) violent intestinal disorders. By the logic of homeopathy, a dilute solution of arsenic should, therefore, cure intestinal disorders. How dilute? The homeopathic designation of “30c” indicates that one part arsenic has been diluted with 100 parts water (C being the Roman numeral for 100) thirty times. So after the first dilution it would be 1/100 arsenic. After the second it would be 1/10,000 arsenic. After 30 such dilutions it would be one part arsenic per 100^30 (100 to the 30th power) parts water. That is a 1 followed by 60 zeroes, or 10^60, ten to the sixtieth power. To give you a sense of how dilute that is, there are estimated to be about 10^80 atoms in the universe. And a cup of water contains about 10^25 molecules of water (see https://www.thenakedscientists.com/forum/index.php?topic=12443.0 for the calculation). So one part arsenic in 10^60 parts water would be less than 1 atom of arsenic per cup of water, which is just... water. Each of the “homeopathic 30c arsenic” tablets I could buy would probably contain zero atoms of arsenic if the contents are exactly what the label says they are.
So if a homeopathic remedy is manufactured as advertised it would contain no active ingredients. It would, in other words, be a placebo.
Yet WebMD says that “Research is mixed. Some studies show that homeopathic remedies are helpful, while others don’t.” ( https://www.webmd.com/balance/what-is-homeopathy#1). What?? How can some studies have found that remedies containing no actual active ingredients are helpful? Is that just an outright falsehood? No, it is perhaps worse than that. It is technically true but misleading. There have indeed been some published studies that show benefits from homeopathic remedies. But those studies are of poor quality (failing to effectively control for placebo effects or observer bias), and the best studies have consistently found no evidence that homeopathic remedies have any effect (see https://sciencebasedmedicine.org/reference/homeopathy/ for useful summaries of the double-blind studies and meta-analyses that make this clear. Useful summaries can also be found in https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6399603/. The UK's national health service also has a very good summary of homeopathic remedies in general: https://www.nhs.uk/conditions/homeopathy/)
The entire WebMD article on homoepathy appears to me to contain no statements that are individually technically untrue, yet as a whole the article gives the extremely misleading impression that homeopathy is a legitimate and perhaps effective medical practice. Read the article and see if you can find examples of technically true but misleading content. Here is one that really walks a fine line: “Doctors are divided because some of the theories behind homeopathy don’t line up with the principles of chemistry and physics.” I suppose it is true that “doctors are divided” in that there probably are one or more medical doctors who believe in homeopathy. And “some of the theories behind homeopathy don’t line up with the principles of chemistry and physics” is certainly true, but it makes it sound like there are minor quibbles with some peripheral parts of homeopathy, when in fact the primary theories of homeopathy are contradicted by the known laws of physics and chemistry.
Homeopathy is a great case study for how hard it is to find reliable information. It is hard to imagine anything that could be more obviously ineffective than what homeopathic remedies, by the definition of homeopathy, claim to be: substances that contain no active ingredients. Barring magic or the inclusion of other active ingredients, it is not possible that homeopathic remedies could have any effect other than as placebos. Yet there are published studies that seem to show evidence for their effectiveness, and well-known web sites that seem to give the impression that they are legitimate medical treatments. If it can be this confusing trying to find reliable evidence about homeopathy, what hope do we have of figuring things out for less obvious questions? What can you do?
First, try to get into the habit of thinking probabilistically. When deciding what you should believe, remember that none of us knows everything, and not everything is known. We have a natural tendency to want to classify things absolutely: definitely true or definitely false - 100% confident or 0%. For most questions the most accurate answer for me is “I don’t know.” And even for almost everything that I DO know I don’t know it to be true or false with 100% confidence. Instead of saying “I believe X” or “I don’t believe X”, get in the habit of saying to yourself “I’m N% confident that X is true.” I’m 85% sure that cognitive behavioral therapy is an effective treatment for anxiety disorders, for example. I think there is only a 15% chance that Freudian psychodynamic therapy is effective. I’ve read quite a bit about the research on both of those, the evidence is pretty clear in my view, and therefore I’m pretty confident. For some therapy that I am unfamiliar with I might start out completely unsure: 50%. Then, as evidence accumulates, I will change that probability up or down in proportion to how much evidence there is and how good it is. Belief should be on a sliding scale, depending on the strength of the evidence.
Second, be disciplined about what you accept as evidence. Empirical scientific research should be given some weight. How much? That depends on how many studies there are and how good each of them are. The spacing effect and the retrieval practice effect in learning have both been repeatedly demonstrated in well-controlled experiments (see https://www.retrievalpractice.org/). I’m 99% confident in each of them. Nutrition studies are mostly based on retrospective self-report data and correlational data, so even the things I most believe about nutrition (more vegetables and fiber good; more sugar bad) I’m only about 80% confident in. And I have very little confidence in any individual new nutritional study I see reported in the news. I will give it almost no weight until it has been replicated repeatedly and consistently.
Correlational evidence from scientific research and self-report evidence from survey data should be given only limited weight. It is not as reliable as well-controlled experimental evidence (like double or triple blind clinical trials), and it takes longer to accumulate a convincing body of this kind of evidence.
Personal experience, casual observation, and “stories” should be given almost no weight as evidence. Following this advice is hard and takes self-discipline, because it is human nature to find anecdotal evidence to be the most persuasive. How many people give up smoking because of statistics about lung cancer rates, versus how many give up smoking because a loved one who smoked died of lung cancer? The individual story is more persuasive. But individual stories can be misleading: How many people have delayed vaccinating their child because they were frightened by a story from someone who said vaccinations caused their child’s autism, even though scientific studies have conclusively shown that vaccines do not cause autism?
Third, try to identify sources of information that are generally trustworthy that you can use as a “reality check” when you encounter conflicting claims. Government science agencies and scientific professional societies are usually more trustworthy; groups advocating for a particular idea or practice are more suspect. Here are a couple of sites that I’ve found to be pretty reliable on a wide range of topics:
