discussion

Confirmation bias is a major issue in science. When we discussed p-hacking in Notes 2, that was really an issue with confirmation bias. Nobody sets out to consciously fudge their data, but they're so sure that the hypothesis is right that they unconsciously find patterns that support what they already believe.

There are a lot of famous examples of confirmation biases. Here's an example from Snopes.com of an amazing coincidence regarding world leaders during World War II (that is actually nothing but confirmation bias): https://www.snopes.com/fact-check/war-games-2/

Here is a list of amazing Lincoln-Kennedy coincidences from Snopes.com that are also the product of confirmation bias: https://www.snopes.com/fact-check/linkin-kennedy/

Some people have made claims of extraordinary similarities between identical twins reared apart. These similarities have been interpreted as evidence for paranormal phenomena like astrology (the twins are similar because of the similarity between their astrological charts) or ESP (twins are psychically linked). Wyatt et al. (1984; https://skepticalinquirer.org/1984/10/natural-levels-of-similarities/ ) argued that any pair of people could appear to have amazing similarities if you could sift through an infinite number of variables and only keep the matches.

"For example, one of our unrelated pairs of women shared these characteristics: Both are Baptist; volleyball and tennis are their favorite sports; their favorite subjects in school were English and Math (and both listed shorthand as their least favorite); both are studying nursing; and both prefer vacations at historical places. Had these similarities been found in a pair of identical twins (who had been reared apart) they might have been used as evidence for astrology, ESP, and the like" (p. 64).

You can evaluate the likelihood of coincidences and confirmation biases using a similar task. Find two people who don't know each other and get them to complete a version of the Dimensions Alike and Different Scale (DADS) that Wyatt et al. (1984) created. It was "a list of 41 areas of functioning, including political beliefs, musical interests, religious preferences, jobs held, hobbies, favorite foods, and so on" (p. 63). Have your two people discuss each item and rate how similar they are on a scale from "very strongly different" (0) to "very strongly alike" (10) with "about as many similarities as differences" (5) in the middle.

It might be easiest to find two friends of yours who don't know each other, or a friend and a relative. They need to have the same gender identity and be about the same age (like identical twins). Setting up a Zoom meeting to let them work together might be effective. Note that friend-friend pairs are over-represented in the data, so if you could make pairs that were friend-other (e.g., friend-relative) that would be helpful. They just need to not know each other before you have them do the survey.

After you have your pair rate their similarity on a number of dimensions, post your results here.

It will help us to assess the data if you include these things:

1. Write a paragraph similar to the example from Wyatt et al. (1984) that I copied into the notes. How amazing of a group of similarities can you put together?

2. Tell us how many things you measured to get that paragraph, and how many were rated as highly similar.

3. If you want to post your questions, that will let us evaluate what everyone came up with as measures.