Research methods

A Crash Course in Statistics at FIU – The Chi Square Test (#4) – Summer 2021

So you took Stats I and Methods One at FIU and passed. But do you remember what you did, how you did it, and why you did it? If you need some basic reminders for the chi square, then this is the lecture for you! I am going to talk about a chi square example in this document that corresponds to the example you saw in the Descriptive Statistics Crash Course (#1), t-Test Crash Course (#2), and ANOVA Crash Course (#3) where participants were asked to recall how much money they spent on textbooks the prior semester. However, for this Chi Square crash course, we are going to focus on nominal variables (categorical) rather than scaled variables (like ratio or interval). The good news is that this mini-lecture will sum up the basics of the chi square for you as we look at this study, but you can find additional information about the chi square in your textbooks. On the final page are several questions based on this crash course. Answer these questions and go into your “Crash Course in Statistics – The Chi Square Quiz #4” in your Canvas assessments menu and copy over your answer. Each Crash Course Quiz counts 5 points.

How, when, and why do a Chi Square?

Before we get to the example, let me give you some basic information about the chi square. The chi square is used to compare two or more levels of a categorical (nominal or ordinal) variable. I recommend looking at the crash course on descriptive statistics for more info on interval and ratio scales, but I want to highlight nominal and ordinal scales here.

Nominal scales are based on assigning items to categories. For example, you can have “yes” versus “no” categories, or “male” versus “female” categories, or “Honda” versus “Toyota” versus “Subaru” versus “Ford”. For nominal scales, are just different categories, but one option isn’t “better” or “higher” than another (After all, which is higher: males or females?).

Ordinal scales have more order to them. That is, they are ranked. Thus there might be a better or worse ranking here (Pizza is ranked highest, Salad second highest, Sandwiches third highest, Liver lowest for food preference). We might know the order, but we may not know how spread out those preferences are. That is, maybe pizza, salad, and sandwiches are all ranked very high in preference but liver is ranked really, really low! Or think about a race. The first, second, and third place finishers may come in a few seconds apart while the fourth place finisher is over a minute behind.

For this crash course, I want to focus only on the nominal scale. A chi square essentially looks at the percentage of cases that fall into categories of the variable. Let’s say I look at gender as a variable in my study. I sample 200 people at random. There is a good chance I would get 100 men and 100 women, but I could also be off a little from that 50/50 ratio. That is, I could wind up with 95 men and 105 women just based on natural fluctuations of my data collection and selection procedure. The question becomes, “Are the observed differences between the number of men and women I observe significantly different than the data that I expected to see?” As another example, I might look to see if there are more guilty than not-guilty verdicts in a trial. If I poll jurors and find that 60% found guilt while 40% found no guilt, I might want to run a chi square to see if the difference is based on chance factors or something more.

Because we are looking at categorical variables (men versus women, or guilt versus no guilt), it is not appropriate to look at means and standard deviations. After all, what is a mean gender? No such thing, right? Thus we cannot use a t-Test or ANOVA for this analysis, as those tests are based on mean scores. The chi square, however, is designed to look at percentages and frequencies of data. There are two different types of chi squares:

1). Chi Square – Goodness of Fit: In this test, we look at only one variable. Thus we can determine whether we have more men in our study than we should by chance, or we can see if there are more not guilty verdicts than we should have by chance. Although I will briefly discuss this Goodness of Fit test in the lecture material, we will focus mostly on the second kind of chi square test in this crash course document …

2). Chi Square – Test of Independence: For this test, we want to see if two variables are independent. Consider juror gender and verdict in the same chi square model. We might want to know if juror gender has an impact on verdict or whether verdict is independent of gender. BOTH variables here are nominal in nature (two levels to gender: male versus female – two levels to verdict: guilty or not guilty).

The nice thing about chi squares is that there really are no assumptions about the shape of the distribution (we don’t need a bell-shaped curve like we do for t-Tests and ANOVAs). But the variables can’t be on scales – just categories. Let’s run a chi square and interpret it in SPSS.

Textbook Study – How Much Did You Spend On Textbooks (High or Low Conditions)

Recall the basic set-up for our money spent on textbooks. Researchers ask participants to recall how much they spent on textbooks the prior semester, and has each participant write their answer on a survey sheet. In all conditions, the first ten answer slots are already filled in, presumably by other respondents. However, the researcher actually completed those ten slots, and manipulated the dollar amounts so that in in the High Dollar Condition, the dollar amounts ranged from $350 to $450 (see Figure 1). In the Low Dollar Condition, amounts ranged from $250 to $350 (see Figure 2). For right now, we are going to briefly omit the “Control” condition (we’ll get back to this third level later in this document) and focus on the High Dollar Condition and the Low Dollar Condition only.

Imagine we have eight real participants in the High Dollar Condition and eight real participants in the Low Dollar Condition (and no, we are not including the original dollar amounts on the sheet passed out by the researcher, as those are not real participants!). However, instead of asking participants to recall how much they spent on textbooks, we ask them to look at the other responses on the survey and decide if other respondents spent more or less than average. We thus have a nominal dependent variable with two answer options: “higher than average” versus a “lower than average”. See Figure 1 for the High Dollar Condition survey and Figure 2 for the Low Dollar Condition survey.

Consider the data below, noting that I have a dichotomized dependent variable. I will label “higher” as 1 and “lower” as 2 (though I could just as easily have labeled “higher” as 2 and “lower” as 1 – for nominal variables, the exact value does not matter, just that different numbers represent different conditions).

Condition A (High Dollar Condition)	Condition B (Low Dollar Condition)
1 (more than average)	1 (more than average)
1 (more than average)	1 (more than average)
1 (more than average)	1 (more than average)
1 (more than average)	2 (less than average)
1 (more than average)	2 (less than average)
1 (more than average)	2 (less than average)
1 (more than average)	2 (less than average)
2 (less than average)	2 (less than average)

Eyeballing this, it looks like participants selected the “Others spent higher than average” option (#1 option) response more frequently in the High Dollar Condition while participants selected the “Others spent lower than average” option (#2 option) more frequently in the Low Dollar Condition. But is this difference based simply on chance or something else? If it is based on chance, then I would say that both columns are pretty equal, and thus the High versus Low Dollar manipulation had no impact. If it is based on something other than chance (i.e. it is significant), then I can conclude that there is something else at work here (probably the High versus Low dollar amounts from prior participants!). That is, we can answer the question “Did our manipulation work, with those in the High Dollar Condition saying that the prior respondents spent a higher amount of money than average on books while those in the Low Dollar Condition spent a lower amount of money than average on books?”

Let’s see how to run this in SPSS. Since this example involves two different variables with categorical levels, we will run a chi square test of independence. For the next section, I am going to open SPSS and run a chi square test of independence. I’ll use screenshots from SPSS as I go, but feel free to run these analyses yourself. Just set up your SPSS file like mine (I also included this SPSS file for you in Canvas if you prefer to use that. It is called “Crash Course Quiz #4– Textbook Money (Chi Square Practice)”, but it is a short data set so I recommend setting up your own SPSS file using the values from the table above). I am just going to give you the basics here, but you can refer to other sources to figure out some of the info we get from the chi square not covered in this lecture).

SPSS – Our Textbook Money Study

1. Click Analyze > Descriptive Statistics > Crosstabs... on the top menu as shown below.

You will be presented with the following (though note that I changed the “Did you spend more” variable into a nominal variable, as denoted by the symbol. I also chose to display the dependent variable name as “HigherLower” rather than the mouthful phrase “Did others spend higher or lower than average”):

Transfer one of the variables into the "Row(s):" box and the other variable into the "Column(s):" box. In our example we will transfer the "DollarCondition" variable into the "Row(s):" box and "HigherLower" into the "Column(s):" box. There are two ways to do this. You can highlight the variable with your mouse and then use the button to transfer the variables or you can drag-and-drop the variables. How do you know which variable goes in the row box and which goes in the column box? There is no right or wrong way. It will depend on how you want to present your data, so feel free to try it either way.

If you want to display clustered bar charts, then make sure that "Display clustered bar charts" checkbox is ticked. I usually don’t need the chart, but if you want a visual aide about what the data looks like, the bar chart might help.

You will end up with a screen similar to the one below:

1. Click on the button. Select the "Chi-square" and "Phi and Cramer's V" options as shown below:

Click the button.

2. Click the button. Select "Observed" from the "Counts" area and "Row” from the "Percentages" area as shown below:

1. After clicking “Continue”, click the button to generate your output.

Output of the Independent Chi Square in SPSS

You will see several tables for the chi square, but ignore the case processing table.

Condition (1 = High, 2 = Low) * Money Spent (1 = Higher, 2 = Lower) Crosstabulation

The nice thing about this table is it tells you exactly what you ran in the name of the table!

As you can see, those in the High Dollar Condition chose the “Higher (than average)” option most frequently while those in the Low Dollar Condition chose the “Lower (than average)” option most frequently. But is this statistically significant? For that, we look to the next table.

Chi-Square Tests

Focus on the Pearson Chi-Square row (and ignore the others). Based on Pearson, we can see that the chi square is significant, χ2(1) = 4.27, p = .039. Keep in mind that our degree of freedom here is 1. We calculate the df by looking at the formula (k1 – 1) X (k2 – 1), where k refers to each variable. That is, k1 focuses on the two levels for Dollar Condition (which has 2 levels, High vs Low) and k2 focuses on the two levels for money spent (which also has 2 levels, Higher than average and Lower than average. Plug in those number of levels into our (k1 – 1) X (k2 – 1) formula, and we get (2 – 1) X (2 – 1), or 1 X 1 = 1.

Symmetric Measures

Our final table looks at symmetric measures. As you can see below, both Phi and Cramer’s V are very high (.516 on a 0 to +1 scale, which makes .516 pretty high). Both are significant at p = .039. However, we use the “Phi” row for designs like the one we just described (a 2 X 2 study design). Thus we would focus on that Phi row to assess our 2 X 2 design. Use Cramer’s V for all other chi squares (like a 2 X 3 design, or a 3 X 3 design).

Interpreting the Independent Chi Square in SPSS

If a significant result occurs, the write up looks like this:

A chi square test of independence was calculated comparing how much participants in the High Dollar Condition versus Low Dollar Condition thought others spent on average for books (higher than average versus lower than average). A significant relationship emerged, χ2(1) = 4.27, p = .039. Most participants in the High Dollar Condition (87.5%) thought other respondents spent more money than average on books while most participants in the Low Dollar Condition (62%) thought other respondents spent less money than average on books. Phi showed a large effect. This indicates that the High versus Low Dollar manipulation worked as intended.

If non-significant, the write up for the chi square is even easier. You simply write:

A chi square test of independence was calculated comparing how much participants in the High Dollar Condition versus participants in the Low Dollar Condition thought others spent on average for books (higher than average versus lower than average). No significant relationship was found, χ2(1) = 1.27, p > .05, and phi did not show a large effect. This indicates that the High versus Low Dollar manipulation did not work, as the frequency of answers were equally distributed among cells.

Another quick example – Three levels to your main independent variable.

Now, assume that we still had two levels to our independent variable of Dollar Condition: High, and Low. However, we add another level to our dependent variable, with the available options including “Higher than average”, “Lower than average”, and a new “Average” option. Consider our SPSS output (next page). For the data on the next page, we have the following interpretation:

A chi square test of independence was calculated comparing how much participants in the High Dollar Condition versus Low Dollar Condition thought others spent on average for books (higher than average vs. lower than average vs. average). A significant relationship emerged, χ2(2) = 8.57, p = .014. Most participants in the High Dollar Condition (75%) thought other respondents spent more money than average on books while most participants in the Low Dollar Condition (62.5%) thought other respondents spent less money than average on books. Cramer’s V showed a large effect. This indicates that the High versus Low Dollar manipulation worked as intended.

Note that our df changed a bit. We now have three conditions in k2, so (k1 – 1) X (k2 – 1) gives us (2 – 1) X (3 – 1), or 1 X 2 = 2. Also, note that the write up uses Cramer’s V rather than Phi (Phi is best used for a 2 X 2 table, but here we have a 2 X 3 table)

A final example – Three levels to your main independent variable and three levels to your independent variable

Finally, assume that we have three levels to our independent variable of Dollar Condition: High, Low, and Average. We also have three levels to our dependent variable, with the options including “Higher than average”, “Lower than average”, and “Just about average”. See Figure 1 for an example of the High Dollar Condition with three dependent variable answer options.

Figure 1: High Dollar Condition with Three Answer Options

If this 3 X 3 test is significant, we would conclude the following (based on the SPSS output below)

A chi square test of independence was calculated comparing how much participants in the High Dollar Condition versus Low Dollar Condition versus Average Dollar Condition thought others spent on average for books (higher than average vs. lower than average vs. average). A significant relationship emerged, χ2(4) = 19.79, p = .001. Most participants in the High Dollar Condition (75%) thought other respondents spent more money than average on books, most participants in the Low Dollar Condition (62.5%) thought other respondents spent less money than average on books, and most participants in the Average Dollar Condition (75%) thought other respondents spent an average amount on books. Cramer’s V showed a large effect. This indicates that the High versus Low versus Average Dollar manipulation worked as intended.

Notice that our df is now 4. Using (k1 – 1) X (k2 – 1), we now have (3 – 1) X (3 – 1), or 2 X 2 = 4

Crash Course In Statistics – The Chi Square – Quiz #4 (Summer 2021, Fake News)

Instructions: In your prior Crash Course Quizzes (#2 and #3), you focused on a study looking at a Facebook user who posted “Fake News”, and then responded to people who called him out for posting the fake news in either a rudely disagreeing manner, a politely disagreeing manner, or a politely agreeing manner. That is, in the “Rude Disagree” condition, the user got angry, calling those who said his post was fake “morons, losers, and uninformed idiots”, and rudely disagreed that he shared fake news (nice guy, huh!). In the “Polite Disagree” condition, the user still disagreed that he posted fake news, but he did so by politely disagreeing that he shared fake news (noting “I know you might have good intentions, but the news I posted is actually correct”). In the “Polite Agree” condition, he politely noted that he was incorrect and did, in fact, post fake news. Using this study, complete the questions below and transfer those answers to your Crash Course in Statistics – The Chi Square Quiz #4 in Canvas (1 point per question). IMPORTANT: The answer options on Canvas may not be in the same order you see them below, so make sure to copy over the CONTENT of the answer and not simply the answer letter (A, B, C, D, or E).

Chi Square Crash Course Quiz Part A: We conduct a similar study using the same two groups as our t-Test (Rudely Disagrees condition versus Politely Disagrees condition. For now, ignore the “Politely Agrees” condition). For our current analysis, we want to determine if participants are able to recall the manner in which the Facebook user responded. That is, participants are asked to recall whether they Facebook user Rudely Disagreed that he posted fake news or whether he Politely Disagreed that he posted fake news. Note: If you want to run these analyses yourself, look for the SPSS file called “Chi Square Crash Course Data – Fake News A” in Canvas – Running the analysis is not required as the data are presented below, but it is definitely recommended if you want some SPSS practice!). You get the following data:

1). How many participants in the Rudely Disagrees condition recalled the Facebook user rudely disagreeing, and how many participants in the Politely Disagrees condition recalled the Facebook user politely disagreeing?

A. A total of 47 participants (100%) in the Rudely Disagrees Condition recalled the Facebook user rudely disagreeing. A total of 47 participants (100%) in the Politely Disagrees Condition recalled the Facebook user politely disagreeing.

B. A total of 51 participants (54.3%) in the Rudely Disagrees Condition recalled the Facebook user rudely disagreeing. A total of 43 participants (45.7%) in the Politely Disagrees Condition recalled the Facebook user politely disagreeing.

C. A total of 27 participants (66%) in the Rudely Disagrees Condition recalled the Facebook user rudely disagreeing. A total of 31 participants (57.4%) in the Politely Disagrees Condition recalled the Facebook user politely disagreeing.

D. A total of 27 participants (57.4%) in the Rudely Disagrees Condition recalled the Facebook user rudely disagreeing. A total of 31 participants (66%) in the Politely Disagrees Condition recalled the Facebook user politely disagreeing.

E. A total of 31 participants (66%) in the Rudely Disagrees Condition recalled the Facebook user rudely disagreeing. A total of 27 participants (57.4%) in the Politely Disagrees Condition recalled the Facebook user politely disagreeing.

2). We used a chi square test to look at whether the Rudely Disagree or Politely Agree participants recalled seeing the appropriate response from the Facebook user (either “The user rudely disagreed that he shared fake news” or “The user politely disagreed that he shared fake news”), but can we use a t-Test for this analysis instead? Select the appropriate answer

A. Yes, we can run a t-Test. The t-Test relies on continuous variables, and the dependent variable here is continuous (scaled).

B. Yes, we can run a t-Test. Since the dependent variable here is a yes/no response (or a dichotomous or nominal variable), we a t-Test is appropriate to use

C. No, we cannot run a t-Test. Since the dependent variable here is a categorical-based response (or a dichotomous or nominal variable), we cannot run a t-Test. A chi square is more appropriate

D. No, we cannot run a t-Test. Since the dependent variable here is a continuous (scaled) response, we cannot run a t-Test. A chi square is more appropriate

E. There is not enough information in this study to decide if we can run a t-Test

3). Which of the following represents the correct way to write out the results for this chi square in an APA formatted results section?

A. A chi square test of independence was calculated to see if participants in the Rudely Disagree and Politely Disagree conditions recalled the Facebook user’s response to being called out for posting fake news (The user rudely disagreed versus The user politely disagreed, respectively). A significant relationship emerged, χ2(2) = 5.19, p < .05. In the Rude Disagree condition, 31 participants (66%) recalled seeing a rudely disagreeing post from the user while 27 participants (57.4%) in the Polite Disagree condition recalled seeing a politely disagreeing post from the Facebook user. Phi showed a strong effect. This indicates that participants paid attention to the study manipulation.

B. A chi square test of independence was calculated to see if participants in the Rudely Disagree and Politely Disagree conditions recalled the Facebook user’s response to being called out for posting fake news (The user rudely disagreed versus The user politely disagreed, respectively). A significant relationship emerged, χ2(1) = 5.19, p < .05. In the Rude Disagree condition, 31 participants (66%) recalled seeing a rudely disagreeing post from the user while 27 participants (57.4%) in the Polite Disagree condition recalled seeing a politely disagreeing post from the Facebook user. Phi showed a strong effect. This indicates that participants paid attention to the study manipulation.

C. A chi square test of independence was calculated to see if participants in the Rudely Disagree and Politely Disagree conditions recalled the Facebook user’s response to being called out for posting fake news (The user rudely disagreed versus The user politely disagreed, respectively). A significant relationship emerged, χ2(1) = 5.19, p < .05. In the Rude Disagree condition, 27 participants (57.4%) recalled seeing a rudely disagreeing post from the user while 31 participants (66%) in the Polite Disagree condition recalled seeing a politely disagreeing post from the Facebook user. Phi showed a strong effect. This indicates that participants paid attention to the study manipulation.

D. A chi square test of independence was calculated to see if participants in the Rudely Disagree and Politely Disagree conditions recalled the Facebook user’s response to being called out for posting fake news (The user rudely disagreed versus The user politely disagreed, respectively). A significant relationship did not emerge, χ2(1) = 5.19, p > .05. Although 31 participants (66%) in the Rude Disagree condition recalled seeing a rudely disagreeing post from the user and 27 participants (57.4%) in the Polite Disagree condition recalled seeing a politely disagreeing post from the Facebook user, condition recall did not differ significantly between conditions. This indicates that participants did not pay attention to the study manipulation.

E. A chi square test of independence was calculated to see if participants in the Rudely Disagree and Politely Disagree conditions recalled the Facebook user’s response to being called out for posting fake news (The user rudely disagreed versus The user politely disagreed, respectively). A significant relationship did not emerge, χ2(2) = 5.19, p > .05. Although 31 participants (66%) in the Rude Disagree condition recalled seeing a rudely disagreeing post from the user and 27 participants (57.4%) in the Polite Disagree condition recalled seeing a politely disagreeing post from the Facebook user, condition recall did not differ significantly between conditions. This indicates that participants did not pay attention to the study manipulation.

Chi Square Crash Course Quiz Part B: You design a new study in which you look at all three conditions (Rudely Disagree vs. Politely Disagree vs. Politely Agree). You also ask them if they recall the correct response from the Facebook user that accompanied their condition: “The user rudely disagreed”, “The user politely disagreed”, or a new “The user politely agreed” option that reflects what participants in the new “Politely Agreed” condition should recall. You get the following data (Note: This does differ from the tables used for questions 1, 2, and 3! If you want to run the data yourself, use the “Chi Square Crash Course Data – Fake News B” in Canvas – not required, but recommended!):

4). How many participants in the Rudely Disagree Condition recalled the Facebook user rudely disagreeing, how many in the Politely Disagree Condition recalled the Facebook user politely disagreeing, and how many in the Politely Agree condition recalled the Facebook user politely agreeing that he shared fake news?

A. A total of 47 participants (100%) in the Rudely Disagree Condition recalled the Facebook user rudely disagreeing, a total of 47 participants (100%) in the Politely Disagree condition recalled the Facebook user politely disagreeing, and a total of 29 participants (100%) in the Politely Agree Condition recalled the Facebook user politely agreeing that he shared fake news.

B. A total of 34 participants (72.3%) in the Rudely Disagree Condition recalled the Facebook user rudely disagreeing, a total of 29 participants (61.7%) in the Politely Disagree condition recalled the Facebook user politely disagreeing, and a total of 29 participants (61.7%) in the Politely Agree Condition recalled the Facebook user politely agreeing that he shared fake news.

C. A total of 4 participants (8.5%) in the Rudely Disagree Condition recalled the Facebook user rudely disagreeing, a total of 6 participants (12.8%) in the Politely Disagree condition recalled the Facebook user politely disagreeing, and a total of 29 participants (61.7%) in the Politely Agree Condition recalled the Facebook user politely agreeing that he shared fake news.

D. A total of 34 participants (72.3%) in the Rudely Disagree Condition recalled the Facebook user rudely disagreeing, a total of 29 participants (19.1%) in the Politely Disagree condition recalled the Facebook user politely disagreeing, and a total of 29 participants (8.5%) in the Politely Agree Condition recalled the Facebook user politely agreeing that he shared fake news.

E. A total of 55 participants (39%) in the Rudely Disagree Condition recalled the Facebook user rudely disagreeing, a total of 47 participants (33.3%) in the Politely Disagree condition recalled the Facebook user politely disagreeing, and a total of 39 participants (27.7%) in the Politely Agree Condition recalled the Facebook user politely agreeing that he shared fake news.

5). Which of the following represents the correct way to write out the results for this chi square in an APA formatted results section?

A. A chi square test of independence was calculated comparing whether participants in the Rudely Disagree, Politely Disagree, and Politely Agree conditions recalled the correct Facebook user response (The user rudely disagreed, the user politely disagreed, or the user politely agreed, respectively). A significant relationship did not emerge, χ2(4) = 67.04, p > .05. In the Rudely Disagree Condition, 34 participants (72.3%) recalled the Facebook user rudely disagreeing. In the Politely Disagree Condition, 29 participants (61.7%) recalled the Facebook user politely disagreeing. Finally, in the Politely Agree Condition, 29 participants (61.7%) recalled the Facebook user politely agreeing. Cramer’s V showed a strong effect. This indicates that participants did not pay attention to the study manipulation.

B. A chi square test of independence was calculated comparing whether participants in the Rudely Disagree, Politely Disagree, and Politely Agree conditions recalled the correct Facebook user response (The user rudely disagreed, the user politely disagreed, or the user politely agreed, respectively). A significant relationship emerged, χ2(1) = 67.04, p > .05. In the Rudely Disagree Condition, 34 participants (72.3%) recalled the Facebook user rudely disagreeing. In the Politely Disagree Condition, 29 participants (61.7%) recalled the Facebook user politely disagreeing. Finally, in the Politely Agree Condition, 29 participants (61.7%) recalled the Facebook user politely agreeing. Cramer’s V showed a strong effect. This indicates that participants did not pay attention to the study manipulation.

C. A chi square test of independence was calculated comparing whether participants in the Rudely Disagree, Politely Disagree, and Politely Agree conditions recalled the correct Facebook user response (The user rudely disagreed, the user politely disagreed, or the user politely agreed, respectively). A significant relationship emerged, χ2(4) = 67.04, p < .001. In the Rudely Disagree Condition, 29 participants (61.7%) recalled the Facebook user rudely disagreeing. In the Politely Disagree Condition, 29 participants (61.7%) recalled the Facebook user politely disagreeing. Finally, in the Politely Agree Condition, 34 participants (72.3%) recalled the Facebook user politely agreeing. Cramer’s V showed a strong effect. This indicates that participants paid attention to the study manipulation.

D. A chi square test of independence was calculated comparing whether participants in the Rudely Disagree, Politely Disagree, and Politely Agree conditions recalled the correct Facebook user response (The user rudely disagreed, the user politely disagreed, or the user politely agreed, respectively). A significant relationship emerged, χ2(4) = 67.04, p < .001. In the Rudely Disagree Condition, 34 participants (72.3%) recalled the Facebook user rudely disagreeing. In the Politely Disagree Condition, 29 participants (61.7%) recalled the Facebook user politely disagreeing. Finally, in the Politely Agree Condition, 29 participants (61.7%) recalled the Facebook user politely agreeing. Cramer’s V showed a strong effect. This indicates that participants paid attention to the study manipulation.

E. A chi square test of independence was calculated comparing whether participants in the Rudely Disagree, Politely Disagree, and Politely Agree conditions recalled the correct Facebook user response (The user rudely disagreed, the user politely disagreed, or the user politely agreed, respectively). A significant relationship emerged, χ2(1) = 67.04, p < .001. In the Rudely Disagree Condition, 34 participants (72.3%) recalled the Facebook user rudely disagreeing. In the Politely Disagree Condition, 29 participants (61.7%) recalled the Facebook user politely disagreeing. Finally, in the Politely Agree Condition, 29 participants (61.7%) recalled the Facebook user politely agreeing. Cramer’s V showed a strong effect. This indicates that participants paid attention to the study manipulation.