Discussion

In all conditions, we will tell participants that we want them to read and rate a new AI policy that FIU is thinking of adopting that focuses on the student use of AI (like Chat GPT). All participants will read the proposed policy and rate how fair and respectful the policy is as well as their intentions to follow the policy. The twist here is that while the basic information in the AI policy is identical across conditions, the language tone differs between three study conditions. One condition presents the basic AI policy information in a neutral language, using neutral words like “appropriate” and “allowed”. A second condition, however, employs high controlling language, using harsh words like “prohibited” and “must never use”. Our third condition employs low controlling language, using words like “encouraged” and “should”. The main research question is whether participants will experience more reactance in the high controlling language condition compared to the low controlling and neutral language conditions.

1). For your first experimental study, you will play the role of researcher, and you will collect data from three different participants. There are two phases to this study. In the first phase, you will orally ask participants if they are willing to participate in a research study. In the second phase, participants will complete a multi-part survey. First, they will read an FIU proposed “AI Policy” that governs how and when students can use AI programs like Chat GPT. Then they provide ratings about the policy, indicating how they feel about it and whether they would follow the policy guidelines. Finally, they will provide demographic information and complete a manipulation check question. As you collect data from participants, use the following steps:

A). Your first task is to approach three different participants (not all at the same time!). Preferably, they will be people that you do not know and did not take a psychology research methods class during the Spring or Summer semesters of 2023 or the Fall 2023 semester. Please DO NOT complete this study yourself. If possible, use only FIU students as participants (no family / friends. You will use them in a later replication study at the end of the term, and they cannot participate twice). There are usually around 48 students for this class section, so with each student collecting data from 3 people, our final sample will be around 140 participants total.

1). Note that there is a “Covid alternative” to data collection if you are unable to collect data yourself. Ask your instructor about that option, but there is a good chance that you will already see some pre-completed “Covid Alternative” documents in Canvas.

2). Even if you use the “Covid Alternative”, read the information below, as it will help you write your future papers. You do not need to mention that you used the Covid Alternative, but you will pretend like you collected participant survey responses yourself.

B). Phase I: Informed Consent

1). Informed Consent:

· Ask the potential participant if he or she is willing to participate in a study for your research methods class. You will get their informed consent verbally. Tell them:

“Hello, I am conducting a study for my research methods class. I was wondering if you would be willing to participate. The study takes about five to ten minutes. There are no risks to participating, and the main benefit is that I can complete my class assignment. Will you participate?”

· An oral Yes or No response is fine. If they say no, thank them and find a different participant. If they say yes, move to the next step ( Phase II – Questionnaire).

C). Phase II: “Questionnaire”

1). General Instructions

· After getting participant’s oral informed consent, randomly give them ONE of the three “Research Study – Florida International University – Spring 2024” documents. These three documents contain our independent variable for the study. Note an “independent variable” is the variable that the researchers control. We have one independent variable with three levels in our study, with each survey representing one of the three levels/conditions for that single independent variable. One third of our participants will be in the “High Controlling Language” condition, one third will be in the “Low Controlling Language” condition, and one third will be in the “Neutral Language” condition (You will read more about these conditions below!).

· Ask participants to follow the instructions at the top of the questionnaire. Tell them that they can go at their own pace. Make sure they complete all questionnaire parts (though note that they can leave some demographic questions blank in Part D if they do not want to provide those details).

2). Questionnaire

· Study Instructions

· In this section, all participants read the same instructions about the general purpose of the study and the tasks they will complete. They will read:

· Part A: FIU Generative Artificial Intelligence (AI) Policy (e.g. Chat GPT)

· Following the initial instructions, participants will read the proposed AI policy. Although the basic informational content of the policy is similar across the three different study conditions, the tone with which the policy is written differs, with some versions having more “controlling language” than other versions.

· In the “Low controlling language” condition, participants read an AI policy that contains language that is typically implicit and polite, using qualifiers such as “choose” or “should,” which suggest free choice (e.g. “AI is discouraged” or “the FIU faculty wants to help you understand when using AI is encouraged versus discouraged.”).

· In the “ Neutral Language” condition, participants read an AI policy that contains neutral language that only imparts information (e.g. “AI is inappropriate” or “the FIU faculty want to make sure you know when using AI is either appropriate or inappropriate”).

· Note #1: You can see the three different versions of the survey in the appendix at the end of this document (which better highlights how the language differs). Just keep in mind that the appendix is not the actual surveys – download the “Main Questionnaire” documents for the full survey versions to give to participants you recruit.

· Note #2: I think that participants will have higher levels of reactance (more freedom threat), more anger, and will be more likely to disregard the AI policy in the “High Controlling Language” condition than in both the “Low Controlling Language” and “Neutral Language” conditions. While I suspect the Neutral Language condition will lead to higher levels of reactance than the Low Controlling Language condition, I am not sure if neutral versus encouraging language will lead to differences between these two conditions. That’s why we do research! For now, I will just equate the two.

· Then ALL participants go to page two where they provide ratings (dependent variables) about the AI policy.

· Theme #1: Reactance – Several statements focus on reactance itself, such as Statement #1 (“The AI policy gives me too little freedom to decide how I can use AI”), Statement #3 (“The AI policy threatens my freedom to choose how I complete my assignments”), and Statement #4 (“The AI policy is not respectful of my right to make my own decisions”).

· Theme #2: Feelings – Several statements focus on how the policy made the participant feel, such as Statement #6 (“The AI policy made me feel angry”) and Statement #7 (“The AI policy made me feel valued”).

· Theme #3: Controlling Language – Several statements focus on whether the AI policy language was controlling, such as Statement #8 (“The AI policy language is too extreme”) and Statement #9 (“The AI language is demanding/bossy”).

· Additional statements cross themes, like Statement #2 (“The AI policy provides helpful AI guidance”), Statement #5 (“The AI policy impressed me”), and Statement #10 (“The AI policy seems fair and reasonable”).

· Data Analysis Note:

· Each of these ten statements is a “dependent variable”, or a response. We have a separate hypothesis for each DV but will only analyze a few of them

· We will analyze some of these variables using a One-Way ANOVA since there are three levels / groups in the single independent variable and all of these dependent variables are interval scales. We will discuss this more for Paper Two later this semester!

· After completing Part B, all participants move to Part C

· In this section, participants rate four statements using the same interval scale of 1 (Strongly Disagree) to 7 (Strongly Agree). These statements focus on participant intentions to follow the AI Policy

· The statements focus on intentions to follow the AI Policy when it is allowed (Statement 1) or when it is not allowed (Statement 2). Other statements focus on their intention to ignore the policy (Statement 3) or use their own judgment (Statement 4).

· Data Analysis Note:

· Like Part B, we will not analyze all these statements, but they will help us assess participant behavioral intentions.

· Like Part B, I think participants will rate themselves as more likely to follow the policy in the Low Controlling and Neutral Language conditions compared to the High Controlling Language condition.

· Part D: Demographic Information

· In this section, participants provide their demographic information in questions 1 through 4. Although most of these items are easy to complete without violating participant’s privacy, let them know they are free to leave blank any questions they do not wish to answer.

· In question 5, we ask participants to recall the tone of the AI policy they read. This is an important question, as it provides our official study manipulation check (to see if participants paid attention to our manipulation).

· Data Analysis Note:

· We report the mean and standard deviation (or SD) for age in Question 2 and descriptive statistics for gender and race/ethnicity (for Questions 1 / 3)

· For Question 5, we will use a chi square to see if participants paid attention to the tone of the policy. That is, we asked participants to complete the following: “The AI policy noted that there are times when students should not use AI, but what specific term or language did the AI policy use making this request (Select one)?”

· We want participants in the High Controlling Language condition to select the “Using AI is prohibited” option. Participants in the Low Controlling Language condition should select the “Using AI is discouraged” option. Participants in the Neutral Language condition should select the “Using AI is inappropriate” option.

A quick note for you (the researcher): If you look at the bottom of the questionnaire on the first page, you will see the letters HC, LC, and NL. Those relate to the condition for that specific survey. The HC stands for High Controlling, the LC stands for Low Controlling, and the NL stands for Neutral Language. These initials will help YOU quickly know which survey is which, though the initials should be meaningless for participants. No need to mention this in your future papers, but it is a good way to keep track of your surveys

D). Once participants complete the questionnaire, debrief them regarding the Psychological Reactance Theory study. That is, tell them about the study and hypothesis. Read the following:

“This study investigates “Psychological Reactance Theory”, or reactance. Reactance involves a person’s motivation to protect or restore a lost sense of freedom, which may involve doing the very thing they are told not to do! For example, if you tell someone to not sign a petition, they might sign it simply to rebel against your order. As a real-life example, telling people that they had to wear a mask during the Covid pandemic led some people to rebel and refuse to wear a mask, often angrily saying: “You can’t tell me what to do.” High-controlling language can lead to reactance. Phrases like “You must adhere to this policy” or “These actions are prohibited” can lead people to feel like they have no choice (and thus no freedom) compared to low-controlling language like “You might consider the benefits of the policy” or “These actions are encouraged”).

In our study, we asked participants to read a proposed Artificial Intelligence policy designed to guide how and when students at FIU use AI (like Chat GPT) for their assignments. While all participants read the same content for the proposed policy, we altered the language and tone of the policy across three study conditions. Participants in our high-controlling language condition read an AI policy with harsh and controlling language (such as telling participants that they MUST follow the AI policy, which “permitted” AI in some situations but “prohibited” it in others). Participants in our low-controlling language condition read an AI policy that used more encouraging language (such as telling participants that they would benefit from following the AI policy, which “encouraged” AI in some situations but “discouraged” it in others. This condition also highlighted participant choice in using AI). Participants in our neutral language condition read an AI policy that included information only (such as telling participants when it was appropriate or inappropriate to use AI).

Based on Psychological Reactance Theory, we have several predictions in this study.

First, if participants read an AI policy that uses high controlling language, then they will more strongly agree that a) the policy gives them too little freedom to decide how to use AI, b) the policy threatens their freedom to choose how to complete assignments, and c) the policy is not respectful of their right to make their own decisions when compared to participants who read an AI policy with either low controlling or neutral language, which will not differ from each other.

Second, if participants read an AI policy that uses high controlling language, then they will more strongly agree that a). the policy made them angry, b) the policy was too extreme, and c) the policy language was demanding when compared to participants who read an AI policy with either low controlling or neutral language, which will not differ from each other.

Finally, if participants read an AI policy that uses high controlling language, then they will more strongly agree that a) they intend to use AI even in situations where the AI policy does not allow them to use it, b) they intend to ignore the AI policy, and c) they intend to use AI when they deem is appropriate when compared to participants who read an AI policy with either low controlling or neutral language, which will not differ from each other.

Once again, thank you for participating in this study!

**Methods Students: The underlined paragraph above will be helpful when you write Paper I! In fact, you can use the underlined paragraphs in your first paper if you like (literally copy and paste them into your literature review, though feel free to rephrase or revise them as you see fit). However, the predictions ARE NOT INCLUDED in your minimum page count. That is, you can copy/paste the predictions, but they do not count in the page minimum!

Please note that we have 14 different dependent measures in this study from Parts B and C (excluding variables in the demographic section), which means that we have 14 different hypotheses, one for each DV. For your Paper Two analysis, you only need to look at two dependent variables / predictions. Of course, the most important dependent variables are in Part B and Part C. As such, you can look at any of the 14 statements in those sections. My advice is to focus on DVs that you find interesting but also help you support (or reject) your predictions. For example, in Part B you might want to focus on only one of the ten statements (I suggest either statement 1, 3, or 4 – choose your favorite from those three options as they are all good and overlap!). For Part C, I suggest focusing on either statement 2, 3, or 4 – but my suggestion is statement 2. That gives you a lot of flexibility in writing both your literature review and your eventual methods / results section.

2). Hold onto the completed questionnaires, as you will need them in an upcoming lab. You will enter data into SPSS and analyze it during your lab. Important note: Each student researcher is responsible for collecting data from three participants (one participant for each study condition – High controlling language, Low controlling language and Neutral language). However, we will combine survey data from each student in your lab section, so your final sample will include at least 100 to 140 or so participants. In your papers (especially Paper II), you will use this total set of research participants (at least 100), NOT just the three participants that you collected yourself. Don’t even discuss “Three participants”, as that is not correct. Discuss ALL 140 participants in your study.

3). Pay close attention to these instructions! You can use them as the basis for Paper II later this semester when you discuss your methods section. That being said, these instructions are too long for a methods section, and includes information you will need to omit for Paper II. When writing that paper, make sure to only report the important aspects (what you actually did in the study). Write about what you actually did in the study!

4). ALTERNATIVE PREDICTIONS: Below are alternative predictions broken down into the three main areas of interest to us (reactance, feelings, and intentions). They get a bit long for the debriefing statement, so I included a briefer version there. But for your Paper I predictions, you are welcome to use/alter those below if you prefer them, though you are also welcome to use the ones present in the debriefing statement. Your study one analyses will focus on two specific dependent variables, so you do not need all of predictions from below. In fact, I suggest you look at one DV from the first set of predictions (either a, b, or c for reactance) and one DV from the third set of predictions (either a, b, or c for intent).

Our first set of predictions focuses on reactance. We predict that if participants read an AI policy that uses high controlling language, then they will experience more reactance when compared to participants who read an AI policy with either low controlling or neutral language, which will not differ from each other. Specifically, participants in the high controlling language condition will more strongly agree that a) the policy gives them too little freedom to decide how to use AI, b) the policy threatens their freedom to choose how to complete assignments, and c) is not respectful of their right to make their own decisions when compared to participants in the low controlling and neutral conditions.

Our second set of predictions focuses on participant feelings about the AI policy. We predict that if participants read an AI policy that uses high controlling language, then they will experience more negative feelings about the policy compared to participants who read an AI policy with either low controlling or neutral language, which will not differ from each other. Specifically, high controlling language participants will more strongly agree that a). the policy made them angry, b) the policy was too extreme, and c) the language was demanding when compared to participants in the low controlling and neutral conditions.

Our final set of predictions focuses on participant intentions to follow the AI policy. We predict that if participants read an AI policy that uses high controlling language, then they will have lower intentions to follow the policy compared to participants who read an AI policy with either low controlling or neutral language, which will not differ from each other. Specifically, high controlling language participants will more strongly agree that a) they intend to use AI even in situations where the AI policy does not allow them to use it, b) they intend to ignore the AI policy, and c) they intend to use AI when they deem is appropriate when compared to participants in the low controlling and neutral conditions.

Appendix – Language differences in the three surveys (highlighted in bold)

At Florida international University (FIU), we believe that writing is integral to thinking. We also know that writing is hard. Recent technological advances in Artificial Intelligence (AI) like Chat GPT are useful tools that can help students improve both their writing and stimulate their thinking. However, AI must NEVER serve as a substitute for either skill.   

Think about the help you get from AI as a less sophisticated version of the assistance that you can receive (for free!) from the FIU Center for Excellence in Writing. An FIU writing tutor might help you in a wide variety of ways, like asking you questions to help jump start your imagination, or steering you away from poor grammar, or helping you to identify a poorly organized paragraph. While a tutor can be helpful, they must not do your writing for you! The help you get from AI is similar, though there are key differences. AI is not a person. It is a machine that is adept at recognizing patterns and reflecting those patterns back at you. It will not think for itself, and it must not think for you. It often makes mistakes, sometimes even inventing facts or providing a biased interpretation of reality. Given both the benefits and problems associated with AI, the FIU faculty must be sure that you know when using AI is permitted versus prohibited.

Permitted uses of AI ( AI is allowed ONLY for the following items)

· AI is permitted for checking spelling, grammar, or providing synonyms (similar to the support you can receive when using Grammarly or Microsoft Word).

· AI is permitted for rephrasing sentences or reorganizing paragraphs that you have drafted yourself. 

· AI is permitted for tweaking outlines or paper content that you have personally written.

· AI is permitted to help you brainstorm ideas for your writing project.

Prohibited uses of AI ( AI is NEVER allowed for the following items)

· AI is prohibited when it is uncited (Since it is not your idea, using it is plagiarism).

· AI is prohibited for use on assessments (quizzes, tests, examinations, etc.)

· AI is prohibited for paper content that should be original to you.

· AI is prohibited if it violates copyright or intellectual property laws.

Evidence of prohibited AI use will result in an Academic Integrity report, with sanctions ranging from a zero for the assignment to an F for the course. At FIU, our goal is to train you how to write and think, and strict adherence to these AI policies will accomplish those goals.

At Florida international University (FIU), we believe that writing is integral to thinking. We also know that writing is hard. Recent technological advances in Artificial Intelligence (AI) like Chat GPT provide useful tools that can help students improve both their writing and stimulate their thinking. However, AI is a poor substitute for learning these skills.   

Think about the help you get from AI as a less sophisticated version of the assistance that you can receive (for free!) from the FIU Center for Excellence in Writing. An FIU writing tutor might help you in a wide variety of ways, like asking you questions to help jump start your imagination, or steering you away from poor grammar, or helping you to identify a poorly organized paragraph. While a tutor can be helpful, they let you do your own writing! The help you get from AI is similar, though there are key differences. AI is not a person. It is a machine that is adept at recognizing patterns and reflecting those patterns back at you. Since it cannot think for itself, you should probably avoid letting it think for you. It often makes mistakes, sometimes even inventing facts or providing a biased interpretation of reality. Given both the benefits and problems associated with AI, the FIU faculty wants to help you understand when using AI is encouraged versus discouraged.

Encouraged uses of AI ( AI is encouraged for the following items)

· AI is encouraged for checking spelling, grammar, or providing synonyms (similar to the support you can receive when using Grammarly or Microsoft Word).

· AI is encouraged for rephrasing sentences or reorganizing paragraphs that you have drafted yourself. 

· AI is encouraged for tweaking outlines or paper content that you have personally written.

· AI is encouraged to help you brainstorm ideas for your writing project.

Discouraged uses of AI ( AI is discouraged for the following items)

· AI is discouraged when it is uncited (Since it is not your idea, it may be plagiarism).

· AI is discouraged for use on assessments (quizzes, tests, examinations, etc.)

· AI is discouraged for paper content that should be original to you.

· AI is discouraged if it violates copyright or intellectual property laws.

Evidence of discouraged AI use might result in an Academic Integrity report, with sanctions ranging from a zero for the assignment to an F for the course. At FIU, our goal is to train you how to write and think, and we hope that you choose to follow these AI policies and help us accomplish those goals.

Neutral Language Version (403 words)

At Florida international University (FIU), we believe that writing is integral to thinking. We also know that writing is hard. Recent technological advances in Artificial Intelligence (AI) like Chat GPT provide useful tools that can help students improve both their writing and stimulate their thinking. However, AI does not serve as a substitute for either skill.   

Think about the help you get from AI as a less sophisticated version of the assistance that you can receive (for free!) from the FIU Center for Excellence in Writing. An FIU writing tutor might help you in a wide variety of ways, like asking you questions to help jump start your imagination, or steering you away from poor grammar, or helping you to identify a poorly organized paragraph. While a tutor can be helpful, they do not do your writing for you! The help you get from AI similar, but there are key differences. AI is not a person. It is a machine that is adept at recognizing patterns and reflecting those patterns back at you. It cannot think for itself, nor can it think for you. It often makes mistakes, sometimes even inventing facts or providing a biased interpretation of reality. Given both the benefits and problems associated with AI, the FIU faculty wants to make sure you know when using AI is appropriate versus inappropriate.

Appropriate uses of AI  

· AI is appropriate for checking spelling, grammar, or providing synonyms (similar to the support you can receive when using Grammarly or Microsoft Word).

· AI is appropriate for rephrasing sentences or reorganizing paragraphs that you have drafted yourself. 

· AI is appropriate for tweaking outlines or paper content that you have personally written.

· AI is appropriate to help you brainstorm ideas for your writing project.

Inappropriate uses of AI  

· AI is inappropriate when it is uncited (Since it is not your idea, it constitutes plagiarism).

· AI is inappropriate for use on assessments (quizzes, tests, examinations, etc.)

· AI is inappropriate for paper content that should be original to you.

· AI is inappropriate if it violates copyright or intellectual property laws.

Evidence of inappropriate AI use may result in an Academic Integrity report, with sanctions ranging from a zero for the assignment to an F for the course. At FIU, our goal is to train you how to write and think, and these AI policies are designed to achieve those goals.