assig 9

Page

of 6

Pages

Assignment #9 – Identify the Threat to Internal Validity Using MR SMITH ID – Fall, 2022 5 Points Possible – Smartphone-Delivered Mindful Eating Intervention Instructions: Below is a description of a study. Read over the description in Part One. In Part Two, read variations of the study and determine which (if any) MR SMITH ID threat is present. PART ONE: Study Description Healthy eating is always a struggle in the United States, especially with cheap (but unhealthy) food so readily available. As a result, numerous diets are available that claim that they can help people better regulate their eating. One such diet involves “mindfulness”, which avoid being too judgmental when it comes to eating but rather teaches the dieter to be curious about what, when, and why they are craving food. Such mindfulness eating diets work by drawing the dieter’s awareness to their eating habits through three steps: 1) Increasing the person’s awareness of triggers that may lead them to eat (e.g. What emotions are they experiencing, Who they are with, What foods are available, etc.); 2) Having the dieter think about the outcomes that follow from their behavior (e.g. Will eating these cookies really make me feel better?), and 3) Having the dieter learn to live with their cravings rather than trying to ignore them. While mindfulness dieting does show promise, how to deliver the lessons of mindfulness diets to participants can vary dramatically. Such lessons can be delivered in person (as in a dieting class), though literature from a doctor or dietician, through websites, and through weight support groups, to name just a few. But one method that might be very beneficial is delivering mindfulness lessons through a smartphone app. After all, most Americans have access to a smartphone, so why not use it to help them with dieting! There are a lot of benefits to the smartphone approach. Users can access the app anywhere at any time without being tied to a computer screen. A mobile phone avoids hassles involving time, travel, and scheduling commitments. Set-up: Imagine you design a smartphone app that presents users with a mindfulness diet program that encourages them to become aware of the “what, when, and why” questions surrounding their current eating habits. You design a series of short modules (5 to 10 minutes) that provide dieters with information about mindfulness and their diets. These modules include animations, short lectures, or basic information about how to be mindful. You ask some participants to login to a computer website to review the module information (control group). You ask other participants to use a smartphone app that includes the same modules (experimental group). Before the study officially begins, you gather a wide variety of pre-study information about each participant. This includes information about how often they cave into cravings, self-reporting on why they eat (to cope with emotions, when socializing, when merely hungry, etc.), whether they binge eat, their starting weight, etc. These measures are also taken at the conclusion of the study. The study design looks like the following: Smartphone Intervention (Experimental Group) Pre-study questionnaire Post-study questionnaire Website Intervention (Control Group) You predict that dieters in the Smartphone group will benefit more (e.g. lose more weight, be more “mindful”, etc.) from the intervention than those in the Website group.

Identify which internal validity threat is most likely at work in each of the following Part Two statements. You will receive .5 points for each correct response. Note that each threat should be used only once, with one statement of the ten statements not having a threat. Also keep in mind internal validity threats may overlap – in these instances, choose the BEST threat for the scenario. Also note that the question order and answer order might differ in Canvas, so make sure to focus on the CONTENT of the question and answer rather than their order. PART TWO: Spot the Internal Validity Threat that is MOST likely at work for each of the following scenarios: Use the key below to match the study validity description with the specific validity threat. a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat 1). After completing the pre-study questionnaire, all dieters were randomly assigned to either the Website intervention or the Smartphone intervention. Once the dieters were assigned to their condition, they did not socialize with other dieters, which was easy to control as dieters completed all study questions remotely. To adequately assess the effectiveness of the intervention programs, the researchers made sure to use the exact same questions for the pre- and post-study questionnaires, including information about how often they cave into their cravings, self-reporting on why they eat (to cope with emotions, when socializing, when merely hungry, etc.), whether they binge eat, their starting weight, and other items. Some of the dieters even commented that they recalled seeing the same post-study questions before on the pre-study questionnaire and that they based their post-study answers on their pre-study answers. After comparing the post-exposure scores for the two conditions, you found no differences between students in the Website intervention and the Smartphone intervention. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat 2). After seeing the pre-study questionnaire scores, you were concerned about dieters who were morbidly obese, and you had heightened medical concerns about these dieters. To make sure such high-risk dieters would get the greatest benefit from your study, you made sure to

assign them to the Smartphone intervention while dieters with lower weights were randomly assigned to either the Smartphone or Website interventions. After comparing the post-study scores for the two conditions, you found that dieters in the Smartphone intervention lost significantly more weight and had heightened mindfulness scores compared to those in the Website intervention. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat 3). To compare the effectiveness of the Smartphone intervention compared to the Website intervention, you kept careful records of how often dieters logged in for the modules on both platforms throughout the month-long study. You noticed that while 90% of dieters in the Smartphone intervention logged in at least once per day throughout the study, only 60% of participants in the Website intervention logged in at least one per day throughout the study. You found that dieters in the Smartphone intervention lost more weight and were more aware of why, when, and how they experienced cravings than dieters in the Website intervention. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat 4). As part of the pre-study questionnaire, you asked participants about their prior dieting experiences. Reasoning that the dieters with previous diet failures might benefit more from the Smartphone intervention, you make sure that those who failed at dieting at least five times were placed in the Smartphone intervention. You found that dieters in the Smartphone intervention benefited more from the mindfulness diet than those in the Website intervention. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing

g. History h. Interactions i. Diffusion j. No Threat 5). Despite being assured that the Smartphone and Website modules were ready to run, you found that there were a lot of glitches with the Website intervention program. It was often sluggish, loading videos and images at a very slow rate. Dieters in this condition often needed to logout and then log back in to get the Website program to work. Dieters in the Smartphone intervention did not have the same glitches, with all dieters in that condition reporting that the modules were very smooth. You found that dieters in the Smartphone intervention benefited more from the mindfulness program than those in the Website intervention. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat 6). About two weeks into the study, a new TV program started airing in town. The program targeted potential dieters using entertaining videos to teach dieters concepts like being aware of the source of their cravings or deciding whether the consequences of eating unhealthy food made eating such food worthwhile. Many of the dieters in your study noted that they became aware of the TV program and decided to check it out since it seemed to overlap with what they were learning in your study. However, the show mostly appealed to women in your study, who raved about the program. You found that there was no difference between female dieters in the Smartphone intervention and female dieters in the Website intervention, though male dieters seemed to benefit more from the Smartphone intervention than the Website intervention. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat 7). After completing the pre-study questionnaire, all dieters were randomly assigned to either the Website intervention or the Smartphone intervention. Once the dieters were assigned to

their condition, they did not socialize with other dieters, which was easy to control as dieters completed all study questions remotely. Although pre-study measures and post-study measures had a lot in common in terms of content area (including an emphasis on being mindful), the questions on the post-study questionnaire differed from the pre-study questionnaire. The researchers found that dieters in the Smartphone intervention benefited more from the modules than dieters in the Website intervention. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat 8). About two weeks into the study, a new TV program started airing in town. The program targeted potential dieters using entertaining videos to teach dieters concepts like being aware of the source of their cravings or deciding whether the consequences of eating unhealthy food made eating such food worthwhile. Many of the dieters in your study noted that they became aware of the TV program and decided to check it out since it seemed to overlap with what they were learning in your study. You found that there was no difference between the Smartphone intervention and the Website intervention. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat

9). At the start of the study, you noticed that dieters in both the Smartphone condition and the Website condition had a tough time understanding how to access the modules and how to answer the questionnaires electronically. As the study progressed over the course of the month, they became more familiar and comfortable with the electronic platforms. You found no differences between dieters in the Smartphone intervention condition and the Website intervention condition. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat 10). Before the study began, a lot of the participants knew each other from their attendance in a weight loss support group. You did not know this when you randomly assigned them to the Smartphone or Website interventions. Outside of the study, the participants continued get together for support, and they started to compare notes about their experiences in your study. Dieters in the Smartphone intervention learned about the Website intervention and vice-versa. Since they often met in public and thus did not have access to a computer, they checked out the Smartphone app instead. Thus, dieters in both the Website intervention and the Smartphone intervention saw information in the Smartphone app. After comparing the post- study results, you find no difference between the Smartphone intervention and the Website intervention. Which internal validity threat is most likely occurring in this scenario? a. Maturation b. Regression to the mean c. Selection d. Mortality e. Instrumentation f. Testing g. History h. Interactions i. Diffusion j. No Threat