just paraphrase

Assignment #2

Experimental, quasi-experimental, and ex post facto designs

1. A researcher wants to investigate whether college students’ knowledge of the negative consequences of the overuse of credit will affect their attitudes about credit use. He plans to recruit one family studies class of 30 students from a local college campus (already enrolled in one class) and measure their attitudes toward credit with an attitude scale. Then, on four successive days, the researcher will teach them four lessons on the consequences of the overuse of credit: wage garnishment, repossession, foreclosure, and bankruptcy. On the fifth day, the researcher will administer an alternate form of the credit attitude scale (one intended to measure the same attitude). Then, he will try to infer whether college students’ knowledge of the negative consequences of the overuse of credit affects their attitudes about credit use.

a.. What is the independent variable and what is the treatment that the researcher will use to manipulate that variable?

The independent variable is the college student's knowledge of the negative consequences of overusing credit.

The treatment is educating students on the negative consequences of overusing credit including lessons on wage garnishment, foreclosure, repossession and bankruptcy .

b. What is the dependent variable in this study?

The dependent variable is the change in the college students attitudes towards overusing credit.

c. What type of research design is this researcher planning to use? Diagram it.

The researcher may use pre-Experimental Design, One Group Pretest Posttest study.

Measure Baseline Administer Program Measure Outcome

O X O

d. State the most likely alternative hypothesis of this researcher.

When college students increase their knowledge of the negative consequences of the overuse of credit, their positive attitude toward credit use will decrease.

e. What are the most important threats to the internal validity of the study? (Give an example of each threat in the language of the research problem.)

Selection- The effect may be due to nonequivalent subjects with different education levels, ages, and experience with credit history

Maturation: student’s attitude could change during the course of the experiment due to emotional or intellectual development between pre- and post test.

Mortality- Students could drop out between the pre- and post test.

Instrumentation- the effect on attitude may be due to the researcher using a different form of the credit attitude scale.

History- The students’ attitude on credit use changed as a result of an outside event such as seminar or personal experiences with creditors.

f. What two threats to internal validity are involved in this researcher’s plan to use an alternate form of the credit attitude scale? What is the trade-off?

Instrumentation threat is involved in this research, as the researcher’s plan to use an alternate form of the credit attitude scale. This means the effect on students’ attitude about credit use may be due to the researcher using a different form of the credit attitude scale. When the research uses different form of credit attitude scale, he or she would like to avoid testing threat, which can occur when the researcher uses the same form of scale in pre and post- test. If the research uses the same form of scale throughout the study, the testing threat may cause the student to be primed or be aware during the pre-test. As a result, the students’ performance in posttest might be due to the pre-test (methodological independent variable) instead of the treatment itself.

Trade- off: It is impossible for both instrumentation and testing threat exist in the same study. If the researcher wants to use the same credit attitude scale in pre-posttest to avoid instrumentation threat, then testing threat might occur in his or her study. In contrast, if the researcher wants to use different form of credit attitude scale in pre-posttest to avoid testing threat, then instrumentation threat might occur in his or her study.

g. Briefly describe the analysis this researcher would use to answer his research question (i.e., what would he compare to what?)

The researcher would compare the results from pre and post tests. Any difference, such as increase or decrease from posttest to pretest is possibly related to the treatment which is represented by X.

Suppose that instead of the design described, the researcher randomly assigned each of the 30 students to two groups. Suppose he did everything the same as before, except a teacher took one group of students on a field trip each of the four “treatment” days during their family studies class to a local historical site.

h. What type of research design is he now using? Diagram it.

Experimental Pretest-posttest control group design experimental.

R O X O

R O O

i. Does this design now have a classic “placebo” control group?

Yes. The control group is the group who was measure their attitudes toward credit with attitude scale before and after the event of field trip, but they were not brought to the field trip for “treatment”.

j. Compare and contrast these two research designs’ internal and external validity.

Threats to internal validity in these two designs include selection bias. It is possible that the groups are not formed equivalently . It could be possible that subjects in one have a characteristic that would affect the dependent variable. One of the threats to external validity would be pre-test treatment interaction. The subjects’ reactions to the treatment might be affected by their exposure to a pretest gaging their attitude toward credit. Furthermore, the experimental pretest-posttest control group design is better control of the threats to internal validity.

2. A research team wants to study the effect of handbag advertising on women’s choice of handbags. They are concerned about the potential effects of the women having any clue before the ads are shown about the content of the study (particularly the treatment). They randomly assign 100 women (19-22 years old) to four groups. They ask two groups of women to choose a handbag from a group of four handbags before the treatment. Two other groups will not be asked initially about their handbag choices. Of the first two groups, one group will be shown a television program containing 40-second handbag ads on one of the handbags and the other group will be shown the program without ads. Of the last two groups, one group will see the program with the ads and the other group without the ads. Then, each woman will be asked to choose a handbag from a set of four possible handbags (including the one in the ads). Then, the researchers will try to infer whether women shown the handbag advertisements choose different handbags from the women not shown the ads (i.e., whether advertising affects women’s choice of handbags).

a. What is the substantive independent variable in this study?

The advertisement of the handbags

b. What additional (methodological) independent variable is planned?

The pretest- having women choose handbags before the treatment.

c. What is the dependent variable in this study?

The women’s choice of handbag

(which means how women react to advertising and whether or not it has an an effect on their choice of handbags)

d. What research design is this research team planning? Diagram it.

Solomon Four Group Design

R O X O

R O O

R X O

R O

e. What are the threats to internal validity of this research design?

History - The women might choose a certain handbag due to a particular well-known fashion show instead of 40-seconds handbag ads.

Selection : The effect might be different due to the difference in the income levels of the women. Some might prefer more expensive brands than others based upon their income level.

Maturation: The women might choose a particular bag due to their maturation and growth. Some women might read different fashion magazines, watching different fashion shows, or discussing with their friends regarding fashion throughout their lifetime. All of these factors (intellectual/ emotional) might contribute to the women choose a particular bag, and this is not due to the 40-seconds handbag ads.

Testing: The women are asked to do the same task in pre- posttest, which is asking the women to choose a handbag from a set of four possible handbags. s. The women’s choice of handbag in posttest might be due to the pretest instead of the 40-seconds handbag ad

f. What is the third (methodological) null hypothesis in this study?

There is no effect of interaction between giving women a pretest (choosing handbags), when it comes to the choice they make in handbags during the posttest.

g. (extra credit) Briefly explain how this team would analyze their data to answer their research question and deal with their methodological concern, including the two different possibilities that could occur in the test of the third null hypothesis.

Group1 O X O

Group2 O O

Group3 X O

Group4 O

In order to observe the effect from pretest, the team should compare the posttest results from group 3 and 4 to the posttests result in group 1 and 2. Therefore, the researcher would know whether the posttest result is due to the effect from pretest.

In order to observe the interaction effect, the researcher has to compare the posttest results in group 1 and 2, group 2 and 3, group 3 and 4, etc.

3. A researcher wants to investigate whether college students’ knowledge of the negative consequences of the overuse of credit will affect their attitudes about credit use. He plans to recruit two professors to help him in his research. One professor of family and consumer sciences is planning a unit of four lessons of credit, such as wage garnishment, repossession, foreclosure, and bankruptcy. The other professor in social studies is teaching a unit about Australia. Each professor’ class has 30 students. At the beginning of the study, the researcher will measure each student’s attitudes toward credit with an attitude scale. Then, each professor will teach her lessons. After that, the researcher will administer an alternate form of the credit attitude scale to each student. Then, he will try to infer whether college students’ knowledge of the negative consequences of the overuse of credit affects their attitudes toward credit use.

a. What type of research design is this researcher planning to use? Diagram it.

Quasi-experimental design-non-equivalent control group design.

O x O

-------------------

O O

b. What do you believe is the one most important threat to the internal validity of this study? (Use the specific languages of the problem.)

Instrumentation - because different forms of credit attitude scale were used between pretest and posttest

Selection: Students come from different majors and educational background. Students from FCS might already have some knowledge related to credit use. In contrast, students from social studies might not have prior or limited exposure to the subject matter related to credit use. In this study there is no random assignment of the experimental groups. Selection bias is a threat which may create the formation of non-equivalent groups.

c. What could this researcher do to address this threat?

To address selection threat, the researchers should take students either from FCS or social sciences so the educational background will be similar.

1) Preliminary matching

2) Assign a subgroup of subjects randomly to groups

3) Analysis of covariance

To address instrumentation threat, the researcher should use the same form of credit attitude scale in pre-posttest.

d. Compare and contrast this design to a previous research design (question #1: h). Which do you believe is the better research design? Why?

The research design in question 1:h, pretest- posttest control group design is a better research design since it includes random assignment of participants. If random assignment is not used there may be unobserved factors causing the differences between two groups, not the treatment. If the subjects are assigned randomly then the only difference between the groups, on average, is whether they receive the treatment.

4. A researcher wants to investigate whether college students’ knowledge of the negative consequences of the overuse of credit will affect their attitudes about credit use. The researcher will select a random sample of 100 college students from a list of all college students at CSUN. He will measure each student’s knowledge of the negative consequences (delinquency, wage garnishment, foreclosure, bankruptcy, etc) of the overuse of credit with a 20-item self-report objective test. At the same time, he will measure each student’s attitudes toward credit with an attitude scale. Then, he will try to infer whether college students’ knowledge of the negative consequences of the overuse of credit affects their attitudes toward credit use. (He wants to infer cause-and-effect relationship.)

a. What type of research design is this researcher planning to use (be precise)? Diagram it.

Ex post facto, Cross sectional design.

X O

Y

· Population of CSUN students

· Subgroup:100 that are randomly selected to represent

· Two tests given at the same time

· 1) 20-Item Self-Report Objective Test (X)

· 2) Attitude Scale (Y)

· Outcomes (O) of the students’ knowledge regarding the negative consequences of credit overuse and its effect on their attitudes toward credit use.

b. Describe its advantage over other ex post facto designs.

The advantage that this ex post facto, cross sectional design has over other ex post facto designs is that there are no problem with changes in conditions, data is much easier to obtain, and it can be used even when the subject matter can be observed only once. Also, because this study does not monitor changes in its participants over a period of time, the cross sectional design is the most appropriate design. Meanwhile, other ex post facto designs, such as trend studies, view the changes within a general population over time. Cohort studies will look at more specific sub-populations as they change over time, and panel studies will look at the same set of people each time. Time series will look at the long view of a study, which would be the exact opposite of a cross sectional, since cross sectional chooses one period of time.

c. Is history a threat to the internal validity of this design? Why or why not?

No, history is not a threat to the internal validity of this design. This study design does not involve pre and posttest. The researcher only measure once on the student’s attitude toward credit use with an attitude scale.

d. Is maturation a threat to the internal validity of this design? Why or why not?

No, maturation is not a threat to the internal validity of this design. This study design does not involve pre and posttest. The researcher only measures once on the student’s attitude toward credit use with an attitude scale.

e. It testing a threat to the internal validity of this design? Why or why not?

No, testing is not a threat to the internal validity of this design. Testing threat only occur in pre-post design studies. The researcher measures each student’s knowledge of the negative consequences of the overuse of credit, and he will measure each student’s attitudes toward credit with an attitude scale at the same time. There are no pre-post design involved in this study.

f. Is instrumentation a threat to the internal validity of this design? Why or why not?

No, instrumentation is not a threat to the internal validity of this design. Similar to testing threat, instrumentation only occur in pre-post design studies or between two measurement period. It is related to any pre-post gain that is attributable to the change of instrument, which is the human observer. For example, the observer might getting better in measuring the variables in post test than in the pre test. However, in this case, the observer measures each student’s knowledge of the negative consequences of the overuse of credit at the same time with the measurement of each student’s attitudes toward credit.

g. Is mortality a threat to the internal validity of this design? Why or why not?

No, mortality is not a threat to the internal validity of this design. Mortality threat only occurs when there are dropouts of samples in between two measurement points in a study, which are pre and posttest. This study design does not involve pre and posttest. The researcher only measure once on the student’s attitude toward credit use with an attitude scale.

h. What is the most important threat to the internal validity of this study? What could this researcher do to address this threat? What could this researcher do to address this threat? Be specific in discussing this in relation to THIS particular study.

The most important threat to the internal validity of this study is ambiguity of cause and effect relationship, since there is no causal relationship that can be identified. A researcher could create a longer time-frame, so that it would be easier to determine whether the outcome resulted from the exposure in time.

i. Describe another important threat to the internal validity of this design that is NOT a threat to any previous design we have considered.

The type of people selected cannot be generalized for the entire population

The ex post facto design is only measured once. There is no way to know whether or not there is a difference of performance after the treatment because there is no pretest. There is also no random sampling and no statistical control because it is only measured a single time.

j. Compare and contrast this research design to the research designs in question #1 (note: there are two designs in question #1: c & h) and question #3 in terms of both internal and external validity. Please be organized in this answer.

The ex post facto design can be used to to establish cause and effect when experimental design is not possible. lt is less expensive and time consuming than experimental research.

The limitation of post facto design are that the researcher is unable to manipulate the independent variable purposively, and data may be unreliable due to the fact that subjects are not assigned randomly. The major threat to internal validity of ex post facto design is ambiguity about causal direction, the inability to be sure whether A causes B or B causes A.

k. Which of these four designs do you believe is the best research design? Why?

Although there is no perfect design, Solomon four-group design might be the best research design. It is appropriate for research that contains large sample group. It can eliminate almost all both internal and external threats for large sample population. However, it might be hard to perform solomon four-group design if the research contains small sample population.