Discussion Questions

The Subject-Experimenter Contract: A Reexamination of Subject Pool Contamination

Kitty Klein Brian Cheuvront North Carolina State University

W e conducted three experiments to investigate: ( a ) the extent to which student research participants believe they will disclose de- tuils of their experiences, ( b ) how much subjects uctually will disclose immediately following a request not to reveal i n f o m - tion, and ( c ) how much they will disclose after a 2-week interval. Disclosure rates increused across the experiments. In the first experiment, one fifth of the subjects indicated that they probably would not preserve confidentiality, whether or not they believed it was important to do so. In the second experiment, signing a pledge not to discuss the experiment resulted in less disclosure, but over one third o f the subjects revealed information to a confederate. In the third experiment, promising students anonymity increased disclosure; 85% of the subjects disclosed information, The results indicate thut failure to heed requests for confidentiality may be ( 1 widespread problem in college subject pools.

In recent years, we have seen a renewed concern that undergraduate students enjoy an ethical and educational experience when they participate in psychological experi- ments. Britton (1987) expressed concern that the direct educational benefits of serving as a research participant are sufficient t o justify the costs to the individuals involved. Kimble (1987) and Korn (1988) outlined the educational benefits students should expect. They emphasized that re- search participants have a right to a discussion of the topic of interest, a n explanation of the procedures followed, and a summary of the results. Nonetheless, Coulter (1986) re- ported that a substantial number of her undergraduates con- sidered their research participation as "boring, irrelevant, and a waste of time" (p. 3 17). She urged experimenters to prepare debriefings that will educate, rather than simply dismiss, student subjects.

Another side of this ethical coin has received little discus- sion. With the exception of Korn's (1988) list of participant responsibilities, the rights of the experimenter are rarely addressed. O n e of these rights, the participant's responsihil- ~ t y to honor the researcher's request not to discuss the study with anyone else who might be a participant, was the impe- tus for our article. T h e problem is essentially one of subject pool contamination. Revealing information about a n ex- perimental procedure or outcome may affect the willingness of other students to volunteer for a particular experiment and may alter their behavior if they do participate. As Mar- ans (1988) noted, leaked information has the potential to threaten the internal validity of our results.

Subject pool contamination has received some empirical attention, alheit very little. Evidence appears mixed on the

severity of t h e problem. After using a deception procedure, Aronson (1966) employed an extensive debriefing that de- scribed how a n experimental participant who is aware of the hypothesis could lead researchers to publish erroneous re- sults. All nine of his subjects admitted to a confederate that they had participated in the study, but none revealed its true nature. Two subjects provided a false cover story.

Diener, Matthews, and Smith (1972) also reported low rates of subject pool contamination. They found that, in ;I class of 440 nonsubjects, only 11 had knowledge o i a decep- tive experiment in which 52 of their classmates had participated.

Other researchers have reported alarming rates of dis- closure. Wuebben's (1967) subjects participated in a persu;l- sion experiment in which fear of an unknown disease was induced. A debriefing, which explained the importance ot not discussing their participation, was given to the subjects after t h e first e x ~ e r i m e n t a l session. All subiects were asked to nod their head to indicate that they would c o m ~ l v with - , the researcher's request not to talk to anyone regarding the details of the experlment. W h e n they returned for the sec - ond part of the experlment 1 week later, subjects were ques- t ~ o n e d about therr compl~ance w ~ t h the expenmenter's re- quest. S~xty-four percent a d m ~ t t e d talking to ,it least one other person about the experlment

Student subjects appear willing to admit that they possess prior knowledge. In a study using the Taffel verbal condi- tioning procedure, Farrow, Lohss, Farrow, and Taub ( 1975) reported that 24% of their subjects acknowledged h a v u n ~ prior information about the experiment.

Lichtenstein (1970) also tound high rates o f contamina- tion in a study that directly measured disclosure, as opposed to relying o n self-reports. When subjects were ,jpproached by a confederate identified as a future subiect, no one dis- closed rnformatlon. W h e n the confederate was ~ d e n t ~ f i e d ,I. a nonsubject, 210/0 d~sclosed ~ n f o r m a t ~ o n . After ,In lntercnl from 1 dav to 2 weeks. 79% d~sclosed ~nform,itlon to C\ tele phone ~nterv~ewer's request for ~ n f o r m , ~ t ~ o n .

G w e n the e v ~ d e n c e that subject pool contamlndtlon oc curs, one must ask whether sublects' behav~or 1s affected h \ prlor knowledge of the procedures, p,lrttcularl\ when decep- tlon 1s not a n ~ssue. Levv (1967) > t u d ~ e d the effects of prlor awareness o n performance In a Tdftel verbal c o n d l t ~ o n l n ~ task. Subjects glven full cl~sclosure 11) ,I contederate ju\r before partlclpatlon performed better th,in thaw not In formed, hut speed of l e a r n ~ n g nns nor ,~ffecteJ

Hork'i and Farrow ( 1 970) % t u d ~ e d the hell,~cIor of elelnen

'Teaching of Psychology

tary school children who could win 5 0 ~ for responding cor- rectly to an embedded figures problem. Twenty-eight per- cent of the subjects tested in the afternoon sessions were correct, compared to 13% of those tested in the morning. The authors concluded that intersubject communication ac- counted for this difference in performance.

From these studies, three conclusions seem warranted. First, rates of reported or actual disclosure vary considerably and appear to be related to type of debriefing, delay follow- ing the experiment, and whether or not the discloser be- lieves the target is a potential subject. The effects of other factors on disclosure, such as liking for the experiment, have not been investigated. Second, prior knowledge of experi- mental procedure affects performance even when deception is not an issue. Third, although disclosure rates may be high, only a small proportion of subjects are affected by the leak- age. In institutions with small subject pools, however, the problem may be more severe than in large institutions in which multiple sections of the introductory course supply potential participants.

Taken together, these earlier studies suggest that con- tamination occurs and that results may be tainted. Our ex- periments investigated several variables that might affect undergraduates' willingness to comply with an experiment- er's request not to discuss their participation. In the first experiment, we examined the relation between beliefs about the scientific value of participation, general evaluation of the experience, and likelihood of disclosure. In the second experiment, we asked half the participants in a laboratory experiment to sign a pledge to keep the experimenter's con- fidence. In the third experiment, we telephoned these same participants to see whether the passage of time or promises of anonymity would affect disclosure.

Experiment 1

Method

Sixty-five introductory psychology students volunteered to participate in the experiment, for which they received 112 hr of experimental credit. Students in the course have the option of writing a 3-page review of a recent journal article or serving as subjects for a total of 3 hr. Subjects read brief descriptions of two different studies. The first study used a Scholastic Aptitude Test (SAT) type task presented on a computer. The second study was a perceptuomotor tracking task. Only the procedures were described; no rationale for the studies was presented. Both descriptions concluded with a statement that the experimenter requested that partici- pants not discuss the experiment with others "because the results might be affected if other psychology students who might participate in the study found out what it was about." Following each scenario were six questions asking: (a) how much the respondent would enjoy participating in such a study, (b) whether the respondent thought the experiment- er would find anything interesting, (c) the importance of complying with the request for confidentiality, (d) the like- lihood the respondent would talk about the experiment with another student in the subject pool, (e) the likelihood the respondent would talk about the experiment with another student not in the pool, and (0 whether the respondent

would expect to learn anything from participating. Each question was followed by a 7-point response scale ranging from not at all likely (1) to extremely likely ( 7 ) .

Resulfs and Discussion

A 2 x 6 (Items x Scenario) analysis of variance (ANOVA) was performed. There were, of course, dif- ferences in responses to the items, F(5, 315) = 34.51, p < .001, but the only other significant effect was the Scenario x Item interaction, F(5, 315) = 3.36, p < .005. Inspection of the means indicated that students believed they would enjoy participating in the SAT study (M = 3.8) less than in the tracking task study (M = 4.4). Consistent with Britton's ( 1 987) findings, the means on the items measuring the value of the studies did not indicate negative evaluations. The - mean response to the question of whether the experimenter might discover something interesting was 4.4 for the SAT and 4.7 for the tracking task. For the issue of whether the participants would learn anything, the mean for the SAT was 3.9 and 3.6 for the tracking task. Students perceived the importance of confidentiality (M = 5.9 for the SAT; M = 5.7 for the tracking task), but indicated some likelihood (M = 2.4 for the SAT; M = 2.7 for the tracking task) that they would talk to another student who was in the subject pool. The likelihood of talking to a student not in the subject pool was greater (M = 4.2 for the SAT; M = 4.1 for the tracking task).

To determine the relation among the items, we con- ducted separate principal component factor analyses for each scenario. In both cases, two factors emerged. The eval- - uative items (enjoyment, learning opportunity, and like- lihood of scientific discovery) loaded together on Factor 1 in both analyses; all had factor loadings greater than .75. Both of the likelihood of talking items appeared together on Fac- tor 2, with loadings in excess of .80. The item assessing views of the importance of complying with the experiment- er's request showed weaker loadings on the first factor (.55 and .56) and low, negative loadings on the second (-. 11 and -. 19). This factor pattern indicated that the evaluative items and the likelihoc~d of talking items were not correlated.

The results of our first experiment suggest that evaluative judgments concerning the merits of an experiment are unre- lated to whether or not subjects believe that honoring an experimenter's request for confidentiality is important. Fur- thermore, there is no relation between the latter belief and likelihood of revealing information to others. In general, respondents indicated a fairly positive evaluation of the re- search scenarios but high likelihoods of not honoring an experimenter's request for confidentiality.

Experiment 2

In this experiment, we observed actual disclosure behav- ior as contrasted with students' self-reports. All our subjects served initially in either the SAT computer study or the perceptuomotor tracking task. Subsequently, a confederate made a direct request for information. We also wanted to determine if signing an agreement would inhibit disclosure.

Vol. 17, No. 3, October 1990

Walsh and Stillman (1974) found less disclosure when sub- jects were sworn t o secrecy but Diener, Kelem, Beaman, and Fraser (cited in Diener et al., 1972) did not.

Method

Twenty-eight students enrolled in the introductory psy- chology course, who had not served in our first role-playing experiment, participated in the second. These students were selected from a group of 102 participants described in our first experiment. T h e 28 subjects (13 men and 15 women) were selected because they were scheduled for the SATJtracking experiment at a time when one of our two confederates was available.

As part of t h e debriefing, we asked all subjects not to discuss their experiences with anyone else, ~ a r t i c u l a r l ~ oth- ers who might be future participants (i.e., other introducto- ry psychology students). In addition, 12 of the subjects were asked to sign a n agreement stating that they understood the request and agreed to comply. Three different experimenters conducted the debriefings. T h e y were counterbalanced across the two experimental conditions.

After t h e debriefing, the subject was asked to sit o n a bench outside the experimental room while the experiment- er went t o the psychology office to get the form t h e subject needed to verify participation. A female confederate was already seated o n t h e bench when the subject arrived. After the experimenter left, the confederate told the subject that she was a student in a n introductory psychology course who had not participated in any experiments and wanted to know what it was like t o be in one. She asked the subjects to describe the experiment they had just left. T h e amount each subject disclosed was rated o n a 3-point scale ranging from no disclosure ( 0 ) t o full disclosure (2) by the confederate and the first author, both of whom were blind as to whether or not the subject had signed a pledge not to talk. Interrater relia- bility, using gamma, was .899.

Results and Discussion

W e were surprised to discover that more than one third of our subiects revealed something to the confederate, even - though t h e experimenter was in t h e adjacent office and was expected t o return at any moment. In two instances, the experimenter stood by while the subject finished a disclosure to the confederate. W e were also chagrined when a student not scheduled as one of our subjects entered the S A T experi- ment with the question, "Is this the experiment where I sign the paper promising not t o talk?"

Of the 12 subjects who signed a n agreement, 1 1 refused to reveal any information to the confederate and 1 subject gave some disclosure. but n o one wrovided detailed information. In this condition, we witnessed our only fabrication: O n e atudent told the confederate she would be wired to a ma- chine and "shocked like hell." Of the 16 subjects asked to honor the exwerimenter's request for nondisclosure but not asked to sign a n agreement, 7 gave n o disclosure, 3 gave partial disclosure, and 6 gave detailed disclosure t o the con- federate. T o ensure adequate expected values, we collapsed the full- and partial-disclosure cells and performed a chi-

square test, which indicated that signing the agreement In- hibited subsequent disclosure, x Z ( l , N = 28) = 6.86 P < .01. O u r actual disclosure rate 5 min after the debriefing and not more than 50 ft from the experimenter came as a sur- prise. Thirty-nine percent of our subjects disclosed informa- tion t o a self.identified subject who was shopping for experi- mental credits.

Experiment 3

Although signing a n agreement appeared to increase compliance in the short run, we were concerned about its effectiveness over a longer period. W e were also interested in how much disclosure would occur when respondents knew such information had the to contaminate the entire subject pool.

Method

Two to 3 weeks after participating in the second expert- ment, 10 of the signers and 10 of the nonsigners were tele- phoned by a female research assistant blind to the experimental conditions. T h e caller introduced herself as an undergraduate psychology major who was writing an article o n t h e research experiences of students enrolled in introduc- tory psychology. T h e article would appear in the student newspaper during the same semester in which the students had served as subjects.

For each experiment in which they had participated, the subjects were asked what they had been required to do, what they thought the researcher was trying to figure out, and whether they thought the experiment was a worthwhile ex- perience. Half the subjects were told that thetr responses would be anonymous, and half were asked to Lrerify their name so that they could be quoted in the article. The degrer of detail that subjects disclosed was rated o n a 5-point scale ranging from no disclosure ( 0 ) to full disclosure (4).

A t the conclusion of the third experiment, we sent all subjects who had participated in our second experiment ; I postcard stating:

Following your participation In an experiment In 635 Poe Hall, you were approached, and in some cases telephoned, by one of our research assistants who asked you tc) descrihe your research experiences. The purpose of these ~nquirteh was to evaluate the effectiveness of our debriefing tech- niques. As in the experiments themselves, your responses cannot be linked to you as an individual. We thank y o u for your cooperation.

W e also provided the name and telephone number of thc supervising faculty member.

Results and Discussion

N o subject was able to glve a coherent explandt~on of the researcher's goals. T h e most common response was to "we how we would d o o n those ktnds of problem5 " All the subjects belleved the experiment had been worthwhtle, ,ll- though one sald "worthwhlle for the experimenter, but not for me " Seventeen of the 20 5uhjects descrthed the pro

Teaching of Psychology

cedure t o varying extents. Four indicated that they were asked not t o discuss their participation but "guessed it was alright now." Students who were guaranteed anonymity re- vealed more (M = 2.27) than did those whose names were verified ( M = 1.00), t(18) = 2.39, p < .05. Signing or not signing a n agreement in t h e actual experiment had no effect o n disclosure. Magnitude of initial disclosure t o t h e confed- erate did not predict magnitude of disclosure in the tele- phone interview. As in Experiment 1, evaluation of the experience was unrelated t o disclosure.

General Discussion

O u r results paint a pessimistic picture of how undergradu- ates honor the clause of our research contract that holds them responsible for not discussing t h e details of their par- ticipation with anyone who might later serve as a subject. In our first study, 22% of the role-players indicated that they would probably n o t honor the experimenter's request not t o discuss t h e procedure with a potential subject. In the second experiment, 39% of our subjects gave at least some dis- closure t o a confederate. In the third experiment, after a 2 - week interval, 85% of these participants disclosed informa- tion that could contaminate the entire subject pool. These rates are particularly alarming, given that our subjects re- vealed information t o a stranger. Even more disclosure might occur with friends or classmates. T h e student who asked about t h e nondisclosure pledge presumably had re- ceived such information.

W e identified several factors that seem t o affect disclosure rates. Asking subjects t o sign a pledge decreased immediate but not later disclosure t o a confederate. Promising students anonymity increased disclosure rates, suggesting a n awareness t h a t such disclosure might have damaging effects. Neither liking for t h e research nor a belief that it was impor- t a n t was related t o compliance with the experimenter's re- quest for confidentiality.

Does this research suggest any improvements in debriefing t h a t might decrease t h e frequency of disclosure? Asking par- ticipants t o sign a n agreement reduced disclosure t o a poten- tial subject in the short run, but this commitment had n o effect o n later willingness t o contaminate the entire pool. Perhaps providing more information about the tasks would have proved effective. T h e tasks used in Experiment 2 in- volved no deception, and the studies' rationales were de- scribed a t length. Nevertheless, none of our subjects was able, 2 weeks later, to describe the researcher's goals. If we had used a more extensive debriefing, subjects may have felt a greater stake in the outcome and been less revealing when questionned. Using the collaborative model that Korn (1988) described might decrease intersubject communica- t ion.

O u r findings regarding the ability of introductory psychol- ogy instructors t o minimize subject pool contamination are discouraging. Results of our first study suggest that apprecia- tion of the potential scientific contribution of undergradu- ates, as Kimble (1987) suggested, is unrelated to honoring a request not t o disclose, although such a n emphasis may im- prove overall attitudes toward t h e research experience.

A final issue concerns our own adherence t o the contract. Experiments 2 and 3 involved deception by the confederate. W e argue that there were n o realistic alternatives for study- ing t h e extent t o which subjects reveal information about their research experiences. Results of Experiment 1 , in which students role-played t h e same tasks we used in Experi- ment 2, indicate that they underestimated the amount of disclosure that actually occurred. Our own ethical difficulty arose in trying t o settle o n the language t o use o n the postcard. W e took care not t o suggest that students had violated their promises.

In conclusion, our experiments suggest that subject pool contamination is a problem. Researchers whose work re- quires naive participants need to ascertain the extent of their subjects' prior knowledge of the experimental tasks and procedures.

References

Aronson, E. (1966). Avoidance of intersubject communication. Psychological Reports, 19, 238.

Britton, B. K. (1987). Comment on Coulter and Daniel. American Psychologist, 42, 268-269.

Coulter, X. (1986). Academic value of research participation by undergraduates. American Psychologist, 4 1, 3 17.

Diener, E., Matthews, R . , & Smith, R. (1972). Leakage of experi- mental information to potential future subjects by debriefed sub- jects. Journal of Experimental Research in Personality, 6, 264-267.

Farrow, J. M., Lohss, W. E., Farrow, B. J . , & Taub, S. (1975). Intersubject communication as a contaminating factor in verbal conditioning. Perceptual and Motor Skills, 40, 975-982.

Horka, S. T., & Farrow, B. J . (1970). A methodological note on intersubject communication as a contaminating factor in psy- chological experiments. Journal ofExperimentu1 Child Psychology, 10, 363-366.

Kimble, G. A. (1987). The scientific value of undergraduate re- search participation. American Psychologist, 42, 267-268.

Korn, J . H. (1988). Students' roles, rights, and responsibilities as research participants. Teaching of Psychology, 15, 74-78.

Levy, L. H. (1967). Awareness, learning, and the beneficient sub- - ject as expert witness. Journal of Personality and Social Psychology, 6. 365-370.

~ i c h t e n s t e i n , E. (1970). Please don't talk to anyone about this experiment: Disclosure of deception by debriefed subjects. Psy- chologrcal Reports, 26, 485-486.

Marans, D. G. (1988). Addressing research practitioner and sub- ject needs: A debriefing-disclosure procedure. American Psychol- ogist, 43, 826-827.

Walsh, W . B., & Stillman, S. M. (1974). Disclosure of deception by debriefed subjects. Journal of Counselingand Clinical Psycholo- gy, 21, 315-319.

Wuebben, P. L. (1967). Honesty of subjects and birth order. Jour- nal of Personality and Social Psychology, 5, 350-352.

Notes

1. We thank Kim Toussignant, Gabie Smith, and Stella Anderson who helped collect and analyze these data.

2. Requests for reprints should be sent to Kitty Klein, Department of Psychology, North Carolina State University, Box 7801, Raleigh, N C 27695-7801.

Vol. 17, No. 3, October 1990