Wk 4 Individual: Multicultural Project Planning [due Mon] Wk 4 Individual: Multicultural Project Planning

Methodological and Conceptual Issues in Cross-Cultural Research

Bernard C. Beins

Culture exerts a notable impact on virtually every aspect of one’s behavior, thought, and attitude. Curiously, though, psychologists lost sight of this proposition for much of the twentieth century. This chapter will identify some of the issues that the current generation of psychologists has rediscovered as being critical in researching and understanding the wide variety of human psychological experience.

Several issues merit attention here. First, psychologists will benefit by understanding the degree to which psychological responses reflect tendencies that are universal as opposed to particular to a given culture. Psychologists have concluded in some cases that they identified universals, but upon closer examination, the certainty has faded.

Second, one’s Weltanschauung clearly drives one’s thought processes. Of specific attention has been the difference in perspective as a function of whether one’s origins are within a collectivistic or individualistic culture. Again, what seemed to have been relatively clear distinctions have blurred as psychologists have moved from the level of culture to the level of the individual. A third important element in cultural research involves the very pattern of thought processes in people of different cultures. What is obvious and apparent to one is foreign to another.

Fourth, methodological issues per se have turned out to be of importance in understanding psychological processes. New techniques like neural imaging appear to affect even basic processes that one might assume are impervious to culture. Furthermore, on a larger level, how one categorizes participants from different cultures is a thorny issue that remains unsolved. Finally, ethical issues involved in research lurk in unexpected ways. What might be ethical according to one set of standards may not be in another.

Researchers have identified these various issues, but it would be premature to claim that they have achieved resolution. The discipline has made notable progress, but as with any complex area, more questions remain than have been answered with certainty.

Historical Elements

Cross-cultural research in psychology is growing in scope and quantity, but it has had only a short history. Consequently, psychology is still grappling with fundamental methodological and conceptual issues pertaining to culturally relevant research. PsycINFO© indicates only 12 articles in peer-reviewed journals with a descriptor of cross-cultural research through 1959. In contrast, in 2008, there were 95 peer-reviewed articles listed. Naturally, a single descriptor represents only a minute slice of relevant research, but this datum reveals the trend.

Social researchers used to know that behaviors differed across cultures and that those behaviors were mediated by cultural factors. As Linton (1945) noted, “personalities, cultures and societies are all configurations in which the patterning and organization of the whole is more important than any of the component parts” (p. 2). But for a number of decades, many psychologists forgot this fact. During the heyday of behaviorism, it seemed that there was little need to attend to culture for two important reasons.

First, animals did not have cultures, so researchers studying rats (which themselves were white) did not have to consider this construct. Second, if behaviors resulted from reinforcement contingencies, researchers may have reasoned that they needed only to understand reward and punishment, and culture would not have been particularly germane in many cases.

Even among users of projective tests like the Thematic Apperception Test (TAT), there was some belief that cultural background may not have played a role in responses. For instance, Riess, Schwartz, and Cottingham (1950) administered the TAT to Black and White participants and recorded the lengths of participants’ responses; the results revealed that the length of utterances did not relate to race. This reliance on the quite objective and measurable number of words uttered was characteristic of the behavioral approach. Interestingly, the researchers did not attend to the content of those responses.

The lack of attention to culture belies the awareness by earlier psychological researchers and by anthropologists that studying culture was an intrinsic part of studying people. Current students of culture would undoubtedly disagree with some interpretations of differences, but the important issue here is that the researchers recognized the importance of those differences. For example, Tylor (1889) attributed differences across cultures through the perspective of Herbert Spencer’s model of cultural evolution, with Western “races” at the pinnacle. However, Thomas’s (1937) ideas anticipated current thoughts about differences in culture leading to different behaviors; he did not accord different status to what others called the “higher” and “lower” races.

By the 1930s, psychology had adopted a tone that resembles today’s. For example, Herskovits (1935) dismissed the notion that African cultures (of which there were many) were primitive or savage. Rather, he noted that strong family ties, strong adherence to governmental and legal principles, and established religions characterized African cultures of that era. He also distinguished the cultures of Black people in Africa and North America, noting that cultures change as people from one culture come into contact with people from another.

Revisions in the ideas about people of non-Western or of southern and eastern European descent in the 1930s seem to have resulted from the influx of a new type of psychologist. Psychology became populated with people from a wide range of ethnicities, particularly Jewish psychologists, who may have been more sensitized to the different life experiences of minorities, and therefore more aware of the cultural factors that eventuate in particular behaviors (Samelson, 1978).

The current focus in psychology on the importance of culture in affecting behavior may similarly result from the influx of a different set of ethnic minorities. Between 1996 and 2004, there was an increase in doctoral degrees among American minorities of 16.6%, an increase in master’s level degrees of 90.8%, and an increase in bachelor’s degrees of 36% (American Psychological Association, 2008). In addition, as an organization, the American Psychological Association (APA) has focused on the internationalization of the discipline, as evidenced by its 2008 Education Leadership Conference that focused on international connections in psychology.

The current cultural climate in psychology will pave the way for changes in two important aspects of research—the way psychologists conduct their research and the way they interpret their results. Just as understanding the relation between culture and behavior is complex, so must be the way psychologists develop research questions, identify appropriate methodologies, and interpret their data.

Universals and Measurements Across Cultures

Cross-cultural research poses difficulties that much other research may not face because, in addition to developing designs that show high levels of internal validity, cross-cultural psychologists have to worry about appropriate external validity. There are also practical issues of generating appropriate participant samples from varied cultures. Furthermore, drawing inferences from data can be difficult because researchers need to understand the intricacies of each culture. Attention to cross-cultural research is relatively new in psychology, thus the methodological and interpretive issues still merit critical scrutiny.

An important component of both the internal and the external validity is the nature of measurements across cultures. As Chen (2008) has noted, measurements across cultures may not be comparable because of factors such as translation issues, failure of items being measured to capture the same construct across cultures, different response styles, and social desirability.

Compounding this dilemma is the fact that some apparently etic traits have emic components in a particular culture that may be irrelevant in another. For instance, in Chinese culture, the etic construct of dependability includes emic components like being gracious to others, truthful, and family oriented (Cheung & Leung, 1998).

Some cross-cultural investigations on universal processes (e.g., Ekman’s, 1972, work on recognition of emotions) have attained the status of “classic” research. But conclusions even about the supposed universality of the ability to recognize emotions must be modified to take culture into account. Ekman’s (1972) research on how accurately people recognize facial expression of emotions provides an appropriate backdrop with which to begin a discussion of how questions evolve across the various phases of culturally oriented research. Ekman discovered that some emotions are universally recognized. He displayed faces to people in five different cultures and asked them to identify the emotion expressed in the picture. Respondents from the United States, Chile, Brazil, Argentina, and Japan all identified the emotions at well above chance levels, implying that a display of emotion has a universality that transcends individual cultures.

As Elfenbein and Ambady (2003) pointed out, though, cultural factors are not irrelevant to the recognition of emotion. For example, the various groups identified the emotions with noticeably different accuracy, with Americans showing the highest average degree of accuracy (86%) and the Japanese the lowest (78%). Elfenbein and Ambady noted, however, that the task involved the recognition of emotions expressed by Americans. It turns out that there is a reliable in-group effect such that, although people are accurate in identifying emotional expressions in general, they tend to be better at identifying those emotions when expressed by people within their own culture. Furthermore, even though there is consistently high recognition of emotions in photographs, the accuracy rate is affected by how many generations a family of Chinese origin had lived in the United States and by whether the emotional expressions were displayed by people living in China or in the U.S.

Similarly, John Bowlby’s (1988) conceptualization and Mary Ainsworth’s research on attachment (Ainsworth, Blehar, Waters, & Wall, 1978) are predicated on the universality of the construct. Ainsworth’s findings seem descriptively useful, but some (e.g., Keller, 2008) have argued that the conclusions regarding differences in attachment styles across cultures represent a bias toward Western culture. Keller stated that some conclusions include a heavy dose of judgment. For example, she suggested that inferences about maternal sensitivity associated with attachment reflect “a judgment on maternal adequacy, a way of distinguishing good from bad mothers” (p. 410).

An assessment of the research findings and subsequent interpretations suggests that attachment behaviors deemed undesirable or even pathological in one culture may be desirable and entirely acceptable in another (Rothbaum, Weisz, Pott, Kiyake, & Morelli, 2000). Thus, one could convincingly argue that attachment theory, often regarded as culture-free, is highly culture-bound.

Psychologists have regarded recognition of emotion and attachment styles as being universal. Thus, one might conclude that these topics should be easy to research: It would not matter where one conducted the research because people would not differ notably across different cultures. Unfortunately, current research on these topics belies the idea that cultural specifics are irrelevant. Consideration of attachment and of recognition of emotion reveals the difficulty that researchers face in trying to study constructs that may have some degree of universality.

A further indication of the complexity of these issues is that constructs that may be regarded as similar may require very different cultural assessment. For example, Smith, Spillane, and Annus (2006) have argued that anorexia nervosa is culturally invariant but that bulimia nervosa is highly Western in origin and prevalence.

Issues in Culturally Relevant Research: Individualism and Collectivism

Psychologists have developed an awareness that they need to exercise caution in discussing supposedly universal traits or patterns of behavior. Even apparently simple cognitive acts like categorizing stimuli or remembering details of an experience involve important cultural components. For instance Ji, Zhang, and Nisbett (2004) reported that Chinese participants may group words according to how they relate whereas American participants group them according to taxonomy (e.g., in a monkey–banana–panda triad, Chinese participants group monkey–banana, but Americans pair monkey–panda).

Considerations of complex personality issues raise even more difficulties. For example, researchers have assumed that the need for self-esteem is a universal trait because the well-established body of literature has consistently shown such a need. However, that literature is based on North American culture; when psychologists have investigated self-esteem among the Japanese, the results diverge in critical ways from those involving North American participants (Heine, Lehman, Markus, & Kitayama, 1999). In fact, self-esteem as conceptualized in Western psychology may be fairly irrelevant in Eastern psychology.

Given that psychological theory is typically Western-based, the constructs that psychologists use and even the language may make it difficult to understand cultural differences and even to speak about them. Thus, researchers may be discussing what they think are etics that are really emics. In fact, there is still disagreement as to whether some psychological constructs are basically universal with different cultural manifestations or whether they are largely cultural (Smith et al., 2006).

One of the dimensions receiving considerable attention because of its cultural relevance involves the role of individualism and collectivism. According to Matsumoto and Yoo (2006), this is the most widely studied dimension in the field, and varied research projects have illustrated consistent behavioral differences associated with the individualistic-collectivist (IC) continuum. It will serve as a useful illustration of the nature of the problems associated with studying behaviors across cultures.

An initial caveat in discussing the effects of one’s location on the IC continuum is that one must recognize that attributing differences in behavior to IC orientation is difficult because the investigator must be able to separate effects of culture (e.g., IC orientation) from other sources of variability. On an ecological level, there is a myriad of factors that affect behavior in different countries including affluence, population density, religious practice, and climate. As psychologists define culture, these ecological variables stand apart from cultural variables (Matsumoto & Yoo, 2006, p. 237).

Research on individualism and collectivism has revealed that people from collectivist cultures remember events differently than people from individualistic cultures. Individualist people tend to remember situations from the viewpoint of themselves as part of the situation. On the other hand, collectivist people tend to focus on the social situation with the perspective of an outsider looking in. A typical interpretation is that individualists focus on self and collectivists focus on other. If this difference in memory is true, then simple measurement of memories may be problematic across cultures because of the different ways that people conceptualize their world (Cohen & Gunz, 2002).

However, even if one were to resolve this dilemma, another problem arises in cultural research. Multiple cultures may exist and notable individual differences appear within a given country; unfortunately, researchers often equate country with culture, making the assumption that what is true generally within a country reflects differences in culture compared to another country. Thus, Iwata and Higuchi (2000) studied state and trait anxiety, finding that their Japanese participants had less positive views of themselves and higher levels of both state and trait anxiety than Americans. They attributed the variation across countries to differences in Japan and the United States on the IC continuum. But, as Matsumoto and Yoo (2006) pointed out, Iwata and Higuchi interpreted their data under the assumption that Japan is a collectivist society in which people are compliant in order to maintain social harmony and underestimate their own positive traits.

Matsumoto and Yoo identified seven assumptions that Iwata and Higuchi made regarding Japanese people. None of these assumptions was empirically tested. Matsumoto and Yoo did point out that Iwata and Higuchi’s assumptions may be correct and that differences between Japanese and American students resulted from the relevant differences on the IC continuum. Their point was that psychologists should engage in research to verify such assumptions. The attribution of differences across groups to cultural factors when there is no empirical support for such an interpretation is known as the cultural attribution fallacy, a specific case of a larger problem that Campbell (1961) called the ecological fallacy.

Interestingly, Iwata and Higuchi (2000) couched their discussion of the Japanese-American differences using the American pattern as the norm. This type of inference poses behaviors of Americans as a standard against which others are compared, which Arnett (2008) has pointed out is common in discussing and interpreting research findings. This orientation to interpretation of research holds true for other types of comparisons as well, including gender differences (Hegarty & Buechel, 2006).

Another complicating factor here is that the IC dimension, as dominant as it is in cultural research, may not be the only construct with explanatory power. For example, as Matsumoto and Yoo (2006) pointed out, psychologists have listed 25 additional dimensions that distinguish cultures. As they noted, countries that differ on one dimension may also differ on others. For instance, on the IC continuum, the United States stands at first in individualism, whereas Japan is 27th out of 70 countries (which puts both Japan and the United States below the median in collectivist tendencies). At the same time, Japan is 8th in uncertainty avoidance, whereas the United States is 59th (Hofstede, 2001). Matsumoto and Yoo also noted that Japan ranks 4th in long-versus short-term orientation, with the United States at 26th out of 36 countries, which might explain differences in anxiety level differences among Japanese and Americans (Iwata & Higuchi, 2000) as well as or better than IC.

Finally, researchers must provide empirical assessment of characteristics of people in a particular culture in order to draw inferences about behaviors based on perceived differences across countries. Noting the great within-group variability in traits within a culture and, often, small across-group differences, McCrae and Terraccianno (2006) demonstrated that “there does not appear to be even a kernel of truth in the stereotypes of national character” (p. 160). There may be consensus about people in a given culture (e.g., the belief by Americans that the English have no sense of humor), but McCrae and Terraccianno note that consensus (i.e., reliability) does not equal validity. Hence, valid measurements of traits in a sample must accompany inferences of behaviors based on those traits.

Culture and Its Relation to Thought and Language

The discussion above centers on the role of culture in attitudes, behaviors, and personality. But there is reason to believe that culture may affect the way people actually think about the world. The belief that Inuit people have a multitude of words for snow and that this linguistic abundance leads them to conceptualize their snow-filled world differently than people in moderate climates is just a myth. But the point behind that myth has a reality. One’s cultural background, including one’s language, seems to be causal in the development of a world view.

For example, as noted previously, Ji et al. (2004) presented their participants with word triads such as monkey–panda–banana. Participants selected two items that formed a group. A grouping like monkey–panda is categorical, whereas a grouping like monkey–banana is relational. The question that the researchers addressed was whether people of Chinese background and of American background responded similarly. The results confirmed that the Chinese participants were more likely to choose relational pairings (e.g., monkey–banana because monkeys eat bananas), whereas the American participants favored categorical pairings (e.g., monkey– panda because both are animals).

These results support the notion that people of East Asian backgrounds preferentially focus on overall context but that those with Western backgrounds make use of taxonomic concepts. Such findings are compatible with a collectivist, context-oriented perspective versus an individualistic perspective. Such sensitivity to context has appeared in numerous studies, including accuracy in identifying one’s orientation in a rod-and-frame test (Ji, Peng, & Nisbett, 2000) and memory for focal versus background elements in visual display (Masuda & Nisbett, 2001).

However, explanations are always fraught with rival hypotheses. Simply because Ji et al.’s (2004) participants may have differed on the IC continuum, it does not necessarily follow that the differences in their patterns of responses in choosing a pair of words from the triad were due to collectivism versus individualism. Drawing such a conclusion may reflect an example of the cultural attribution fallacy.

For example, Matsumoto and Yoo (2006) have speculated that differences in memory for individual items in a visual display between Japanese and American participants may be due either to the greater emphasis on rote memorization in schools or on the visual computer game culture in Japan. Such memorial differences may be consistent across cultures, but those differences do not always signal cultural differences (as the term culture is used by psychologists). This difficulty with interpretations lies at the heart of the interpretation paradox (van de Vijver & Leung, 2000), which reflects the fact that large differences across groups are easy to spot, but hard to explain, whereas small differences are hard to spot, but easy to explain.

Similarly, differences in responses to stimuli may reflect a combination of culture and other factors. As noted above, Ji et al. (2004) reported that Chinese participants organized words relationally but Americans organized words categorically. At the same time, the language in which participants were tested affected their responses. Response patterns of some Chinese participants differed depending on whether the language of the experimental session was Chinese or English. This finding corresponds to differences that other investigators have noted. For instance, Marian and Neisser (2000) found that language itself provides a context for recall. When Russian–English bilinguals received a prompt in Russian, their memories for autobiographical details associated with living in Russia, whereas English prompts spurred better memory for events associated with speaking English. In addition, the researchers have noted differences in cognitive processes associated with language itself.

Similarly, cross-cultural research has revealed differences in recognition of emotions. Matsumoto, Anguas-Wong, and Martinez (2008) discovered that Spanish– English bilinguals recognized facial emotions more accurately when they were tested in English than in Spanish, and Matsumoto and Assar (1992) showed that Hindi–English bilinguals recognized facial emotions more accurately when tested in English than in Hindi.

These results based on language may extend beyond language itself. That is, it is possible that bilinguals who are bicultural engage in a kind of cultural frame switching, viewing their world in different ways depending on the context.

For example, Benet-Martínez, Leu, Lee, and Morris, (2002) assessed the degree to which participants saw their two cultures as being consistent with one another or contradictory to one another. They characterized the participants as either high in bicultural identity integration (BII, cultures were consistent) or low in BII (cultures were contradictory). In the experimental session, the researchers primed participants to think either of American culture (e.g., by presenting pictures of Mickey Mouse, the U.S. Capitol, etc.) or Chinese culture (e.g., by presenting pictures of a rice farmer, the Great Wall of China, etc.). Participants subsequently viewed an animation of a single fish swimming in front of a school of fish, and then the participants gave an explanation as to why the single fish was leading the pack. The researchers coded the responses as involving factors that were either internal (personal characteristics of the fish) or external (the fish being influenced by other fish).

The results revealed that people who saw their two cultures as consistent with one another (those high in BII) responded with internal attributes of the fish when primed with American symbols and with external attributes when primed with Chinese symbols. On the other hand, participants who saw their two cultures as contradictory (those low in BII) showed internal attributions when primed with Chinese symbols and external attributions when primed with American symbols.

The researchers concluded that those high in BII readily engaged in cultural frame switching because they were comfortable with both worldviews. In contrast, those low in BII reacted to cues in one culture by drawing to mind their affiliation with the other culture. This response resembles the behavior of tourists to foreign countries who have a heightened awareness of their affiliation with their culture but who rarely think of that affiliation in their home country. As Markus (2008) pointed out, ethnicity can affect the psychological experience even when people are not aware of it.

As these results show, culture plays a significant role in how people observe and respond to the world around them. But the issue is complex because behaviors change due to individual differences like the language that the person uses and the degree to which the person feels assimilated into the various cultures to which he or she is exposed. Depending on the sample used in a research project, results might show vastly different patterns. Furthermore, the nature of a prime could affect the outcome significantly. For example, Shih, Pittinsky, and Ambady (1999) primed women of Asian descent to their gender identities or to their cultural identities before asking them to solve math problems. When the women received the gender prime, they performed less well on the math task than when they received the culture prime.

Inzlicht and Ben-Zeev (2000) provided an example of what may be a very subtle prime that had a major effect on women’s math performance. Simply being in the presence of men depressed women’s math performance compared to being in a single-sex research session.

Priming is a useful technique for assessing the effect of group membership. Much of the work on performance after a prime as it systematically affects different groups has involved sex and gender (e.g., Inzlicht & Ben-Zeev, 2000); it is well known, but less studied, in the context of race (e.g., Aronson, Lustina, Good, Keough, Steele, & Brown, 1999; Steele & Aronson, 1995). There is a collection of additional research on other cultural groups whose members are affected by appropriate primes, including Canadian (Walsh, Hickey, & Duffy, 1999), Chinese (Lee & Ottati, 1995), French (Croizet & Claire, 1998), German (Keller, 2007), Italian (Muzzatti & Agnoli, 2007), and poor people (Croizet & Claire, 1998).

This complex pattern of factors that influence one’s outlook and behavior reveals the caution one must exercise in attributing differences in performance simply to culture. As noted before, culture might be very important, but differences in behavior may reflect the moderating effect of other variables and of individual differences among participants. Thus, significant differences across groups might actually reflect a confounding variable that is correlated with cultural affiliation but the differences may not be due to that affiliation itself.

The Council of National Psychological Associations for the Advancement of Ethnic Minority Interests (2000) has developed guidelines for research in ethnic minority communities in the U.S. These guidelines were developed in the context of American culture, but they pertain to cross-cultural research of any kind. The principles involve (a) avoidance of treating all members of a group as being the same, (b) development of cultural competence by researchers, (c) use of multiple, valid, convergent measurements, (d) understanding of cultural context, (e) use of representative samples, and (f) avoidance of using race as a predictive construct.

Methodological Issues

The cognitive and intellectual effects of culture on thought and behavior pose serious issues in interpretations of research results. But methodological questions that appear fairly simple in research with samples from a single culture turn out to be complicated in and of themselves. Failure to account for the effects of methodology across cultures can result in data that are, at best, uninterpretable or, at worst, entirely misleading. The present discussion of cross-cultural research will include a broad definition of cross-cultural, accepting that different cultures can exist across countries but also within countries.

In describing their research, authors should provide enough demographic information about participants to (a) allow valid interpretation of results, (b) generalize the results, and (c) provide useful information for further research in the area (American Psychological Association, 2010, pp. 29–30). This information includes variables like age, race or ethnicity, sex, and socioeconomic status. As noted below, race and ethnicity are not useful explanatory constructs, but when demographic information appears in a published article, readers can use that information to put the results into context regarding to what groups the results might pertain.

A cursory examination of most research reports reveals a paucity of such information. Interestingly, the third edition of APA’s publication manual (American Psychological Association, 1983) makes no mention of the importance of including culturally relevant information about participants. Not until the fourth edition does the manual provide detailed guidance about describing cultural characteristics of participants (American Psychological Association, 1994). This change across the publication manuals mirrors the increase in attention to the importance of culture in research.

The importance of these variables in behavior is obvious. But there are also potential surprises lurking where researchers are not likely to expect them. For example, Park and Gutchess (2006) have noted that the results of brain imaging studies show that culture is associated with differences in neural processing. Consequently, failure to recognize such cultural effects could blind researchers to confounding variables and problems in drawing inferences, and failure to indicate both age and ethnicity in a research report could hamper further research in the area.

Furthermore, some psychologists have questioned whether culturally relevant research involving race (as opposed to ethnicity) is well grounded. That is, does race actually represent a “naturally occurring” causal variable? Helms, Jernigan, and Mascher (2005) have argued that there are methodological problems associated with conceptualizing people from a given racial category as constituting an intact group. As a rule, researchers do not use empirical measures to verify that the members in a group actually share important characteristics other than race.

Even when investigators report demographics, one needs to be aware of potential problems with this information. When people self-report their race, they may alter their choices depending on type of question (e.g., forced choice vs. selecting all that apply) or perceived advantage of a given selection in a particular situation (Panter, Daye, Allen, Deo,& Wightman, 2009; Phinney, 1996). Thus, even if race is a viable conceptual variable, assignment to racial categories, whether by the researchers or by the participants, may be problematic. In fact, with respect to health-related research, Shields et al. (2005) have recommended that self-reported race may be useful for recruiting research participants, but that statistical analysis should not be based on self-reported race.

Another, related problem with categorizing by race or ethnicity is that such categories are often based on bureaucratic rather than on systematic and scientific denotations. Rodriguez (2000) pointed out that, if a researcher uses categories established by the United States government, the same individual may fall into different groups depending on what agency has collected the data. The Census Bureau does not include Hispanic as a racial category, but agencies responsible for civil rights issues do include such a racial category.

The Pew Research Center has illustrated the problem with categorization with a “quick primer” related to being Hispanic, at least in the eyes of the U.S. Census Bureau:

Q: I immigrated to Phoenix from Mexico. Am I Hispanic?

A: You are if you say so.

Q: My parents moved to New York from Puerto Rico. Am I Hispanic?

A: You are if you say so.

Q: My grandparents were born in Spain but I grew up in California. Am I Hispanic?

A: You are if you say so.

Q: I was born in Maryland and married an immigrant from El Salvador. Am I Hispanic?

A: You are if you say so.

Q: I was born in Argentina but grew up in Texas. I don’t consider myself Hispanic. Does the Census count me as an Hispanic?

A: Not if you say you aren’t. (Passel & Taylor, 2009)

With research involving comparisons across countries, the situation is equally or more murky. Researchers have often equated country and culture, so that all participants from a given country are seen as having the same cultural roots. Chang (2000) provided an example of the drawback in this approach, explaining that there is really no single cultural group that one can designate as Chinese. Considering historic, linguistic, and anthropological factors, one could identify over 50 different ethnic groups that are all labeled Chinese. In fact, some of these groups speak variants of Chinese that would be mutually incomprehensible. Some researchers have used strategies like labeling as Asian anybody who was born in Asia, which could include people from a vast array of countries that show notable cultural differences among themselves (Cohen & Gunz, 2002; Kim, Atkinson, & Yang, 1999).

Further difficulties arise in interpreting data based on cultural category. Kim et al. (1999) suggested that as people from Asian countries become familiar with the culture in the U.S., their behaviors may change quickly. However, their attitudes may change slowly, if at all. So in one sense, they become members of both cultures, with an outlook that may differ across individuals, depending on the degree to which they feel comfortable in both cultures (Benet-Martínez, Leu, Lee, & Morris, 2002).

Stimulus Materials

A further, significant issue in cross-cultural research involves the materials (e.g., psychological tests, experimental stimuli) that investigators use. These materials might seem straightforward and noncontroversial within the context of single-culture research, but there are notable reasons to attend to them in cross-cultural research.

Researchers who measure a construct across different groups often assume that the measurement is invariant across groups (Chen, 2008). A second assumption is that the construct itself is the same across the groups (Arnett, 2008; Heine & Buchtel, 2009). Neither of these assumptions is guaranteed. For instance, Chen has pointed out regarding the question of measurement invariance that reliability (hence validity) is generally lower in educational measurement in underperforming groups. Furthermore, across groups, different factors may underlie a construct like self-esteem, so measuring the construct comparably in the different cultures is difficult or maybe not even possible (Smith et al., 2006).

When it is possible to assess personality across cultures that are related (e.g., American and European), language and stimulus issues can still be important. Studies of the Big Five model of personality lead to somewhat different results as a function of translating English words into other European languages (Peabody & De Raad, 2002). Similarly, the addition of contextual or situational information in self-reports of personal characteristics can affect the reliability of the measurements of the Big Five traits (De Raad, Sullot, & Barelds, 2008).

In addition to the problem of characterizing a construct, if a measurement instrument is to be used in groups with different languages, translation issues arise. The concept of feeling blue has been used to exemplify linguistic concerns. How does one translate it? A direct translation into Chinese produces an item with little meaning (Chen, 2008); trying to render it directly into Spanish leads to problems because the word for blue, azul, does not carry the same connotation as in English (Rogler, 1999). The use of back translation can help (Banville, Desrosiers, & Genet-Volet, 2000), although there are still reasons for concern when researchers attempt this. For example, Rogler reported that 36% of test items on the Spanish version of the Clinical Analysis Questionnaire had grammatical errors or direct translation of idioms that made no sense in Spanish.

There is a growing literature regarding the adequacy of assessment materials across cultures, but the issue is far from settled. The question of cultural competence on many levels in research remains an important concern.

Ethics

A final set of issues in cross-cultural research revolves around ethics in research. In the present context, these issues are not associated with anonymity, confidentiality, informed consent, and other elements of planning and conducting research, although researchers must attend to them. Rather, the discussion here involves the implications of research with diverse populations.

Markus (2008) raised the question of whether research on race and ethnicity, in and of itself, leads to stereotyping or other ethically troublesome outcomes. The question is not moot. For example, Kirmayer and Young (1998) reported that somatization of psychiatric symptoms varies across cultures. In addition, some behaviors that are stereotypically associated with certain ethnic groups in the United States are also associated with personality disorders (Iwamasa, Larrabee, & Merritt, 2000). Researchers who are not sensitive to cultural variations could draw inferences that result in stigmatization and generally unwarranted conclusions. APA’s ethical principles and standards are especially relevant here.

First, among the aspirational general principles cited by APA (2002), the issue of justice is quite relevant here. That is, psychologists need to recognize the limitations to their expertise and competence and to recognize their biases.

Second, psychologists must respect people’s rights and dignity. This principle pertains to recognition of and respect for individual differences associated with “age, gender, gender identity, race, ethnicity, culture, national origin, religion, sexual orientation, disability, language, and socioeconomic status” (p. 1063). Furthermore, psychologists are enjoined to consider these factors when working with members of such groups.

Within the enforceable ethical standards set forth by APA, there are several specific elements related to culturally relevant research, including competence (boundaries and maintenance of competence, bases for scientific judgments) and assessments (use of assessments, interpreting assessment results, assessment by unqualified persons).

Researchers should not ignore the fact that research on populations with which they are unfamiliar is fraught with potential conceptual and methodological problems. The conclusions based on research with participants from diverse backgrounds have both theoretical and social implications that are connected to ethical issues.

Conclusion

Cross-cultural research in American psychology has moved through several stages over the past century, from a relatively simple, descriptive depiction of people in varied cultures to a complex and nuanced recognition of the complex interplay between culture and behavior. The recent impetus among many psychologists to consider the relation of culture and behavior seems to mirror society’s increased interest in cultural issues as American society has become more diverse; along the way ethnicity has become a more prominent facet of many people’s lives.

Kim (2007; see also Chapter 1, this volume) has suggested that, just as societal ideas change, so do research ideas, positing that there are four “cultures” of cultural research. These so-called cultures include (a) the pre-encounter research culture: “I’m not interested”; (b) the initial encounter: “culture is a nuisance”; (c) the Captain Cook research culture: “let’s explore and compare”; and (d) the paradigm shift research culture: “beyond ethnocentric paradigms.” One might argue that psychology has entered the exploration/comparison phase. Development of ideas that may move psychological theory beyond ethnocentric paradigms would require a paradigm shift, according to Kim. So it is not immediately clear what the next phase would look like.

However, within the current paradigm, psychologists are only now successfully separating some culture-independent and -dependent processes. It has become apparent that some supposedly universal processes (e.g., recognition of facial emotions) are more intertwined with specifics of language than initially supposed, as shown in the recent research on the Big Five model of personality (Peabody & De Raad, 2002).

Furthermore, the use of an American-centered model of psychological processes still predominates. Models of attitude, personality, and behavior may undergo additional modification when what is considered standard (i.e., the American perspective) instead becomes simply one of many useful perspectives. And as researchers compare and explore psychological processes across cultures, it will be critical to define samples precisely and to document the validity of assumptions that participants show certain cultural characteristics, a practice that is largely honored in the breach.

Finally, because issues of race and ethnicity carry considerable emotional valence, researchers must show awareness of the implications of their findings. Obviously, all researchers need to show such awareness, but problems with stereotyping, prejudice, or simple misunderstanding are largely irrelevant in monocultural research with participants of the dominant culture.

Breaking new ground in any area of research poses distinct challenges. Cross-cultural research is no different than any other area. As noted here, psychologists have identified problems and posed solutions to a constellation of new issues with which researchers must contend when expanding the horizons of psychology to include the study and understanding of behavior across cultures.

Why Diversity Matters:The Power of Inclusion in Research Methods

Linda M. Woolf and Michael R. Hulsizer

As we look around the globe, one fact becomes immediately apparent— remarkable diversity, as well as similarity, characterizes humanity. The term diversityoften refers to the varieties of human characteristics found within a culture, but also sometimes implies similar variety across cultures. Just within the United States, the population reflects a strikingly diverse tapestry composed of variations in gender, race, ethnicity, sexual orientation, language, ability, income, culture, and other elements of diversity. We live in a multicultural and international community with multi-layered, intersecting threads representing potential areas of study and understanding. Researchers may elect to study not only one element of diversity but also how this element may differ across cultures and/or interact with other elements of diversity. For example, one might choose to study gender differences within a culture, differences between two or more cultures related to gender, or the intersection of gender and sexual orientation either within or across cultures. Regardless, it is imperative that researchers conduct their investigations in a methodologically valid and culturally appropriate manner. It is not enough simply to include some element of diversity as a variable to study. Rather investigators must take care to insure that the methods chosen to research that variable are adequate, appropriate, and ethical. Failure to conduct cross-cultural or diversity research with an eye toward appropriate methodology may result in faulty results and erroneous conclusions.

Despite the spectrum of diversity as one looks around the globe, psychological research and study within the U.S. has failed to reflect that diversity. According to Louis Kincannon, Director of the United States Census Bureau (2007a),

About one in three U.S. residents is a minority. To put this into perspective, there are more minorities in this country today than there were people in the United States in 1910. In fact, the minority population in the U.S. is larger than the total population of all but 11 countries. (para. 1)

The U.S. Census Bureau (2007b) also reported that the minority population is not localized in one region. Indeed, “nearly one in every 10 of the nation’s 3,141 counties has a population that is more than 50 percent minority” (para. 1). Despite these figures, several researchers have noted a paucity of published research focusing on racial or ethnic minorities within psychology journals (e.g., Bernal, Trimble, Burlew, & Leong, 2003; Carter, Akinsulure-Smith, Smailes, & Clauss, 1998; Graham, 1992, Sue, 1999). According to a number of investigators, some researchers simply fail to report the racial and ethnic background of study participants (Buboltz, Miller, & Williams, 1999; Delgado-Romero, Galván, Maschino, & Rowland, 2005; Munley et al., 2002). Delgado-Romero and colleagues (2005) reported that only 57% of examined counseling articles published between 1990 and 1999 reported race and ethnicity. The results of their analysis revealed that White and Asian Americans were overrepresented, whereas African Americans, Hispanics, and Native American Indians tended to be underrepresented as compared to the U.S. population. Finally, it is also important to note that much of the traditional psychological curriculum has historically neglected issues of diversity (Guthrie, 1998). For example, despite the fact that approximately 45% of articles indexed in PsycINFO© included authors from outside the United States (Adair, Coelho, & Luna, 2002), introduction to psychology, life-span developmental psychology, and social psychology textbooks included little of this research (Woolf, Hulsizer, & McCarthy, 2002). Ultimately, it would seem that Betancourt and López’s (1993) assertion that “the study of culture and related variables occupies at best a secondary place in American (mainstream) psychology” (p. 629) is as true today as it was almost two decades ago.

Fortunately, there has been a recent push to increase both an awareness of and research concerning diversity, cross-cultural issues, and international research within psychology. In January 2000, the Council of National Psychological Associations for the Advancement of Ethnic Minority Interests (CNPAAEMI) produced the Guidelines for research in ethnic minority communities. Five ethnic minority associations within the American Psychological Association (APA) collaboratively worked to create a document reflecting issues relevant to research with African American, Asian American, Hispanic, and American Indian populations. In 2003, the APA published the Guidelines on multicultural education, training, research, practice, and organizational change for psychologistsand in 2005 and 2008 hosted Education Leadership Conferences devoted to diversity and international issues, respectively. Indeed, the APA (2009) vision statement now identifies the Association as

A principal leader and global partner promoting psychological knowledge and methods to facilitate the resolution of personal, societal and global challenges in diverse, multicultural and international contexts. (para. 9)

In 2005, the APA Working Group on Internationalizing the Undergraduate Psychology Curriculum, in concert with the American Council on Education (ACE), published a set of recommended learning outcomes for psychology courses (Lutsky et al., 2005). The report noted that psychology has long been international both as a science and as a profession with early psychologists working in such diverse countries as Argentina, India, Japan, Mexico, New Zealand, and Russia. Goal 2 of the report focused on research methods and highlighted that “Students should be aware of research methods and skills necessary for international research competence” (p. 3). The report presented five learning outcomes in relation to research methods, specifically that students will develop: (1) the ability to access and read journals published outside the United States; (2) the ability to identify and appreciate the range of research methodologies used by psychologists around the globe (e.g., ethnographies, observations); (3) an awareness of ethical issues that may be of concern in other countries (e.g., protection from harm); (4) an understanding that many constructs may have different meanings in different cultures; and (5) an appreciation of the dangers associated with making broad generalizations based on small and potentially unrepresentative samples.

In this chapter, we focus on a number of issues related to the role of diversity in understanding research methods. It is important to note that we do not discuss all issues related to diversity concerns and research. Rather, we seek to introduce the topic by providing representative examples which highlight the complexity involved in conducting research responsibly—particularly research that involves diverse populations. Moreover, much of what we discuss in each section below describes research focusing on differences between groups (e.g., differences between two cultures). However, we would like to highlight that this does not negate the need for research to be more inclusive of diverse populations in non-diversity focused research. It is important that researchers endeavor to be more inclusive of diverse populations in all of their work.

Validity

Marsella (2001) suggested that many psychologists are unwilling to “accept a very basic ‘truth’—that western psychology is rooted in an ideology of individualism, rationality, and empiricism that has little resonance in many of the more than 5,000 cultures found in today’s world” (p. 7). Indeed, as we saw in Chapter 1, psychology textbooks and journals are filled with psychological concepts, theories, and research findings that, at the surface, appear to be applicable to all humans. Yet, there is a great deal of cross-cultural as well as intra-cultural variability. Moreover, there is growing awareness that the use of homogeneous samples (e.g., introductory psychology students within the U.S.) has painted a somewhat distorted picture of psychological phenomena (Graham, 1992; Sears, 1986). Unfortunately, although homogeneous samples may increase the internal validity of a study, it is often at the expense of the ability to generalize to a range of diverse populations. All of the above highlights concerns related to the topic of research validity, specifically construct, internal, and external validity.

Construct validity

Construct validity typically refers to the degree to which a variable has been operationally defined so that it captures the essence of a hypothetical construct under study. For example, does a particular depression scale adequately measure depression or is an IQ test a good measure of intelligence? If the answer is “yes,” then the measure has good construct validity. If the answer is “no,” then any research conclusions drawn from the use of the measure are, at best, limited. An examination of construct validity is necessary in any cross-cultural or diversity research.

When conducting cross-cultural research, experimenters must monitor the degree to which their methodology is equivalent across cultures and groups. In fact, Matsumoto (2003) suggested that researchers should strive to create the perfect cross-cultural study. To that end, Brislin (2000) asserted that investigators need to develop an awareness of three sources of nonequivalence—translation, conceptual, and metric. Translation equivalence is necessary when conducting research on one population using experimental measures developed and standardized with another population. For example, if a researcher plans to use a standardized questionnaire developed in the U.S., translation equivalence needs to be achieved prior to its use in another country. So, how should the researcher go about translating a scale or experimental measure? According to the Guidelines for research in ethnic minority communities (CNPAAEMI, 2000), the proper translation of experimental measures involves the use of the back-translation method. First, a bilingual translator converts the measure to the target language. Next, a second translator converts the translated measure back to the original language. Finally, a researcher compares the original version to the back-translated version to examine any existing differences. The process is repeated until translation equivalence is achieved.

Conceptual equivalence is the degree to which theoretical concepts or constructs are the same between two cultures. Unfortunately, researchers can introduce conceptual nonequivalence in a variety ways, even through something as seemingly innocuous as demographics. Although it is often useful to control for certain demographic variables, such as age, sex, and religious orientation, this may become problematic when conducting cross-cultural research. For example, a demographic question such as “age” would seem relatively innocuous. Yet, for the Ju/’hoansi, also known as the !Kung of the Kalahari, this would be not be a useful question, as the Ju/’hoansi use a culturally specific age categorization system as opposed to thinking of age in chronological years (Hames & Draper, 2004). The inclusion of an indigenous (local) investigator or collaborator may serve to increase the validity of cross-cultural research because this inclusion can better enable the research team to ask the right questions (CNPAAEMI, 2000).

Finally, researchers need to be concerned with metric equivalence—the ability to compare the specific scores on a scale of interest across cultures. For example, would a researcher interpret a score of 25 on the Beck Depression Inventory (Beck, Ward, Mendelson, Mock, & Erbaugh, 1961) the same way across cultures? Would individuals living under repressive regimes, where government intrusion is a daily occurrence, score differently on the paranoia scale of the Minnesota Multiphasic Personality Inventory (MMPI-2; Butcher et al., 2001)? Additionally, metric equivalence may not exist within a culture when studying diverse populations. For example, researchers have rarely standardized measures of assessment used in research, particularly those related to personality assessment and psychopathology, for use with a disabled population (Elliott & Umlauf, 1995; Pullin, 2002). In addition, disabled individuals may score differently than others on the Hypochondriasis Scale of the MMPI-2 (Butcher et al., 2001), with seeming obsession over body functions. Varying cultural norms about time, differences in physical ability, and age may all impact completion of timed tests of intellectual ability.

Inadequately standardized measures that lack a representative normative group threaten construct validity by introducing the possibility that individuals are evaluated on performance as opposed to ability. Unfortunately, researchers rarely design studies to accommodate participants who are differently abled. Tasks designed to measure cognitive, personality, or other psychological abilities may instead be measuring primarily noncognitive or nonpsychological variables in research inclusive of participants with a range of abilities. For example, research on memory using computers to present stimuli may be more challenging for some older adults due to an increased incidence in susceptibility to glare from cataracts. Research participants of any age who are in ill health may fatigue more easily and perform poorly on outcome measures. Thus, differences found between participants may reflect visual or health differences as opposed to memory differences. Unless the researcher takes into account the impact of disability, age, or other factors, the resulting research conclusions may be biased and inaccurate.

Internal validity

Internal validity reflects the confidence with which we can draw cause and effect conclusions from our research results. Of primary concern for researchers is the ability to eliminate, minimize, or hold constant all extraneous or confounding variables. Unfortunately, studies examining differences between various elements of diversity (e.g., gender, culture) are by their very nature quasi-experimental and hence limited in relation to cause and effect conclusions. Indeed, this issue may contribute to the paucity of diversity research within psychology. In a provocative article, Chang and Sue (2005) suggested that the current scientific paradigm, with its focus on experimental designs, has introduced a bias in psychology. Specifically, they asserted that mainstream journals systematically exclude multicultural research, thus devaluing existing multicultural research and impeding the growth of research in this area. Ideally, experimental research should be high in both internal validity and external validity. Unfortunately, it is very difficult to achieve both in experimental designs, and extremely problematic when using alternative designs (e.g., quasi-experimental, correlational, observational, ethnographic). Chang and Sue argued that the importance psychologists have placed on determining causality has led researchers to value internal validity over external validity.

Chang and Sue (2005) further asserted that the emphasis psychology places on causality (internal validity) has led to a host of additional research problems, which only serve to devalue external validity and the very nature of multicultural research. The problems they discussed included: (a) overuse of college students as research participants; (b) willingness to assume research conducted on one population (e.g., White, middle-class, U.S. citizens) can be generalized to other groups or contexts; (c) disregard for research seeking to explore cross-cultural differences as opposed to explaining such differences; (d) the tendency of journal reviewers to insist that researchers add a White control group when conducting research on ethnic minority groups; and (e) the formation of aggregate non-White populations to obtain a large enough sample size. The last issue is especially problematic due to the implied assumption of homogeneity among the non-White population, when in fact non-White groups may differ widely. Although the exclusion of diverse participants may enhance internal validity, the resultant homogeneous sample does not reflect the diversity of human experience.

External validity

External validity is the degree to which research results can be generalized beyond one’s sample. Often psychology focuses on the universality of human cognition and behavior. Researchers often refer to these universal concepts as etics. Concepts thought to be specific to a particular culture are called emics. Matsumoto (1994) suggested that, “most cross-cultural psychologists would agree that there are just as many, if not more, emics as there are etics” (p. 5). Many psychology textbook authors present traditional descriptions and recommended treatments for posttraumatic stress disorder (PTSD) as universal, or etics. Yet, PTSD symptoms and treatment are frequently culture-specific (Atlani & Rousseau, 2000; Bracken, 1998). The primary lesson of external validity is that researchers should exercise great care when extrapolating beyond their sample and this is particularly true when conducting cross-cultural or diversity research.

One of the primary issues at the core of external validity in relation to cross-cultural research is that variability exists within cultures. For example, the Guidelines for research in ethnic minority communities (CNPAAEMI, 2000), in relation to issues of using Hispanics or Asian/Pacific Islanders in research, stated that above all else, researchers need to be aware of the diversity of these populations. These broadly defined groups vary with respect to acculturation, language, race/ethnicity, beliefs, socioeconomic status, and educational background. Indeed, they do not represent homogenous cultures. Anyone critiquing research involving Asian/Pacific Islander or Hispanic participants must consider all these issues. These same issues are also fundamental to research guidelines associated with Native American Indians. In addition to research design concerns, investigators need to be prepared for small sample sizes. In fact, researchers may end up dealing with a single and consequently small population, given the fact that each tribe is a distinct cultural entity.

The issue of external validity also applies to diversity research within cultures. For example, Eichler (1991) cited the problem of overgeneralization in gender research—the propensity to extend concepts, theories, and research developed from studies involving participants of one gender to all individuals. Examples of overgeneralization ranged from the use of sexist language to the tendency of researchers to generalize the results of a study to all individuals when in fact the initial sample was composed only of men. Hyde (1996) also asserted that researchers often employ a “female deficit model” when drawing conclusions—the tendency to frame the results in such a fashion that women’s behavior is seen as deficient or non-normative. Research conducted in highly patriarchal cultures may magnify this issue.

Methodological Design Concerns

Many of the concerns we have discussed under validity intersect with methodological design concerns. However, several topics deserve further discussion and include methodological concerns related to demographics and sampling, experimenter bias, and the use of quasi-experimental design.

Demographics and sampling

At first glance, the issue of demographics and sampling may appear to be simple. However, when one includes diverse populations or studies diversity both between and within cultures, these are highly complex and require careful thought and foresight. Some of the major issues impacting demographics and sampling involve:

(1) self-identification; (2) hidden populations; (3) exclusion; and (4) faulty biologically based conclusions. Experimenter preconceptions about groups and cultures influence all of these issues.

When conducting research, it is important that investigators use culturally relevant and appropriate demographic categories regardless of whether the research involves participants within or between cultures. Particularly important is an appreciation for the role that culture may play in shaping a participant’s self-identification. For example, if you are conducting cross-cultural research, is it more appropriate to use the word “Hispanic” or “Latino/a” in the United States? Can you include a broad category such as Asian or does one need to be more specific: “Japanese,” “Chinese,” “Hmong,” or “Karen” (noting that this listing represents just a tiny fraction of the various ethnicities/cultures often lumped together under the term “Asian”)? To better address questions such as these, researchers often collaborate with individuals who are native or knowledgeable about the cultures under study. Familiarity with the culture, including diversity within the culture under study (e.g., ethnic groups, language, religion), is essential to avoid bias and stereotypes on the part of the experimenter.

The issue of self-identification applies not just to cultural differences but also variations within culture, such as gender and sexual orientation. Gays and lesbians have often eschewed the term “homosexuality” because “homosexuality” has been historically associated with disease and mental illness. Today, some researchers elect to use “queer” as a more inclusive term replacing gay and lesbian and inclusive of bisexuals and transgendered individuals (Grace, Hill, Johnson, & Lewis, 2004). Moreover, it is not uncommon to see the abbreviation GLBTQI (gay, lesbian, bisexual, transgendered or transsexual, queer or questioning, intersex) used by sexual minorities within the U.S. Even more challenging, from a research perspective, is the fact that some individuals who have had same-sex sexual relationships may still self-identify as heterosexual (Savin-Williams, 2001) and transgendered or intersex (biological sex is ambiguous) individuals may identify as either male or female. This ambiguity and interplay among all these constructs makes simple, non-fluid gender and sexual orientation self-identification more ambiguous. Certainly, the juxtaposition of identity issues and the need for demographics creates the basis for problematic research hypotheses and sampling challenges. Nonetheless, researchers should avoid overly simplistic categorizations for the sake of expediency as this may introduce a source of bias into a research project.

Many groups around the globe are hidden from view due to cultural, religious, or legal strictures. For example, in some cultures, adultery may result in honor killings, homosexuality is punishable by death or banishment, and certain religious groups face imprisonment. Therefore, these populations often remain hidden as a matter of safety and to avoid oppression. Attempts to include and study such populations are thus difficult.

Even within the U.S., there remain hidden populations due to stigma and stereotype (see Chapter 12 in this volume). For example, attitudes concerning the subject of sexual orientation have changed significantly in the past 20 years. According to recent Gallup Polls (Saad, 2008), Americans have become increasingly more tolerant of homosexuality. In 1982, only 34% of participants agreed that homosexuality was an acceptable alternative lifestyle. This number had risen to 57% when participants responded to the same question in 2008. Despite these changing attitudes, research inclusive of or concerning sexual orientation remains a small fraction of the literature and appears largely in specialized journals or special issues of journals (Bowman, 2003). Many individuals may be reticent to self-identify as lesbian, gay, or bisexual due to fears concerning confidentiality or internalized homophobia. Thus, researchers are often limited to a very select group of the population who self-identify as lesbian, gay, or bisexual—typically White, formally educated, urban, and upper middle class. In response to the challenges associated with obtaining a representative sample, researchers are currently exploring new methodologies such as the use of the Internet for research purposes (Koch & Emrey, 2001) and the use of alternative and qualitative methodologies related to the study of sexual orientation (Morrow, 2003). It is important to note that this reflects just one instance of the difficulty of sampling hidden populations. Researchers studying any number of populations (e.g., the mentally ill or minority religions) that are not immediately identifiable based on some external characteristic, but belong to a class of individuals socially stigmatized because of group status, will experience difficulties, particularly with regard to sampling.

Unfortunately, researchers often exclude individuals, both intentionally and unintentionally, from research participation based on a variety of factors. Such factors include, but are not limited to, the ability to get to the site of the research, literacy, gender, language, political oppression, socioeconomic status, religious affiliation, and disability. Such exclusions limit the generalizability of the research. The pool of available participants may be limited if participants are unable to read a questionnaire, lack the means to travel to the research site, or fear governmental reprisals for taking part in a research study. Consequently, those who become involved in the research may be different than those individuals who do not elect to or cannot participate.

Individuals with disabilities are often systematically excluded from research both in the U.S. and abroad. According to the World Programme of Action Concerning Disabled Persons (United Nations, 2009) at least 10% of the world’s population is impaired by a physical, mental, or sensory disability. Within the U.S., researchers estimate that there are almost 44.1 million community dwelling people (5 years and older) with disabilities (U.S. Census Bureau, 2007c). Within the U.S. definitions of disability include sensory, physical, mental, self-care, homebound, and employment categories. Disabilities can be visible or hidden (e.g., heart defect), and individuals may or may not define themselves as disabled depending on the degree of limitation the disability has on their lives. Nonetheless, some investigators fail to accommodate disabled participants, thereby systematically excluding them from research. Other researchers simply bar individuals with disabilities from research participation altogether, often as the result of experimenter bias—the assumption that disability has broad negative effects on cognition, personality, affect, or ability. Of course, this assumption is grounded largely in stereotype rather than reality (Dunn, 2000).

The five ethnic minority associations that compose the CNPAAEMI worked collaboratively to create the Guidelines for research in ethnic minority communities (2000). The guidelines provided a wealth of information concerning research with ethnically diverse populations, with an important caveat. The CNPAAEMI warned that it is important for researchers to take care when using race as a demographic category, lest the variable become a proxy for biological explanations of behavior (Wang & Sue, 2005). Similar concerns surround the use of gender in psychological research. For example, McHugh and Cosgrove (2002) suggested that by attributing differences in men and women to gender differences “we risk essentializing gender, promoting a view of women as a homogenous group, and reinforcing the very mechanisms of oppression against which we are fighting” (p. 13) and note that the use of only “male/men” or “female/women” as categorical terms excludes those who identify as transsexual, transgender, or intersex.

Certainly, race and gender as variables are focal social and cultural constructs that, if used judiciously, can provide a wealth of information. The Guidelines for research in ethnic minority communities invited researchers to learn more about the rich diversity among individuals defined as belonging to a particular race and avoid treating racial categories as homogenous groups. They also recommended that researchers avoid non-representative but perhaps easy-to-reach samples. For example, it may be easy to conduct research on adolescents abroad, using as a sample those who attend a private English-speaking boarding school in another country, but this population may not be representative of the majority population in that country.

Experimenter bias

In addition to the impact on participant demographics and samples, experimenter bias can also affect the nature of an investigator’s worldview and subsequent research hypotheses. For any cross-cultural study, researchers must be familiar with all aspects of the society (e.g., language, religion, kinship patterns, etc.). Without such knowledge, any research questions, methods, and conclusions may be biased and untrustworthy. Moreover, it is imperative that researchers take into consideration the role of other diversity concerns within that culture as well. For example, research around the globe may be impacted by what Eichler (1991) termed androcentricity—the tendency for researchers to view the world from a male perspective. The research process perpetuates androcentrism in many ways. In an influential article, Denmark, Russo, Frieze, and Sechzer (1988) discussed several examples of sexism in research. For instance, the authors discussed the tendency for researchers to examine concepts such as leadership style in a fashion emphasizing male stereotypes (e.g., dominance, aggression) as opposed to a range of leadership styles, including those that are more egalitarian. Another example cited by Denmark and colleagues involved the propensity for some researchers to assume that topics relevant to White males are inherently more important than issues related to other groups. Binion (2006) argued that cultures have traditionally viewed women’s human rights (e.g., marital rape and other forms of domestic abuse) as existing within the private sphere of the home outside of public view, and hence have often ignored them. The false dichotomy of public/private sphere has unfortunately negatively influenced the actions of communities and law enforcement, and has limited political asylum opportunities for women around the globe.

Experimenter bias has also impacted the conclusions reached by researchers studying disability both within and outside of the United States. According to Dunn (2000), “Insiders (people with disabilities) know what disability is like, whereas outsiders (people without disabilities) make assumptions, including erroneous ones, about it” (p. 574). Most notably, non-disabled individuals overestimate the effect of a disability on an individual’s capacities and well-being. In addition, researchers explicitly studying participants with known disabilities may fail to consider all possible alternative explanations, such as cultural, situational, or interpersonal factors, and simply attribute the results to the disability. For example, researchers comparing math scores among disabled and non-disabled populations need to take into account the role of stereotype threat (Steele, Spencer, & Aronson, 2002) when discussing any performance differences between the groups.

Research design

Typically, research examining cross-cultural differences or focusing on diversity-related variables such as gender, age, or sexual orientation, employs a quasi-experimental design. A researcher examining differences in age may compare younger and older adults or a cross-cultural psychologist may research differences between ethnic groups either within or across nations. Unfortunately, the use of distinct samples in a quasi-experimental design typically results in low internal validity. These samples may differ on any number of variables other than the one under investigation. In a cross-sectional study (in which two or more age groups are compared), age differences may be confounded with generational, non-shared cultural, or historical cohort experiences—making it difficult to determine causality. For example, early cross-sectional studies suggested progressive declines in intelligence following late adolescence and early young adulthood. However, research taking account of cohort effects (e.g., differences in educational levels) did not support this gloomy picture (Schaie, 1986; 2000).

Similarly, comparisons of ethnic groups or race within a culture may be confounded by differences in religion, language, socioeconomic status, or other factors not under study by the researchers. Unfortunately, researchers may assume differences in heredity or genetics to explain the results of cross-cultural studies or those involving race or ethnicity. This faulty assumption can lead to erroneous and potentially damaging conclusions. For example, throughout the twentieth century some researchers were still asserting that White Americans were intellectually superior to minority groups—specifically African Americans (e.g., Hernstein & Murray, 1994; Jensen, 1969). However, Sternberg, Grigorenko, and Kidd (2005) noted, “Race is a social construction, not a biological construct, and studies currently indicating alleged genetic bases of racial differences in intelligence fail to make their point even for these social defined groups” (p. 57).

Meta-analyses have become an increasingly popular tool for psychologists. For example, a search of PsycINFO© revealed dozens of meta-analyses examining gender as well as cultural differences across a variety of issues. Although meta-analyses have almost certainly added to psychologists’ knowledge of humanity, researchers need to exercise caution when drawing conclusions from such studies. For example, in relation to gender, Halpern (1995) expressed concern that researchers ignore context when using benchmarks to determine whether a gender-related effect size is meaningful or important. The tendency to ignore context is further compounded by the fact that gender research often fails to use representative samples. Halpern cautioned that “as long as the research literature is based on a narrow selection of participants and assessment procedures, the results of any analysis will be biased” (p. 83).

Research Ethics

The APA’s Ethical principles of psychologists and code of conduct (2002) highlights a range of research ethics topics such as confidentiality, informed consent, the use of deception, and the obligation to be accurate in reporting results. Moreover, according to the Ethical principles, all psychologists must “strive to benefit those with whom they work and take care to do no harm” (p. 3). Although all of these concerns are fundamental to psychological research, issues of informed consent and confidentiality are especially important in cross-cultural research, particularly in relation to protection from harm.

In addition, according to the Ethical principles, researchers conducting any project must inform participants about a variety of features such as purpose, procedures, risks and benefits, the limits of confidentiality, and the right to withdraw from participation. As with all statements of informed consent, it is essential that the document be clear and unambiguous—this is particularly important in cross-cultural research. Researchers must develop consent forms with an understanding of the culture of participants in mind, inclusive of special needs related to language, reading ability, nation-specific legal standards, and cultural norms. Consequently, a “boiler-plate” statement simply translated into a different language often does not meet the standards of informed consent. Moreover, in some cultures, women may not be culturally free to give informed consent but must first receive the consent of a male guardian.

Individuals with developmental or cognitive disabilities represent a challenge to informed consent (Dresser, 2001). Informed consent rests on the premise that research participants can fully understand the procedures, risks, and benefits associated with a particular study and agree to participate. Unfortunately, individuals with significant cognitive impairment due to developmental disabilities, injury, or dementia may not fully understand the material provided to give informed consent. Researchers must take special care to protect participants who are unable to provide consent on their own, including situations where a guardian provides consent but the participants undergo experimental testing without their knowledge. Unfortunately, due to the potential risks involved, many researchers simply avoid research with cognitively disabled populations (Yan & Munir, 2004).

Researchers need to be particularly aware of issues of confidentiality when conducting cross-cultural research. Seltzer and Anderson (2001) noted that researchers should use care even with aggregate data, as some individuals or small groups may remain identifiable. Within this discussion, they highlighted the role that official statistics have played in historical incidents of forced migration, internment, and genocide. The United Nations Statistical Commission (1994) passed the Fundamental principles of official statistics, which stated, “Official statistics provide an indispensable element in the information system of a democratic society, serving the government, the economy and the public with data about the economic, demographic, social and environmental situation.” Nonetheless, the UNSC asserted that researchers must protect individuals’ privacy and guard against the misuse of statistics. It is clear that researchers must carefully protect the confidentiality of all participants (e.g., including no easily identifiable information about the participants on forms, questionnaires, etc.) in cultures where the researcher’s data may be taken by family members, religious officials, or governmental agents for use against the well-being of the participant.

Conclusion

Increasingly, we hear that we are living within a global community. If this is to be a true and inclusive statement, then research needs to reflect the tapestry of human existence, noting variations across a range of biological and social constructs (e.g., sex and race, respectively) and cultures. In this chapter, we have discussed a host of constructs such as gender and race but this does not represent the full spectrum of human expression. Other areas of diversity significantly impacting individuals’ identities and their place in communities include, but are by no means limited to, socioeconomic status, linguistic variability, educational level, differing sects within religious groups, profession, marital status, physical and mental health, social class/caste, computer literacy, access to resources, political affiliation, and physical appearance. Ultimately, how we identify ourselves is incredibly variable and influenced by the currents of time, our communities, and our culture.

To become an effective and critical consumer and producer of research today requires cross-cultural and diversity-related understanding, knowledge, and skills. Individuals need to expand their horizons by reading research from around the globe, on a variety of topics and perhaps inclusive of research from related fields such as sociology, anthropology, history, and international studies. It is important to think of psychological research not so much as a template through which all topics can be studied (e.g., independent/dependent variables; experimental/ control groups) but as a flexible tool that can be used creatively to explore topics of relevance across a range of populations and communities. With an awareness of alternative methodologies, input from collaborators with varying backgrounds, and an understanding of the unique ethical concerns involved in cross-cultural research, individuals can engage in exciting and important research that is both inclusive and reflective of human diversity across the global community.

References

Adair, J. G., Coelho, A. E. L., & Luna, J. R. (2002). How international is psychology? International Journal of Psychology, 37, 160–170.

American Psychological Association (APA) (2002). Ethical principles of psychologists and code of conduct. Retrieved September 13, 2009, from http://apa.org/ethics/code2002.xhtml

American Psychological Association (APA) (2003). Guidelines on multicultural education, training, research, practice, and organizational change for psychologists. American Psychologist, 58, 377–402.

American Psychological Association (APA) (2009). Vision statement. Retrieved September 1, 2009, from http://www.apa.org/about

Atlani, L., & Rousseau, C. (2000). The politics of culture in humanitarian aid to women refugees who have experienced sexual violence. Transcultural Psychiatry, 37, 435–449.

Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561–571.

Bernal, G., Trimble, J. E., Burlew, A. K., & Leong, F. T. L. (2003). Introduction: The psychological study of racial and ethnic minority psychology. In G. Bernal, J. E. Trimble, A. K. Burlew, & F. T. L. Leong (Eds.), Handbook of racial and ethnic minority psychology (pp. 1–12). Thousand Oaks, CA: Sage.

Betancourt, H., & López, S. R. (1993). The study of culture, ethnicity, and race in American psychology. American Psychologist, 48, 629–637.

Binion, G. (2006). Human rights: A feminist perspective. In B. B. Lockwood (Ed.), Women’s rights: A Human Rights Quarterly reader (pp. 70–86). Baltimore, MD: Johns Hopkins University Press.

Bowman, S. L. (2003). A call to action in lesbian, gay, and bisexual theory building and research. The Counseling Psychologist, 31, 63–69.

Bracken, P. J. (1998). Hidden agendas: Deconstructing post traumatic stress disorder. In P. J. Bracken & C. Petty (Eds.), Rethinking the trauma of war (pp. 38–59). New York: Free Association Books.

Brislin, R. (2000). Understanding culture’s influence on behavior (2nd ed.). New York: Wadsworth.

Buboltz, W. C., Jr., Miller, M., & Williams, D. J. (1999). Content analysis of research in the Journal of Counseling Psychology (1973–1998). Journal of Counseling Psychology, 46, 496–503.

Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, W. G., & Kaemmer, B. (2001). Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for administration, scoring and interpretation (rev. ed.). Minneapolis, MN: University of Minnesota Press.

Carter, R. T., Akinsulure-Smith, A. M., Smailes, E. M., & Clauss, C. S. (1998). The status of racial/ethnic research in counseling psychology: Committed or complacent? Journal of Black Psychology, 24, 322–334.

Chang, J., & Sue, S. (2005). Culturally sensitive research: Where have we gone wrong and what do we need to do now? In M. G. Constantine & D. W. Sue (Eds.), Strategies for building multicultural competence in mental health and educational settings (pp. 229–246). Hoboken, NJ: John Wiley & Sons.

Council of National Psychological Associations for the Advancement of Ethnic Minority Interests (CNPAAEMI) (2000). Guidelines for research in ethnic minority communities. Washington, DC: American Psychological Association.

Delgado-Romero, E. A., Galván, N., Maschino, P., & Rowland, M. (2005). Race and ethnicity in empirical counseling and counseling research: A 10-year review. The Counseling Psychologist, 33, 419–448.

Denmark, F., Russo, N. F., Frieze, I. H., & Sechzer, J. A. (1988). Guidelines for avoiding sexism in psychological research: A report of the ad hoc committee on nonsexist research. American Psychologist, 43, 582–585.

Dresser, R. (2001). Research participants with mental disabilities: The more things change. In L. E. Frost & R. J. Bonnie (Eds.), Evolution of mental health law(pp. 57–74). Washington, DC: American Psychological Association.

Dunn, D. S. (2000). Social psychological issues in disability. In R. G. Frank & T. R. Elliott (Eds.), Handbook of rehabilitation psychology (pp. 565–584). Washington, DC: American Psychological Association.

Eichler, M. (1991). Nonsexist research methods: A practical guide. New York: Routledge.

Elliott, T. R., & Umlauf, R. L. (1995). Measurement of personality and psychopathology following acquired physical disability. In L. A. Cushman & M. J. Scherer (Eds.), Psychological assessment in medical rehabilitation (pp. 301–324). Washington, DC: American Psychological Association.

Grace, P. A., Hill, R. J., Johnson, C. W., & Lewis, J. B. (2004). In other words: Queer voices/ dissident subjectivities impelling social change. International Journal of Qualitative Studies in Education, 17, 301–324.

Graham, S. (1992). “Most of the subjects were White and middle class”: Trends in published research on African Americans in selected APA journals, 1970–1989. American Psychologist, 47, 629–639.

Guthrie, R. V. (1998). Even the rat was white: A historical view of psychology (2nd ed.). Boston: Allyn and Bacon.

Halpern, D. F. (1995). Cognitive gender differences: Why diversity is a critical research issue. In H. Landrine (Ed.), Bringing cultural diversity to feminist psychology: Theory, research, and practice (pp. 77–92). Washington, DC: American Psychological Association.

Hames, R., & Draper, P. (2004). Women’s work, child care, and helpers-at-the-nest in a hunter-gatherer society. Human Nature, 15, 319–341. Hernstein, R. J., & Murray, C. (1994). The bell curve: Intelligence and class structure in American life. New York: Free Press.

Hyde, J. S. (1996). Half the human experience: The psychology of women (5th ed.). Lexington, MA: DC Heath and Company.

Jensen, A. R. (1969). How much can we boost IQ and scholastic achievement? Harvard Educational Review, 33, 1–123.

Koch, N. S., & Emrey, J. A. (2001). The Internet and opinion measurement: Surveying marginalized populations. Social Science Quarterly, 82, 131–138.

Lutsky, N., Torney-Purta, J., Veleyo, R., Whittlesy, V., Woolf, L., & McCarthy, M. (2005). American Psychological Association Working Group on Internationalizing Undergraduate Psychology Curriculum: Report and recommended learning outcomes for internationalizing the undergraduate curriculum. Retrieved September 14, 2009, from http://www.apa.org/ed/pcue/international.pdf

Marsella, A. J., (2001). Internationalizing the psychology curriculum. Psychology International, 12 (2), 7–8.

Matsumoto, D. (1994). Cultural influences on research methods and statistics. Belmont, CA: Wadsworth.

Matsumoto, D. (2003). Cross-cultural research. In S. F. Davis (Ed.), Handbook of research methods in experimental psychology (pp. 189–208). Malden, MA: Blackwell.

McHugh, M. C., & Cosgrove, L. (2002). Gendered subjects in psychology: Satirical and dialectic perspectives. In L. H. Collins, M. R. Dunlap, & J. C. Chrisler (Eds.), Charting a new course for feminist psychology (pp. 3–19). Westport, CT: Praeger.

Morrow, S. L. (2003). Can the master’s tools ever dismantle the master’s house? Answering silences with alternative paradigms and methods. The Counseling Psychologist, 31, 70–77.

Munley, P. H., Anderson, M. Z., Baines, T. C., Borgman, A. L., Briggs, D., Dolan, J. P., Jr., et al. (2002). Personal dimensions of identity and empirical research in APA journals. Cultural Diversity and Ethnic Minority Psychology, 8, 357–365.

Pullin, D. (2002). Testing individuals with disabilities: Reconciling social science and social policy. In R. B. Ekstrom, & D. K. Smith (Eds.), Assessing individuals with disabilities in educational, employment, and counseling settings (pp. 11–31). Washington, DC: American Psychological Association.

Saad, L. (2008, June 18). Americans evenly divided on morality of homosexuality: However, majority supports legality and acceptance of gay relations. Princeton, NJ: The Gallup Organization.

Savin-Williams, R. C. (2001). Mom, Dad. I’m gay. How families negotiate coming out. Washington, DC: American Psychological Association.

Schaie, K. W. (1986). Beyond calendar definitions of age, time, and cohort: The general developmental model revisited. Developmental Review, 6, 252–277.

Schaie, K. W. (2000). The impact of longitudinal studies on understanding development from young adulthood to old age. International Journal of Behavioral Development, 24, 257–266.

Sears, D. O. (1986). College sophomores in the laboratory: Influences of a narrow data base on social psychology’s view of human nature. Journal of Personality and Social Psychology, 51, 515–530.

Seltzer, W., & Anderson, M. (2001). The dark side of numbers: The role of population data systems in human rights abuses. Social Research, 68, 481–513.

Steele, C. M., Spencer, S. J., & Aronson, J. (2002). Contending with group image: The psychology of stereotype and social identity threat. In M. Zanna (Ed.), Advances in experimental social psychology, Vol. 34 (pp. 379–440). San Diego, CA: Academic Press.

Sternberg, R. J., Grigorenko, E. L., & Kidd, K. K. (2005). Intelligence, race, and genetics. American Psychologist, 60, 46–59.

Sue, S. (1999). Science, ethnicity, and bias: Where have we gone wrong? American Psychologist, 54, 1070–1077.

United Nations (2009). World Programme of Action Concerning Disabled Persons. Retrieved September 14, 2009, from http://www.un.org/disabilities/

United Nations Statistical Commission (1994). Fundamental principles of official statistics. Retrieved September 14, 2009, from http://unstats.un.org/unsd/methods/statorg/FP-English.htm

United States Census Bureau. (2007a). Minority population tops 100 million. Retrieved September 15, 2009, from http://www.census.gov/Press-Release/www/releases/archives/population/010048.xhtml

United States Census Bureau. (2007b). More than 300 counties now “Majority-Minority”. Retrieved September 15, 2009, from http://www.census.gov/Press-Release/www/releases/archives/population/010482.xhtml

United States Census Bureau. (2007c). 2005–2007 American community survey. Retrieved September 15, 2009, from http://www.census.gov/acs/www/index.xhtml

Wang, V. O., & Sue, S. (2005). In the eye of the storm: Race and genomics in research and practice. American Psychologist, 60, 37–45.

Woolf, L. M., Hulsizer, M. R., & McCarthy, T. (2002). International psychology: A compendium of textbooks for selected courses evaluated for international content. Office of Teaching Resources in Psychology. Retrieved September 15, 2009, from http://teachpsych.org/otrp/resources/woolf02intcomp.pdf

Yan, E. G., & Munir, K. M. (2004). Regulatory and ethical principles in research involving children and individuals with developmental disabilities. Ethics and Behavior, 14, 31–49.