Challenges to Cross-Cultural Research

TypesofComparativeStudiesinCross-CulturalPsychology.pdf

Home >Psychology homework help >Challenges to Cross-Cultural Research

Unit 2 Theoretical and Methodological Issues Subunit 2 Methodological Issues in Psychology and Culture Article 2

3-1-2009

Types of Comparative Studies in Cross-Cultural Psychology Fons J. R. van de Vijver Tilburg University, The Netherlands, [email protected]

This Online Readings in Psychology and Culture Article is brought to you for free and open access (provided uses are educational in nature)by IACCP and ScholarWorks@GVSU. Copyright © 2009 International Association for Cross-Cultural Psychology. All Rights Reserved. ISBN 978-0-9845627-0-1

Recommended Citation van de Vijver, F. J. (2009). Types of Comparative Studies in Cross-Cultural Psychology. Online Readings in Psychology and Culture, 2(2). https://doi.org/10.9707/2307-0919.1017

http://scholarworks.gvsu.edu/orpc/

https://scholarworks.gvsu.edu/orpc/vol2

https://scholarworks.gvsu.edu/orpc/vol2/iss2

https://scholarworks.gvsu.edu/orpc/vol2/iss2/2

https://doi.org/10.9707/2307-0919.1017

http://scholarworks.gvsu.edu/orpc/

http://www.iaccp.org/drupal/

mailto:[email protected]

Types of Comparative Studies in Cross-Cultural Psychology

AbstractAbstract From a methodological perspective cross-cultural studies in psychology differ in three dimensions. First, cross-cultural psychological studies can be exploratory or test specific hypotheses. Second, some cross-cultural studies compare countries or ethnic groups while other cross-cultural studies relate specific characteristics of a country or ethnicity (e.g., socialization patterns or religiosity) to psychological variables. Third, studies can compare either constructs (e.g., do Chinese and Kenyans mean the same when they say that a person is intelligent?) or score levels (e.g., are Americans more extravert than Italians?). A classification of cross-cultural psychological studies, based on the three dimensions, is presented and examples are given.

Creative Commons License

This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

This article is available in Online Readings in Psychology and Culture: https://scholarworks.gvsu.edu/orpc/vol2/iss2/2

http://creativecommons.org/licenses/by-nc-nd/3.0/

https://scholarworks.gvsu.edu/orpc/vol2/iss2/2

INTRODUCTION

Types of Cross-Cultural Studies

The Argentinean author Jorge Borges once proposed a

taxonomy of animals that he claimed to have found in a

Chinese encyclopedia

(http://www.multicians.org/thvv/borges-animals.html):

1. those that belong to the Emperor,

2. embalmed ones,

3. those that are trained,

4. suckling pigs,

5. mermaids,

6. fabulous ones,

7. stray dogs,

8. those included in the present classification,

9. those that tremble as if they were mad,

10. innumerable ones,

11. those drawn with a very fine camelhair brush,

12. others,

13. those that have just broken a flower vase,

14. those that from a long way off look like flies.

The wonderful creativity of the taxonomy provides a good introduction for the central topic

of the present chapter: How can we categorize cross-cultural studies? The classification by

Borges clearly illustrates that there are various ways of classifying and that some are more

useful than others. The same issue plays a role in categorizing cross-cultural studies.

Many categorizations can be envisaged, but not all of them are equally consequential.

Common distinctions are between cross-national and intranational studies; the

former involves different countries, while in the latter different cultural groups are studied

that live in a single country. Examples of the latter are the numerous studies in which

European Americans and African Americans or Hispanics are involved. In European

countries intranational studies often compare majority group members and migrants or

refugees or examine topics that are specific for migrants, such as acculturation processes.

Examples of cross-national studies are the numerous comparisons of American and East

Asian countries, such as Japan and China. Another categorization system can be based

on the various psychological disciplines in which cross-cultural studies are carried out,

such as social psychology, personality, and developmental psychology. An application of

this perspective is particularly useful if one wants to identify areas of interest in cross-

cultural psychology; social behavior is the most frequently examined behavioral domain in

cross-cultural psychology. Still another perspective refers the distinction between cultural

van de Vijver: Types of Comparative Studies in Cross-Cultural Psychology

Produced by The Berkeley Electronic Press, 2011

and cross-cultural psychology; the latter is culture comparative (a journal mainly devoted

to comparative studies is the Journal of Cross-Cultural Psychology, see

http://jcc.sagepub.com/), while the former is more focused on an in-depth analysis of

cultural phenomena in a specific culture (journal: Culture and Psychology, see

http://cap.sagepub.com/). The present chapter only deals with comparative studies.

A Methodological Classification of Cross-Cultural Studies

The perspective to categorize studies of the current chapter is methodological. Three

dimensions are introduced. The first dimension refers to the question as to whether

contextual factors are included in a study. Contextual factors refer here to a wide variety of

variables that could influence the cross-cultural differences observed; these variables may

involve either participant characteristics (such as socioeconomic status, education, and

age) or culture characteristics (such as a country's affluence and institutions). Many cross-

cultural studies do not include contextual factors; these studies are interested in the

comparison of countries. Examples are the large cross-national comparisons of

educational achievement, such as the TIMMS project (http://timss.bc.edu), in which the

performance of secondary school students in mathematics and science is compared

across many countries. One of the most essential parts of reports of such studies is a table

in which the scores of the countries are ranked. In other cross-cultural studies, however,

more attention is paid to cultural factors and countries are deliberately chosen because of

some characteristic they have. An example is the currently popular distinction between

individualistic and collectivistic countries (see the chapter on individualism and collectivism

on this website).

Both types of studies have their own strengths and weaknesses. If studies include

many countries, they almost always have an intrinsic interest for cross-cultural

psychologists. The studies increase our insight in the cross-cultural differences and

similarities across these countries. Historically, they are often the precursors of more

focused studies in which country differences are seen as related to differences in

underlying dimensions. The latter type study is more precise than studies in which no

contextual factors were examined; therefore, they are often easier to interpret. When a

cross-cultural study involves only a few countries, problems of interpretability of

differences are often large. As an example, suppose that a self-esteem questionnaire has

been administered to adults in the USA and Iran, and that the mean score of the

Americans was higher. The seemingly obvious conclusion would be that American adults

on average have a higher self-esteem than Iranian adults. The conclusion might be valid,

but various alternative explanations can be envisaged. In many Islamic countries there is a

norm to be humble and not to brag about one's personal qualities; as a consequence,

Iranians may show lower scores. Also, unintended sample differences may account for the

difference in scores; the Americans may have had more education (which is known to be

positively related to self-esteem). It may well be that if one would examine samples from

the two countries that have the same average educational level, the differences in scores

on the questionnaire may become smaller or may even disappear altogether. Problems of

Online Readings in Psychology and Culture, Unit 2, Subunit 2, Chapter 2

https://scholarworks.gvsu.edu/orpc/vol2/iss2/2

interpretability are more salient in studies in which no cultural factors are included than in

studies that include contextual factors. If a measure of the educational level of the

participants as well as a questionnaire on the perceived norm about humility would have

been administered in both countries, statistical tools, such as regression analysis or

covariance analysis, could have been used to evaluate their influence on the cross-cultural

score differences observed.

The second dimension to classify cross-cultural studies refers to the distinction

between exploratory and hypothesis-testing studies that is commonly described in

introductory textbooks to methods of research. Exploratory studies attempt to increase

our understanding of cross-cultural differences by assuming a perspective that is as open

and unprejudiced as possible about the nature and size of cross-cultural differences; no

prior ideas are formulated about where these differences and similarities are to be

expected. Researchers often want to stay "close to the data" and are not inclined to make

large inferential jumps. Exploratory studies are helpful in initial stages of a research

paradigm in which it is not yet clear to what extent a theory, model, or instrument "works"

in another culture.

After the initial stage of exploratory studies, a researcher may feel more confident

about what to expect in other cultures. In these cases hypothesis-testing studies can be

carried out. In such studies theories or models about the relationship between

psychological and cultural phenomena are specified at beforehand and tested for

accuracy.

Both types of studies have their own strengths and weaknesses. The power to

detect differences and similarities in a large variety of domains in a single study is a

strength of exploratory studies. A broadband approach to cross-cultural differences

provides an efficient means to collect much information in an efficient way. The openness

of the exploratory approach also constitutes its weakness: exploratory studies can easily

become "fishing trips" in which the researcher wants to "catch" as much as possible. In

their most extreme form such studies address a multitude of cultural differences and

similarities without providing any overarching framework for the patterning of the

similarities of differences. In sum, exploratory studies are usually good at identifying cross-

cultural differences and similarities, but poor at providing a framework to interpret these

differences. The latter is the stronghold of hypothesis-testing studies, which combine

theoretical precision (testing specific cultural aspects) and statistical rigor.

The third dimension to classify cross-cultural studies refers to the kind of research

question addressed in a study. A distinction is made between structure-oriented and

level-oriented studies. As an example of a structure-oriented study, one could ask

whether the nature of intelligence differs across countries; the question is not how much

samples from various countries differ in intelligence, but whether intelligence is different

across countries. For example, it has been argued that Westerners tend to approach

problems in an analytic way, which means that a problem is reduced to its constituent

parts and solving the problem amounts to successfully dealing with all the parts in

succession. Easterners, on the other hand, are said to opt more frequently for a holistic

type of reasoning, in which the relations between the parts of a problem rather than the

van de Vijver: Types of Comparative Studies in Cross-Cultural Psychology

Produced by The Berkeley Electronic Press, 2011

parts themselves form the essence of a problem. Whatever the validity of the latter claim, it

is helpful to illustrate the issue that in many cases cross-cultural researchers are not

interested in quantitative differences, but in qualitative similarities and differences. These

structure-oriented studies focus on relationships among variables and attempt to identify

similarities and differences in these relationships across cultures.

Level-oriented studies examine the size of cross-cultural differences. Examples are

the numerous studies in which the level of individualism and collectivism are compared

across countries and the studies in which the school performances of American pupils and

Eastern Asian pupils are compared.

Structure- and level-oriented studies are complementary and often follow each other

in time. Studies that examine the similarity of structures across countries are often done

first. They pave the way for a second wave of studies in which scores are compared

across countries. Although the two kinds of studies often do not have the neat temporal

separation suggested here, it is important to realize that they address different questions

and that a numerical comparison of scores requires that an instrument measures the same

in each cultural group considered. Take the cross-cultural study of depression as an

example. Depression has a somatic component (e.g., sleeplessness and loss of appetite)

as well as a psychological component (e.g., feeling down and being pessimistic). There

are indications that individuals from different cultures with depressive symptoms show

more agreement in their somatic symptoms than in their psychological complaints. To

some extent this may be a consequence of differences in norms about expressing

personal feelings to others. A comparison of depression scores obtained in different

cultures can show misleading results if the symptoms (or at least the tendency to report

these) are not identical across cultures.

Examples

The three classification dimensions (i.e., contextual factors included or not included;

exploratory vs. hypothesis-testing; structure-oriented vs. level-oriented) produce a total of

8 (= 2 x 2 x 2) studies, as can be seen in Table 1.

The eight possibilities are illustrated on the basis of a fictitious set of studies (Table 2

briefly presents real examples, which are not further discussed here). Suppose that we

have a theory of emotions according to which each human emotion is a combination of

two, independent components: valence (positive and negative emotions) and intensity (low

and high intensity) and an instrument that has shown this structure in samples of British

psychology students. Each emotion is then seen a point in a two-dimensional space. In the

first type of study, structure-oriented psychological differences studies (the names and

order of Table 1 are followed here), the researcher may develop a new instrument for a

culture in which the instrument has not yet been administered; the development should

start from a thorough knowledge of the specific culture. The newly developed instrument is

then administered and the researcher examines whether the two-dimensional structure is

also present in the new sample.

Online Readings in Psychology and Culture, Unit 2, Subunit 2, Chapter 2

https://scholarworks.gvsu.edu/orpc/vol2/iss2/2

Table 1.

Types of Studies in Cross-Cultural Psychology (after van de Vijver & Leung, 1997).

Orientation more on

Consideration of

contextual factors Exploration Hypothesis testing

(a) Structure-oriented

No (1) Structure-oriented

psychological differences

(2) Structure-oriented

generalizability

Yes (3) Structure-oriented ecological

linkage

(4) Structure-oriented

contextual theory

(b) Level-oriented

No (5) Level-oriented psychological

differences

(6) Level-oriented

generalizability

Yes (7) Level-oriented ecological

linkage

(8) Level-oriented contextual

theory

In the second type of study, structure-oriented generalizability studies, one would

accumulate data from various countries with the instrument and check to what extent the

two-dimensional structure is found in each of these. In other words, the generality of the

structure elsewhere is addressed.

The third type of study, a structure-oriented ecological linkage study, could be used if

the two-dimensional structure would not be replicated everywhere. It is the challenge for

the researcher to determine which contextual factors influence the poor replicability (e.g.,

two other factors have been found in some countries). We can investigate whether the

countries in which the two factors were not found, differ from the countries in which the

two-dimensional British structure was found in country indicators, such as average income,

educational level, or extraversion.

In ecological linkage studies one often needs country indicators. The Internet is a

rich source of country-level data. Examples of interesting sites are http://www.un.org and

http://www.oecd.org, and www.worldbank.org (and its World Development Indicators for which a

subscription is required), http://www.adherents.com/ (for religion data). Anthropologists have

built a large database, the Human Relations Area File (HRAF;http://www.yale.edu/hraf), with

information on a large number of cultural characteristics, ranging from birth practices to

death rites, mainly from pre-industrial societies.

The fourth type, structure-oriented contextual theory-based studies, tests theories

about cross-cultural differences (or similarities) in structure. In particular when the

generalizability studies just described (type 2) would show that the two-dimensional

structure does not hold in all cultures examined, the need will arise to learn more about the

background of the differences. Structure-oriented contextual theory-based studies could

van de Vijver: Types of Comparative Studies in Cross-Cultural Psychology

Produced by The Berkeley Electronic Press, 2011

test to what extent the two-dimensional model of emotion fits better in countries with a

higher level of formal education, collectivism, higher proportions of religious people, with

less stringent socialization patterns, to name but a few (arbitrary) examples; in general, in

structure-oriented contextual theory-based studies a researcher tests hypotheses that

could explain the differences in fit.

Table 2.

Description for Each Type of Study

Type of study Source and description

Structure-oriented

psychological

differences

Source: Russell, J. A., & Sato, K. (1995). Comparing emotion words between languages. Journal of Cross-Cultural Psychology, 26, 384-391.

Description: The authors studied the meaning and equivalence of emotion words among English speaking, Japanese speaking, and Cantonese speaking individuals. A set of 14 photographs of faces was shown to the subjects and they were asked to judge to what extent the face shown in the picture was an expression of each of 14 emotion words. For any two language groups, a correlation index for an emotion word can be calculated based on the ratings of these groups on the 14 photographs. The higher the correlation, the more similar is the meaning of the emotion word across the two languages. Three comparison groups could be formed: English/Japanese, English/Cantonese, and Japanese/Cantonese, and these three groups could be compared on the correlations of the 14 emotion words. Results showed that the correlations were similar across the three comparison groups for 12 of the 14 emotion words

Structure-oriented

generalizability

Source: McCrae, R. R., Terraciano, A., & 79 Members of the Personality Profiles of Cultures Project (2005). Personality profiles of cultures: Aggregate personality traits. Journal of Personality and Social Psychology, 89, 407-425.

Description: The authors studied the generalizability of a well-known Western model of personality, the Five-Factor Model of personality, in many cultures, both Western and non-Western.

Structure-oriented

ecological linkage

Source: Van de Vijver, F. J. R., & Poortinga, Y. H. (2002). Structural equivalence in multilevel research. Journal of Cross-Cultural Psychology, 33, 141-156.

Description: They compared the meaning of Inglehart’s concept of postmaterialism across more than 30 countries. It was found that the concept does not have an identical meaning in countries with low and high Gross National Product.

Structure-oriented

contextual theory

Source: Sidanius, J., Pratto, F., & Rabinowitz, J. L. (1994). Gender, ethnic status, and ideological asymmetry. Journal of Cross-Cultural Psychology, 25, 194-216.

Description: Based on social dominance theory, the authors proposed that for members of high-status ethnic groups, social dominance orientation (i.e., the desire to establish hierarchical social relationships among social groups) should be positively related to group salience and differential group closeness. Group salience refers to the experienced salience of one's ethnic group membership, and differential group closeness refers to the emotional closeness of one's ethnic group to other ethnic groups. The

Online Readings in Psychology and Culture, Unit 2, Subunit 2, Chapter 2

https://scholarworks.gvsu.edu/orpc/vol2/iss2/2

stronger the social dominance orientation, the more salient is one's ethnic identity and the closer one feels toward one's ethnic group. For members of low-status groups, however, the relationship between social dominance orientation and group salience and differential group closeness should be weaker. This prediction was tested in the US with a group of whites, the high-status ethnic group, and a group of Blacks and Hispanics, the low- status groups.

Level-oriented

psychological

differences

Source: Guida, F. V., & Ludlow, L. H. (1989). A cross-cultural study of test anxiety. Journal of Cross-Cultural Psychology, 20, 178-190.

Description: The authors examined cross-cultural differences in test anxiety between American and Chilean school children. The latter group was found to display higher levels of test anxiety. No attempt was made to evaluate causal antecedents for these differences in this study.

Level-oriented

generalizability

Source: Amir, Y., & Sharon, I. (1987). Are social psychological laws cross- culturally valid? Journal of Cross-Cultural Psychology, 18, 383-470.

Description: The authors replicated a number of well-known Western social psychological studies with Israeli subjects. The authors were interested in the generalizability of findings from experimental social psychology obtained among Western subjects to an Israeli context. Significant main effects could often be replicated but interaction effects did not travel well.

Level-oriented

ecological linkage

Source: Van Hemert, D. D. A., Van de Vijver, F. J. R., Poortinga, Y. H., & Georgas, J. (2002). Structure and score levels of the Eysenck Personality Questionnaire across individuals and countries. Personality and Individual Differences.

Description: Differences in country scores on the three personality dimensions in Eysenck’s theory (psychoticism, neuroticism, and extraversion) have been reported. The question was addressed to which country-level variables these differences were related.

Level-oriented

contextual theory

Source: Van de Vijver, F. J. R. (1997). Meta-analysis of cross-cultural comparisons of cognitive test performance. Journal of Cross-Cultural Psychology, 28, 678-709.

Description: A meta-analysis was carried out in which different models that could presumably explain cross-cultural differences in scores on mental tests were tested. As an example, support was found for the hypothesis that differences in GNP and educational expenditure (per head) between cultural groups are positively related to differences on cognitive test scores.

Level-oriented psychological differences studies test the presence of cross-cultural

differences, often using a t test or analysis of variance. These studies are popular in the

literature. Suppose, that we administer a questionnaire measuring our two emotion

dimensions in different cultures. A level-oriented psychological differences study could test

the presence of differences in valence and intensity across cultures. In such studies the

researcher typically does not have prior ideas about where to expect cultural differences

on any dimension, but employs well-established statistical techniques (e.g., a t test) to

determine if the score differences observed reflect real differences or are mere sample

fluctuations that are so small that they can be safely ignored.

van de Vijver: Types of Comparative Studies in Cross-Cultural Psychology

Produced by The Berkeley Electronic Press, 2011

Level-oriented generalizability studies usually build on studies in Western countries

and examine to what extent differences observed there can be generalized to other

cultures. Suppose that we have asked participants to indicate the valence and intensity of

emotions experienced during the last week and that we consistently find that in Western

countries women are more expressive than men, as indicated that women show a higher

variation in reported emotions. Level-oriented generalizability studies would be studies in

new cultures that address the generality of the Western gender differences.

If these studies would find that the gender differences are not universal, the next

question would be to examine which country factors could be held responsible for the

difference. A level-oriented ecological linkage study could address this question by linking

the gender differences observed in the various studies to various country indicators, such

as gross national product and average level of education.

Finally, level-oriented contextual theory-based studies test a theory of such

differences. For example, Williams and Best (1990; reference: Williams, J. E., & Best, D. L.

(1990). Measuring sex stereotypes: A multination study. Beverly Hills, CA: Sage) have

argued that "national development may be accompanied by a reduction in the degree in

which women and men are viewed as 'psychologically different'" (p. 253). Based on this

model, we would predict a negative correlation between gender differences in standard

deviation of the valence and intensity of reported emotions on the one hand and some

indicator of national development (such as Gross Domestic Product per head) on the other

hand.

Conclusion

Hundreds of cross-cultural studies are published each year. These studies can be seen as

belonging to different types: they are exploratory or test hypotheses, they include or do not

include contextual variables, and they focus either on the structure of psychological

phenomena or they compare score levels obtained in different cultures. The rank numbers

of the eight types of study in Table 1 should not be seen as rankings going from less to

more (or from more to less) valuable studies. Rather, depending on the level of theory and

availability of data, co-researchers from other countries, resources and various other

issues, each of the eight cells can be appropriate. In each of the cells of Table 1 good and

bad studies can be carried out.

The dimensions underlying Table 1 may help to think about existing and new

studies. The dimensions may help researchers to appreciate the strengths and

weaknesses of studies and may help to think about design and analysis prior to the data

collection, which tends to improve the quality of a study.