social research
Aplia for The Practice of Social Research
After reading, go to “Online Study Resources” at the end of this chapter for instructions on how to use Aplia’s homework and learning resources.
C H A P T E R 7
Typologies, Indexes, and Scales
Introduction
Indexes versus Scales
Index Construction Item Selection Examination of Empirical
Relationships Index Scoring Handling Missing Data Index Validation The Status of Women:
An Illustration of Index Construction
Scale Construction Bogardus Social Distance
Scale Thurstone Scales Likert Scaling Semantic Differential Guttman Scaling
Typologies
Researchers often need to employ
multiple indicators to measure a
variable adequately and validly.
Indexes, scales, and typologies are
useful composite measures made
up of several indicators of variables.
C H A P T E R O V E R V I E W
50094_ch07.indd 197 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
198 ■ Chapter 7: Typologies, Indexes, and Scales
Introduction As we saw in Chapter 6, many social science con- cepts have complex and varied meanings. Making measurements that capture such concepts can be a challenge. Recall our discussion of content validity, which concerns whether we have captured all the different dimensions of a concept.
To achieve broad coverage of the various di- mensions of a concept, we usually need to make multiple observations pertaining to that concept. Thus, for example, Bruce Berg (1989: 21) advises in-depth interviewers to prepare essential ques- tions, which are “geared toward eliciting specific, desired information.” In addition, the researcher should prepare extra questions: “questions roughly equivalent to certain essential ones, but worded slightly differently.”
Multiple indicators are used with quantitative data as well. Suppose you’re designing a survey. Although you can sometimes construct a single questionnaire item that captures the variable of interest—“Sex: Male Female” is a simple example—other variables are less straightforward and may require you to use several questionnaire items to measure them adequately.
Quantitative data analysts have developed specific techniques for combining indicators into a single measure. This chapter discusses the con- struction of two types of composite measures of variables—indexes and scales. Although these measures can be used in any form of social re- search, they are most common in survey research and other quantitative methods. A short section at the end of this chapter considers typologies, which are relevant to both qualitative and quantitative research.
Composite measures are frequently used in quantitative research, for several reasons. First, social scientists often wish to study variables that have no clear and unambiguous single indicators. Single indicators do suffice for some variables, such as age. We can determine a survey respondent’s age by simply asking, “How old are you?” Similarly, we can determine a newspaper’s circulation by merely
looking at the figure the newspaper reports. In the case of complex concepts, however, researchers can seldom develop single indicators before they actually do the research. This is especially true with regard to attitudes and orientations. Rarely can a survey researcher, for example, devise single ques- tionnaire items that adequately tap respondents’ degrees of prejudice, religiosity, political orientation, alienation, and the like. More likely, the researcher will devise several items, each of which provides some indication of the variables. Taken individually, each of these items is likely to prove invalid or un- reliable for many respondents. A composite mea- sure, however, can overcome this problem.
Second, researchers may wish to employ a rather refined ordinal measure of a particular variable (alienation, say), arranging cases in several ordinal categories from very low to very high, for example. A single data item might not have enough categories to provide the desired range of variation. However, an index or scale formed from several items can provide the needed range.
Finally, indexes and scales are efficient devices for data analysis. If considering a single data item gives us only a rough indication of a given variable, considering several data items can give us a more comprehensive and more accurate indication. For example, a single newspaper editorial may give us some indication of the political orientations of that newspaper. Examining several editorials would probably give us a better assessment, but the ma- nipulation of several data items simultaneously could be very complicated. Indexes and scales (es- pecially scales) are efficient data-reduction devices: They allow us to summarize several indicators in a single numerical score, while sometimes nearly maintaining the specific details of all the individual indicators.
Indexes versus Scales The terms index and scale are typically used im- precisely and interchangeably in social research literature. The two types of measures do have some
50094_ch07.indd 198 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Indexes versus Scales ■ 199
characteristics in common, but in this book we’ll distinguish between the two. However, you should be warned of a growing tendency in the literature to use the term scale to refer to both indexes and scales, as they are distinguished here.
First, let’s consider what they have in com- mon. Both scales and indexes are ordinal measures of variables. Both rank-order the units of analysis in terms of specific variables such as religiosity, alienation, socioeconomic status, prejudice, or intellectual sophistication. A person’s score on ei- ther a scale or an index of religiosity, for example, gives an indication of his or her relative religiosity vis-à-vis other people.
Further, both scales and indexes are compos- ite measures of variables—that is, measurements based on more than one data item. Thus, a survey respondent’s score on an index or scale of religios- ity is determined by the responses given to several questionnaire items, each of which provides some indication of religiosity. Similarly, a person’s IQ score is based on answers to a large number of test questions. The political orientation of a newspaper might be represented by an index or scale score reflecting the newspaper’s editorial policy on vari- ous political issues.
Despite these shared characteristics, it’s useful to distinguish between indexes and scales. In this book, we’ll distinguish them by the way scores are assigned in each. We construct an index sim- ply by accumulating scores assigned to individual attributes. We might measure prejudice, for exam- ple, by adding up the number of prejudiced state- ments each respondent agreed with. We construct a scale, however, by assigning scores to patterns of responses, recognizing that some items reflect a relatively weak degree of the variable while others reflect something stronger. For example, agreeing that “Women are different from men” is, at best, weak evidence of sexism compared with agree- ing that “Women should not be allowed to vote.” A scale takes advantage of differences in intensity among the attributes of the same variable to iden- tify distinct patterns of response.
Let’s consider this simple example of sexism a bit further. Imagine asking people to agree or disagree with the two statements just presented.
Some might agree with both, some might disagree with both. But suppose I told you someone agreed with one and disagreed with the other: Could you guess which statement they agreed with and which they did not? I’d guess the person in question agreed that women were different but disagreed that they should be prohibited from voting. On the other hand, I doubt that anyone would want to prohibit women from voting, while asserting that there is no difference between men and women. That would make no sense.
Now consider this. The two responses we wanted from each person would technically yield four response patterns: agree/agree, agree/disagree, disagree/agree, and disagree/disagree. We’ve just seen, however, that only three of the four patterns make any sense or are likely to occur. Where in- dexes score people based on their responses, scales score people on the basis of response patterns: We determine what the logical response patterns are and score people in terms of the pattern their re- sponses most closely resemble.
Figure 7-1 provides a graphic illustration of the difference between indexes and scales. Let’s assume we want to develop a measure of political activism, distinguishing those people who are very active in political affairs, those who don’t participate much at all, and those who are somewhere in between.
The first part of Figure 7-1 illustrates the logic of indexes. The figure shows six different politi- cal actions. Although you and I might disagree on some specifics, I think we could agree that the six actions represent roughly the same degree of politi- cal activism.
Using these six items, we could construct an index of political activism by giving each person 1 point for each of the actions he or she has taken.
index A type of composite measure that summa- rizes and rank-orders several specific observations and represents some more-general dimension.
scale A type of composite measure composed of several items that have a logical or empirical structure among them. Examples of scales include Bogardus social distance, Guttman, Likert, and Thurstone scales.
50094_ch07.indd 199 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
200 ■ Chapter 7: Typologies, Indexes, and Scales
If you wrote to a public official and signed a peti- tion, you’d get a total of 2 points. If I gave money to a candidate and persuaded someone to change her or his vote, I’d get the same score as you. Using this approach, we’d conclude that you and I had the same degree of political activism, even though we had taken different actions.
The second part of Figure 7-1 describes the logic of scale construction. In this case, the actions clearly represent different degrees of political ac- tivism, ranging from simply voting to running for office. Moreover, it seems safe to assume a pattern of actions in this case. For example, all those who contributed money probably also voted. Those who
worked on a campaign probably also gave some money and voted. This suggests that most people will fall into only one of five idealized action pat- terns, represented by the illustrations at the bottom of the figure. The discussion of scales, later in this chapter, describes ways of identifying people with the type they most closely represent.
As you might surmise, scales are generally superior to indexes, because scales take into con- sideration the intensity with which different items reflect the variable being measured. Also, as the example in Figure 7-1 shows, scale scores convey more information than index scores do. Again, be aware that the term scale is commonly misused to
Fig. 6-11-133-04979-6
Babbie: The Practice of Social Research, 13/e
C e n g a g e L e a r n i n g
F I G U R E 7 1 Indexes versus Scales. Both indexes and scales seek to measure variables such as political activism. Whereas indexes count the number of indica- tors of the variable, scales take account of the differing intensities of those indicators.
50094_ch07.indd 200 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Index Construction ■ 201
refer to measures that are only indexes. Merely calling a measure a scale instead of an index doesn’t make it better.
There are two other misconceptions about scal- ing that you should know about. First, whether the combination of several data items results in a scale almost always depends on the particular sample of observations under study. Certain items may form a scale within one sample but not within another. For this reason, do not assume that a given set of items is a scale simply because it has turned out that way in an earlier study.
Second, the use of specific scaling techniques— such as Guttman scaling, to be discussed—does not ensure the creation of a scale. Rather, such techniques let us determine whether or not a set of items constitutes a scale.
An examination of actual social science re- search reports will show that researchers use in- dexes much more frequently than they do scales. Ironically, however, the methodological literature contains little if any discussion of index construc- tion, whereas discussions of scale construction abound. There appear to be two reasons for this disparity. First, indexes are more frequently used because scales are often difficult or impossible to construct from the data at hand. Second, methods of index construction seem so obvious and straight- forward that they aren’t discussed much.
Constructing indexes is not a simple undertak- ing, however. The general failure to develop index- construction techniques has resulted in many bad indexes in social research. With this in mind, I’ve devoted over half of this chapter to the methods of index construction. With a solid understanding of the logic of this activity, you’ll be better equipped to try constructing both indexes and scales.
Index Construction Let’s look now at four main steps in the construc- tion of an index: selecting possible items, examin- ing their empirical relationships, scoring the index, and validating it. We’ll conclude this discussion by examining the construction of an index that provided interesting findings about the status of women in different countries.
Item Selection The first step in creating an index is selecting items for a composite index, which is created to measure some variable.
Face Validity The first criterion for selecting items to be included in an index is face validity (or logical validity). If you want to measure political conservatism, for ex- ample, each of your items should appear on its face to indicate conservatism (or its opposite, liberalism). Political party affiliation would be one such item. Another would be an item asking people to ap- prove or disapprove of the views of a well-known conservative public figure. In constructing an index of religiosity, you might consider items such as at- tendance at religious services, acceptance of certain religious beliefs, and frequency of prayer; each of these appears to offer some indication of religiosity.
Unidimensionality The methodological literature on conceptualization and measurement stresses the need for unidimen- sionality in scale and index construction. That is, a composite measure should represent only one dimension of a concept. Thus, items reflecting reli- gious fundamentalism should not be included in a measure of political conservatism, even though the two variables might be empirically related to each other.
General or Specific Although measures should tap the same dimen- sion, the general dimension you’re attempting to measure may have many nuances. In the example of religiosity, the indicators mentioned previously— ritual participation, belief, and so on—represent different types of religiosity. If you want to focus on ritual participation in religion, you should choose items specifically indicating this type of religiosity: attendance at religious services and other rituals such as confession, bar mitzvah, bowing toward Mecca, and the like. If you want to measure reli- giosity in a more general way, you should include a balanced set of items, representing each of the
50094_ch07.indd 201 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
202 ■ Chapter 7: Typologies, Indexes, and Scales
different types of religiosity. Ultimately, the na- ture of the items you include will determine how specifically or generally the variable is measured.
Variance In selecting items for an index, you must also be concerned with the amount of variance they provide. If an item is intended to indicate political conservatism, for example, you should note what proportion of respondents would be identified as conservatives by that item. If a given item identi- fied no one as a conservative or everyone as a conservative—for example, if nobody indicated approval of a radical-right political figure—that item would not be very useful in the construction of an index.
To guarantee variance, you have two options. First, you may select several items the responses to which divide people about equally in terms of the variable, for example, about half conservative and half liberal. Although no single response would jus- tify the characterization of a person as very conser- vative, a person who responded as a conservative on all items might be so characterized.
The second option is to select items differing in variance. One item might identify about half of the subjects as conservative, while another might iden- tify few of the respondents as conservative. Note that this second option is necessary for scaling, and it is reasonable for index construction as well.
Examination of Empirical Relationships The second step in index construction is to exam- ine the empirical relationships among the items being considered for inclusion. (See Chapter 14 for more.) An empirical relationship is established when respondents’ answers to one question—in a questionnaire, for example—help us predict how they’ll answer other questions. If two items are empirically related to each other, we can reason- ably argue that each reflects the same variable, and we may include them both in the same index. There are two types of possible relationships among items: bivariate and multivariate.
Bivariate Relationships A bivariate relationship is, simply put, a relationship between two variables. Suppose we want to mea- sure respondents’ support for U.S. participation in the United Nations. One indicator of different levels of support might be the question “Do you feel the U.S. financial support of the UN is Too high
About right Too low?” A second indicator of support for the
United Nations might be the question “Should the United States contribute military personnel to UN peacekeeping actions? Strongly approve
Mostly approve Mostly disapprove Strongly disapprove.”
Both of these questions, on their face, seem to reflect different degrees of support for the United Nations. Nonetheless, some people might feel the United States should give more money but not provide troops. Others might favor sending troops but cutting back on financial support.
If the two items both reflect degrees of the same thing, however, we should expect re- sponses to the two items to correspond with each other. Specifically, those who approve of military support should be more likely to favor financial support than those who disapprove of military support would. Conversely, those who favor financial support should be more likely to favor military support than those disapproving of financial support would. If these expectations are met, we say there is a bivariate relationship between the two items.
Here’s another example. Suppose we want to determine the degree to which respondents feel women have the right to an abortion. We might ask (1) “Do you feel a woman should have the right to an abortion when her pregnancy was the result of rape?” and (2) “Do you feel a woman should have the right to an abortion if continu- ing her pregnancy would seriously threaten her life?”
Granted, some respondents might agree with item (1) and disagree with item (2); others will do just the reverse. However, if both items tap into some general opinion people have about the issue of abortion, then the responses to these two items
50094_ch07.indd 202 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Index Construction ■ 203
should be related to each other. Those who support the right to an abortion in the case of rape should be more likely to support it if the woman’s life is threatened than those who disapproved of abortion in the case of rape would. This would be another example of a bivariate relationship.
You should examine all the possible bivari- ate relationships among the several items being considered for inclusion in an index, in order to determine the relative strengths of relationships among the several pairs of items. Percentage tables, correlation coefficients (see Chapter 16), or both may be used for this purpose. How we evaluate the strength of the relationships, however, can be rather subtle. The Tips and Tools feature “‘Cause’ and ‘Effect’ Indicators” examines some of these subtleties.
Be wary of items that are not related to one another empirically: It’s unlikely that they measure the same variable. You should probably drop any item that is not related to several other items.
At the same time, a very strong relation- ship between two items presents a different problem. If two items are perfectly related to each other, then only one needs to be included in the index; because it completely conveys the indications provided by the other, nothing more would be added by including the other item. (This problem will become even clearer in the next section.)
Here’s an example to illustrate the testing of bivariate relationships in index construction. I once conducted a survey of medical school faculty members to find out about the consequences of a “scientific perspective” on the quality of patient care provided by physicians. The primary intent was to determine whether scientifically inclined doctors treated patients more impersonally than other doctors did.
The survey questionnaire offered several pos- sible indicators of respondents’ scientific perspec- tives. Of those, three items appeared to provide
Tips and Tools
“Cause” and “Effect” Indicators
Kenneth Bollen Department of Sociology, University of North Carolina, Chapel Hill
While it often makes sense to expect indicators of the same variable to be positively related to one another, as discussed in the text, this is not always the case.
Indicators should be related to one another if they are essentially “effects” of a variable. For example, to measure self-esteem, we might ask a person to indicate whether he or she agrees or disagrees with the statements (1) “I am a good person” and (2) “I am happy with myself.” A person with high self-esteem should agree with both statements while one with low self-esteem would probably disagree with both. Since each indicator depends on or “reflects” self-esteem, we expect them to be positively correlated. More generally, indicators that depend on the same variable should be associated with one another if they are valid measures.
But, this is not the case when the indicators are the “cause” rather than the “effect” of a variable. In this situation the indicators may cor- relate positively, negatively, or not at all. For example, we could use sex and race as indicators of the variable exposure to discrimination. Being
nonwhite or female increases the likelihood of experiencing discrimina- tion, so both are good indicators of the variable. But we would not expect the race and sex of individuals to be strongly associated.
Or, we may measure social interaction with three indicators: time spent with friends, time spent with family, and time spent with coworkers. Though each indicator is valid, they need not be positively correlated. Time spent with friends, for instance, may be inversely related to time spent with family. Here, the three indicators “cause” the degree of social interaction.
As a final example, exposure to stress may be measured by whether a person recently experienced divorce, death of a spouse, or loss of a job. Though any of these events may indicate stress, they need not correlate with one another.
In short, we expect an association between indicators that depend on or “reflect” a variable, that is, if they are the “effects” of the variable. But if the variable depends on the indicators—if the indicators are the “causes”—those indicators may be either positively or negatively corre- lated, or even unrelated. Therefore, we should decide whether indicators are causes or effects of a variable before using their intercorrelations to assess their validity.
50094_ch07.indd 203 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
204 ■ Chapter 7: Typologies, Indexes, and Scales
especially clear indicati0ons of whether the doctors were scientifically oriented:
1. As a medical school faculty member, in what capacity do you feel you can make your greatest teaching contribution: as a prac- ticing physician or as a medical researcher?
2. As you continue to advance your own medical knowledge, would you say your ultimate medical interests lie primarily in the direction of total patient management or the understanding of basic mechanisms? [The purpose of this item was to distinguish those who were mostly interested in over- all patient care from those mostly inter- ested in biological processes.]
3. In the field of therapeutic research, are you generally more interested in articles reporting evaluations of the effectiveness of various treatments or articles exploring the basic rationale underlying the treatments? [Similarly, I wanted to distinguish those more interested in articles dealing with patient care from those more interested in biological processes.]
(Babbie 1970: 27–31)
For each of these items, we might conclude that those respondents who chose the second answer are more scientifically oriented than respondents who chose the first answer. Though this comparative con- clusion is reasonable, we should not be misled into thinking that respondents who chose the second answer to a given item are scientists in any absolute sense. They are simply more scientifically oriented than those who chose the first answer to the item.
To see this point more clearly, let’s examine the distribution of responses to each item. From the first item—greatest teaching contribution— only about one-third of the respondents appeared scientifically oriented. That is, approximately one- third said they could make their greatest teaching contribution as medical researchers. In response to the second item—ultimate medical interests— approximately two-thirds chose the scientific answer, saying they were more interested in learn- ing about basic mechanisms than learning about
total patient management. In response to the third item—reading preferences—about 80 percent chose the scientific answer.
These three questionnaire items can’t tell us how many “scientists” there are in the sample, for none of them is related to a set of criteria for what constitutes being a scientist in any absolute sense. Using the items for this purpose would present us with the problem of three quite different estimates of how many scientists there were in the sample.
However, these items do provide us with three independent indicators of respondents’ relative inclinations toward science in medicine. Each item separates respondents into the more scientific and the less scientific. But each grouping of more or less scientific respondents will have a somewhat different membership from the oth- ers. Respondents who seem scientific in terms of one item will not seem scientific in terms of an- other. Nevertheless, to the extent that each item measures the same general dimension, we should find some correspondence among the several groupings. Respondents who appear scientific in terms of one item should be more likely to appear scientific in their response to another item than would those who appeared nonscientific in their response to the first. In other words, we should find an association or correlation between the re- sponses given to two items.
Figure 7-2 shows the associations among the responses to the three items. Three bivariate tables are presented, showing the distribution of responses for each possible pairing of items. An examination of the three bivariate relationships presented in the figure supports the suggestion that the three items all measure the same variable: scientific orientation. To see why this is so, let’s begin by looking at the first bivariate relationship in the table. The table shows that faculty who responded that “researcher” was the role in which they could make their greatest teaching contribution were more likely to identify their ultimate medical inter- ests as “basic mechanisms” (87 percent) than were those who answered “physician” (51 percent). The fact that the “physicians” are about evenly split in their ultimate medical interests is irrelevant for our purposes. It is only relevant that they are less
50094_ch07.indd 204 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Index Construction ■ 205
scientific in their medical interests than the “re- searchers.” The strength of this relationship may be summarized as a 36 percentage point difference.
The same general conclusion applies to the other bivariate relationships. The strength of the relationship between reading preferences and ul- timate medical interests may be summarized as a 38 percentage point difference, and the strength of
the relationship between reading preferences and greatest teaching contribution as a 21 percentage point difference. In summary, then, each single item produces a different grouping of “scientific” and “nonscientific” respondents. However, the re- sponses given to each of the items correspond, to a greater or lesser degree, to the responses given to each of the other items.
Initially, the three items were selected on the basis of face validity—each appeared to give some indication of faculty members’ orientations to science. By examining the bivariate relationship between the pairs of items, we have found support for the expectation that they all measure basically the same thing. However, that support does not sufficiently justify including the items in a compos- ite index. Before combining them in a single index, we need to examine the multivariate relationships among the several variables.
Multivariate Relationships among Items Figure 7-3 categorizes the sample respondents into four groups according to (1) their greatest teaching contribution and (2) their reading preferences. The numbers in parentheses indicate the number of respondents in each group. Thus, 66 of the faculty members who said they could best teach as physi- cians also said they preferred articles dealing with the effectiveness of treatments. For each of the four groups, the figure presents the percentage of those who say they are ultimately more interested in basic mechanisms. So, for example, of the 66 fac- ulty mentioned, 27 percent are primarily interested in basic mechanisms.
The arrangement of the four groups is based on a previously drawn conclusion regarding scientific orientations. The group in the upper left corner of the table is presumably the least scientifically ori- ented, based on greatest teaching contribution and reading preferences. The group in the lower right corner is presumably the most scientifically ori- ented in terms of those items.
Recall that expressing a primary interest in basic mechanisms was also taken as an indication of scientific orientation. As we should expect, then, those in the lower right corner are the most likely to give this response (89 percent), and those in the
Fig. 6-21-133-04979-6
Babbie: The Practice of Social Research, 13/e
C e n g a g e L e a r n i n g
F I G U R E 7 2 Bivariate Relationships among Scientific Orientation Items. If several indicators are measures of the same variable, then they should be empirically correlated with one another, as you can observe in this case. Those who choose the scientific orientation on one item are more likely to choose the scientific orientation on other items.
50094_ch07.indd 205 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
206 ■ Chapter 7: Typologies, Indexes, and Scales
upper left corner are the least likely (27 percent). The respondents who gave mixed responses in terms of teaching contributions and reading prefer- ences have an intermediate rank in their concern for basic mechanisms (58 percent in both cases).
This table tells us many things. First, we may note that the original relationships between pairs of items are not significantly affected by the pres- ence of a third item. Recall, for example, that the relationship between teaching contribution and ultimate medical interest was summarized as a 36 percentage point difference. Looking at Figure 7-3, we see that among only those respon- dents who are most interested in articles dealing with the effectiveness of treatments, the relation- ship between teaching contribution and ultimate medical interest is 31 percentage points (58 per- cent minus 27 percent: first row). The same is true among those most interested in articles dealing with the rationale for treatments (89 percent minus 58 percent: second row). The original rela- tionship between teaching contribution and ulti- mate medical interest is essentially the same as in Figure 7-2, even among those respondents judged as scientific or nonscientific in terms of reading preferences.
We can draw the same conclusion from the columns in Figure 7-3. Recall that the original rela- tionship between reading preferences and ultimate medical interest was summarized as a 38 percentage
point difference. Looking only at the “physicians” in Figure 7-3, we see that the relationship between the other two items is now 31 percentage points. The same relationship is found among the “researchers” in the second column.
The importance of these observations becomes clearer when we consider what might have hap- pened. In Figure 7-4, hypothetical data tell a much different story than the actual data in Figure 7-3 do. As you can see, Figure 7-4 shows that the orig- inal relationship between teaching role and ulti- mate medical interest persists, even when reading preferences are introduced into the picture. In each row of the table, the “researchers” are more likely to express an interest in basic mechanisms than the “physicians” are. Looking down the columns, how- ever, we note that there is no relationship between reading preferences and ultimate medical interest. If we know whether a respondent feels he or she can best teach as a physician or as a researcher, knowing the respondent’s reading preference adds nothing to our evaluation of his or her scientific orientation. If something like Figure 7-4 resulted from the actual data, we would conclude that read- ing preference should not be included in the same index as teaching role, because it contributed noth- ing to the composite index.
This example used only three questionnaire items. If more were being considered, then more- complex multivariate tables would be in order, constructed of four, five, or more variables. The purpose of this step in index construction, again,
F I G U R E 7 3 Trivariate Relationships among Scientific Orientation Items. Indicators of the same variable should be correlated in a multivariate analysis as well as in bivariate analyses. Those who choose the scientific re- sponses on greatest teaching contribution and reading preferences are the most likely to choose the scientific response on the third item.
Fig. 6-31-133-04979-6
Babbie: The Practice of Social Research, 13/e
C e n g a g e L e a r n i n g
F I G U R E 7 4 Hypothetical Trivariate Relationship among Scientific Orientation Items. This hypothetical relationship suggests that not all three indica- tors would contribute effectively to a composite index.
Fig. 6-41-133-04979-6
Babbie: The Practice of Social Research, 13/e
C e n g a g e L e a r n i n g
50094_ch07.indd 206 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Index Construction ■ 207
is to discover the simultaneous interaction of the items in order to determine which should be in- cluded in the same index. These kinds of data anal- yses are easily accomplished using programs such as SPSS and MicroCase. They are usually referred to as cross-tabulations.
Index Scoring When you’ve chosen the best items for your index, you next assign scores for particular responses, thereby creating a single composite measure out of the several items. There are two basic decisions to be made in this step.
First, you must decide the desirable range of the index scores. A primary advantage of an index over a single item is the range of gradations it offers in the measurement of a variable. As noted earlier, political conservatism might be measured from “very conservative” to “not at all conservative” or “very liberal.” How far to the extremes, then, should the index extend?
In this decision, the question of variance enters once more. Almost always, as the possible ex- tremes of an index are extended, fewer cases are to be found at each end. The researcher who wishes to measure political conservatism to its greatest extreme (somewhere to the right of Attila the Hun, as the saying goes) may find there is almost no one in that category. At some point, additional grada- tions do not add meaning to the results.
The first decision, then, concerns the con- flicting desire for (1) a range of measurement in the index and (2) an adequate number of cases at each point in the index. You’ll be forced to reach some kind of compromise between these conflicting desires.
The second decision concerns the actual as- signment of scores for each particular response. Basically you must decide whether to give items in the index equal weight or different weights. Although there are no firm rules, I suggest—and practice tends to support this method—that items be weighted equally unless there are compelling reasons for differential weighting. That is, the bur- den of proof should be on differential weighting; equal weighting should be the norm.
Of course, this decision must be related to the earlier issue regarding the balance of items cho- sen. If the index is to represent the composite of slightly different aspects of a given variable, then you should give each aspect the same weight. In some instances, however, you may feel that two items reflect essentially the same aspect, and the third reflects a different aspect. If you want to have both aspects equally represented by the index, you might give the different item a weight equal to the combination of the two similar ones. For instance, you could assign a maximum score of 2 to the dif- ferent item and a maximum score of 1 to each of the similar ones.
Although the rationale for scoring responses should take such concerns as these into account, typically researchers experiment with different scoring methods, examining the relative weights given to different aspects but at the same time worrying about the range and distribution of cases provided. Ultimately, the scoring method chosen will represent a compromise among these several demands. Of course, as in most research activities, such a decision is open to revision on the basis of later examinations. Validation of the index, to be discussed shortly, may lead the researcher to re- cycle his or her efforts by constructing a completely different index.
In the example taken from the medical school faculty survey, I decided to weight the items equally, since I’d chosen them, in part, because they represent slightly different aspects of the over- all variable scientific orientation. On each of the items, the respondents were given a score of 1 for choos- ing the “scientific” response to the item and a score of 0 for choosing the “nonscientific” response. Each respondent, then, could receive a score of 0, 1, 2, or 3. This scoring method provided what I considered a useful range of variation—four index categories— and also provided enough cases for analysis in each category.
Here’s a similar example of index scoring, from a study of work satisfaction. One of the key vari- ables was job-related depression, measured by an index composed of the following four items, which asked workers how they felt when thinking about them- selves and their jobs:
50094_ch07.indd 207 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
208 ■ Chapter 7: Typologies, Indexes, and Scales
• “I feel downhearted and blue.” • “I get tired for no reason.” • “I find myself restless and can’t keep still.” • “I am more irritable than usual.”
The researchers, Amy Wharton and James Baron, report, “Each of these items was coded: 4 = often, 3 = sometimes, 2 = rarely, 1 = never.” They go on to explain how they measured another variable, job-related self-esteem:
Job-related self-esteem was based on four items asking respondents how they saw themselves in their work: happy/sad; successful/not suc- cessful; important/not important; doing their best/not doing their best. Each item ranged from 1 to 7, where 1 indicates a self-perception of not being happy, successful, important, or doing one’s best.
(1987: 578)
As you look through the social research lit- erature, you’ll find numerous similar examples of cumulative indexes being used to measure variables.
Although it is often appropriate to examine the relationships among indicators of a variable being measured by an index or scale, you should realize that the indicators are sometimes independent of one another. For example, Stacy De Coster notes that the indicators of family stress may be indepen- dent of one another, though they contribute to the same variable.
Family Stress is a scale of stressful events within the family. The experience of any one of these events—parent job loss, parent separation, par- ent illness—is independent of the other events. Indeed, prior research on events utilized in stress scales has demonstrated that the events in these scales typically are independent of one another and reliabilities on the scales low.
(2005: 176)
If the indicators of a variable are logically related to one another, on the other hand, it is important to use that relationship as a criterion for determining which are the better indicators.
Handling Missing Data Regardless of your data-collection method, you’ll frequently face the problem of missing data. In a content analysis of the political orientations of blogs, for example, you may discover that a par- ticular blog has never taken an editorial position on one of the issues being studied. In an experimental design involving several retests of subjects over time, some subjects may be unable to participate in some of the sessions. In virtually every survey, some respondents fail to answer some questions (or choose a “don’t know” response). Although missing data present problems at all stages of analy- sis, they’re especially troublesome in index con- struction. There are, however, several methods of dealing with these problems.
First, if there are relatively few cases with missing data, you may decide to exclude them from the construction of the index and the analy- sis. (I did this in the medical school faculty ex- ample.) The primary concerns in this instance are whether the numbers available for analysis will remain sufficient and whether the exclusion will result in an unrepresentative sample whenever the index, excluding some of the respondents, is used in the analysis. The latter possibility can be examined through a comparison—on other rel- evant variables—of those who would be included in and excluded from the index.
Second, you may sometimes have grounds for treating missing data as one of the available responses. For example, if a questionnaire has asked respondents to indicate their participation in various activities by checking “yes” or “no” for each, many respondents may have checked some of the activities “yes” and left the remainder blank. In such a case, you might decide that a failure to answer meant “no,” and score missing data in this case as though the respondents had checked the “no” space.
Third, a careful analysis of missing data may yield an interpretation of their meaning. In con- structing a measure of political conservatism, for example, you may discover that respondents who failed to answer a given question were generally as conservative on other items as those who gave
50094_ch07.indd 208 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Index Construction ■ 209
the conservative answer were. In another example, a recent study measuring religious beliefs found that people who answered “don’t know” about a given belief were almost identical to the “disbeliev- ers” in their answers about other beliefs. (Note: You should take these examples not as empirical guides in your own studies but only as suggestions of gen- eral ways to analyze your own data.) Whenever the analysis of missing data yields such interpreta- tions, then, you may decide to score such cases accordingly.
There are many other ways of handling the problem of missing data. If an item has several pos- sible values, you might assign the middle value to cases with missing data; for example, you could as- sign a 2 if the values are 0, 1, 2, 3, and 4. For a con- tinuous variable such as age, you could similarly assign the mean to cases with missing data (more on this in Chapter 14). Or, missing data can be sup- plied by assigning values at random. All of these are conservative solutions because they weaken the “purity” of your index and reduce the likeli- hood that it will relate to other variables in ways you may have hypothesized.
If you’re creating an index out of a large num- ber of items, you can sometimes handle missing data by using proportions based on what is ob- served. Suppose your index is composed of six indicators, and you only have four observations for a particular subject. If the subject has earned 4 points out of a possible 4, you might assign an index score of 6; if the subject has 2 points (half the possible score on four items), you could as- sign a score of 3 (half the possible score on six observations).
The choice of a particular method to be used depends so much on the research situation that I can’t reasonably suggest a single “best” method or rank the several I’ve described. Excluding all cases with missing data can bias the representativeness of the findings, but including such cases by assign- ing scores to missing data can influence the nature of the findings. The safest and best method is to construct the index using more than one of these methods and see whether you reach the same con- clusions using each of the indexes. Understanding your data is the final goal of analysis anyway.
The Research in Real Life feature, “How Healthy Is Your State,” illustrates one use of indexing that you might find interesting. In addition to the rank listing, be sure to examine the health measures in- cluded in the index.
Index Validation Up to this point, we’ve discussed all the steps in the selection and scoring of items that result in an index purporting to measure some variable. If each of the preceding steps is carried out carefully, the likelihood of the index actually measuring the variable is enhanced. To demonstrate success, however, we must show that the index is valid. Following the basic logic of validation, we assume that the index provides a measure of some variable; that is, the scores on the index arrange cases in a rank order in terms of that variable. An index of political conservatism rank-orders people in terms of their relative conservatism. If the index does that successfully, then people scored as relatively conservative on the index should appear relatively conservative in all other indications of political ori- entation, such as their responses to other question- naire items. There are several methods of validating an index.
Item Analysis The first step in index validation is an internal validation called item analysis. In item analysis, you examine the extent to which the index is related to (or predicts responses to) the individual items it comprises. Here’s an illustration of this step.
In the index of scientific orientations among medical school faculty, index scores ranged from 0 (most interested in patient care) to 3 (most inter- ested in research). Now let’s consider one of the items in the index: whether respondents wanted to advance their own knowledge more with regard
item analysis An assessment of whether each of the items included in a composite measure makes an independent contribution or merely duplicates the contribution of other items in the measure.
50094_ch07.indd 209 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
210 ■ Chapter 7: Typologies, Indexes, and Scales
Research in Real Life
How Healthy Is Your State?
Since 1990, United Health Foundation, the American Public Health Association, and Partnership for Prevention have collaborated on an annual evaluation of the health status of each of the 50 states. The fol- lowing table displays the findings for overall rankings from the 2010 report. The scores indicate where each state stands in comparison to the
nation as a whole. The scores are shown as standard deviations from the national average. While you may not have studied this statistical technique, you can still tell whether your state is above or below the national average. The healthiest state in 2010 was Vermont; Mississippi was the least healthy.
You may be interested in seeing how your state ranks.
2010 Overall Rankings
Rank Order
Rank State Score* Rank State Score*
1 Vermont 1.131 26 California 0.230
2 Massachusetts 0.906 27 Pennsylvania 0.046
3 New Hampshire 0.892 28 Alaska 0.033
4 Connecticut 0.873 29 Illinois 0.031
5 Hawaii 0.852 30 Michigan 0.024
6 Minnesota 0.844 31 Arizona 0.009
7 Utah 0.825 32 Delaware 20.032
8 Maine 0.627 33 New Mexico 20.056
9 Idaho 0.569 34 Ohio 20.070
10 Rhode Island 0.553 35 North Carolina 20.181
11 Nebraska 0.550 36 Georgia 20.207
11 Washington 0.550 37 Florida 20.210
13 Colorado 0.545 38 Indiana 20.322
14 Iowa 0.524 39 Missouri 20.325
15 Oregon 0.516 40 Texas 20.364
16 North Dakota 0.511 41 South Carolina 20.397
17 New Jersey 0.487 42 Tennessee 20.423
18 Wisconsin 0.468 43 West Virginia 20.449
19 Wyoming 0.419 44 Kentucky 20.456
20 South Dakota 0.324 45 Alabama 20.519
21 Maryland 0.274 46 Oklahoma 20.521
22 Virginia 0.266 47 Nevada 20.533
23 Kansas 0.258 48 Arkansas 20.605
24 New York 0.250 49 Louisiana 20.664
25 Montana 0.243 50 Mississippi 20.768
*Scores presented in this table indicate the weighted number of standard deviations a state is above or below the national norm.
50094_ch07.indd 210 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Index Construction ■ 211
Weight of Individual Measures
Name of Measure % of Total Effect on Score
DETERMINANTS
BEHAVIORS
Prevalence of Smoking 7.5 Negative
Prevalence of Binge Drinking 5.0 Negative
Prevalence of Obesity 7.5 Negative
High School Graduation 5.0 Positive
COMMUNITY AND ENVIRONMENT
Violent Crime 5.0 Negative
Occupational Fatalities 2.5 Negative
Infectious Disease 5.0 Negative
Children in Poverty 5.0 Negative
Air Pollution 5.0 Negative
PUBLIC AND HEALTH POLICIES
Lack of Health Insurance 5.0 Negative
Public Health Funding 2.5 Positive
Immunization Coverage 5.0 Positive
CLINICAL CARE
Early Prenatal Care 5.0 Positive
Primary Care Physicians 5.0 Positive
Preventable Hospitalizations 5.0 Negative
OUTCOMES
Poor Mental Health Days 2.5 Negative
Poor Physical Health Days 2.5 Negative
Geographic Disparity 5.0 Negative
Infant Mortality 5.0 Negative
Cardiovascular Deaths 2.5 Negative
Cancer Deaths 2.5 Negative
Premature Death 5.0 Negative
OVERALL HEALTH RANKING 100.0 —
Since you are, by now, a critical consumer of social research, I can hear you demanding, “Wait a minute, how did they measure healthy?” Good question. The table, “Weight of Individual Measures,” provides a summary of the components included in the report’s definition of what constitutes good or bad health. You’ll see that the
indicators encompass a number of categories. Some represent positive indications (e.g., high school graduation rates) and some are negative indicators (e.g., smoking and binge drinking). Moreover, the table shows the weight assigned to each indicator in the construction of a state’s overall score.
(Continued)
50094_ch07.indd 211 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
212 ■ Chapter 7: Typologies, Indexes, and Scales
Research in Real Life (Continued)
to total patient management or more in the area of basic mechanisms. The latter were treated as being more scientifically oriented than the former. The following empty table shows how we would ex- amine the relationship between the index and the individual item.
Index of Scientific Orientations
0 1 2 3
Percent who said they were more interested in basic mechanisms
??
??
??
??
If you take a minute to reflect on the table, you may see that we already know the numbers that go in two of the cells. To get a score of 3 on the index, respondents had to say “basic mechanisms” in response to this question and give the “scientific” answers to the other two items as well. Thus, 100 percent of the 3’s on the index said “basic mechanisms.” By the same token, all the 0’s had to answer this item with “total patient management.” Thus, 0 percent of those respondents said “basic mechanisms.” Here’s how the table looks with the information we already know.
Index of Scientific Orientations
0 1 2 3
Percent who said they were more interested in basic mechanisms
0
??
??
100
If the individual item is a good reflection of the overall index, we should expect the 1’s and 2’s to fill in a progression between 0 percent and 100 percent. More of the 2’s should choose “basic mechanisms” than 1’s. This result is not guaranteed
It would be a good idea for you to review each indicator and see if you agree that it reflects on how healthy states are. Perhaps you can think of other indicators that might have been used.
The full report provides a wealth of thoughtful discussion on why each of these indicators was chosen, and I’d encourage you to check it out at the URL shown below.
Source: United Health Foundation, Public Health Association, and Partnership for Prevention, “America’s Health Rankings: A Call to Action for Individuals and Their Communities.” ©2010 United Health Foundation. Table 1 taken from page 8, Table 36 from page 41. You may download a copy of the report at: (http://www .americashealthrankings.org/2010/AHR2010Edition-compact.pdf ).
by the way the index was constructed, however; it is an empirical question—one we answer in an item analysis. Here’s how this particular item analysis turned out.
Index of Scientific Orientations
0 1 2 3
Percent who said they were more interested in basic mechanisms
0
16
91
100
As you can see, in accord with our assumption that the 2’s are more scientifically oriented than the 1’s, we find that a higher percentage of the 2’s (91 percent) say “basic mechanisms” than the 1’s (16 percent).
An item analysis of the other two components of the index yields similar results, as shown here.
Index of Scientific Orientations
0 1 2 3
Percent who said they could teach best as medical researchers
0
4
14
100
Percent who said they preferred reading about rationales
0
80
97
100
Each of the items, then, seems an appro- priate component in the index. Each seems to reflect the same quality that the index as a whole measures.
In a complex index containing many items, this step provides a convenient test of the independent contribution of each item to the index. If a given item is found to be poorly related to the index, it may be assumed that other items in the index can- cel out the contribution of that item, and it should
50094_ch07.indd 212 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Index Construction ■ 213
is wrong with the index. But if the index fails to predict strongly the external validation items, the conclusion to be drawn is more ambiguous. In this situation we must choose between two possibili- ties: (1) the index does not adequately measure the variable in question, or (2) the validation items do not adequately measure the variable and thereby do not provide a sufficient test of the index.
Having worked long and conscientiously on the construction of an index, you’ll likely find the second conclusion compelling. Typically, you’ll feel you have included the best indicators of the variable in the index; the validating items are, therefore, second-rate indicators. Nevertheless, you should recognize that the index is purportedly a very powerful measure of the variable; thus, it should be somewhat related to any item that taps the variable, even if poorly.
When external validation fails, you should reexamine the index before deciding that the
external validation The process of testing the validity of a measure, such as an index or scale, by examining its relationship to other, presumed indi- cators of the same variable. If the index really mea- sures prejudice, for example, it should correlate with other indicators of prejudice.
be excluded from the index. If the item in question contributes nothing to the index’s power, it should be excluded.
Although item analysis is an important first test of an index’s validity, it is not a sufficient test. If the index adequately measures a given variable, it should successfully predict other indications of that variable. To test this, we must turn to items not included in the index.
External Validation In our example of the scientific orientation in- dex, several questions in the questionnaire of- fered the possibility of external validation. Table 7-1 presents some of these items, which provide several lessons regarding index valida- tion. First, we note that the index strongly pre- dicts the responses to the validating items in the sense that the rank order of scientific responses among the four groups is the same as the rank order provided by the index itself. That is, the percentages reflect greater scientific orientation as you read across the rows of the table. At the same time, each item gives a different description of scientific orientation overall. For example, the last validating item indicates that the great majority of all faculty were engaged in research during the preceding year. If this were the only indicator of scientific orientation, we would conclude that nearly all faculty were scientific. Nevertheless, those scored as more scientific on the index are more likely to have engaged in research than were those scored as relatively less scientific. The third validating item provides a different descriptive picture: Only a minority of the faculty overall say they would prefer duties limited exclusively to research. Nevertheless, the relative percentages giving this answer corre- spond to the scores assigned on the index.
Bad Index versus Bad Validators Nearly every index constructor at some time must face the apparent failure of external items to vali- date the index. If the internal item analysis shows inconsistent relationships between the items in- cluded in the index and the index itself, something
TA B L E 7 1 Validation of Scientific Orientation Index
Index of Scientific Orientation
Low 0 1 2
High 3
Percent interested in attend- ing scientific lectures at the medical school
34
42
46
65
Percent who say faculty mem- bers should have experience as medical researchers
43
60
65
89
Percent who would prefer fac- ulty duties involving research activities only
0
8
32
66
Percent who engaged in research during the preceding academic year
61
76
94
99
50094_ch07.indd 213 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
214 ■ Chapter 7: Typologies, Indexes, and Scales
validating items are insufficient. One way to do this is to examine the relationships between the validating items and the individual items included in the index. If you discover that some of the index items relate to the validators and others do not, you’ll have improved your understanding of the index as it was initially constituted.
There’s no cookbook solution to this problem; it is an agony serious researchers must learn to survive. Ultimately, the wisdom of your decision to accept an index will be determined by the useful- ness of that index in your later analyses. Perhaps you’ll initially decide that the index is a good one and that the validators are defective, but you’ll later find that the variable in question (as measured by the index) is not related to other variables in the ways you expected. You may then have to com- pose a new index.
The Status of Women: An Illustration of Index Construction For the most part, our discussion of index construc- tion has focused on the specific context of survey research, but other types of research also lend themselves to this kind of composite measure. For example, when the United Nations (1995) set out to examine the status of women in the world, they chose to create two indexes, reflecting two different dimensions.
The Gender-related Development Index (GDI) compared women to men in terms of three indica- tors: life expectancy, education, and income. These indicators are commonly used in monitoring the status of women in the world. The Scandinavian countries of Norway, Sweden, Finland, and Den- mark ranked highest on this measure.
The second index, the Gender Empowerment Measure (GEM), aimed more at power issues and comprised three different indicators:
• The proportion of parliamentary seats held by women
• The proportion of administrative, managerial, professional, and technical positions held by women
• A measure of access to jobs and wages
Once again, the Scandinavian countries ranked high but were joined by Canada, New Zealand, the Netherlands, the United States, and Austria. Having two different measures of gender equality rather than one allowed the researchers to make more-sophisticated distinctions. For ex- ample, in several countries, most notably Greece, France, and Japan, women fared relatively well on the GDI but quite poorly on the GEM. Thus, while women were doing fairly well in terms of income, education, and life expectancy, they were still denied access to power. And whereas the GDI scores were higher in the wealthier na- tions than in the poorer ones, GEM scores showed that women’s empowerment depended less on national wealth, with many poor, developing countries outpacing some rich, industrial ones in regard to such empowerment.
By examining several different dimensions of the variables involved in their study, the UN researchers also uncovered an aspect of women’s earnings that generally goes unnoticed. Population Communications International (1996: 1) summa- rizes the finding nicely:
Every year, women make an invisible con- tribution of eleven trillion U.S. dollars to the global economy, the UNDP [United Nations Development Programme] report says, count- ing both unpaid work and the underpayment of women’s work at prevailing market prices. This “underevaluation” of women’s work not only undermines their purchasing power, says the 1995 HDR [Human Development Report], but also reduces their already low social status and affects their ability to own property and use credit. Mahbub ul Haq, the principal author of the report, says that “if women’s work were accurately reflected in national statistics, it would shatter the myth that men are the main breadwinners of the world.” The UNDP report finds that women work longer hours than men in almost every country, including both paid and unpaid duties.
“Research in Real Life: Indexing the World” provides some other examples of indexes that have been created to monitor the state of the world.
50094_ch07.indd 214 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Scale Construction ■ 215
As you can see, indexes can be constructed from many different kinds of data for a variety of purposes. Now we’ll turn our attention from the construction of indexes to an examination of scal- ing techniques.
Scale Construction Good indexes provide an ordinal ranking of cases on a given variable. All indexes are based on this kind of assumption: A senator who voted for seven con- servative bills is considered to be more conservative than one who voted for only four of them. What an index may fail to take into account, however, is that not all indicators of a variable are equally important or equally strong. The first senator might have voted in favor of seven mildly conservative bills, whereas the second senator might have voted in favor of four extremely conservative bills. (The second senator might have considered the other seven bills too lib- eral and voted against them.)
Scales offer more assurance of ordinality by tapping the intensity structures among the indi- cators. The several items going into a composite
measure may have different intensities in terms of the variable. Many methods of scaling are avail- able. We’ll look at four scaling procedures to illus- trate the variety of techniques available, along with a technique called the semantic differential. Although these examples focus on questionnaires, the logic of scaling, like that of indexing, applies to other research methods as well.
Bogardus Social Distance Scale Let’s suppose you’re interested in the extent to which U.S. citizens are willing to associate with, say, sex offenders. You might ask the following questions:
1. Are you willing to permit sex offenders to live in your country?
2. Are you willing to permit sex offenders to live in your community?
3. Are you willing to permit sex offenders to live in your neighborhood?
4. Would you be willing to let a sex offender live next door to you?
5. Would you let your child marry a sex offender?
Research in Real Life
Indexing the World
If you browse the web in search of indexes, you’ll be handsomely re- warded. Here are just a few examples of the ways in which people have used the logic of social indexes to monitor the state of the world. Go to your Sociology CourseMate at www.cengagebrain.com for links to each of the following examples:
• The well-being of nations is commonly measured in economic terms, such as the Gross Domestic Product per capita, average in- come, or stock market averages. In 1972, however, the mountainous kingdom of Bhutan drew global attention by proposing an index of “Gross National Happiness,” augmenting economic factors with measures of physical and mental health, freedom, environment, marital stability, and other indicators of noneconomic well-being. The World Data Base of Happiness expands this general idea to 24 countries.
• Columbia University’s Environmental Sustainability Index is one of several measures that seek to monitor the environmental impact of the nations of the planet.
• The well-being of America’s young people is the focus of the Child and Youth Well-Being Index, housed at Duke University.
• Money Magazine has indexed the 100 best places to live in America, using factors such as economics, housing, schools, health, crime, weather, and public facilities.
• The Heritage Foundation offers the Index of Economic Freedom for those planning business ventures around the world.
• For Christians who believe in prophecies of the end of times, the Rapture Index uses 45 indicators—including inflation, famine, floods, liberalism, and Satanism—and offers a gauge of how close or far away the end is.
Can you find other, similar indexes online?
50094_ch07.indd 215 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
216 ■ Chapter 7: Typologies, Indexes, and Scales
These questions increase in terms of the close- ness of contact with sex offenders. Beginning with the original concern to measure willingness to associate with sex offenders, you have thus devel- oped several questions indicating differing degrees of intensity on this variable. The kinds of items presented constitute a Bogardus social distance scale (created by Emory Bogardus). This scale is a measurement technique for determining the will- ingness of people to participate in social relations— of varying degrees of closeness—with other kinds of people.
The clear differences of intensity suggest a structure among the items. Presumably if a person is willing to accept a given kind of association, he or she would be willing to accept all those preced- ing it in the list—those with lesser intensities. For example, the person who is willing to permit sex offenders to live in the neighborhood will surely accept them in the community and the nation but may or may not be willing to accept them as next- door neighbors or relatives. This, then, is the logical structure of intensity inherent among the items.
Empirically, one would expect to find the larg- est number of people accepting co-citizenship and the fewest accepting intermarriage. In this sense, we speak of “easy items” (for example, residence in the United States) and “hard items” (for example, intermarriage). More people agree to the easy items than to the hard ones. With some inevitable excep- tions, logic demands that once a person has refused a relationship presented in the scale, he or she will also refuse all the harder ones that follow it.
The Bogardus social distance scale illustrates the important economy of scaling as a data-reduction device. By knowing how many relationships with
sex offenders a given respondent will accept, we know which relationships were accepted. Thus, a single number can accurately summarize five or six data items without a loss of information.
Motoko Lee, Stephen Sapp, and Melvin Ray (1996) noticed an implicit element in the Bogardus social distance scale: It looks at social distance from the point of view of the majority group in a society. These researchers decided to turn the tables and create a “reverse social dis- tance” scale: looking at social distance from the perspective of the minority group. Here’s how they framed their questions (1996: 19):
Considering typical Caucasian Americans you have known, not any specific person nor the worst or the best, circle Y or N to express your opinion. Y N 5. Do they mind your being a citizen in
this country? Y N 4. Do they mind your living in the same
neighborhood? Y N 3. Would they mind your living next to
them? Y N 2. Would they mind your becoming a
close friend to them? Y N 1. Would they mind your becoming their
kin by marriage?
As with the original scale, the researchers found that knowing the number of items minority respondents agreed with also told the researchers which ones were agreed with, 98.9 percent of the time in this case.
Thurstone Scales Often, the inherent structure of the Bogardus social distance scale is not appropriate to the variable being measured. Indeed, such a logical structure among several indicators is seldom ap- parent. A Thurstone scale (created by Louis Thurstone) is an attempt to develop a format for generating groups of indicators of a variable that have at least an empirical structure among them. A group of judges is given perhaps a hundred items that are thought to be indicators of a given variable. Each judge is then asked to estimate how strong an indicator of a variable each item
Bogardus social distance scale A measurement technique for determining the willingness of people to participate in social relations—of varying degrees of closeness—with other kinds of people. It is an especially efficient technique in that one can sum- marize several discrete answers without losing any of the original details of the data.
Thurstone scale A type of composite measure, constructed in accord with the weights assigned by “judges” to various indicators of some variables.
50094_ch07.indd 216 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Scale Construction ■ 217
is—by assigning scores of perhaps 1 to 13. If the variable were prejudice, for example, the judges would be asked to assign the score of 1 to the very weakest indicators of prejudice, the score of 13 to the strongest indicators, and intermediate scores to those felt to be somewhere in between.
Once the judges have completed this task, the researcher examines the scores assigned to each item by all the judges, then determines which items produced the greatest agreement among the judges. Those items on which the judges disagreed broadly would be rejected as ambiguous. Among those items producing general agreement in scor- ing, one or more would be selected to represent each scale score from 1 to 13.
The items selected in this manner might then be included in a survey questionnaire. Respondents who appeared prejudiced on those items repre- senting a strength of 5 would then be expected to appear prejudiced on those having lesser strengths, and if some of those respondents did not appear prejudiced on the items with a strength of 6, it would be expected that they would also not appear prejudiced on those with greater strengths.
If the Thurstone scale items were adequately developed and scored, the economy and effective- ness of data reduction inherent in the Bogardus social distance scale would appear. A single score might be assigned to each respondent (the strength of the hardest item accepted), and that score would adequately represent the responses to several ques- tionnaire items. And as is true of the Bogardus scale, a respondent who scored 6 might be regarded as more prejudiced than one who scored 5 or less.
Thurstone scaling is not often used in research today, primarily because of the tremendous expen- diture of energy and time required to have 10 to 15 judges score the items. Because the quality of their judgments would depend on their experience with the variable under consideration, they might need to be professional researchers. Moreover, the meanings conveyed by the several items indicating a given variable tend to change over time. Thus, an item having a given weight at one time might have quite a different weight later on. For a Thurstone scale to be effective, it would have to be updated periodically.
Likert Scaling I’m sure you are familiar with questionnaire items containing response categories such as “strongly agree,” “agree,” “disagree,” and “strongly disagree Rensis Likert (pronounced “LICK-ert”) created this commonly used question format. Likert also cre- ated a technique for combining the items into a scale, but while Likert’s scaling technique is rarely used, his answer format is one of the most fre- quently used in survey research.
The particular value of this format is the unam- biguous ordinality of response categories. If respon- dents were permitted to volunteer or select such answers as “sort of agree,” “pretty much agree,” “really agree,” and so forth, you would find it im- possible to judge the relative strength of agreement intended by the various respondents. The Likert format solves this problem.
Though seldom used, Likert’s scaling method is fairly easy to understand, based on the relative intensity of different items. As a simple example, suppose we wish to measure prejudice against women. To do this, we create a set of 20 state- ments, each of which reflects that prejudice. One of the items might be “Women can’t drive as well as men.” Another might be “Women shouldn’t be allowed to vote.” Likert’s scaling technique would demonstrate the difference in intensity between these items as well as pegging the intensity of the other 18 statements.
Let’s suppose we ask a sample of people to agree or disagree with each of the 20 statements. Simply giving one point for each of the indica- tors of prejudice against women would yield the possibility of index scores ranging from 0 to 20. A true Likert scale goes one step beyond that
Likert scale A type of composite measure devel- oped by Rensis Likert, in an attempt to improve the levels of measurement in social research through the use of standardized response categories in sur- vey questionnaires, to determine the relative inten- sity of different items. Likert items are those using such response categories as strongly agree, agree, disagree, and strongly disagree. Such items may be used in the construction of true Likert scales as well as other types of composite measures.
50094_ch07.indd 217 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
218 ■ Chapter 7: Typologies, Indexes, and Scales
and calculates the average index score for those agreeing with each of the individual statements. Let’s say that all those who agreed that women are poorer drivers than men had an average index score of 1.5 (out of a possible 20). Those who agreed that women should be denied the right to vote might have an average index score of, say, 19.5—indicating the greater degree of prejudice reflected in that response.
As a result of this item analysis, respondents could be rescored to form a scale: 1.5 points for agreeing that women are poorer drivers, 19.5 points for saying women shouldn’t vote, and points for other responses reflecting how those items related to the initial, simple index. If those who disagreed with the statement “I might vote for a woman for president” had an average index score of 15, then the scale would give 15 points to people disagreeing with that statement.
As I’ve said earlier, Likert scaling is seldom used today. The item format devised by Likert, however, is one of the most commonly used formats in contemporary questionnaire design. Typically, it is now used in the creation of simple indexes. With, say, five response categories (in- cluding “no opinion” or something similar), scores of 0 to 4 or 1 to 5 might be assigned, taking the direction of the items into account (for example, assign a score of 5 to “strongly agree” for posi- tive items and to “strongly disagree” for negative items). Each respondent would then be assigned an overall score representing the summation of the scores he or she received for responses to the individual items.
Semantic Differential Like the Likert format, the semantic differential asks questionnaire respondents to choose between
two opposite positions by using qualifiers to bridge the distance between the two opposites. Here’s how it works.
Suppose you’re evaluating the effectiveness of a new music-appreciation lecture on subjects’ ap- preciation of music. As a part of your study, you want to play some musical selections and have the subjects report their feelings about them. A good way to tap those feelings would be to use a seman- tic differential format.
To begin, you must determine the dimen- sions along which subjects should judge each selection. Then you need to find two opposite terms, representing the polar extremes along each dimension. Let’s suppose one dimension that interests you is simply whether subjects en- joyed the piece or not. Two opposite terms in this case could be “enjoyable” and “unenjoyable.” Similarly, you might want to know whether they regarded the individual selections as “complex” or “simple,” “harmonic” or “discordant,” and so forth.
Once you have determined the relevant dimensions and have found terms to represent the extremes of each, you might prepare a rat- ing sheet each subject would complete for each piece of music. Figure 7-5 shows what it might look like.
On each line of the rating sheet, the subject would indicate how he or she felt about the piece of music: whether it was enjoyable or unenjoyable, for example, and whether it was “somewhat” that way or “very much” so. To avoid creating a biased pattern of responses to such items, it’s a good idea to vary the placement of terms that are likely to be related to each other. Notice, for example, that “discordant” and “traditional” are on the left side of the sheet, with “harmonic” and “modern” on the right. Most likely, those selections scored as “dis- cordant” would also be scored as “modern” rather than “traditional.”
Both the Likert and semantic differential for- mats have a greater rigor and structure than other question formats do. As I indicated earlier, these formats produce data suitable to both indexing and scaling.
semantic differential A questionnaire format in which the respondent is asked to rate something in terms of two, opposite adjectives (e.g., rate text- books as “boring” or “exciting”), using qualifiers such as “very,” “somewhat,” “neither,” “somewhat,” and “very” to bridge the distance between the two opposites.
50094_ch07.indd 218 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Scale Construction ■ 219
Guttman Scaling Researchers today often use the scale developed by Louis Guttman. Like Bogardus, Thurstone, and Likert scaling, Guttman scaling is based on the fact that some items under consideration may prove to be more-extreme indicators of the variable than others. Here’s an example to illustrate this pattern.
In the earlier example of measuring scientific orientation among medical school faculty members, you’ll recall that a simple index was constructed. As it happens, however, the three items included in the index essentially form a Guttman scale.
The construction of a Guttman scale begins with some of the same steps that initiate index con- struction. You begin by examining the face validity of items available for analysis. Then, you examine the bivariate and perhaps multivariate relations among those items. In scale construction, however, you also look for relatively “hard” and “easy” indi- cators of the variable being examined.
Earlier, when we talked about attitudes regard- ing a woman’s right to have an abortion, we dis- cussed several conditions that can affect people’s opinions: whether the woman is married, whether her health is endangered, and so forth. These dif- fering conditions provide an excellent illustration of Guttman scaling.
Here are the percentages of the people in the 2006 GSS sample who supported a woman’s right to an abortion, under three different conditions:
Woman’s health is seriously endangered 87%
Pregnant as a result of rape 77%
Woman is not married 38%
The different percentages supporting abortion under the three conditions suggest something about the different levels of support that each item indicates. For example, if some- one supported abortion when the mother’s life is seriously endangered, that’s not a very strong indicator of general support for abortion, because almost everyone agreed with that. Supporting abortion for unmarried women seems a much stronger indicator of support for abortion in general—fewer than half the sample took that position.
Guttman scaling is based on the idea that any- one who gives a strong indicator of some variable will also give the weaker indicators. In this case, we would assume that anyone who supported abor- tion for unmarried women would also support it in the case of rape or of the woman’s health being threatened. Table 7-2 tests this assumption by pre- senting the number of respondents who gave each of the possible response patterns.
The first four response patterns in the table compose what we would call the scale types: those patterns that form a scalar structure. Following those respondents who supported abortion under all three conditions (line 1), we see (line 2) that those with only two pro-choice responses have chosen the two easier ones; those with only one such response (line 3) chose the easiest of the three (the woman’s health
Fig. 6-51-133-04979-6
Babbie: The Practice of Social Research, 13/e
C e n g a g e L e a r n i n g
F I G U R E 7 5 Semantic Differential: Feelings about Musical Selections. The semantic differential asks respondents to describe something or someone in terms of opposing adjectives.
Guttman scale A type of composite measure used to summarize several discrete observations and to represent some more-general variable.
50094_ch07.indd 219 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
220 ■ Chapter 7: Typologies, Indexes, and Scales
being endangered). And finally, there are some respondents who opposed abortion in all three circumstances (line 4).
The second part of the table presents those response patterns that violate the scalar structure of the items. The most radical departures from the scalar structure are the last two response patterns: those who accepted only the hardest item and those who rejected only the easiest one.
The final column in the table indicates the number of survey respondents who gave each of the response patterns. The great majority (1,788, or 97 percent) fit into one of the scale types. The presence of mixed types, however, indicates that the items do not form a perfect Guttman scale. (It would be extremely rare for such data to form a Guttman scale perfectly.)
Recall at this point that one of the chief func- tions of scaling is efficient data reduction. Scales provide a technique for presenting data in a sum- mary form while maintaining as much of the origi- nal information as possible. When the scientific orientation items were formed into an index in our earlier discussion, respondents were given one point for each scientific response they gave. If these same three items were scored as a Guttman scale, some respondents would be assigned scale scores that would permit the most accurate reproduction of their original responses to all three items.
In the present example of attitudes regarding abortion, respondents fitting into the scale types would receive the same scores as would be as- signed in the construction of an index. Persons se- lecting all three pro-choice responses (+ + +) would still be scored 3, those who selected pro-choice re- sponses to the two easier items and were opposed on the hardest item (+ + −) would be scored 2, and so on. For each of the four scale types we could predict accurately all the actual responses given by all the respondents based on their scores.
The mixed types in the table present a problem, however. The first mixed type (− + −) was scored 1 on the index to indicate only one pro-choice response. But, if 1 were assigned as a scale score, we would predict that the 43 respondents in this group had chosen only the easiest item (approving abortion when the woman’s life was endangered), and we would be making two errors for each such respondent: thinking their response pattern was (+ − −) instead of (− + −). Scale scores are assigned, therefore, with the aim of minimizing the errors that would be made in reconstructing the original responses.
Table 7-3 illustrates the index and scale scores that would be assigned to each of the response patterns in our example. Note that one error is made for each respondent in the mixed types. This is the minimum we can hope for in a mixed-type pattern. In the first mixed type, for example, we would erroneously predict a pro-choice response to the easiest item for each of the 43 respondents in this group, making a total of 43 errors.
The extent to which a set of empirical re- sponses form a Guttman scale is determined by the accuracy with which the original responses can be reconstructed from the scale scores. For each of the 1,846 respondents in this example, we’ll predict three questionnaire responses, for a total of 5,538 predictions. Table 7-3 indicates that we’ll make 58 errors using the scale scores assigned. The percentage of correct predictions is called the coefficient of reproducibility: the percentage of original responses that could be reproduced by knowing the scale scores used to summarize them. In the present example, the coefficient of reproducibility is 99 percent.
TA B L E 7 2 Scaling Support for Choice of Abortion
Women’s Health
Result of Rape
Woman Unmarried
Number of Cases
Scale types + + + 763
+ + − 633
+ − − 201
− − − 191
Total = 1,788
Mixed types − + − 43
+ − + 7
− − + 4
− + + 4
Total = 58
50094_ch07.indd 220 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Typologies ■ 221
Except for the case of perfect (100 percent) re- producibility, there is no way of saying that a set of items does or does not form a Guttman scale in any absolute sense. Virtually all sets of such items ap- proximate a scale. As a general guideline, however, coefficients of 90 or 95 percent are the commonly used standards. If the observed reproducibility ex- ceeds the level you’ve set, you’ll probably decide to score and use the items as a scale.
The decision concerning criteria in this regard is, of course, arbitrary. Moreover, a high degree of reproducibility does not ensure that the scale con- structed in fact measures the concept under con- sideration. What it does is increase confidence that all the component items measure the same thing. Also, you should realize that a high coefficient of reproducibility is most likely when few items are involved.
One concluding remark with regard to Gutt- man scaling: It’s based on the structure observed among the actual data under examination. This is an important point that is often misunder- stood. It does not make sense to say that a set of
questionnaire items (perhaps developed and used by a previous researcher) constitutes a Guttman scale. Rather, we can say only that they form a scale within a given body of data being analyzed. Scalability, then, is a sample-dependent, empirical matter. Although a set of items may form a Gutt- man scale among one sample of survey respon- dents, for example, there is no guarantee that this set will form such a scale among another sample. In this sense, then, a set of questionnaire items in and of itself never forms a scale, but a set of empiri- cal observations may.
This concludes our discussion of indexing and scaling. Like indexes, scales are composite mea- sures of a variable, typically broadening the mean- ing of the variable beyond what might be captured by a single indicator. Both scales and indexes seek to measure variables at the ordinal level of mea- surement. Unlike indexes, however, scales take advantage of any intensity structure that may be present among the individual indicators. To the extent that such an intensity structure is found and the data from the people or other units of analysis comply with the logic of that intensity structure, we can have confidence that we have created an ordinal measure.
Typologies Indexes and scales, then, are constructed to provide ordinal measures of given variables. We attempt to assign index or scale scores to cases in such a way as to indicate a rising degree of prejudice, religios- ity, conservatism, and so forth. In such cases, we’re dealing with single dimensions.
Often, however, the researcher wishes to sum- marize the intersection of two or more variables, thereby creating a set of categories or types—a nominal variable—called a typology. You may, for
TA B L E 7 3 Index and Scale Scores
Response Pattern
Number of Cases
Index Scores
Scale Scores
Total Scale Errors
Scale types + + + 763 3 3 0
+ + − 633 2 2 0
+ − − 201 1 1 0
− − − 191 0 0 0
Mixed types − + − 43 1 2 43
+ − + 7 2 3 7
− − + 4 1 0 4
− + + 4 2 3 4
Total scale errors = 58
= 1 − 58 1,846 × 3
= 1 − 58 5,538
= .9895 = 99%
Coefficient of reproducibility = 1 − number of errors number of guesses
This table presents one common method for scoring mixed types, but you should be advised that other methods are also used.
typology The classification (typically nominal) of observations in terms of their attributes on two or more variables. The classification of newspapers as liberal-urban, liberal-rural, conservative-urban, or conservative-rural would be an example.
50094_ch07.indd 221 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
222 ■ Chapter 7: Typologies, Indexes, and Scales
example, wish to examine the political orientations of newspapers separately in terms of domestic is- sues and foreign policy. The fourfold presentation in Table 7-4 describes such a typology.
Newspapers in cell A of the table are conserva- tive on both foreign policy and domestic policy; those in cell D are liberal on both. Those in cells B and C are conservative on one and liberal on the other.
As another example, Rodney Coates (2006) created a typology of “racial hegemony” from two dimensions:
1. Political Ideology
a. Democratic
b. Non-Democratic
2. Military and Industrial Sophistication
a. Low
b. High
He then used the typology to examine modern examples of colonial rule, with specific reference to race relations. The specific cases he examined allowed him to illustrate and refine the typology. He points out that such a device represents Max Weber’s “ideal type”: “As stipulated by Weber, ideal types represent a type of abstraction from reality. These abstractions, constructed from the logical extraction of elements derived from spe- cific examples, provide a theoretical model by which and from which we may examine reality” (2006: 87).
Frequently, you arrive at a typology in the course of an attempt to construct an index or scale. The items that you felt represented a single vari- able appear to represent two. We might have been attempting to construct a single index of political orientations for newspapers but discovered— empirically—that foreign and domestic politics had to be kept separate.
In any event, you should be warned against a difficulty inherent in typological analysis. When- ever the typology is used as the independent vari- able, there will probably be no problem. In the preceding example, you might compute the per- centages of newspapers in each cell that normally endorse Democratic candidates; you could then
easily examine the effects of both foreign and do- mestic policies on political endorsements.
It’s extremely difficult, however, to analyze a typology as a dependent variable. If you want to discover why newspapers fall into the different cells of typology, you’re in trouble. That becomes ap- parent when we consider the ways you might con- struct and read your tables. Assume, for example, that you want to examine the effects of community size on political policies. With a single dimension, you could easily determine the percentages of rural and urban newspapers that were scored conserva- tive and liberal on your index or scale.
With a typology, however, you would have to present the distribution of the urban newspapers in your sample among types A, B, C, and D. Then you would repeat the procedure for the rural ones in the sample and compare the two distributions. Let’s suppose that 80 percent of the rural news- papers are scored as type A (conservative on both dimensions), compared with 30 percent of the urban ones. Moreover, suppose that only 5 percent of the rural newspapers are scored as type B (con- servative only on domestic issues), compared with 40 percent of the urban ones. It would be incorrect to conclude from an examination of type B that urban newspapers are more conservative on domestic issues than rural ones are, because 85 percent of the rural newspapers, compared with 70 percent of the urban ones, have this character- istic. The relative sparsity of rural newspapers in type B is due to their concentration in type A. It should be apparent that an interpretation of such data would be very difficult for anything other than description.
In reality, you’d probably examine two such dimensions separately, especially if the dependent
TA B L E 7 4 A Typology of Newpapers
Foreign Policy
Conservative Liberal
Domestic policy Conservative A B
Liberal C D
50094_ch07.indd 222 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
variable has more categories of responses than the given example does.
Don’t think that typologies should always be avoided in social research; often they provide the most appropriate device for understanding the data. To examine the pro-life orientation in depth, for example, you might create a typology involving both abortion and capital punishment. Libertarian- ism could be seen in terms of both economic and social permissiveness. You’ve now been warned, however, against the special difficulties involved in using typologies as dependent variables.
M A I N P O I N T S
Introduction
• Single indicators of variables seldom (1) cap- ture all the dimensions of a concept, (2) have sufficiently clear validity to warrant their use, or (3) permit the desired range of variation to allow ordinal rankings. Composite measures, such as scales and indexes, solve these problems by including several indicators of a variable in one summary measure.
Indexes versus Scales
• Although both indexes and scales are intended as ordinal measures of variables, scales typically sat- isfy this intention better than indexes do.
• Whereas indexes are based on the simple cumula- tion of indicators of a variable, scales take advan- tage of any logical or empirical intensity structures that exist among a variable’s indicators.
Index Construction
• The principal steps in constructing an index in- clude selecting possible items, examining their empirical relationships, scoring the index, and validating it.
• Criteria of item selection include face validity, unidimensionality, the degree of specificity with which a dimension is to be measured, and the amount of variance provided by the items.
• If different items are indeed indicators of the same variable, then they should be related empirically to one another. In constructing an index, the researcher needs to examine bivari- ate and multivariate relationships among the items.
• Index scoring involves deciding the desirable range of scores and determining whether items will have equal or different weights.
• There are various techniques that allow items to be used in an index in spite of missing data.
• Item analysis is a type of internal validation, based on the relationship between individual items in the composite measure and the measure itself. Exter- nal validation refers to the relationships between the composite measure and other indicators of the variable—indicators not included in the measure.
Scale Construction
• Four types of scaling techniques are represented by the Bogardus social distance scale, a device for measuring the varying degrees to which a person would be willing to associate with a given class of people; Thurstone scaling, a technique that uses judges to determine the intensities of different in- dicators; Likert scaling, a measurement technique based on the use of standardized response catego- ries; and Guttman scaling, a method of discover- ing and using the empirical intensity structure among several indicators of a given variable. Gutt- man scaling is probably the most popular scaling technique in social research today.
• The semantic differential is a question format that asks respondents to make ratings that lie between two extremes, such as “very positive” and “very negative.”
Typologies
• A typology is a nominal composite measure often used in social research. Typologies may be used effectively as independent variables, but interpre- tation is difficult when they are used as dependent variables.
K E Y T E R M S
The following terms are defined in context in the chapter and at the bottom of the page where the term is introduced, as well as in the comprehensive glossary at the back of the book.
Bogardus social distance scale Likert scale
external validation scale
Guttman scale semantic differential
index Thurstone scale
item analysis typology
Key Terms ■ 223
50094_ch07.indd 223 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
224 ■ Chapter 7: Typologies, Indexes, and Scales
P R O P O S I N G S O C I A L R E S E A R C H :
C O M P O S I T E M E A S U R E S
This chapter has extended the issue of measurement to include those in which variables are measured by more than one indicator. What you have learned here may extend the discussion of measurement in your proposal. As in the case of operationalization, you may find this easier to formulate in the case of quantitative studies, but the logic of multiple indicators may be ap- plied to all research methods.
If your study will involve the use of composite measures, you should identify the type(s), the indica- tors to be used in their construction, and the methods you’ll use to create and validate them. If the study you are planning in this series of exercises will not include composite measures, you can test your understand- ing of the chapter by exploring ways in which they could be used, even if you need to temporarily vary the data-collection method and/or variables you have in mind.
R E V I E W Q U E S T I O N S A N D E X E R C I S E S
1. In your own words, describe the difference be- tween an index and a scale.
2. Suppose you wanted to create an index for rat- ing the quality of colleges and universities. Name three data items that might be included in such an index.
3. Make up three questionnaire items that measure attitudes toward nuclear power and that would probably form a Guttman scale.
4. Construct a typology of pro-life attitudes as dis- cussed in the chapter.
5. Economists often use indexes to measure eco- nomic variables, such as the cost of living. Go to the Bureau of Labor Statistics link on your Sociol- ogy CourseMate at www.cengagebrain.com and
find the Consumer Price Index survey. What are some of the dimensions of living costs included in this measure?
S P S S E X E R C I S E S
See the booklet that accompanies your text for ex- ercises using SPSS (Statistical Package for the Social Sciences). There are exercises offered for each chapter, and you’ll also find a detailed primer on using SPSS.
Online Study Resources Access the resources your instructor has assigned. For this book, you can access:
CourseMate for The Practice of Social Research
Login to CengageBrain.com to access chapter-specific learning tools including Learning Objectives, Practice Quizzes, Videos, Internet Exercises, Flash Cards, Glossaries, Web Links, and more from your Sociology CourseMate.
If your professor has assigned Aplia homework:
1. Sign into your account.
2. After you complete each page of questions, click “Grade It Now” to see detailed explanations of every answer.
3. Click “Try Another Version” for an opportunity to improve your score.
Visit www.cengagebrain.com to access your account and purchase materials.
50094_ch07.indd 224 11/18/11 5:24 PM
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.