Week 4

Part Four

Measurement Concepts

Chapter 13

Measurement and Scaling Concepts

AT-A-GLANCE

Zikmund, W., Babin, B. J., Carr, J., & Griffin, M. (2013). Business research methods (9th ed.). Mason, OH: Cengage Learning.

I. What Do I Measure?

A. Concepts

B. Operational definitions

· Variables

· Constructs

II. Levels of Scale Measurement

A. Nominal scale

B. Ordinal scale

C. Interval scale

D. Ratio scale

E. Mathematical and statistical analysis of scales

· Discrete measures

· Continuous measures

III. Index Measures

A. Indexes and composites

B. Computing scale values

IV. Three Criteria for Good Measurement

A. Reliability

· Internal consistency

· Test-retest reliability

B. Validity

· Establishing validity

C. Reliability versus validity

D. Sensitivity

LEARNING OUTCOMES

1. Determine what needs to be measured to address a research question or hypothesis

2. Distinguish levels of scale measurement

3. Know how to form an index or composite measure

4. List the three criteria for good measurement

5. Perform a basis assessment of scale reliability and validity

CHAPTER VIGNETTE: Money Matters?

Griff Mitchell is the Vice-President of Customer Relationship Management (CRM) for one of the world’s largest suppliers of industrial heavy equipment, and in this role he oversees all sales and service operations. The company has decided to perform a CRM employee evaluation process that will allow an overall ranking of all CRM employees, and the ranking will be used to single out the best performers to be recognized at the annual CRM conference. The rankings will also be used to identify the lowest 20 percent of performers, who will be put on a probationary list with specific improvement goals. Griff’s key question is, What is performance? So he calls a meeting of senior CRM managers to discuss how the ranking decisions should be made. One manager argued that sales volume should be the sole criterion because it is objective. Another said to use the manager’s opinion to classify an employee as a top performer, a good performer, or an under performer. Another said to use margins. Finally, another said that because they are CRM, customer satisfaction should be used. This did not help Griff much in his quest to develop a valid performance measure that treated all employees fairly. So he seeks the help of an outside research consultant, Robin Donald.

SURVEY THIS!

Students are asked to identify one of each of the four categories of scale measurement and to look at the question shown. What scale measurement do these items represent? Each set of items is designed to capture a single construct—the top portion assesses how much work-life interferes with nonwork-life and the bottom portion assesses self-perceived performance. Compute a coefficient α for each scale to estimate reliability and then create a composite scale by summing the items that make up that particular scale.

RESEARCH SNAPSHOTS

· Peer Pressure and Investing Behavior

While we all have experienced “peer pressure,” research has shown that some individuals are more susceptible than others. This interpersonal influence is typically thought to be present in conspicuous consumption and socially visible products, but recent research shows such influence can occur even in the selection of less visible products and services, such as investments. Researchers used the construct susceptibility to interpersonal influence (SCII) to study this. First, this construct had to be conceptualized and measured and was thought to be composed of two parts—susceptibility to informational influences and susceptibility to normative influences. Questions were developed to measure both parts to capture SCII. Results of the study are given.

· Recoding Made Easy

Most computer software makes scale recoding easy, and a screenshot from SPSS, perhaps the most widely used statistical software in business-related research, is shown. The steps include: (1) click on transform, (2) click on recode, (3) choose to recode into the same variable, (4) select the variable(s) to be recoded, (5) click on old and new values, (6) use the menu that appears to enter the old values and the matching new values, and click add after entering each pair, and (7) click continue.

OUTLINE

I. WHAT DO I MEASURE?

· The decision statement, corresponding research questions and research hypotheses can be used to decide what concepts need to be measured.

· Measurement is the process of describing some property of a phenomenon of interest usually by assigning numbers in a reliable and valid way.

· When numbers are used, the researcher must have a rule for assigning a number to an observation in a way that provides an accurate description.

· All measurement, particularly in the social sciences, contains error.

· Concepts

· A researcher has to know what to measure before knowing how to measure something.

· A concept is a generalized idea that represents something of meaning.

· Concepts such as age, sex, education and number of children are relatively concrete properties and present few problems in either definition or measurement.

· Concepts such as brand loyalty, corporate culture, and so on are more abstract and are more difficult to both define and measure.

· Operational Definitions

· Researchers measure concepts through a process known as operationalization, which is a process that involves identifying scales that correspond to variance in the concept.

· Scales provide a range of values that correspond to different values in the concept being measured.

· Scales provide correspondence rules that indicate that a certain value on a scale corresponds to some true value of a concept, hopefully in a truthful way.

· Variables

· Researchers use variance in concepts to make diagnoses.

· Variables capture different concept values.

· Scales capture variance in concepts and as such, the scales provide the researcher’s variables.

· For practical purposes, once a research project is underway, there is little difference between a concept and a variable.

· Constructs

· Sometimes a single variable cannot capture a concept alone.

· Using multiple variables to measure one concept can often provide a more complete account of some concept than could any single variable.

· A construct is a term used for concepts that are measured with multiple variables.

· Can be very helpful in operationlizing a concept.

II. LEVELS OF SCALE MEASUREMENT

· The level of scale measurement is important because it determines the mathematical comparisons that are allowed.

· The four levels of scale measurement are:

1. nominal

2. ordinal

3. interval

4. ratio

· Nominal Scale

· Nominal scales represent the most elementary level of measurement.

· Assigns a value to an object for identification or classification purposes.

· The value can be but does not have to be a number since no quantities are being represented.

· A qualitative scale.

· Useful even though they can be considered elementary.

· Business researchers use nominal scales quite often.

· Nominal scaling is arbitrary in the sense that each label can be assigned to any of the categories without introducing error.

· Examples of nominal numbering are uniform numbers, airport terminals, and school bus numbers.

· Ordinal Scale

· Ordinal scales have nominal properties, but they also allow things to be arranged based on how much of some concept they possess.

· A ranking scale.

· Somewhat arbitrary, but not nearly as much as a nominal scale.

· Example: “win,” “place,” and “show” in a horse race ( tells which horse was first, second, and third, but does not tell by how much a horse won.

· Interval Scale

· Interval scales have both nominal and ordinal properties, but they also capture information about differences in quantities of a concept.

· Classic example is the Fahrenheit temperature scale:

· 80° F is hotter than 40° F, but you cannot conclude that the 40° is twice as cold as 80° because this is a scaling system.

· To see the point above, convert the temperatures to the Celsius scale. 80° F = 26.7° C, and 40° F = 4.4° C.

· The scale is not iconic, meaning that it does not exactly represent some phenomenon.

· Interval scales are very useful because they capture relative quantities in the form of distances between observations.

· Ratio Scale

· Ratio scales represent the highest form of measurement in that they have all the properties of interval scales with the additional attribute of representing absolute quantities.

· Interval scales represent only relative meaning while ratio scales represent absolute meaning.

· In other words, ratio scales provide iconic measurement.

· Zero, therefore, has meaning in that it represents an absence of some concept.

· An absolute zero is a defining characteristic in determining between ratio and interval scales.

· For example money is a way to measure economic value.

· Mathematical and Statistical Analysis of Scales

· Although you can put numbers into formulas and perform calculations with almost any numbers, the researcher has to know the meaning behind the numbers before useful conclusions can be drawn (e.g., averaging the numbers used to identify school busses is meaningless).

· Discrete Measures

· Discrete measures are those that take on only one of a finite number of values.

· Most often used to represent a classificatory variable and thus do not represent intensity of measures, only membership.

· Common discrete scales include any yes-no response, matching, color choice or practically any scale that involves selecting from a small number of categories.

· Nominal and ordinal scales are discrete measures.

· Certain statistics are most appropriate for discrete measures (shown in Exhibit 13.5).

· The central tendency of discrete measures is best captured by the mode (i.e., most frequent level).

· Continuous Measures

· Continuous measures are those assigning values anywhere along some scale range in a place that corresponds to the intensity of some concept.

· Ratio measures are continuous measures.

· Strictly speaking, interval scales are not necessarily continuous.

· e.g., Likert item ranging from 1=strongly disagree to 5=strongly agree.

· This is a discrete scale because only the values 1, 2, 3, 4, or 5 can be assigned.

· The mean is not an appropriate way of stating central tendency and we really shouldn’t use many common statistics on these responses.

· However, as a scaled response of this type takes on more values, the error introduced by assuming that the differences between the discrete points are equal are smaller.

· Therefore, business researchers generally treat interval scales containing 5 or more categories of response as interval.

· Researchers should keep in mind, however, the distinction between ratio and interval measures.

· Errors in judgment can be made when interval measures are treated as ratio.

· e.g., An attitude of 0 means nothing as attitude only has meaning in a relative sense (i.e., compared to another’s attitude).

· The means and standard deviation may be calculated from continuous data.

· Using the actual quantities from arithmetic operations is permissible with ratio scales.

III. INDEX MEASURES

· An attribute is a single characteristic or fundamental feature of an object, person, situation, or issue.

· Indexes and Composites

· Multi-item instruments for measuring a construct are called index measures, or composite measures.

· An index measure assigns a value based on how much of the concept being measured is associated with an observation.

· An index is often formed by putting several variables together.

· Composite measures also assign a value based on a mathematical derivation of multiple variables.

· For most practical applications, composite measures and indexes are computed in the same way.

· Computing Scale Values

· Exhibit 13.6 demonstrates how a composite measure can be created from common rating scales.

· A summated scale is created by simply summing the response to each item making up the composite measure.

· A researcher may sometimes choose to average the scores rather than summing them because the composite measure is expressed on the same scale as is the items that make it up.

· Reverse coding means that the value assigned for a response is treated oppositely from the other items.

IV. THREE CRITERIA FOR GOOD MEASUREMENT

· The three major criteria for evaluating measurements are:

1. reliability

2. validity

3. sensitivity

· Reliability

· Reliability is an indicator of a measure’s internal consistency.

· A measure is reliable when different attempts at measuring something converge on the same result.

· Internal Consistency

· Internal consistency represents a measure’s homogeneity.

· The set of items that make up a measure are referred to as a battery of scale items.

· Internal consistency of a multiple-item measure can be measured by correlating scores on subsets of items making up a scale.

· The split-half method of checking reliability is performed by taking half the items from the scale and checking them against the results from the other half.

· The two scale halves should correlate highly.

· They should also produce similar scores.

· Coefficient alpha (α) is the most commonly applied estimate of a multiple item scale’s reliability.

· Represents internal consistency by computing the average of all possible split-half reliabilities for a multiple item scale.

· The coefficient demonstrates whether or not the different items converge.

· Ranges in value from 0 (no consistency) to 1 (complete consistency).

· Generally, scales with a coefficient α:

· 0.80 - 0.95: very good reliability

· 0.70 - 0.80: good reliability

· 0.60 - 0.70: fair reliability

· below 0.60: poor reliability

· Test-Retest Reliability

· The test-retest method of determining reliability involves administering the same scale or measure to the same respondents at two separate times to test for stability.

· If the measure is stable over time, the test, administered under the same conditions each time, should obtain similar results.

· Represents a measure’s repeatability.

· Two problems common to all longitudinal studies:

· the premeasure, or first measure, may sensitize the respondents to their participation in a research project and subsequently influence the results of the second measure.

· If the time between measures is long, there may be an attitude change or other maturation of the subjects.

· Validity

· Good measures should be both precise (i.e., reliable) and accurate (i.e., valid).

· Validity is the accuracy of a measure or the extent to which a score truthfully represents a concept.

· Achieving validity is not a simple matter.

· Establishing Validity

· The four basic approaches to establishing validity are face validity, content validity, criterion validity, and construct validity.

· Face validity refers to the subjective agreement among professionals that a scale logically reflects the concept being measured.

· Content validity refers to the degree that a measure covers the domain of interest.

· Criterion validity addresses the question: “Does my measure correlate with measures of similar concepts or known quantities?”

· May be classified as either concurrent validity or predictive validity depending on the time sequence in which the new measurement scale and the criterion measure are correlated.

· If measures taken at the same time ( concurrent validity.

· If measures taken at different times ( predictive validity.

· Construct validity exists when a measure reliably measures and truthfully represents a unique concept and consists of several components:

· Face validity

· Content validity

· Criterion validity

· Convergent validity – another way of expressing internal consistency; highly reliable scales contain convergent validity.

· Discriminant validity – represents how unique or distinct is a measure; a scale should not correlate too highly with a measure of a different construct.

· Reliability versus Validity

· The differences between the two are illustrated with rifle targets (see Exhibit 13.7).

· A: The shots from the older gun are scattered ( low reliability.

· B: The shots from the newer gun are closely clustered and on target ( high reliability and validity.

· C: The shots from a newer gun are closely clustered but off target ( high reliability but low validity.

· Sensitivity

· The sensitivity of a scale is an important measurement concept, particularly when changes in attitudes or other hypothetical constructs are under investigation.

· Sensitivity refers to an instrument’s ability to accurately measure variability in a concept.

· Sensitivity is generally increased by adding more response points or adding scale items.

QUESTIONS FOR REVIEW AND CRITICAL THINKING/ANSWERS

1. Define measurement. How is your performance in your research class being assessed?

Measurement is the process of describing some property of a phenomenon of interest usually by assigning numbers in a reliable and valid way. Students’ performance in a research class will be assessed through their performance on exams, projects, presentations, etc. Each element will be assigned a score (a ratio measure), which is typically translated into a letter grade of A, B, C, D, or F (a nominal measure).

2. What is the difference between a concept and a construct?

A concept can be thought of as a generalized idea that represents something of meaning. Concepts such as age, sex, education, and number of children are relatively concrete properties and present few problems in either definition or measurement. Concepts such as loyalty, personality, channel power, trust, corporate culture, etc. are more difficult both to define and measure. Researchers measure concepts through a process known as operationalization, which involves identifying scales that correspond to variance in the concept. Scales provide a range of values that correspond to different values in the concept being measured. Researchers use variance in concepts to make diagnoses, and scales capture variance in concepts.

Sometimes, a single variable cannot capture a concept alone. Using multiple variables to measure one concept can often provide a more complete account of some concept than could any single variable. A construct is a term used for concepts that are measured with multiple measures.

3. Suppose a researcher takes over a project only after a proposal has been written by another researcher. Where will the researcher find the things that need to be measured?

The problem definition should suggest the concepts that must be measured.

4. Describe, compare, and contrast the four different levels of scale measurement.

Nominal Scales – represent the most elementary level of measurement. A nominal scale assigns a value to an object for identification or classification purposes. The value can be but does not have to be a number because no quantities are being represented.

Ordinal Scales – have nominal properties, but they also allow things to be arranged based on how much of some concept they possess. In other words, an ordinal scale is a ranking scale. However, it does not tell the value of the interval between rankings.

Interval Scales – have both nominal and ordinal properties, but they also capture information about differences in quantities of a concept. This type of scale captures relative quantities in the form of distances between observations, but it is not iconic, meaning that it does not exactly represent some phenomenon.

Ratio Scales – represent the highest form of measurement in that they have all the properties of interval scales with the additional attribute of representing absolute quantities. Provide iconic measurement. Zero, therefore, has meaning in that it represents an absence of some concept.

5. Consider the different grading measuring scales described at the beginning of the chapter. Describe what level of measurement is represented by each. Which method do you think contains that least opportunity for error?

The first type of grading assigned a letter corresponding to a student’s performance, and it represents an interval scale because the descending order of letter represents rank.

The second type of grading assigned a number from 1 to 20 and represents an ordinal scale.

The third type of grading assigned a number corresponding to a percentage performance scale and represents a ratio scale. This type contains the least opportunity for error.

The final type of grading assigned a P or F representing passing or failing and is a nominal scale.

6. Look at the responses to the following survey items that describe how stressful consumers believed a Christmas shopping trip was using a ten-point scale ranging form 1 (= no stress at all) to 10 (= extremely stressful):

a. How stressful was finding a place to park? 7

b. How stressful was the checkout procedure? 5

c. How stressful was trying to find exactly the right product? 8

d. How stressful was finding a store employee? 6

i. What would be the stress score for this respondent based on a summated scale score?

A summated scale is created by simply summing the response to each item making up the composite measure, and in this case it is 26 (i.e., 7+5+8+6 = 26).

ii. What would be the stress score for this respondent based on an average composite scale score?

The average composite scale score is simply the average of the responses and is equal to 6.5 (i.e., 26 ( 4 = 6.5).

iii. Do any items need to be reverse coded? Why or why not?

No because each item was asking how stressful something was for the respondent.

7. How is it that business researchers can justify treating a seven-point Likert scale as interval?

A 7-point Likert scale is an interval scale, and, strictly speaking, interval scales are not necessarily continuous. It is a discrete scale because only the values 1, 2, 3, 4, 5, 6, or 7 can be assigned. Furthermore, it is an ordinal scale because it only orders based on agreement. We really have no way of knowing that the difference in agreement of somebody marking a 5 instead of a 4 is the same as the difference in agreement of somebody marking a 2 instead of a 1. Therefore, the mean is not an appropriate way of stating central tendency, and we really shouldn’t use many common statistics on these responses. However, as a scaled response of this type takes on more values, the error introduced by assuming that the differences between the discrete points are equal become smaller. Therefore, business researchers generally treat interval scales containing five or more categories of response as interval.

8. What are the components of construct validity? Describe each.

Construct validity exists when a measure reliably measures and truthfully represents a unique concept. Construct validity consists of several components:

a. Face – refers to the subjective agreement among professionals that a scale logically reflects the concepts being measured.

b. Content validity – refers to the degree that a measure covers the domain of interest.

c. Convergent validity – another way of expressing internal consistency.

d. Criterion validity – addresses the question, “Does my measure correlate with measures of the similar concepts or known quantities?” It may be classified as either concurrent validity or predictive validity depending on the time sequence in which the new measurement scale and the criterion measure are correlated.

e. Discriminant validity – represents how unique or distinct is a measure. A scale should not correlate too highly (i.e., above .75) with a measure of a different construct.

9. Why might a researcher wish to use more than one question to measure satisfaction with a particular aspect of retail shopping?

The answer to this question requires a recognition that there are several dimensions to shopping. Like many aspects of our lives, shopping may be a source of both disappointment and satisfaction at the same time. It may be appropriate to have a battery of measures to make sure that the scale measures what it is supposed to measure. Because shopping is a complex multidimensional concept, several questions may be required to design scales that are valid and reliable.

10. How can a researcher assess the reliability and validity of a multi-item composite scale?

Reliability is an indicator of a measure’s internal consistency. A measure is reliable when different attempts at measuring something converge on the same result. Internal consistency represents a measure’s homogeneity. Internal consistency of a multiple-item measure can be measured by correlating scores on subsets of items making up a scale. The split-half method of checking reliability is performed by taking half the items from a scale and checking them against the results from the other half. The two scale halves should correlate highly, and they should produce similar scores. Coefficient alpha (α) is the most commonly applied estimate of a multiple item scale’s reliability and represents internal consistency by computing the average of all possible split-half reliabilities. The test-retest method of determining reliability involves administering the same scale or measure to the same respondents at two separate times to test for stability. If the measure is stable over time, the test, administered under the same conditions each time, should obtain similar results. Test-retest reliability represents a measure’s repeatability.

Validity is the accuracy of a measure on the extent to which a score truthfully represents a concept. Achieving validity is not a simple matter. It addresses the problem of whether a measure indeed measures what it is supposed to measure. Researchers have attempted to assess validity in many ways. Three basic approaches to establishing validity are face or content validity, criterion validity, and construct validity, which are described in the answer to Question 8.

11. Comment on the validity and reliability of the following:

a. A respondent’s report of an intention to subscribe to Consumer Reports is highly reliable. A researcher believes that this constitutes a valid measurement of dissatisfaction with the economic system and alienation from big business.

There is a problem with validity in this instance. There may be other reasons for reading Consumer Reports other than alienation from big business. It has been said that a bent ruler may consistently provide the same results, but this does not necessarily indicate accuracy of measurement. This probably is the case when a researcher uses this magazine as an indicator of alienation.

b. A general-interest magazine claimed that it was a better advertising medium than television programs with similar content. Research had indicated that for a soft drink and other test products, recall scores were higher for the magazine ads than for 30-second commercials.

This question deals with advertising effectiveness. The question indirectly asks, “What is advertising effectiveness?” Recall—consumers’ ability to remember commercials—is a standard form of measuring advertising effectiveness. However, it has been argued that the persuasive power of television is substantially greater than magazines. Television has the ability to involve the prospect. Recall may not be a valid measure of advertising effectiveness if the goal is to measure persuasiveness rather than ability to remember the ads.

c. A respondent’s report of frequency of magazine reading consistently indicates that she regularly reads Good Housekeeping and Gourmet and never reads Cosmopolitan.

This question implies a longitudinal study reporting magazine readership at several points in time. Because answers are consistent on all occasions, the results are reliable. However, the question may not be valid. Suppose, for example, a respondent’s longitudinal report of magazine readership gave the following responses over a one-year period: period one, never read Cosmopolitan, period two, subscribe to Cosmopolitan, period three, occasionally read Cosmopolitan. These results show a lack of reliability. While it is possible that the subject has changed her behavior radically over the course of the year, it is more likely that response bias is inherent in the question concerning the reading of Cosmopolitan magazine. Even with consistent answers a response bias could occur, hence a lack of validity.

12. Indicate whether the following measures use a nominal, ordinal, interval, or ratio scale:

a. Prices on the stock market

These are ratio scales. They have an absolute zero point.

b. Marital status, classified as “married” or “never married”

When it is classified as “married” or “never married,” marital status is a nominal scale because it indicates a category of marital status.

c. A yes/no question asking whether a respondent has ever been unemployed

Whether or not a respondent has ever been unemployed is a nominal scale. The two categories are “yes” and “no.”

d. Professorial rank: assistant professor, associate professor, or professor

Professorial rank: assistant professor, associate professor, or professor is an ordinal scale because it indicates an ordered rank, according to a hierarchical status. However, depending on the context of the research, it may be considered to be a nominal scale.

e. Grades: A, B, C, D, or F

These show order, but not necessarily interval order, so it is an ordinal scale.

RESEARCH ACTIVITIES

1. Go to the library and find out how Sales and Marketing Management magazine constructs its buying power index.

The buying power index has three variables that are weighted for importance. They are:

Variable Weight

Population .2

(% of USA)

Retail Sales .3

(% of USA)

Buying Income .5

(% of USA)

1.0

2. Define each of the following concepts, and then operationally define each one by providing correspondence rules between the definition and the scale:

a. A good bowler

Conceptually a good bowler is someone who regularly bowls and scores above average.

Operationally a good bowler might be defined as someone who bowls in a league and has a 185 average.

b. The television audience for The Tonight Show

A conceptual definition could be a member of the audience is an individual who watches a portion of a TV program.

An operational definition could be an individual who watches at least 15 minutes of the program when it is being broadcast or tape delayed as measured by the people-meter system.

c. Purchasing intention for an iPhone

Purchasing intention is an individual’s plan to buy an iPhone. An operational definition might be an individual who indicates “definitely will” or “probably will” on a scale that reads: Do you plan to purchase an iPhone in the next six months?

Definitely Probably Uncertain Probably Definitely

Will Will Will Not Will Not

d. Consumer involvement with cars

Conceptually involvement is the level of personal importance and interest evoked by automobiles.

Operationally it might be a combination of scores on the following two questions:

How important are cars in your personal life?

Very Important Somewhat Important Not at All Important

How interesting are cars to you?

Very Interesting Somewhat Interesting Not at All Interesting

e. A workaholic

Most students will have a feel for this concept and conceptually define it like an alcoholic—as someone who works all the time, who is addicted to work, or who works to excess.

The first operational definition that students will suggest is often someone who works more than x hours per week, perhaps 70 hours per week. This will not, however, tap the cognitive domain of someone who loves to work—who cannot wait for Monday mornings. It may not tap someone’s self perception that they are a workaholic.

Some operational definitions might be answers to “How many hours per week do you work?” or “How many hours did you work this week?”

Or you could use a simple self-report statement, such as “Are you a workaholic?”

A series of attitude statements might also be used as an operational definition. For example:

“I love to go to work on Monday mornings.”

strongly agree agree disagree strongly disagree

“Work is the most important thing in my life.”

strongly agree agree disagree strongly disagree

f. Outstanding supervisory skills

This is a construct that may be describe with several variables such as empathy, delegation of authority, empowerment of employees, fairness, and so on.

g. A risk averse investor

This may be conceptualized through attitudes toward risky investing. Behavior could also be used to infer risk aversion, such as investing in lower risk investments (i.e., annuities).

3. [Internet Question] Use the ACSI scores found at http://www.theasci.org to respond to this question: Using the most recent two years of data, test the following two hypotheses:

a. American consumers are more satisfied with breweries than they are with wireless telephone services.

ACSI stands for American Customer Satisfaction Index. The answers here are given for the years 2010 and 2011, and instructors need to visit this site for subsequent years as they become available.

For 2010 and 2011, the ASCI scores for breweries were 82 for both years, and they were 72 and 71, respectively, for wireless telephone services. So this hypothesis appears to be supported.

b. American consumers are more satisfied with discount and department stores than they are with automobile companies.

For 2010 and 2011, the scores for discount and department stores were 75 and 76, respectively, and they were 82 and 83, respectively, for automobiles and light vehicles. This hypothesis is not supported.

4. Refer back to the opening vignette. Use a search engine to find stories dealing with job performance. In particular, pay attention to stories that may be related to CRM. Make a recommendation to Griff concerning a way that job performance should be measured. Would your scale be nominal, ordinal, interval or ratio?

Students’ responses will vary.

5. Go to http://www.queendom.com/tests . Click on the lists of personality tests. Take the hostility test. Do you think this is a reliable and valid measure of how prone someone is to generally act in a hostile manner?

When I clicked on the personality test, I did not see the hostility test. However, when I searched for it, it came up. Students’ responses regarding whether or not this is a reliable and valid measure will probably be influenced by the score they received. Mine was 32, indicating that I was not a very hostile person, which I think is a pretty accurate description of me. However, if the score is higher, and thus having a more socially undesirable meaning, students may say that it is not a valid measure; indeed, those students might even get angry and want to hit someone!

CASE 13.1 FLYAWAY AIRLINES

Objectives: This case offers an opportunity to discuss the nature of service quality, the measurement of service quality, specific measurement issues and considerations, and related consumer behavior aspects. Specific objectives are:

1. To provide students with an opportunity to critically compare alternative methods for measuring a perceptual concept such as the quality of a service.

2. To require students to think comprehensively about the fact that choice of method has implications for credibility and usefulness of research.

3. To challenge students to discover and compare critical elements of different measurement processes for the same concept.

4. To demonstrate the concept of validity for alternate methods and to stimulate discussion on details of that research concept.

Summary: Quality of airline service is of concern to consumers and industry watchers alike. The perceptual nature of quality makes for a difficult measurement problem. Taking the direct consumer survey approach has its merits in the subjective richness of the results. This type of an approach is also more easily understood by a wider audience. It does, however, suffer from potential sample bias due to lack of respondent experience with multiple airlines and from the cumbersome aspects of a survey process. In addition, the survey approach does not address the need to monitor quality on an on-going and timely basis. There are certainly many qualitative factors that are important to customers in judging quality. These subjective aspects are only assessable by direct inquiry of the consumer. This does not make them less important, just less accessible. Elaborate surveying efforts are necessary to monitor this type of consumer opinion. Most of the major airlines already do this type of quality assessment and use the results to improve the service they offer the consumer. This information is, however, proprietary and not available to the public or to competitor airlines for their use in making better choices involving airline use or for taking competitive action.

Use of a more quantitative and objective approach avoids some of the problems just mentioned for a survey approach. Quality of service can be compared across airlines on exactly the same factors for a comparable time period using data available to all consumers and competitive airlines. Comparisons can be made as frequently as monthly, but more realistically the comparisons should be made across several months. Using a weighted average approach is more difficult to understand, but once understood, it offers a depth of detail and competitive information that surpasses the opinion based survey method.

Taken in tandem we believe the survey method and the AQR are reflective of critical quality aspects that an airline must consider if they are to be responsive to the customer and remain competitive. The Airline Quality Rating offers a way to compare the quality of airlines by using strictly quantitative, comparable, regularly published data. This does not take all aspects of quality into account and does not tell the whole story. For many of the more subjective aspects, such as food quality, pleasurableness of the experience, atmospherics, comfort, and employee attitude, a periodic consumer survey is still necessary. The addition of the AQR to a complete quality assessment program provides a more responsive and comparable method for judging service quality for all airlines.

Questions

1. How comparable are the two different methods? In what ways are they similar? In what ways are they different?

Both methods use consumer based data. The survey method is more direct, while the weighted average method uses reported consumer complaints as a large part of the data base. Surveying relies on the use of single “overall” rating to determine a rank order of airlines as to quality. The AQR produces a weighted average number that, when compared for each airline, can be treated as a continuously scaled variable. This weighted average score can then be translated into a ranking. Only the rankings are fully comparable between the two methods.

The survey method examines perceptual, qualitative, more subjective aspects of the service experience. The weighted average method examines performance, quantitative, and more objective aspects of the service experience.

2. What are the positive and negative aspects of each approach that Shocker should consider before recommending a course of action for FlyAway Airways?

Positive aspects of survey method:

+ Easily understood by management.

+ Represents broad based opinion of actual consumers of airline services.

+ Looks at the perceptual, subjective aspects of service quality.

Negative aspects of survey method:

- Cumbersome and costly to accomplish.

- Respondents lack of experience with all airlines taints the value of the opinion expressed.

- Most probably long time lags between survey efforts.

Positive aspects of weighted average method:

+ Because of standardized factors, the results are very comparable from airline to airline and from time period to time period.

+ Easily repeatable, uses inexpensive publicly available data, and is quick to accomplish.

+ Responds to consumer concerns via the inclusion of several customer complaint factors.

+ Looks at performance oriented factors that are more objective.

Negative aspects of weighted average method:

- Uses only limited data points.

- Performance data that is used is self-reported by the airlines under Federal mandates.

- Weights assigned to each factor can be questioned as to their true representativeness of consumer opinion.

- The direction of impact (+/-) for each factor can be argued as well.

3. What aspects of service quality does each approach address well and not so well?

A survey approach captures perceptual aspects very well, but misses the finer detail of multiple encounters. The survey approach forces the respondent to give an “average” opinion, based on experience to that point in time. These are very meaningful, but everything might change with the next encounter, news of an air crash, or announcement of financial woes for an airline. Also, each respondent’s “average” opinion carries the same weight whether they are heavy, infrequent, or non-users of airline services.

The weighted average of the AQR uses performance data or outcomes as its mainstay. The weighing system allows each factor to have a varying impact on quality. Weights reflect the importance of critical factors such as on-timeness, accidents, flight problems, denied boardings, and lost baggage. The positive or negative sign associated with each weighted factor reflects the impact that the factor has on the consumer’s perception of quality.

4. Considering the two methods outlined, what types of validity would you consider to be demonstrated by the two approaches to measuring quality? Defend your position.

Types of validity demonstrated:

A. Content/face validity: Definition: Judgment of experts in the field that suggests that the items being used in the scale are representative of the concept being investigated. Both methods use similar, but not identical, elements to evaluate the service quality concept that has been recognized by researchers as pertinent when evaluating the quality or satisfaction experienced by a customer (i.e., on-timeness, lost baggage/hassles, safety, ability to serve the customer, financial stability).

B. Criterion validity: Definition: Correlating the results of one scale with the results of another measure or scale designed to measure the same concept. There are really no other scales available for comparison at this point in time. Also, the AQR approach does not allow the capturing of the “overall” quality rating that the survey method is based on, so comparison between the two is difficult.

Reliability: The reliability score (Chronbach’s Alpha) reported represents the scale’s freedom from experimental error and reflects the scale’s consistency of results when applied to other group results at other times. A perfect reliability of 1.00 indicates that exactly the same results would be obtained with other similar samples using this scale.

5. Which of the methods should Shocker recommend? Why?

He should recommend doing both. If FlyAway Airways is already doing a customer survey regarding quality issues, they should continue to monitor this side of the quality concept. In addition, FlyAway Airways should begin to gather the data and monitor the AQR type scores as well. Having regular, comparable, quantitative information to compliment the less frequent survey results offers a more complete picture of service quality for management decision making. Where possible, the two different efforts at assessing service quality should use common definitions and/or question areas. This attention to the content of both methods will result in much better comparison of findings.