Week 7

Chapter 12

Experimental Research

Zikmund, W., Babin, B. J., Carr, J., & Griffin, M. (2013). Business research methods (9th ed.). Mason, OH: Cengage Learning.

AT-A-GLANCE

I. Creating an Experiment

A. An illustration: Can a Self-Efficacy Intervention Enhance Job Attitude?

· Experimental subjects

· Independent variables

· Experimental outcome

· Independent variable main effects and interaction

II. Designing an Experiment to Minimize Experimental Error

A. Manipulation of the independent variable

· Experimental and control groups

· Several experimental treatment levels

· More than one independent variable

· Repeated measures

B. Selection and measurement of the dependent variable

C. Selection and assignment of test units

· Sample selection and random sampling error

· Randomization

· Matching

· Control over extraneous variables

· Experimental confounds

· Extraneous variables

III. Demand Characteristics

A. What are demand characteristics?

B. Experimenter bias and demand effects

C. Hawthorne effect

D. Reducing demand characteristics

· Experimental disguise

· Isolate experimental subjects

· Use a “blind” experimental administrator

· Administer only one experimental condition per subject

IV. Establishing Control

A. Problems controlling extraneous variables

V. Ethical Issues in Experimentation

VI. Practical Experimental Design Issues

A. Basic versus factorial experimental designs

B. Laboratory experiments

C. Field experiments

D. Within-subject and between-subjects designs

VII. Issues of Experimental Validity

A. Internal validity

· Manipulation checks

· History

· Maturation

· Testing

· Instrumentation

· Selection

· Mortality

B. External validity

· Student subjects

C. Trade-offs between internal and external validity

VIII. Classification of Experimental Designs

A. Symbolism for diagramming experimental designs

B. Three examples of quasi-experimental designs

· One-shot design

· One-group pretest-posttest design

· Static group design

C. Three alternative experimental designs

· Pretest-posttest control group design (Before-after with control)

· Posttest-only control group design (After-only with control)

· Compromise designs

D. Time series designs

E. Complex experimental designs

· Completely randomized design

· Randomized-block design

· Factorial designs

LEARNING OUTCOMES

1. Identify the independent variable, dependent variable, and construct a valid simple experiment to assess a cause and effect relationship

2. Understand and minimize the systematic experimental error

3. Know ways of minimizing experimental demand characteristics

4. Avoid unethical experimental practices

5. Understand the advantages of between-subjects experimental design

6. Weigh the trade-off between internal and external validity

7. Use manipulations to implement a completely randomized experimental design, a randomized-block design, and a factorial experimental design

CHAPTER OPENING VIGNETTE: Testing Web Protocols for Financial Markets

Technology has drastically changed the way we conduct banking and related financial services. For most of us, how it all happens is not that relevant. However, for information technology directors and financial managers, how this process occurs—and how to make it occur faster and cheaper—is vitally important. An experiment of two protocols (FIX and SOAP) for transferring information over computer networks is described. Researchers designed a laboratory experiment to compare the relative performance of the two in business computing scenarios. While the results are not important to us, an understanding of the process of testing these two approaches is very important to business researchers.

SURVEY THIS!

Students are asked to look at the questions on prospective careers. The questions actually represent a very simple experimental design in which the type of occupation described may cause the subjects’ responses to these questions. Students are asked to look at the data and see if they can determine whether or not students’ beliefs about careers are altered by the type of occupation they were assigned to rate. Here, job type becomes the experimental manipulation. Do you know the treatment levels?

RESEARCH SNAPSHOTS

· Talking While Driving: Are Cell Phone Conversations Different from Passenger Conversations?

The increased use of cell phones while driving has led to increased driver distractions and the potential for accidents and even fatalities. But people engage in conversations with fellow passengers as they drive as well. Researchers designed an experiment with a driving simulator to test whether engaging in a conversation on a cell phone is more dangerous. Students were assigned to one of three groups: a group with no passenger as a control group, a group that had a passenger who asked them questions while they simulated driving, and a group where the driver engaged in a cell phone conversation. The results indicate that driving while engaged in a conversation on a cell phone was indeed much more distracting and dangerous than having a conversation with a passenger, perhaps because a passenger can anticipate changing driving conditions and adjust their engagement in conversation accordingly.

· The Hidden in Hidden Valley Ranch

Hidden Valley Ranch (HVR) conducted a field market experiment to test three new flavors of salad dressings (i.e., three levels of the experimental variable flavor). These types of tests are expensive because small batches of each flavor is produced, bottled, and shipped to sales representative who stock the dressings in the stores. The first day involved placing the products in retail store. On the second day, the reps returned to record sales and saw that all flavors had sold. It turns out that a competitor bought every bottle, and HVR was unable to produce any sales data (the dependent variable). Moreover, the competitor was able to determine the recipe in their labs. So the risks with field tests are: no more secrets once the product is for sale and the risk of espionage rendering the experiment invalid.

OUTLINE

I. CREATING AN EXPERIMENT

· The purpose of experimental research is to allow the investigator to control the research situation so that causal relationships among variables may be evaluated.

· The experimenter manipulates one or more independent variables and holds constant all other possible independent variables while observing the effects on dependent variables.

· Independent variables are expected to determine the outcomes of interest.

· Dependent variables are the outcomes of interest.

· An Illustration: Can a Self-Efficacy Intervention Enhance Job Attitude?

· An experiment investigating how self-efficacy might influence employee’s attitude toward their job is described (Self-efficacy is a person’s confidence and belief in their own abilities to accomplish tasks at hand).

· Experimental Subjects

· Participants in experimental research are referred to as subjects rather than respondents because the experimenter subjects them to some experimental treatment.

· In this experiment, 35 of the subjects were given positive feedback and encouragement from their supervisors as the experimental treatment. The other 36 subjects were not provided the positive feedback.

· Independent Variables

· Experiment involved one relevant independent variable—whether or not the employee received positive feedback.

· While not a true independent variable, the length of time each employee had worked with the firm was also important to the researchers.

· Variables such as this are referred to as blocking variables—a categorical (less than interval) variable that is not manipulated as in an experimental variable but is included in the statistical analysis of experiments.

· Experimental condition refers to one of the possible levels of an experimental variable manipulation.

· Subjects were divided into “newcomers” (the new employees) and “insiders” (current employees) and then randomly assigned to either the treatment condition or the control group.

· Differences between groups are analyzed.

· Experimental Outcome

· The key outcomes, or dependent variables, is the subject’s job satisfaction (only one looked at in this illustration), organizational commitment, professional commitment, intent to quit the organization, and intent to quit the profession.

· In addition, the researchers followed up later to see if the subjects had actually left the firm.

· A rating scale assessed job satisfaction, with higher scores representing higher job satisfaction.

· Means are given in Exhibit 12.2.

· Looks like the difference in job satisfaction is primarily between the current and new employees.

· Independent Variable Main Effects and Interaction

· Length of time that the employee works at the firm clearly appears to matter, but attempts to enhance self-efficacy shouldn’t be dismissed so quickly.

· The research must examine both the effects of each experimental variable alone and the effects due to combinations of variables.

· A main effect refers to the experimental difference in means between the different levels of any single experimental variable.

· An interaction effect is due to a specific combination of independent variables.

· It’s possible that the combination of length of employment and self-efficacy treatment creates effects that are not clearly represented in the main effects.

· Experimental results are often shown with a line graph (see Exhibit 12.3).

· Main effects are illustrated when the lines are at different heights.

· When the lines have very different slopes, an interaction is likely present.

· In this example:

· The worst situation is the current employees who do not receive positive feedback.

· The best scenario occurs when the treatment is given to new employees.

· It appears that job satisfaction decreases over time, and the benefit of the self-efficacy treatment is greater for the employees that have been with the organization than for new employees.

II. DESIGNING AN EXPERIMENT TO MINIMIZE EXPERIMENTAL ERROR

· Experimental designs involve no less than four important design elements:

1. Manipulation of the independent variable(s).

2. Selection and measurement of the dependent variable(s).

3. Selection and assignment of experimental subjects.

4. Control over extraneous variables.

· Manipulation of the Independent Variable

· The researcher creates the values of the independent variables.

· Experimental independent variables are hypothesized to be causal influences.

· An experimental treatment is the term referring to the way an experimental variable is manipulated, and it often involves treatments with more than two levels.

· Experimental variables are often categorical variables that take on a value to represent some classifiable or qualitative aspect (e.g., protocol is either FIX or SOAP – see chapter vignette).

· Independent variables may truly be continuous variables, and the researcher must select appropriate levels of that variable as experimental treatments.

· Researcher decides on levels that would be relevant to study.

· Levels should be noticeably different and realistic.

· Experimental and Control Groups

· In the simplest experiment, an independent variable is manipulated over two treatment levels resulting in two groups: an experimental group and a control group.

· Experimental group – one in which an experimental treatment is administered.

· Control group – one in which no experimental treatment is administered.

· By holding conditions constant in the control group, the researcher controls for potential sources of error in the experiment.

· Several Experimental Treatment Levels

· By analyzing more groups each with a different treatment level, a more precise result may be obtained than in the simple experimental group-control group experiment described above.

· May still involve a control group.

· Design can produce only a main effect.

· More Than One Independent Variable

· Include the effect of another experimental variable.

· The term cell is used to refer to treatment combinations within an experiment.

· The number of cells involved in any experiment can be easily computed as follows: K = (T1)(T2)…(Tm).

· K = the number of cells

· T1 = the number of treatment levels for experimental group number one.

· T2 = the number of treatment levels for experimental group number two and so on.

· Including multiple variables allows a comparison of experimental treatments on the dependent variables.

· This design involves both main effects and interactions.

· Repeated Measures

· Repeated measures designs – experiments in which an individual subject is exposed to more than one level of an experimental treatment.

· Possesses several drawbacks even though it is more economical.

· Selection and Measurement of the Dependent Variable

· Choosing the right dependent variable is part of the problem definition process.

· The amount of time needed for effects to become evident should be considered.

· Selection and Assignment of Test Units

· Test units are the subjects or entities whose responses to the experimental treatment are measured or observed (i.e. individual consumers, employees, organizational units, sales territories, market segments, brands, stores, etc.).

· People are the most common test units in most business experiments.

· Sample Selection and Random Sample Errors

· As in other forms of research, random sampling errors and sample selection errors may occur.

· Sample selection error occurs because of flaws in procedures used to assign experimental test units.

· Systematic or non-sampling error may occur if the sampling units in an experimental cell are somehow different than the units in another cell, and this difference affects the dependent variable.

· Randomization

· Randomization – the random assignment of subject and treatments to groups – is one device for equally distributing the effects of extraneous variables to all conditions.

· Nuisance variables—items that may affect the dependent measures but are not of primary interest—will not be eliminated, but they will be controlled because they are likely to exist to the same degree in every experimental cell.

· Matching

· Assigning subjects in a way that their characteristics are the same in each group.

· This is best thought of in terms of demographic characteristics (e.g., If a subject’s sex is expected to influence the dependent variable, make sure there are equal numbers of men and women in each experimental cell.).

· While useful, the researcher can never be sure that sampling units are matched on all characteristics.

· Control Over Extraneous Variables

· This is related to the various types of experimental error.

· Recall that total survey error was classified into two basic types: random sampling error and systematic error.

· Same dichotomy applies to all research designs, but the terms random (sampling) error and systematic error are more frequently used when discussing experiments.

· Experimental Confounds

· A confound in an experiment means that there is an alternative explanation beyond the experimental variables for any observed differences in the dependent variable.

· Once identified, the validity of the experiment is severely questioned.

· Extraneous Variables

· Marketing mix variables (i.e., price, product, promotion, and distribution) interact with uncontrollable forces in the market (i.e., competitors’ activities, consumer trends).

· Thus, many marketing experiments are subject to the effect of extraneous variables.

· Must be identified before the experiment if at all possible so the experimenter can control or eliminate such variables.

III. DEMAND CHARACTERISTICS

· What Are Demand Characteristics?

· Demand characteristic – an experimental design element that unintentionally provides subjects with hints about the research hypothesis.

· Knowledge of the experimental hypothesis creates a particular type of confound known as a demand effect.

· Experimenter Bias and Demand Effects

· Demand characteristics are aspects of an experiment that demand (encourage) that the subjects respond in a particular way ( a source of systematic error.

· Prominent demand characteristics are often presented by the person administering experimental procedures.

· Experimenter bias – an experimenter’s presence, actions, or comments influence the subjects’ behavior or sway the subjects to slant their answers to cooperate with the experimenter.

· Hawthorne Effect

· Hawthorne Effect – people will perform differently when they know they are experimental subjects.

· Named after a famous management experiment that attempted to study the effects on productivity of various working conditions (i.e., hours of work, rest periods, lighting, methods of pay) at the Western Electric Hawthorne plant in Cicero, IL.

· Researchers found that workers’ productivity increased regardless of the manipulation.

· Investigators realized that the workers’ morale was higher because they were aware of being part of a special experimental group.

· Social interaction should be restricted in laboratory experiments because conversations might produce joint decisions rather than a desired individual one.

· Reducing Demand Characteristics

· It is practically impossible to eliminate demand characteristics, but the following steps can by taken to reduce them:

1. Use an experimental disguise.

2. Isolate experimental subjects.

3. Use a “blind” experimental administrator.

4. Administer only one experimental treatment level to each subject.

· Experimental Disguise

· Subjects can be told that the purpose of the experiment is somewhat different than the actual purpose.

· Most often, they are simply told less than the complete “truth” about what is going to happen.

· In other cases, more deceit may be needed.

· A placebo is an experimental deception involving a false treatment.

· A placebo effect refers to the corresponding effect in a dependent variable that is due to the psychological impact that goes along with knowledge of the treatment.

· Isolate Experimental Subjects

· Discussion among subjects may lead them to guess the experimental hypotheses.

· Integrity will be higher when each only knows enough to participate in the experiment.

· Use a “Blind” Experimental Administrator

· The people administering the experiment may not be told the hypotheses.

· Less likely to give off clues that result in demand effects.

· Administer Only One Experimental Treatment Per Subject

· When subjects know more than one experimental treatment condition, they are much more likely to guess the experimental hypotheses.

IV. ESTABLISHING CONTROL

· The major difference between experimental research and descriptive research is an experimenter’s ability to control variables by either holding conditions constant or manipulating the experimental variable.

· When extraneous variables cannot be eliminated, experimenters may strive for constancy of conditions, which means that subjects in all experimental groups are exposed to identical conditions except for the differing experimental treatments.

· If an experimental method requires that the same subjects be exposed to two or more experimental treatments, an error may occur due to the order of presentation.

· Counterbalancing attempts to eliminate the confounding effects of order of presentation.

· Problems Controlling Extraneous Variables

· It is not always possible to control every possible extraneous variable (e.g., competitors may bring out a product during a test market).

· Competitors aware of a company’s test market may knowingly change its marketing to confound the test.

V. ETHICAL ISSUES IN EXPERIMENTATION

· Although deception is necessary in most experiments, when subjects can be returned to their prior condition through debriefing, then it is probably consistent with high moral standards.

· If debriefing will not return subjects to their former condition because they are injured significantly or truly psychologically harmed, then the experiment should not proceed.

· In test-markets, when a company puts a product out for public consumption, competitors may also now freely consume the product.

· When attempts to interfere with a test market are aimed solely at invalidating test results or infringing on some copyright protection, those acts are ethically questionable.

VI. PRACTICAL EXPERIMENTAL DESIGN ISSUES

· Basic Versus Factorial Experimental Designs

· In basic experimental designs a single independent variable is manipulated to observe its effect on a single dependent variable.

· Factorial experimental designs are more sophisticated than basic experimental designs and allow for an investigation of the interaction of two or more independent variables.

· Laboratory Experiments

· An experiment can be conducted in a natural setting (a field experiment) or in an artificial setting (a laboratory experiment).

· In a laboratory experiment the researcher has more complete control over the research setting and extraneous variables.

· Some laboratory experiments may be more controlled or artificial.

· An example of a device used in a very controlled experiment is a tachistoscope, which can be used to experiment with the visual impact of advertising, packaging, and so on, by controlling the amount of time a visual image is exposed to a subject.

· Field Experiments

· Field experiments are research projects involving manipulations that are implemented in a natural environment.

· Test markets are field experiments.

· A researcher manipulates experimental variables but cannot possibly control all the extraneous variables.

· Generally, subjects know when they are participating in a laboratory experiment, but in field experiments, subjects do not even know they have taken part in an experiment.

· With field experiments, the consent is implied since subjects are not asked to do anything other than their normal behavior.

· Controlled store tests – products are put into stores in a number of small cities or into selected supermarket chains.

· Within-Subjects and Between-Subjects Designs

· A basic question is how many treatments should a subject receive?

· For economical reasons, the researcher may wish to apply multiple treatments to the same subject, which is called a within-subjects design that involves repeated measures because with each treatment the same subject is measured.

· Between-subjects design – each person receives only one treatment combination and each dependent variable is measured only once for every subject.

· More costly but more advantageous.

· Validity is higher because demand characteristics are greatly reduced.

· Statistical analyses are simpler.

· Results are easier to report and explain to management.

VII. ISSUES OF EXPERIMENTAL VALIDITY

· Internal Validity

· Internal validity exists to the extent that an experimental variable is truly responsible for any variance in the dependent variable.

· If the observed results were influenced or confounded by extraneous factors, the researcher will have problems making valid conclusions about the relationship between the experimental treatment and the dependent variable.

· A lab experiment enhances internal validity because it maximizes control of outside forces.

· Manipulation Checks

· Internal validity depends in large part on successful manipulations.

· Manipulations should be carried out in a way that varies the experimental variable over meaningfully different levels.

· If the levels are too close together, the experiment may lack the power necessary to observe differences in the dependent variable.

· The validity of manipulations can often be checked with a manipulation check.

· In business, it is often done by asking a survey question or two.

· Should always be administered after dependent variables in self-response format experiments to keep the manipulation check item from becoming a troublesome demand characteristic.

· Extraneous variables can jeopardize internal validity, and six types are:

1. history

2. maturation

3. testing

4. instrumentation

5. selection

6. mortality

· History Effect

· A history effect occurs when some change other than the experimental treatment occurs during the course of an experiment that affects the dependent variable.

· A common history effect occurs when competitors change their strategies during a test marketing experiment.

· Particularly prevalent in repeated measures experiments that take place over an extended time.

· A special case is the cohort effect, which refers to a change in the dependent variable that occurs because members of one experimental group experienced different historical situations than members of other experimental groups.

· Maturation

· Maturation effects are effects that are function of time and the naturally occurring events that coincide with growth and experience.

· Testing

· Testing effects are also called pretesting effects because the initial measurement or test alerts subjects in a way that affects their response to the experimental treatments.

· Only occur in a before-and-after study (i.e., one in which an initial baseline measure is taken before an experimental treatment is administered).

· May increase awareness of socially approved answers, increase attention to experimental conditions, or make the subject more conscious than usual of the dimensions of a problem.

· Instrumentation

· A change in the wording of questions, a change in interviewers, or a change in other procedures used to measure the dependent variable causes an instrumentation effect.

· Problematic with any type of repeated measures design.

· Selection

· The selection effect is a simple bias that results from differential selection of respondents for the comparison groups, or sample selection error, discussed earlier.

· Mortality

· If an experiment is conducted over a period of a few weeks or more, some sample bias may occur due to mortality, or sample attrition.

· Attrition occurs when some subjects withdraw from the experiment before it is completed.

· Mortality effects may occur if subjects drop from one experimental treatment group disproportionately from other groups.

External Validity

· External validity is the accuracy with which experimental results can be generalized beyond the experimental subjects.

· Increased when the sample truly represents some population and when the results extend to market segments or other groups of people.

· The higher the external validity, the more one can count on the fact that any results observed in an experiment will also be seen in the “real world.”

· Lab experiments are associated with low external validity because the limited set of experimental conditions does not adequately represent all the influences existing in the real world.

· Lack of external validity results in difficulty repeating the experiment with any change in subjects, settings or time.

· Student Subjects

· Basic researchers often use college students as subjects.

· Students are easily accessible, but they often are not representative of the total population.

· However, when behaviors are studied for which students have some particular expertise, then they are appropriate.

· Trade-Offs Between Internal and External Validity

· Laboratory experiments with many controlled factors usually are high in internal validity, while field experiments generally have less internal validity, but greater external validity.

· Ideally, results from lab experiments would be followed up with some type of field test.

VIII. CLASSIFICATION OF EXPERIMENTAL DESIGNS

· There are various types of experimental designs.

· If only one variable is manipulated, the experiment is a basic experimental design.

· If the experimenter wishes to investigate several levels of the independent variable (e.g., four price levels), or to investigate the interaction effects of two or more independent variables, then the experiment requires a complex, or statistical, experimental design.

· Symbolism for Diagramming Experimental Designs

· The following symbolism facilitates the description of the various experimental designs:

X = exposure of a group to an experimental treatment.

O = observation or measurement of the dependent variable. If more than one observation is taken, subscripts will be given to indicate temporal order.

R = random assignment of test units.

· As we diagram designs utilizing these symbols, the reader should assume a time flow from left to right.

· Three Examples of Quasi-Experimental Designs

· Quasi-experimental designs do not involve random allocation of subjects to treatment combinations.

· They do not qualify as true experimental designs because they do not adequately control for the problems associated with loss of internal validity.

· One-Shot Design

· Also known as after-only design and is diagrammed as follows:

X O1

· This one-shot design is a case study fraught with problems:

· subjects participate because of voluntary self-selection or arbitrary assignment

· study lacks any kind of comparison or any means of controlling extraneous influences

· Under certain circumstances, though, it is the only viable choice.

· One-Group Pretest-Posttest Design

O1 X O2

· This design offers a comparison on the same individuals before and after treatment (i.e., training).

· Although this is an improvement over the one-shot design, this design still has several weaknesses that may jeopardize internal validity (i.e., maturation, testing effect, and mortality).

· However, despite its weaknesses, this design is used frequently in business research.

· Static Group Design

· Each subject is identified as a member or either an experimental group or a control group.

· Experimental group is measured after being exposed to the treatment, and the control group is measured without having been exposed to the treatment:

Experimental group: X O1

Control group: O2

· The results of a static control group are computed by subtracting the observed results in the control group from those in the experimental group (O1 - O2 ).

· A major weakness of this design is that we have no assurance that the groups were equal on variables of interest before the experimental group received the treatment.

· If the groups were selected arbitrarily by the investigator or if entry into either group was voluntary, systematic differences between the groups could invalidate the conclusions about the effect of the treatment.

· Random assignment of subjects may minimize problems with group differences.

· If the groups are established by the experimenter rather than existing as a function of some other causation, the static group design is referred to as an after-only design with control group.

· On many occasions, an after-only design is the only possible option (i.e., conducting use tests for new products or brands).

· Three Alternative Experimental Designs

· In the next three designs, the symbol R to the left of the diagram indicates that the first step in a true experimental design is the randomization of subject assignment.

· Pretest-Posttest Control Group Design (Before-After With Control)

· This is the classic experimental design:

Experimental group: R O1 X O2

Control group: R O3 X O4

· This design has the advantage of the before-after design with the additional advantages gained from having a control group.

· The effect of the experimental treatment equals (O2 - O1) - (O4 - O3).

· It is assumed that the effect of extraneous variables will be the same on both the experimental and the control groups.

· This assumption is also made for effects of other events between the before and after measurements (history), changes within the subjects that occur with the passage of time (maturation), testing effects, and instrumentation effects.

· However, a testing effect is possible when subjects are sensitized to the subject of the research.

· This weakness in the before-after with control group design can be corrected (see the next two designs).

· Posttest-Only Control Group Design (After-Only With Control)

Experimental group: R X O1

Control group: R O2

· The effect of the experimental treatment is equal to O2 – O1..

· With only posttest measurement, the effects of testing and instrument variation are eliminated.

· Further, all the same assumptions about extraneous variables are made; that is, they operate equally on both groups.

· Compromise Designs

· In many instances of business research true experimentation is not possible—the best the researcher can do is approximate an experimental design.

· A compromise design is one that falls short of assigning subjects or treatments randomly to groups.

· The alternative to the compromise design when random assignment of subjects is not possible is to conduct the experiment without a control group.

· Generally this is considered a greater weakness than utilizing groups that have already been established.

· When the experiment involves a longitudinal study, circumstances usually dictate a compromise with true experimentation.

· Time Series Designs

· Experiments that are investigating long-term structural change may require a time series design.

· When experiments are conducted over long periods of time, they are most vulnerable to historical changes.

· In such cases the following quasi-design is utilized:

O1 O2 O3 X O4 O5 O6

· Several observations are taken to identify trends before the treatment is administered.

· After the treatment, several observations are made to determine if the patterns after the treatment are similar to those before.

· Of course, this time series design cannot give the researcher complete assurance that the treatment caused the change in the trend, but it does enable the researcher to distinguish temporary changes from permanent changes.

· Complex Experimental Designs

· Complex experimental designs are statistical designs that isolate the effects of confounding extraneous variables or allow for the manipulation of more than one independent variable.

· Completely Randomized Designs

· A completely randomized design is an experimental design that uses a random process to assign subjects to treatment levels of an experimental variable.

· Randomization is an attempt to control extraneous variables while manipulating potential causes.

· A random number process can be used to assign subjects to one of the treatment groups.

· Randomized Block Design

· The randomized block design is an extension of the completely randomized design.

· A form of randomization is utilized to control for most extraneous variables.

· However, if the researcher has identified a single extraneous variable that might affect subjects’ responses systematically, then the researcher will attempt to isolate the single variable by blocking out its effects.

· A blocking variable is a categorical variable that is expected to be associated with different values of a dependent variable for each group (e.g., biological sex).

· The term randomized block originated in agricultural research that applied several levels of a treatment variable to each of several blocks of land.

· In business research, the researcher may wish to isolate block effects such as store size, territory location, market shares of the test brand or its major competition, per capita consumption levels for a product class, city size, etc.

· Factorial Designs

· Allow for the testing of the effects of two or more treatments (factors) at various levels.

· Main effects are differences (in the dependent variable) between treatment levels.

· Interactions produce differences (in the dependent variable) between experimental cells based on combinations of variables.

· For example, with an experiment with three levels of one factor (e.g., three levels of price) and two levels of another (e.g., two packaging designs), we have a 3 x 2 (read “three by two”) factorial design because the first factor is varied in three ways and the second factor is varied in two ways.

· A 3 x 2 design requires six cells, or six experimental groups (3 x 2 = 6).

· If the subjects each receive only one combination of experimental variables, then we use the term 3 x 2 between-subjects design to describe the experiment.

· The number of treatments (factors) and the number of levels of each treatment identify the factorial design.

· The important idea is that each treatment level is combined with every other treatment level.

· In addition to the advantage of investigating two or more independent variables simultaneously, factorial designs allow researchers to measure interaction effects.

· If the effect of one treatment differs at various levels of another treatment, interaction occurs.

QUESTIONS FOR REVIEW AND CRITICAL THINKING/ANSWERS

1. Define experimental condition, experimental treatment, and experimental group. How are these related to the implementation of a valid manipulation?

An experimental condition refers to one of the possible levels of an experimental variable manipulation. An experimental treatment is the term referring to the way an experimental variable is manipulated. An experimental group is one in which an experimental treatment is administered. Internal validity depends in large part on successful manipulations. Manipulations should be carried out in a way that varies the experimental variable over meaningfully different levels. If the levels are too close together, the experiment may lack the power necessary to observe differences in the dependent variable. A manipulation check may be performed by asking the experimental group a survey question or two. A valid manipulation would produce substantially difference average responses between the experimental groups receiving varying levels of the experimental treatment.

2. A tissue manufacturer that has the fourth-largest market share plans to experiment with a 50¢ off coupon during November and a buy one, get one free coupon during December. The experiment will take place at Target stores in St. Louis and Kansas City. Sales will be recorded by scanners from which mean tissue sales for each store for each month can be computed and interpreted.

a. What is the independent variable and the dependent variable?

The independent variable is the price promotion, either a 50¢ off coupon or a buy one, get one free coupon. One might argue that city of the experiment, either St. Louis or Kansas City, is also an independent variable. The dependent variable is sales.

b. Prepare a “dummy” table that would describe what the results of the experiment would look like.

Treatment	Sales
Price Promotion
50¢ off coupon (Nov)	$XXX
Buy one, get one coupon (Dec)	$XXX

3. What is the difference between a main effect and an interaction in an experiment? In question 2, what will create a main effect? Is an interaction possible?

A main effect refers to the experimental difference in means between the different levels of any single experimental variable. In question 2, there is a main effect for type of price promotion. An interaction effect is due to a specific combination of independent variables. In question 2, an interaction is not possible because there is only one experimental variable. That is, there are two levels (i.e., 50¢ off coupon or buy one, get one coupon) of only one experimental variable (i.e., price promotion). However, is a student states that city is also an independent variable, then an interaction is possible.

4. In what ways might the design in question 2 yield systematic or non-sampling error?

Systematic or non-sampling error may occur if the sampling units in an experimental cell are somehow different than the units in another cell, and this difference affects the dependent variable. While the location (i.e., St. Louis or Kansas City) is not an independent variable, it could have a systematic effect on the dependent variable.

5. What purpose does the random assignment of subjects serve?

Random assignment to a group is one device for equally distributing or scattering the effects of extraneous variables to all conditions. The presence of nuisance variables will not be eliminated, but they will be controlled because they are likely to exist to the same degree in every experimental cell. Thus, all cells would be expected to yield similar average scores on the dependent variables if it were not for the experimental treatments administered in a particular cell.

6. Why is an experimental confound so damaging to the conclusions drawn from an experiment?

A confound in an experiment means that there is an alternative explanation beyond the experimental variables for any observed differences in the dependent variable. Once a potential confound is identified, the validity of the experiment is severely questioned.

7. What are demand characteristics? How can they be minimized?

The term demand characteristics refers to an experimental design element that unintentionally provides subjects with hints about the research hypothesis. Knowledge of the experimental hypothesis creates a confound, and this particular type of confound is known as a demand effect. Although it is practically impossible to eliminate demand characteristics from experiments, there are steps that can be taken to reduce them, which include: (1) use an experimental guise, (2) isolate experimental subjects, (3) use a “blind” experimental administrator, and (4) administer only one experimental treatment level to each subject.

8. [Ethics Question] Suppose researchers were experimenting with how much more satisfied consumers are with a “new and improved” version of some existing product. How might the researchers design a placebo within an experiment testing this research question? Is using such a placebo ethical or not?

A placebo is an experimental deception involving a false treatment. A placebo effect refers to the corresponding effect in a dependent variable that is due to the psychological impact that goes along with knowledge of the treatment. To test this research question, one group of subjects may be asked to evaluate a “new and improved” product that really is new and improved. Another group could be asked to evaluate a “new and improved” product that is really not any different than it originally was, which is the placebo. With regard to ethics, deception is necessary in most experiments. However, when subjects can be returned to their prior condition through debriefing, then the experiment is probably consistent with high moral standards, which is the case in this example. However, when subjects have been injured significantly or truly psychologically harmed, debriefing will not return them to their formal condition and the experiment should not proceed.

9. If a company wanted to know whether to implement a new management training program based on how much it would improve ROI in its southwest division, would you recommend a field or lab experiment?

Most likely a field experiment is appropriate in this situation. While a laboratory experiment allows the researcher more complete control over the research setting and extraneous variables, it is done at the expense of external validity. Field experiments are research projects involving experimental manipulations (i.e., training) that are implemented in a natural environment. However, in a field experiment, the researcher manipulates experimental variables but cannot possibly control all the extraneous variables, which causes it to have lower internal validity. However, in this situation, external validity is probably of higher priority to management than internal validity.

10. [Internet Question] Suppose you wanted to test the effect of three different e-mail requests inviting people to participate in a survey posted on the Internet. One simply contained a hyperlink with no explanation, the other said if someone participated $10 would be donated to charity, and the other said if someone participated he or she would have a chance to win $1,000. How would this experiment be conducted differently based on whether it was a between-subjects or within-subjects design? What are the advantages of a between-subjects design?

Within-subjects designs involve repeated measures because with each treatment the same subject is measured. In contrast, the researcher could decide that each person will receive only one treatment condition, which is referred to as a between-subjects design. Each dependent variable is measured only once for every subject. Between-subjects designs are usually advantageous although they are usually more costly. The validity of between-subjects designs is usually higher because by applying only one treatment combination to one subject, demand characteristics are greatly reduced. In addition, statistical analysis of between-subjects designs are simpler than within-subjects designs. This also means the results are easier to report and explain to management.

11. What is a manipulation check? How does it relate to internal validity?

The validity of manipulations can often be checked with a manipulation check. It is often conducted by asking a survey question or two. Internal validity depends in large part on successful manipulations. Manipulations should be carried out in a way that varies the experimental variable over meaningfully different levels. If the levels are too close together, the experiment may lack the power necessary to observe differences in the dependent variable.

12. [Ethics Question] What role does debriefing play in ensuring that experimental procedures are consistent with good ethical practice?

Debriefing experimental subjects by communicating the purpose of the experiment and the researcher’s hypotheses about the nature of consumer behavior is expected to counteract negative effects of deception, relieve stress, and provide an educational experience for the subject.

RESEARCH ACTIVITIES

1. Consider the following scenario (See text for the scenario):

a. Provide a critique of the procedures used to support the claim that Sea Snapper’s product is superior. Prepare it in a way that it could be presented as evidence in court.

One could argue that color of the plate on which the fish sticks were offered to subjects and the quality of the product were confounding variables that influenced the dependent variable.

b. Design an experiment that would provide a more valid test of the research question, “Do consumers prefer Sea Snapper fish sticks compared to Captain John’s fish sticks?”

The most obvious changes would be to use the product in the same degree of quality and to use the same color of plate. That is, instead of getting Sea Snapper’s fish sticks from the test kitchen and Captain John’s from a package, they should both come from a package, especially since this is a more realistic scenario for consumers.

2. Conduct a taste test involving some soft drinks with a group of friends. Pour them several ounces of three popular soft drinks and simply label the cups A, B and C. Make sure they are blind to the actual brands. Then, let them drink as much as they want and record how much of each they drink. You may also ask them some questions about the drinks. Then, allow other subjects to participate in the same test, but this time, let them know what the three brands are. Record the same data and draw conclusions. Does brand knowledge affect behavior and attitudes about soft-drinks?

This will interesting for students. Some research has shown that in blind taste tests, consumers prefer Pepsi over Coke, but when subjects are allowed to see the brand names, Coke is preferred. It will be interesting to see if students find similar results.

CASE 12.1 Tooheys

Objective: To encourage students to think about validity and ethical issues surrounding experiments.

Summary: Sixty-six willing Australian drinkers helped a Federal Court judge decide that Tooheys did not engage in misleading or deceptive advertising for its 2.2 beer, which contains 2.2 percent alcohol, compared to 6 percent for other beers leading to a claim that could be interpreted as implying it was non-alcoholic. Drunken driving laws prohibit anyone with a blood-alcohol level above 0.50 from driving in Australia. An experiment was conducted to see what happens when a lot of 2.2 is consumed. However, some subjects couldn’t drink the required 10 “middies,” an Aussie term for a beer of 10 fluid ounces within an hour. Some got sick and were excluded, and a few more could not even drink the minimum number of drinks. The judged observed that consuming enough 2.2 in an hour to reach the legal limit was “uncomfortable and therefore and unlikely process.” Because none of the ads mentioned such extreme quantities, he ruled that the ads could not be found misleading or deceptive.

Questions:

1. Would a lab experiment or a field experiment be more “valid” in determining whether Tooheys could cause a normal beer consumer to become intoxicated? Explain.

If the purpose is to demonstrate, as in this case, that consumers cannot possibly drink enough within a one hour period to become intoxicated, then a lab experiment would be appropriate. It would support Toohey to show that it is not feasible to do it in such a controlled environment, which would only make their case stronger when they argue that consumers do not really drink that way (at least most don’t drink that way). A field experiment, where consumers would consume this product at a much slower rate and most likely have food as well, would show the same result, but much less dramatically.

2. Describe an alternative research design that would have higher validity.

This lab experiment had high internal validity. If the goal is high external validity, then a field experiment would be more appropriate.

3. Is the experiment described in this story consistent with good ethical practice? Likewise, comment on how the design described in part 2 would be made consistent with good ethical practices.

Students will have varying opinions on the ethical nature of this research. Forcing participants, however willing, to consume a product to the point of throwing up, is causing physical harm, albeit short-lived. A field experiment where consumers are drinking at their own leisure may be considered more ethical.

4. Is validity or ethics more important?

This should generate a lively discussion. Hopefully, the students will agree that ethics is more important.