reading discussion

wsa543

Architectural_Research_Methods_----_Chapter_9_Experimental_and_QuasiExperimental_Research.pdf

Home >Architecture and Design homework help >reading discussion

313

C h a p t e r 9

Experimental and Quasi‐Experimental Research

9 . 1 I n t R o d u c t I o n

Research on the performance of various building components has constituted a significant and long-standing domain within architectural research. Although much of this research has focused on improving various building technologies in the advanced industrialized world, a research study by Givoni, Gulich, Gomez, and Gomez focuses instead on radiant cooling by metal roofs, a significant issue for housing in developing countries.1 Givoni et al. noted that, although corrugated metal roofs are effective for cooling in the evening, they are prone to overheating houses in the daylight hours. The researchers hypothesized that the installation of operable hinged interior insulating plates under the roof would reduce daytime heating while simultaneously not interfering with the nighttime cooling function of the metal roofs.

To test this hypothesis, the researchers built a small‐scale mock‐up of the typi- cal house (termed a “test cell”) whereby the heating/cooling effect of various test conditions could be measured (see Figure 9.1). To be specific, Givoni et al. tested three distinct conditions of insulation operation: (1) with the insulation panels closed both day and night; (2) with the insulation panels open at night and closed during the day; and (3) with the insulation positioned as in 2, but with the addition of a small ventilating fan from midnight to 5:00 a.m. In addition, two levels of ther- mal mass (as represented by water-filled bottles) were also tested.

Based on their testing of these conditions, the authors conclude that the com- bination of both insulating panels and fan venting (condition 3) provides better daytime cooling than without the fan ventilation. However, no appreciable

Wang, David, and Linda N. Groat. Architectural Research Methods, John Wiley & Sons, Incorporated, 2013. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/rit/detail.action?docID=1166322. Created from rit on 2020-09-13 16:55:11.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

314 Part II: Seven Research Strategies

difference in cooling was noted as a consequence of the thermal mass condition. Finally, based on these data, the authors were able to develop predictive formulae for calculating the indoor maximum temperature as a function of the swing of the outdoor temperature.

Taking on a very different topic area, researcher Ann Sloan Devlin sought to discover the extent to which gender might have an effect on how job applicants are evaluated in architectural practice.2 To be more specific, she hypothesized that “women architects would be less favorably rated than male architects,” and more so at the more senior level.3

To test this hypothesis, Devlin created both a junior‐level and senior‐level ré- sumé, the junior level with 4 years of architectural experience and the senior level with 13 years of experience. Half of each résumé type (junior or senior) was desig- nated by a fictitious female name, and half by a fictitious male name. Each résumé included a career objective, professional experience, affiliation, registration, educa- tion, skills, and honors and awards. By using identical gender-designated résumés, Devlin is adapting a long‐standing experimental design employed by researchers who have similarly tested out gender biases in other fields, including, for example, a study of faculty applicants to a psychology department.4

Figure 9.1 Test cell used by Givoni et al. Courtesy of American Solar Energy Society, Inc.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 315

Respondents in Devlin’s study were over 200 architects (156 men and 48 women) licensed in the state of Connecticut, but representing all regions of the country. Respondents were told that the study was about “the perception architects have of the characteristics possessed by those practicing architecture.” These re- spondents then randomly received one of the four fictitious résumés and were asked to evaluate the candidates on a 7-point scale for the following qualities: technical aspects of the job, administrative aspects, interpersonal aspects, contribution to growth of firm’s client base, creative contribution, advancement, and overall rating. Of particular significance, respondents were also asked whether they would accept or reject the candidate for hire.

The most salient result of Devlin’s study was that the “male architect respon- dents were more likely to hire male applicants than female applicants as senior ar- chitects.”5 Devlin reaches this conclusion by comparing the hiring decisions of the respondents in relation to the four résumé conditions (male or female; intern or senior), using inferential statistical measures (see Chapter 8, section 8.3.1). She concludes that women in architecture may indeed “experience discrimination as they advance through the ranks.”6

9 . 2 S t R at E g y: g E n E R a l c h a R a c t E R I S t I c S o f E x p E R I m E n ta l R E S E a R c h

In some very obvious respects, these two studies may seem to be worlds apart. On a thematic level, the Givoni et al. study tackles an aspect of environmental technol- ogy, while the Devlin study seeks to clarify the dynamics of gender discrimination in architectural practice. Second, the research contexts are very different. The for- mer is conducted in a laboratory setting, while the latter makes use of a real‐life or “field” setting. Third, the variables being investigated are quite different. The Givoni et al. study considers only physical variables, whereas the Devlin study focuses on behavioral or social conditions.

Despite this variety of notable differences, both the Givoni et al. and Devlin studies are nevertheless examples of experimental research design. Many readers are likely to read into that factual statement either a commendation of high praise or an invitation to disparage such research. This is because experimental research is so frequently portrayed as the standard against which all other research strategies should be judged. In general, readers who adhere to the postpositivist system of inquiry are likely to see the experimental strategy as the essence of credible “scien- tific” research. However, many researchers who adhere to the intersubjective or subjective paradigmatic positions have argued persuasively that the experimental

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

316 Part II: Seven Research Strategies

design is often either inappropriate or insufficient for research about certain social and cultural dimensions of designed environments. We will address some of these concerns later in this chapter (see section 9.6). Nevertheless, we would argue that just as is the case with each of the several research strategies, experimental research can yield both outstanding or flawed research depending on how appropriately it is applied to a particular research question.

What, then, are the underlying commonalities that define the Givoni et al. and Devlin studies as experimental research? Briefly, the defining characteristics of an experimental research design include: the use of a treatment, or independent vari- able; the measurement of outcome, or dependent, variables; a clear unit of assign- ment (to the treatment); the use of a comparison (or control) group; and a focus on causality.7 These five characteristics will be discussed in some detail in the follow- ing chapter segments.

9.2.1 The Use of a Treatment, or Independent Variable

In each of the two studies described earlier, the researchers are seeking to study the impact of one or more specific, identifiable variables on the phenomenon under study. In the case of the metal roof research, the researchers are seeking to test the thermal impact of several conditions, both in isolation and in combination, including insulation, venting fan, and thermal mass. Similarly, in her research on gender issues in professional practice, Ann Sloan Devlin is seeking to clarify the impact of gender des- ignations on how architects evaluate job applicants. Although quite different in nature, these variables are manipulated or controlled by the researchers in some specified way, and as such these are considered to be treatments in the experimental strategy.

9.2.2 The Measurement of One or More Outcome Variables

In each of these studies, the researchers were able to specify the impact of the ex- perimental treatment by carefully measuring certain outcome measures, or depen- dent variables. For Givoni et al.’s study of metal roofs, the dependent variables were the temperature readings for indoor areas of the test cell environments, including both the attic and the indoor living environment. More specifically, the researchers were able to ascertain how much the indoor temperatures were cooled by the sev- eral experimental conditions (see Figures 9.2 and 9.3). In a similar way, Devlin was able to assess the impact of gender designations through two measures: a question- naire instrument whereby prospective employers could register their evaluation on a 1‐to‐7 rating scale, and a hiring decision to accept or reject. Again, although quite different in nature, both the temperature and evaluation measures are the outcome measures (or dependent variables) of these experiments.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 317

9.2.3 The Designation of a Unit of Assignment

In each of these studies, the researchers applied the experimental treatment to a specified unit of assignment. In the case of Givoni et al.’s research, the treatment con- ditions (various combinations of insulation, venting fans, and mass) are all applied to a test cell. This test cell was a small‐scale mock‐up of a metal‐roofed residential unit in a hot climate, a 1‐meter cube with metal-roofed gable (see Figure 9.1).

Figure 9.2 Temperature variation by each condition tested. From Givoni et al. Courtesy of American Solar Energy Society, Inc.

Figure 9.3 Temperature variation by each condition tested. From Givoni et al. Courtesy of American Solar Energy Society, Inc.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

318 Part II: Seven Research Strategies

However, in Devlin’s study the “unit of assignment” was not an inanimate object, but rather the individual architects who were asked to evaluate the fictitious job applicants. Each of these “units”—whether test cells or individual architects— received a treatment manipulated by the researcher.

9.2.4 The Use of a Comparison or Control Group

A fourth common feature of these two studies is their use of a comparison or con- trol group. The control condition in Givoni et al.’s study is achieved with the insula- tion panels closed both day and night, such that no heating or cooling occurs. In all other conditions (i.e., treatment conditions), the insulation panels are closed dur- ing the day and opened at night to allow for cooling. In other words, the control condition is defined as one to which the treatment is not applied. However, in Devlin’s study, it is more accurate to say that comparison groups received different treatments. This is because all architect respondents received some treatment, one of four combinations of male or female applicant, and junior or senior level. The purpose of using either a control or comparison groups is to allow measurement of the relative effect of the treatment, or independent variable, against the units that received either no treatment or a different treatment.

9.2.5 A Focus on Causality

The combined effect of these several defining features of the experimental research design (i.e., treatment, outcome measures, unit of assignment, and control or com- parison groups) is to enable the researcher to credibly establish a cause‐effect relationship. In general, the experimental researcher is seeking to ascertain and measure the extent to which one or more treatments cause a clearly measured out- come within a specified research setting, whether in a laboratory or in the field.

Although the underlying structure of the experimental research design is es- sentially consistent across diverse topic areas, there are nevertheless some differ- ences in emphasis, specifically the extent to which the issue of “causality” can be taken for granted.8 To be specific, experimental research in environmental technol- ogy (such as the metal roof study) is more likely to take causality for granted than research on sociocultural aspects of architecture (such as the gender designation research). This is because environmental technology, like much research in many fields of science and engineering, tends to incorporate the following characteristics: (1) the use of laboratory settings where relevant variables can be easily controlled; (2) variables that are in many instances inert, and therefore likely to remain consis- tent and amenable to accurate measurement; (3) explicit theories that enable

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 319

researchers to specify the expected effects of a particular treatment; and (4) mea- surement instruments that are precisely calibrated to measure such effects. Given these more easily measurable conditions, then, causality in such research can often be assumed without much discussion or argument.

However, in research that involves people’s reactions to physical and/or social variables (especially in field settings, as is the case in Devlin’s research), researchers tend to be more explicit about how they have met the basic requirements of experi- mental design. For example, Devlin explicitly emphasizes the random assignment of résumé recipients to the four treatment conditions, random assignment being a sig- nificant hallmark of experimental design. Likewise, in drawing their conclusions, researchers who explore socio‐physical dynamics in architecture tend to emphasize the conditions and limitations of a causal interpretation.

Similarly, this is exactly the case in the way Devlin qualifies her conclusion that male respondents tended to rate senior female applicants less positively than the senior male applicants. Devlin specifically mentions two limitations to a causal interpretation: (1) many respondents explained that they found it hard to rate the applicants because the résumé information was so limited; and (2) the response rate was only 30% and therefore the extent of generalizability to the larger popula- tion of architect employers is limited. Such problems and limitations in experimen- tal research will be discussed in greater detail in segment 9.5 of this chapter.

Box 9.1

The Effect of Intelligibility on Place Legibility

This study by Yixiang Long and Perver Baran aims to address the question: To what extent do certain objective physical features of cities, mea- sured by Space Syntax analyses, affect people’s subjective experience of the urban environment?a It is a notable undertaking in several respects. First, it builds on Kevin Lynch’s classic and influential study, The Image of the City, and seeks to identify potentially causal objective measures that lead to people’s experience of legibility encoded in Lynch’s concepts of nodes, landmarks, districts, edges, and paths. Second, it employs Space Syntax, a school of thought and analytical framework developed by Bill Hillier and colleagues, to analyze how morphologies of space embody social and

a Yixiang Long and Perver K. Baran, “Does Intelligibility Affect Place Legibility? Understanding the Relationship between Objective and Subjective Evaluations of the Urban Environment,” Environment and Behavior, in press.

(Continued )

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

320 Part II: Seven Research Strategies

cognitive logic. (See Chapters 8 and 11 for more discussion of Space Syn- tax.) By investigating the relationship between these two well‐established conceptual frameworks, this study represents a innovative integration of two significant theoretical contributions. Third, although the correlational research strategy represents the most common methodology for investi- gating the relationship of spatial form to subjective cognitive responses, the authors have used a decisively experimental strategy to good effect.

The research design for this study entails a field experiment conducted in the city of Changsha, the capital city of Hunan Province in China. A space syntax analysis of the entire city was used to identify the two study areas for the experiment; a standard axial maps analysis was conducted for each neighborhood separately. A combination of measures (global integration, local integration, and connectivity) was used to differentiate the overall “intelligibility” of the two neighborhoods (see Figures 9.4 and 9.5). The first neighborhood, Dong‐pai‐lou, is characterized by a system of streets that is “highly permeable inward as well as outward . . . , indicating a clear rela- tionship between global and local structure.” However, the second neigh- borhood, Rong‐wan‐zhen, has a more treelike structure that “does not connect well with north and south sub‐areas of the neighborhood . . . and there is an unclear relationship between the global and local structure.”

Figure 9.4 Dong‐pai‐lou system of highly permeable streets. Courtesy of SAGE Publications.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 321

The stated hypothesis for the experiment is: “[T]his difference in intel- ligibility (our independent variable) will play an important role in indi- vidual’s [sic] spatial cognition (i.e., place legibility).” To this end, the authors employed a “posttest‐only two experimental group design,” whereby university student volunteers unfamiliar with these neighbor- hoods were initially assessed on a spatial/visual‐ability test and matched as comparable pairs. Students with the same gender and spatial ability

Figure 9.5 Rong‐wan‐zhen treelike structure of streets. Courtesy of SAGE Publications.

(Continued )

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

322 Part II: Seven Research Strategies

9 . 3 S t R at E g y: d I S t I n g u I S h I n g b E t w E E n E x p E R I m E n ta l a n d Q u a S I ‐ E x p E R I m E n ta l R E S E a R c h

So far in our discussion, we have discussed only the general requirements of ex- perimental research, without recognizing the very important distinction between experimental and quasi‐experimental designs. This distinction rests on the manner in which the units of assignment (whether test cells, people, etc.) are selected for either experimental or control treatments. Although the goal for both experimental and quasi‐experimental research is to achieve comparability among the units in each treatment group, such comparability is more precisely established in experi- mental research through random assignment. In contrast, the quasi‐experimental research design is often employed in field settings where people or physical vari- ables cannot be randomly assigned because of either ethical or practical reasons. In such cases, the researcher seeks to ascertain or establish effective comparability across as many variables as possible. These considerations are discussed in greater detail in this chapter section.

test scores were then randomly assigned to one of two groups of 24 participants.

Each of the two treatment groups met in the specific neighborhood location and was asked to “freely explore the neighborhood for an hour.” Following the neighborhood exploration, they were asked to complete three tasks: (1) to draw a sketch map of the neighborhood they explored, (2) a scene‐recognition test, and (3) a brief survey. In the brief survey, participants were asked to indicate their confidence regarding drawing the sketch map, the accuracy of their map and scene recognition, and giving directions. In a nutshell, the participants who explored the more intelligible neighborhood demonstrated more accurate path knowledge, recognized more scenes, and had more confidence in their spatial‐cognitive abilities.

Over all, the hypothesis was generally supported: intelligibility (mea- sured by space syntax analyses) does influence perceived legibility. The practical significance of this finding is that space syntax measures are eas- ier and less time consuming to implement and compute than many other wayfinding performance measures taken with respondents either in real environments or simulated environments. In particular, space syntax mea- sures can be taken during the design phase of an urban design or large architectural project such that the design can be modified for legibility before it is built.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 323

9.3.1 Random Assignment in Experimental Research

Random assignment is an important criterion in experimental research where there is reason to believe that the units of assignment may not always be equivalent. In such instances, random assignment is considered the most effective way to ensure the essential comparability of treatment groups. If the “units” within treatment groups are truly equivalent, the observed differences in outcome measurements can then be credibly attributed to the treatment itself.

In the case of the gender discrimination study, Devlin was actually able to em- ploy random assignment, even though the respondents were not conducting their evaluations in a laboratory setting. By choosing to manipulate the résumé condi- tions rather than depend on the real-life applicant resumes received by these archi- tects, Devlin could assign résumé treatments randomly to the list of architects registered in Connecticut (Devlin’s home state). This provides a greater level of as- surance that the gender of the applicant actually had a measurable effect on the male architects’ evaluations.

However, in experimental research based on inert materials (such as the Givoni et al study), the comparability of assigned “units” does not necessarily require the sort of randomization measures essential for studies about people’s reactions to so- cial or physical conditions. In most circumstances, the essential comparability of test cells or mock‐ups can be assumed either because: (1) materials of the same physical specifications are used; or (2) the same physical unit can be reused in a different treatment condition. As a consequence, the authors of the metal roof study can claim that, given certain specified climatic conditions, the different measured cool- ing outcomes can be attributed to the differences in treatment conditions.

9.3.2 Nonrandom Assignment in Quasi‐Experimental Research

As mentioned earlier, research studies conducted in the field frequently entail situ- ations in which random assignments cannot be achieved because of either ethical or practical reasons. For example, if a researcher wanted to test the effect of four lighting systems on employee productivity in four separate office areas, it is unlikely that management would agree to assign the employees randomly to the four office areas such that important work group functions would be disrupted.

In this situation, researchers would likely adopt a quasi‐experimental design in which they would identify four existing work groups, each of which would receive a different lighting treatment. In doing so, the researchers would attempt to find work groups comparable in as many respects as possible, including task or work objectives, mix of job types, gender mix, age range, level of education, and so on. If, for instance, the work groups’ tasks were quite dissimilar, it would then be more

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

324 Part II: Seven Research Strategies

difficult to attribute measured differences in productivity to the lighting treatment rather than differences in the tasks.

Another example of quasi‐experimental design is a small research project con- ceived and conducted by students in one of Groat’s research methods classes.9 The students had raised in discussion the example of a small gallery area near the school offices that had been created to function as both an exhibit space and a lounge area for faculty and students. In the students’ view, the space was seldom used as lounge. Discussion soon revolved around what sort of changes would have to be made for the space to function more as a lounge and social space. The students hypothesized that the gallery would be used more if the arrangement of furniture were more in- formal and if small screening elements were used to block the view through the glass wall along the doorway side of the space.

The students’ research design involved two sets of observations of the space: the first observations recorded people’s use of the space in its existing condition, and the second recorded its use under the experimental treatment. The observa- tions were made on the Monday (studio day) and Tuesday (nonstudio day) of two successive weeks, starting at 8:30 in the morning and continuing to 7:30 at night. Each observation period was for 15 minutes duration starting on the half‐hour and ending at 45 minutes after each hour.

The experimental treatment condition, used in the second two‐day observation period, was designed to create a more “inviting” ambience; it entailed alteration of the furniture arrangement, lighting levels, and ambient sound (see Figures 9.6 and 9.7). More specifically, the following alterations were made: addition of screening elements to create more visual privacy from the hallway windows; relocation of some furniture elements for more privacy and to create groupings; lowering of fluo- rescent lighting levels; addition of incandescent table lamps; introduction of reading materials on the tables; use of soft background music; and introduction of plants.

Finally, the students also developed a one‐page observation sheet that included the following information: a count of the number of people using the space during that observation period; a plan of the gallery including the furniture arrangement in which the people’s movement and activities were mapped; and a coding system by which people’s specific activities could be described (i.e., speaking, writing/ reading, sleeping).

The general conclusion that the students were able to draw was that although the numbers of people using the space did not change substantially, the average amount of time each person spent in the gallery increased, and the nature of their activities changed as well (see Figures 9.8, 9.9, 9.10, 9.11, 9.12, and 9.13). Indeed, by the second day of the treatment condition, the proportion of staying activities was more than double that of the previous Tuesday in the control condition.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 325

Figure 9.6 Existing and modified condition of the space observed. Courtesy of Barnes et al.

Figure 9.7 Existing and modified condition of the space observed. Courtesy of Barnes et al.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

326 Part II: Seven Research Strategies

Figure 9.8 Comparison of the total observed activities. Courtesy of Barnes et al.

Figure 9.9 Comparison of the total observed activities. Courtesy of Barnes et al.

Figure 9.10 Comparison of moving/staying activities for each day of observation. Cour- tesy of Barnes et al.

Figure 9.11 Comparison of moving/staying activities for each day of observation. Courtesy of Barnes et al.

Figure 9.12 Comparison of moving/staying activities for each day of observation. Cour- tesy of Barnes et al.

Figure 9.13 Comparison of moving/staying activities for each day of observation. Courtesy of Barnes et al.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 327

How much of this change can be attributed to the treatment effect? Due to the circumstances of the field setting, the students were unable to assign gallery users randomly to the two conditions, and so they adopted a quasi‐experimental design. But since no specific measures of the gallery users were taken, it is not possible to gauge precisely how the users of the control condition compared with those in the treatment condition. Still, there were no obvious indicators that the groups were substantially nonequivalent. It is therefore likely, but not certain, that the “informal, inviting” condition did encourage and enable a change in the use patterns of the gallery space.10

9 . 4 d I a g R a m m I n g E x p E R I m E n ta l R E S E a R c h d E S I g n S

From the experience of the architectural design process, we know that it is often helpful, sometimes even essential, to diagram the singular qualities of a design con- cept or parti. In a similar vein, experimental researchers have devised a way of dia- gramming the particular details of experimental research designs, using the following coding system:

{R = Random assignment} {X = Experimental treatment} {O = Observation of dependent variables (e.g., pretest or posttest)}

Although there are a great many typical or standard experimental research de- signs designated by an established nomenclature,11 for our purposes it is a sufficient introduction to diagram the three exemplar studies that have been discussed thus far in the chapter.

Taking the Givoni et al. study of radiant cooling first, this research design can be represented as follows. Each row represents, from left to right, the sequence en- tailed in each treatment condition.

O {Observation only, with no prior treatment} X1 O {Treatment 1, and subsequent observation} X2 O {Treatment 2, and subsequent observation} X3 O {Treatment 3, and subsequent observation}

This notation system conveys the following essential points about the design of this study: (1) no explicit attention is paid to random assignment, as all the rel- evant procedures deal with standardized inert materials; (2) there are three different treatment conditions in addition to the control condition; and (3) only posttest (i.e., no pretest) observations are made.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

328 Part II: Seven Research Strategies

Devlin’s study of gender issues in architectural practice presents a slightly dif- ferent research design in the following respects: (1) random assignment is an ex- plicit and important consideration for establishing comparability across treatment groups; and (2) there is no explicit control condition. However, similar to Givoni et al.’s study, no pretest observations are made. Thus, the notation system for this study can be represented this way:

R X1 O {Random assignment, followed by treatment 1, observation} R X2 O {Random assignment, followed by treatment 2, observation} R X3 O {Random assignment, followed by treatment 3, observation} R X4 O {Random assignment, followed by treatment 4, observation}

Finally, the Barnes et al. study of behavioral patterns in a gallery space presents a slightly more ambiguous research design. This is because the researchers were not able to determine the extent to which the people who experienced the original gal- lery arrangement were likewise the people who experienced the modified arrange- ment. (In retrospect, this might have been achieved by asking users if they had come into the gallery anytime during the previous Monday or Tuesday.) If the gal- lery users had been substantially the same group, then the notation of the research design would be as follows:

O O X O O {Two observations, treatment, followed by two observations}

This design is known as a “single‐group interrupted time‐series design,” whereby two pretest observations were made, after which the treatment (physical modification) was applied, followed by two posttest observations.

However, if the two sets of users were substantially or completely different, then it would be more accurate to diagram the research design in the following way:

O O {No treatment, two observations only} X O O {Treatment, followed by two observations}

This second diagram presumes that the group that experienced the original gallery arrangement constitutes the control group, whereas the group that experi- enced the new arrangement was the experimental treatment group. Both control and treatment groups were observed twice, the treatment group only as a posttest.

Finally, the Long and Baran study made use of a posttest‐only two experimen- tal group design. In other words, each of the two randomly assigned comparison groups received a different treatment condition (one of the two city neighborhoods explored), and there was no control group.

X O {Treatment, followed by one observation} X O {Treatment, followed by one observation}

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 329

Readers who choose to make use of experimental research procedures are ad- vised to consult some of the books cited in the chapter endnotes for further exam- ples of specific experimental designs. These diagrammatic notations can be exceedingly useful to the researcher for clarifying the precise nature and assump- tions of the experimental design he or she selects.

9 . 5 ta c t I c S : t h E S E t t I n g S , t R E at m E n t S , a n d m E a S u R E S f o R E x p E R I m E n ta l R E S E a R c h

Thus far, our discussion of experimental and quasi‐experimental research has fo- cused on the defining characteristics of the research strategy itself. However, within the experimental design, there are numerous options regarding the tactics for achieving such an experimental strategy. For instance, the experimental setting can range from a highly controlled laboratory to less well‐controlled field sites. Similarly, the treatment conditions can range from highly calibrated physical manipulations to categorical, nonphysical conditions, such as the gender designations in Devlin’s study. Finally, measurement of the outcome variables can range from the instru- mented measures of physical changes (such as air temperature measurement in the Givoni et al. research) to less finely measured indexes of a behavioral response (such as in Devlin’s study).

In the examples that follow, the broad range and combinations of tactics avail- able to experimental and quasi‐experimental research will be discussed in the con- text of several specific research studies.

9.5.1 Clarifying the Tactics of the Previously Discussed Studies

Before considering additional examples of experimental research, we would like to characterize more explicitly the tactics selected by the researchers of the previously cited studies. For instance, Givoni et al.’s study of radiant cooling employs the sort of tactics typically associated with experimental research in environmental technol- ogy. The construction and treatment of the test cells was carefully monitored within a university lab setting. The physical treatment conditions of the test cells could be precisely specified and controlled by the experimenters; likewise, the outcome measures of air temperature could be exactly measured by laboratory instruments. (See Figure 9.14 for a complete summary of tactics used in the experimental studies cited in this chapter section.)

In contrast, the Devlin study represents a set of experimental procedures starkly different from the Givoni et al. study. Indeed, one could argue that the

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

330 Part II: Seven Research Strategies

combination of the setting, treatments, and measures in Devlin’s study represent virtually the opposite end of the spectrum. First, the research setting is not only a field setting, but one that is in effect dispersed across the country, to offices where the architects received the résumé conditions. Second, although the treatment con- ditions were conveyed physically in print through gendered names and stated levels of employment experience, the physical and interactive reality of a real‐life appli- cant was absent. Finally, the outcome measures of evaluation and employment de- cision were rendered through scores on a questionnaire. In all of these ways, the focus of the study was on the social‐cultural implications of nonphysical treatment conditions, measured through attitudinal responses.

Third, the Barnes et al. student study of the architecture gallery, though quasi‐ experimental in design, represents an intermediate range of tactics. First, although the study employs a field setting rather than a lab, the setting itself is relatively small and easily manipulated by the experimenters. Secondly, the treatment conditions are all physically based (i.e., arrangement of furniture, the type of lighting, etc.); as such, they can be clearly specified and measured in physical terms. Finally, although the outcome is behavioral and requires some interpretation, the standards for counting people and classifying behavior can be clearly standardized.

Study Setting Treatment Outcome Measures

1. Radiant cooling (Givoni et al.)

Lab Environmental modifications insulation venting mass

Instrumented measures air temperature

2. Gender issues (Devlin)

Field Résumés gender seniority

Attitudinal response applicant evaluation hiring decision

3. Gallery behavior (Barnes et al.)

Field Environmental modifications furniture lighting ambient sound screens

Behavioral change staying/moving

4. Place legibility (Long and Baran)

Field Neighborhood setting low vs. high intelligibility

Place legibility sketch maps recognition tests surveys

Figure 9.14 Summary of tactics in cited studies.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 331

Lastly, the Long and Baran study on the effects of the qualities of urban form on people’s experience of place legibility is likewise a field experiment. In this case the treatment condition entailed the exposure of two comparable groups of students to one of two neighborhood locations—one with a higher level of intelli- gibility, and the other with a much lower level. All subjects were instructed to “freely explore the neighborhood for an hour.” To test the extent to which the individual students were able to perceive legibility in the neighborhood they explored, three outcome measures were used: (1) the drawn accuracy of a sketch map; (2) a scene recognition test; and (3) a survey questionnaire aimed to test the subjects’ sense of confidence in their sketch map, scene recognition, and direction giving.

9.5.2 Environmental Performance of Automated Blinds in Office Buildings: Using a Behavioral Survey Prior to Lab Experiment

Research by Kim et al. focuses in particular on the use of blinds to conserve energy and improve comfort in modern office buildings.12 Their study focuses on the poten- tial efficacy of automated Venetian blind systems as compared to the use of manual or motorized systems. Considerable research on the use of the latter two options has previously demonstrated the limitations of both these systems, largely because office occupants rarely modify the position of the blinds in response to changing environ- mental conditions. Thus, the authors hypothesize that automated blinds may have relatively greater potential for energy savings and improved comfort.

To test this hypothesis, the researchers began first by conducting a survey of blind usage by the occupants of a 22‐story office building in Seoul, Korea. The blinds in each office were operated either directly by the occupants themselves or through a central control center. Blind operations were monitored over two clear days and two cloudy days, at 10‐minute intervals. The overall conclusion confirmed the general re- sults of much previous research, that is, that most blinds were never or rarely operated (see Figure 9.15). Moreover, the pattern of operation varies by building facade expo- sure, with the fewest adjustments made by occupants of south‐facing offices. Overall, this pattern of usage “is not sufficient to meet the energy savings requirements and environmental demands for comfort.”13 Based on the survey of blind operations, the authors selected for their experiment a south‐facing office condition, with the blinds set at an occlusion index of 75%, the average reading for clear sky conditions.

To conduct their set of experiments, the authors built two full‐size, side‐by‐ side mock‐up test rooms. The rooms were built to the same dimensions, and with identical heat loss and gain properties; test room 1 was fitted with the automated blinds, while test room 2 was fitted with manual or motorized blinds. To assess the thermal performance of the blinds, measurements were taken of both the difference between indoor and outdoor temperatures, and the rate at which the temperature

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

332 Part II: Seven Research Strategies

decreased over time. Secondly, to assess the visual performance of the blinds, mea- surements were taken of both the interior and exterior illuminance.

Test conditions as indicated in Figure 9.16 were run on six days in August, and measurements for three performance criteria were taken: temperature difference between indoor and outdoor, energy consumption, and visual comfort. The au- thors conclude that given the conditions tested in the mock‐up offices, the auto- mated blinds system demonstrated both potential energy savings and comfort enhancement. In addition, the authors noted that the automation of the blinds in these experimental cases was based solely on changes in the outdoor conditions; however, a significant enhancement of the automated system could be achieved by modifying the algorithm to include indoor conditions.

9.5.3 Occupant Comfort from Air Movement: Using a Lab Setting , Physical Treatments, Instrumentation, and Subjective Measures

Although much environmental technology research relies on combining lab set- tings with exclusively instrumented measures of physical outcome variables, many other variations of lab setting research are possible. One such example is a study by Edward Arens et al. concerning the use of personally controlled air fans to achieve cooling and perceived comfort.14 The goal of this study was to evaluate the effec- tiveness of using fans, instead of compressor‐based air conditioning, as a means to achieve cooling comfort. In doing so, the study was conducted in an environmental chamber (i.e., lab setting) where individual subjects could be exposed to a controlled range of warm temperatures (see Figure 9.17). The environmental chamber was designed to “appear as a realistic residential or office space.”15

Type D (operated 3 times or more)

Overcast Clear

West

Overcast Clear

East

Overcast Clear

P er

ce nt

ag e

(% )

20%

40%

60%

80%

100%

South

Type C (operated twice)

Type B (operated once)

Type A (not operated)

Figure 9.15 Blind operation frequency analysis. Courtesy of Elsevier.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 333

Case Date Test Room 1 Test Room 2 Cooling Remarks

1 8/12 Automated blind: Energy‐saving mode

Manual (fully opened)

X Evaluate temperature difference

2 8/16 (see Table 1) Manual (fully closed)

X Evaluate temperature difference

3 8/19 Manual (fully opened)

O Evaluate energy consumption

4 8/20 Manual (fully closed)

O Evaluate energy consumption

5 8/18 Automated blind: Comfort mode

Manual (fully opened)

O Evaluate comfort

6 8/29 (see Table 2) Motorizeda O Evaluate comfort aOcclusion index 75%, slat angle: 90°.

Figure 9.16 Summary of six experimental cases utilized to study the impact of the automated blind on environmental performance. Redrawn from Building and Environment 44, Kim, Ji‐Hyun, Park, Young‐Joon, Yeo, Myoung‐Souk, & Kim, Kwang‐Woo. “An experimental study on the envi- ronmental performance of the automated blind in summer,” 1517–1527 (2009).With permission from Elsevier.

The 119 subjects (57 female, 62 male) were divided into two comparison groups. One group was asked to control the fan settings in a fluctuating mode; the second group used the fan’s constant mode, “in which the inherent turbulence of the airstream was at higher frequencies than in the fluctuating mode.”16 During both experimental protocols, the subjects’ time in the experimental chamber in- cluded two distinct activity segments generating two distinct metabolic rates: one which included both sitting and step‐climbing (1.2 met), and another which was entirely sedentary (1.0 met). Throughout all sessions, the subjects experienced a range of temperatures from 25ºC to 30ºC. Thus, the treatments represented a com- bination of both lab‐based controls and behavioral regimens.

The outcome measures included both instrumentation and subjective ratings of perceived comfort. The former was achieved by recording the subject’s choice of fan speed, and the latter was measured by a 7‐point scale from cold to hot indicating how the subject experienced the temperature of the environment. More than 80% of the subjects in the 1.2‐met condition were able to maintain comfort up to 29ºC. As a re- sult, the researchers are able to conclude that within certain temperature zones, the use of personal air fans can serve as an effective alternative to mechanical air conditioning.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

334 Part II: Seven Research Strategies

Figure 9.17 Temperature range versus fan speed level. Arens, 1998. Courtesy of Prof. Edward Arens.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 335

Box 9.2

Experiment: Energy Conservation in Housing

Malcolm Bell and Robert Lowe sought to test the impact of various energy saving techniques in housing administered by the Housing Authority of York, UKa (see Figure 9.18). As such, it therefore represents a field setting experiment.

As part of a larger three‐stage program in energy conservation monitor- ing, the authors report on a 30‐house scheme in which the impact of en- ergy saving improvements were measured against a “control group of dwellings in the same modernization scheme but with no additional

Figure 9.18 Typical house type in Malcolm Bell and Robert Lowe’s energy‐ efficient modernization study. Reprinted from Energy and Buildings 32 (2000), with permission from Elsevier Science.

a Malcolm Bell and Robert Lowe, “Energy Efficient Modernization of Housing: A UK Case Study,” Energy and Buildings 32 (2000): 267–280.

(Continued )

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

336 Part II: Seven Research Strategies

9.5.4 Experimental Monitoring of Thermal Comfort and Simulation of Energy Usage: Using a Purpose‐Built Testing Prototype, Physical Treatments, Instrumented Measures, and Numerical Simulations

An increasingly common strategy in experimental research is to augment it, either it- eratively or in distinct phases, with simulation modeling. Such is the case in a study of solar walls in residential buildings conducted by Stazi, Mastrucci, and di Perna, and briefly described earlier in Chapter 3.17 The goal of the study was to test whether the use of a Trombe wall design (a solar wall with vents at the top and bottom for ventila- tion) combined with a shading device would result in improved thermal comfort and energy savings over a standard nonventilated solar wall. In particular, the authors aimed to develop potential modifications to solar wall and/or Trombe wall designs suitable for the Mediterranean climate, where the use of solar walls is advantageous in winter months but prone in summer months to increased cooling requirements and overheating. Three operating conditions (or treatments) were tested: (1) a nonventi- lated solar wall; (2) a Trombe wall in winter mode with air thermo‐circulation; and (3) a Trombe wall in summer mode with cross‐ventilation. (See Figure 3.1 for treat- ment of the solar wall and the Trombe wall in winter and summer conditions.)

energy efficiency works.”b The 21 houses in the experimental group were modernized with a combination of clearly specified physical treatments: insulation, draft‐proofing of doors and windows, central heating with gas condensing boiler, and a gas fire as a secondary heat source. The 11 houses in the control group, with no additional energy efficiency works, were well matched with the experimental houses in terms of the initial energy char- acteristics. As a consequence, any consistent differences in energy con- sumption could then be attributed to the experimental treatment.

Monitoring measures included internal temperatures and gross energy con- sumption for the entire period, both of which are based on instrumentation. Although the difference of 5,536 kWh between the experimental and control groups is statistically significant at the .03 level, the measured savings are about half of what was predicted by energy modeling. Further investigation, including interviews with residents, indicated that some residents used the secondary heat source, the gas fire, for so many hours on a daily basis that the energy efficiency of the gas boiler was compromised. In this regard, the mon- itoring of energy efficient modifications in a field, or real‐world, housing set- ting, provided important insights about the limits of conservation hardware, when it is not accompanied by changes in human behavior.

b Ibid., 272.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 337

These treatments were built into a prototype south‐facing residential building, consisting of nine apartment units, in central Italy. The several treatment conditions were monitored for several years and over different seasons; measurements were taken of the thermal behavior of the solar walls, indoor thermal comfort conditions, and energy consumption (see Figures 9.19 and 9.20).

Figure 9.19 View of the building. Courtesy of Elsevier.

Figure 9.20 Exterior view of the Trombe wall. Courtesy of Elsevier.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

338 Part II: Seven Research Strategies

Once the researchers had collected an extensive set of data from the case study experiments, numerical simulations were performed using an existing software pro- gram using an algorithm already validated for Trombe walls. Taking other modifica- tions and calculations into account, the authors then compared the values obtained from the simulation model with the experimental data “in order to verify the reliability of the simulation tools in reproducing real situations. Once the model had been calibrated, it was possible to generalize the results running the calculation for the whole year.”18

As a result of the combination of the case study experiments and simulation modeling, the authors then returned to the experimental mode with the goal of testing modifications of the Trombe wall design for summer conditions, including shading solar walls with overhangs, use of opaque shutters, activating the cross- ventilation of the Trombe wall, and improving natural ventilation. Although the monitoring of this treatment condition occurred during a period of extreme heat, the modified Trombe wall design nevertheless maintained operative temperature within the comfort range for the entire period. The authors conclude that the tested Trombe wall configuration in Mediterranean climate conditions can be an efficient system for both energy savings and thermal comfort.

See Figure 9.21 for a summary of the tactics used in the studies discussed in this chapter section.

Study Setting Treatment Outcome Measures

1. Blind operation systems (Kim et al.)

Field Blind systems manual motorized automated

Instrumented measures Temperatures indoors and outdoors Illuminance indoors and outdoors

2. Personally controlled air fans (Arens et al.)

Lab Physical treatments temperature activity level fan type

Instrumented measures and behavioral response fan speed choice perceived comfort

3. Energy use in housing (Bell and Lowe)

Field Environmental modifications gas boiler insulation draft proofing secondary heat

Instrumented measures internal temperature gross energy consumption

Figure 9.21 Summary of tactics in cited studies.

(Continued )

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 339

Box 9.3

Experiment: A Case Study of Facade Treatments

Stamps’s study of the effects of design features on people’s perceptions of architectural mass is based on an experimental design, and in that regard it is unusual.a Many, probably most, studies of nonarchitects’ or users’ responses to building facades have employed a correlational design involving assessments of actual buildings. Stamps’s research design in- volved the use of computer‐generated sketches of building facades that systematically varied the architectural treatment of each facade. Four key variables, based on a previous pilot study, were identified as having a po- tential impact on respondent assessments: visual area, partitioning of fa- cade elements, fenestration, and articulation (e.g., bays or notches) of the facade plane. Using an experimental design protocol that enables multiple treatments to be combined across a limited number of stimuli (i.e., the facades), Stamps generated the nine facade examples represented in Fig- ure 9.22. To achieve a random selection of respondents, Stamps relied on a survey research firm to recruit a random selection of respondents from the local area. Each respondent was asked to view paired sets of the facades and indicate which facade appeared to be more massive.

a Arthur Stamps, “Measures of Architectural Mass: From Vague Impressions to Definite Design Features,” Environment and Planning B: Planning and Design (1998): 825–836.

Figure 9.21 (Continued)

Study Setting Treatment Outcome Measures

4. Solar walls (Stazi et al.)

Field/ Prototype

Solar/Trombe modifications glazing ventilation shading

Instrumented measures temperature readings energy use/savings simulation modeling

5. Perceptions of facades (Stamps)

Lab Treatment of facade features visual area façade elements fenestration articulation

Perception of architectural mass

(Continued )

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

340 Part II: Seven Research Strategies

The results of Stamps’s study indicate that the most influential variable by far was visual area, which can be modified in situ by setback require- ments. Fenestration treatments had a much more modest impact on per- ception of mass, and both articulation of the facade plane and the partitioning of facade elements had minimal impact.

9 . 6 t h E c o m p l E m E n ta Ry n at u R E o f E x p E R I m E n ta l c u lt u R E S I n d E S I g n a n d R E S E a R c h

In Chapter 2, we argued that the relationship between design and research is far more nuanced and multifaceted than a black‐and‐white statement of equivalence or difference. Here we want to address the application of the term experimental to de- sign studio and practice endeavors in recent discourse with how we have discussed the notion of experimental research design in this chapter.

Figure 9.22 Computer‐generated facade stimuli from Arthur Stamps. Courtesy of Pion Limited, London.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 341

It should already be clear by this point that for the purposes of peer‐reviewed scholarly research, the use of the term experiment (including quasi‐experiment) is restricted to the relatively precise characteristics already discussed in sections 9.2 and 9.3. However, some proponents of the role of research in design studios and practice have pointed out the utility of understanding the comparably experimental culture of research and design.19 This is true in a very general sense, and consistent with our discussion in Chapter 2 concerning the equivalence of logics in use, and so on. Still, many instances of the research enterprise in studio and practice contexts are experimental in a more generic or metaphorical sense. While many valuable insights are generated through inductive exploration (i.e., the logic of discovery), often the iterative sequence of testing and documentation through deductive logic is missing or less developed.

In this context, a JAE article by Stephen Kieran (of Kieran Timberlake) estab- lishes a legitimate claim to design experimentation that meets the claim of experi- mental research.20 He describes in some detail the increasing emphasis on research as the core of his practice. He then discusses how over recent years the firm has “introduced the process of monitoring what we have planned and built.”21 In this endeavor, their designs for technically innovative curtain walls have been built as prototypes at the University of Pennsylvania’s research and teaching facility for the School of Engineering. And, in collaboration with Professor Ali Malkawi, a system of monitoring devices has been employed.

Finally, Kieran’s description of the design for a residence in Maryland details a thoughtful process of integrating natural ventilation, adjustable solar shading, and a bifolding hanger door as a thermal pocket over the glazing layer. As he ex- plains it, the monitoring data from this design proposal suggested further lines of development, including the introduction of thermal mass into the cavity to store heat for evening hours. He concludes by suggesting an experiment to draw heated air out of the top of the cavity, thus “using the facade as a type of Trombe wall”22 (see Figure 9.23).

In sum, what Kieran describes is not too much different than the Trombe wall study by Stazi et al. Both “experiments” involved an iterative process of tinkering with and testing out empirically different modifications to a wall sys- tem; both also extensively used monitoring devices to collect data and evaluate the effectiveness of the wall treatments. In some contrast, Stazi and his col- leagues, as established researchers, began with an intention to test out solar/ Trombe wall systems, and then developed a broader analysis through numerical simulations to thermal comfort and energy savings that might be generalized throughout the seasons and in similar climate conditions. Moreover, Stazi et al. systematically tested multiple treatment conditions, whereas Kieran Timberlake

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

342 Part II: Seven Research Strategies

Figure 9.23 Varying positions of the accordion‐style glass doors of the Loblolly House by Kieran- Timberlake Associates LLP. © Kieran Timberlake.

(a)

(b)

(c)

aimed to develop a design for a particular client that gradually evolved into con- siderations for the design of a Trombe wall system. Despite these differences, the distance between these examples of experimental design and experimental research is close indeed.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 343

Box 9.4

Applications of Experimental Research in Practice and Education

In the vast majority of practice and educational settings, design decision making in the most technical areas of architecture typically relies on a foundation of extensive experimental research. This is certainly the case with issues such as building skin design and materials development.

In a notable collaboration between a professional firm (Perkins + Will) and an architectural design studio (University of Cincinnati), the studio employed computational design techniques, analytical tools, and digital fabrication to achieve performance goals for the design of a building fa- cade retrofit.a Prior experimental research is the basis for not only the various performance criteria but also the development of various simula- tion and analytical tools.

The particular building site was an actual project from the Perkins + Will office, and office personnel participated as resources to the studio. The task of the studio was to “reskin” the former cold storage facility near downtown Chicago that was being converted to a commercial office build- ing. Figure 9.24 outlines the various stages and techniques that served as the framework for the studio work.

In this studio context, the integration of simulation techniques for parametric design and fabrication led to a variety of solution types. These included: (1) an adaptable building skin responsive to daily or sea- sonal changes; (2) a double skin with a kinetic shading system; (3) an external shading system; and (4) a tectonic building form. The develop- ment of this range of solution types demonstrates the integrative poten- tial of experimentally developed analytical, visualization, and fabrication tools. Not only can such design processes lead to a much higher level of building performance criteria, but they can also offer a venue for effec- tive collaboration and knowledge transfer between professional and academic settings.

a A. Aksamija, T. Snapp, M. Hodge, and M. Tang, “Re‐skinning: Performance‐ Based Design and Fabrication of Building Facade Components: Design Computing, Analytics and Prototyping,” Perkins + Will Research Journal 4(1) (2012): 15–28.

(Continued )

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

344 Part II: Seven Research Strategies

Figure 9.24 Building skin digital design and fabrication process. Courtesy of Dr. Ajla Aksamija, Perkins + Will Tech Lab.

9 . 7 c o n c lu S I o n S : S t R E n g t h S a n d w E a k n E S S E S

Of all the research design strategies commonly employed by researchers, the ex- periment is, in all likelihood, the most controversial. On the one hand, experimen- tal design is considered by postpositivist researchers to represent the highest standard of research.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 345

The best method—indeed the only fully compelling method—of establishing causation is to conduct a carefully designed experiment in which the effects of possible lurking variables are controlled. To experiment means to actively change {x} and observe the response {y}.23

This quotation is revealing because it so crisply encapsulates the essence of what is seen as experimentalism’s major strength: the most credible device for de- termination of causality, observed through a sequence of a specified treatment and its outcome.

On the other hand, the experimental design strategy is widely criticized, for a variety of reasons, by researchers representing both the intersubjective and subjectiv- ist paradigms. Feminist scholars, in particular, have articulated a number of major concerns.24 Most center around one of the following issues: (1) efficacy and accuracy, (2) misapplication of experimental procedure, or (3) ethical issues (see Figure 9.25).

Efficacy and Accuracy. The essence of the argument concerning the efficacy of experimental method is that most real‐life settings or sociocultural phenomena are far too complex to be reduced to a small set of treatment and outcome variables. Moreover, the laboratory setting is seen not as a “neutral social environment” but rather as a “specific social environment that exerts its own effects.”25 Critics argue that instead, settings and phenomena must be studied in complex and messy natu- ral settings. As Michelle Fine and Susan Gordon put it:

If you really want to know either of us, don’t put us in the laboratory, or hand us a survey, or even interview us separately alone in our homes. Watch me (MF) with women friends, my son, his father, my niece or my mother and you will see what feels most authentic to me. These very moments, which construct who I am when I am most me, remain remote from psychological studies of individu- als or even groups.26

Strengths Weaknesses

Potential for establishing causality Reduction of complex reality to identify “causal” or independent variables

Potential for generalizing results to other settings and phenomena

Misuse by overgeneralization to different ethnic, gender populations

Ability to control all aspects of experimental design enables attribution of causality

Overemphasis on control yields ethical prob- lems, dehumanization

Figure 9.25 Strengths and weaknesses of experimental research.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

346 Part II: Seven Research Strategies

Misapplication. Critics who cite the misuse or misapplication of experimental protocol frequently focus on the way biases or oversights can inadvertently influ- ence the results of such research. This critique is articulated quite clearly by the well‐known feminist researcher, Shulamit Reinharz. She argues:

[P]ublication practices and experimental design highlight differences and hide similarities between groups. Overgeneralization that masks differences in race, age, education, and other factors is clearly inappropriate and possibly danger- ous. Too often studies done on white populations are generalized to all groups, just as studies done on men are generalized to all people, thereby producing distorted results.27

However, a number of feminists and others affiliated with various schools of thought (including transformative, phenomenological, and others) have proposed a more nuanced and pragmatic perspective whereby the experimental research design is actually employed to reveal gendered and racist practices. Indeed, Devlin’s study of gender discrimination in hiring is one such example. Implicit in this exploitation of the experimental method is the belief that, given the power and respect it commands in so many quarters, feminist and other emancipatory research will only be seen as credible if it is conveyed in the form of the influential experimental strategy.

Ethical Issues. The core of the ethical concerns that have been raised about experimental design is that the manipulative control exercised by the researcher puts research “subjects” in an essentially powerless position. Treatments are applied to subjects without their consultation. Or alternatively, a potentially advantageous treatment (i.e., better lighting or gender-neutral pedagogy) might be withheld from the “control” group of subjects. Indeed, even the objectified language of “sub- jects”—as opposed to people or individuals—tends to dehumanize the people who participate in such studies.

In the end, it would seem that the selection of the experimental design offers the potential to confer both profound benefits and potentially serious weaknesses. The former includes the attribution of causality, as well as prestige and credibility in some circles. Indeed, in some areas of research—notably in the more technical areas—the premises of experimental work remain unchallenged, although now fre- quently complemented by computer simulation models.

However, its shortcomings, as identified earlier, include: (1) inappropriate simplification of complex research issues; (2) potential for misapplication; and (3) the potential for serious ethical problems. Yet even feminist critic Shulamit Reinharz argues that despite its apparent weaknesses, researchers may do well to exploit its strengths:

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

Experimental and Quasi‐Experimental Research 347

Combining the strengths of the experimental method with the strengths of other methods is probably the best way to avoid its weaknesses while utilizing its power. Similarly, combining the strength of research with the power of other forms of persuasion is probably a useful approach for creating change.28

The notion of combining distinctly different research strategies is one that has become increasingly popular among researchers in diverse fields and disciplines. It is a topic to which we will return in Chapter 12.

n o t E S

1. Baruch Givoni, Michael Gulich, Carlos Gomez, and Antulio Gomez, “Radiant Cooling by Metal Roofs in Developing Countries,” Proceedings of the 21st National Passive Solar Conference (Boulder, CO: American Solar Energy Society, April 1996), 83–87.

2. Ann S. Devlin, “Architects: Gender‐Role and Hiring Decisions,” Psychological Report 81 (1997): 667–676.

3. Ibid., 670. 4. L. S. Fidell, “Empirical Verification of Sex Discrimination in Hiring Practices in

Psychology,” American Psychologist 25 (1970): 1094–1097. 5. Devlin, 667. 6. Ibid., 674. 7. William R. Standish, Thomas D. Cook, and Donald T. Campbell, Experimental and

Quasi‐Experimental Designs for Generalized Causal Inference (Boston: Houghton Mifflin, 2002).

8. Ibid. 9. Janice Barnes, Kaninika Bhatnagar, Fernando Lara, Satoshi Nakamura, Pirasri

Povatong, Tien Chien Tsao, and Victoria Turkel, “Results of a Quasi‐Experimental Treatment in the Architecture Gallery.” Unpublished student paper, University of Michigan, Ann Arbor, MI, 1997.

10. Despite these conclusions, a major obstacle to changing the furniture arrangement on a permanent basis was that the intended lack of visibility meant the security of the ex- hibits could not be monitored from the offices across the hall. As a result of this quasi‐ experiment, then, the student group made a policy recommendation to the administration that a separate student/faculty lounge area should be provided. Several years later, a new lounge was built.

11. Standish et al.; John Creswell, Research Design: Qualitative, Quantitative and Mixed Methods Approaches (Thousand Oaks, CA: Sage, 2009); D. Mertens, Research Methods in Education and Psychology, 3rd ed. (Thousand Oaks, CA: Sage, 2010).

12. Ji‐Hyun Kim, Young‐Joon Park, Myooung‐Souk Yeo, and Kwang‐Woo Kim, “An Experimental Study on the Environmental Performance of the Automated Blind in Summer,” Building and Environment (44) (2009): 1517–1527.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

348 Part II: Seven Research Strategies

13. Ibid., 1521. 14. Edward Arens, Tengfang Xu, Katsuhiro Miura, Zhang Hui, Marc Fountain, and Fred

Bauman, “A Study of Occupant Cooling by Personally Controlled Air Movement,” Energy and Building 27 (1998): 45–59.

15. Ibid., 46–47. 16. Ibid., 47. 17. Francesca Stazi, Alessio Mastrucci, and Constanzo di Perna, “The Behaviour of Solar

Walls in Residential Buildings with Different Insulation Levels: An Experimental and Numerical Study,” Energy and Buildings 47 (2012): 217–229.

18. Ibid., 220. 19. David Salomon, “Experimental Cultures: On the ‘End’ of the Design Thesis and the

Rise of the Research Studio,” Journal of Architectural Education 65(1) (2011): 33–44. 20. Stephen Kieran, “Research in Design: Planning Doing Monitoring Learning,” Journal

of Architectural Education 61(1) (2007): 27–31. 21. Ibid., 29. 22. Ibid., 31. 23. D. Moore and D. McCabe, Introduction to the Practice of Statistics (New York: Freeman,

1993). 24. P. Lather, “This IS your Father’s Paradigm,” Qualitative Inquiry 10(1) (2004): 15–34;

E. A. St. Pierre, “Scientifically Based Research in Education: Epistemology and Ethics,” Adult Education Quarterly 56(4): 239–266.

25. Shulamit Reinharz, Feminist Methods in Social Research (New York: Oxford University Press, 1992): 100.

26. Michelle Fine and Susan M. Gordon, “Feminist Transformations of/Despite Psychology,” in M. Crawford and M. Gentry (eds.), Gender and Thought: Psychological Perspectives (New York: Springer‐Verlag, 1989), 106.

27. Reinharz, 107. 28. Ibid., 108.

C op

yr ig

ht ©

2 01

3. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed