Research again

30 CONCEPTS FUNDAMENTAL TO PROPOSAL WRITING

WORKSHEET 2.~

Proposal Function Review

What Are Expectafillll'for My Proposal'

For the type(s) of study you expect to propose for your dissertation, describe the extent to which your proposal will be expected to serve each of the functions identified. It may be hel~ful to consult faculty advisors, more senior dissertation students, and prior local disser­ tation proposals.

To What Extent Will My Proposal Need To ... ?

Provide an argumen t for justifying my study?

Include a work plan?

Provide evidence of my ability to do the study?

Serve as a request for commitment to work with me?

Serve as a contract for how my study is to be conducted?

Be used later to judge the quality of my dissertation work?

Serve as a partial draft of my final dissertation report? I

1"

CHAPTER 3

The Proposal as a Chain of Reasoning

CHAPTER CONTENTS

The Proposal as a Chain of Reasoning 31 The Chain of Reasoning in Studies with Generalizable Findings 32

The Links in the Chain 32 Details of the Links from Procedure to Data 34

Four Useful Characteristics of the Chain Analogy 36 Relation of the Chain Analogy to the Proposal 38 The Chain of Reasoning in Development and Problem-Solving Studies 39 Worksheet 3.1: Chain of Reasoning Analysis 42

THE PROPOSAL AS A CHAIN OF REASONING

As noted in the previous chapter, not all studies produce findings involving a generalization; some simply describe, leaving the range of application to be supplied by the reader.' But, as we will describe in this section, those studies that do seek to generalize, whether the generality is found by prespecified ~r emergent means, present their findings as a chain of reasollirzg. If the study is pre­ specified, the initial links of the chain will be developed in the proposal. If the study is emergent, then building as much of the chain as is feasible at the study'S outset provides the strongest proposal.

Let us carry this point a bit further. The end goal of research that produces or supports a generalization is the development of a carefully constructed chain of reasoning. Both the write-up of the proposal and the dissertation itself follow a logical, deductive sequence of presentation. The process of doing research, especially in the case of prespecified studies, often follows a similar sequence.

1 Some qlldlit<ltivl.-~ research, such <15 \o\i'hyte's Street Corner Society (1Q93), results more in d('scrip­

lion of situations than in generalizdtions about them. Though the proposal for such a study may be writ­ ten deductively, the di5sertation is not.

31

32 CONCEPTS FUNDAMENTAL TO PROPOSAL WRITING

But it also may not, and in that instance the process is reconstructed as a logical sequence in the write-up. Even though the logic involved in developing generalizations from emergent studies is inductive rather than deductive (as is most apparent in exploratory research), research reports presenting findings sup­ portive of a generalization do so deductively as a chain of reasoning.

The basic logic underlying the chain of reasoning not only applies to stud­ ies seeking generalizable findings, but may also be interpreted so as to apply to the developmental and problem-solving efforts described in the previous chapter as local application studies. For example, the production of a new measuring instrument or curriculum, the solving of a local problem, and con­ ducting an evaluation all follow a series of steps comparable to those involved in studies seeking generalizations. Let us first examine how the chain of rea­ soning applies in the latter case, and we will then take up the former.

THE CHAIN OF REASONING IN STUDIES

WITH GENERALIZABLE FINDINGS

Our most beneficial research studies provide results that are generalizable be­ yond the context in which they were carried out. Figure 3.1 represents the logic underlying the write-up of such studies as a chain of reasoning analogous to a metal chain. Each of the links in the chain successively develops a logical path from the onset of the study to the presentation of findings. This is described in the discussion of each of the links in the following section. It also shows the value of the metal chain analogy. The chain of reasoning logic also underlies the research proposal.

The Links in the Chain

In the presentation of new findings, as well as in beginning to do a study, one usually links back to what was already known about the phenomenon in terms of published work or experience. Thus, the first link in the chain is Links to Previous Research It shows how the idea for this study arose out of this background.

How much background on the intended study already exists determines the nature of the next link, Explanation, Rationale, Theory, or Point of View. With little prior knowledge or experience, it leads to a rationale for doing the study and perhaps a point of view about what to study. With more background, one may have an explanation of a phenomenon; with still more, perhaps a theory about a process.

The specificity of the prior link determines the Questions, Hypotheses, Mod­ els link. With little background, one may pose a question describing the initial focus of attention for the study. With an explanation, one may be able to make a prediction that is presented as a hypothesis. If there is extensive prior research so the underlying causative variables may be fathomed, this leads to a model in-

THE CHAIN OF REASONING 33

~~

Expl.anation, rationale, theory or point of view

Links to previou... research

Questions, hypotheses, models

It'U.... -:.1 a

Figure 3.1. The chain of reasoning in the presentation of findings (adapted from Krathwohl,

1998/2004 with suggestions from John T. Behrens).

dicating the interrelation of the variables in a process. The study would then seek data to test that model.

The question, hypothesis, or model forms the basis for the Prespecijied and emergent designs of the study, the next link. With little prior knowledge, an emergent design is usually indicated; you don't know where the "handles" are on the phenomena. The more that is known about the phenomena, the more certainty you can preplan the study. Whereas beginning with a question most likely leads to an emergent study, starting with a hypothesis or model leads to a prespecified study. Some studies begin in an emergent mode, and as more is learned a planned study becomes possible.

Just as the question, hypothesis, or model translates into the choice of emergent or prespecified design, the la tter choice determines the Procedure. The procedure spells out the who, where, what, when, and how of the study. In an emergent study it will tell who will be studied, what will be the focus, when and how it will be done, etc. In a prespecified study, the nature of a treatment or

34 CONCEPTS FUNDAMENTAL TO PROPOSAL WRITI"I/G

intervention, the measures of effect, the pattern of treatment and measurement, etc., are decided upon and the details specified of when, how, where, etc., the observations, interviews, measures, treatments, etc., will take place. The link tying Procedure to Data is detailed below.

Carrying out the deSign leads to gathering Data, the next link. For example, the scores on measures, the observation notes, the recordings or transcripts of interviews, and the results of surveys are the data.

I In both emergent and prespecified studies, the data are usually volumi­ nous, more than can be grasped by just looking at them. This requires data re­ duction using the methods of Statistical and Narrative Analyses, the next link. Narrative analysis usually involves finding the significant themes in the obser­ vation notes, interviews or documents; statistical analysis, descriptive sum­ mary statistics, relationship and pattern-seeking statistics and displays, and singling out findings unlikely to have resulted from chance.

The results of these analyses are summarized in a final section of the report, the Conclusion. These conclusions are read by other researchers and lead in turn to continuing the chain of reasoning as these findings are built on by new re­ search. This is indicated in the last two links of the chain, which, although not part of the study report, show each study as part of the continuing research process.

Details of the Links from Procedure to Data

Figure 3.2 provides a more detailed look at the link between Procedure and Data in the chain. It reveals that instead of a single link, it has been split in order to describe the who, where, what, how, and when of the procedure. 2

Figure 3.2. Detail of the connections between the Procedure and Data links.

Figure 3.3 spreads these links out to bridge across the Procedure and Vata links and labels them to indicate the who, where, what, how, and when of pro­ cedure that must be described.

More specifically, these six links are:

2. Rudyard Kipling's little ditty usefully describes the elements to describe: "[ keep six honest Serv­

ing men, / They taught me all J knew, / Their names are What and Why and When / And How.nd

Where and Who." We covered Why in the Explanation, rationale link. The others are covered in Procedure.

THE CHAIN OF REASONING 35

1. Who, the Participants-these are all the persons selected for the study or present in the situation being observed.

2. Where, the Situation-this is the situation and context in which the ex­ periment is carried out, that in which observation is done, that inter­ viewing takes place, etc.

3. What, the Focus(es) ofAction-that is, for experimental research, the inde­ pendent variable, treatment or experimental variable(s) (cause), the de­ pendent variable (effect), and any control variables (e.g., measures of ability where one wants to rule it out as an alternative cause). For quali­ tative and nonexperimental research, it is those processes and activities that are the focus of attention.

Figure 3.3. The connections in the chain of reasoning between the Procedure and Data links.

4. Also a What, the Records from (3) above-these are the data resulting from observations or measures, the field notes and the answers to tests, questionnaires, etc. The latter are scored and interpreted at the next stage.

5. How, the Comparison or Contrast-in experimental research, that which forms the basis for sensing that the treatment or experimental variable had some effect, or, in nonexperimental research, how things changed as the process or activity continued.

6. And When, the Time Schedule-when things are done, such as what ob­ servations are made, when, where, and of whom, and, if there are meas­ ures and treatments, how, where, when, and to whom they were administered.

For example, consider Rowe's (1974) hypothesis. She hypothesized that after posing a question to the class, increa~ing the amount of time the teacher i·typically waits before calling on a student would improve the nature of class­ room discourse. She found a normal "wait-time" of one second on average could be increased to three to five by training. The six rings translated the above general hypothesis in these ways:

• The Participants were the teachers and students in the classrooms where this effect was demonstrated.

36 CONCEPTS FUNDAMENTAL TO PROPOSAL WRITING

• The Situation and context were those found in the classrooms. In this in­ stance, as in many, the choice of the "participants and/ or informants" de­ termined the "situation."

• The Focuses of Action were the treatment, the teacher's increase in "wait­ time" (cause), and the change in the students' responses to the treatment (effect). To attain control for variations in what was meant by "delayed wait-time" that might result from embarrassment or discomfort, training of the teachers ensured the "treatment" was administered uniformly and as intended.

• Records included measures of effect such as recordings of classroom dis­ course to determine who talked and what kind of teacher-pupil inter­ change took place. There were also pre- and posttraining measures of the teacher's wait-time to show that it actually increased.

• The Comparison and Contrast involved contrasting measures of both wait­ time and classroom discourse prior to teacher training with those after wait-time training.

• Finally, Time Schedule involved a procedural plan indicating when and where the training would take place and of whom, and when, and of what, observations would be made.

Were this Rowe's dissertation, her proposal would have provided detail on the links of the chain of reasoning model from the previous literature at the top, down through the six rings of the "procedure" links in Figures 3.1 and 3.3. In addition, it would have included a general description of the data that would be gathered and the methods of analysis of the data.

Rowe's data showed that higher-level thinking appeared in the answers following longer wait-times as well as other positive changes. Note that in fig­ ure 3.1 and the next figure, 3.4, Rowe's findings, in tum, link to the beginning of subsequent studies when researchers used her findings to build their studies' chains of reasoning. (To determine the extent to which this occurred, one would look up "Rowe, Mary Budd," in the Social Science Citation Index.)

The complete chain of reasoning with the labels attached to the connections between Procedure and Data is shown in Figure 3.4. This figure may be useful for you to refer to as you read the rest of this book.

FOUR USEFUL CHARACTERISTICS OF THE CHAIN ANALOGY

The chain analogy is useful because many characteristics of a metal chain carry over to research chains of reasoning (Krathwohl, 1998/2004). For example, it is an old truism that a metal chain is only as strong as its weakest link. Similarly, a research proposal's logical chain ofreasoning is only as strong as its weakest link. If one of the links in the chain is weak-for example, if training is omitted from the

THE CHAIN OF REASONING 37

IA\ IA\ Links 10 previous research

Explanation, rationale, theory or point of view,­" ~ &I Questions, hypotheses, model::.

# 's=' , Prespecified and emergent designs

Data

'Procedure

:f~:S Record·s· ~~~~~~ Tinu:

Figure 3.4. The complete chain of reasoning with all the labels (adapted from Krathwohl,

1998i2004 with suggestions from John T. Behrens).

proposal so the teachers do not increase their wait-time-then you can hardly attribute any change to the treatment. Like a metal chain, the research argu­

ment is only as strong as the weakest part of it. A second feature of the analogy, and a corollary of the first, is that each link

in the chain should have the same strength. It would make little sense to have one link in a metal chain as thick as that for a ship's anchor and others as thin as sewing thread. Similarly, in planning a project, for the most efficient use of your resources, each of the links should be the same strength as the ot/lers. Why spend re­ sources refining measures of the effect of a treatment to great sensitivity when resources to ensure that the treatment itself is administered as it should be are not allocated? You should allocate resources to the various links in the chain of reasoning so that each level in the chain can appropriately support the

argument. A third aspect is that just as a chain picks up the load at the beginning and

successively transfers it to each link, thereby determining the nature of each

38 CONCEPTS FUNDAMENTAL TO PROPOSAL WRITING

successive link in terms of the load it has to carry, the same occurs in the chain of reasoning. Each link in the chain determines the nature of the next link. This as­ pect became apparent as the chain was described: past research leads to the present explanation, that explanation to a hypothesis, question, or model that determines the choice of design that is translated into procedure, and so on.

A final feature of a metal chain is that at any point in the chain where sev­ eral horizontal links across the chain's breadth serve jointly to connect the links above and below them, each of the horizontal links shares the load. In the re­ search chain of reasoning, this occurs as shown in Figures 3.2 and 3.3 between Procedure and Data. In a metal chain, where links share the load, one of them may be made stronger in order to compensate for weakness in another in order to carry the load from the levels above to the levels below them. In the same way, one of the links connecting Procedure and Data may be strengthened to compen­ sate for another facet that is weak. For instance, assume the "wait-time" effect, even with training, is so small it is hard to notice the change-a thin, weak" comparison and contrast" link. You may compensate by strengthening any or all of several of the other design links. For example, you could strengthen the Participants link by both increasing the size of the sample and using especially bright students who are likely to be particularly responsive to the treatment. You could strengthen the Records link by using tests or observations especially designed to catch the small changes that are expected to occur. Thus, various design trade-offs can be made to achieve the strongest overall chain, each of these horizontal links compensating for one another.

RELATION OF THE CHAIN ANALOGY TO THE PROPOSAL

As noted in the Rowe example, the report of the study follows the chain of rea­ soning, and to the extent possible, the proposal should as well. Insofar as the nature of the study can be anticipated before beginning the actual dissertation data collection, the proposal encompasses all of the upper part of Figure 3.4 through the six horizontal rings of the study's procedure. To the extent possible in anticipation of what is expected to occur, it usually also describes the data that will be collected and the process of analysis to be used. The relations of the successive links described above and shown graphically in Figure 3.4 should be reflected in the preparation of the proposal. The problem statement should be built so that the project's hypotheses, questions, or models flow logically from it. The statement of objectives and method of attack should build upon and move beyond the review of past research, showing how this study will add to prior accomplishments, and remedy past failures. These, in tum, will suggest the population and sample and the rest of the research design. The kind of data gathered will determine what analysis, statistical or narrative, is appropriate.

All research studies presenting the case for a generalization are logical

THE CHAIN OF REASONING 39

chains of reasoning. A strong proposal intended to demonstrate or validate such a generalization reflects this chain by the plan of its structure, by its inter­ nal logical consistency, and by the appropriate development of each section. Each section reflects the previous material and carries it a step further in a con­ sistent way. Study details are not overlooked: objectives are not slighted, plans for data collection are not included in the analysis section of the plan, and the like. Resources are properly allocated to strengthen weak aspects, and design trade-offs are appropriately made.

The idea of a proposal as a chain of reasoning underlies the advice given in the next two parts of the book. Part 2 provides general advice about how to de­ velop the core proposal components of the problem statement and method statement. Part 3 deals with adapting the core proposal to fit particular types of inquiry approaches. So both parts should be consulted, part 2 in its entirety and such chapters of part 3 as seem relevant to what you plan to do.

THE CHAIN OF REASONING IN DEVELOPMENT AND PROBLEM-SOLVING STUDIES

Earlier, we noted that the chain of reasoning analogy usefully applies as well to local application studies, whether development studies such as those creating a product (e.g., an instrument or curriculum) or problem-solving studies such as performing an evaluation or conducting a cost analysis. When such studies are described in a formal report, they also follow a chain of reasoning sequence, and the chain analogy, together with its applicable properties (strong as its weakest link, etc.), also applies. The interpretation of each of the steps in the conceptualized chain, however, must be adjusted to fit the context-develop­ ment or problem solving-of the study. Table 3.1 suggests for each of the links in the chain how it may be interpreted for development and problem-solving studies. In some instances, entries are examples of what would represent that link in a particular kind of study.

Most of the entries in the table are self-explanatory, but a word might be said about the data and analysis steps and formative and summative evalua­ tion. What occurs at these steps depends on how quickly you complete the de­ velopment or problem-solving process, succeeding on the first attempt or requiring several trials. If a prototype or trial, at the data link, you may seek di­ agnostic information intended to help learn how the product, intervention, or process can be improved. Since you are seeking diagnostic capability, the in­ struments or measures used may be different from those used once past this stage. And if the data indicate improvement is needed or possible, you may then cycle back through the earlier stages. How far back depends on whether you must start from scratch or are on the right track and satisfaction lies in ad­ justment. If the evaluation of the prototype or trial yields data that look as though only a bit of tweaking is needed, or if it has been developed as far as you

!!'

40 CONCEPTS FUNDAMENTAL TO PROPOSAL WRITING

TABLE 3.1

The Chain of Reasoning in Development and Problem Solving Studies

Interpreted for Links in the Chain Development Studies

Links to previous studies Lessons learned in previous,

similar studies

Explanation, rationale, Problem sDlving ratiDnale,

theDry, or point of view development models

Questions, hypothesis, Criteria which product must meet

models

Planned and/ Dr emergent Plan for product development

design

Procedure

Participants

SiluatiDn

FDcus of action

Records

Comparison and contrast

Time schedule

Persons used in tryouts

Defined by location Df persons

used in tryouts

Variables invDlved in

develDpment Df product

Measures or instruments used to

evaluate product

Basis used to determine

improvement

Procedural steps invDlved in

develDping product and

its evaluation

Data Prototype product and fDrmative

evaluation

Or

Product and summative

evaluation

Statistical and narrative Analyses appropriate to data

~lnalysis gathered

Concllblon Description of product,

its uses, advantages,

weaknesses, dnd limitClti~m5

How Successive Links in the Chain May Be:

Interpreted for Problem Solving Studies

Analysis of strengths and weaknesses

of previous solutions or alternative

processes

Intervention strategies, diffusiDn theory,

cost-analysis models

Criteria which solution or

process must satisfy

Plan for development Df solution Dr

application of process

Defined by IDcale Df problem

Defined by lDcale Df problem

Variables involved in

solution or process

Measures or instruments used to

evaluate sDlutiDn Dr process

Basis used tD determine

improvement or success of process

Pnxedural steps invDlved in

solurion Dr in process and

cv.l1uiltiun of outcomes

Trial interventiDn or process

and formative evaluation

Or

Implementation of

intervention or process and

summative evaluation data

Analyses appropriate to data

gathered .. DescriptiDn of sDlution or

process, other pOSSible uses,

adv811tages and weilknesses

THE CHAIN OF REASONING 41

intend to take it, or you are satisfied, then you proceed to the summative evalu­ ation. It leads to the conclusion and wrap-up of the project.

A variation for problem-solVing studies should be noted. Once you have solved a problem at the local level, even though that is all you intended at the outset, you may realize that the solution or process has more general implica­ tions. This may result in cycling back to the design and procedure links of the chain and making new choices in the six aspects of procedure. This would allow you to determine how well the intervention or process works in other situations, with other persons-whether, as they say, "it has legs" and is

generalizable. Worksheet 3.1: Chain of Reasoning Analysis is provided here to give you

practice in analyzing how well a dissertation proposal builds a chain of reason­ ing. Use Worksheet 3.1 to review the chain of reasoning in chapter 11, one of the annotated proposals included in this book. Then, once you have a draft of your own proposal, come back to this worksheet to review the strength of its chain of reasoning so you can make the most convincing case for conducting your

study. ;, .,

I

,

4

WORKSHEET J.1

Chain of Reasoning Analysis

How Strong Is My Proposal's Om;" of Reasoning'

In reviewing your proposal's argument for the study proposed, first describe each element in the proposal's chain of reasoning, and then review its strengths, weaknesses, ways to cor­ rect those weaknesses, and, finally, how well it follows from prior elements and contributes to subsequent elements in building a convincing overall argument.

PART TWO

Advice Common to Most Proposals

This part gets down to the nuts and bolts of writing a proposal and making it hold together as a logically integrated chain of reasoning (the chain of reason­ ing analogy was described in the previous chapter; pick it up if you missed it). Although it gives advice that will apply to most proposals, the advice is de­ scribed in terms that may make it appear to apply primarily to prespecified proposals (the prespecified/emergent distinction is described in chapter 2; go back and pick it up too if you missed it). This is necessary in order to make it specific enough to be helpful. But wherever advice for an emergent or qualita­ tive dissertation would differ from the advice given, that is noted in this part's chapters and then is further developed in appropriate sections of part 3. Part 2 consists of three chapters that cover successive sections of the proposal, the four topmost rings of the chain of reasoning.

Chapter 4 describes how to present the problem and foreshadows the rest of the proposal. It shows how the review of literature develops and refines the problem statement as well as suggesting appropriate refinements in method. The refined problem statement leads to a question, hypothesis, or model, de­ pending on how advanced the state of knowledge is in the area being studied, or leads to a more detailed description of the phenomenon to be focused upon, in the case of emergent studies.

Chapter 5 describes how to present your research method or process in suf­ ficient detail that gatekeepers will be comfortable in approving the proposal as a basis for proceeding with the dissertation.

Chapter 6 describes what kind of additional evidence may be helpful in presenting your case and how this may be marshaled so as to be convincing to gatekeepers.

42