I need this assignment by 9:00pm tomorrow 04/04/2016
mnsdbwree26
84935915.pdf
Teaching critical questions about argumentation through the revising process: effects of strategy instruction on college students’ argumentative essays
Yi Song • Ralph P. Ferretti
Published online: 24 May 2012
� Springer Science+Business Media B.V. 2012
Abstract The effects of self-regulated strategy development revising instruction for college students that targeted the use of argumentation schemes and critical
questions were assessed in three conditions. In the first condition, students were
taught to revise their essays by asking and answering critical questions about the
argument from consequences and argument from example schemes while writing about controversial topics. In the second condition, students were taught to revise
their essays by using argumentation schemes to justify their standpoint, but did not
learn the critical questions. In the third condition, students received no instruction
about either the argumentation schemes or the critical questions. Compared to
students in the contrasting conditions, those who were taught to ask and answer
critical questions wrote essays that were of higher quality, and included more
counterarguments, alternative standpoints, and rebuttals. These findings indicate
that strategy instruction that includes critical standards for argumentation increases
college students’ sensitivity to alternative perspectives.
Keywords Argumentation schemes � Critical questions � Strategy instruction � Argumentative writing � Revision
Introduction
Communication about controversial issues presumes the capacity to consider
potentially relevant evidence, entertain alternative perspectives, and arrive at a
Y. Song (&) School of Education, University of Delaware, Newark, DE 19716, USA
e-mail: [email protected]
R. P. Ferretti
School of Education, University of Delaware, 015 C Willard Hall, Newark, DE 19716, USA
e-mail: [email protected]
123
Read Writ (2013) 26:67–90
DOI 10.1007/s11145-012-9381-8
reasonable standpoint based on an evaluation of the justificatory strategies used to
accomplish their discursive purposes (van Eemeren & Grootendorst, 2004; van
Eemeren, Grootendorst, & Henkemans, 1996; Walton, Reed, & Macagno, 2008).
Unfortunately, the evidence shows that people often ignore relevant information
that is inconsistent with their perspective, i.e., my-side bias (Perkins, Farady, &
Bushey, 1991), do not consider potential counterarguments (Kuhn, 1991), lack
standards for evaluating argumentative strategies (Ferretti, Lewis, & Andrews-
Weckerly, 2009; Nussbaum & Edwards, 2011), and fail to adapt their strategies to
the communicative circumstances (Felton & Kuhn, 2001). As a result, people’s
argumentation is often poorly developed and insensitive to alternatives perspectives.
The aforementioned limitations in thinking are clearly evident in students’
argumentative essays, which are usually shorter and underdeveloped compared to
narrative and informative writing (Applebee, Langer, Mullis, Latham, & Gentile,
1994). The NAEP Writing Report Card (Persky, Daane, & Jin, 2003) showed that
only 17 % of fourth-graders, 18 % of eighth-graders, and 31 % of twelfth-graders
wrote argumentative essays that were judged to be ‘‘skillful’’ or better. Skillful
essays generally offered a thesis and some supporting reasons and examples, but
lacked clear transitions among arguments and may not have considered alternative
perspectives. The 2007 NAEP assessment showed that only 27 % of twelfth-
graders’ argumentative essays reached this standard (The Nation’s Writing Report
Card, 2007). Unfortunately, students rarely acknowledge opposing positions,
consider the merits of different views, or attempt to systematically integrate or
rebut alternative perspectives (Ferretti, MacArthur, & Dowdy, 2000; Ferretti et al.,
2009).
Various interventions have been designed to improve students’ argumentative
writing (De La Paz & Graham, 1997a, b; Deatline-Buchman & Jitendra, 2006;
Ferretti et al., 2000; Graham & Harris, 1989; Nussbaum & Kardash, 2005; Sexton,
Harris, & Graham, 1998), including the self-regulated strategy development (SRSD)
model (Graham & Perin, 2007; Graham, Harris, & McKeown, in press). In SRSD
instruction, the teacher provides explicit support for the acquisition of specific
strategies. This support includes activating the student’s background knowledge, as
well as discussing and modeling strategy use (Graham et al., in press). SRSD
instruction positively impacts the writing of all elementary and secondary students,
including those with learning disabilities (LD) (Graham & Perin, 2007; Graham,
et al., in press). However, there is a dearth of evidence about the effectiveness of
SRSD instruction with college students. A recent meta-analysis (Graham et al., in
press) about the efficacy of SRSD writing interventions identified 116 reports, only
two of which targeted the writing of college students. Both of these studies were
excluded from the meta-analysis because of design limitations (Graham et al., in
press). Clearly, credible evidence is absent about the efficacy of SRSD instruction
for college students.
A number of argumentative writing strategies have been developed in the SRSD
tradition, including TREE (Graham & Harris, 1989; Sexton et al., 1998) and STOP
and DARE (De La Paz & Graham, 1997a, b). TREE is a mnemonic that helps
remind students to generate critical parts of a basic argumentative essay: note topic sentence, note reasons, examine reasons, and note ending. STOP reminds students
68 Y. Song, R. P. Ferretti
123
to stop, reflect, and plan before writing (Suspend judgment, Take a side, Organize
ideas, Plan more as you write), and DARE reminds students to include the four parts
of an argumentative essay (Develop a topic sentence, Add supporting ideas, Reject
arguments for the other side, End with a conclusion). Both of these strategies focus
on the planning process (Graham & Harris, 2005), and are designed to help students
include a position, reasons, and a conclusion in their writing. The STOP and DARE
strategy also encourages students to consider alternative viewpoints. These
strategies have been shown to positively impact the quality of younger students’
written arguments and increase the number of argumentative discourse elements.
However, arguments are not simply reducible to the elements of which they are
comprised (Ferretti, Andrews-Weckerly, & Lewis, 2007; Ferretti et al., 2009) because
they possess a structure that in its totality increases the acceptability of the writer’s
standpoint (van Eemeren et al., 1996). Furthermore, arguments have different aims
(Walton, 1992) and different argumentative strategies are best suited for different
discursive purposes. These strategies, which have been called argumentation schemes (van Eemeren & Grootendorst, 1992; van Eemeren, Grootendorst, Jackson, & Jacobs,
1993; van Eemeren, Grootendorst, & Henkemans, 2002; Walton, 1996; Walton et al.,
2008), are conventionalized ways of representing the relationship between what is
stated in the standpoint and its supporting justificatory structure. For example, Ferretti
et al. (2007, 2009) discussed two argumentative strategies that are often used in writing
about policy deliberations (see also Aristotle, tran. 1991; Walton et al., 2008). The
argument from consequences strategy justifies a contemplated policy based on the potential positive or negative consequences that may result from its enactment. The
argument from example strategy uses particular cases or instances to illustrate a generalized claim.
The relevance and acceptability of these argumentation schemes must be
evaluated against critical standards of evaluation (Coirer, Andriessen, & Chanquoy,
1999; van Eemeren & Grootendorst, 1992; van Eemeren et al., 1993, 2002). Critical questions provide a normative standard that can be used to evaluate the relevance of an argument’s justificatory structure (van Eemeren & Grootendorst, 1992; van
Eemeren et al., 1993, 2002; Walton, 1996; Walton et al., 2008). Consider, for
example, that the proponent of a policy might argue for its enactment because it will
yield favorable consequences, and that these consequences are further bolstered
with supporting examples. The opponent might ask a series of critical questions
(i.e., potential counterarguments) about the acceptability and relevance of these
argumentative strategies (see Fig. 1). With respect to the favorable consequences,
the opponent might ask: (1) How sure are you that the good consequences will
actually happen?; (2) Do you have evidence (facts, data, support) that these
consequences probably will happen?; (3) Are there potentially bad consequences
that might happen if we implement the policy? In turn, the policy’s proponent could
ask these questions about the reasons for the opponent’s perspective (i.e., rebuttals).
In other words, critical questions can help establish the relevance of an
argumentation scheme by encouraging consideration of the alternative perspective
through the generation of counterarguments and rebuttals (Ferretti et al., 2009;
Nussbaum & Edwards, 2011). Instruction about critical questions can be incorpo-
rated into the SRSD model (Ferretti et al., 2007).
Teaching critical questions about argumentation 69
123
As we previously mentioned, SRSD strategies have been developed for planning
and composing written arguments (De La Paz & Graham, 1997a, b; Graham &
Harris, 1989; Sexton et al., 1998), but only one strategy has been studied for
revising argumentative essays (Graham & MacArthur, 1988). This is unfortunate
because revision affords an opportunity for students to reflect on their ideas, develop
and apply evaluative criteria, and improve their writing skills (Scardamalia &
Bereiter, 1986). Despite its importance, most students ignore the revision process or
have an impoverished understanding of its purposes (Graves & Murray, 1980).
Expert writers revise their work to improve its overall quality and to clarify
important ideas, while novices revise to correct grammar, spelling, diction, and
punctuation (Faigley & Witte, 1981; MacArthur, Graham, & Schwartz, 1991;
Sommers, 1980). Although research shows that revising skill develops over time
Fig. 1 Argumentation schemes and critical questions for argument from consequences and argument from example. (Adapted from Walton, 1996)
70 Y. Song, R. P. Ferretti
123
(Fitzgerald & Markman, 1987), many college students are unable to revise
effectively (Kinsler, 1990).
Graham and MacArthur (1988) taught three 5th- and 6th-grade students with LD
the SCAN strategy for revising argumentative essays on a word processor. SCAN represents the procedure of reflecting on four questions—Does it make sense? Is it connected to my belief? Can I add more? Note errors. They found that instruction in the use of the SCAN strategy had a positive effect on students’ revising behavior.
Prior to instruction, only 31 % of the revisions involved the addition of reasons to
support author’s standpoint; after instruction, 62 % of the revisions involved the
addition of reasons. Furthermore, after instruction, students’ essays were longer and
of higher quality. The criteria for revising argumentative essays embedded in the
SCAN strategy included presenting a clear belief, providing reasons to support this
belief, and removing mechanical errors. These criteria are appropriate for younger
students and students with LD, but do not focus attention on the critical questions
that college students should use to test their argumentation.
The purpose of the current study was to assess the effects of an SRSD revising
strategy that taught college students to ask and answer critical questions about their
arguments concerning policy controversies. Students were assigned to one of the
following conditions. In the first condition, students were taught the Ask and
Answer Critical Questions (ASCQ) revising strategy to ask and answer critical
questions about the argument from consequences and argument from example schemes (Ferretti et al., 2009; Walton, 1992; Walton et al., 2008). In the second
condition, students were taught the Argumentation Schemes (AS) revising strategy
to include these argumentation schemes in their essays, but did not learn the critical
questions. In the third, students received no instruction about either the argumen-
tation schemes or the critical questions.
The inclusion of critical questions in the ASCQ condition should encourage
greater consideration of the alternative standpoint than either the AS condition or
the untreated control condition. In contrast, the AS condition should encourage the
writers to provide more supporting reasons for their standpoints compared to the
ASCQ or control conditions. Therefore, we expected that compared to the other
conditions, the ASCQ strategy should impact students’ first drafts and revisions by
(1) improving the overall quality of students’ essays; and (2) increasing the number
of counterarguments, alternative standpoints, and rebuttals. In addition, students
who learned the AS strategy were expected to develop more ‘‘my-side’’ reasons than
those in the other conditions. This is because students in this condition were taught
to use the argumentation schemes to develop reasons for their standpoint.
Method
Participants
Thirty undergraduate students enrolled in elementary education classes at a mid-
Atlantic University participated in the study. Participants received extra course
credit upon the completion of the study. All the participants were native English
Teaching critical questions about argumentation 71
123
speakers, and their average age was 19 years old. Among them, 27 were Caucasian
(90 %), two were Hispanic (7 %), and one was African American (3 %). All but one
of the participants was female. These proportions were representative of the student
population in elementary education classes at this university.
Writing prompts
A survey of controversial topics was informally conducted in three undergraduate
Education classes prior to the study. More than 100 students were asked to identify
three controversial issues that they thought were of interest to college students. Five
argumentative writing prompts about controversial policy issues were developed
based on their responses (see Table 1). Two of them were used as the pretest topics,
another two prompts were used as the posttest topics, and the last one served as the
maintenance test topic. All prompts were presented in a WORD file on a computer,
and each participant wrote and revised their essays on a computer.
Design and general procedure
An experimental design, which included a pretest, strategy instruction (AS or
ASCQ) or an untreated control, a posttest, and a 2-weeks maintenance test, was
employed. During pretest phase, all participants individually wrote two argumen-
tative essays on the first day, and revised their essays on a second day. The
following instruction was provided when participants wrote their first drafts:
‘‘Please read the writing prompt carefully. Write an argumentative essay to persuade
the reader to accept your opinion about this issue. There is no requirement on the
length of the essay. You have 45 min to complete the essay.’’ After completing their
first drafts, participants were told that they would revise their essays on a different
day. The instruction for revising tasks was as follows: ‘‘This is an essay you wrote.
Read it carefully. Think about any changes you would like to make and anything
that you would like to fix. Use everything you learned about revising essays to help
you. You have 45 min to revise your essay.’’
Table 1 Writing prompts used in study
Pretest prompts
Should Facebook be allowed to collect information about people’s purchasing preferences and sell it
to other companies?
Should university students be required to take a variety of courses outside their major field of study?
Posttest prompts
Should the university require students to be accountable for behavior discovered on the students’
Facebook account?
Should US government require all high school students to study a national curriculum before
entering college?
Maintenance test prompt
Should high school students be allowed to use cellphones while in school to communicate with
family and friends?
72 Y. Song, R. P. Ferretti
123
Based on their pretest performance, participants were matched and assigned to
three conditions: the control condition, the Argumentation Scheme (AS) strategy
condition, and the Ask and Answer Critical Questions (ASCQ) strategy condition.
The researchers ensured that the three conditions were roughly comparable and
statistically indistinguishable with respect to four argumentative elements (i.e.,
reasons, counterarguments, alternative standpoints, and rebuttals) before instruction
as a result of the assignment. Statistics about the comparability of the conditions are
provided in the results section.
After the pretest, participants in the two treatment conditions received instruction
in one of the two revising strategies over seven lessons. The AS strategy taught
participants to provide reasons for their standpoint based on the argument from consequences and argument from example schemes, and the ASCQ strategy taught participants to ask and answer critical questions about the use of these schemes in
addition to learning about them. In other words, the effects of the ASCQ strategy
were contrasted with the AS strategy that taught the argumentation schemes without
critical questions. Each lesson lasted 45–60 min, and all the lessons were completed
within 2–3 weeks. Participants in the untreated control condition received no
strategy instruction during the study, i.e., they simply were told to write and revise
their essays at each phase of the study as they did during the pretest. However,
control participants were taught the ASCQ strategy after the study’s conclusion.
After strategy instruction, all the participants individually wrote two essays and
revised them at posttest. The procedure was exactly the same as that of the pretest.
The maintenance test was conducted 2 weeks after the posttest was completed. At
maintenance test, participants wrote an essay and revised it on two separate days.
The ask and answer critical questions strategy instructional procedures
Participants learned the Ask and Answer Critical Questions strategy (ASCQ) during
seven lessons under the structural framework of the SRSD model. The six stages of
SRSD provided a general guideline for designing the lessons, which included
developing background knowledge, discussion, memorizing the steps of the
strategy, modeling how to use the strategy, practicing the strategy with support,
and independent use of the strategy. In lesson one, the instructor discussed the
general goal of this instruction. Participants received explicit instruction on the
basic argumentative elements (i.e., standpoint, reason, counterargument, rebuttal of
the counterarguments, alternative standpoint, reason for the alternative standpoints,
rebuttal of the alternative standpoints, and conclusion). The instructor also
explained that good arguments use relevant argumentative strategies, which
foreshadowed the use of argumentation schemes to justify a standpoint.
In lesson two, participants learned the concepts of argumentation schemes and
critical questions. They received explicit instruction in using argument from consequences and asking critical questions to evaluate the reasonableness of these arguments (see Fig. 1). By asking these critical questions, participants anticipated
potential counterarguments and alternative standpoints. They also learned how to
answer these critical questions (i.e., to rebut the counterarguments and alternative
standpoints) to strengthen their arguments. In lesson three, participants learned the
Teaching critical questions about argumentation 73
123
argument from example scheme, and received explicit instruction in how to ask and answer critical questions associated with it (Fig. 1).
In lesson four, the instructor discussed the importance of revision and then
introduced the ASCQ strategy for improving revising skills. This strategy prompted
participants not only to consider the Argumentation Schemes they used in their
essay but also to ask and answer Critical Questions to improve their arguments. The
instructor described the steps of the ASCQ strategy: (1) Read your essay and find
the sentence that states your standpoint about the issue; (2) Ask CQ—What
Argumentation Scheme(s) did I use to support my standpoint? What Critical
Questions should I use to test the argumentation scheme(s)?; (3) Answer each
critical question (Answer CQ)—Explain how your responses justify the use of the
argumentation scheme and respond to potential criticisms of your argument; and (4)
Re-read the essay and make final changes. After participants had memorized all the
steps, the instructor modeled the use of the ASCQ strategy for revising a standard
essay. While modeling, the instructor used self-instructions (e.g., problem
definition, planning, strategy use, self-evaluation, and self-reinforcement). The
instructor also demonstrated how to use the ‘‘Insert Comment’’ and ‘‘Track
changes’’ functions in the word processor during revision. Specifically, the
instructor used these functions to record each argumentation scheme and list
relevant critical questions, as well as identify the changes to the essays between
drafts.
In lessons 5–7, participants practiced using the ASCQ strategy to revise three
standard essays. Participants took increasing responsibility for controlling the
strategy. The instructor provided only as much help as was necessary at this stage of
instruction. Instruction was completed when participants were able to apply all the
steps of the ASCQ strategy independently.
The argumentation scheme strategy instructional procedures
Participants learned the Argumentation Scheme strategy (AS) during seven SRSD
lessons. Lesson one was exactly the same as that of the ASCQ condition. In lesson
two, participants learned the concept of argumentation scheme. They received
explicit instruction in using argument from consequences but without the inclusion of critical questions. In lesson three, participants learned about the argument from example scheme, and to develop reasons based on this scheme (see Fig. 1).
In lesson four, the instructor discussed the importance of revision and then
introduced the steps of the AS strategy for improving revising skills: (1) Read your
essay and find the sentence that states your standpoint about the issue; (2) Add
reasons based on argument from consequences to support your standpoint; (3)
Develop reasons based on argument from example; and (4) Re-read the essay and
make final changes. After participants had memorized all the steps, the instructor
modeled the use of the AS strategy for revising a standard essay. The modeling
process was similar to that in the ASCQ condition. In lessons 5–7, participants
practiced using the AS strategy to revise three standard essays. The procedure was
exactly same as that in the ASCQ condition. Instruction was completed when
participants were able to apply all the steps of the AS strategy independently.
74 Y. Song, R. P. Ferretti
123
Treatment validity
To ensure that the instructional procedures were implemented according to plan, we
implemented the following safeguards. First, Yi Song practiced applying all
instructional procedures. Second, she administered written lesson plans that
included each step of a daily lesson. Third, she checked off all the steps of
instruction when they were completed. Finally, all the instructional sessions were
videotaped and Yi Song randomly checked 20 % of the sessions. All of the planned
instructional procedures were completed for both the AS and ASCQ conditions (i.e.,
100 %).
Measures
This section first describes the measure for the overall quality of the essays, and then
explains how the argumentative structures were analyzed to identify various
argumentative elements and their relationships.
Overall quality
We used a persuasive writing rating scale that was based on Ferretti et al.’s (2000)
study to evaluate the overall quality of the essays. Overall quality refers to the
effectiveness of the essay in influencing a particular audience to take some action or
to change their thinking about a controversial issue. In evaluating overall quality,
the following components were considered: (a) whether students’ arguments stated
a clear opinion, (b) whether students provided supporting reasons for their opinion,
(c) how well students elaborated their reasons, (d) whether students anticipated and
responded to alternative perspectives, and (e) how well students organized their
arguments including an introduction and a conclusion. Essays were rated on a scale
from 1 to 7. The low-level standards (e.g., no opinion, no reasons) were removed
from Ferretti et al.’s (2000) rubric because college students were able to present an
opinion and give reasons to support their opinion. The high-level standard (i.e., level
7) included consideration of potential counterarguments and alternative perspec-
tives. This required the provision of rebuttals that strengthen the participant’s
standpoint.
Three undergraduate students and one graduate student unfamiliar with the
purpose and design of the study received six, 1-h training and practice sessions in
using the primary trait scoring guide. The scoring guide included anchor essays, and
it directed raters to judge the overall persuasiveness of an essay. The raters then
independently scored either first or revised drafts of all the essays. As recommended
by Rushton, Brainerd, and Pressley (1983), the scores for essays for which there
were two raters were averaged for a final persuasiveness score. These four raters
formed two pairs. Interrater agreement of pair 1 (agreement = agreements/
agreements ? disagreements) within 1 point was found to be 90 %, and interrater
agreement of pair 2 was 94 %.
Teaching critical questions about argumentation 75
123
Structural analysis of essays
We evaluated the structure of participants’ argumentative essays by using the
process derived from the pragma-dialectical theory of argumentation (Ferretti et al.,
2009; van Eemeren & Grootendorst, 1992, 2004; van Eemeren et al., 2002). In brief,
we developed a coding system that allowed us to distinguish the elements of an
argumentative structure and the relationships among elements, and then to
graphically depict these structural relationships.
As in Ferretti et al. (2009), we used the following process to graphically represent
the structure of participants’ written arguments: (a) identify the participant’s
standpoint(s) about the controversial issue (i.e., the standpoint advanced by the participant), (b) identify the participant’s reasons that are offered as support for the
standpoint (i.e., the propositions that support the participant’s standpoint or serve as
elaborations of reasons), (c) identify counterarguments that could be used to object to or undermine the participant’s standpoint (i.e., potential criticisms of either the
participant’s standpoint or reasons for the participant’s standpoint), (d) identify
rebuttals of the counterarguments (i.e., the participant’s explanation of why the counterarguments are wrong or bad reasons), (e) identify alternative standpoint(s) of the controversial issue (i.e., standpoints of other people that the participant
disagrees with), (f) identify reasons for the alternative standpoint (i.e., the propositions that support other people’s standpoint), (g) identify rebuttals of the alternative standpoint (i.e., propositions that attack an alternative standpoint or its reasons and thereby strengthen the participant’s standpoint), (h) identify an
introduction that foreshadows what is to follow in the participant’s presentation of the argument, (i) identify a conclusion that summarizes what the participant has written, (j) identify functional repetitions that restate previously expressed reasons and/or standpoints, and (k) identify non-functional elements that include informa- tion irrelevant for supporting an argument about the issue. By using these elements,
we were able to analyze and graphically depict the individual components as well as
the structure of participants’ essays.
Two undergraduate students unfamiliar with the purpose and design of the study
received six, 1-h training sessions in using the structural analysis scoring guide. The
scoring guide included anchor essays that illustrated the relationships among
argumentative writing elements. The raters then practiced graphically depicting the
arguments for 15 essays in the scoring guide. These structures were evaluated and
discussed to improve the graphing of arguments. After training and practice, the raters
and the first author formed three pairs and each of them independently graphed the
structure of two-thirds of the total number of the essays. Interrater agreement of 3 pairs
(agreement = agreements/agreements ? disagreements) within 1 instance was com-
puted for each argumentative element: author’s standpoint(s) = 100, 100, 98 %;
reasons for author’s standpoint(s) = 71, 73, 72 %; counterarguments = 93, 97,
93 %; alternative standpoint(s) = 100, 100, 99 %; reasons for alternative stand-
point(s) = 100, 98, 99 %; rebuttals = 92, 94, 95 %; introductions = 100, 100,
100 %; conclusions = 100, 100, 100 %; functional repetitions = 98, 92, 96 %;
nonfunctional elements = 100, 100, 100 %. The structural analyses of the raters were
76 Y. Song, R. P. Ferretti
123
reconciled, yielding a final structure from which all dependent measures were
computed.
Results
In what follows, we report the findings of the data analyses on the overall quality of
participants’ argumentative essays and the argumentative structures. As mentioned
previously, we expected that compared to the AS and control conditions, the ASCQ
strategy should impact participants’ first drafts and revisions by (1) improving the
overall quality of their essays; and (2) increasing the number of counterarguments,
alternative standpoints, and rebuttals. In addition, participants who learned the AS
strategy should develop more ‘‘my-side’’ reasons than those in the other conditions.
Statistical analyses
We used the following analytic approach to test the aforementioned predictions for
each of the dependent measures. We first computed an average score on each
dependent measure for the two writing prompts administered during the pretest and
posttest, respectively, because prompts did not have a significant effect (p [ .05) on essay quality at any phase of testing. The means and standard deviations for each of
the dependent measures are presented in Table 2 (Essay Quality: Pretest, Posttest,
Maintenance Test), Table 3 (Structural Measures: Pretest and Posttest), and Table 4
(Structural Measures: Maintenance test). Then, the effects of the instructional
condition, phase, and draft were examined for each measure by running a 3-way
ANOVA with repeated measures. Within-subjects factors included phase (i.e.,
pretest, posttest, and maintenance test), and draft (i.e., first and revised drafts), and
the between-subjects factor was instructional condition (i.e., ASCQ, AS, and
control). The factors contributing to a 3-way interaction were analyzed by running a
series of 2-way ANOVAs. A one-way ANOVA was run to determine the source of
significant 2-way ANOVAs, and paired-sample t tests were run to test for simple
effects. In general, we ran ANOVAs on the appropriate lower order interaction
effects and simple effects to decompose statistically significant higher order
interaction effects. For each of our predictions, we report the core findings with
respect to the effects of instructional condition, phase, and draft on each of the
dependent measures.
Overall quality
Table 2 presents the means and standard deviations for essay quality by
instructional condition, draft, and phase. It shows that the quality of participants’
essays ranged from 3.43 to 4.08 at pretest. Their essays were partially developed on
the 7-point scale and were not persuasive. After instruction, the mean scores of
essays written by participants in the ASCQ conditions were 5 or higher, while those
of participants in the AS condition were 4 or higher. The control condition was
unchanged.
Teaching critical questions about argumentation 77
123
A 3-way repeated-measures ANOVA was run to assess the effect of instructional
condition, phase, and draft on overall quality. The 3-way interaction effect was not
statistically significant. However, significant main effects occurred for phase,
F(1.830,49.404) = 21.66, p = .000, partial g2 = .45; draft, F(1, 27) = 27.97, p = .000, partial g2 = .51; and instructional condition, F(2, 27) = 9.70, p = .001, partial g2 = .42. It also revealed significant interactions between phase and instructional condition, F (3.660, 49.404) = 9.73, p = .000, partial g2 = .42; and between draft and instructional condition, F(2, 27) = 4.75, p = .017, partial g2 = .26.
To explore the interaction between phase and instructional condition, the two
drafts were averaged at each phase, and a one-way ANOVA was conducted to
compare mean scores across three conditions at each phase. No significant
difference was found at the pretest, p [ .05. However, instructional condition had a significant effect at the posttest, F(2, 27) = 15.86, p = .000, partial g2 = .54; and maintenance test, F(2, 27) = 20.26, p = .000, partial g2 = .60. Post hoc compar- isons indicated that the essay quality of the ASCQ condition was higher than that of
the other conditions at the posttest, p \ .003; and at the maintenance test, p = .000. These results show that ASCQ condition resulted in higher quality essays after
instruction than either of the other two conditions.
To explore the interaction between draft and instructional condition, the scores at
different phases were averaged for each draft, and a paired sample t test was
conducted for each condition to compare mean scores of different drafts. The
quality of the draft was not significant for the control condition, p [ .05. However, the revised draft in the ASCQ condition, p = .003; and AS, p = .027, had a significantly higher score than the first draft. These results confirmed our
expectation that learning the ASCQ and AS strategies would positively impact
the quality of students’ revised draft.
These analyses suggest that the participants in the ASCQ condition wrote better
essays for the first and revised drafts. However, it is unclear whether these differences
were solely attributable to the quality of their first drafts, i.e., whether the differences in
essay quality among conditions after instruction were evident for the revised draft after
controlling for the first draft. To assess this possibility, a one-way analysis of
covariance (ANCOVA) was conducted for the posttest and maintenance test,
respectively. The dependent variable was the quality of the revised draft, the
independent variable was condition, and the covariate was the quality of the first draft.
The ANCOVA was significant for the posttest, F(2, 26) = 8.63, p = .001, partial
Table 2 Means for overall quality in the pretest, posttest, and maintenance test
Control AS ASCQ
1st D 2nd D 1st D 2nd D 1st D 2nd D
Pretest 3.93 (0.41) 3.95 (0.45) 3.45 (1.10) 3.90 (1.14) 3.43 (0.71) 4.08 (0.66)
Posttest 3.78 (0.79) 3.82 (0.71) 4.10 (0.53) 4.43 (0.61) 5.05 (1.01) 5.73 (0.72)
Maintenance 3.80 (0.54) 4.10 (0.77) 4.40 (0.46) 4.30 (0.35) 5.30 (0.79) 5.55 (0.64)
1st D = 1st draft; 2nd D = 2nd draft. The overall quality ranged 1–7. Standard deviations in parentheses
78 Y. Song, R. P. Ferretti
123
T a
b le
3 M
e a n s
fo r
st ru
c tu
ra l
sc o re
s o f
p re
te st
a n d
p o st
te st
e ss
a y s
V a ri
a b
le C
o n
tr o l
A S
A S
C Q
P re
te st
P o
st te
st P
re te
st P
o st
te st
P re
te st
P o
st te
st
1 st
D 2
n d
D 1
st D
2 n
d D
1 st
D 2
n d
D 1
st D
2 n
d D
1 st
D 2
n d
D 1
st D
2 n
d D
A u
th o
r’ s
re a so
n
1 9
.6 0
(9 .8
6 )
2 0
.6 0
(9 .8
1 )
2 1
.7 5
(8 .2
2 )
2 2
.1 0
(8 .5
1 )
2 0
.2 0
(5 .5
9 )
2 0
.8 0
(5 .4
0 )
2 5
.3 5
(7 .4
1 )
3 5
.8 5
(9 .9
7 )
2 1
.7 0
(6 .9
3 )
2 3
.1 5
(7 .3
2 )
1 6
.9 5
(6 .7
8 )
1 7
.7 5
(6 .4
7 )
C o u n te
ra rg
u m
e n t
0 .7
5
(1 .4
6 )
0 .7
5
(1 .4
6 )
0 .1
0
(0 .3
2 )
0 .1
0
(0 .3
2 )
0 .4
0
(0 .7
0 )
0 .4
0
(0 .7
0 )
0 .1
5
(0 .2
4 )
0 .1
5
(0 .2
4 )
0 .4
5
(0 .8
3 )
0 .4
5
(0 .8
3 )
3 .9
5
(3 .7
4 )
6 .6
0
(2 .8
3 )
A lt
e rn
a ti
v e
st a n
d p
o in
t
0 .1
5
(0 .3
4 )
0 .1
5
(0 .3
4 )
0 .1
0
(0 .2
1 )
0 .1
0
(0 .2
1 )
0 .0
5
(0 .1
6 )
0 .0
5
(0 .1
6 )
0 .0
0
(0 .0
0 )
0 .0
0
(0 .0
0 )
0 .0
5
(0 .1
6 )
0 .0
5
(0 .1
6 )
0 .6
5
(0 .4
7 )
0 .7
5
(0 .4
2 )
R e a so
n fo
r
a lt
e rn
a ti
v e
st a n
d p
o in
t
0 .8
5
(2 .2
1 )
0 .8
5
(2 .2
1 )
0 .1
0
(0 .3
2 )
0 .1
0
(0 .3
2 )
0 .2
5
(0 .7
9 )
0 .2
5
(0 .7
9 )
0 .0
0
(0 .0
0 )
0 .0
0
(0 .0
0 )
0 .0
0
(0 .0
0 )
0 .0
0
(0 .0
0 )
0 .6
0
(0 .5
7 )
0 .8
5
(0 .6
3 )
R e b
u tt
a l
1 .4
5
(2 .9
8 )
1 .5
5
(2 .9
6 )
0 .6
0
(1 .7
3 )
0 .6
0
(1 .7
3 )
1 .5
5
(3 .2
9 )
1 .5
0
(3 .2
7 )
0 .9
0
(1 .5
8 )
1 .0
0
(1 .7
8 )
0 .4
5
(0 .8
6 )
0 .6
0
(0 .9
9 )
1 4
.2 5
(9 .7
9 )
2 1
.2 5
(6 .3
0 )
1 st
D =
1 st
d ra
ft ;
2 n
d D
= 2
n d
d ra
ft .
S ta
n d
a rd
d e v
ia ti
o n
s in
p a re
n th
e se
s
Teaching critical questions about argumentation 79
123
g2 = .40; and the maintenance test, F(2, 26) = 3.75, p = .037, partial g2 = .22. Pairwise comparisons indicated that during the posttest the ASCQ condition wrote
significantly better revised drafts, controlling for the effect of the first drafts, than the
other two conditions, p \ .03. Comparisons also showed that the revised drafts in the ASCQ condition were of higher quality, controlling for the effect of the first draft, than
those in the AS condition, p = .033, at the maintenance test. These results indicate that the ASCQ condition resulted in better initial and revised drafts in comparison to the
other conditions, and that it had an effect on the revised draft even after removing the
effects of the first draft.
Essay structure
The means and standard deviations for each basic argumentative component are
presented in Table 3 (pretest and posttest) and Table 4 (maintenance test). The
effects of instructional condition, phase, and draft on each structural element were
examined using a series of 3-way repeated-measures ANOVAs on each of the
following measures: reasons for the author’s standpoint, counterarguments,
alternative standpoints, reasons for the alternative standpoints, and rebuttals.
Reasons for the author’s standpoint
Participants were taught to use the AS strategy to develop their argument by adding
reasons for their standpoint. Thus, participants in the AS condition should include
more reasons in their revised essays after instruction than participants in the other
conditions.
The 3-way repeated measures ANOVA on the number of author’s reasons
revealed significant main effects for draft, F(1, 27) = 23.65, p = .000, partial g2 = .47; and instructional condition, F(2, 27) = 5.42, p = .011, partial g2 = .29. It also revealed significant interactions between phase and instructional condition,
F (4, 54) = 5.10, p = .001, partial g2 = .27; between draft and instructional
Table 4 Means for structural scores of maintenance test essays
Maintenance test Control AS ASCQ
Variable 1st D 2nd D 1st D 2nd D 1st D 2nd D
Reason for
author’s standpoint
18.40
(7.66)
20.10
(8.21)
26.60
(10.89)
35.70
(11.06)
18.80
(6.60)
18.80
(6.37)
Counterargument 0.90
(1.45)
0.90
(1.45)
0.40
(0.52)
0.40
(0.52)
3.90
(2.47)
5.40
(1.17)
Alternative standpoint 0.00
(0.00)
0.00
(0.00)
0.10
(0.32)
0.10
(0.32)
0.70
(0.48)
0.80
(0.42)
Reason for
alternative standpoint
0.00
(0.00)
0.00
(0.00)
0.00
(0.00)
0.00
(0.00)
1.80
(2.44)
2.00
(2.36)
Rebuttal 2.50
(4.25)
3.20
(5.83)
2.90
(3.57)
3.10
(3.67)
13.60
(8.02)
19.00
(8.88)
1st D = 1st draft; 2nd D = 2nd draft. Standard deviations in parentheses
80 Y. Song, R. P. Ferretti
123
condition, F(2, 27) = 11.22, p = .000, partial g2 = .45; and between phase and draft, F(2, 54) = 4.23, p = .020, partial g2 = .14. In addition, the three-way interaction phase X instructional condition X draft was significant, F(4, 54) = 6.28, p = .000, partial g2 = .32.
A series of 2-way ANOVAs were then conducted at each phase with draft and
instructional condition as the independent variables. At the pretest, draft had a
significant effect, F(1, 27) = 12.11, p = .002, partial g2 = .31. However, the three conditions did not differ with respect to the number of reasons for either the first
draft or the revised draft (p [ .05), which means that they were comparable with respect to the production of reasons prior to instruction. At the posttest, there were
significant effects for draft, F(1, 27) = 24.35, p = .000, partial g2 = .47; instruc- tional condition, F(2, 27) = 7.67, p = .002, partial g2 = .36; and the interaction between draft and instructional condition, F(2, 27) = 17.70, p = .000, partial g2 = .57. A one-way ANOVA analysis revealed a significant difference in the number of reasons in the revised draft, F(2, 27) = 12.53, p = .000, partial g2 = .48, but not in the first draft. Post hoc comparisons indicated that participants in the AS condition included significantly more reasons in the revised draft than did
participants in the other two conditions, p \ .005. A paired sample t test indicated that numbers of reasons in the revised draft were significantly greater than in the
first draft for the AS condition, p = .001, but there were no differences between drafts for the control and ASCQ conditions.
Like the posttest, analyses of the maintenance test revealed effects of draft,
F(1, 27) = 8.68, p = .007, partial g2 = .24; instructional condition, F(2, 27) = 7.67, p = .002, partial g2 = .36; and the interaction between draft and instructional condition, F(2, 27) = 5.23, p = .012, partial g2 = .28. A one-way ANOVA was then computed for each draft. There was a significant difference in the number of reasons in
the revised draft, F(2, 27) = 11.52, p = .000, partial g2 = .46, but not in the first draft. Post hoc comparisons indicated that the AS condition included significantly
more reasons in the revised draft than did participants in the other two conditions,
p = .001. The number of reasons in the revised draft was greater than in the first draft for the AS condition, p = .031, but there were no differences between drafts for the control and ASCQ conditions. These results are consistent with the conclusion that
participants who learned the AS strategy added more reasons to support their
standpoint in the revised draft than participants in the other conditions.
Counterarguments
The ASCQ strategy taught participants to ask and answer critical questions to
anticipate potential criticisms of their written arguments, i.e., counterarguments.
Thus, participants in the ASCQ condition should include a greater number of
counterarguments in their revised draft after instruction as compared to the other
conditions.
A 3-way repeated-measures ANOVA was run to determine the effects of phase,
draft, and instructional condition on the number of counterarguments. Significant
main effects were found for phase, F(2, 54) = 14.66, p = .000, partial g2 = .35; draft, F(1, 27) = 10.53, p = .003, partial g2 = .28; and instructional condition,
Teaching critical questions about argumentation 81
123
F(2, 27) = 31.22, p = .000, partial g2 = .70. In addition, significant interactions were obtained between phase and instructional condition, F (4, 54) = 19.69, p = .000, partial g2 = .59; and between draft and instructional condition, F(2, 27) = 10.53, p = .000, partial g2 = .44. The three-way interaction phase X instructional condition X draft was also significant, F(3.055, 41.239) = 2.90, p = .045, partial g2 = .18.
In the pretest, participants in all three conditions generated few counterarguments
in either draft. This pattern continued during the posttest and maintenance test for
participants in the control and AS conditions. However, participants in the ASCQ
condition demonstrated an increased sensitivity to potential criticisms by including
an average of more than 3.8 counterarguments after instruction. A series of 2-way
ANOVAs were then conducted at each phase with draft and instructional condition
as the independent variables. No differences were found in the pretest, p [ .05, which means that the three conditions were comparable with respect to the
production of counterarguments prior to instruction.
The repeated measures ANOVA at the posttest revealed the following significant
effects: draft, F(1, 27) = 5.31, p = .029, partial g2 = .16; instructional condition, F(2, 27) = 32.19, p = .000, partial g2 = .71; and the interaction between draft and instructional condition, F(2, 27) = 5.31, p = .011, partial g2 = .28. A one-way ANOVA was then conducted for each draft, and these analyses showed that there
was a significant difference in the number of counterarguments in both the first
draft, F(2, 27) = 10.36, p = .000, partial g2 = .43; and the revised draft, F(2, 27) = 49.82, p = .000, partial g2 = .79. Post hoc comparisons indicated that participants in the ASCQ condition included significantly more counterarguments
than did participants in the other two conditions in the first draft, p \ .05; and the revised draft, p = .000. Furthermore, ASCQ participants produced more counter- arguments in their revised draft than the first draft, p = .047. The number of counterarguments did not change from the first draft to the revised draft for the
control and AS conditions, p [ .05. Like the posttest, analyses of the maintenance test revealed effects of draft,
F(1, 27) = 5.26, p = .030, partial g2 = .16; instructional condition, F(2, 27) = 32.14, p = .000, partial g2 = .70; and the interaction between draft and instructional condition, F(2, 27) = 5.26, p = .012, partial g2 = .28. A one-way ANOVA was then computed for each draft, and a significant difference in the
number of counterarguments was found for both the first draft, F(2, 27) = 12.70, p = .000, partial g2 = .48; and the revised draft, F(2, 27) = 60.76, p = .000, partial g2 = .82. Post hoc comparisons indicated that participants in the ASCQ condition included significantly more counterarguments than did participants the
other two conditions for the first draft, p \ .002; and the revised draft, p = .000. Furthermore, participants in the ASCQ condition produced more counterarguments
in the revised draft than the first draft, p = .043. The number of counterarguments did not change from the first draft to the revised draft for the control and AS
conditions, p [ .05. The analyses of the posttest and maintenance test data show that participants in the ASCQ condition produced more counterarguments
for both drafts than those in the other conditions, but the effects were especially
pronounced for the revised draft.
82 Y. Song, R. P. Ferretti
123
Alternative standpoints
Participants in the ASCQ condition were taught to explicitly consider the alternative
perspective about the controversial policies. Therefore, they should have included
more alternative standpoints in their arguments after instruction in comparison to
participants in the AS and control conditions.
A 3-way ANOVA with repeated measures on the number of alternative
standpoints revealed a significant main effect for phase, F(2, 54) = 6.08, p = .004, partial g2 = .18; instructional condition, F(2, 27) = 18.75, p = .000, partial g2 = .58; and a significant interaction between phase and instructional condition, F(4, 54) = 10.19, p = .000, partial g2 = .43. Few alternative standpoints were included in either draft by participants during the pretest. In the posttest and
maintenance test, participants in the control and AS conditions rarely presented
alternative standpoints. However, participants in the ASCQ condition usually
included an alternative standpoint after receiving instruction.
To explore the interaction between phase and instructional condition, the two
drafts were averaged at each phase, and a one-way ANOVA was conducted to
compare mean numbers of alternative standpoints across three conditions at each
phase. No significant difference was found at the pretest, p [ .05. However, instructional condition had a significant effect at the posttest, F(2, 27) = 19.35, p = .000, partial g2 = .59; and maintenance test, F(2, 27) = 17.73, p = .000, partial g2 = .57. Post hoc comparisons indicated that participants in the ASCQ condition included significantly more alternative standpoints than did participants in
the other two conditions after instruction, p \ .005. These analyses suggest that the participants in the ASCQ condition produced
more alternative standpoints than those in the AS and control conditions for both
drafts. However, it is unclear whether these differences were solely attributable to
the inclusion of alternative standpoints in their first drafts, i.e., whether the
differences in alternative standpoints among conditions after instruction were
evident for the revised draft after controlling for the first draft. To assess this
possibility, a one-way analysis of covariance (ANCOVA) was conducted for the
posttest and maintenance test, respectively. The dependent variable was alternative
standpoint in the revised draft, the independent variable was condition, and the
covariate was alternative standpoint in the first draft. The ANCOVA was significant
for the posttest, F(2, 26) = 4.09, p = .029, partial g2 = .24; and the maintenance test, F(2, 26) = 3.85, p = .034, partial g2 = .23. Pairwise comparisons indicated that during the posttest the ASCQ condition had significantly more alternative
standpoints in the revised draft, controlling for the effect of the first draft, than the
other two conditions, p \ .05. Comparisons also showed that the ASCQ condition had significantly more alternative standpoints in the maintenance test revised draft,
controlling for the effect of the first draft, than the control condition, p = .046. These results suggest that the ASCQ condition resulted in a greater number of
alternative standpoints in their initial and revised drafts in comparison to the other
conditions, and that it had an effect on the revised draft even after removing the
effects of the first draft.
Teaching critical questions about argumentation 83
123
Reasons for the alternative standpoint
Participants in the ASCQ condition were taught to explicitly consider the alternative
perspective about controversial issues. Therefore, they should include more reasons
for alternative standpoints after instruction in comparison to participants in the AS
and control conditions.
A 3-way analysis of variance (ANOVA) with repeated measures was run to
assess the effects of phase, draft, and instructional condition on the reasons for the
alternative standpoint. The 3-way interaction effect was not statistically significant.
However, significant main effects occurred for draft, F(1, 27) = 4.31, p = .047, partial g2 = .14; and instructional condition, F(2, 27) = 3.42, p = .047, partial g2 = .20. It also revealed significant interactions between phase and instructional condition, F (3.435, 46.367) = 4.36, p = .006, partial g2 = .24; and between draft and instructional condition, F(2, 27) = 4.31, p = .024, partial g2 = .24.
To explore the interaction between phase and instructional condition, the two
drafts were averaged at each phase, and a one-way ANOVA was conducted to
compare mean numbers of reasons for the alternative standpoint across three
conditions at each phase. No significant difference was found at the pretest, p [ .05. However, instructional condition had a significant effect at the posttest, F(2, 27) = 13.00, p \ .010, partial g2 = .49; and maintenance test, F(2, 27) = 6.32, p \ .010, partial g2 = .32. Post hoc comparisons indicated that the ASCQ condition included significantly more reasons for the alternative standpoint than the AS
condition, p = .004; and the control condition, p = .012, for the posttest. However, post hoc comparisons did not reveal any significant effects for the maintenance test,
p [ .05. To explore the interaction between draft and instructional condition, the number
of reasons for the alternative standpoint at different phases was averaged for each
draft. There was no difference in the number of reasons for the alternative
standpoint between the two drafts produced by either the AS or the control
condition, p [ .05. Although the ASCQ condition had a small effect on the number of reasons for the alternative standpoint between the two drafts, it was not
statistically significant, p [ .05. These analyses suggest that the participants in the ASCQ condition produced
more reasons for the alternative standpoint for the first and revised drafts during the
posttest. However, it is unclear whether these differences were solely attributable to
the inclusion of reasons for the alternative standpoint in their first drafts, i.e.,
whether the differences in reasons for the alternative standpoint among conditions
after instruction were evident for the revised draft after controlling for the first draft.
To assess this possibility, a one-way analysis of covariance (ANCOVA) was
conducted for the posttest. The dependent variable was reasons for the alternative
standpoint in the revised draft, the independent variable was condition, and the
covariate was reasons for the alternative standpoint in the first draft. The ANCOVA
was significant for the posttest, F(2, 26) = 4.08, p = .029, partial g2 = .24. Pairwise comparisons indicated that the ASCQ condition had significantly more
reasons for the alternative standpoint in the posttest revised draft, controlling for the
effect of the first draft, than the AS condition, p = .043. These results suggest that
84 Y. Song, R. P. Ferretti
123
learning the ASCQ strategy encouraged participants to consider reasons for the
alternative standpoint in the first draft and the revised draft, and that during the
posttest, it had an effect on the revised draft even after removing the effects of
the first draft.
Rebuttals
The ASCQ strategy taught participants to rebut potential criticisms to strengthen
their standpoint. Therefore, participants in the ASCQ condition should include more
rebuttals after instruction than did participants in other conditions.
A 3-way ANOVA with repeated measures on the number of rebuttals revealed
significant main effects for phase, F(2, 54) = 24.63, p = .000, partial g2 = .48; draft, F(1, 27) = 13.58, p = .001, partial g2 = .34; and instructional condition, F(2, 27) = 29.42, p = .000, partial g2 = .69. It also revealed significant interactions between phase and instructional condition, F (4, 54) = 22.17, p = .000, partial g2 = .62; between draft and instructional condition, F(2, 27) = 10.63, p = .000, partial g2 = .44; and between phase and draft, F(2, 27) = 5.03, p = .010, partial g2 = .16. The three-way interaction phase X instructional condition X draft was also significant, F(4, 54) = 4.37, p = .004, partial g2 = .24.
A series of 2-way ANOVAs were then conducted at each phase with draft and
instructional condition as the independent variables. No significant difference was
found in the pretest, p [ .05. However, the repeated measures ANOVA on the posttest revealed significant effects for draft, F(1, 27) = 9.01, p = .006, partial g2 = .25; instructional condition, F(2, 27) = 48.32, p = .000, partial g2 = .78; and the interaction between draft and instructional condition, F(2, 27) = 8.64, p = .001, partial g2 = .39. One-way ANOVAs revealed a significant difference in the number of rebuttals in both the first draft, F(2, 27) = 17.98, p = .000, partial g2 = .57; and the revised draft, F(2, 27) = 91.15, p = .000, partial g2 = .87. Post hoc compar- isons indicated that participants in the ASCQ condition included significantly more
rebuttals than did participants in the other two conditions in the first draft, p \ .01; and the revised draft, p = .000. A paired sample t test revealed that the revised draft produced by the ASCQ condition included more rebuttals than the first draft in the
posttest, p = .016. The repeated measures ANOVA on the maintenance test revealed similar
significant effects: draft, F(1, 27) = 8.67, p = .007, partial g2 = .24; instructional condition, F(2, 27) = 18.03, p = .000, partial g2 = .57; and the interaction between draft and instructional condition, F(2, 27) = 5.39, p = .011, partial g2 = .29. One- way ANOVAs showed a significant difference in the number of rebuttals in both the
first draft, F(2, 27) = 12.51, p = .000, partial g2 = .48; and the revised draft, F(2, 27) = 19.90, p = .000, partial g2 = .60. Post hoc comparisons indicated that participants in the ASCQ condition included significantly more rebuttals than did
participants in the other two conditions in both drafts, p \ .05. A paired sample t test found a significant difference between the two drafts in the ASCQ condition,
p = .021. These results indicate that the revised draft produced by the ASCQ condition included more rebuttals than the first draft in the maintenance test. In sum,
these results from the posttest and maintenance test show that participants in the
Teaching critical questions about argumentation 85
123
ASCQ condition produced more rebuttals for both drafts than those in the other
conditions, but the effects were especially pronounced for the revised draft.
Discussion
Previous research (De La Paz & Graham, 1997a, b; Graham & Harris, 1989; Sexton
et al., 1998) demonstrated that strategy instruction positively impacts the quality of
students’ written arguments and increases the number of argumentative elements.
However, arguments are a structured constellation of propositions that in sum increase
the acceptability of the writer’s standpoint (van Eemeren & Grootendorst, 1992, 2004;
van Eemeren et al., 1993, 1996). In addition, arguments depend on the use of specific
argumentation schemes that are meant to achieve their particular discursive purposes
(van Eemeren & Grootendorst, 1992, 2004; van Eemeren et al., 1993; Walton, 1992,
1996). Finally, the relevance and acceptability of these arguments should be evaluated
by the application of scheme-relevant critical questions. The latter issues were not
considered in the aforementioned strategy instruction studies (Ferretti et al., 2007),
and we are aware of only one intervention study (Nussbaum & Edwards, 2011) that
taught seventh-grade students critical questions. Therefore, we sought to determine
whether the ASCQ revising strategy promoted greater consideration of the alternative
perspective, as evidenced by the inclusion of counterarguments, alternative
standpoints, and rebuttals, and also resulted in higher quality argumentative essays
than the other two conditions. Furthermore, we contrasted the effects of the ASCQ
strategy with the AS strategy, and expected the AS strategy to be associated with the
production with more ‘‘my-side’’ reasons, i.e., reasons for the author’s standpoint.
We found that teaching participants to ask and answer critical questions about
their argumentation schemes improved the quality of their essays. After ASCQ
instruction, the quality of participants’ revised drafts was above 5.5 points on a
7-point quality scale, far exceeding the rated quality of essays produced by
participants in either the AS or control conditions. Learning the AS strategy also led
to a slight improvement of the essay quality, but the impact was modest. These
results indicate that a strategy that taught college students to ask and answer critical
questions about argumentation schemes resulted in higher quality written arguments
(Ferretti et al., 2007; 2009; Nussbaum & Edwards, 2011; Walton, 1996).
Measures derived from the structural analyses of participants’ essays showed that
participants who learned the AS strategy included more reasons for their standpoint as
a result of instruction, but rarely anticipated potential counterarguments or rebutted
alternative standpoints. In contrast, participants in the ASCQ condition not only
generated more counterarguments but also produced more alternative standpoints,
reasons for the alternative standpoint, and rebuttals. Thus, the ASCQ strategy resulted
in more critical reflection about the potential weaknesses of participants’ written
arguments. Our finding is consistent with prior research on teaching critical questions
to seventh graders, which found that the experimental group made more arguments
that integrated both sides of an issue (Nussbaum & Edwards, 2011).
We note, however, that the effects of the ASCQ condition were especially
notable for the inclusion of counterarguments and rebuttals, and these effects were
86 Y. Song, R. P. Ferretti
123
less distinctive for alternative standpoints and reasons for the alternative standpoint.
Two reasons probably account for these findings. First, almost all participants in the
ASCQ condition explicitly acknowledged a single alternative standpoint that was
contrasted with their own standpoint. Therefore, while effective, the ASCQ
condition did not result in a marked increase in the number of alternative
standpoints. Second, the addition of a relatively large number of counterarguments
and associated rebuttals by participants in the ASCQ condition may mitigate the
inclusion of many reasons for the alternative standpoint. This is because
counterarguments are potential criticisms of the author’s standpoint that could
have been leveled by proponents of the alternative standpoint. Said differently, these
counterarguments could have served as reasons for the alternative standpoint. Once
used as counterarguments, the need to include additional reasons for the alternative
standpoint is reduced.
Producing a greater number of elaborated reasons for one’s standpoint certainly
results in a more persuasive argument (Bensley & Haynes, 1995; Ferretti et al.,
2000; Graham & MacArthur, 1988). Participants who learned the AS strategy
learned to develop additional supporting reasons and provide concrete examples of
these reasons. However, their essays were only somewhat more persuasive than they
were before instruction because they failed to address the alternative perspective.
Like participants in the control condition, most of those in the AS condition did not
generate any counterarguments, rebuttals, or alternative standpoints. This is
consistent with previous findings that students often fail to consider alternative
perspectives, i.e., their written arguments evidence a clear my-side bias (Ferretti
et al., 2000; Knudson, 1992; Leitão 2003; MacCann 1989; Nussbaum & Kardash,
2005; Perkins et al., 1991). Our findings show that the ASCQ strategy was
successful in helping participants overcome my-side bias by teaching participants to
anticipate potential counterarguments and rebut them.
Both strategies had a positive impact on participants’ revising behavior. Partic-
ipants who learned the AS strategy added more reasons to support their own standpoint
when revising their essays, and those who learned the ASCQ strategy made many
revisions that included counterarguments, alternative standpoints, and rebuttals. In
contrast, participants in the control condition did not effectively revise their essays.
This finding is consistent with the extant literature (Faigley & Witte, 1981; Graves &
Murray, 1980; MacArthur et al., 1991; Sommers, 1980), which shows that novice
writers ignore the revision process or have little idea about how to do it well, especially
when they lack knowledge of genre considerations (Scardamalia & Bereiter, 1986).
Furthermore, learning the ASCQ and AS strategies impacted both drafts of their essays. We noted, for example, that some participants in the ASCQ condition
explicitly justified the reasonableness of their arguments by anticipating counterar-
guments, addressing alternative standpoints, and rebutting these potential criticisms
while writing their first drafts. This pattern is educationally desirable (MacArthur in
press) because standards for good writing acquired during the revising process should
positively impact the quality of students’ writing.
Most of the strategy instruction research has focused on teaching argumentative
writing to late elementary and middle school (grades 5–9) students with LD. The
existing literature documents the powerful effects of strategy instruction for lower
Teaching critical questions about argumentation 87
123
achieving and younger writers (Graham & Perin, 2007; Graham et al., in press). As
mentioned earlier, there is a dearth of research about the effects of SRSD instruction at
the college level. This is unfortunate because argumentation is the primary skill essential
to college and career success. Regardless of their disciplinary focus, college students
must learn to participate in academic discourse, which is inherently argumentative
(Graff, 2003). The current study contributes to our understanding of teaching
argumentative writing at the college level. Our findings show that strategy instruction
is an effective intervention for normally achieving college students when the strategy
supports the acquisition of cognitively challenging knowledge and skills. In this study,
this includes knowledge about argumentation schemes and the application of standards
for written arguments, i.e., asking and answering critical questions.
Nevertheless, future research should address methods for simplifying the Ask and
Answer Critical Questions strategy for younger students and students with LD.
Ferretti et al. (2009) found that regardless of goal condition, disability status, or
grade, fourth- and sixth-grade students overwhelmingly used the argument from
consequences scheme in their argumentative essays about a controversial policy
issue. Other argumentative schemes, including argument from slippery slope,
argument from example, and argument from verbal classification, were presented to
support the argument from consequences scheme, but were used infrequently. These
concepts are challenging, but learning critical questions pertaining to these schemes
may encourage younger writers to anticipate alternative perspectives. Nussbaum
and Edwards (2011) rephrased the critical questions to make them succinct. For
example, their handout included some critical questions for argument from
consequences: (1) What’s the likelihood? (2) How important? (3) How do you
know? This could help reduce the complexity of learning about critical questions,
but Nussbaum and Edwards (2011) did not introduce the concept of argumentation
schemes to students. Future research should explore how to support students’
acquisition of commonly used argumentation schemes.
In addition, the sample size was relatively small because individual participants
in the ASCQ and AS conditions were each taught to a criterion of mastery.
Therefore, future investigations about the efficacy of whole-class instruction
involving the ASCQ strategy are warranted. Finally, we used a sample of
convenience because the college students, all of whom were either majors or minors
in a teacher education program, received course credit for their participation. This
may limit the generalizability of our findings. For this reason, future studies should
be designed to assess the efficacy of SRSD instruction with college students drawn
from a wider pool of participants. Nevertheless, our findings suggest that an SRSD
revising strategy that targets critical standards of evaluation is a promising method
for enhancing college students’ argumentative essays.
References
Applebee, A. N., Langer, J. A., Mullis, I. V. S., Latham, A. S., & Gentile, C. A. (1994). NAEP 1992: Writing report card. Washington, DC: US Government Printing Office.
Aristotle (1991). The art of rhetoric (H. Lawson-Tancred, Trans.). London: Penguin Classics.
88 Y. Song, R. P. Ferretti
123
Bensley, D. A., & Haynes, C. (1995). The acquisition of general purpose strategic knowledge for
argumentation. Teaching of Psychology, 22, 41–45. Coirer, P., Andriessen, J., & Chanquoy, L. (1999). From planning to translating: The specificity of
argumentative writing. In J. Andriessen & P. Coirier (Eds.), Foundations of argumentative text processing (pp. 1–28). Amsterdam: Amsterdam University Press.
De La Paz, S., & Graham, S. (1997a). Strategy instruction in planning: Effects on the writing performance
and behavior of students with learning disabilities. Exceptional Children, 63, 167–181. De La Paz, S., & Graham, S. (1997b). Effects of dictation and advanced planning instruction on the
composing of students with writing and learning problems. Journal of Educational Psychology, 89, 203–222.
Deatline-Buchman, A., & Jitendra, A. K. (2006). Enhancing argumentative essay writing of fourth-grade
students with learning disabilities. Learning Disability Quarterly, 29, 39–54. Faigley, L., & Witte, S. (1981). Analyzing Revision. College Composition and Communication, 32,
400–414.
Felton, M., & Kuhn, D. (2001). The development of argumentative discourse skill. Discourse Processes, 32(2&3), 135–153.
Ferretti, R. P., Andrews-Weckerly, S., & Lewis, W. E. (2007). Improving the argumentative writing of
students with learning disabilities: Descriptive and normative considerations. Reading and Writing Quarterly, 23, 267–285.
Ferretti, R. P., Lewis, W. E., & Andrews-Weckerly, S. (2009). Do goals affect the structure of students’
argumentative writing strategies? Journal of Educational Psychology, 101, 577–589. Ferretti, R. P., MacArthur, C. A., & Dowdy, N. S. (2000). The effects of an elaborated goal on the
persuasive writing of students with learning disabilities and their normally achieving peers. Journal of Educational Psychology, 92, 694–702.
Fitzgerald, J., & Markman, L. (1987). Teaching children about revision in writing. Cognition and Instruction, 4, 3–24.
Graff, G. (2003). Clueless in academe: How schooling obscures the life of the mind. Yale: Yale University Press.
Graham, S., & Harris, K. R. (1989). A components analysis of cognitive strategy training: Effects of
learning disabled students’ compositions and self-efficacy. Journal of Educational Psychology, 81, 353–361.
Graham, S., & Harris, K. R. (2005). Writing better: Effective strategies for teaching students with learning difficulties. Baltimore, MD: Paul H. Brookes.
Graham, S., Harris, K. R., & McKeown, D. (in press). The writing of students with LD and a meta-
analysis of SRSD writing intervention studies: Redux. In L. Swanson, K. R. Harris, & S. Graham
(Eds.). Handbook of research in learning disabilities (2nd edn.). New York: Guilford. Graham, S., & MacArthur, C. (1988). Improving learning disabled students’ skills at revising essays
produced on a word processor: Self-instruction strategy training. Journal of Special Education, 22, 133–152.
Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve writing of adolescents in middle and high schools. New York: Carnegie Corporation.
Graves, D. H., & Murray, D. M. (1980). Revision: In the writer’s workshop and in the classroom. Journal of Education, 162, 38–56.
Kinsler, K. (1990). Structured peer collaboration: Teaching essay revision to college students needing
writing remediation. Cognition and Instruction, 7, 303–321. Knudson, R. E. (1992). The development of written argumentation: An analysis and comparison of
argumentative writing at four grade levels. Child Study Journal, 22, 167–184. Kuhn, D. (1991). The skills of argument. Cambridge: Cambridge University Press. Leitão, S. (2003). Evaluating and selecting counterarguments: Studies of children’s rhetorical awareness.
Written Communication, 20(3), 269–306. MacArthur, C. A. (in press). Evaluation and revision processes in writing. In V. W. Berninger (Ed.), Past,
present, and future contributions of cognitive writing research to cognitive psychology. New York: Psychology Press.
MacArthur, C. A., Schwartz, S., & Graham, S. (1991). Effects of a reciprocal peer revision strategy in
special education classrooms. Learning Disabilities Research and Practice, 6, 201–210. MacCann, T. M. (1989). Student argumentative writing: Knowledge and ability at three grade levels.
Research in the Teaching of English, 23, 62–76.
Teaching critical questions about argumentation 89
123
Nussbaum, E. M., & Edwards, O. V. (2011). Critical questions and argument stratagems: A framework
for enhancing and analyzing students’ reasoning practices. Journal of the Learning Sciences, 20, 443–488.
Nussbaum, M. E., & Kardash, C. M. (2005). The effects of goal instructions and text on the generation of
counterarguments during writing. Journal of Educational Psychology, 97, 157–169. Perkins, D. N., Faraday, M., & Bushey, B. (1991). Everyday reasoning and the roots of intelligence. In J.
F. Voss, D. N. Perkins, & J. W. Segal (Eds.), Informal reasoning and education (pp. 83–105). Hillsdale, NJ: Erlbaum.
Persky, H. R., Daane, M. C., & Jin, Y. (2003). The nation’s report card: Writing 2002 (US Department of Education Publication No. NCES 2003-529). Washington, DC: US Government Printing Office.
Rushton, J., Brainerd, C., & Pressley, M. (1983). Behavioral development and construct validity: The
principle of aggregation. Psychological Bulletin, 94, 18–38. Scardamalia, M., & Bereiter, C. (1986). Research on written composition. In M. C. Wittrock (Ed.),
Handbook of research on teaching (pp. 778–803). New York: Macmillan. Sexton, M., Harris, K. R., & Graham, S. (1998). Self-regulated strategy development and the writing
process: Effects on essay writing and attributions. Exceptional Children, 64(3), 295–311. Sommers, N. (1980). Revision strategies of student writers and experienced adult writers. College
Composition and Communication, 31, 378–388. The Nation’s Report Card: Writing 2007. (2007). Retrieved February 25, 2011, from http://nces.ed.gov/
nationsreportcard/pdf/main2007/2008468.pdf.
van Eemeren, F. H., & Grootendorst, R. (1992). Argumentation, communication, and fallacies: A pragma-dialectical perspective. Mahwah, NJ: Erlbaum.
van Eemeren, F. H., & Grootendorst, R. (2004). A systematic theory of argumentation: The pragma- dialectical approach. Cambridge, England: Cambridge University Press.
van Eemeren, F. H., Grootendorst, R., & Henkemans, F. S. (1996). Fundamentals of argumentation theory: A handbook of historical backgrounds and contemporary developments. Mahwah, NJ: Erlbaum.
van Eemeren, F. H., Grootendorst, R., & Henkemans, F. S. (2002). Argumentation: Analysis, evaluation, presentation. Mahwah, NJ: Erlbaum.
van Eemeren, F. H., Grootendorst, R., Jackson, S., & Jacobs, S. (1993). Reconstructing argumentative discourse. Tuscaloosa: University of Alabama Press.
Walton, D. (1992). Plausible argument in everyday conversation. Albany, NY: State University of New York Press.
Walton, D. N. (1996). Argumentation schemes for presumptive reasoning. Mahwah, NJ: Lawrence Earlbaum Associates.
Walton, D., Reed, C., & Macagno, F. (2008). Argumentation schemes. Cambridge: Cambridge University Press.
90 Y. Song, R. P. Ferretti
123
Copyright of Reading & Writing is the property of Springer Science & Business Media B.V. and its content
may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express
written permission. However, users may print, download, or email articles for individual use.
