Accountability Under Constraint:
sososays123
CopyofTheRelationshipBetweenCollectiveBargainingAgreementsandCaliforniaSchools.pdf
Accountability Under Constraint: The Relationship Between Collective Bargaining Agreements and California
Schools’ and Districts’ Performance Under No Child Left Behind
Katharine O. Strunk Andrew McEachin
University of Southern California
The authors examine how the collective bargaining agreement (CBA) negoti- ated between teachers’ unions and districts is associated with schools’ and dis- tricts’ performance under accountability pressures in California. They find that CBA restrictiveness is associated with the increased likelihood that districts will be in Program Improvement (PI) and at higher levels of PI, and with lower school- and district-level proficiency and graduation rates. They also show that strong contract schools and districts that have higher proportions of minority, low-income, and low-achieving student are even less likely to meet performance targets and have even lower proficiency rates.
KEYWORDS: accountability, collective bargaining agreements, education policy, teachers’ unions
KATHARINE O. STRUNK is an assistant professor of education and policy in the Rossier School of Education, and by courtesy the School of Policy, Planning and Development at the University of Southern California, 901E Waite Philips Hall, 3470 Trousdale Parkway, Los Angeles, CA 90089; e-mail: [email protected]. Her research cen- ters on K–12 education policy, focusing on teacher labor markets, education governance, and education finance. She is motivated by a desire to understand how teachers’ unions and the collective bargaining agreements they negotiate with school districts impact school and district operations and achievement, and in particular how they may con- strain or facilitate districts’ and schools’ responses to accountability incentives.
ANDREW MCEACHIN is a Provost Fellow and PhD student in education and policy in the Rossier School of Education at the University of Southern California, 500 Waite Philips Hall, 3470 Trousdale Parkway, Los Angeles, CA 90089; e-mail: [email protected]. His research centers on K–12 education policy, focusing on accountability, program evaluation, and special education. The 2 years he spent as a Teach for America corps member sparked his interest in better understanding how school and district contex- tual factors (e.g., teachers’ unions) affect student achievement in an era of high-stakes accountability.
American Educational Research Journal
August 2011, Vol. 48, No. 4, pp. 871–903
DOI: 10.3102/0002831211401006
� 2011 AERA. http://aerj.aera.net
I. Introduction
Accountability policies create systems of inducements and sanctions that are intended to incentivize teachers, school administrators, and district ad- ministrators to improve student performance (Elmore, Abelmann, & Fuhrman, 1996; Ladd, 1996). Such policies became increasingly widespread throughout the 1990s, culminating with the 2002 implementation of the No Child Left Behind Act (NCLB), which created a federal accountability system that tied important Title I monies to student achievement on standardized tests at the school and district levels. Specifically, NCLB requires every sub- group of students identified within a school and a district to meet achieve- ment targets in math and English language arts (ELA), as well as to meet other targets including set student participation and graduation rates. Schools and districts that fail to meet these targets face increasing conse- quences as they enter and then progress to higher levels of ‘‘Program Improvement’’ status. With the proposed reauthorization of the Elementary and Secondary Education Act seemingly maintaining the key accountability provisions set by NCLB, it appears that accountability policies will continue to define federal, and as a result state, education policy (U.S. Department of Education, 2010).
The theoretical framework behind the use of accountability policies holds that the threat (and use) of sanctions, at times combined with support for failing schools, will incentivize teachers, principals, superintendents, and other school staff to implement reforms that will increase student achieve- ment (Figlio & Ladd, 2008; Ladd, 1996). Implicit in this theory of action is the assumption that districts and schools have sufficient capacity to allow and implement reforms that will greatly enhance student achievement. Not only must teachers and administrators have the capacity to quickly improve instruction to reach these goals, but schools and districts must also have the ability to change their structures and practices. However, not all schools and districts that face accountability pressures have sufficient autonomy and flex- ibility to make changes that they believe will have the most impact on stu- dent achievement.
Although the ways in which external actors and policies may constrain school and district administrators and teachers from implementing education reforms is not a new topic of interest, little attention has been paid to factors that may restrict administrative flexibility to respond to accountability incen- tives (Figlio & Ladd, 2008; Hill & Lake, 2002). In this article, we examine how one potential source of constraint—the collective bargaining agreement (CBA, or contract) negotiated between teachers’ unions and district admin- istrators or school boards—is associated with schools’ and districts’ perfor- mance under NCLB accountability pressures. Using a measure of contract restrictiveness combined with accountability and academic performance data from California, we ask if contract restrictiveness is associated with
Strunk, McEachin
872
districts’ and schools’ likelihood of meeting NCLB-set achievement targets and with their performance on standardized tests and graduation rates. We also examine whether contract restrictiveness is more or less associated with accountability outcomes in ‘‘hard-to-staff’’ districts and schools—those with higher proportions of minority, low-income, and low-achieving students.
The article proceeds as follows. Section II reviews the literature and con- ceptual background behind accountability policies, focusing on the ways in which CBAs may constrain district administrators from implementing NCLB- mandated and other reforms in California. Section III outlines our data and explains our measures of contract restrictiveness and school and district out- comes. Sections IV and V ask if contract restrictiveness is associated with dis- trict and school performance under NCLB and whether contract restrictiveness is more or less associated with outcomes in hard-to-staff dis- tricts and schools. Section VI discusses study limitations, and Section VII con- cludes with a discussion of our results and policy implications.
II. How Might Contracts Be Associated With Achievement
and Accountability Outcomes?
Extant research shows that accountability policies appear to increase student achievement, both on high-stakes tests as well as low-stakes tests such as the National Assessment of Educational Progress (NAEP) (Carnoy & Loeb, 2002; Figlio & Rouse, 2006; Hanushek & Raymond, 2005; Jacob, 2005; Ladd & Lauen, 2010; Springer, 2008). Although there is some concern about the possibility of ‘‘gaming the system’’ to increase student achievement (e.g., Cullen & Reback, 2006; Jacob & Levitt, 2003), the positive impact of accountability policies on student performance on low-stakes tests imply that such policies do, in fact, improve average student achievement, although test score gains appear to be modest (Figlio & Ladd, 2008). Moreover, some research suggests that schools and districts are achieving these performance gains by implementing instructional and operational re- forms that benefit students and increase their performance (Chaing, 2009; Hamilton, Berends, & Stecher, 2005; Rouse, Hannaway, Goldhaber, & Figlio, 2007).
The theoretical framework upon which standards-based accountability policies rest assumes that districts and schools have sufficient capacity to implement the necessary changes that will enable schools and districts to meet the student achievement goals placed upon then. In order to make the reforms discussed in the studies mentioned above, district and school ad- ministrators need not only to know how to generate instructional change and enhance student performance but also to have sufficient flexibility to enact such changes. However, district and school administrators have vary- ing levels of autonomy and flexibility, and not all administrators will have
Accountability Under Constraint
873
the ability to respond to accountability incentives by implementing reforms that are intended to increase student achievement.
Although researchers have called for a better understanding of how operational flexibility, or the lack thereof, impacts districts’ and schools’ abil- ities to implement accountability reforms (Figlio & Ladd, 2008; Petrelli, 2002), little research has explored the various policy and contextual factors that may make districts and schools more or less successful at responding to accountability pressures. In one of the few studies that links accountability programs and policy flexibility to achievement, Loeb and Strunk (2007) examine how the relationship between accountability policies and student achievement on the NAEP varies when there is more or less local control over decision making. They find that accountability policies are more asso- ciated with student achievement gains in states that allow for more local con- trol over resource allocation and generation. There also are many other feasible mechanisms of constraint that may restrict administrators’ abilities to respond to accountability incentives, including heavily codified state reg- ulations, mayoral control, special interest groups, and contracts negotiated between labor groups and district administrators.
Among these potential sources of constraints, CBAs negotiated between school district personnel and the teachers’ unions have been highlighted as potentially important because they control essential school- and district-level policies that may dictate the set of feasible innovations that administrators can undertake (Hill & Lake, 2002; Petrelli, 2002; Price, 2009; Strunk & Grissom, 2010). For example, these contracts can and often do regulate per- sonnel and staffing decisions, professional development and compensation practices, class size policies, the frequency of school and district meetings, the availability and duration of teacher collaboration time, the number of class preparations a teacher may undertake, the ways in which teachers are evaluated, the scheduling of the school day and year, and more (Hess & Kelly, 2006; Hess & Loup, 2008; Moe, 2009; Strunk, 2009). These policies may be targets for reform as districts and schools attempt to increase student achievement in the face of increasing accountability pressures.
Although there is a good deal of discussion about the mechanisms through which CBAs may constrain district and school administrators in ways that impact important educational outcomes, very little is known about the ways in which contracts actually do or do not impact district flexibility and/or student achievement, much less how they interact with attempted educational reforms. Some research finds that stronger contracts are nega- tively associated with student achievement (Moe, 2009; Strunk, in press) and implies that stronger contracts by their very nature impede administra- tors (Hess & Loup, 2008), although there is as yet no causal evidence to show that restrictive contracts cause lower student performance, or that they do in fact impede administrators’ actions. Some research examines the impacts of contract restrictiveness on the ways districts spend their
Strunk, McEachin
874
limited resources. To this end, Eberts (1983), Eberts and Stone (1984), and Strunk (in press) show that districts with more restrictive contracts spend their resources differently than districts with more flexible contracts. Other studies explore how contracts regulate district policies regarding staffing and personnel allocation, which may in turn impact student achievement (Koski & Horng, 2007; Levin, Mulhern, & Schunck, 2005; Weisberg et al., 2009).
How CBAs might impact school and district performance under account- ability pressures has received even less attention in the literature. Hill and Lake (2002) study the implementation of standards and accountability poli- cies in Washington state and draw attention to CBA regulations that dictate teacher assignment and budgets. They note that some policymakers felt that ‘‘school-level accountability [is] impossible: No organization can be responsible for performance if it cannot select staff members and control money’’ (p. 217). Recent research on CBAs from districts in three states, how- ever, finds that there are multiple flexibilities written into the contracts that enable high school principals to enact necessary reforms even in the face of seemingly restrictive bargaining agreements (Price, 2009). However, we have found no research to date that specifically considers the relationship between the level of constraint present in contracts and school and district performance on accountability indicators.
Given that CBAs are frequently discussed as potential constraints on educator actions, and that the efficacy of accountability policies in raising student achievement in many ways relies on educators maintaining the abil- ity to enact reforms that will raise student achievement, we believe that a brief discussion of the ways in which CBAs may specifically constrain dis- tricts and schools under NCLB is warranted.1 NCLB outlines potential sanc- tions that states can or must impose on districts and schools that are deemed ‘‘failing’’ and placed in Program Improvement (PI). As schools and districts continue to fail to make ‘‘Adequate Yearly Progress’’ (AYP), they advance to higher levels of PI, and with these designations receive increasing levels of intervention from the state.2 Sanctions imposed on schools and districts in PI become more intrusive into schools’ and districts’ operations each year of PI, dramatically increasing in PI3. NCLB also outlines corrective actions that states can choose to impose on districts in PI3, includ- ing replacing the district personnel relevant to the district’s failure to make AYP, removing schools from jurisdiction of the district, establishing alterna- tive arrangements for the governance and supervision of schools, abolishing or restructuring the district, and deferring programmatic funds or reducing administrative funds.
Inherent in NCLB’s structure of punishments and rewards is the assump- tion that sanctions and policy interventions linked to school and district op- erations will help schools and districts meet accountability standards. However, for better or worse, CBAs may constrain districts from taking
Accountability Under Constraint
875
one or more of the possible PI3 corrective actions outlined by NCLB. For instance, one of the corrective action options that districts can choose to implement for their schools is to ‘‘replace school staff (those relevant to school’s failure to make AYP)’’ (California Department of Education [CDE], 2006, p. 15). The CDE, however, warns that
this option requires careful consideration and consultation with the Human Resources department regarding contractual issues and con- sultation with the local bargaining organization. NCLB specifically provides that California shall not ‘‘alter or otherwise affect the rights, remedies, and procedures afforded school or school district employ- ees under . . . the terms of collective bargaining agreements, memo- randa of understanding, or other agreements between such employees and their employers.’’ (CDE, 2006, p. 22)
The PI4 corrective action option to ‘‘replace all or most staff relevant to AYP failure’’ may also be made impossible by districts’ CBAs; again, the CDE (2006) cautions that ‘‘there may be contract implications’’ (p. 44). CBAs may also limit districts’ abilities to ‘‘extend the school year or day.’’ The CDE warns that districts must ‘‘determine whether significantly increasing the school year and/or school day will require a modification or waiver of the current collective bargaining agreement’’ (p. 24).
Many NCLB sanctions for schools and districts are less obvious in the ways that they require district administrations and local teachers’ associa- tions to cooperate and work together, and possibly necessitate contract flex- ibility. For instance, PI-associated sanctions require that proportions of federal Title I funds be reserved for specific professional development activ- ities and supplementary education services, but these funds already may be allocated in ways that are set by the CBA. Administrators may require fairly flexible contracts in order to comply with this and other NCLB sanctions.
CBAs can also restrict state policymakers and district and school admin- istrators from implementing reforms that are not directly required or pro- posed by NCLB but that might be employed in an attempt to help schools and districts meet NCLB targets. For instance, districts may wish to reform teacher evaluation policies, or they may wish to alter staffing provisions in order to place specific teachers in certain classrooms or remove poorly per- forming teachers from the classroom, regardless of seniority. Both of these areas, however, are dictated by regulations in the CBAs, and therefore very restrictive CBAs may make it impossible for administrators or policy- makers to implement such reforms, which may in turn contribute to schools’ and districts’ inability to meet performance standards.
The above discussion outlines the underlying theory of action of accountability policies and the potential for restrictive contracts to affect ad- ministrators’ abilities to enact reforms. However, it is not clear at the outset that the ways in which CBAs constrain administrators from implementing
Strunk, McEachin
876
accountability-driven reforms necessarily will have adverse consequences for school and district performance. Many regulations included in CBAs are intended to protect students and teachers and improve student learning. If contracts limit district and school administrators’ abilities to respond to accountability incentives by impeding administrators’ abilities to reform their schools and districts in ways that will help students, then districts and schools in districts with more restrictive contracts will have lower student performance and will be less likely to meet NCLB targets. Conversely, if these contracts dictate district policy in ways that ensure that the only re- forms enacted serve teachers’ and students’ best interests and improve edu- cational outcomes, then districts and schools in districts with more restrictive contracts will have higher student performance and be more likely to meet achievement, participation, and graduation targets.
It is also possible that contracts restrict certain kinds of districts and schools more than others, and that this increased level of constraint may dif- ferentially impact districts’ and schools’ student achievement levels and their abilities to meet accountability program requirements. Specifically, districts and schools with more difficult working conditions may be particularly affected by CBA-determined policy rigidity. This may occur because districts with equally strong CBAs differentially enforce these contracts depending on the specific district context (Koski & Horng, 2007; Price, 2009). Many poten- tially restrictive provisions within CBAs may be less frequently or strictly en- forced in districts and schools with relatively easy working conditions because teachers in these schools and districts may be content and less likely to feel the need to utilize the protections that have been negotiated into their contracts. On the other hand, administrators and teachers who work in very difficult working conditions may be more likely to enforce contract provi- sions to ensure that teachers receive due protections or that administrators ensure proper compliance. Alternately, schools or districts with particularly difficult working conditions may attempt to provide nonpecuniary benefits in the form of greater enforcement of contract protections in order to attract and retain teachers to hard-to-staff schools or districts.
Although there is a growing body of literature that explores the relation- ships between CBAs, district policies, and student achievement, little is understood about the relationship between CBAs and districts’ abilities to implement accountability reforms or districts’ or schools’ progress under accountability programs. In addition, little is known about how the impact of contract restrictiveness on district operations, flexibility, or student out- comes may vary in different kinds of districts. In order to better understand how CBAs may impact student achievement, specifically in schools and dis- tricts under severe accountability pressures, careful study of these relation- ships is warranted. This study is the first to explore how the degree to which school district CBAs restrict district and school administrators is asso- ciated with student performance under accountability pressures instituted by
Accountability Under Constraint
877
NCLB. In addition, we are the first to explore whether the impact of contract restrictiveness on school and district outcomes under accountability pres- sures varies by district context. We leave it to future work to address how CBAs impact the specific actions taken and policies implemented by school and district administrators.
III. Data
We combine a measure of contract strength with California district and school accountability performance measures and a set of school and district characteristics. Our final sample includes 465 California school districts, rep- resenting 82% of California districts with four or more schools.3 We use as the larger population districts with four or more schools because many of the smaller school districts have quite different contracts than their larger neighbors, simply because many contract policies do not impact districts with only a few schools as much as they might their neighbors with a multi- tude of schools. Summary statistics are provided in Table 1.
Collective Bargaining Agreement Restrictiveness
We borrow our measure of contract restrictiveness from Strunk and Reardon (2010), who generated it through the use of a partial independence item response (PIIR) approach using self-collected data from the same 465 California public school district teachers’ union contracts from the 2005– 2006 school year. California’s CBAs vary dramatically in their scope, tone, and content. While some address only the bare regulations necessary to employ teachers, such as compensation, the length of the school year, and vacation/leave policies, others contain much more detail regarding the ex- pectations of teachers and administrators, including the length of time a teacher may work each day, the specific hours a teacher can be on the school campus, the number of meetings a teacher must attend, the number of students a teacher may instruct, the number of classes or class prepara- tions a teacher can be assigned, the frequency of and the ways in which teachers are evaluated, and so on. Those contracts that regulate even the most detailed aspects of the school day and teachers’ work are likely to con- strain district and school administrators to a greater extent than do contracts that leave the details to the specific administrators to decide. A reliable mea- sure of contract restrictiveness would assess the degree to which a contract restricts administrators’ actions while taking into account the reality that not all regulations included in a CBA restrict administrators—some restrict teach- ers, as well. As described in Strunk and Reardon (2010) and Reardon and Raudenbush (2006), the measure we use in this study does this by using a PIIR model that measures the underlying latent restrictiveness of a teachers’ union contract.
Strunk, McEachin
878
T a b le
1
S u
m m
a ry
S ta
ti s ti
c s
2 0 0 6
2 0 0 7
2 0 0 8
2 0 0 9
N M
e a n
S D
N M
e a n
S D
N M
e a n
S D
N M
e a n
S D
D e p e n d e n t v a ri a b le
s
M a th
p ro
fi c ie
n c y
ra te
4 6 2
0 .5
2 0
0 .1
4 2
4 6 1
0 .5
2 2
0 .1
3 8
4 6 0
0 .5
4 1
0 .1
3 5
4 5 6
0 .5
7 3
0 .1
3 0
G ra
d u a ti o n
ra te
3 1 2
0 .9
1 2
0 .0
8 1
3 1 4
0 .9
0 3
0 .0
7 8
3 1 1
0 .8
6 9
0 .1
1 6
3 0 6
0 .8
7 2
0 .0
8 4
P ro
g ra
m Im
p ro
v e m
e n t (P
I) st
a tu
s 4 6 2
0 .2
3 9
4 5 2
0 .2
3 4
4 5 1
0 .2
8 1
4 4 8
0 .3
7 4
M a th
p ro
fi c ie
n c y
ra te
(s c h o o l)
6 ,2
9 3
0 .5
0 9
0 .1
9 8
6 2 4 8
0 .5
1 5
0 .1
9 5
6 ,2
0 7
0 .5
3 6
0 .1
9 0
6 ,1
1 8
0 .5
6 9
0 .1
8 8
G ra
d u a ti o n
ra te
(s c h o o l)
1 ,4
5 9
0 .8
5 8
0 .1
7 5
1 ,4
6 0
0 .8
3 5
0 .2
0 5
1 ,4
4 9
0 .8
4 6
0 .1
5 5
1 ,3
7 3
0 .8
4 9
0 .1
3 8
P I
st a tu
s (s
c h o o l)
4 ,3
3 2
0 .3
2 7
4 ,2
8 4
0 .4
0 5
4 ,2
2 7
0 .3
9 7
4 ,1
0 0
0 .4
1 2
In d e p e n d e n t v a ri a b le
s
L a g g e d
m a th
p ro
fi c ie
n c y
ra te
4 6 2
0 .4
9 1
0 .1
4 6
4 6 1
0 .5
2 0
0 .1
4 2
4 6 0
0 .5
2 2
0 .1
3 8
4 5 6
0 .5
4 2
0 .1
3 5
C o n tr a c t re
st ri c ti v e n e ss
4 6 2
2 0 .0
0 1
0 .5
0 5
4 6 1
2 0 .0
0 1
0 .5
0 5
4 6 0
2 0 .0
0 1
0 .5
0 5
4 5 6
0 .0
0 0
0 .5
0 6
# si
g n if ic
a n t su
b g ro
u p s
4 6 2
2 9 .8
1 6
8 .7
0 0
4 6 1
2 9 .7
7 9
8 .6
2 0
4 6 0
2 9 .9
7 6
8 .5
4 1
4 5 6
2 9 .9
3 6
8 .5
0 5
S m
a ll
d is
tr ic
t 4 6 2
0 .2
4 7
4 6 1
0 .2
5 0
4 6 0
0 .2
5 0
4 5 6
0 .2
5 1
L a rg
e d is
tr ic
t 4 6 2
0 .2
5 0
4 6 1
0 .2
5 0
4 6 0
0 .2
4 7
4 5 6
0 .2
4 7
% m
in o ri ty
4 6 2
0 .4
5 8
0 .2
7 2
4 6 1
0 .4
6 9
0 .2
7 4
4 6 0
0 .4
7 5
0 .2
7 4
4 5 6
0 .4
7 9
0 .2
7 3
% lo
w in
c o m
e 4 6 2
0 .4
2 6
0 .2
5 1
4 6 1
0 .4
4 3
0 .2
5 2
4 6 0
0 .4
5 2
0 .2
5 2
4 5 6
0 .4
6 3
0 .2
5 5
H ig
h sc
h o o l d is
tr ic
t 4 6 2
0 .1
0 5
4 6 1
0 .1
0 5
4 6 0
0 .1
0 5
4 5 6
0 .1
0 4
U n if ie
d d is
tr ic
t 4 6 2
0 .5
5 1
4 6 1
0 .5
5 1
4 6 0
0 .5
5 1
4 5 6
0 .5
5 5
U rb
a n
d is
tr ic
t 4 6 2
0 .2
5 2
4 6 1
0 .2
5 2
4 6 0
0 .2
5 2
4 5 6
0 .2
5 1
R u ra
l d is
tr ic
t 4 6 2
0 .0
4 4
4 6 1
0 .0
4 4
4 6 0
0 .0
4 4
4 5 6
0 .0
4 4
D is
tr ic
t p e r
p u p il
e x p e n d it u re
s 4 6 1
9 ,1
5 6
1 ,9
8 4
4 5 9
9 ,6
6 1
2 ,0
8 5
4 5 8
1 0 ,7
1 1
2 ,7
9 1
4 5 2
1 1 ,0
9 6
2 ,8
4 8
L a g g e d
m a th
p ro
fi c ie
n c y
ra te
(s c h o o l)
6 ,6
8 3
0 .4
7 8
0 .2
1 1
6 ,8
0 9
0 .5
0 3
0 .2
0 1
6 ,8
9 1
0 .5
0 9
0 .1
9 7
6 ,9
6 2
0 .5
3 1
0 .1
9 3
% m
in o ri ty
(s c h o o l)
6 ,7
4 8
0 .5
5 3
0 .3
0 1
6 ,6
8 1
0 .5
6 3
0 .3
0 1
6 ,6
3 3
0 .5
6 6
0 .3
0 1
6 ,5
7 3
0 .5
7 1
0 .3
0 0
% lo
w in
c o m
e (s
c h o o l)
6 ,7
3 9
0 .5
1 1
0 .3
0 2
6 ,6
7 6
0 .5
2 4
0 .3
0 2
6 ,6
2 4
0 .5
3 4
0 .3
0 0
6 ,5
6 9
0 .5
4 2
0 .3
0 2
H ig
h sc
h o o l (s
c h o o l)
6 ,7
4 8
0 .2
1 2
6 ,6
8 1
0 .2
1 2
6 ,6
3 3
0 .2
1 2
6 ,5
7 3
0 .2
1 2
M id
d le
sc h o o l (s
c h o o l)
6 ,7
4 8
0 .1
5 2
6 ,6
8 1
0 .1
5 3
6 ,6
3 3
0 .1
5 2
6 ,5
7 3
0 .1
5 1
U rb
a n
sc h o o l (s
c h o o l)
6 ,7
4 8
0 .4
4 6
6 ,6
8 1
0 .4
4 6
6 ,6
3 3
0 .4
4 5
6 ,5
7 3
0 .4
4 7
R u ra
l sc
h o o l (s
c h o o l)
6 ,7
4 8
0 .0
1 9
6 ,6
8 1
0 .0
1 8
6 ,6
3 3
0 .0
1 8
6 ,5
7 3
0 .0
1 8
879
Specifically, this method follows Reardon and Raudenbush’s (2006) development of the PIIR model as a generalized hybrid of a discrete-time hazard model and a Rasch model that adjusts for the conditional structure of survey response patterns. Their model, like other applications of item response theory (IRT) models to survey response patterns, assumes that each person who responds to a survey possesses some latent attribute that affects the probability of an affirmative response to each survey question. However, Reardon and Raudenbush adjust for the likelihood that many sur- vey questions are not locally independent but, instead, are predicated on an affirmative response to an earlier survey question. Strunk and Reardon (2010) extend the method to examine district teachers’ union contracts, which can be seen as a set of responses to items ‘‘asked’’ on a larger survey. They conceive of each specific regulation that can be included in a CBA as an item. Contracts that contain the regulation have ‘‘affirmative’’ responses to the item, and contracts that do not include the regulation have ‘‘negative’’ responses. Greater detail on the generation of the PIIR measure can be found in Strunk and Reardon; however, the measure can be roughly thought of as a summation of items with weights, such that any item that restricts the district in any way is coded as a 1; and if there are varying levels of restric- tiveness within a contract regulation, each growing level of constraint is also coded as a 1. For example, if a contract includes a regulation that district ad- ministrators must consider seniority in transfer decisions, that provision, which restricts districts from placing personnel totally according to their own wishes, would be coded as a 1. Then, if the contract further specifies that seniority must be the deciding factor in transfer decisions, that provision is coded with another 1. The PIIR model then effectively assesses the restric- tiveness of the contract by considering all the 1s in each contract and placing greater weight on items that are less frequently found in contracts. Each item can be ranked in order of ‘‘restrictiveness,’’ which is analogous to the diffi- culty of a given test item in traditional IRT models.
The items included in the PIIR model are selected according to standard test theory, winnowing out items that are not associated with the underlying latent trait. This method of item selection and use of the PIIR approach has a number of advantages over other ways of measuring contract strength, two of which are particularly pertinent here. First, the method of item selection allows for an objective and statistically sound approach to measuring con- tract restrictiveness; items are not selected for inclusion based on conjecture or previous assumptions but, rather, are based on unbiased item selection methods. Contracts were initially coded for a total of 639 items spanning all of the subareas of contracts, and then reduced based on the ability of the items to discriminate among contracts, narrowing the set to 334 individ- ual contract provisions. The set of items was then further reduced by running a recursive exploratory Cronbach’s alpha analysis on the items included in our initial model. The item-total correlations produced for each of the 334
Strunk, McEachin
880
items were examined, and items with item-total correlations lower than .25 were dropped, keeping 39 contract items in the final measure of contract restrictiveness that span the breadth of the contract. This process ensures that only the items that capture the same underlying trait of contract restric- tiveness are retained in the final measure.
Second, the use of the PIIR model allows for the creation of a transparent and probabilistically based interval scale along which individual contracts are placed according to their specific level of restrictiveness, as well as stan- dard errors of measurement for each contract. Strunk and Reardon (2010) also assess the reliability of the measure and judge that, with a model-gen- erated reliability of .7, it captures the underlying trait well enough to use in statistical analyses of the relationships between union strength and cova- riates and outcomes of importance. The reliability provides the estimated ratio of true signal to signal-plus-noise in the measure of union strength, and indicates the level to which we are able to differentiate between individ- ual districts according to their estimated union strength ‘‘scores.’’ Summary statistics for this measure of contract restrictiveness are given in Table 1.
As do Strunk (in press) and Strunk and Reardon (2010), we interpret contract restrictiveness as a measure of how greatly the CBA constrains dis- trict administrators in their ability to flexibly respond to situations that arise in the course of day-to-day operations of schools. For example, a highly restrictive contract to district administrators may be one that constrains ad- ministrators from placing teachers in the schools they believe need these teachers the most due to transfer and vacancy regulations within the con- tract. Similarly, a restrictive contract may be one that limits administrators in the frequency of teacher evaluations due to specific clauses within the contract. Such constraint does not always harm students. In fact, contracts that are more restrictive to the district administration can be seen as those that offer more benefits for teachers, and as a result more restrictive contracts may provide benefits to students. Regardless, highly restrictive contracts by construct limit the ability of administrators and teachers to work together or separately to nimbly implement district and school policies.
Accountability Outcomes
We are interested in the multiple outcomes for which NCLB holds schools and districts accountable: (a) the proportion of students who reach proficiency in ELA and math, both overall and by significant subgroups; (b) the proportion of students who participate in ELA and math assessments; (c) high schools’ and unified/high school districts’ graduation rates; and (d) Academic Performance Index (API) score growth. If districts and schools fail to reach the state-set targets for these indicators, they enter and progress to higher levels of PI. We include as outcomes in our models whether or not a school or district is in Program Improvement, the school or district’s year of
Accountability Under Constraint
881
PI, the proportion of students who reach proficiency in math or ELA, and the high school or district’s graduation rate. We use the publicly available AYP, API, and PI data sets from the CDE for the 2005–2006 through 2008–2009 school years. We do not include analyses of whether or not schools and dis- tricts meet API growth or participation rate targets because there is not enough variation in these outcomes to allow for precise estimates of the re- lationships between various covariates of interest and these outcomes.
IV. Is Contract Restrictiveness Associated With District-Level
Accountability Outcomes?
To explore the relationship between contract restrictiveness and district NCLB outcomes, we estimate year fixed effects models of the following form:
Ajt 5 b01b1ðCBA RestrictivenessjÞ1b2ð% Minorityjt�1Þ 1b3ð% Low Incomejt�1Þ 1b4ð% Proficientjt�2Þ1Djt�1d1tt 1njt;
ð1Þ
where Ajt is one of five district outcomes. First, we run Model 1 as a logistic regression, where Ajt is whether or not district j is in PI at time t (1 = in PI). Next, we use an ordinal outcome, in which Ajt indicates the level of PI (0 = not in PI, 1 = PI year 1, 2 = PI year 2, 3 = PI year 3 or 31). Last, we estimate ordinary least squares (OLS) regressions in which Ajt is either the proportion of students in district j who achieve a proficient score in math or ELA or who graduate (if district j is a high school or unified school district). The CBA restrictiveness measure is generated through the use of a PIIR model, with larger values indicating a contract that is more restrictive to district adminis- trators. CBA Restrictiveness data are from only one school year (2005–2006) because CBAs are in place for at least three school years at a time. We attri- bute the 2005–2006 CBA Restrictiveness value to all years in our panel. We are particularly interested in the coefficient b1, which can be interpreted as the relationship between the level of contract restrictiveness and the var- ious district outcomes.
Because extant research has shown that teachers prefer districts and schools with lower proportions of minority, low-income, and low-achieving students (e.g., Hanushek, Kain, & Rivkin, 2004; Lankford, Loeb, & Wyckoff, 2002), we explore the relationship between contract strength and account- ability outcomes in districts that vary according to these student characteris- tics. As such, we include measures of district characteristics that may make the district particularly hard to staff, including the proportion of students who are minorities (African American, Hispanic, American Indian, Filipino, or Pacific Islander), poor (as proxied by whether or not they qualify for the Free- or Reduced-Price Lunch Program), and who achieved a proficient
Strunk, McEachin
882
score in math. We also run all models using the proportion of students who achieved proficiency in ELA, which we do not show because results are sub- stantially the same as those from the models using math proficiency rates. For the logistic and ordinal logistic outcomes, we take all independent var- iables from the year previous to the accountability outcomes except for the previous proficiency rates, which are twice lagged. We do this because a dis- trict’s or school’s PI status in year t is determined by its performance in year t – 1. Djt represents a vector of district-level characteristics to control for important contextual factors that might influence district outcomes, includ- ing the urban or rural location of the district (suburban districts are the ref- erence category), the level of the school district (unified or high school, with elementary as the reference category), and the districts’ per pupil expendi- tures. In addition, we control for district size by including two dichotomous indicators of district enrollment (average daily attendance), with large dis- tricts defined as those in the 75th percentile of enrollment or above, and small districts defined as those in the 25th percentile of enrollment or below. We model size in this manner because a few notable outliers do not allow for the inclusion of district size modeled simply as total enrollment. Last, we include year fixed effects, yt, for each of the school years 2006–2007, 2007–2008, and 2008–2009. We use robust standard errors. We obtain all measures of district demographic data from the Common Core of Data, and the CDE’s AYP, API, and PI data sets for the 2005–2006 through 2008– 2009 school years.
We also ran a number of models estimating the relationship between contract restrictiveness and student achievement for relevant subgroups under NCLB. Specifically, we explored whether contract restrictiveness is associated with the proportion of Black, Hispanic, English language learner, disabled, or low-income students achieving proficiency. Because we found largely inconsistent and insignificant associations, we do not present these results. They are available upon request from the authors.
We find consistent evidence that contract restrictiveness is associated with a greater probability that districts will be in PI, and at higher levels of PI, as well as with lower graduation rates. There is no significant relationship between contract strength and either ELA or math proficiency outcomes. These results are shown in Table 2, Columns 1, 2, 6, and 7 and in Table 3, Columns 1, 2, 6, and 7. We see that a one-unit increase in contract strength is associated with a 7.67% increase in the likelihood that a district will be in PI and a 4.48% increase in the likelihood that a district will progress to higher levels of PI. Although not shown in Table 2, the coefficients on the year in- dicators show that the magnitudes of these relationships for the most part increase each year, indicating that districts with stronger contracts are more likely to fail to meet NCLB requirements, and thus to enter and prog- ress through PI, as it becomes more difficult to meet these requirements over time. We find that a one standard deviation (0.51) increase in contract
Accountability Under Constraint
883
T a b le
2
L o
g is
ti c
a n
d O
rd e re
d L
o g
is ti
c R
e g
re s s io
n A
n a ly
s is
o f
th e
R e la
ti o
n s h
ip B
e tw
e e n
C o
n tr
a c t
R e s tr
ic ti
v e n
e s s
a n
d D
is tr
ic t
P ro
g ra
m Im
p ro
v e m
e n
t (P
I) S
ta tu
s
P I
S ta
tu s
P I
(0 -3
)
(1 )
(2 )
(3 )
(4 )
(5 )
(6 )
(7 )
(8 )
(9 )
(1 0 )
C o n tr a c t re
st ri c ti v e n e ss
(C R )
0 .4
5 6 * *
0 .4
5 0 * *
2 0 .4
0 3
2 0 .7
0 3 1
1 .8
2 6 * *
0 .4
6 1 * *
0 .4
5 4 * *
2 0 .2
2 3
2 0 .5
0 7
1 .7
3 3 *
(0 .1
4 2 )
(0 .1
4 2 )
(0 .4
0 5 )
(0 .3
7 9 )
(0 .6
7 4 )
(0 .1
4 1 )
(0 .1
4 0 )
(0 .4
4 1 )
(0 .4
1 4 )
(0 .7
2 0 )
L a g g e d
m a th
p ro
fi c ie
n c y
ra te
2 1 4 .1
5 2 * * *
2 1 3 .2
1 8 * * *
2 1 3 .7
2 3 * * *
2 1 2 .9
6 7 * * *
2 1 3 .0
8 3 * * *
2 1 4 .5
8 3 * * *
2 1 3 .2
9 4 * * *
2 1 4 .1
8 5 * * *
2 1 3 .0
9 9 * * *
2 1 3 .1
2 5 * * *
(1 .0
5 4 )
(1 .2
1 4 )
(1 .0
7 1 )
(1 .1
8 6 )
(1 .2
3 8 )
(1 .1
0 3 )
(1 .2
2 0 )
(1 .1
4 0 )
(1 .2
0 5 )
(1 .2
4 3 )
L a g g e d
M a th
P ro
fi c ie
n c y
R a te
3 C R
2 2 .8
9 4 *
2 2 .6
8 2 1
(1 .4
1 2 )
(1 .4
8 9 )
% m
in o ri ty
0 .4
8 9
0 .6
7 0 1
0 .6
4 4 1
0 .7
8 3 1
(0 .3
6 4 )
(0 .3
8 0 )
(0 .3
9 0 )
(0 .4
0 5 )
% M
in o ri ty
3 C R
1 .4
7 9 *
1 .1
4 2 1
(0 .6
2 1 )
(0 .6
7 3 )
% fr
e e /r
e d u c e d
lu n c h
1 .0
9 6 *
1 .3
0 3 * *
1 .1
3 0 *
1 .5
1 3 * *
1 .6
1 6 * *
1 .5
3 5 * *
(0 .5
0 1 )
(0 .4
8 6 )
(0 .5
1 0 )
(0 .5
1 1 )
(0 .4
9 9 )
(0 .5
1 6 )
% F re
e /R
e d u c e d
L u n c h
3 C R
2 .1
9 8 * * *
1 .7
5 4 *
(0 .6
5 0 )
(0 .7
0 5 )
U rb
a n
d is
tr ic
t 0 .3
6 7 *
0 .3
2 8 1
0 .3
6 5 *
0 .3
3 4 1
0 .3
5 2 *
0 .3
4 7 *
0 .2
9 9 1
0 .3
4 5 *
0 .3
0 2 1
0 .3
1 8 1
(0 .1
6 8 )
(0 .1
7 0 )
(0 .1
7 1 )
(0 .1
7 4 )
(0 .1
7 4 )
(0 .1
6 6 )
(0 .1
6 7 )
(0 .1
6 9 )
(0 .1
7 1 )
(0 .1
7 1 )
R u ra
l d is
tr ic
t 2
0 .8
9 8 * * *
2 0 .8
8 7 * * *
2 0 .9
4 4 * * *
2 0 .8
9 2 * * *
2 0 .8
4 3 * * *
2 0 .8
6 7 * *
2 0 .8
5 5 * *
2 0 .9
0 3 * * *
2 0 .8
6 2 * *
2 0 .8
2 6 * *
(0 .2
5 8 )
(0 .2
5 9 )
(0 .2
5 3 )
(0 .2
4 9 )
(0 .2
5 2 )
(0 .2
7 2 )
(0 .2
7 4 )
(0 .2
6 9 )
(0 .2
6 5 )
(0 .2
6 6 )
S m
a ll
d is
tr ic
t 2
1 .0
1 4 * * *
2 1 .0
7 6 * * *
2 1 .0
3 6 * * *
2 1 .0
5 4 * * *
2 1 .0
4 1 * * *
2 0 .9
1 4 * * *
2 1 .0
1 4 * * *
2 0 .9
3 2 * * *
2 1 .0
0 1 * * *
2 0 .9
8 6 * * *
(0 .2
1 2 )
(0 .2
0 1 )
(0 .2
1 0 )
(0 .1
9 7 )
(0 .1
9 9 )
(0 .2
2 8 )
(0 .2
1 7 )
(0 .2
2 7 )
(0 .2
1 5 )
(0 .2
1 5 )
L a rg
e d is
tr ic
t 1 .1
2 7 * * *
1 .1
7 7 * * *
1 .1
0 9 * * *
1 .1
6 4 * * *
1 .1
5 9 * * *
1 .1
6 6 * * *
1 .2
2 7 * * *
1 .1
4 6 * * *
1 .2
0 8 * * *
1 .2
0 6 * * *
(0 .1
8 0 )
(0 .1
7 8 )
(0 .1
8 4 )
(0 .1
8 3 )
(0 .1
8 2 )
(0 .1
7 3 )
(0 .1
7 4 )
(0 .1
7 7 )
(0 .1
7 8 )
(0 .1
7 8 )
H ig
h sc
h o o l d is
tr ic
t 2
0 .0
7 3
0 .0
8 1
2 0 .1
0 8
0 .1
0 4
0 .0
7 5
2 0 .1
2 3
0 .0
8 4
2 0 .1
5 2
0 .0
8 6
0 .0
7 5
(0 .3
2 8 )
(0 .3
3 9 )
(0 .3
2 2 )
(0 .3
3 3 )
(0 .3
3 6 )
(0 .3
2 5 )
(0 .3
3 3 )
(0 .3
2 0 )
(0 .3
2 7 )
(0 .3
3 1 )
(c o n
ti n
u ed
)
884
T a b le
2 (c
o n
ti n
u e d
)
P I
S ta
tu s
P I
(0 -3
)
(1 )
(2 )
(3 )
(4 )
(5 )
(6 )
(7 )
(8 )
(9 )
(1 0 )
U n if ie
d d is
tr ic
t 2
1 .1
4 6 * * *
2 1 .1
0 8 * * *
2 1 .1
3 1 * * *
2 1 .0
5 8 * * *
2 1 .0
8 3 * * *
2 1 .1
1 4 * * *
2 1 .0
6 8 * * *
2 1 .0
9 9 * * *
2 1 .0
3 3 * * *
2 1 .0
4 6 * * *
(0 .1
5 7 )
(0 .1
5 7 )
(0 .1
5 8 )
(0 .1
6 0 )
(0 .1
5 8 )
(0 .1
5 8 )
(0 .1
5 8 )
(0 .1
5 9 )
(0 .1
6 1 )
(0 .1
6 0 )
D is
tr ic
t p e r
p u p il
e x p e n d it u re
s 0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
c u
t1 2
5 .8
2 7 * * *
2 4 .8
4 9 * * *
2 5 .5
9 1 * * *
2 4 .7
4 3 * * *
2 4 .7
7 6 * * *
(0 .7
4 1 )
(0 .8
2 7 )
(0 .7
6 4 )
(0 .8
1 5 )
(0 .8
4 3 )
c u
t2 2
5 .1
4 6 * * *
2 4 .1
6 3 * * *
2 4 .9
0 7 * * *
2 4 .0
5 4 * * *
2 4 .0
8 9 * * *
(0 .7
3 5 )
(0 .8
2 4 )
(0 .7
5 9 )
(0 .8
1 2 )
(0 .8
4 0 )
c u
t3 2
4 .4
5 1 * * *
2 3 .4
6 4 * * *
2 4 .2
1 1 * * *
2 3 .3
5 3 * * *
2 3 .3
8 9 * * *
(0 .7
3 0 )
(0 .8
1 9 )
(0 .7
5 5 )
(0 .8
0 7 )
(0 .8
3 5 )
C o n st
a n t
4 .9
2 6 * * *
4 .2
7 7 * * *
4 .6
7 8 * * *
4 .1
0 4 * * *
4 .2
0 9 * * *
(0 .6
7 0 )
(0 .7
7 2 )
(0 .6
8 5 )
(0 .7
5 1 )
(0 .7
9 0 )
P se
u d o
R 2
.3 9 4
.3 9 6
.3 9 7
.4 0 1
.3 9 8
.2 9 1
.2 9 4
.2 9 2
.2 9 6
.2 9 5
N 1 ,7
9 4
1 ,7
9 4
1 ,7
9 4
1 ,7
9 4
1 ,7
9 4
1 ,3
4 2
1 ,3
4 2
1 ,3
4 2
1 ,3
4 2
1 ,3
4 2
N o te
. A ll
m o d e ls
e st
im a te
d w
it h
y e a r fi x e d
e ff e c ts
a n d
ro b u st
st a n d a rd
e rr
o rs
. M
o d e ls
1 th
ro u g h
5 w
e re
a ls
o ru
n u si
n g
E n g li sh
la n g u a g e
a rt s
(E L A ) p e rc
e n t p ro
fi c ie
n c y
o u tc
o m
e s
a n d
la g s,
w it h
su b st
a n ti a ll y
th e
sa m
e re
su lt s.
M o d e ls
5 a n d
1 0
w e re
a ls
o e st
im a te
d w
it h
in te
ra c ti o n s
b e tw
e e n
% m
in o ri ty
a n d
c o n tr a c t re
st ri c ti v e n e ss
w it h
si m
il a r re
su lt s.
T h e se
re su
lt s
a re
a v a il a b le
u p o n
re q u e st
. 1
p \
.1 0 . * p
\ .0
5 . * * p
\ .0
1 . * * * p
\ .0
0 1 .
885
T a b le
3
R e g
re s s io
n A
n a ly
s is
o f
th e
R e la
ti o
n s h
ip B
e tw
e e n
C o
n tr
a c t
R e s tr
ic ti
v e n
e s s
a n
d D
is tr
ic t
M a th
a n
d G
ra d
u a ti
o n
P ro
fi c ie
n c y
R a te
s
M a th
P ro
fi c ie
n c y
R a te
s G
ra d u a ti o n
R a te
s
(1 )
(2 )
(3 )
(4 )
(5 )
(6 )
(7 )
(8 )
(9 )
(1 0 )
C o n tr a c t re
st ri c ti v e n e ss
(C R )
2 0 .0
0 3
2 0 .0
0 3
0 .0
0 3
0 .0
0 3
2 0 .0
1 4 *
2 0 .0
1 3 *
2 0 .0
1 4 *
0 .0
2 0 *
0 .0
1 8 *
2 0 .3
3 1 * * *
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 3 )
(0 .0
0 3 )
(0 .0
0 6 )
(0 .0
0 6 )
(0 .0
0 6 )
(0 .0
0 8 )
(0 .0
0 8 )
(0 .0
9 1 )
L a g g e d
m a th
p ro
fi c ie
n c y
ra te
0 .9
4 8 * * *
0 .9
2 5 * * *
0 .9
4 5 * * *
0 .9
2 3 * * *
0 .9
2 5 * * *
0 .5
9 7 * * *
0 .5
7 1 * * *
0 .5
7 9 * * *
0 .5
5 2 * * *
0 .6
0 1 * * *
(0 .0
0 8 )
(0 .0
1 1 )
(0 .0
0 8 )
(0 .0
1 2 )
(0 .0
1 1 )
(0 .0
8 9 )
(0 .0
9 2 )
(0 .0
9 0 )
(0 .0
9 3 )
(0 .0
7 1 )
L a g g e d
M a th
P ro
fi c ie
n c y
R a te
3 C R
0 .0
2 0 1
0 .3
5 3 * * *
(0 .0
1 1 )
(0 .0
9 9 )
% m
in o ri ty
2 0 .0
0 1
2 0 .0
0 3
2 0 .0
5 8 * * *
2 0 .0
6 3 * * *
(0 .0
0 5 )
(0 .0
0 5 )
(0 .0
1 3 )
(0 .0
1 4 )
% M
in o ri ty
3 C R
2 0 .0
1 1 1
2 0 .0
6 4 * * *
(0 .0
0 6 )
(0 .0
1 5 )
% fr
e e /r
e d u c e d
lu n c h
2 0 .0
1 6 *
2 0 .0
1 8 *
2 0 .0
1 6 *
2 0 .0
7 8 * * *
2 0 .0
8 2 * * *
2 0 .0
6 2 * * *
(0 .0
0 7 )
(0 .0
0 7 )
(0 .0
0 7 )
(0 .0
1 8 )
(0 .0
1 8 )
(0 .0
1 4 )
% F re
e /R
e d u c e d
L u n c h
3 C R
2 0 .0
1 2 1
2 0 .0
6 9 * * *
(0 .0
0 7 )
(0 .0
1 5 )
U rb
a n
d is
tr ic
t 0 .0
0 1
0 .0
0 2
0 .0
0 1
0 .0
0 2
0 .0
0 1
2 0 .0
0 2
0 .0
0 0
2 0 .0
0 2
0 .0
0 1
0 .0
0 2
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 4 )
(0 .0
0 4 )
(0 .0
0 4 )
(0 .0
0 4 )
(0 .0
0 3 )
R u ra
l d is
tr ic
t 0 .0
0 0
2 0 .0
0 1
0 .0
0 1
0 .0
0 0
2 0 .0
0 1
0 .0
0 0
0 .0
0 6
0 .0
0 3
0 .0
0 7
0 .0
0 9
(0 .0
0 3 )
(0 .0
0 3 )
(0 .0
0 3 )
(0 .0
0 3 )
(0 .0
0 3 )
(0 .0
0 9 )
(0 .0
0 8 )
(0 .0
0 8 )
(0 .0
0 8 )
(0 .0
0 8 )
S m
a ll
d is
tr ic
t 2
0 .0
0 2
2 0 .0
0 2
2 0 .0
0 2
2 0 .0
0 3
2 0 .0
0 3
2 0 .0
0 8
2 0 .0
0 3
2 0 .0
0 6
2 0 .0
0 3
2 0 .0
0 3
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 6 )
(0 .0
0 6 )
(0 .0
0 6 )
(0 .0
0 6 )
(0 .0
0 6 )
L a rg
e d is
tr ic
t 0 .0
0 3 *
0 .0
0 3 *
0 .0
0 4 *
0 .0
0 3 *
0 .0
0 3 *
2 0 .0
0 3
2 0 .0
0 7 1
0 .0
0 0
2 0 .0
0 6
2 0 .0
0 4
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 4 )
(0 .0
0 4 )
(0 .0
0 4 )
(0 .0
0 4 )
(0 .0
0 4 )
(c o n
ti n
u ed
)
886
T a b le
3 (c
o n
ti n
u e d
)
M a th
P ro
fi c ie
n c y
R a te
s G
ra d u a ti o n
R a te
s
(1 )
(2 )
(3 )
(4 )
(5 )
(6 )
(7 )
(8 )
(9 )
(1 0 )
H ig
h sc
h o o l d is
tr ic
t 2
0 .0
0 2
2 0 .0
0 4
2 0 .0
0 1
2 0 .0
0 4
2 0 .0
0 4
2 0 .0
0 5
2 0 .0
1 1
0 .0
0 5
2 0 .0
0 7
2 0 .0
0 7
(0 .0
0 3 )
(0 .0
0 3 )
(0 .0
0 3 )
(0 .0
0 3 )
(0 .0
0 3 )
(0 .0
2 5 )
(0 .0
2 5 )
(0 .0
2 5 )
(0 .0
2 5 )
(0 .0
2 5 )
U n if ie
d d is
tr ic
t 0 .0
0 0
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 6
2 0 .0
0 5
0 .0
0 0
2 0 .0
0 4
2 0 .0
0 6
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
2 5 )
(0 .0
2 5 )
(0 .0
2 5 )
(0 .0
2 5 )
(0 .0
2 5 )
D is
tr ic
t p e r
p u p il
e x p e n d it u re
s 0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
C o n st
a n t
0 .0
5 9 * * *
0 .0
7 6 * * *
0 .0
6 1 * * *
0 .0
7 7 * * *
0 .0
7 6 * * *
0 .4
0 5 * * *
0 .4
3 1 * * *
0 .4
1 7 * * *
0 .4
4 8 * * *
0 .3
9 7 * * *
(0 .0
0 6 )
(0 .0
0 9 )
(0 .0
0 6 )
(0 .0
0 9 )
(0 .0
0 9 )
(0 .0
9 2 )
(0 .0
9 5 )
(0 .0
9 2 )
(0 .0
9 6 )
(0 .0
7 9 )
R 2
.9 5 2
.9 5 2
.9 5 2
.9 5 2
.9 5 2
.4 6 5
.4 7 4
.4 7 3
.4 8 2
.5 0 6
N 1 ,8
3 0
1 ,8
3 0
1 ,8
3 0
1 ,8
3 0
1 ,8
3 0
1 ,2
3 6
1 ,2
3 6
1 ,2
3 6
1 ,2
3 6
1 ,2
3 6
N o te
. A ll
m o d e ls
e st
im a te
d w
it h
y e a r fi x e d
e ff e c ts
a n d
ro b u st
st a n d a rd
e rr
o rs
. M
o d e ls
1 th
ro u g h
5 w
e re
a ls
o ru
n u si
n g
E n g li sh
la n g u a g e
a rt s
(E L A ) p e rc
e n t p ro
fi c ie
n c y
o u tc
o m
e s
a n d
la g s,
w it h
su b st
a n ti a ll y
th e
sa m
e re
su lt s e x c e p t in
M o d e l 4 , fo
r w
h ic
h th
e in
te ra
c ti o n
b e tw
e e n
c o n tr a c t re
st ri c ti v e n e ss
a n d
p e rc
e n t o f st
u d e n ts
re c e iv
in g
fr e e
o r re
d u c e d
p ri c e
lu n c h e s
is n e g a ti v e , b u t n o t st
a ti st
ic a ll y
si g n if ic
a n t (p
= .1
8 7 ).
M o d e ls
5 a n d
1 0
w e re
a ls
o e st
im a te
d w
it h
in te
ra c ti o n s
b e tw
e e n
% m
in o ri ty
a n d
c o n tr a c t re
st ri c ti v e n e ss
w it h
si m
il a r
re su
lt s.
T h e se
re su
lt s
a re
a v a il a b le
u p o n
re q u e st
. 1
p \
.1 0 . * p
\ .0
5 . * * p
\ .0
1 . * * * p
\ .0
0 1 .
887
restrictiveness is associated with approximately a 1.3% decrease in gradua- tion rates, on average.
We are also interested in the relationships between elements of teachers’ working conditions and student achievement. We find that districts with lower previous math proficiency rates and greater proportions of minority and poor students are more likely to be in PI and at advanced stages of PI and to have lower proficiency and graduation rates. In addition, we see that urban and large districts are more likely to be in PI and at higher levels of PI, whereas rural and small districts are less likely to be failing under NCLB.
Given that we find that districts with more restrictive CBAs have worse performance outcomes under NCLB, we are driven to question whether this relationship is accentuated in certain kinds of districts. We hypothesize that contract restrictiveness may be more associated with outcomes in districts with difficult working conditions because the protections guaranteed to teachers in the CBAs in such districts may be more utilized and adhered to than in districts with easier working conditions. Contracts are to a great extent negotiated to provide teachers with protections and benefits that are likely to be more important in districts with harder working conditions. Districts with difficult working conditions, such as high proportions of minority, low-income, and low-achieving students, may need to enhance their working conditions in order to attract and retain teachers (Loeb & Page, 2000). Districts may be able to enhance their working conditions by assuring teachers that their contractually agreed upon protections will be en- forced. If this is the case, then hard-to-staff districts will be more likely to enforce the provisions that make CBAs restrictive to district and school ad- ministrators, which may in turn impact district administrators’ abilities to implement accountability-driven reforms.
To explore the degree to which contract restrictiveness may be differen- tially associated with student achievement when there are more or less attractive working conditions, proxied by the proportion of low-income, minority, and low-achieving students in the districts, we run a series of mod- els of the following form:
Ajt 5 b01b1ðCBA RestrictivenessjÞ1b2ð% Minorityjt�1Þ þ b3ðCR 3 % Minorityjt�1Þ1b4ð% Low Incomejt�1Þ þ b5ðCR 3 % Low Incomejt�1Þ1b6ð% Proficientjt�2Þ þ b7ðCR 3 % Proficientjt�2Þ1Djt�1d1tt 1njt:
ð2Þ
These models have the same set of outcomes and covariates as outlined in Equation 1 and also include interactions between the indicators of district working conditions and contract restrictiveness. We are specifically inter- ested in parameters b3, b5, and b7, which can be interpreted as the differen- tial relationship between contract restrictiveness and the district-level
Strunk, McEachin
888
outcome in districts with varying proportions of minority, low-income, and proficient students.
We find that contract restrictiveness is particularly negatively associated with districts’ PI status and level in districts with difficult working conditions. Columns 3 through 5 in Table 2 show that districts with higher proportions of minority, low-income, and low-achieving students are more likely to be in PI and that this relationship is exacerbated when these hard-to-staff districts also have more restrictive contracts. Columns 8 through 10 show that these particularly challenging districts are also even more likely to be at higher lev- els of PI when they have more restrictive contracts. These findings appear at least in part to be driven by lower student performance in hard-to-staff dis- tricts with restrictive contracts; Columns 3 through 5 and 8 through 10 in Table 3 show that districts with higher proportions of minority, low-income, and low-achieving students are even more likely to have lower math, ELA and graduation rates in districts with stronger contracts. For example, on average, a district with a high proportion of minority students (one standard deviation above the mean) and a restrictive contract (one standard deviation above the mean) has 0.2% fewer students obtaining proficiency and 3.5% fewer students graduating than a similar district with an average proportion of minority students and an average contract. Similarly, a district with a high proportion of poor students (again, defined as one standard deviation above the mean) and a restrictive contract has 0.6% fewer students obtaining pro- ficiency and 3.1% fewer students graduating than a similar district with the average proportion of poor students and an average contract.
V. Is Contract Restrictiveness Associated With School-Level
Accountability Outcomes?
It is possible that contracts, which are negotiated at the district level, are differently associated with school-level outcomes under NCLB pressures than they are with district-level outcomes. Schools may interpret or act on contract provisions in disparate ways depending on their particular contexts and the threats to them based on accountability sanctions. To better understand the relationship between contract strength and school-level outcomes under NCLB structures, our third and fourth sets of analyses repeat those explained above, except we now consider outcomes and characteristics at the school level. We use hierarchical linear models (HLMs) and hierarchical general linear models (HGLMs) for schools (Level 1) nested within districts (Level 2) to con- trol for both the clustered nature of our data and the correlation between dis- trict-level variables and school-level outcomes (Raudenbush & Bryk, 2002). At Level 1, the school-level model takes the form:
Aijt 5 b01b1ð% Minorityijt�1Þ1b2ð% Low Incomeijt�1Þ 1b3ð% Proficientijt�2Þ1Sijt�1j1eijt;
ð3aÞ
Accountability Under Constraint
889
where Aijt is each outcome in turn at the school level. The proportion of stu- dents who are minority, low-income, and proficient in math are also mea- sured at the school level. Sijt–1 is a vector of school characteristics that includes school location (urban and rural) and school type (high school and middle school). We group-mean center our Level 1 variables in our models with continuous outcomes, allowing us to interpret b0j as the mean outcome (for example, proficiency rate) of the average school in dis- trict j. We grand-mean center our Level 1 variables in our models with dichotomous and ordinal outcomes for ease of interpretation. In this case, b0j is the grand average likelihood of being in PI. The Level 1 parameters b1j through b6j can be interpreted as the average relationship between the independent variable and the outcome. At Level 1 we are primarily inter- ested in the relationships between the proportions of minority, low-income, and proficient students and school outcomes, b1, b2, and b3, controlling for schools’ geographic location and type. This is because we are interested in testing whether contract restrictiveness is differentially associated with school outcomes in districts and schools with harder working conditions.
At Level 2, the district-level model is first specified as
b0jt 5g001g01ðCBA RestrictivenessjÞ1Djt21d1tt 1u0jt b1jt; � � � ; b7jt 5g10; . . . ; g70:
ð3bÞ
The Level 2 predictors are grand-mean centered. In this set of analyses, we only include district-level predictors in the Level 1 intercept equation, such that we are exploring the relationships between district-level predictors and the average school-level outcomes of interest. We are most interested in the parameter g01, which indicates the relationship between contract restrictiveness and average school outcomes. Djt–1 is now a vector of district characteristics that includes two dichotomous indicators of district size (small and large) and a measure of ‘‘diversity,’’ which controls for the number of subgroup scores for which a district may be held accountable. We control for this measure because Balfanz, Legters, West, and Weber (2007) find that the number of significant subgroups for which schools are held account- able strongly and negatively predicts their likelihood of meeting AYP. We do not control for the proportions of minority and low-income students in the district given their high correlations with the diversity measure. We also include year fixed-effects.
Tables 4 and 5 provide the results of these analyses. We do not find a sig- nificant association between contract strength and schools’ likelihood of ‘‘failing’’ under NCLB. On average, schools in districts with more restrictive contracts appear no more likely to be in PI or at higher levels of PI than do schools with less restrictive contracts (Table 4, Columns 1, 2, 6, and 7). Contract strength also is not significantly associated with schools’ proficiency or graduation rates (Table 5, Columns 1, 2, 6, and 7).
Strunk, McEachin
890
T a b le
4
L o
g is
ti c
a n
d O
rd e re
d L
o g
is ti
c H
ie ra
rc h
ic a l G
e n
e ra
l L
in e a r
M o
d e l (H
G L
M )
A n
a ly
s is
o f
th e
R e la
ti o
n s h
ip B
e tw
e e n
C o
n tr
a c t
R e s tr
ic ti
v e n
e s s
a n
d
S c h
o o
l P
ro g
ra m
Im p
ro v e m
e n
t (P
I) S
ta tu
s
P I
S ta
tu s
P I
(0 -3
1 )
(1 )
(2 )
(3 )
(4 )
(5 )
(6 )
(7 )
(8 )
(9 )
(1 0 )
In te
rc e p t
2 0 .7
0 6 * * *
2 0 .8
4 6 * * *
2 0 .6
9 9 * * *
2 0 .8
6 2 * * *
2 0 .8
1 4 * * *
2 1 .7
5 0 * * *
2 1 .8
7 0 * * *
2 1 .7
5 7 * * *
2 1 .8
8 9 * * *
2 1 .8
3 5 * * *
(0 .0
7 3 )
(0 .0
7 7 )
(0 .0
7 5 )
(0 .0
7 7 )
(0 .0
7 6 )
(0 .0
8 6 )
(0 .0
8 6 )
(0 .0
8 6 )
(0 .0
8 6 )
(0 .0
8 5 )
C o n tr a c t re
st ri c ti v e n e ss
0 .2
0 9
0 .1
4 1
0 .2
0 6
0 .1
7 3
0 .1
1 2
.2 3 9 1
0 .1
6 1
.2 3 8 1
0 .1
8 0
0 .0
8 9
(0 .1
7 0 )
(0 .1
8 5 )
(0 .1
7 0 )
(0 .1
8 4 )
(0 .1
8 0 )
(0 .1
4 3 )
(0 .1
4 5 )
(0 .1
4 2 )
(0 .1
4 7 )
(0 .1
4 4 )
# o f A d e q u a te
Y e a rl y
P ro
g re
ss 0 .0
1 4
0 .0
2 0 .0
1 4
0 .0
2 0 .0
1 8
0 .0
1 4
0 .0
1 6
0 .0
1 5
0 .0
1 7
0 .0
1 4
(A Y
P )
c ri te
ri a
(0 .0
1 4 )
(0 .0
1 5 )
(0 .0
1 4 )
(0 .0
1 5 )
(0 .0
1 5 )
(0 .0
1 5 )
(0 .0
1 5 )
(0 .0
1 5 )
(0 .0
1 4 )
(0 .0
1 5 )
S m
a ll
d is
tr ic
t 2
0 .0
1 6
2 0 .4
9 1 *
2 0 .0
0 1
2 0 .5
0 2 *
2 0 .4
7 1 *
2 0 .0
2 3
2 .5
5 0 *
2 0 .0
3 6
2 .5
6 1 *
2 .5
2 0 *
(0 .2
2 0 )
(0 .2
2 7 )
(0 .2
2 1 )
(0 .2
2 4 )
(0 .2
2 5 )
(0 .2
5 0 )
(0 .2
6 0 )
(0 .2
4 8 )
(0 .2
6 1 )
(0 .2
5 3 )
L a rg
e d is
tr ic
t 2
0 .0
5 8
0 .1
6 5
2 0 .0
5 2
0 .1
4 7
0 .1
5 5
2 0 .1
0 3
0 .1
1 0
2 0 .1
1 0
0 .0
9 5
0 .1
0 8
(0 .2
1 3 )
(0 .2
3 7 )
(0 .2
1 4 )
(0 .2
3 6 )
(0 .2
3 0 )
(0 .1
9 2 )
(0 .1
9 2 )
(0 .1
9 8 )
(0 .1
9 3 )
(0 .1
9 2 )
P e r
p u p il
e x p e n d it u re
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
(0 .0
0 0 )
L a g g e d
m a th
p ro
fi c ie
n c y
ra te
2 9 .6
7 6 * * *
2 1 0 .0
3 8 * * *
2 9 .6
8 5 * * *
2 1 0 .0
4 9 * * *
2 9 .4
7 4 * * *
2 9 .4
5 7 * * *
2 9 .7
2 8 * * *
2 9 .4
4 6 * * *
2 9 .7
4 0 * * *
2 9 .1
8 7 * * *
(0 .1
3 9 )
(0 .1
3 3 )
(0 .1
4 1 )
(0 .1
4 5 )
(0 .1
5 3 )
(0 .4
0 5 )
(0 .4
0 8 )
(0 .3
9 6 )
(0 .4
0 9 )
(0 .4
2 3 )
C o n tr a c t re
st ri c ti v e n e ss
2 2 .8
6 7 * * *
2 2 .6
6 5 * * *
(0 .3
1 4 )
(0 .7
0 8 )
% m
in o ri ty
3 .8
5 8 * * *
3 .9
0 4 * * *
3 .7
3 6 * * *
3 .6
9 5 * * *
(0 .0
9 5 )
(0 .1
3 6 )
(0 .3
6 6 )
(0 .3
1 7 )
C o n tr a c t re
st ri c ti v e n e ss
2 0 .2
4 2
0 .2
4 9
(0 .2
1 1 )
(0 .8
1 2 )
% fr
e e /r
e d u c e d
lu n c h
3 .3
2 3 * * *
3 .1
0 4 * * *
3 .3
5 2 * * *
3 .3
8 3 * * *
3 .1
2 2 * * *
3 .4
1 0 * * *
(0 .0
7 0 )
(0 .0
8 7 )
(0 .0
6 9 )
(0 .4
4 4 )
(0 .4
4 0 )
(0 .4
4 6 )
C o n tr a c t re
st ri c ti v e n e ss
0 .9
5 8 * * *
1 .1
7 3 1
(0 .1
7 7 )
(0 .6
8 1 )
(c o n
ti n
u ed
)
891
T a b le
4 (c
o n
ti n
u e d
)
P I
S ta
tu s
P I
(0 -3
1 )
(1 )
(2 )
(3 )
(4 )
(5 )
(6 )
(7 )
(8 )
(9 )
(1 0 )
U rb
a n
sc h o o l
2 0 .0
7 9
2 0 .0
7 9
2 0 .0
7 7
2 0 .0
8 3
2 0 .0
7 9
2 0 .0
9 3
2 0 .0
9 5
2 0 .0
9 5
2 0 .0
9 9
2 0 .0
9 4
(0 .0
5 0 )
(0 .0
5 8 )
(0 .0
5 0 )
(0 .0
5 7 )
(0 .0
5 7 )
(0 .1
1 5 )
(0 .1
0 5 )
(0 .1
1 6 )
(0 .1
0 5 )
(0 .1
0 5 )
R u ra
l sc
h o o l
0 .8
8 3 * *
0 .4
3 3
0 .8
9 6 * *
0 .4
1 9
0 .3
5 6
1 .0
6 7 * * *
0 .5
7 6
1 .0
5 3 * * *
0 .5
6 4
0 .4
9 4
(0 .3
1 5 )
(0 .2
8 3 )
(0 .3
1 7 )
(0 .2
8 1 )
(0 .2
7 9 )
(0 .3
0 3 )
(0 .3
6 3 )
(0 .3
0 1 )
(0 .3
5 9 )
(0 .3
5 1 )
H ig
h sc
h o o l
2 0 .8
2 5 * * *
2 0 .5
3 6 * * *
2 0 .8
2 6 * * *
2 0 .5
3 1 * * *
2 0 .5
3 1 * * *
2 .8
7 9 * * *
2 .5
5 7 * *
2 .8
7 7 * * *
2 .5
5 4 * *
2 .5
5 2 * *
(0 .0
4 7 )
(0 .0
5 0 )
(0 .0
4 8 )
(0 .0
5 3 )
(0 .0
4 9 )
(0 .2
0 4 )
(0 .2
0 7 )
(0 .2
0 2 )
(0 .3
6 0 )
(0 .2
1 1 )
M id
d le
sc h o o l
1 .3
1 6 * * *
1 .3
7 4 * * *
1 .3
1 5 * * *
1 .3
7 3 * * *
1 .3
9 8 * * *
1 .2
7 1 * * *
1 .3
4 7 * * *
1 .2
7 3 * * *
1 .3
4 7 * * *
1 .3
7 0 * * *
(0 .0
4 4 )
(0 .0
4 3 )
(0 .0
4 6 )
(0 .0
4 5 )
(0 .0
4 4 )
(0 .1
5 3 )
(0 .1
6 5 )
(0 .1
5 0 )
(0 .1
6 4 )
(0 .1
6 3 )
T h re
sh o ld
2 0 .5
5 1 * * *
.5 5 5 * * *
0 .5
5 1 * * *
.5 5 6 * * *
.5 5 6 * * *
(0 .0
2 9 )
(0 .0
2 9 )
(0 .0
2 9 )
(0 .0
2 9 )
(0 .0
2 9 )
T h re
sh o ld
3 1 .1
6 9 * * *
1 .1
7 5 * * *
1 .1
6 9 * * *
1 .1
7 6 * * *
1 .1
7 8 * * *
(0 .0
4 1 )
(0 .0
4 1 )
(0 .0
4 1 )
(0 .0
4 1 )
(0 .0
4 1 )
N _ 1
1 6 ,0
0 4
1 6 ,0
0 4
1 6 ,0
0 4
1 6 ,0
0 4
1 6 ,0
0 4
1 6 ,0
0 4
1 6 ,0
0 4
1 6 ,0
0 4
1 6 ,0
0 4
1 6 ,0
0 4
N _ 2
4 4 9
4 4 9
4 4 9
4 4 9
4 4 9
4 4 9
4 4 9
4 4 9
4 4 9
4 4 9
N o te
. A ll
m o d e ls
e st
im a te
d w
it h
y e a r fi x e d
e ff e c ts
o n
th e
L e v e l 1
in te
rc e p t e q u a ti o n . M
o d e ls
1 th
ro u g h
5 w
e re
a ls
o ru
n u si
n g
E n g li sh
la n g u a g e
a rt s
(E L A )
p e rc
e n t p ro
fi c ie
n c y
o u tc
o m
e s
a n d
la g s,
w it h
su b st
a n ti a ll y
th e
sa m
e re
su lt s.
M o d e ls
5 a n d
1 0
w e re
a ls
o e st
im a te
d w
it h
in te
ra c ti o n s
b e tw
e e n
% m
in o ri ty
a n d
c o n tr a c t re
st ri c ti v e n e ss
w it h
si m
il a r re
su lt s.
T h e se
re su
lt s a re
a v a il a b le
u p o n
re q u e st
. 1
p \
.1 0 . * p
\ .0
5 . * * p
\ .0
1 . * * * p
\ .0
0 1 .
892
T a b le
5
H ie
ra rc
h ic
a l
L in
e a r
M o
d e l
(H L
M )
A n
a ly
s is
o f
th e
R e la
ti o
n s h
ip B
e tw
e e n
C o
n tr
a c t
R e s tr
ic ti
v e n
e s s
a n
d S
c h
o o
l
M a th
a n
d G
ra d
u a ti
o n
P ro
fi c ie
n c y
R a te
s
M a th
P ro
fi c ie
n c y
R a te
s G
ra d u a ti o n
P ro
fi c ie
n c y
R a te
s
(1 )
(2 )
(3 )
(4 )
(5 )
(6 )
(7 )
(8 )
(9 )
(1 0 )
In te
rc e p t
0 .5
4 6 * * *
0 .5
4 6 * * *
0 .5
4 6 * * *
0 .5
4 6 * * *
0 .5
4 6 * * *
0 .8
7 4 * * *
0 .8
7 4 * * *
0 .8
7 4 * * *
0 .8
7 4 * * *
0 .8
7 4 * * *
(0 .0
0 6 )
(0 .0
0 6 )
(0 .0
0 6 )
(0 .0
0 6 )
(0 .0
0 6 )
(0 .0
0 5 )
(0 .0
0 5 )
(0 .0
0 5 )
(0 .0
0 5 )
(0 .0
0 5 )
C o n tr a c t re
st ri c ti v e n e ss
0 .0
2 1
0 .0
2 1
0 .0
2 1
0 .0
2 1
0 .0
2 1
2 0 .0
1 5
2 0 .0
1 5
2 0 .0
1 5
2 0 .0
1 5
2 0 .0
1 5
(0 .0
1 3 )
(0 .0
1 3 )
(0 .0
1 3 )
(0 .0
1 3 )
(0 .0
1 3 )
(0 .0
1 0 )
(0 .0
1 0 )
(0 .0
1 0 )
(0 .0
1 0 )
(0 .0
1 0 )
# o f A d e q u a te
Y e a rl y
P ro
g re
ss (A
Y P )
c ri te
ri a
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
2 0 .0
0 1
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
S m
a ll
d is
tr ic
t 2
0 .0
4 0 *
2 0 .0
4 0 *
2 0 .0
4 0 *
2 0 .0
4 0 *
2 0 .0
4 0 *
2 0 .0
1 0
2 0 .0
1 0
2 0 .0
1 0
2 0 .0
1 0
2 0 .0
1 0
(0 .0
2 0 )
(0 .0
2 0 )
(0 .0
2 0 )
(0 .0
2 0 )
(0 .0
2 0 )
(0 .0
1 3 )
(0 .0
1 3 )
(0 .0
1 3 )
(0 .0
1 3 )
(0 .0
1 3 )
L a rg
e d is
tr ic
t 2
0 .0
3 0 1
2 0 .0
3 0 1
2 0 .0
3 0 1
2 0 .0
3 0 1
2 0 .0
3 0 1
2 0 .0
1 8
2 0 .0
1 8
2 0 .0
1 8
2 0 .0
1 8
2 0 .0
1 8
(0 .0
1 8 )
(0 .0
1 8 )
(0 .0
1 8 )
(0 .0
1 8 )
(0 .0
1 8 )
(0 .0
1 6 )
(0 .0
1 6 )
(0 .0
1 6 )
(0 .0
1 6 )
(0 .0
1 6 )
P e r
p u p il
e x p e n d it u re
s 0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
0 .0
0 0
L a g g e d
m a th
p ro
fi c ie
n c y
ra te
0 .8
5 4 * * *
0 .8
7 6 * * *
0 .8
5 3 * * *
0 .8
7 6 * * *
0 .8
6 2 * * *
0 .5
7 0 * * *
0 .5
7 8 * * *
0 .5
6 9 * * *
0 .5
7 6 * * *
0 .5
2 8 * * *
(0 .0
0 8 )
(0 .0
0 9 )
(0 .0
0 8 )
(0 .0
0 9 )
(0 .0
1 1 )
(0 .0
2 8 )
(0 .0
2 8 )
(0 .0
2 8 )
(0 .0
2 8 )
(0 .0
3 3 )
C o n tr a c t re
st ri c ti v e n e ss
0 .0
4 7 * * *
0 .1
7 2 * *
(0 .0
1 1 )
(0 .0
5 4 )
% m
in o ri ty
2 0 .0
6 3 * * *
2 0 .0
5 5 * * *
2 0 .0
6 0 * * *
2 0 .0
4 4 * *
(0 .0
0 5 )
(0 .0
0 4 )
(0 .0
1 2 )
(0 .0
1 7 )
C o n tr a c t re
st ri c ti v e n e ss
2 0 .0
2 5 * * *
2 0 .0
6 1 *
(0 .0
0 6 )
(0 .0
2 5 )
(c o n
ti n
u ed
)
893
T a b le
5 (c
o n
ti n
u e d
)
M a th
P ro
fi c ie
n c y
R a te
s G
ra d u a ti o n
P ro
fi c ie
n c y
R a te
s
(1 )
(2 )
(3 )
(4 )
(5 )
(6 )
(7 )
(8 )
(9 )
(1 0 )
% F re
e /r
e d u c e d
lu n c h
2 0 .0
3 2 * * *
2 0 .0
2 9 * * *
2 0 .0
3 1 * * *
2 0 .0
1 8 1
2 0 .0
0 8
2 0 .0
1 5
(0 .0
0 4 )
(0 .0
0 4 )
(0 .0
0 4 )
(0 .0
1 0 )
(0 .0
0 9 )
(0 .0
1 0 )
C o n tr a c t re
st ri c ti v e n e ss
2 0 .0
1 1 1
2 0 .0
6 7 * * *
(0 .0
0 6 )
(0 .0
1 9 )
U rb
a n
sc h o o l
0 2
0 .0
0 1
0 2
0 .0
0 1
2 0 .0
0 1
0 .0
0 2
0 .0
0 1
0 .0
0 3
0 .0
0 1
0 .0
0 0
(0 .0
0 2 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 1 )
(0 .0
0 5 )
(0 .0
0 5 )
(0 .0
0 5 )
(0 .0
0 5 )
(0 .0
0 5 )
R u ra
l sc
h o o l
0 .0
0 4
0 .0
0 3
0 .0
0 4
0 .0
0 3
0 .0
0 4
0 .0
2 4
0 .0
2 3
0 .0
2 3
0 .0
2 6
0 .0
3 0
(0 .0
0 9 )
(0 .0
0 9 )
(0 .0
0 9 )
(0 .0
0 9 )
(0 .0
1 0 )
(0 .0
1 9 )
(0 .0
1 8 )
(0 .0
1 8 )
(0 .0
1 7 )
(0 .0
2 1 )
H ig
h sc
h o o l
2 0 .0
2 0 * * *
2 0 .0
2 0 * * *
2 0 .0
2 0 * * *
2 0 .0
2 0 * * *
2 0 .0
2 0 * * *
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
(0 .0
0 2 )
M id
d le
sc h o o l
2 0 .0
2 7 * * *
2 0 .0
2 4 * * *
2 0 .0
2 7 * * *
2 0 .0
2 4 * * *
2 0 .0
2 5 * * *
(0 .0
0 1 )
(0 .0
0 2 )
(0 .0
0 1 )
(0 .0
0 2 )
(0 .0
0 2 )
N _ 1
2 4 ,5
6 4
2 4 ,5
6 4
2 4 ,5
6 4
2 4 ,5
6 4
2 4 ,5
6 4
5 ,6
4 9
5 ,6
4 9
5 ,6
4 9
5 ,6
4 9
5 ,6
4 9
N _ 2
4 5 6
4 5 6
4 5 6
4 5 6
4 5 6
3 0 5
3 0 5
3 0 5
3 0 5
3 0 5
N o te
. A ll
m o d e ls
e st
im a te
d w
it h
y e a r fi x e d
e ff e c ts
o n
th e
L e v e l 1
in te
rc e p t e q u a ti o n . M
o d e ls
1 th
ro u g h
5 w
e re
a ls
o ru
n u si
n g
E n g li sh
la n g u a g e
a rt s
(E L A )
p e rc
e n t p ro
fi c ie
n c y
o u tc
o m
e s
a n d
la g s,
w it h
su b st
a n ti a ll y
th e
sa m
e re
su lt s.
M o d e ls
5 a n d
1 0
w e re
a ls
o e st
im a te
d w
it h
in te
ra c ti o n s b e tw
e e n
% m
in o ri ty
a n d
c o n tr a c t re
st ri c ti v e n e ss
w it h
si m
il a r re
su lt s.
T h e se
re su
lt s a re
a v a il a b le
u p o n
re q u e st
. 1
p \
.1 0 . * p
\ .0
5 . * * p
\ .0
1 . * * * p
\ .0
0 1 .
894
It is interesting to note the relationships between other school and dis- trict characteristics and school-level outcomes. In both Tables 4 and 5, we see that higher performing schools are less likely to be in PI or at higher lev- els of PI and have higher proficiency and graduation rates. In addition, schools with higher proportions of minority and poor students are more likely to be deemed ‘‘failing’’ under NCLB and have lower graduation and math proficiency rates. We find that, controlling for school characteristics, no district characteristics have consistent significant relationships with school-level outcomes.
Next, we explore the ways in which district CBA restrictiveness impacts the relationship between school working conditions and school outcomes. The Level 1 equation remains the same as in Equation 3a; however, we now include contract restrictiveness in the g1, g2, and g3 equations. The pa- rameters g11, g21, and g31 can be interpreted as the degree to which contract strength exacerbates or attenuates the relationships between school out- comes and the proportions of minority students, low-income students, and proficient students, respectively.
b0jt 5g001g01ðCBA Restrictivenessjt21Þ1Djt21d1tt 1u0jt b1jt 5g101g11ðCBA Restrictivenessjt21Þ b2jt 5g201g21ðCBA Restrictivenessjt21Þ b3jt 5g201g21ðCBA Restrictivenessjt21Þ b4j; � � � ; b7j 5g40; . . . ; g70:
ð4Þ
Columns 3 through 5 and 8 through 10 in Tables 4 and 5 outline the relation- ships between contract restrictiveness and school-level outcomes in schools with more difficult working conditions. Table 4 indicates that schools with higher proportions of low-achieving and poor students are more likely to be labeled as failing under NCLB and that this relationship is accentuated in districts with stronger contracts. This appears driven in part by the nega- tive interaction between contract restrictiveness and working conditions in predicting school-level proficiency and graduation rates. For instance, Table 5 shows that schools with high proportions of students who qualify for free- or reduced-price lunches in strong contract districts have approxi- mately 6.7% lower graduation rates and 1.1% lower math proficiency rates than districts with average strength contracts.
However, Table 4 shows that, although there is a strong negative asso- ciation between the proportion of minority students in a school and that school’s likelihood of being in PI or at higher levels of PI, the interaction between contract restrictiveness and the proportion of minority students in a school is not significant. Nonetheless, we still see that schools with higher proportions of minority students fare worse in terms of math proficiency and graduation rates when in districts with stronger contracts; Table 5 shows that
Accountability Under Constraint
895
schools with high proportions of minority students in strong contract districts have approximately 6.1% lower graduation rates and 2.5% lower proficiency rates than districts with average strength contracts.
VI. Limitations
Our findings should be interpreted with a few limitations in mind. First, endogeneity concerns remind us that poor student performance and failure to meet accountability targets may lead to rather than result from stronger contracts. Because our data and analytic strategy do not enable us to defin- itively state that strong contracts lead to lower performance outcomes, we attempt to provide some explanation of the mechanism through which this might occur. We find that contract rigidity is more associated with reduced outcomes in schools and districts with characteristics that proxy less desirable working conditions. We find no evidence that suggests that stronger contracts are specifically associated with lower proficiency and graduation rates for most NCLB-classified subgroups, including Black, White, low-income, special needs, and English language learner groups. We do find some evidence that contract restrictiveness is associated with lower proficiency and graduation rates for Hispanic students.4 This indicates that contract restrictiveness is not particularly associated with subgroup out- comes. Rather, contracts that exhibit the same degree of inherent rigidity are more negatively associated with outcomes in districts with more minority, low-achieving, and low-income students. It may be something about the kinds of districts with higher proportions of low-income and minority stu- dents that interacts with contract rigidity to reduce student achievement and schools’ and districts’ abilities to meet NCLB standards.
In addition, we hesitate to draw causal conclusions from our analyses because other factors not included in our analyses may lead to both more restrictive contracts and lower student outcomes. For instance, the amount of trust or goodwill between the district administration and the teachers’ union or teaching staff may impact both contract strength and student out- comes. For both of these reasons, our results should be interpreted as sug- gestive of a causal relationship and not as causally determinative themselves.
To begin to understand the direction of the relationship between con- tract restrictiveness and performance outcomes, researchers might explore whether changes in CBA strength are associated with changes in outcomes. However, this still will not definitively address the issue of cause and effect. Given the changes occurring in state- and district-level policies across the country that require drastic amendments to CBAs, future research may be able to exploit these policy shifts to determine if these changes impact dis- tricts’ and schools’ abilities to raise student performance and meet account- ability standards.
Strunk, McEachin
896
Third, researchers have questioned the ability of states to reliably mea- sure student achievement in a post-NCLB accountability era. There is some evidence that high-stakes state assessments may inaccurately measure stu- dent performance, especially of subgroups, and may understate proficiency gaps between poor and minority students and wealthier White students (e.g., Fuller, Gesicki, Kang, & Wright, 2006; Lee, 2006). Lee (2006) notes that states’ high-stakes tests ‘‘significantly inflate proficiency levels and pro- ficiency gains as well as deflate racial and social achievement gaps’’ when compared to national NAEP scores (p. 11). If this is the case, our results should be interpreted as suggestive. On the other hand, the overstated levels of student performance and improvement in the achievement gaps may in fact understate the differential relationship between contract strength in dis- tricts with higher proportions of poor and minority students.
Fourth, our results do not shed light on the specific mechanisms through which stronger contracts influence or restrict the actions of school and dis- trict administrators. We cannot directly link the negative relationship between stronger contracts and school and district outcomes to district ad- ministrators’ discretion to enact specific reforms, such as those surrounding teacher staffing practices, evaluations, or other policies outlined in the CBAs. Nor do we specifically link contract restrictiveness to teachers’ instructional practices that may be enacted or changed in response to accountability threats. This work instead focuses on determining whether more restrictive contracts are associated with school and district performance on student out- comes required by NCLB, regardless of the specifics of what actions and pol- icies are or are not constrained as a result of contract restrictiveness. To that end, we find evidence of significant negative relationships between contract restrictiveness and school and district outcomes, which suggest that more restrictive contracts, regardless of the differences in specific provisions among districts, are associated with diminished performance, especially in schools and districts’ with difficult working conditions.
Future research would benefit from looking at how restrictive contracts influence the actions of school and district administrators in their attempts to raise student achievement in light of accountability pressures. For example, research might explore whether stronger contracts restrict schools and dis- tricts in higher stages of PI from firing low-performing teachers, assigning certain students to specific teachers, or choosing specific curricula for use in classrooms more than in districts or schools facing less accountability pressure. Similarly, future research might address the relationship between contract restrictiveness and the turnover of key educational actors such as teachers, principals, and superintendents, especially in districts and schools facing accountability pressures. The results from addressing these research questions might help district and union officials negotiate contracts that allow schools and districts more flexibility in setting policies and
Accountability Under Constraint
897
implementing practices to help them improve student achievement and meet accountability goals.
VII. Discussion and Policy Implications
In the current high-stakes world of K–12 education policy under NCLB, school and district administrators are held directly responsible for student performance. The incentives are high for these individual actors to facilitate change in schools that will enable more students to achieve proficiency in core subject areas and to graduate. However, it is not clear that schools and districts have the capacity—either in terms of technical knowledge or in terms of flexibility to enact reforms—to rise to the high standards set by NCLB. This study is one of the first to explore one of a number of factors that may constrain district and school administrators as they attempt to meet the goals set out by NCLB. Specifically, we are the first to examine how the level of contract rigidity may or may not be associated with school and district outcomes under NCLB.
Contracts are negotiated by teachers’ unions and district administrations or school boards with the intent of setting regulations that will protect both teachers and districts or schools from arbitrary or unwarranted actions. What constitutes arbitrary or unwarranted may depend on which side is negotiating the contract—the union or the district. What constitutes capri- cious actions may also depend on the specific district context. Although contracts may contain the same provisions in two different districts, one or the other side may be less inclined to enforce those regulations if work- ing conditions are good and relations are favorable between the union and the administration. As such, two very similar contracts, with the same degree of bargained rigidity, may have different impacts on school and dis- trict administrators’ actions, and on student outcomes in districts with dif- ferent working conditions. Specifically, teachers, union representatives, and administrators in districts and schools with harder working conditions may be more inclined to enforce contract regulations that will improve working conditions. If these regulations constrain administrators or teach- ers from acting in ways that improve student performance and enable dis- tricts and schools to meet NCLB targets, then we might see the negative interactions that are apparent in our results.
We find that contract restrictiveness is negatively associated with student performance and districts’ abilities to meet accountability goals. We also find that contract restrictiveness is more consistently and more negatively associ- ated with district than with school performance in NCLB (PI status and level), even though contract rigidity is associated with the proportion of students meeting proficiency and graduation rates in both schools and districts. That contract strength is similarly associated with actual student performance in both schools and districts makes sense; students in districts with more
Strunk, McEachin
898
restrictive contracts appear to fare worse, on average, and this impacts both districts’ and schools’ performance levels. It is not obvious, however, why this does not translate into a greater likelihood of schools entering into and progressing through PI. It is possible that, because contracts are nego- tiated at the district level but many provisions are enforced at the school level, teachers’ unions and administrators at the school level have more lee- way to work around contract provisions and adjust contract enforcement to fit their particular needs.
In addition, we see that graduation rates are always more directly sensi- tive to contract restrictiveness than ELA or math proficiency rates. This find- ing supports extant research on the impacts of teachers’ unions themselves on students across the performance distribution. Specifically, Hoxby (1996) finds that unionized districts and districts with stronger unions have higher dropout rates. This is contrary to Eberts and Stone’s (1987) finding that unions are positively associated with average student performance. Together, these results can be interpreted to suggest that unions themselves may increase average student performance but reduce it at the lower tail of the performance distribution. Our results support the latter half of this con- clusion—that stronger unions, or at least stronger contracts, are associated with higher dropout rates. That we find that districts and schools in districts with more restrictive contracts have lower proficiency rates is also in line with more recent research that finds negative associations between CBA restrictiveness and API scores in California school districts (Moe, 2009; Strunk, in press).
Our results have several implications for policymakers at the local, state, and national levels. While we do not provide evidence on how to improve collective bargaining agreements, the central lesson is about the ability of districts and schools to meet accountability requirements when administra- tors’ and teachers’ abilities to change their practice and processes are con- strained by restrictive contracts. Policymakers who would hold school and district administrators and teachers accountable for student performance under high-stakes accountability regimes such as NCLB should consider the capacity such actors have to meet set requirements. Incentives only bring about intended results when those receiving the incentives can actually change their actions. It is possible that unintended consequences of account- ability policies, such as cheating or teaching to the test, may be more likely to occur in districts and schools that cannot implement other reforms that may increase student achievement.
Policymakers also may wish to consider the ways in which accountabil- ity policies such as NCLB may be more or less effective in districts with dif- ferent working conditions. We provide suggestive evidence that contractual limits on administrators and teachers may be more harmful in districts that are also constrained by harder working conditions.
Accountability Under Constraint
899
Our results can also inform the current debates occurring across the country and within many states about whether and how to reform the col- lective bargaining process and the resulting collective bargaining agree- ments. Our results indicate that state policymakers and teachers’ associations may wish to consider whether there is a place for state regula- tions concerning the items districts may include in their CBAs. If this and future research continues to indicate that restrictive contracts or restrictive elements of contracts inhibit district and school administrators and/or teach- ers from enacting important reforms or instructional strategies, there may be a role for state regulation of certain elements of collective bargaining. In addition, our results suggest that district administrators and local unions may need to take a close look at the contents of their CBAs. For instance, if restrictive CBAs inherently protect the working conditions and job security of teachers regardless of their effectiveness in raising student performance, and constrain district administrators from taking actions to promote student learning, then the negative association between contract rigidity and student outcomes can be diminished if these protections are removed.
On the other hand, our results indicate that certain kinds of districts—specifically those that are particularly hard to staff—may face greater performance challenges when constrained by strong CBAs. This points to the need for locally determined contracts that allow local district actors to consider the specific needs of their district in contract negotia- tions. Our results also imply a warning for district policymakers—the union representatives and administrators who negotiate and enforce CBAs. When negotiating and enforcing contract regulations, it may be important to con- sider what flexibilities may allow districts and schools to respond to accountability pressures. In today’s context, in which districts and schools, teachers, and ultimately students, will be greatly impacted by failures to achieve and/or to meet specific achievement targets, the regulations included in contracts and the extent to which they are enforced can have strong implications for student, school, and district success under NCLB and similar accountability policies.
Acknowledgement
The authors gratefully acknowledge helpful comments from Tammy Kolbe and from discussants and participants at the 2010 Annual Conference of the American Education Finance Association. We also appreciate the helpful suggestions from the editor and anonymous referees. All remaining errors are our own.
Notes 1Because our study focuses on California schools and districts, we primarily discuss
California’s specific implementation of NCLB.
Strunk, McEachin
900
2There are only three levels of Program Improvement status for districts (PI1 through PI3), after which districts are deemed PI31. There are five levels of Program Improvement for schools (PI1 through PI5), after which schools are deemed PI51. Because California had its own accountability system before NCLB was implemented, schools were able to be in Program Improvement as early as 2000–2001 and could be in PI3 as early as 2003–2004. Districts could be in PI for the first time in 2004–2005 and in PI3 in 2006–2007.
3The sample of school districts has substantively similar district characteristics as the larger population of California school districts with four or more schools.
4 Results available from the authors upon request.
References
Balfanz, R., Legters, N., West, T. C., & Weber, L. M. (2007). Are NCLB’s measures, in- centives, and improvement strategies the right ones for the nation’s low- performing schools? American Educational Research Journal, 44(3), 559–593.
California Department of Education. (2006, September). PI training for PI status. Retrieved June 29, 2010, from http://www.cacompcenter.org/pi-ta/PI_YR3.pdf and from http://www.cde.ca.gov/ta/lp/vl/documents/pitrngyr4and5.docwww .cde.ca.gov/ta/lp/vl/documents/pitrngyr4and5.doc
Carnoy, M., & Loeb, S. (2002). Does external accountability affect student outcomes? A cross-state analysis. Educational Evaluation and Policy Analysis, 24(4), 305–331.
Chaing, H. (2009). How accountability pressure on failing schools affects student achievement. Journal of Public Economics, 93, 1045–1057.
Cullen, J., & Reback, R. (2006). Tinkering towards accolades. In T. Gronberg & D. Jansen (Eds.), Advances in microeconomics (14th ed.). Cambridge, MA: Emerald Publishing.
Eberts, R. W. (1983). How unions affect management decisions: Evidence from public schools. Journal of Labor Research, 4(3), 239–247.
Eberts, R. W., & Stone, J. A. (1984). Unions and public schools: The effect of collective bargaining on American education. Lexington, MA: Lexington Books.
Eberts, R. W., & Stone, J. A. (1987). Teachers unions and the productivity of public schools. Industrial and Labor Relations Review, 40(3), 354–363.
Elmore, R. F., Abelmann, C., & Fuhrman, S. (1996). The new accountability in state education reform: From process to performance. In H. F. Ladd (Ed.), Holding schools accountable: Performance-based reform in education (pp. 65–98). Washington, DC: Brookings Institution Press.
Figlio, D., & Ladd, H. F. (2008). School accountability and student achievement. In H. F. Ladd & E. B. Fiske (Eds.), Handbook of research in education finance and policy (pp. 166–182). New York, NY: Routledge.
Figlio, D., & Rouse, C. (2006). Do accountability and voucher threats improve low- performing schools? Journal of Public Economics, 89(2), 381–394.
Fuller, B., Gesicki, K., Kang, E., & Wright, J. (2006). Is the No Child Left Behind Act working? The reliability of how states track achievement (PACE Working Paper 06-1). Sacramento, CA: PACE.
Hamilton, L., Berends, M., & Stecher, B. (2005). Teachers’ responses to standards- based accountability (RAND Working Paper 259-EDU). Santa Monica, CA: RAND.
Hanushek, E. A., Kain, J. F., & Rivkin, S. G. (2004). Why public schools lose teachers. Journal of Human Resources, 39(2), 326–354.
Accountability Under Constraint
901
Hanushek, E. A., & Raymond, M. (2005). Does school accountability lead to improved student performance? Journal of Policy Analysis and Management, 24(2), 297–329.
Hess, F. M., & Kelly, A. P. (2006). Scapegoat, albatross, or what? In J. Hannaway & A. J. Rotherham (Eds.), Collective bargaining in education: Negotiating change in today’s schools (pp. 53–87). Cambridge, MA: Harvard Education Press.
Hess, F. M., & Loup, C. (2008). The leadership limbo: Teacher labor agreements in America’s fifty largest school districts. Washington, DC: Thomas B. Fordham Institute.
Hill, P. T., & Lake, R. J. (2002). Standards and accountability in Washington State. Brookings Papers on Education Policy, 5, 199–225.
Hoxby, C. M. (1996). How teachers unions affect education production. The Quarterly Journal of Economics, 111(3), 617–718.
Jacob, B. A. (2005). Accountability, incentives and behavior. Journal of Public Economics, 89(5–6), 761–796.
Jacob, B. A., & Levitt, S. (2003). Rotten apples: An investigation of the prevalence and predictors of teacher cheating. Quarterly Journal of Economics, 118(3), 843–877.
Koski, W. S., & Horng, E. L. (2007). Curbing or facilitating inequality? Law, collective bargaining, and teacher assignment among schools in California. Journal of Education Finance and Policy, 2(3), 262–300.
Ladd, H. (1996). Holding schools accountable: Performance-based reform in educa- tion. Washington, DC: Brookings Institute Press.
Ladd, H., & Lauen, D. L. (2010). Status versus growth: The distributional effect of school accountability policies. Journal of Policy Analysis and Management, 29(3), 426–450.
Lankford, H., Loeb, S., & Wyckoff, J. (2002). Teacher sorting and the plight of urban schools: A descriptive analysis. Educational Evaluation and Policy Analysis, 24(1), 37–62.
Lee, J. (2006). Tracking achievement gaps and assessing the impact of NCLB on the gaps: An in-depth look into national and state reading and math outcome trends. Cambridge, MA: The Civil Rights Project at Harvard University.
Levin, J., Mulhern, J., & Schunck, J. (2005). Unintended consequences: The case for reforming staffing rules in urban teachers union contracts. New York, NY: New Teacher Project.
Loeb, S., & Page, M. (2000). Examining the link between teacher wages and out- comes: The importance of alternative labor market opportunities and non- pecuniary variation. Review of Economics and Statistics, 82(3), 393–408.
Loeb, S., & Strunk, K. O. (2007). Accountability and local control: Response to incen- tives with and without authority over resource generation and allocation. Education Finance and Policy, 2(1), 10–39.
Moe, T. M. (2009). Collective bargaining and the performance of the public schools. American Journal of Political Science, 53(1), 156–174.
Petrelli, M. J. (2002). Comment on standards and accountability in Washington state. Brookings Papers on Education Policy, 5, 226–228.
Price, M. (2009). Teacher union contracts and high school reform (Center of Reinventing Public Education Working Paper). Seattle: University of Washington.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.
Reardon, S. F., & Raudenbush, S. W. (2006). A partial independence item response model for surveys with filter questions. Sociological Methodology, 36, 257–300.
Strunk, McEachin
902
Rouse, C. E., Hannaway, J., Goldhaber, D., & Figlio, D. (2007). Feeling the Florida heat? How low performing schools respond to voucher and accountability pres- sure (CALDER Working Paper 13). Washington, DC.
Springer, M. G. (2008). The influence of an NCLB accountability plan on the distribu- tion of student test score gains. Economics of Education Review, 27(5), 556–563.
Strunk, K. O. (2009). California teachers’ union contracts and negotiations: Moving beyond the stereotype. Sacramento, CA: PACE.
Strunk, K. O. (in press). Are teachers’ unions really to blame? The relationships between teachers’ union contract restrictiveness, district resource allocation and student test scores in California. Education Finance and Policy.
Strunk, K. O., & Grissom, J. A. (2010). Do strong unions shape district policies? Collective bargaining, teacher contract restrictiveness, and the political power of teachers’ unions. Educational Evaluation and Policy Analysis, 32(3), 389–406.
Strunk, K. O., & Reardon, S. (2010). Measuring union strength: A partial indepen- dence item response approach to measuring the restrictiveness of teachers’ union contracts. Journal of Educational and Behavioral Statistics, 35(6), 629–670.
U.S. Department of Education, Office of Planning, Analysis and Policy Development. (2010, March). A blueprint for reform: Reauthorizing the Elementary and Secondary Education Act. Washington, DC: Author.
Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness. New York, NY: The New Teacher.
Manuscript received June 29, 2010 Final revision received January 22, 2011
Accepted January 24, 2011
Accountability Under Constraint
903
<< /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /None /Binding /Left /CalGrayProfile (Dot Gain 20%) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Error /CompatibilityLevel 1.3 /CompressObjects /Off /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages true /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.1000 /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams false /MaxSubsetPct 100 /Optimize false /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness true /PreserveHalftoneInfo false /PreserveOPIComments false /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Remove /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true /ACaslon-Bold /ACaslon-BoldItalic /ACaslon-Italic /ACaslon-Ornaments /ACaslon-Regular /ACaslon-Semibold /ACaslon-SemiboldItalic /AdobeCorpID-Acrobat /AdobeCorpID-Adobe /AdobeCorpID-Bullet /AdobeCorpID-MinionBd /AdobeCorpID-MinionBdIt /AdobeCorpID-MinionRg /AdobeCorpID-MinionRgIt /AdobeCorpID-MinionSb /AdobeCorpID-MinionSbIt /AdobeCorpID-MyriadBd /AdobeCorpID-MyriadBdIt /AdobeCorpID-MyriadBdScn /AdobeCorpID-MyriadBdScnIt /AdobeCorpID-MyriadBl /AdobeCorpID-MyriadBlIt /AdobeCorpID-MyriadLt /AdobeCorpID-MyriadLtIt /AdobeCorpID-MyriadPkg /AdobeCorpID-MyriadRg /AdobeCorpID-MyriadRgIt /AdobeCorpID-MyriadRgScn /AdobeCorpID-MyriadRgScnIt /AdobeCorpID-MyriadSb /AdobeCorpID-MyriadSbIt /AdobeCorpID-MyriadSbScn /AdobeCorpID-MyriadSbScnIt /AdobeCorpID-PScript /AGaramond-BoldScaps /AGaramond-Italic /AGaramond-Regular /AGaramond-RomanScaps /AGaramond-Semibold /AGaramond-SemiboldItalic /AGar-Special /AkzidenzGroteskBE-Bold /AkzidenzGroteskBE-BoldEx /AkzidenzGroteskBE-BoldExIt /AkzidenzGroteskBE-BoldIt /AkzidenzGroteskBE-Ex /AkzidenzGroteskBE-It /AkzidenzGroteskBE-Light /AkzidenzGroteskBE-LightEx /AkzidenzGroteskBE-LightOsF /AkzidenzGroteskBE-Md /AkzidenzGroteskBE-MdEx /AkzidenzGroteskBE-MdIt /AkzidenzGroteskBE-Regular /AkzidenzGroteskBE-Super /AlbertusMT /AlbertusMT-Italic /AlbertusMT-Light /Aldine401BT-BoldA /Aldine401BT-BoldItalicA /Aldine401BT-ItalicA /Aldine401BT-RomanA /Aldine401BTSPL-RomanA /Aldine721BT-Bold /Aldine721BT-BoldItalic /Aldine721BT-Italic /Aldine721BT-Light /Aldine721BT-LightItalic /Aldine721BT-Roman /Aldus-Italic /Aldus-ItalicOsF /Aldus-Roman /Aldus-RomanSC /AlternateGothicNo2BT-Regular /AmazoneBT-Regular /AmericanTypewriter-Bold /AmericanTypewriter-BoldA /AmericanTypewriter-BoldCond /AmericanTypewriter-BoldCondA /AmericanTypewriter-Cond /AmericanTypewriter-CondA /AmericanTypewriter-Light /AmericanTypewriter-LightA /AmericanTypewriter-LightCond /AmericanTypewriter-LightCondA /AmericanTypewriter-Medium /AmericanTypewriter-MediumA /Anna /AntiqueOlive-Bold /AntiqueOlive-Compact /AntiqueOlive-Italic /AntiqueOlive-Roman /Arcadia /Arcadia-A /Arkona-Medium /Arkona-Regular /ArrusBT-Black /ArrusBT-BlackItalic /ArrusBT-Bold /ArrusBT-BoldItalic /ArrusBT-Italic /ArrusBT-Roman /AssemblyLightSSK /AuroraBT-BoldCondensed /AuroraBT-RomanCondensed /AuroraOpti-Condensed /AvantGarde-Book /AvantGarde-BookOblique /AvantGarde-Demi /AvantGarde-DemiOblique /Avenir-Black /Avenir-BlackOblique /Avenir-Book /Avenir-BookOblique /Avenir-Heavy /Avenir-HeavyOblique /Avenir-Light /Avenir-LightOblique /Avenir-Medium /Avenir-MediumOblique /Avenir-Oblique /Avenir-Roman /BaileySansITC-Bold /BaileySansITC-BoldItalic /BaileySansITC-Book /BaileySansITC-BookItalic /BakerSignetBT-Roman /BaskervilleBE-Italic /BaskervilleBE-Medium /BaskervilleBE-MediumItalic /BaskervilleBE-Regular /Baskerville-Bold /BaskervilleBook-Italic /BaskervilleBook-MedItalic /BaskervilleBook-Medium /BaskervilleBook-Regular /BaskervilleBT-Bold /BaskervilleBT-BoldItalic /BaskervilleBT-Italic /BaskervilleBT-Roman /BaskervilleMT /BaskervilleMT-Bold /BaskervilleMT-BoldItalic /BaskervilleMT-Italic /BaskervilleMT-SemiBold /BaskervilleMT-SemiBoldItalic /BaskervilleNo2BT-Bold /BaskervilleNo2BT-BoldItalic /BaskervilleNo2BT-Italic /BaskervilleNo2BT-Roman /Baskerville-Normal-Italic /BauerBodoni-Black /BauerBodoni-BlackCond /BauerBodoni-BlackItalic /BauerBodoni-Bold /BauerBodoni-BoldCond /BauerBodoni-BoldItalic /BauerBodoni-BoldItalicOsF /BauerBodoni-BoldOsF /BauerBodoni-Italic /BauerBodoni-ItalicOsF /BauerBodoni-Roman /BauerBodoni-RomanSC /Bauhaus-Bold /Bauhaus-Demi /Bauhaus-Heavy /BauhausITCbyBT-Bold /BauhausITCbyBT-Heavy /BauhausITCbyBT-Light /BauhausITCbyBT-Medium /Bauhaus-Light /Bauhaus-Medium /BellCentennial-Address /BellGothic-Black /BellGothic-Bold /Bell-GothicBoldItalicBT /BellGothicBT-Bold /BellGothicBT-Roman /BellGothic-Light /Bembo /Bembo-Bold /Bembo-BoldExpert /Bembo-BoldItalic /Bembo-BoldItalicExpert /Bembo-Expert /Bembo-ExtraBoldItalic /Bembo-Italic /Bembo-ItalicExpert /Bembo-Semibold /Bembo-SemiboldItalic /Benguiat-Bold /Benguiat-BoldItalic /Benguiat-Book /Benguiat-BookItalic /BenguiatGothicITCbyBT-Bold /BenguiatGothicITCbyBT-BoldItal /BenguiatGothicITCbyBT-Book /BenguiatGothicITCbyBT-BookItal /BenguiatITCbyBT-Bold /BenguiatITCbyBT-BoldItalic /BenguiatITCbyBT-Book /BenguiatITCbyBT-BookItalic /Benguiat-Medium /Benguiat-MediumItalic /Berkeley-Black /Berkeley-BlackItalic /Berkeley-Bold /Berkeley-BoldItalic /Berkeley-Book /Berkeley-BookItalic /Berkeley-Italic /Berkeley-Medium /Berling-Bold /Berling-BoldItalic /Berling-Italic /Berling-Roman /BernhardBoldCondensedBT-Regular /BernhardFashionBT-Regular /BernhardModernBT-Bold /BernhardModernBT-BoldItalic /BernhardModernBT-Italic /BernhardModernBT-Roman /BernhardTangoBT-Regular /BlockBE-Condensed /BlockBE-ExtraCn /BlockBE-ExtraCnIt /BlockBE-Heavy /BlockBE-Italic /BlockBE-Regular /Bodoni /Bodoni-Bold /Bodoni-BoldItalic /Bodoni-Italic /Bodoni-Poster /Bodoni-PosterCompressed /Bookman-Demi /Bookman-DemiItalic /Bookman-Light /Bookman-LightItalic /Boton-Italic /Boton-Medium /Boton-MediumItalic /Boton-Regular /Boulevard /BremenBT-Black /BremenBT-Bold /BroadwayBT-Regular /CaflischScript-Bold /CaflischScript-Regular /Caliban /CarminaBT-Bold /CarminaBT-BoldItalic /CarminaBT-Light /CarminaBT-LightItalic /CarminaBT-Medium /CarminaBT-MediumItalic /Carta /Caslon224ITCbyBT-Bold /Caslon224ITCbyBT-BoldItalic /Caslon224ITCbyBT-Book /Caslon224ITCbyBT-BookItalic /Caslon540BT-Italic /Caslon540BT-Roman /CaslonBT-Bold /CaslonBT-BoldItalic /CaslonOpenFace /CaslonTwoTwentyFour-Black /CaslonTwoTwentyFour-BlackIt /CaslonTwoTwentyFour-Bold /CaslonTwoTwentyFour-BoldIt /CaslonTwoTwentyFour-Book /CaslonTwoTwentyFour-BookIt /CaslonTwoTwentyFour-Medium /CaslonTwoTwentyFour-MediumIt /CastleT-Bold /CastleT-Book /Caxton-Bold /Caxton-BoldItalic /Caxton-Book /Caxton-BookItalic /CaxtonBT-Bold /CaxtonBT-BoldItalic /CaxtonBT-Book /CaxtonBT-BookItalic /Caxton-Light /Caxton-LightItalic /CelestiaAntiqua-Ornaments /Centennial-BlackItalicOsF /Centennial-BlackOsF /Centennial-BoldItalicOsF /Centennial-BoldOsF /Centennial-ItalicOsF /Centennial-LightItalicOsF /Centennial-LightSC /Centennial-RomanSC /Century-Bold /Century-BoldItalic /Century-Book /Century-BookItalic /CenturyExpandedBT-Bold /CenturyExpandedBT-BoldItalic /CenturyExpandedBT-Italic /CenturyExpandedBT-Roman /Century-HandtooledBold /Century-HandtooledBoldItalic /Century-Light /Century-LightItalic /CenturyOldStyle-Bold /CenturyOldStyle-Italic /CenturyOldStyle-Regular /CenturySchoolbookBT-Bold /CenturySchoolbookBT-BoldCond /CenturySchoolbookBT-BoldItalic /CenturySchoolbookBT-Italic /CenturySchoolbookBT-Roman /Century-Ultra /Century-UltraItalic /CharterBT-Black /CharterBT-BlackItalic /CharterBT-Bold /CharterBT-BoldItalic /CharterBT-Italic /CharterBT-Roman /CheltenhamBT-Bold /CheltenhamBT-BoldCondItalic /CheltenhamBT-BoldExtraCondensed /CheltenhamBT-BoldHeadline /CheltenhamBT-BoldItalic /CheltenhamBT-BoldItalicHeadline /CheltenhamBT-Italic /CheltenhamBT-Roman /Cheltenham-HandtooledBdIt /Cheltenham-HandtooledBold /CheltenhamITCbyBT-Bold /CheltenhamITCbyBT-BoldItalic /CheltenhamITCbyBT-Book /CheltenhamITCbyBT-BookItalic /Christiana-Bold /Christiana-BoldItalic /Christiana-Italic /Christiana-Medium /Christiana-MediumItalic /Christiana-Regular /Christiana-RegularExpert /Christiana-RegularSC /Clarendon /Clarendon-Bold /Clarendon-Light /ClassicalGaramondBT-Bold /ClassicalGaramondBT-BoldItalic /ClassicalGaramondBT-Italic /ClassicalGaramondBT-Roman /CMR10 /CMR8 /CMSY10 /CMSY8 /CMTI10 /CommonBullets /ConduitITC-Bold /ConduitITC-BoldItalic /ConduitITC-Light /ConduitITC-LightItalic /ConduitITC-Medium /ConduitITC-MediumItalic /CooperBlack /CooperBlack-Italic /CooperBT-Bold /CooperBT-BoldItalic /CooperBT-Light /CooperBT-LightItalic /CopperplateGothicBT-Bold /CopperplateGothicBT-BoldCond /CopperplateGothicBT-Heavy /CopperplateGothicBT-Roman /CopperplateGothicBT-RomanCond /Copperplate-ThirtyThreeBC /Copperplate-ThirtyTwoBC /Coronet-Regular /Courier /Courier-Bold /Courier-BoldOblique /Courier-Oblique /Critter /CS-Special-font /DellaRobbiaBT-Bold /DellaRobbiaBT-Roman /Della-RobbiaItalicBT /Della-RobbiaSCaps /Del-NormalSmallCaps /Delphin-IA /Delphin-IIA /Delta-Bold /Delta-BoldItalic /Delta-Book /Delta-BookItalic /Delta-Light /Delta-LightItalic /Delta-Medium /Delta-MediumItalic /Delta-Outline /DextorD /DextorOutD /DidotLH-OrnamentsOne /DidotLH-OrnamentsTwo /DINEngschrift /DINEngschrift-Alternate /DINMittelschrift /DINMittelschrift-Alternate /DINNeuzeitGrotesk-BoldCond /DINNeuzeitGrotesk-Light /Dom-CasItalic /DomCasual /DomCasual-Bold /Dom-CasualBT /Ehrhard-Italic /Ehrhard-Regular /EhrhardSemi-Italic /EhrhardtMT /EhrhardtMT-Italic /EhrhardtMT-SemiBold /EhrhardtMT-SemiBoldItalic /EhrharSemi /ELANGO-IB-A03 /ELANGO-IB-A75 /ELANGO-IB-A99 /ElectraLH-Bold /ElectraLH-BoldCursive /ElectraLH-Cursive /ElectraLH-Regular /ElGreco /EnglischeSchT-Bold /EnglischeSchT-Regu /ErasContour /ErasITCbyBT-Bold /ErasITCbyBT-Book /ErasITCbyBT-Demi /ErasITCbyBT-Light /ErasITCbyBT-Medium /ErasITCbyBT-Ultra /Euclid /Euclid-Bold /Euclid-BoldItalic /EuclidExtra /EuclidExtra-Bold /EuclidFraktur /EuclidFraktur-Bold /Euclid-Italic /EuclidMathOne /EuclidMathOne-Bold /EuclidMathTwo /EuclidMathTwo-Bold /EuclidSymbol /EuclidSymbol-Bold /EuclidSymbol-BoldItalic /EuclidSymbol-Italic /EUEX10 /EUFB10 /EUFB5 /EUFB7 /EUFM10 /EUFM5 /EUFM7 /EURB10 /EURB5 /EURB7 /EURM10 /EURM5 /EURM7 /EuropeanPi-Four /EuropeanPi-One /EuropeanPi-Three /EuropeanPi-Two /EuroSans-Bold /EuroSans-BoldItalic /EuroSans-Italic /EuroSans-Regular /EuroSerif-Bold /EuroSerif-BoldItalic /EuroSerif-Italic /EuroSerif-Regular /Eurostile /Eurostile-Bold /Eurostile-BoldCondensed /Eurostile-BoldExtendedTwo /Eurostile-BoldOblique /Eurostile-Condensed /Eurostile-Demi /Eurostile-DemiOblique /Eurostile-ExtendedTwo /EurostileLTStd-Demi /EurostileLTStd-DemiOblique /Eurostile-Oblique /EUSB10 /EUSB5 /EUSB7 /EUSM10 /EUSM5 /EUSM7 /ExPonto-Regular /FairfieldLH-Bold /FairfieldLH-BoldItalic /FairfieldLH-BoldSC /FairfieldLH-CaptionBold /FairfieldLH-CaptionHeavy /FairfieldLH-CaptionLight /FairfieldLH-CaptionMedium /FairfieldLH-Heavy /FairfieldLH-HeavyItalic /FairfieldLH-HeavySC /FairfieldLH-Light /FairfieldLH-LightItalic /FairfieldLH-LightSC /FairfieldLH-Medium /FairfieldLH-MediumItalic /FairfieldLH-MediumSC /FairfieldLH-SwBoldItalicOsF /FairfieldLH-SwHeavyItalicOsF /FairfieldLH-SwLightItalicOsF /FairfieldLH-SwMediumItalicOsF /Fences /Fenice-Bold /Fenice-BoldOblique /FeniceITCbyBT-Bold /FeniceITCbyBT-BoldItalic /FeniceITCbyBT-Regular /FeniceITCbyBT-RegularItalic /Fenice-Light /Fenice-LightOblique /Fenice-Regular /Fenice-RegularOblique /Fenice-Ultra /Fenice-UltraOblique /FlashD-Ligh /Flood /Folio-Bold /Folio-BoldCondensed /Folio-ExtraBold /Folio-Light /Folio-Medium /FontanaNDAaOsF /FontanaNDAaOsF-Italic /FontanaNDCcOsF-Semibold /FontanaNDCcOsF-SemiboldIta /FontanaNDEeOsF /FontanaNDEeOsF-Bold /FontanaNDEeOsF-BoldItalic /FontanaNDEeOsF-Light /FontanaNDEeOsF-Semibold /FormalScript421BT-Regular /Formata-Bold /Formata-MediumCondensed /ForteMT /FournierMT-Ornaments /FrakturBT-Regular /FrankfurterHigD /FranklinGothic-Book /FranklinGothic-BookItal /FranklinGothic-BookOblique /FranklinGothic-Condensed /FranklinGothic-Demi /FranklinGothic-DemiItal /FranklinGothic-DemiOblique /FranklinGothic-Heavy /FranklinGothic-HeavyItal /FranklinGothic-HeavyOblique /FranklinGothicITCbyBT-BookItal /FranklinGothicITCbyBT-Demi /FranklinGothicITCbyBT-DemiItal /FranklinGothicITCbyBT-Heavy /FranklinGothicITCbyBT-HeavyItal /FranklinGothic-Medium /FranklinGothic-MediumItal /FranklinGothic-Roman /Freeform721BT-Bold /Freeform721BT-BoldItalic /Freeform721BT-Italic /Freeform721BT-Roman /FreestyleScrD /FreestyleScript /Freestylescript /FrizQuadrataITCbyBT-Bold /FrizQuadrataITCbyBT-Roman /Frutiger-Black /Frutiger-BlackCn /Frutiger-BlackItalic /Frutiger-Bold /Frutiger-BoldCn /Frutiger-BoldItalic /Frutiger-Cn /Frutiger-ExtraBlackCn /Frutiger-Italic /Frutiger-Light /Frutiger-LightCn /Frutiger-LightItalic /Frutiger-Roman /Frutiger-UltraBlack /Futura /FuturaBlackBT-Regular /Futura-Bold /Futura-BoldOblique /Futura-Book /Futura-BookOblique /FuturaBT-Bold /FuturaBT-BoldCondensed /FuturaBT-BoldCondensedItalic /FuturaBT-BoldItalic /FuturaBT-Book /FuturaBT-BookItalic /FuturaBT-ExtraBlack /FuturaBT-ExtraBlackCondensed /FuturaBT-ExtraBlackCondItalic /FuturaBT-ExtraBlackItalic /FuturaBT-Heavy /FuturaBT-HeavyItalic /FuturaBT-Light /FuturaBT-LightCondensed /FuturaBT-LightItalic /FuturaBT-Medium /FuturaBT-MediumCondensed /FuturaBT-MediumItalic /Futura-CondensedLight /Futura-CondensedLightOblique /Futura-ExtraBold /Futura-ExtraBoldOblique /Futura-Heavy /Futura-HeavyOblique /Futura-Light /Futura-LightOblique /Futura-Oblique /Futura-Thin /Galliard-Black /Galliard-BlackItalic /Galliard-Bold /Galliard-BoldItalic /Galliard-Italic /GalliardITCbyBT-Bold /GalliardITCbyBT-BoldItalic /GalliardITCbyBT-Italic /GalliardITCbyBT-Roman /Galliard-Roman /Galliard-Ultra /Galliard-UltraItalic /Garamond-Antiqua /GaramondBE-Bold /GaramondBE-BoldExpert /GaramondBE-BoldOsF /GaramondBE-CnExpert /GaramondBE-Condensed /GaramondBE-CondensedSC /GaramondBE-Italic /GaramondBE-ItalicExpert /GaramondBE-ItalicOsF /GaramondBE-Medium /GaramondBE-MediumCn /GaramondBE-MediumCnExpert /GaramondBE-MediumCnOsF /GaramondBE-MediumExpert /GaramondBE-MediumItalic /GaramondBE-MediumItalicExpert /GaramondBE-MediumItalicOsF /GaramondBE-MediumSC /GaramondBE-Regular /GaramondBE-RegularExpert /GaramondBE-RegularSC /GaramondBE-SwashItalic /Garamond-Bold /Garamond-BoldCondensed /Garamond-BoldCondensedItalic /Garamond-BoldItalic /Garamond-Book /Garamond-BookCondensed /Garamond-BookCondensedItalic /Garamond-BookItalic /Garamond-Halbfett /Garamond-HandtooledBold /Garamond-HandtooledBoldItalic /GaramondITCbyBT-Bold /GaramondITCbyBT-BoldCondensed /GaramondITCbyBT-BoldCondItalic /GaramondITCbyBT-BoldItalic /GaramondITCbyBT-BoldNarrow /GaramondITCbyBT-BoldNarrowItal /GaramondITCbyBT-Book /GaramondITCbyBT-BookCondensed /GaramondITCbyBT-BookCondItalic /GaramondITCbyBT-BookItalic /GaramondITCbyBT-BookNarrow /GaramondITCbyBT-BookNarrowItal /GaramondITCbyBT-Light /GaramondITCbyBT-LightCondensed /GaramondITCbyBT-LightCondItalic /GaramondITCbyBT-LightItalic /GaramondITCbyBT-LightNarrow /GaramondITCbyBT-LightNarrowItal /GaramondITCbyBT-Ultra /GaramondITCbyBT-UltraCondensed /GaramondITCbyBT-UltraCondItalic /GaramondITCbyBT-UltraItalic /Garamond-Kursiv /Garamond-KursivHalbfett /Garamond-Light /Garamond-LightCondensed /Garamond-LightCondensedItalic /Garamond-LightItalic /GaramondNo4CyrTCY-Ligh /GaramondNo4CyrTCY-LighItal /GaramondThree /GaramondThree-Bold /GaramondThree-BoldItalic /GaramondThree-BoldItalicOsF /GaramondThree-BoldSC /GaramondThree-Italic /GaramondThree-ItalicOsF /GaramondThree-SC /GaramondThreeSMSIISpl-Italic /GaramondThreeSMSitalicSpl-Italic /GaramondThreeSMSspl /GaramondThreespl /GaramondThreeSpl-Bold /GaramondThreeSpl-Italic /Garamond-Ultra /Garamond-UltraCondensed /Garamond-UltraCondensedItalic /Garamond-UltraItalic /GarthGraphic /GarthGraphic-Black /GarthGraphic-Bold /GarthGraphic-BoldCondensed /GarthGraphic-BoldItalic /GarthGraphic-Condensed /GarthGraphic-ExtraBold /GarthGraphic-Italic /Geometric231BT-HeavyC /GeometricSlab712BT-BoldA /GeometricSlab712BT-ExtraBoldA /GeometricSlab712BT-LightA /GeometricSlab712BT-LightItalicA /GeometricSlab712BT-MediumA /GeometricSlab712BT-MediumItalA /Giddyup /Giddyup-Thangs /GillSans /GillSans-Bold /GillSans-BoldCondensed /GillSans-BoldExtraCondensed /GillSans-BoldItalic /GillSans-Condensed /GillSans-ExtraBold /GillSans-ExtraBoldDisplay /GillSans-Italic /GillSans-Light /GillSans-LightItalic /GillSans-LightShadowed /GillSans-Shadowed /GillSans-UltraBold /GillSans-UltraBoldCondensed /Gill-Special /Giovanni-Bold /Giovanni-BoldItalic /Giovanni-Book /Giovanni-BookItalic /Glypha /Glypha-Bold /Glypha-BoldOblique /Glypha-Oblique /Gothic-Thirteen /Goudy /Goudy-Bold /Goudy-BoldItalic /GoudyCatalogueBT-Regular /Goudy-ExtraBold /GoudyHandtooledBT-Regular /GoudyHeavyfaceBT-Regular /GoudyHeavyfaceBT-RegularCond /Goudy-Italic /GoudyOldStyleBT-Bold /GoudyOldStyleBT-BoldItalic /GoudyOldStyleBT-ExtraBold /GoudyOldStyleBT-Italic /GoudyOldStyleBT-Roman /GoudySans-Black /GoudySans-BlackItalic /GoudySans-Bold /GoudySans-BoldItalic /GoudySans-Book /GoudySans-BookItalic /GoudySansITCbyBT-Black /GoudySansITCbyBT-BlackItalic /GoudySansITCbyBT-Bold /GoudySansITCbyBT-BoldItalic /GoudySansITCbyBT-Light /GoudySansITCbyBT-LightItalic /GoudySansITCbyBT-Medium /GoudySansITCbyBT-MediumItalic /GoudySans-Medium /GoudySans-MediumItalic /Granjon /Granjon-Bold /Granjon-BoldOsF /Granjon-Italic /Granjon-ItalicOsF /Granjon-SC /GreymantleMVB-Ornaments /Helvetica /Helvetica-Black /Helvetica-BlackOblique /Helvetica-Black-SemiBold /Helvetica-Bold /Helvetica-BoldOblique /Helvetica-Compressed /Helvetica-Condensed /Helvetica-Condensed-Black /Helvetica-Condensed-BlackObl /Helvetica-Condensed-Bold /Helvetica-Condensed-BoldObl /Helvetica-Condensed-Light /Helvetica-Condensed-Light-Light /Helvetica-Condensed-LightObl /Helvetica-Condensed-Oblique /Helvetica-Condensed-Thin /Helvetica-ExtraCompressed /Helvetica-Fraction /Helvetica-FractionBold /HelveticaInserat-Roman /HelveticaInserat-Roman-SemiBold /Helvetica-Light /Helvetica-LightOblique /Helvetica-Narrow /Helvetica-Narrow-Bold /Helvetica-Narrow-BoldOblique /Helvetica-Narrow-Oblique /HelveticaNeue-Black /HelveticaNeue-BlackCond /HelveticaNeue-BlackCondObl /HelveticaNeue-BlackExt /HelveticaNeue-BlackExtObl /HelveticaNeue-BlackItalic /HelveticaNeue-Bold /HelveticaNeue-BoldCond /HelveticaNeue-BoldCondObl /HelveticaNeue-BoldExt /HelveticaNeue-BoldExtObl /HelveticaNeue-BoldItalic /HelveticaNeue-Condensed /HelveticaNeue-CondensedObl /HelveticaNeue-ExtBlackCond /HelveticaNeue-ExtBlackCondObl /HelveticaNeue-Extended /HelveticaNeue-ExtendedObl /HelveticaNeue-Heavy /HelveticaNeue-HeavyCond /HelveticaNeue-HeavyCondObl /HelveticaNeue-HeavyExt /HelveticaNeue-HeavyExtObl /HelveticaNeue-HeavyItalic /HelveticaNeue-Italic /HelveticaNeue-Light /HelveticaNeue-LightCond /HelveticaNeue-LightCondObl /HelveticaNeue-LightExt /HelveticaNeue-LightExtObl /HelveticaNeue-LightItalic /HelveticaNeueLTStd-Md /HelveticaNeueLTStd-MdIt /HelveticaNeue-Medium /HelveticaNeue-MediumCond /HelveticaNeue-MediumCondObl /HelveticaNeue-MediumExt /HelveticaNeue-MediumExtObl /HelveticaNeue-MediumItalic /HelveticaNeue-Roman /HelveticaNeue-Thin /HelveticaNeue-ThinCond /HelveticaNeue-ThinCondObl /HelveticaNeue-ThinItalic /HelveticaNeue-UltraLigCond /HelveticaNeue-UltraLigCondObl /HelveticaNeue-UltraLigExt /HelveticaNeue-UltraLigExtObl /HelveticaNeue-UltraLight /HelveticaNeue-UltraLightItal /Helvetica-Oblique /Helvetica-UltraCompressed /HelvExtCompressed /HelvLight /HelvUltCompressed /Humanist521BT-Bold /Humanist521BT-BoldCondensed /Humanist521BT-BoldItalic /Humanist521BT-ExtraBold /Humanist521BT-Italic /Humanist521BT-Light /Humanist521BT-LightItalic /Humanist521BT-Roman /Humanist521BT-RomanCondensed /Humanist521BT-UltraBold /Humanist521BT-XtraBoldCondensed /Humanist531BT-BlackA /Humanist531BT-BoldA /Humanist531BT-RomanA /Humanist531BT-UltraBlackA /Humanist777BT-BlackB /Humanist777BT-BlackCondensedB /Humanist777BT-BlackItalicB /Humanist777BT-BoldB /Humanist777BT-BoldCondensedB /Humanist777BT-BoldItalicB /Humanist777BT-ExtraBlackB /Humanist777BT-ExtraBlackCondB /Humanist777BT-ItalicB /Humanist777BT-LightB /Humanist777BT-LightCondensedB /Humanist777BT-LightItalicB /Humanist777BT-RomanB /Humanist777BT-RomanCondensedB /Humanist970BT-BoldC /Humanist970BT-RomanC /HumanistSlabserif712BT-Black /HumanistSlabserif712BT-Bold /HumanistSlabserif712BT-Italic /HumanistSlabserif712BT-Roman /ICMEX10 /ICMMI8 /ICMSY8 /ICMTT8 /Iglesia-Light /ILASY8 /ILCMSS8 /ILCMSSB8 /ILCMSSI8 /Imago-Book /Imago-BookItalic /Imago-ExtraBold /Imago-ExtraBoldItalic /Imago-Light /Imago-LightItalic /Imago-Medium /Imago-MediumItalic /Industria-Inline /Industria-InlineA /Industria-Solid /Industria-SolidA /Insignia /Insignia-A /IPAExtras /IPAHighLow /IPAKiel /IPAKielSeven /IPAsans /ITCGaramondMM /ITCGaramondMM-It /JAKEOpti-Regular /JansonText-Bold /JansonText-BoldItalic /JansonText-Italic /JansonText-Roman /JansonText-RomanSC /JoannaMT /JoannaMT-Bold /JoannaMT-BoldItalic /JoannaMT-Italic /Juniper /KabelITCbyBT-Book /KabelITCbyBT-Demi /KabelITCbyBT-Medium /KabelITCbyBT-Ultra /Kaufmann /Kaufmann-Bold /KeplMM-Or2 /KisBT-Italic /KisBT-Roman /KlangMT /Kuenstler480BT-Black /Kuenstler480BT-Bold /Kuenstler480BT-BoldItalic /Kuenstler480BT-Italic /Kuenstler480BT-Roman /KunstlerschreibschD-Bold /KunstlerschreibschD-Medi /Lapidary333BT-Black /Lapidary333BT-Bold /Lapidary333BT-BoldItalic /Lapidary333BT-Italic /Lapidary333BT-Roman /LASY10 /LASY5 /LASY6 /LASY7 /LASY8 /LASY9 /LASYB10 /LatinMT-Condensed /LCIRCLE10 /LCIRCLEW10 /LCMSS8 /LCMSSB8 /LCMSSI8 /LDecorationPi-One /LDecorationPi-Two /Leawood-Black /Leawood-BlackItalic /Leawood-Bold /Leawood-BoldItalic /Leawood-Book /Leawood-BookItalic /Leawood-Medium /Leawood-MediumItalic /LegacySans-Bold /LegacySans-BoldItalic /LegacySans-Book /LegacySans-BookItalic /LegacySans-Medium /LegacySans-MediumItalic /LegacySans-Ultra /LegacySerif-Bold /LegacySerif-BoldItalic /LegacySerif-Book /LegacySerif-BookItalic /LegacySerif-Medium /LegacySerif-MediumItalic /LegacySerif-Ultra /LetterGothic /LetterGothic-Bold /LetterGothic-BoldSlanted /LetterGothic-Slanted /Life-Bold /Life-Italic /Life-Roman /LINE10 /LINEW10 /Linotext /Lithos-Black /LithosBold /Lithos-Bold /Lithos-Regular /LOGO10 /LOGO8 /LOGO9 /LOGOBF10 /LOGOSL10 /LOMD-Normal /LubalinGraph-Book /LubalinGraph-BookOblique /LubalinGraph-Demi /LubalinGraph-DemiOblique /LucidaHandwritingItalic /LucidaMath-Symbol /LucidaSansTypewriter /LucidaSansTypewriter-Bd /LucidaSansTypewriter-BdObl /LucidaSansTypewriter-Obl /LucidaTypewriter /LucidaTypewriter-Bold /LucidaTypewriter-BoldObl /LucidaTypewriter-Obl /LydianBT-Bold /LydianBT-BoldItalic /LydianBT-Italic /LydianBT-Roman /LydianCursiveBT-Regular /Machine /Machine-Bold /Marigold /MathematicalPi-Five /MathematicalPi-Four /MathematicalPi-One /MathematicalPi-Six /MathematicalPi-Three /MathematicalPi-Two /MatrixScriptBold /MatrixScriptBoldLin /MatrixScriptBook /MatrixScriptBookLin /MatrixScriptRegular /MatrixScriptRegularLin /Melior /Melior-Bold /Melior-BoldItalic /Melior-Italic /MercuriusCT-Black /MercuriusCT-BlackItalic /MercuriusCT-Light /MercuriusCT-LightItalic /MercuriusCT-Medium /MercuriusCT-MediumItalic /MercuriusMT-BoldScript /Meridien-Bold /Meridien-BoldItalic /Meridien-Italic /Meridien-Medium /Meridien-MediumItalic /Meridien-Roman /Minion-Black /Minion-Bold /Minion-BoldCondensed /Minion-BoldCondensedItalic /Minion-BoldItalic /Minion-Condensed /Minion-CondensedItalic /Minion-DisplayItalic /Minion-DisplayRegular /MinionExp-Italic /MinionExp-Semibold /MinionExp-SemiboldItalic /Minion-Italic /Minion-Ornaments /Minion-Regular /Minion-Semibold /Minion-SemiboldItalic /MonaLisa-Recut /MrsEavesAllPetiteCaps /MrsEavesAllSmallCaps /MrsEavesBold /MrsEavesFractions /MrsEavesItalic /MrsEavesPetiteCaps /MrsEavesRoman /MrsEavesRomanLining /MrsEavesSmallCaps /MSAM10 /MSAM10A /MSAM5 /MSAM6 /MSAM7 /MSAM8 /MSAM9 /MSBM10 /MSBM10A /MSBM5 /MSBM6 /MSBM7 /MSBM8 /MSBM9 /MTEX /MTEXB /MTEXH /MTGU /MTGUB /MTMI /MTMIB /MTMIH /MTMS /MTMSB /MTMUB /MTMUH /MTSY /MTSYB /MTSYH /MTSYN /MusicalSymbols-Normal /Myriad-Bold /Myriad-BoldItalic /Myriad-CnBold /Myriad-CnBoldItalic /Myriad-CnItalic /Myriad-CnSemibold /Myriad-CnSemiboldItalic /Myriad-Condensed /Myriad-Italic /MyriadMM /MyriadMM-It /Myriad-Roman /Myriad-Sketch /Myriad-Tilt /NeuzeitS-Book /NeuzeitS-BookHeavy /NewBaskerville-Bold /NewBaskerville-BoldItalic /NewBaskerville-Italic /NewBaskervilleITCbyBT-Bold /NewBaskervilleITCbyBT-BoldItal /NewBaskervilleITCbyBT-Italic /NewBaskervilleITCbyBT-Roman /NewBaskerville-Roman /NewCaledonia /NewCaledonia-Black /NewCaledonia-BlackItalic /NewCaledonia-Bold /NewCaledonia-BoldItalic /NewCaledonia-BoldItalicOsF /NewCaledonia-BoldSC /NewCaledonia-Italic /NewCaledonia-ItalicOsF /NewCaledonia-SC /NewCaledonia-SemiBold /NewCaledonia-SemiBoldItalic /NewCenturySchlbk-Bold /NewCenturySchlbk-BoldItalic /NewCenturySchlbk-Italic /NewCenturySchlbk-Roman /NewsGothic /NewsGothic-Bold /NewsGothic-BoldOblique /NewsGothicBT-Bold /NewsGothicBT-BoldCondensed /NewsGothicBT-BoldCondItalic /NewsGothicBT-BoldExtraCondensed /NewsGothicBT-BoldItalic /NewsGothicBT-Demi /NewsGothicBT-DemiItalic /NewsGothicBT-ExtraCondensed /NewsGothicBT-Italic /NewsGothicBT-ItalicCondensed /NewsGothicBT-Light /NewsGothicBT-LightItalic /NewsGothicBT-Roman /NewsGothicBT-RomanCondensed /NewsGothic-Oblique /New-Symbol /NovareseITCbyBT-Bold /NovareseITCbyBT-BoldItalic /NovareseITCbyBT-Book /NovareseITCbyBT-BookItalic /Nueva-BoldExtended /Nueva-Roman /NuptialScript /OceanSansMM /OceanSansMM-It /OfficinaSans-Bold /OfficinaSans-BoldItalic /OfficinaSans-Book /OfficinaSans-BookItalic /OfficinaSerif-Bold /OfficinaSerif-BoldItalic /OfficinaSerif-Book /OfficinaSerif-BookItalic /OnyxMT /Optima /Optima-Bold /Optima-BoldItalic /Optima-BoldOblique /Optima-ExtraBlack /Optima-ExtraBlackItalic /Optima-Italic /Optima-Oblique /OSPIRE-Plain /OttaIA /Otta-wa /Ottawa-BoldA /OttawaPSMT /Oxford /Palatino-Bold /Palatino-BoldItalic /Palatino-Italic /Palatino-Roman /Parisian /Perpetua /Perpetua-Bold /Perpetua-BoldItalic /Perpetua-Italic /PhotinaMT /PhotinaMT-Bold /PhotinaMT-BoldItalic /PhotinaMT-Italic /PhotinaMT-SemiBold /PhotinaMT-SemiBoldItalic /PhotinaMT-UltraBold /PhotinaMT-UltraBoldItalic /Plantin /Plantin-Bold /Plantin-BoldItalic /Plantin-Italic /Plantin-Light /Plantin-LightItalic /Plantin-Semibold /Plantin-SemiboldItalic /Poetica-ChanceryI /Poetica-SuppLowercaseEndI /PopplLaudatio-Italic /PopplLaudatio-Medium /PopplLaudatio-MediumItalic /PopplLaudatio-Regular /ProseAntique-Bold /ProseAntique-Normal /QuaySansEF-Black /QuaySansEF-BlackItalic /QuaySansEF-Book /QuaySansEF-BookItalic /QuaySansEF-Medium /QuaySansEF-MediumItalic /Quorum-Black /Quorum-Bold /Quorum-Book /Quorum-Light /Quorum-Medium /Raleigh /Raleigh-Bold /Raleigh-DemiBold /Raleigh-Medium /Revival565BT-Bold /Revival565BT-BoldItalic /Revival565BT-Italic /Revival565BT-Roman /Ribbon131BT-Bold /Ribbon131BT-Regular /RMTMI /Rockwell /Rockwell-Bold /Rockwell-BoldItalic /Rockwell-Italic /Rockwell-Light /Rockwell-LightItalic /RotisSansSerif /RotisSansSerif-Bold /RotisSansSerif-ExtraBold /RotisSansSerif-Italic /RotisSansSerif-Light /RotisSansSerif-LightItalic /RotisSemiSans /RotisSemiSans-Bold /RotisSemiSans-ExtraBold /RotisSemiSans-Italic /RotisSemiSans-Light /RotisSemiSans-LightItalic /RotisSemiSerif /RotisSemiSerif-Bold /RotisSerif /RotisSerif-Bold /RotisSerif-Italic /RunicMT-Condensed /Sabon-Bold /Sabon-BoldItalic /Sabon-Italic /Sabon-Roman /SackersGothicLight /SackersGothicLightAlt /SackersItalianScript /SackersItalianScriptAlt /Sam /Sanvito-Light /SanvitoMM /Sanvito-Roman /Semitica /Semitica-Italic /SIVAMATH /Siva-Special /SMS-SPELA /Souvenir-Demi /Souvenir-DemiItalic /SouvenirITCbyBT-Demi /SouvenirITCbyBT-DemiItalic /SouvenirITCbyBT-Light /SouvenirITCbyBT-LightItalic /Souvenir-Light /Souvenir-LightItalic /SpecialAA /Special-Gali /Sp-Sym /StempelGaramond-Bold /StempelGaramond-BoldItalic /StempelGaramond-Italic /StempelGaramond-Roman /StoneSans /StoneSans-Bold /StoneSans-BoldItalic /StoneSans-Italic /StoneSans-PhoneticAlternate /StoneSans-PhoneticIPA /StoneSans-Semibold /StoneSans-SemiboldItalic /StoneSerif /StoneSerif-Italic /StoneSerif-PhoneticAlternate /StoneSerif-PhoneticIPA /StoneSerif-Semibold /StoneSerif-SemiboldItalic /Swiss721BT-Black /Swiss721BT-BlackCondensed /Swiss721BT-BlackCondensedItalic /Swiss721BT-BlackExtended /Swiss721BT-BlackItalic /Swiss721BT-BlackOutline /Swiss721BT-BlackRounded /Swiss721BT-Bold /Swiss721BT-BoldCondensed /Swiss721BT-BoldCondensedItalic /Swiss721BT-BoldCondensedOutline /Swiss721BT-BoldExtended /Swiss721BT-BoldItalic /Swiss721BT-BoldOutline /Swiss721BT-BoldRounded /Swiss721BT-Heavy /Swiss721BT-HeavyItalic /Swiss721BT-Italic /Swiss721BT-ItalicCondensed /Swiss721BT-Light /Swiss721BT-LightCondensed /Swiss721BT-LightCondensedItalic /Swiss721BT-LightExtended /Swiss721BT-LightItalic /Swiss721BT-Medium /Swiss721BT-MediumItalic /Swiss721BT-Roman /Swiss721BT-RomanCondensed /Swiss721BT-RomanExtended /Swiss721BT-Thin /Swiss721BT-ThinItalic /Swiss921BT-RegularA /Symbol /Syntax-Black /Syntax-Bold /Syntax-Italic /Syntax-Roman /Syntax-UltraBlack /Tekton /Times-Bold /Times-BoldA /Times-BoldItalic /Times-BoldOblique /Times-Italic /Times-NewRoman /Times-NewRomanBold /TimesNewRomanPS-BoldItalicMT /TimesNewRomanPS-ItalicMT /TimesNewRomanPSMT /Times-Oblique /Times-PhoneticAlternate /Times-PhoneticIPA /Times-Roman /Times-RomanSmallCaps /Times-Sc /Times-SCB /Times-special /TimesTenGreekP-Upright /TradeGothic /TradeGothic-Bold /TradeGothic-BoldCondTwenty /TradeGothic-BoldCondTwentyObl /TradeGothic-BoldOblique /TradeGothic-BoldTwo /TradeGothic-BoldTwoOblique /TradeGothic-CondEighteen /TradeGothic-CondEighteenObl /TradeGothicLH-BoldExtended /TradeGothicLH-Extended /TradeGothic-Light /TradeGothic-LightOblique /TradeGothic-Oblique /Trajan-Bold /TrajanPro-Bold /TrajanPro-Regular /Trajan-Regular /Transitional521BT-BoldA /Transitional521BT-CursiveA /Transitional521BT-RomanA /Transitional551BT-MediumB /Transitional551BT-MediumItalicB /Univers /Universal-GreekwithMathPi /Universal-NewswithCommPi /Univers-BlackExt /Univers-BlackExtObl /Univers-Bold /Univers-BoldExt /Univers-BoldExtObl /Univers-BoldOblique /Univers-Condensed /Univers-CondensedBold /Univers-CondensedBoldOblique /Univers-CondensedOblique /Univers-Extended /Univers-ExtendedObl /Univers-ExtraBlackExt /Univers-ExtraBlackExtObl /Univers-Light /Univers-LightOblique /UniversLTStd-Black /UniversLTStd-BlackObl /Univers-Oblique /Utopia-Black /Utopia-BlackOsF /Utopia-Bold /Utopia-BoldItalic /Utopia-Italic /Utopia-Ornaments /Utopia-Regular /Utopia-Semibold /Utopia-SemiboldItalic /VAGRounded-Black /VAGRounded-Bold /VAGRounded-Light /VAGRounded-Thin /Viva-BoldExtraExtended /Viva-Regular /Weidemann-Black /Weidemann-BlackItalic /Weidemann-Bold /Weidemann-BoldItalic /Weidemann-Book /Weidemann-BookItalic /Weidemann-Medium /Weidemann-MediumItalic /WindsorBT-Elongated /WindsorBT-Light /WindsorBT-LightCondensed /WindsorBT-Roman /Wingdings-Regular /WNCYB10 /WNCYI10 /WNCYR10 /WNCYSC10 /WNCYSS10 /WoodtypeOrnaments-One /WoodtypeOrnaments-Two /ZapfCalligraphic801BT-Bold /ZapfCalligraphic801BT-BoldItal /ZapfCalligraphic801BT-Italic /ZapfCalligraphic801BT-Roman /ZapfChanceryITCbyBT-Bold /ZapfChanceryITCbyBT-Demi /ZapfChanceryITCbyBT-Medium /ZapfChanceryITCbyBT-MediumItal /ZapfChancery-MediumItalic /ZapfDingbats /ZapfDingbatsITCbyBT-Regular /ZapfElliptical711BT-Bold /ZapfElliptical711BT-BoldItalic /ZapfElliptical711BT-Italic /ZapfElliptical711BT-Roman /ZapfHumanist601BT-Bold /ZapfHumanist601BT-BoldItalic /ZapfHumanist601BT-Demi /ZapfHumanist601BT-DemiItalic /ZapfHumanist601BT-Italic /ZapfHumanist601BT-Roman /ZapfHumanist601BT-Ultra /ZapfHumanist601BT-UltraItalic /ZurichBT-Black /ZurichBT-BlackExtended /ZurichBT-BlackItalic /ZurichBT-Bold /ZurichBT-BoldCondensed /ZurichBT-BoldCondensedItalic /ZurichBT-BoldExtended /ZurichBT-BoldExtraCondensed /ZurichBT-BoldItalic /ZurichBT-ExtraBlack /ZurichBT-ExtraCondensed /ZurichBT-Italic /ZurichBT-ItalicCondensed /ZurichBT-Light /ZurichBT-LightCondensed /ZurichBT-LightCondensedItalic /ZurichBT-LightExtraCondensed /ZurichBT-LightItalic /ZurichBT-Roman /ZurichBT-RomanCondensed /ZurichBT-RomanExtended /ZurichBT-UltraBlackExtended ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages true /ColorImageMinResolution 150 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Bicubic /ColorImageResolution 300 /ColorImageDepth -1 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.50000 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages true /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 150 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox false /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (U.S. Web Coated \050SWOP\051 v2) /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /Unknown /CreateJDFFile false /SyntheticBoldness 1.000000 /Description << /FRA <FEFF004f007000740069006f006e00730020007000650072006d0065007400740061006e007400200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000500044004600200064006f007400e900730020006400270075006e00650020007200e90073006f006c007500740069006f006e002000e9006c0065007600e9006500200070006f0075007200200075006e00650020007100750061006c0069007400e90020006400270069006d007000720065007300730069006f006e00200061006d00e9006c0069006f007200e90065002e00200049006c002000650073007400200070006f0073007300690062006c0065002000640027006f00750076007200690072002000630065007300200064006f00630075006d0065006e007400730020005000440046002000640061006e00730020004100630072006f0062006100740020006500740020005200650061006400650072002c002000760065007200730069006f006e002000200035002e00300020006f007500200075006c007400e9007200690065007500720065002e> /JPN <FEFF3053306e8a2d5b9a306f30019ad889e350cf5ea6753b50cf3092542b308000200050004400460020658766f830924f5c62103059308b3068304d306b4f7f75283057307e30593002537052376642306e753b8cea3092670059279650306b4fdd306430533068304c3067304d307e305930023053306e8a2d5b9a30674f5c62103057305f00200050004400460020658766f8306f0020004100630072006f0062006100740020304a30883073002000520065006100640065007200200035002e003000204ee5964d30678868793a3067304d307e30593002> /DEU <FEFF00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e0020005000440046002d0044006f006b0075006d0065006e00740065006e0020006d00690074002000650069006e006500720020006800f60068006500720065006e002000420069006c0064006100750066006c00f600730075006e0067002c00200075006d002000650069006e0065002000760065007200620065007300730065007200740065002000420069006c0064007100750061006c0069007400e400740020007a0075002000650072007a00690065006c0065006e002e00200044006900650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f0062006100740020006f0064006500720020006d00690074002000640065006d002000520065006100640065007200200035002e003000200075006e00640020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /PTB <FEFF005500740069006c0069007a006500200065007300740061007300200063006f006e00660069006700750072006100e700f5006500730020007000610072006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000500044004600200063006f006d00200075006d00610020007200650073006f006c007500e700e3006f00200064006500200069006d006100670065006d0020007300750070006500720069006f0072002000700061007200610020006f006200740065007200200075006d00610020007100750061006c0069006400610064006500200064006500200069006d0070007200650073007300e3006f0020006d0065006c0068006f0072002e0020004f007300200064006f00630075006d0065006e0074006f0073002000500044004600200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002c002000520065006100640065007200200035002e0030002000650020007300750070006500720069006f0072002e> /DAN <FEFF004200720075006700200064006900730073006500200069006e0064007300740069006c006c0069006e006700650072002000740069006c0020006100740020006f0070007200650074007400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006d006500640020006800f8006a006500720065002000620069006c006c00650064006f0070006c00f80073006e0069006e006700200066006f00720020006100740020006600e50020006200650064007200650020007500640073006b00720069006600740073006b00760061006c0069007400650074002e0020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e006500730020006d006500640020004100630072006f0062006100740020006f0067002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e> /NLD <FEFF004700650062007200750069006b002000640065007a006500200069006e007300740065006c006c0069006e00670065006e0020006f006d0020005000440046002d0064006f00630075006d0065006e00740065006e0020007400650020006d0061006b0065006e0020006d00650074002000650065006e00200068006f0067006500720065002000610066006200650065006c00640069006e00670073007200650073006f006c007500740069006500200076006f006f0072002000650065006e0020006200650074006500720065002000610066006400720075006b006b00770061006c00690074006500690074002e0020004400650020005000440046002d0064006f00630075006d0065006e00740065006e0020006b0075006e006e0065006e00200077006f007200640065006e002000670065006f00700065006e00640020006d006500740020004100630072006f00620061007400200065006e002000520065006100640065007200200035002e003000200065006e00200068006f006700650072002e> /ESP <FEFF0055007300650020006500730074006100730020006f007000630069006f006e006500730020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f0073002000500044004600200063006f006e0020006d00610079006f00720020007200650073006f006c00750063006900f3006e00200064006500200069006d006100670065006e00200070006100720061002000610075006d0065006e0074006100720020006c0061002000630061006c006900640061006400200061006c00200069006d007000720069006d00690072002e0020004c006f007300200064006f00630075006d0065006e0074006f00730020005000440046002000730065002000700075006500640065006e00200061006200720069007200200063006f006e0020004100630072006f00620061007400200079002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /SUO <FEFF004e00e4006900640065006e002000610073006500740075007300740065006e0020006100760075006c006c006100200076006f0069006400610061006e0020006c0075006f006400610020005000440046002d0061007300690061006b00690072006a006f006a0061002c0020006a006f006900640065006e002000740075006c006f0073007400750073006c00610061007400750020006f006e0020006b006f0072006b006500610020006a00610020006b007500760061006e0020007400610072006b006b007500750073002000730075007500720069002e0020005000440046002d0061007300690061006b00690072006a0061007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f006200610074002d0020006a00610020004100630072006f006200610074002000520065006100640065007200200035002e00300020002d006f0068006a0065006c006d0061006c006c0061002000740061006900200075007500640065006d006d0061006c006c0061002000760065007200730069006f006c006c0061002e> /ITA <FEFF00550073006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e00740069002000500044004600200063006f006e00200075006e00610020007200690073006f006c0075007a0069006f006e00650020006d0061006700670069006f00720065002000700065007200200075006e00610020007100750061006c0069007400e00020006400690020007300740061006d007000610020006d00690067006c0069006f00720065002e0020004900200064006f00630075006d0065006e00740069002000500044004600200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /NOR <FEFF004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f00700070007200650074007400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006d006500640020006800f80079006500720065002000620069006c00640065006f00700070006c00f80073006e0069006e006700200066006f00720020006200650064007200650020007500740073006b00720069006600740073006b00760061006c0069007400650074002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e006500730020006d006500640020004100630072006f0062006100740020006f0067002000520065006100640065007200200035002e00300020006f0067002000730065006e006500720065002e> /SVE <FEFF0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006e00e40072002000640075002000760069006c006c00200073006b0061007000610020005000440046002d0064006f006b0075006d0065006e00740020006d006500640020006800f6006700720065002000620069006c0064007500700070006c00f60073006e0069006e00670020006f006300680020006400e40072006d006500640020006600e50020006200e400740074007200650020007500740073006b00720069006600740073006b00760061006c0069007400650074002e0020005000440046002d0064006f006b0075006d0065006e00740065006e0020006b0061006e002000f600700070006e006100730020006d006500640020004100630072006f0062006100740020006f00630068002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006100720065002e> /ENU (Use these settings for creating PDF files for submission to The Sheridan Press. These settings configured for Acrobat v6.0 08/06/03.) >> >> setdistillerparams << /HWResolution [2400 2400] /PageSize [612.000 792.000] >> setpagedevice
