Attachment in Early Childhood 2
Abstract
When institutions administer student evaluations of teaching (SETs) online, response rates are lower
relative to paper-based administration. We analyzed average SET scores from 364 courses taught during
the fall term in 3 consecutive years to determine whether administering SET forms online for all courses
in the 3rd year changed the response rate or the average SET score. To control for instructor
characteristics, we based the data analysis on courses for which the same instructor taught the course in
each of three successive fall terms. Response rates for face-to-face classes declined when SET
administration occurred only online. Although average SET scores were reliably lower in Year 3 than in
the previous 2 years, the magnitude of this change was minimal (0.11 on a five-item Likert-like scale).
We discuss practical implications of these findings for interpretation of SETs and the role of SETs in the
evaluation of teaching quality.
Keywords: college teaching, student evaluations of teaching, online administration, response
rate, assessment
Attachment in Early Childhood 3
Introduction
Student ratings and evaluations of instruction have a long history as sources of information
about teaching quality (Berk, 2013). Student evaluations of teaching (SETs) often play a significant role in
high-stakes decisions about hiring, promotion, tenure, and teaching awards. As a result, researchers
have examined the psychometric properties of SETs and the possible impact of variables such as race,
gender, age, course difficulty, and grading practices on average student ratings (Griffin et al., 2014;
Nulty, 2008; Spooren et al., 2013). They have also examined how decision makers evaluate SET scores
(Boysen, 2015a, 2015b; Boysen et al., 2014; Dewar, 2011). In the last 20 years, considerable attention
has been directed toward the consequences of administering SETs online (Morrison, 2011; Stowell et al.,
2012) because low response rates may have implications for how decision makers should interpret SETs.
Body
Administering SETs online creates multiple benefits. Online administration enables instructors to
devote more class time to instruction (vs. administering paper-based forms) and can improve the
integrity of the process. Students who are not pressed for time in class are more likely to reflect on their
answers and write more detailed comments (Morrison, 2011; Stowell et al., 2012; Venette et al., 2010).
Because electronic aggregation of responses bypasses the time-consuming task of transcribing
comments (sometimes written in challenging handwriting), instructors can receive summary data and
verbatim comments shortly after the close of the term instead of weeks or months into the following
term.
Despite the many benefits of online administration, instructors and students have expressed
concerns about online administration of SETs. Students have expressed concern that their responses are
not confidential when they must use their student identification number to log into the system
(Dommeyer et al., 2002). However, breaches of confidentiality can occur even with paper-based
administration. For example, an instructor might recognize student handwriting (one reason some
Attachment in Early Childhood 4
students do not write comments on paper-based forms), or an instructor might remain present during
SET administration (Avery et al., 2006).
In-class, paper-based administration creates social expectations that might motivate students to
complete SETs. In contrast, students who are concerned about confidentiality or do not understand how
instructors and institutions use SET findings to improve teaching might ignore requests to complete an
online SET (Dommeyer et al., 2002). Instructors in turn worry that low response rates will reduce the
validity of the findings if students who do not complete an SET differ in significant ways from students
who do (Stowell et al., 2012). For example, students who do not attend class regularly often miss class
the day that SETs are administered. However, all students (including nonattending students) can
complete the forms when they are administered online. Faculty also fear that SET findings based on a
low-response sample will be dominated by students in extreme categories (e.g., students with grudges,
students with extremely favorable attitudes), who may be particularly motivated to complete online
SETs, and therefore that SET findings will inadequately represent the voice of average students (Reiner
& Arnold, 2010).
Conclusion
Psychology has a long history of devising creative strategies to measure the “unmeasurable,”
whether the targeted variable is a mental process, an attitude, or the quality of teaching (e.g., Webb et
al., 1966). In addition, psychologists have documented various heuristics and biases that contribute to
the misinterpretation of quantitative data (Gilovich et al., 2002), including SET scores (Boysen, 2015a,
2015b; Boysen et al., 2014). These skills enable psychologists to offer multiple solutions to the challenge
posed by the need to objectively evaluate the quality of teaching and the impact of teaching on student
learning.
Online administration of SET forms presents multiple desirable features, including rapid
feedback to instructors, economy, and support for environmental sustainability. However, institutions
Attachment in Early Childhood 5
should adopt implementation procedures that do not undermine the usefulness of the data gathered.
Moreover, institutions should be wary of emphasizing procedures that produce high response rates only
to lull faculty into believing that SET data can be the primary (or only) metric used for high-stakes
decisions about the quality of faculty teaching. Instead, decision makers should expect to use multiple
measures to evaluate the quality of faculty teaching.
Attachment in Early Childhood 6
References
Astuti, U., Hartono, H., & Sunawan, S. (2020). The influence of parental attachment toward early
childhood children’s separation anxiety. Journal of Primary Education, 9(5), 501–510.
https://doi.org/10.15294/jpe.v9i5.43210
Lewis-Morrarty, E., Degnan, K. A., Chronis-Tuscano, A., Henderson, H. A., Pine, D. S., & Fox, N. A. (2015).
Infant Attachment Security and Early Childhood Behavioral Inhibition Interact to Predict
Adolescent Social Anxiety Symptoms. Child Development, 86(2), 598–613.
http://www.jstor.org/stable/24696608
Morley, T. E., & Moran, G. (2011). The origins of cognitive vulnerability in early childhood: Mechanisms
linking early attachment to later depression. Clinical Psychology Review, 31(7), 1071–1082.
https://doi.org/10.1016/j.cpr.2011.06.006
Murphy, T. P., McCurdy, K., Jehl, B., Rowan, M., & Larrimore, K. (2020). Jealousy behaviors in early
childhood: Associations with attachment and temperament. International Journal of Behavioral
Development, 44(3), 266-272. https://doi.org/10.1177/0165025419877974
O'Connor, T.G., Woolgar, M., Humayun, S., Briskman, J.A. and Scott, S. (2019), Early caregiving predicts
attachment representations in adolescence: findings from two longitudinal studies. J Child
Psychol Psychiatr, 60: 944-952. https://doi.org/10.1111/jcpp.12936
Powered by TCPDF (www.tcpdf.org)
