Discussion

Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=hbas20

Basic and Applied Social Psychology

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/hbas20

“I’ll Be There”: Improving Online Class Attendance with a Commitment Nudge during COVID-19

Robert J. Weijers, Lesya Ganushchak, Kim Ouwehand & Björn B. de Koning

To cite this article: Robert J. Weijers, Lesya Ganushchak, Kim Ouwehand & Björn B. de Koning (2022) “I’ll Be There”: Improving Online Class Attendance with a Commitment Nudge during COVID-19, Basic and Applied Social Psychology, 44:1, 12-24, DOI: 10.1080/01973533.2021.2023534

To link to this article: https://doi.org/10.1080/01973533.2021.2023534

© 2022 The Author(s). Published with license by Taylor & Francis Group, LLC.

View supplementary material

Published online: 25 Jan 2022. Submit your article to this journal

Article views: 5968 View related articles

View Crossmark data Citing articles: 2 View citing articles

“I’ll Be There”: Improving Online Class Attendance with a Commitment Nudge during COVID-19

Robert J. Weijers, Lesya Ganushchak, Kim Ouwehand, and Bj€orn B. de Koning

Erasmus Universiteit Rotterdam

ABSTRACT Class attendance is an important predictor of academic success, but students encounter behavioral barriers preventing them from attending. In this experimental study, we investi- gated a commitment intervention to improve online attendance among university students (n¼ 973) during the COVID-19 pandemic. In the experimental condition, we asked students to commit to attending all classes and divided this group into students who made the com- mitment and those who did not commit. The data was analyzed from a psychological per- spective (the effect on the individuals who responded positively to the commitment request) and a policy perspective (the effect for all individuals that received the request). No intervention effect was found when comparing students’ attendance in the experimental condition to the control condition, but students who made the commitment attended class more often than non-committing students and those in the control condition. Exploratory analyses revealed that the intervention effect was found in the course with lower attend- ance, indicating that a ceiling effect possibly prevented the intervention from showing results regarding attendance. However, exploratory analyses also revealed selection bias as a possible explanation for the effects. Additionally, the intervention backfired for non-commit- ting students, reducing their attendance. Future research should focus on different strat- egies to improve online attendance.

One of the most important behaviors for students to be successful in higher education is class attendance (Bijsmans & Schakel, 2018). Class attendance in higher education is positively linked to educational outcomes such as course engagement (B€uchele, 2021), improved course grades, and higher completion rates (Sund & Bignoux, 2018). The importance of class attendance for educational attainment is also high- lighted by research showing that even when controlled for student characteristics such as motivation and study habits, class attendance significantly contributes to academic achievement (Cred�e et al., 2010; Golding, 2011). However, still many students in higher educa- tion do not always attend their classes. In fact, teach- ers in higher education, as well as in vocational education, describe students skipping classes as one of the greatest issues they are facing (Weijers et al., n.d), as a low attendance rate results in less engaging meet- ings, discussions and worse attuned class preparation. One reason that students do not attend class is that

they encounter behavioral barriers throughout educa- tion that prevent them from showing the desired behavior (Ruggeri, 2019). Previous research has shown that while students have good intentions to attend class and are capable of doing so, they often fall vic- tim to barriers like lack of willpower, postponing, or even simply forgetting to attend (Weijers et al., 2021). Giving in to these barriers leads to undesired behavior (i.e., not attending class), which endangers students’ possibilities to successfully complete their courses and more generally their whole educational program. It is therefore important to investigate possibilities to increase the chance that students overcome these bar- riers and attend class.

Online attendance

In this article, we focus on improving online class attendance. Online courses have been on the rise for the past decade and recently have gained momentum

CONTACT Robert J. Weijers [email protected] Department of Psychology, Education, and Child Studies, Erasmus Universiteit Rotterdam, Burgemeester Oudlaan 50, Rotterdam 3062 PA, The Netherlands.

Supplemental data for this article can be accessed at publisher’s website.

� 2022 The Author(s). Published with license by Taylor & Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by- nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.

BASIC AND APPLIED SOCIAL PSYCHOLOGY 2022, VOL. 44, NO. 1, 12–24 https://doi.org/10.1080/01973533.2021.2023534

due to the physical distancing measures as a result of the COVID-19 pandemic. To comply with govern- ment advice and regulations, worldwide universities have switched almost completely to an online course program. For example, in the Netherlands universities were instructed to switch to an online course program where possible (Rijksoverheid, 2020), and more than 90% of classes subsequently moved online (Association of Universities the Netherlands, 2020). Although class attendance is equally important in physical and online education (Nieuwoudt, 2020; Rapposelli, 2014), there are specific challenges related to attending online classes. For example, for an online class, it is easier for students to pretend to be present or to not login into the online class altogether, and it is more difficult to enforce class attendance compared to physical classes (Archambault et al., 2013). During the COVID-19 pandemic, online class attendance has become even more challenging and teachers have reported a drop in class attendance (Meeter et al., 2020; The New York Times, 2020). As an example, at several universities, the attendance requirement that was practiced during physical lessons was loosened when teaching moved to online lessons. This was done to be accommodating toward students during the unusual circumstances of the COVID- 19 pandemic.

With online education, especially when not volun- tarily chosen but forced by the circumstances, there is more room for students to fall victim to the afore- mentioned behavioral barriers, preventing them from attending. One possible reason for this is that such an online environment is less motivating or stimulating. According to the self-determination theory (SDT: Deci & Ryan, 2012; Ryan & Deci, 2000) perceived autonomy, together with perceived relatedness and competency are central human needs that need to be fulfilled for an individual to be proactive and engaged. Autonomy means that a person feels in control of their own goals and actions, relatedness indicates that the person feels a sense of belonging to others, and competency is achieved when a person feels their skills match up to a task (Deci & Ryan, 2012). Note that in the current operationalization, the online edu- cational context does not meet these basic needs. The involuntary aspect of the online education (i.e., stu- dents did not choose distance education, but the regu- lations forced the students to stay home) jeopardizes autonomy, as the majority of students would prefer physical classrooms, but the COVID-19 measures pre- vent them from choosing this option (Chakraborty et al., 2021). Also, the need for relatedness is

jeopardized, as many students miss social connections (Meeter et al., 2020) and report feeling lonely because of the COVID-19 measures (Wickens et al., 2021): not going to campus means no bumping in acquaintances, meeting other students in the lecture hall, studying and commuting together, etc. The need for compe- tency is also at risk because students are both figura- tively and literally less visible (Onyema et al., 2020). This provides less behavior for a teacher to comment and compliment on, which may lead to decreased feel- ings of competence. Furthermore, negative motiv- ational outcomes due to a lack of fulfillment of one need can cascade into another. For example, not hav- ing the autonomy to choose to have lunch at campus can also lead to a decrease in relatedness. As a result, students are less intrinsically motivated online (Meeter et al., 2020) and might fall into a vicious cycle of pas- sivity, which makes it easier to fall victim to behav- ioral barriers like lack of willpower, postponing, or even simply forgetting to attend.

Not only does the forced online environment hin- der students, but they also miss the usual support net- works like social networks, student associations, and campus provisions. Additionally, it is more difficult for teachers to reach out and help students in an online environment. This especially hurts students who are already struggling with their education (Figlio et al., 2013). Research suggests, however, that it is possible for teachers to support students in this situ- ation. Even in an online setting without face-to-face contact, increasing feelings of autonomy can improve motivation and learning. For example, in an online setting, Schneider et al. (2018) found that students who could choose (autonomy-supportive condition) between two texts they had to study, led to better retention and transfer of learning compared with stu- dents who could not choose (control condition). This suggests that an autonomy-enhancing educational intervention can help improve students’ intrinsic motivation, making them less prone to behavioral bar- riers. To this end, we look toward a commitment strategy as a possible autonomy-enhancing (Weijers et al., n.d) means to help students overcome behav- ioral barriers. In the present study, we use a commit- ment intervention as a possible autonomy-supportive way to increase class attendance rates of students in higher education.

Commitment

Cialdini (1993) describes the effect of commitment on behavior as follows: “Once we make a choice or take a

BASIC AND APPLIED SOCIAL PSYCHOLOGY 13

stand, we will encounter personal and interpersonal pressures to behave consistently with that commitment” (p. 52). This pressure is rooted in a desire to be consistent with oneself. When we want to change behavior, we can then make use of this desire to be consistent in different ways (Cialdini, 1993). The following examples of commitment-based interven- tions illustrate the possibilities when appealing to this desire for consistency. As one example, it is possible to ask people to set goals for themselves, so that they will want to pursue these goals later (Michie et al., 2014). A different possibility is the “foot in the door”- technique (Freedman & Fraser, 1966) where a person is first asked a small favor, later to be followed up with a larger favor—which they are more likely to perform to be consistent with their earlier agreement. Another way to elicit consistency with oneself is mak- ing a behavioral contract: a “written specification of the behavior to be performed, signed by the person and witnessed by another” (Michie et al., 2014, p. 261). Commitment-based interventions, in different forms, have been utilized to successfully facilitate behavioral change, for example in healthcare (e.g., Srivastava, 2012), physical exercise (e.g., Bhattacharya et al., 2015), and environmentally friendly behavior (e.g., Baca-Motes et al., 2013), and a meta-analysis by Lokhorst et al. (2013) revealed that commitment inter- ventions generally have a moderate effect size (r¼ 0.27). A commitment intervention ties in with self-determination theory: targets are given a choice instead of external regulation, improving perceived autonomy (Ryan & Deci, 2000); the asked behavior is an affirmation of the belief that the target has the cap- acity to execute the behavior, increasing perceived competence; and the request for commitment is done to a person with whom the target has a connection (e.g., their teacher), increasing perceived relatedness.

An important aspect to consider when discussing commitment interventions is whether participants actually commit to the behavior. Commitment inter- ventions require a participant’s active commitment to have an effect, but usually, not all participants will make the commitment. Because of this, Lokhorst et al. (2013) distinguish two measurable effects of a com- mitment. First, the commitment intervention has an effect we refer to as “psychological effect,” which is the effect on only the participants that positively responded to the ask for commitment (i.e., actually made a commitment). However, not everyone responds to this message. For the effect of this inter- vention when implemented as a class or university policy, we also have to investigate the participants

who were asked to commit but did not respond or even refused to commit. We refer to the effect of the intervention on all participants, regardless of whether the participant made the commitment or not, the “policy effect,” as it represents the effect of the inter- vention on a population if the intervention is imple- mented as a policy. Both the psychological effect and the policy effect of the intervention should be investi- gated to get a complete picture of the effectiveness of a commitment intervention (Lokhorst et al., 2013). Most prior studies only focus on the psychological effect of the intervention, as the commitment is only expected to work when one actually makes the com- mitment (Cialdini, 1993). However, it is also relevant to get insight into the policy effect because it provides information about whether implementing the inter- vention would be effective in practice.

Commitment interventions in education

Different interventions that make use of commitment to facilitating behavioral change have also been suc- cessfully used in educational contexts (Damgaard & Nielsen, 2018). For example, in the study by Duckworth et al. (2013) middle school children who made a commitment during a class exercise to achieve individual educational goals slightly improved their class attendance, conduct, and grades. Oettingen et al. (2015) used commitment strategies to improve time management in both university students and voca- tional students. Committing to doing practice exams increased exam practice completion in high school students (Duckworth et al., 2011). Similarly, in a study by Clark et al. (2020) college students who committed to completing a self-set goal of practice exams com- pleted significantly more practice exams. As a targeted intervention in a live educational context, the behav- ioral contract technique has been documented as suc- cessful in helping children diagnosed with ADHD to achieve specific behavioral goals (DuPaul & Stoner, 2014). In a one-on-one conversation with their teacher, these students created a behavioral contract where they outlined specific desired behaviors and promised to fulfill the contract. Teachers have also already been observed to use various commitment strategies, such as letting students set their own dead- lines (Weijers et al., n.d). These examples demonstrate the potential effectiveness of commitment interven- tions in education.

In the present study, we build on these earlier find- ings and extend them in the following ways: (1) we extend the findings on improved class attendance by

14 R. J. WEIJERS ET AL.

Duckworth et al. (2013) to students in higher educa- tion, (2) test the behavioral contract as a class-wide intervention, delivered via email, instead of the usual face-to-face commitment approach as an individual tool (as in DuPaul & Stoner, 2014), and (3) extend the findings by Duckworth et al. (2013) to an online educational setting, which is more relevant than ever during the COVID-19 pandemic. The goal of this study is to find the effectiveness of a class-wide imple- mented behavioral contract intervention to improve online class attendance in higher education.

Current study

In this study, we attempt to answer the following research question: can a commitment intervention in the form of a behavioral contract improve students’ class attendance rate in an online course? We attempt to answer this question by an experiment in a univer- sity environment, where we asked first- and second- year students per email to commit to attending their online classes. All students in the courses were ran- domly divided into a control condition (not asked for commitment) and an experimental condition (asked for commitment). For the experimental condition, we noted which students made the commitment and which students did not. Consistent with the advice of Lokhorst et al. (2013), this enabled us to distinguish between the policy effect (the effect of the interven- tion on all involved students) and psychological effect (the effect of the intervention on the students who made the commitment) of the intervention.

The first hypothesis concerns the policy effect of the intervention, meaning the effect of the intervention on all participating students. This allows us to determine whether this intervention is useful as a policy tool. Based on the earlier success of commitment interven- tions as a policy tool (for an overview, see Lokhorst et al., 2013), we formulated the following hypothesis:

Hypothesis 1: Students in the experimental condition attend class more often than students in the control condition.

The other two hypotheses address the psychological effect of the commitment intervention. We expect the policy effect of the intervention to be explained by par- ticipants who actually make a commitment (Cialdini, 1993; Lokhorst et al., 2013). Therefore, we compare the attendance of the students who chose to commit with those who did not commit, and with those in the con- trol condition, resulting in the following hypotheses:

Hypothesis 2: Within the experimental condition, students who make a commitment attend class more

often than those who are asked but do not make a commitment.

Hypothesis 3: Students who make a commitment attend class more often than those who are not asked to make a commitment (control condition).

Method

Participants

In this field experiment, 1,046 bachelor students from a large urban university in the Netherlands participated during their regular educational program. Although we did not collect specific demographics, this population of students at the same university was characterized in other studies as mostly female (approximately 75%) adolescents of on average 20 years old (Brouwer et al., 2019; Kickert et al., 2019). Participants followed a first- year Pedagogy course (1.8PED; n¼ 136), a second-year Pedagogy course (2.8PED; n¼ 123), or a first-year Psychology course (1.8PSY; n¼ 787). From the 1,046 students enrolled in the course, 73 students were excluded for data analysis. Students were excluded because they dropped out of university during the experiment (3 students), attendance registration data was lost (31 students), or did not finish the course while not leaving university (39 students), resulting in a remaining 973 students. These courses were held concurrently during the final five weeks of the aca- demic year (June/July 2020) and consisted of small online group meetings (about 10–12 students per group). No students were enrolled in more than one course simultaneously. All three courses had the same pedagogical approach (i.e., problem-based learning), and had two meetings per week. In problem-based learning, the success of the meetings largely depends on student attendance, especially with the small group sizes. Due to the scheduling of the final exam, 1.8PED contained 8 meetings, while 1.8PSY and 2.8PED con- tained 9 meetings. Given that the COVID-19 measures issued by the Dutch government forbade physical on- campus teaching at the time this study took place, the three courses took place online and the 100% class attendance requirement was replaced with a more leni- ent “attendance effort,” meaning that attendance was no longer mandatory but students were expected to attend as many classes as they could.

Design

This study had a between-subjects experimental design with two conditions: an experimental condition and a control condition. In the experimental condition,

BASIC AND APPLIED SOCIAL PSYCHOLOGY 15

students received an email from the course coordin- ator welcoming the students to the course and asking for their commitment to attend all online meetings during the course. The decision to create behavioral contracts per email instead of the more conventional face-to-face approach was made due to both the group size and the COVID-19 restrictions in place. Previous research by Baca-Motes et al. (2013) suggests that the commitment should be as specific as possible. Therefore, the email also specified what days the meetings took place. Students were asked to confirm their commitment before the first meeting of the course by replying via email to the course coordinator. A standardized text was used for this to keep the bar- rier to responding to a minimum; students only needed to fill in their name and choose from “will/ will not” as appropriate. No confirmation of the stu- dent’s reply was sent out. The control condition received an email with the same welcoming text as in the experimental condition but without the inquiry for commitment to the online meetings during the course. The full text of the experimental and control emails can be found in Appendix A.

Approval for this study was obtained from the Ethical Review Committee of the Department of Psychology, Education, and Child Studies; Erasmus University Rotterdam (application 20-052).

Measures

Class attendance For each student (control and experimental condition) attendance to the online meetings was registered: tutors registered per meeting whether a student was present or absent, which resulted in an overall score for each student representing the total number of attended meetings. Students were considered absent only when they were absent during the entire online meeting. Because the three courses did not have an equal number of meetings (two courses had 8 meet- ings, one had 9 meetings), we converted students’ raw attendance scores into percentages, representing how much percent of meetings a student had attended.

Commitment For each student in the experimental condition, tutors registered whether students had replied to the request for commitment, and when a reply was received whether or not students had committed to attend all online meetings. Students who replied that they would attend all meetings were categorized as “committed.” Students who, for any reason, declared not to attend all

meetings or wished not to make a commitment, were categorized as “refused.” Due to their diverse reasons to not commit and small group size (n¼ 16), the refused- group was not included in the analysis of the second and third hypotheses. Students who did not reply were categorized as “non-response” (<1% of total students). As they did not engage in the behavioral contract, we consider these students as not committed. To clarify: “committed” means “committed through the inter- vention,” regardless of how a student may feel toward the course. Replies made after the first-course meeting were categorized as “non-response.”

Of the students in the final sample, 488 were in the control condition and 485 were in the experimental condition. From all students in the experimental con- dition, 300 were in the “committed” group, 16 were in the “refused” group, and 169 were in the “non- response” group. In the final dataset, 746 students participated in the first year Psychology course (1.8PSY), 111 students participated in the first year Pedagogy course (1.8PED), and 116 students partici- pated in the second year Pedagogy course (2.8PED).

To account for leveled data, for each student we retrieved information from the course coordinator about the name of the course they followed, in which tutor group the student participated, and the name of the student’s tutor.

Procedure

Student emails were obtained from the enroll lists of the courses and divided between conditions using ran- dom number generation. A week before the first ses- sion of the course, students in both conditions received an email (matching with their condition) from their course coordinator. Students in the experi- mental condition received a reminder to reply the day before the first meeting of the course.

When students inquired about the ask for commit- ment, the course coordinator responded based on a cover story about creating an indication of actual attendance for the preparation of the tutors who guided the online meetings. After data analysis, stu- dents were debriefed about their participation in an experiment about increased attendance and given the opportunity to remove their data. No students wished to have their data removed.

Analysis

The hypotheses were tested using a mixed-effects model approach in the statistical program R (R Core

16 R. J. WEIJERS ET AL.

Team, 2020), using the lme4 package (Bates et al., 2015). Three models were created to test each of the three hypotheses. The first model was created to test the first hypothesis. In this model, class attendance in the experimental group was compared to the control group. Within the experimental group, we used the second model to compare the committed group to the group of non-responders, testing the second hypoth- esis. A third model was created to investigate the third hypothesis concerning the effect of participating in the intervention, by comparing the control condition to the committed group. The data and the used script can be found in the supplementary materials.

The model used for Hypothesis 1 had the experi- mental condition (control, experiment) as the inde- pendent variable and attendance as a percentage as the dependent variable, with course added as a fixed effect. The models used for Hypotheses 2 and 3 had response conditions (commit or nonresponse for the model for Hypothesis 2; commit or control for the model for Hypothesis 3) as the independent variable and attendance as the dependent variable, with course added as a fixed effect. For all analyses, tutor and tutor groups were added as random intercepts to account for differences in attendance caused by these factors. To optimize the random structure, the sugges- tions of Barr et al. (2013) were used as a guideline. Estimates provided are indicative of effect size. To create an approximation of Cohen’s d for effect sizes, we used the method described by Westfall et al. (2014).

Results

In Table 1, descriptive statistics for attendance are presented for each condition and course.

To test the first hypothesis (the “policy effect”), the attendance of the students in the experimental condi- tion was compared to the attendance of the students in the control condition. No difference was found between these two groups: Estimate ¼ 0.32, SE ¼ 0.49, d¼ 0.15.

The second model was intended to find a differ- ence in attendance between students who responded and students who did not. The students who commit- ted attended a higher percentage of meetings (M¼ 92.69, SD¼ 10.43) compared to students who did not respond to the email asking for commitment (M¼ 84.39, SD¼ 18.63): Estimate ¼ 4.24, SE¼ 0.67, d¼ 1.80.

Lastly, a third model was constructed to find the difference between the attendance of the control

group versus the students who committed to attend all online meetings. Students who committed to attend all online meetings attended a higher percentage of meetings (M¼ 92.69, SD¼ 10.43) than those in the control group (M¼ 90.08, SD¼ 16.25), Estimate ¼ 1.29, SE¼ 0.51, d¼ 0.84. This indicates that there was a positive effect of the intervention.

Explorative analysis

Based on the descriptive statistics (Table 1), we inves- tigated whether the courses significantly differed in attendance. We created a mixed effect model as described in the Analysis section, using the course as a predictor for attendance. The average attendance of 1.8PED was lower than attendance in 1.8PSY (Estimate ¼ 13.09, d¼ 5.96) and in 2.8PED (Estimate ¼ 12.83, d¼ 5.84). Class attendance in 1.8PSY and 2.8PED did not differ (Estimate ¼ 0.25, d¼ 0.11).

To account for the lower attendance of 1.8PED, we added the interaction between course and condition as a fixed effect to the models. We found this interaction for the groups 1.8PED and 1.8PSY for the first (Estimate ¼ 2.55, d¼ 1.18) and third hypothesis (Estimate ¼ 1.52, d¼ 2.99). This interaction was not found for the second hypothesis. No interactions were found when comparing 1.8PED to 2.8PED. In Figures 1 and 2, the interaction is displayed in a bar graph.

An effect of the intervention appears to be only present in the 1.8PED group. For students enrolled for 1.8PED, we found a difference between the control group and the committed group, as well as between the experimental and control conditions, and such a difference did not exist for the other groups. A pos- sible explanation for this difference is a ceiling effect: the higher baseline attendance of the other courses created a ceiling that prevented the intervention from having an effect on attendance. This suggests that the effectiveness of the intervention is dependent on the baseline attendance rate in a course.

Table 1. Descriptive statistics for percentage of course attendance. (n¼ 973) N M (SD)

Condition Control 488 90.08 (16.25) Experiment 485 89.35 (15.20) Committed 300 92.69 (10.43) Non-response 169 84.39 (18.63) Refused 16 79.17 (28.05)

Course 1.8PSY 746 91.24 (13.43) 1.8PED 111 78.15 (25.25) 2.8PED 115 91.11 (12.93)

Total 973 89.72 (15.73)

BASIC AND APPLIED SOCIAL PSYCHOLOGY 17

When testing Hypotheses 2 and 3, we found a dif- ference in attendance between committing students and non-responding students, and students in the control condition, and even found a policy effect (experimental vs. control) in course 1.8PED, where

attendance was lower and as such, the effect of the intervention was not hindered by a ceiling. However, these effects of the intervention could also be attrib- uted to self-selection: it is possible that other, preexist- ing, factors made students who committed more likely

Figure 1. Mean scores and standard errors of attendance per course for Hypothesis 1 (control group compared to experimental group).

Figure 2. Mean scores and standard errors of attendance per course for Hypothesis 3 (control group compared to commit group).

18 R. J. WEIJERS ET AL.

to both attend classes and respond to the intervention.

To further unpack this possibility, we conducted exploratory follow-up analyses of an earlier course. Because class attendance was registered in a general way (pass/fail) in previous courses in the Pedagogy curriculum, the only available data on class attendance per session was from Psychology course 1.7PSY, which took place 5 weeks before course 1.8PSY. Course 1.7PSY was similar in structure and length to the courses in our study (1.8PSY, 1.8PED, 2.8PED), and had been followed by nearly all students who also enrolled for 1.8PSY. It is important to note that no intervention was applied in 1.7PSY. We sampled the students who attended both 1.7PSY and 1.8PSY, and applied the same exclusion criteria as described before, resulting in a dataset of 742 students. Students were divided into groups based on their future responses to the intervention. In Table 2, descriptive statistics are presented for each course and condition of this subgroup. Although attendance data of a previ- ous course was not available for all students, we con- sider this large subsample suitable to investigate the possibility of a self-selection effect. If the same pat- terns in the hypotheses emerge in the course before the intervention was applied (course 1.7PSY), this might indicate that the observed class attendance dif- ferences might not be directly related to our commit- ment intervention but rather to self-selection.

Testing the three main hypotheses with the class attendance data of the earlier course 1.7PSY showed similar patterns for attendance as for the main courses in the study (1.8PSY, 1.8PED, 2.8PED). Students who would later commit to the intervention in 1.8PSY attended 1.7PSY more often (Estimate ¼ 1.78, SE ¼ 0.58, d¼ 0.81) than those who would not respond in 1.8PSY. Students’ attendance for the later control group and later experimental group was equal (Estimate ¼ �0.25, SE ¼ 0.41, d¼ 0.23), confirming successful randomization for our experiment. Lastly, students who would later commit to attending attended classes more often (Estimate ¼ 0.95, SE ¼ 0.47, d¼ 1.25) than those in the control condition.

These findings point toward preexisting factors pre- dicting both attendance and committing to the inter- vention, rather than the intervention affecting class attendance.

Given the high similarity in attendance rates, con- tent, and structure of courses 1.7PSY and 1.8PSY, we could also investigate whether there was an interaction effect of the experimental group between courses. This within-subject design answered the question of whether the intervention affected a student’s attend- ance in 1.8PSY compared to that same student’s attendance in 1.7PSY, and whether this effect differed between experimental groups. This mixed-effects model had the experimental condition (control, com- mit, and nonresponse) and course (1.8PSY and 1.7PSY) as independent variables, and attendance (as a percentage) as the dependent variable. Due to the small group size (n¼ 8), students who refused to commit were removed from this analysis. Tutor, tutorial group, and student ID were added as random intercepts. Interestingly, we found an interaction between course and condition (Estimate: �0.73, SE ¼ 0.31, d¼ 1.26): attendance for students who did not respond to the request for commitment (i.e., non- response) was lower in 1.8PSY (where they were asked to commit) than in 1.7PSY (where they were not asked to commit). This indicates that their attendance dropped after being nudged when compared to their previous attendance in 1.7PSY. No effect was found for any of the other groups: these groups attended equally often in 1.7PSY and 1.8PSY.

Discussion

Class attendance is an important predictor of aca- demic success. In this study, we investigated the effect of a course-wide commitment intervention on online attendance in a large sample of college students. Per email, we asked students to commit to attending all their online classes and investigated whether this increased their attendance. The results showed that, in contrast to Hypothesis 1, there was no general effect of the commitment intervention on class attendance.

Table 2. Descriptive statistics for percentage of course attendance for students attending both 1.7PSY and 1.8PSY. (n¼ 742) N 1.7PSY attendance, M (SD) 1.8PSY attendance, M (SD)

Condition Control 373 92.33 (11.53) 91.92 (12.79) Experiment 369 92.85 (10.83) 90.65 (14.00)

Commit 223 94.23 (10.33) 93.55 (10.06) Non-response 138 90.67 (11.10) 85.87 (17.92) Refused 8 92.19 (9.30) 92.19 (9.30)

Total 742 92.59 (11.18) 91.29 (13.41)

Students are categorized by the nudge they received for 1.8PSY—no nudging took place for 1.7PSY.

BASIC AND APPLIED SOCIAL PSYCHOLOGY 19

In other words, no “policy effect” was found: there was no significant difference in attendance between the experimental group and the control group.

Regarding the psychological effects of the commit- ment intervention, we found a positive effect of mak- ing a commitment on online class attendance: students who committed attended class more often than those in the control condition (Hypothesis 2) and those who did not respond (Hypothesis 3). In the one course (1.8PED), which had a lower baseline of class attendance (<80%) compared to the other two courses (1.8PSY and 2.8PED) (>90%) we also found evidence of the policy effect: students in the experi- mental condition attended class more often than those in the control condition. It is possible that a class attendance rate of >90% leaves too little room for improvement and therefore asking students to commit to attending classes might not have further improved class attendance. This might suggest that the effective- ness of a commitment intervention for improving class attendance is influenced by the initial attendance rate in a course. It is important to note that during the course (1.8PED) in which the effect of the inter- vention was significant, tutors applied the attendance policy in a more lenient way than tutors in other courses did. Therefore, an alternative explanation for this effect might be the perception of autonomy; when students think they have a free choice in attending, a commitment intervention might be more effective in comparison to students who feel constrained by exter- nal rules. This idea is congruent with the concept of behavioral interventions in education functioning as a stepping stone in the transfer between external regula- tion and autonomous decision-making (Weijers et al., n.d). From this perspective, it can be suggested that it is more likely that higher attendance is reached by external rules than by behavioral interventions, but when strict external rules do not apply, a commitment intervention might be a promising manner to increase class attendance.

However, these interpretations need to be consid- ered with caution, because the exploratory analyses in this study also provided an indication that rather than an effect of the intervention, the findings might be the result of a self-selection effect: the idea that students who attend class more often are also more likely to make a commitment, resulting in a selection bias. This finding in itself is interesting, as it not only pro- vides information about the effect of the intervention itself but also highlights the importance of taking other factors into account that might relate to the desired behavior (class attendance). In this light, a

methodological contribution of our study is that investigating the effect of a (commitment) interven- tion might produce different results when its effect is considered at only one point in time, compared to when multiple measurements over time are consid- ered. This might lead to different conclusions and in some cases, as in our study, could lead to overesti- mation of the actual effect that an intervention has on behavior.

Following these exploratory findings, an explan- ation for the presumed lack of psychological effect is that the behavioral contract is not suitable for wide implementation. To be effective, a behavioral interven- tion should be designed based on the determinants of the behavior it intends to change (Hansen, 2018). This would also explain the earlier success in the study by DuPaul and Stoner (2014), who targeted children with ADHD, a specific subgroup with likely similar determinants for their behavior. In our course- wide sample, the determinants for attendance behavior can differ greatly between students, especially during the COVID-19 pandemic. These different determi- nants cannot realistically be targeted with one class- wide intervention. Future research could focus on determining the relevant behavioral barriers and design a behavioral intervention based on circumvent- ing these barriers, specifically targeting the students that encounter those barriers.

Another finding from our exploratory analysis is that attendance rates were similar for both the course with the commitment intervention and the course without the commitment intervention. This was true for all groups except for the students who had the opportunity to commit but did not do so. The attend- ance of those students dropped after the intervention, while attendance for all other groups remained unchanged. From this, we hypothesize a backfire effect caused by the intervention for one of the groups (i.e., non-responders), as they were predicted to attend more often after the intervention, but in practice attended less often. The backfiring of behavioral inter- ventions are not uncommon (Osman et al., 2020), especially for interventions that threaten the perceived autonomy of those targeted by the intervention. In an attempt to reassert one’s autonomy, targets then dis- play reactance (Knowles & Linn, 2004): they respond contrary to the direction suggested by the intervention (Osman et al., 2020). This can be illustrated with nudging interventions (Thaler & Sunstein, 2008) as an example. Nudges are behavioral interventions that utilize psychological mechanisms to influence people’s behavior (Thaler & Sunstein, 2008). However, they

20 R. J. WEIJERS ET AL.

were found to sometimes produce reactance (e.g., Dewies et al., 2021; Sunstein, 2017). Our intervention, utilizing the commitment mechanism, may be consid- ered such a nudge (see Damgaard & Nielsen, 2018; Weijers et al., 2021, for integration of nudges in edu- cation, including commitment interventions). For this intervention, the contrary reaction would be to not respond to the commitment and not attend all online classes, explaining the drop in attendance after the intervention for the nonresponse group. A different explanation for this backfiring is that not responding might have acted as a self-fulfilling prophecy. Non-responders could have had the idea that they influenced the expectancy of the teachers/course coor- dinators that they would not show up and therefore, they did not show up. A last possible explanation is that students knew they would not be able to attend all classes, due to already planned obligations, and did not communicate this explicitly by refusing to commit.

It is, however, difficult to make inferences about the non-response group. Our interpretation of non- response as “not committed to attending” may be an overgeneralization of a diverse group, as reasons to not commit can range from not seeing the email, to consciously deciding not to reply, to know in advance that one will miss one or more sessions. To shed light on the reasons for the non-response, we contacted non-responding students to ask them about their rea- sons for not making the commitment. Unfortunately, but not unexpectedly, students either provided no or uninformative replies making it impossible to derive meaningful conclusions. Future research could investi- gate the commitment intervention in a condition in which all students are obliged to reply to the interven- tion (i.e., when enrolling for the course they have to fill in the commitment question to complete the enrollment) to reduce non-response. Alternatively, future research could integrate a more substantial qualitative component (e.g., interviews, focus groups) to determine behavioral determinants of participants, the effect of the intervention, and students’ reaction to receiving the email. This would especially help inter- pret the underlying motives of the non-response group and the effect the intervention has on them.

Implications

For educational practitioners, an important implica- tion of this study is the presumed backfire effect of the behavioral contract on the non-responders as a means to improve online attendance. The usage of a

course-wide behavioral contract to improve attend- ance is therefore not desirable. Because teachers often use behavioral interventions without being aware of doing so (Weijers et al., n.d) this is an especially important finding. Our findings provide insight not only for this specific instance of course attendance, but could also be extended to attendance in different online contexts, or even online meetings outside of education where attendance is a prerequisite for suc- cess, like company stakeholder meetings.

An important implication for researchers is that this study shows that the initial result of a commit- ment-based intervention can be misleading. In our study, we find that the presumed positive psycho- logical effect of the intervention is more likely explained by a selection bias for those who make the commitment than by the success of the intervention. The main effect of the intervention in this study was likely negative, namely a backfire effect: students who did not respond had a lower attendance rate. Some previous studies have reported a positive result similar to our initial findings (for an overview, see Lokhorst et al., 2013) without investigating the possibility of selection bias. This while it is very important to inves- tigate these possibilities to determine the effectiveness and long-term effects of behavioral interventions (Marchiori et al., 2017) especially in educational con- texts (Weijers et al., 2021), as educational goals are often long-term and focused on student development. Therefore, we urge researchers investigating commit- ment to account for this possible selection bias dis- torting the actual effect of the intervention and to choose multiple moments of measurement when- ever possible.

Conclusion

Attendance is an important predictor for academic success, but student attendance is dropping, especially during the COVID-19 pandemic. Students often fall victim to behavioral barriers that prevent them from attending class. Therefore, new tools for teachers are necessary to overcome these barriers. In this study, we have implemented a version of a commitment inter- vention, the behavioral contract, as a course-wide pol- icy, to increase students’ online attendance. No policy effect was found, but a positive psychological effect was found in commitment: students who committed to attending class did so more often than those who did not respond and those in the control condition. When exploring the data further, the lack of policy effect is possibly due to the high attendance rates

BASIC AND APPLIED SOCIAL PSYCHOLOGY 21

(>90%): in the one course with lower attendance, we do find evidence of a policy effect. While this result seemed promising, after exploratory analysis we have reason to assume that the found results are because of a selection bias. Students who are more likely to attend class are also more likely to commit to the intervention. In fact, analysis of previous attendance revealed that the intervention could have had an adverse effect on students who did not commit, as their attendance rate went down. Further research is needed to find what behavioral determinants are the barriers to attendance. When identified, these barriers can then be circumvented using a behavioral interven- tion strategy targeting them. This can then result in a successful intervention that can increase the attend- ance of students for their (online) classes, ultimately improving their academic success.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Funding

This work was supported by the Dutch Organization for Scientific Research (NWO) under grant [40.5.18540.132].

ORCID

Bj€orn B. de Koning http://orcid.org/0000-0001- 5136-2261

References

Archambault, L., Kennedy, K., & Bender, S. (2013). Cyber- truancy: Addressing issues of attendance in the digital age. Journal of Research on Technology in Education, 46(1), 1–28. https://doi.org/10.1080/15391523.2013.10782611

Association of Universities the Netherlands. (2020). Universiteiten in coronatijd [Universities in corona time]. https://www.vsnu.nl/files/documenten/Universiteiten%20in% 20coronatijd.pdf.

Baca-Motes, K., Brown, A., Gneezy, A., Keenan, E. A., & Nelson, L. D. (2013). Commitment and behavior change: Evidence from the field. Journal of Consumer Research, 39(5), 1070–1084. https://doi.org/10.1086/667226

Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. https://doi.org/10.1016/j.jml. 2012.11.001

Bates, D., M€achler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/ jss.v067.i01

Bhattacharya, J., Garber, A. M., & Goldhaber-Fiebert, J. D. (2015). Nudges in exercise commitment contracts: A randomized trial. National Bureau of Economic Research.

Bijsmans, P., & Schakel, A. H. (2018). The impact of attend- ance on first-year study success in problem-based learn- ing. Higher Education, 76(5), 865–881. https://doi.org/10. 1007/s10734-018-0243-4

Brouwer, J., Jansen, E., Severiens, S., & Meeuwisse, M. (2019). Interaction and belongingness in two student-cen- tered learning environments. International Journal of Educational Research, 97, 119–130. https://doi.org/10. 1016/j.ijer.2019.07.006

B€uchele, S. (2021). Evaluating the link between attendance and performance in higher education: The role of class- room engagement dimensions. Assessment & Evaluation in Higher Education, 46(1), 132–150. https://doi.org/10. 1080/02602938.2020.1754330

Chakraborty, P., Mittal, P., Gupta, M. S., Yadav, S., & Arora, A. (2021). Opinion of students on online educa- tion during the COVID-19 pandemic. Human Behavior and Emerging Technologies, 3(3), 357–365. https://doi.org/ 10.1002/hbe2.240

Cialdini, R. B. (1993). Influence: Science and practice. Harper Collins College Publishers.

Clark, D., Gill, D., Prowse, V., & Rush, M. (2020). Using goals to motivate college students: Theory and evidence from field experiments. The Review of Economics and Statistics, 102(4), 648–645. https://doi.org/10.1162/rest_a_00864

Cred�e, M., Roch, S. G., & Kieszczynka, U. M. (2010). Class attendance in college: A meta-analytic review of the rela- tionship of class attendance with grades and student characteristics. Review of Educational Research, 80(2), 272–295. https://doi.org/10.3102/0034654310362998

Damgaard, M. T., & Nielsen, H. S. (2018). Nudging in edu- cation. Economics of Education Review, 64, 313–342. https://doi.org/10.1016/j.econedurev.2018.03.008

Deci, E. L., & Ryan, R. M. (2012). Motivation, personality, and development within embedded social contexts: An overview of self-determination theory. In R. M. Ryan (Ed.), The Oxford handbook of human motivation (pp. 85–107). Oxford University Press.

Dewies, M., Schop-Etman, A., Rohde, K. I., & Denktaş, S. (2021). Nudging is ineffective when attitudes are unsup- portive: An example from a natural field experiment. Basic and Applied Social Psychology, 43(4), 213. https:// doi.org/10.1080/01973533.2021.1917412

Duckworth, A. L., Grant, H., Loew, B., Oettingen, G., & Gollwitzer, P. M. (2011). Self-regulation strategies improve self-discipline in adolescents: Benefits of mental contrasting and implementation intentions. Educational Psychology, 31(1), 17–26. https://doi.org/10.1080/01443410. 2010.506003

Duckworth, A. L., Kirby, T. A., Gollwitzer, A., & Oettingen, G. (2013). From fantasy to action: Mental contrasting with implementation intentions (MCII) improves aca- demic performance in children. Social Psychological and Personality Science, 4(6), 745–753. https://doi.org/10.1177/ 1948550613476307

DuPaul, G. J., & Stoner, G. (2014). ADHD in the schools: Assessment and intervention strategies. Guilford Publications.

Figlio, D., Rush, M., & Yin, L. (2013). Is it live or is it inter- net? Experimental estimates of the effects of online

22 R. J. WEIJERS ET AL.

instruction on student learning. Journal of Labor Economics, 31(4), 763–784. https://doi.org/10.1086/669930

Freedman, J. L., & Fraser, S. C. (1966). Compliance without pressure: The foot-in-the-door technique. Journal of Personality and Social Psychology, 4(2), 195–202. https:// doi.org/10.1037/h0023552

Golding, J. M. (2011). The role of attendance in lecture classes: You can lead a horse to water… . Teaching of Psychology, 38(1), 40–42. https://doi.org/10.1177/0098628310390915

Hansen, P. G. (2018). BASIC: Behavioural insights toolkit and ethical guidelines for policy makers. OECD Publishing. https://doi.org/10.1787/9ea76a8f-en

Kickert, R., Meeuwisse, M., M. Stegers-Jager, K., V. Koppenol-Gonzalez, G., R. Arends, L., & Prinzie, P. (2019). Assessment policies and academic performance within a single course: The role of motivation and self-regulation. Assessment & Evaluation in Higher Education, 44(8), 1177–1190. https://doi.org/10.1080/02602938.2019.1580674

Knowles, E. S., & Linn, J. A. (Eds.). (2004). Resistance and persuasion. Psychology Press.

Lokhorst, A. M., Werner, C., Staats, H., van Dijk, E., & Gale, J. L. (2013). Commitment and behavior change: A meta- analysis and critical review of commitment-making strat- egies in environmental research. Environment and Behavior, 45(1), 3–34. https://doi.org/10.1177/0013916511411477

Marchiori, D. R., Adriaanse, M. A., & De Ridder, D. T. (2017). Unresolved questions in nudging research: Putting the psychology back in nudging. Social and Personality Psychology Compass, 11(1), e12297. https:// doi.org/10.1111/spc3.12297

Michie, S., Atkins, L., & West, R. (2014). The behaviour change wheel. A guide to designing interventions. Silverback Publishing.

Meeter, M., Bele, T., den Hartogh, C., Bakker, T., de Vries, R. E., & Plak, S. (2020). College students’ motivation and study results after COVID-19 stay-at-home orders. PsyArxiv. https://doi.org/10.31234/osf.io/kn6v9

Nieuwoudt, J. E. (2020). Investigating synchronous and asynchronous class attendance as predictors of academic success in online education. Australasian Journal of Educational Technology, 36(3), 15–25. https://doi.org/10. 14742/ajet.5137

Oettingen, G., Kappes, H. B., Guttenberg, K. B., & Gollwitzer, P. M. (2015). Self-regulation of time manage- ment: Mental contrasting with implementation intentions. European Journal of Social Psychology, 45(2), 218–229. https://doi.org/10.1002/ejsp.2090

Onyema, E. M., Eucheria, N. C., Obafemi, F. A., Sen, S., Atonye, F. G., Sharma, A., & Alsayed, A. O. (2020). Impact of Coronavirus pandemic on education. Journal of Education and Practice, 11(13), 108–121.

Osman, M., McLachlan, S., Fenton, N., Neil, M., L€ofstedt, R., & Meder, B. (2020). Learning from behavioural changes that fail. Trends in Cognitive Sciences, 24(12), 969–980. https://doi.org/10.1016/j.tics.2020.09.009

R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.r-project.org/.

Rapposelli, J. (2014). The correlation between attendance and participation with respect to student achievement in

an online learning environment [Unpublished doctoral dissertation]. https://digitalcommons.liberty.edu/cgi/view- content.cgi?article=1858&context=doctoral

Rijksoverheid. (2020). Veelgestelde vragen over het corona- virus en het hoger onderwijs [Frequently asked questions about the coronavirus and higher education]. https:// www.rijksoverheid.nl/onderwerpen/coronavirus-covid-19/ ouders-scholieren-en-studenten-kinderopvang-en-onder- wijs/hogescholen-en-universiteiten-hoger-onderwijs.

Ruggeri, K. (Ed.). (2019). Behavioral insights for public pol- icy: Concepts and cases. Routledge.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social devel- opment, and well-being. American Psychologist, 33(1), 68–78.

Schneider, S., Nebel, S., Beege, M., & Rey, G. D. (2018). The autonomy-enhancing effects of choice on cognitive load, motivation and learning with digital media. Learning and Instruction, 58, 161–172. https://doi.org/10. 1016/j.learninstruc.2018.06.006

Srivastava, P. (2012). Getting engaged: Giving employees a nudge toward better health. Compensation & Benefits Review, 44(2), 105–109. https://doi.org/10.1177/088636871 2450982

Sund, K. J., & Bignoux, S. (2018). Can the performance effect be ignored in the attendance policy discussion? Higher Education Quarterly, 72(4), 360–374. https://doi. org/10.1111/hequ.12172

Sunstein, C. R. (2017). Nudges that fail. Behavioural Public Policy, 1(1), 4–25. https://doi.org/10.1017/bpp.2016.3

Thaler, R. H., & Sunstein, C. R. (2008). Nudge: Improving decisions about health, wealth, and happiness. Penguin.

The New York Times. (2020). As schools moves online, many students stay logged out. https://www.nytimes. com/2020/04/06/us/coronavirus-schools-attendance-absent. html

Weijers, R. J., de Koning, B. B., & Paas, F. (2021). Nudging in education: From theory towards guidelines for success- ful implementation. European Journal of Psychology of Education, 36(3), 883–902. https://doi.org/10.1007/ s10212-020-00495-0

Weijers, R. J., de Koning, B. B., Klatter, E. B., & Paas, F. (n.d). Nudging autonomous learning behavior: How do teachers in secondary vocational education nudge their students? European Journal of Psychology of Education, 36, 883–902.

Westfall, J., Kenny, D. A., & Judd, C. M. (2014). Statistical power and optimal design in experiments in which sam- ples of participants respond to samples of stimuli. Journal of Experimental Psychology. General, 143(5), 2020–2045. https://doi.org/10.1037/xge0000014

Wickens, C. M., McDonald, A. J., Elton-Marshall, T., Wells, S., Nigatu, Y. T., Jankowicz, D., & Hamilton, H. A. (2021). Loneliness in the COVID-19 pandemic: Associations with age, gender and their interaction. Journal of Psychiatric Research, 136, 103–108. https://doi. org/10.1016/j.jpsychires.2021.01.047

BASIC AND APPLIED SOCIAL PSYCHOLOGY 23

Appendix A. Text used for email

Experimental Dear student, Course [name] is starting soon. Due to the COVID-19

measures, we have tried our best to optimize the course for an online learning environment. Because of this, I’d like to ask you a favor.

I would like to ask you to indicate whether you will join all sessions of the online meetings. We are asking a random sample of all students, because there is no attendance requirement for this course, only an effort requirement. We need this information to put together the right content of these meetings.

Please answer by replying to this e-mail in the follow- ing format:

Hereby I … … (fill in name) declare that I will / will not (delete as appropriate) be present during all online meetings of [course name]. This means that I will attend the meetings every [weekday] and [weekday] in the next five weeks.

Thank you in advance. Kind regards, [name of course coordinator]

Reminder Dear student, A few days ago, you received an email asking you to

indicate whether you will attend all sessions of [course name]. We are asking a random sample of all students, because there is no attendance requirement for this course, only an effort requirement.

I would like to ask you again to indicate whether you will attend all sessions. We use this information to put together the right content of these meetings.

Please answer by replying to this e-mail in the follow- ing format:

Hereby I … … (fill in name) declare that I will / will not (delete as appropriate) be present during all online meetings of [course name]. This means that I will attend the meetings every [weekday] and [weekday] in the next five weeks.

Thank you in advance. Kind regards, [name of course coordinator]

Control Dear student, [Course name] is starting soon. Due to the COVID-19

measures, we have tried our best to optimize the course for an online learning environment. I hope you’ll be able to end the academic year in a good way!

Kind regards, [name of course coordinator]

24 R. J. WEIJERS ET AL.