Assessment 1 - Evaluating Scientific Merit

Academic Engagement and Student Success: Do High-Impact Practices Mean Higher Graduation Rates? Sarah Randall Johnsona and Frances King Stageb

aInstitutional Research, Harvard Business School, Boston, Massachusetts, USA; bHigher and Postsecondary Education, New York University, New York, New York, USA

ABSTRACT This study examined the relationship between 10 high-impact practices and graduation rates at four-year public colleges and universities in the United States. The Association of American Colleges and Universities defined high-impact practices as especially effective for student learning, engagement, and career preparation in the 21st century. While advocacy for these practices and their inclusion in undergraduate curricula is growing, little research has examined their relationship to institutional outcomes. Based on data from 101 participating institutions, this study used both primary and secondary data to investigate whether offering high-impact practices as required for all students, required for some students, or optional was related to an institution’s four or six-year gradua- tion rate. The findings suggest that high-impact practices are in widespread use across different institutional types but have limited relationships with graduation rates. This study contri- butes to the body of literature on college completion. Findings suggest that offering high-impact practices may not lead to increased graduation rates at public institutions.

ARTICLE HISTORY Received 21 October 2016 Accepted 12 February 2018

KEYWORDS Academic engagement; college completion; high- impact practices

Introduction

College completion became a prominent subject of public attention in the United States during the past decade (Bowen, Chingos, & McPherson, 2009; Obama, 2009; E. Porter, 2014). Financial constraints, concerns about accountability, and desire for demonstrable outcomes put pressure on col- leges and universities to increase graduation rates. Although the general population entering college rose in number, the proportion of students completing college degrees remained steady and, in some years, slightly declined (Carey, 2004; Shapiro et al., 2015). The six-year completion rate is about 50% (U.S. Department of Education, 2015), and the time taken to earn a college degree in the United States has consistently risen since the mid- 1980s (Bound, Lovenheim, & Turner, 2007).

An additional, parallel concern in the national discourse is quality of baccalaureate study. Some have suggested there are declines in writing

CONTACT Sarah Randall Johnson [email protected]

THE JOURNAL OF HIGHER EDUCATION 2018, VOL. 89, NO. 5, 753–781 https://doi.org/10.1080/00221546.2018.1441107

© The Ohio State University

instruction, critical thinking, quantitative skills, and moral reasoning (Arum & Roksa, 2011; Bok, 2008). The National Commission on the Future of Higher Education (2006) reported, “There are also disturbing signs that many students who do earn degrees have not actually mastered the reading, writing, and thinking skills we expect of college graduates” (p. 5).

The Association of American Colleges and Universities (AAC&U) is con- cerned with the “quality, vitality, and public standing of undergraduate liberal education” and promotes the adoption of specific higher education practices on behalf of its 1,300 member institutions (Kuh, 2008, p. 48). In 2008, the AAC&U produced a series of reports defining 10 “high-impact practices” as particularly effective in cultivating student learning and prepar- ing students for future careers (Kuh, 2008). These 10 practices are: first-year seminars, core curricula, learning communities, writing-intensive courses, collaborative assignments, undergraduate research, diversity/global learning experiences, service learning, internships, and capstones or senior projects. Using data from the National Survey on Student Engagement (NSSE), researchers at AAC&U cited relationships between student participation in the 10 practices and both personal and academic gains. Further, they argued, the practices were uniquely effective in providing students with a quality undergraduate education and for success in a “new global century” (Kuh, 2008).

This study examined whether four-year public institutions incorporating the AAC&U’s high-impact practices had higher four and six-year gradua- tion rates than institutions that did not implement the practices. Data reported in this study provided an opportunity to connect student persis- tence and success to specific campus practices at a large number of institutions. Examining connections between the AAC&U’s 10 practices and institutional outcomes is important because of their widespread recommendation and adoption at the expense of other possible offerings. In addition, this study came at a time when public institutions faced increased budget constraints. While research has examined further con- nections between high-impact practices and student learning and engage- ment (Hu & McCormick, 2012; Kilgo, Sheets, & Pascarella, 2015), little attention has been paid to whether these 10 practices influence college completion or postcollege outcomes.

Much existing research on college outcomes has centered on student- level data at single institutions. This study was unique in its attention to student outcomes on multiple college campuses. Further, little research has been conducted at the institutional level to understand how institutional decision making and curricular tradeoffs might impact broader outcomes. With this study, we expanded the current body of research on college completion and time to degree by shedding light on the relationship

754 S. R. JOHNSON AND F. K. STAGE

between college graduation and the AAC&U’s institutional quality stan- dards for American higher education.

Literature review

Research on student persistence has primarily focused on campus engage- ment and individual student characteristics. Taken together, studies by Astin (1993) and Tinto (1993) revealed that high school grades, parents’ education levels, family encouragement, and academic and social integra- tion were primary determinants of degree attainment. Other studies con- firmed these variables as important predictors of persistence and degree attainment but also stressed the importance of socioeconomic status and environmental factors (Cabrera, Castaneda, Nora, & Hengstler, 1992; Nora, 2003; Runyan, 2011).

Because persistence is a precondition of completion, factors contributing to successful college completion can best be understood through theories of student persistence (DesJardins, Kim, & Rzonca, 2003). Academic and social engagement are intertwined in relation to student persistence (Stage, 1989). Social engagement is a mechanism for student development and for commitment to college completion (Astin, 1984; Bean, 1980). Student engagement within an institutional environment can have both positive and negative associations with persistence (Tinto, 1993). Students who engaged in campus communities and succeeded academically were likely to persist in college and ultimately obtain a degree (Astin, 1984; Pascarella & Terenzini, 2005; Spady, 1971; Tinto, 1993). Interpretation of relationships between student outcomes and high-impact practices should consider the nature of these practices as interconnected, academically and socially engaging experiences.

Academic obstacles to completion have been comingled with student background characteristics. Black and Hispanic students are dispropor- tionately first-generation college students who face a number of obsta- cles to degree completion, particularly the absence of resources to navigate social and academic cultural practices (Corrigan, 2003). Research has linked higher SAT scores with shorter time to degree and with four, five, and six-year graduation rates (American Federation of Teachers, 2003; Goener & Snaith, 2003; Knight, 1994). Socioeconomic status influences access to college entrance examinations and costly exam preparation. Studies on degree completion have frequently focused on associations with student characteristics. Some have proposed that greater attention should instead be given to sorting—a widening strati- fication of students attending elite private institutions versus those attending resource-strapped public institutions—as the culprit for slowed rates of completion (Bound et al., 2007). The sorting phenomenon

THE JOURNAL OF HIGHER EDUCATION 755

illustrates the importance of identifying institutional differences in understanding how different student populations respond to various academic activities.

Early-career high-impact practices

Freshman seminars Research has demonstrated relationships between student learning, academic outcomes, and the AAC&U’s high-impact practices, particularly in the early years of college. For decades, researchers have emphasized a student’s first year of college as being uniquely important for retention and campus inte- gration (Pascarella & Terenzini, 2005; Tinto, 1993). However, the relation- ship between retention and participation for a first-year seminar course is unclear. More than one study revealed no relationship between second-year retention and first-year seminars in the general college student population (Janz, 2004; Potts & Schultz, 2008). The relationship was significant for one specific academically at-risk population: students whose high school ranks and grade point averages (GPAs) were lower than average for the college’s entering class (Potts & Schultz, 2008).

A national study of more than 20,000 undergraduates revealed that first-year seminars had a positive impact on students’ intent to return in their second year (Porter & Swing, 2006). However, content of the course was important to the seminar’s magnitude of influence. Courses emphasizing good study habits and that educated students on matters related to their health had the greatest impact on persistence (Porter & Swing, 2006).

First-year seminars may have longer-term impacts that are more difficult to measure than grades or retention. Padgett, Keup, and Pascarella (2013) found that participation in first-year seminars enhanced students’ orientations toward lifelong learning years after they left college, and they identified “indirect effects that suggest first- year seminars are fostering meaningful learning objectives that further boost students’ need for cognition” (p. 145).

Core curricula Relationships with peers are among the most powerful socializing factors shaping college student persistence (Astin, 1993; Pascarella & Terenzini, 2005). However, there has been limited research linking core liberal arts curricula with persistence or completion. One study showed that students taking two of the same courses together in their first year were more than twice as likely to complete their degree within 6 years than were students who took one or no common courses in the first year (Nora, Barlow, & Crisp, 2005). Evidence exists that mastery of material as well as cognitive

756 S. R. JOHNSON AND F. K. STAGE

abilities are influenced by the pattern and sequencing of courses taken by undergraduate students (Pascarella & Terenzini, 2005), suggesting that students benefit from purposefully designed curricula and established sequencing of courses.

Learning communities Analyses of NSSE data by Kuh (2008) and AAC&U showed positive relationships between participation in learning communities and self- reported gains in deep learning as well as in personal and practical experi- ences among first-year students. A longitudinal study of participants in one learning community for psychology majors revealed that participants had higher rates of persistence and a higher mean GPA than those who had not participated (Buch & Spaulding, 2008). It is unknown whether students who elected not to participate in the learning community had additional characteristics related to lower GPA and persistence. Drawing direct links between optional cohort programs and outcomes is complicated by the fact that students who elect to take additional academic programming are likely already engaged on campus.

Writing courses Extensive classroom writing was connected to enhanced learning as well as academic and personal development (Bangert-Drowns, Hurley, & Wilkinson, 2004). One reason was that colleges emphasizing the importance of writing across the undergraduate curriculum often implemented parallel program- matic efforts, such as increased attention to quantitative reasoning, oral communication, and information literacy (Kuh, 2008). Multiple studies have linked classroom writing assignments to higher grades and higher test scores (Bangert-Drowns et al., 2004; Cisero, 2006; Horton, Fronk, & Walton, 1985). Bangert-Drowns et al. (2004) concluded that the length of writing assignments had a positive relationship with academic performance and that “assignments requiring students to evaluate their current understandings, confusions, and feelings in relation to the subject matter yielded more positive effects than instruction that did not include such metacognitive stimulation” (p. 47). Focusing on personal reflection in writing may espe- cially benefit students in the math and science fields for which organizing and summarizing data can be useful for understanding what is known and what is left unknown (Horton et al., 1985).

High-impact practices for upper classmen

Collaborative assignments Educators often argue that when instructors organize students in groups to solve a problem, complete a task, or create a product, they experience a wide

THE JOURNAL OF HIGHER EDUCATION 757

range of social, psychological, and academic benefits (Kuh, 2008; Laal & Ghodsi, 2012). Working collaboratively in college has been associated with gains in personal development, understanding of science and technology, appreciation for art, and analytical skills (Cabrera, Nora, Crissman, & Terenzini, 2002). Lei, Kuestermeyer, and Westmeyer (2010) cautioned researchers against attributing too many benefits to group work: “Group composition is always associated with the effects of multiple confounding variables due to the overall nature of collaborative learning at the college level” (p. 317). In assessing bodies of research on student collaboration, one should use caution because “collaborative learning” can have a wide range of definitions, including informal study groups, team-based class assignments, and long-term research projects (Kuh, 2008).

Undergraduate research The AAC&U described the goals of exposing college students to independent research as involving them with “actively contested questions, empirical observation, cutting-edge technologies, and the sense of excitement that comes from working to answer important questions” (Kuh, 2008, p. 10). Using NSSE data, Kuh (2008) found that seniors who assisted faculty on research projects reported significant gains in personal, practical, and general learning. Douglass and Zhao (2013) also found a relationship between stu- dent learning and independent research. However, they concluded that independent student research projects were a more reliable mechanism for student learning gains than assisting faculty with research because of assis- tants’ varied nature of tasks and the limited scope in which a college student could be involved (Douglass & Zhao, 2013).

Study abroad Study abroad and cultural exchange programs are designed to expose stu- dents to new experiences and to develop their cultural awareness, sensitivity, and communication skills (Salisbury, 2011). Researchers have linked such programs to graduation rates, time to degree, retention, and GPA (Ingraham & Peterson, 2004; Metzger, 2006). At best, these studies have demonstrated associations between institutional characteristics and students who partici- pate. They have not addressed issues of socioeconomic status in program participation or characteristics of students who choose to study abroad.

A study of 1,500 students related to intercultural competence and study abroad concluded that study abroad had no significant influence on one’s appreciation of cultural differences or levels of comfort with diversity (Salisbury, 2011). Culturally expansive college experiences may be too diverse in nature and students may be too selective in immersive behaviors to understand significant effects of programs as a comprehensive whole.

758 S. R. JOHNSON AND F. K. STAGE

Service learning Service-learning proponents have argued that college programs should be broadened to give the greatest number of students access to benefits (Berry & Chisholm, 1999; Bill & Melinda Gates Foundation, 2010; Tos, 2015). Service learning is perhaps most recognized as helping students make connections between real-life experiences and classroom content. Research by Maynes, Hatt, and Wideman (2013) supported the integration of community service into the college curriculum to maximize potential for student learning: “Learning may remain tacit rather than explicit unless substantial opportu- nities for reflection are included in the service learning experience” (p. 80).

Personal independence and self-confidence were additional benefits of involvement in community service and service learning (Prentice & Robison, 2010). Participation in service learning was linked to an increased sense of independence and personal responsibility for one’s own learning (Largent & Horinek, 2008; Tos, 2015). Overall, research on community involvement during college has demonstrated a positive impact on classroom learning and personal development, both of which have been linked to college persistence and completion.

Internships Internships provide students with exposure to professional environments they might seek after obtaining their college degrees. Previous studies have shown correlations between participation in college internships and students’ academic outcomes (Binder, Baguley, Crook, & Miller, 2015; Mansfield, 2011). A study of undergraduates in the United Kingdom revealed that students placed in internships through programs at their college scored 4% higher in final-year exams than did students who were not placed in intern- ships (Gomez, Lush, & Clements, 2004). Studies linking internship participa- tion to test scores have been criticized for neglecting to include student background characteristics, particularly family income. Students of higher socioeconomic status may be more likely to seek internships and exhibit other desirable academic behaviors.

Recent research has explored broader outcomes beyond student intern- ships. A study of undergraduate students interested in careers in the life and health sciences showed that internship participation improved essential- learning outcomes and clarified students’ postgraduation goals (Gilbert, Banks, Houser, Rhodes, & Lees, 2014). Binder et al. (2015) concluded that college internships provide a variety of academic advantages to students from diverse backgrounds across a wide range of majors.

Capstone or senior projects Capstone or senior projects are often described as “culminating experiences” or exercises in “integration” of knowledge acquired during the previous 3

THE JOURNAL OF HIGHER EDUCATION 759

years of college. One study on a capstone program at a private university showed that humanities majors who participated graduated with stronger oral and written communication skills and an increased willingness to work collaboratively with others (Brooks, Benton-Kupper, & Slayton, 2004). These findings support arguments by the AAC&U that the benefits of capstone experiences extend beyond classroom content to interpersonal development and increased communication with faculty (Kinzie, 2013).

Collier (2000) examined the role of capstone courses in personal identity development. He found that students participating in a senior capstone exhibited stronger characteristics of personal identity and socialization than did their peers who did not participate (Collier, 2000). Collier also suggested that capstone experiences positively contributed to students’ understanding of their transitional role from college student to professional adult. Although students reaching their senior year of college have a relatively high likelihood of degree completion, capstone courses may contribute to a final push necessary for graduation and pursuit of a career in a student’s major field.

Few studies published independently from AAC&U have examined more than one high-impact practice at the same time. Wolniak and Engberg (2015) reported small and generally inconsistent relationships between participation in certain high-impact practices and early-career outcomes, such as job satisfaction and earnings. Some high-impact practices have demonstrated stronger relationships with student learning than others. Using data from the Wabash National Study of Liberal Arts Education, Kilgo et al. (2015) found that group learning and undergraduate research “had broad-reaching positive effects across multiple liberal arts learning outcomes, such as critical thinking, need for cognition, and intercultural effectiveness” (p. 509). They also concluded that study abroad, internships, service learning, and capstone projects had small positive effects on student learning (Kilgo et al., 2015). However, Hatch (2013) measured association between levels of engagement in community college students and participation in select high-impact prac- tices and found no significant relationships. While students have reported experiencing learning gains following participation in individual high-impact practices, questions remain regarding AAC&U’s high-impact practices as a set of standards in undergraduate education.

Summary

Research has shown relationships between student learning, academic out- comes, and high-impact practices. Multiple studies have demonstrated posi- tive associations that first-year seminars, writing requirements, learning communities, and service learning have with student persistence and aca- demic achievement (Berry & Chisolm, 1999; Buch & Spaulding, 2008; Cisero, 2006; Porter & Swing, 2006; Potts & Schultz, 2008). Given evidence regarding

760 S. R. JOHNSON AND F. K. STAGE

the 10 high-impact practices (Kuh, 2008) as well as broader literature on engagement and persistence (Astin, 1984; Tinto, 1993), we hypothesized that these four practices would have positive associations with graduation rates. In addition, we hypothesized that the freshman seminar would have the largest positive association with graduation rates; first-year experiences have received greater attention in the literature compared with other high- impact practices.

Individual background characteristics are also connected with persistence. This study included institutional variables as well as student demographic characteristics previously associated with graduation rates. We also included expenditures per pupil, selectivity, proportion of the student body receiving financial aid, and the proportion of the student body identifying as White. By controlling for these institutional student variables, we aimed to refine associations of high-impact practices with college graduation rates. Previous research has primarily relied on data from national surveys of student enrollment and on enrollment at single institutions. This study was unique in its attempt to connect research on individual student persistence and enrollment to institutional-level variables.

Method

Data collection

A cross-sectional data set was constructed that included both primary and secondary data. Primary data were collected through an online survey of academic officers regarding the availability of high-impact practices at their institutions. Secondary data were obtained from Barron’s Profile of American Colleges (2015) and from the U.S. Department of Education’s Integrated Postsecondary Education Data System (IPEDS).

A review of institutional Web sites allowed us to identify academic officers and advisors in baccalaureate programs who received our survey by e-mail. Examples of recipients’ titles included director of academic advising, assistant dean for academic advising, and associate dean for academic programs. Two individuals at each institution were identified as primary and secondary contacts. For large universities with more than one undergraduate college, individuals were identified from the institution’s college of arts and sciences or college of liberal arts because high-impact practices are part of the AAC&U recommendations for liberal arts education. An e-mail request for participation was sent to primary contacts. Three follow-up e-mails were sent to nonrespondents. At institutions where the primary contact had not responded, three e-mail attempts were made to secondary contacts.

Survey responses regarding the offering of each high-impact practice were limited to four options measuring the institutional availability of the

THE JOURNAL OF HIGHER EDUCATION 761

practice. Responses included “required for all students,” “required for some students,” “optional for students,” or “not offered.” Responses were assigned a numerical value. Respondents could also select the option “I am not sure” with an open text box for further comment. Assigning ordinal values to participation levels rather than as categorical reflected an assump- tion that student access to high-impact practices has a positive relationship with campus engagement. A second purpose was to measure the extent to which high-impact practices are integrated into each institution’s curricu- lum from the perspective of a senior academic administrator instead of the view from individual classrooms. In addition to using individual practice scores, a composite score was calculated for each institution as a sum of the 10 high-impact practice variables and their corresponding values to mea- sure each institution’s total level of offerings.

Sample

In 2013, 244 four-year public institutions granting baccalaureate degrees in the United States enrolled 10,000 or more undergraduates. The study popu- lation included all 244 institutions. There were 101 institutions in our final analytic sample, a 41.4% response rate. We examined characteristics in large public institutions for two reasons. Institutions with more than 10,000 undergraduates enroll more than half the college student population in the United States. Additionally, students with socioeconomic characteristics associated with lengthened time to degree are more likely to attend public rather than private institutions (Perna & Titus, 2007). More than 60% of Black college students and 81% of Hispanic students attend public institu- tions, and 75% of federal financial aid recipients attend public institutions (U.S. Department of Education, 2014). For these reasons, large public insti- tutions are well suited for examining how academic practices might help diverse students across the United States complete college degrees.

Dependent variables

Two dependent variables were included in the study: four-year and six-year graduation rates (U.S. Department of Education, 2015). The U.S. government defines four-year graduation rate as the proportion of students entering an institution as full-time, first-time degree-seeking undergraduates in a parti- cular year (cohort) who completed their degrees within 4 years. Six-year graduation rates refer to the proportion of the cohort completing their program within 6 years of their first enrollment. Dependent variables were obtained through IPEDS for the 2013–2014 academic year.

762 S. R. JOHNSON AND F. K. STAGE

Independent variables

Independent variables for this study were the 10 high-impact practices recommended by the AAC&U: freshman seminars, core curriculum, learn- ing communities, writing-intensive courses, collaborative assignments, undergraduate research, study abroad, service learning, internships, and senior capstone or thesis. These variables were measured by level of offering: 0 = not offered, 1 = optional, 2 = required for some, 3 = required for all (see Table 1).

Control variables

Selectivity levels and corresponding numerical values were based on rat- ings from Barron’s Profile of American Colleges: 6 = most competitive, 5 = highly competitive, 4 = competitive, 3 = somewhat competitive, 2 = less competitive, and 1 = noncompetitive (Barron’s Profile of American Colleges, 2015). Baccalaureate institutions were also categorized into four Carnegie Classifications and were assigned corresponding values: 1 = doctoral-granting/research universities, 2 = master’s colleges and univer- sities, 3 = baccalaureate colleges, and 4 = associate degree colleges. This study controlled for institutional selectivity and Carnegie Classification because research universities have resources for high-impact practices for

Table 1. High-impact practice definitions. High-Impact Practice Definition

First-Year Seminars and Experiences

The institution offers courses designated as freshman or first-year seminars.

Common Intellectual Experiences

The institution has a common core curriculum for all undergraduate students.

Learning Communities The institution offers courses that are linked together with the exact same group of enrolled students or offers courses that are designated as part of a learning community.

Writing- Intensive Courses

The institution has courses that are designated as writing-intensive.

Collaborative Assignments and Projects

Students work in teams or small groups for class assignments.

Undergraduate Research The institution permits undergraduate students to serve as faculty research assistants, or it establishes independent research opportunities specifically for undergraduates.

Diversity/Global Learning The institution operates programs in study abroad or in domestic cultural exchange.

Service or Community- Based Learning

The institution operates community service programs or offers courses that include community-based projects.

Internships The institution grants academic credit for internships or internships are integrated into the undergraduate curriculum.

Capstone Courses and Projects

The institution offers senior seminar courses or a senior thesis/capstone.

Note. For more detailed definitions, see https://www.aacu.org/leap/hips

THE JOURNAL OF HIGHER EDUCATION 763

students that bachelor’s-level and master’s-level institutions do not. Further, differentiating between institutions by selectivity and by Carnegie Classification could help determine whether high-impact prac- tices can mitigate substantial differences in student outcomes between different types of institutions.

Variables related to student body characteristics were obtained through IPEDS: expenditures per pupil, proportion of students receiving federal financial aid, proportion of students receiving private loans, and proportion of the student body identified as White.

Correlation

We constructed Spearman rank correlation matrices to examine relationships between high-impact practices, institutional variables, and graduation rates. Nonparametric tests, such as Spearman, are most accurate with data that are categorically ranked (Chok, 2010). We constructed separate matrices by selectivity level to understand how relationships between the variables might vary across institutional type.

Regression

Multiple regression models enabled us to examine the influence of high- impact practices across graduation rates. Two separate series of three regression models were created, one for each dependent variable of four and six-year graduation rates. The first model included 10 independent variables, 1 for each high-impact practice. The second model included 10 high-impact practices as well as 5 institutional variables: expenditures per pupil, proportion of students receiving federal financial aid, proportion of students receiving private loans, proportion of the student body identifying as White, and selectivity level. The third model included those 5 variables as well as the high-impact practice composite score and an interaction variable for selectivity and composite score. Multiple regression was an essential tool for this study because it provided an estimate of the influence of multiple institutional variables on graduation rates as well as the magni- tude of influence for each independent and control variable on each dependent variable.

Finally, to preserve sample size, we used multiple imputation (Rubin, 1987) to impute 14 high-impact practice responses that were missing. These 14 imputations made up 1.3% of the study’s high-impact practice responses. Multiple imputation uses information from other observations to generate contextually appropriate values for missing variables. Selectivity level, Carnegie Classification, and four-year graduation rates were used to identify the “nearest neighbor” institution that most closely resembled the

764 S. R. JOHNSON AND F. K. STAGE

institution with missing data. That institution’s value was used for imputa- tion. Multiple-imputation methods are commonly used in higher education research to improve the accuracy of estimation and to avoid sample loss (Oseguera & Hwang, 2014). One limitation of using multiple imputation is that it can result in lower standard errors for multiple regression coefficients and should be taken into consideration in interpreting this study’s analytic results (Gedikoglu & Parcell, 2013).

Results

Table 2 displays characteristics of responding institutions as well as char- acteristics of institutions in the overall population. Of the 101 participating institutions, 74.26% (n = 75) were doctoral-granting, research universities; 22.77% (n = 23) were master’s colleges or universities; and 2.97% (n = 3) were associate degree colleges that also award four-year bachelor’s degrees. No baccalaureate colleges were included; only five public institutions enrol- ling 10,000 or more undergraduates were classified as baccalaureate in the United States. They made up only 2.16% of the population. Additionally, the

Table 2. Institutional characteristics. Respondents Population

Characteristic N = 101 N = 244

Carnegie Classification Doctoral/Research Universities 74.26%* 57.19% Master’s Colleges and Universities 22.77%* 33.45% Baccalaureate Colleges 0%* 2.16% Associate Degree Colleges 2.97%* 7.19% Selectivity Most-Competitive or Highly Competitive 15.84%* 11.28% Competitive 74.26% 73.54% Less-Competitive or Noncompetitive 9.9%* 15.18% Finances Mean Annual Expenditures per Pupil $27,294.74 $23,100.86 Student Body Receiving Federal Aid 48.95% 48.34% Student Body Receiving Private Loans 5.00% 4.28% Student Body Mean Undergraduate Enrollment 19,378.17 18,380.11 Asian 7.27% 6.63% Black or African American 8.48% 10.21% Hispanic 11.39% 13.68% Native American/Alaska Native 0.42% 0.48% Native Hawaiian/Pacific Islander 1.85%* 0.21% White 61.74% 58.87% Two or More Races 2.94% 3.01% Graduation Rates Four-Year 35.09%* 28.47% Six-Year 59.11%* 53.64% AAC&U Membership 84.15%* 75.89%

Note. AAC&U = Association of American Colleges and Universities. Analytic sample differs from the full survey sample at * p < .05.

THE JOURNAL OF HIGHER EDUCATION 765

majority of institutions enrolling more than 10,000 students offer some type of graduate-level education.

According to Barron’s Profile of American Colleges (2015), 15.84% (n = 16) of participating institutions were either most competitive or highly compe- titive, 74.26% (n = 75) were competitive or very competitive, and 9.9% (n = 10) were less competitive or noncompetitive. The mean annual expen- ditures per pupil for participating institutions was $27,294.74. On average, 48.95% of the student body received financial aid from the federal govern- ment and 5% had taken out private loans to help pay tuition.

Mean undergraduate enrollment at participating institutions was 19,378 students. On average, 7.27% of students were Asian American, 8.48% were Black or African American, 11.39% were Hispanic or Latino, 0.42% were Native American or Alaska Natives, 1.85% were Native Hawaiian or Pacific Islanders, 61.74% were White or Caucasian, and 2.94% identified as being two or more races. The mean 4-year graduation rate was 35.09% and the 6- year graduation rate was 59.11%. Members of the AAC&U made up 84.15% (n = 85) of participating institutions. A few statistically significant differences existed between characteristics in the analytic sample and in the full popula- tion of large public institutions. The analytic sample contained a greater proportion of doctoral/research universities and lower proportions of the other Carnegie Classifications.

The proportion of most competitive institutions was larger and the pro- portion of least selective institutions was smaller in the analytic sample than in the full sample. For these reasons, conclusions about Carnegie Classification and nonselective institutions are limited to the participants in this study. Graduation rates were higher for the analytic sample, which may reflect the proportional differences in doctoral-granting and highly competi- tive institutions in the sample but should not impact the relationships examined in this study.

Correlation analysis

Table 3 displays the Spearman correlations for graduation rates, high-impact practices, the institution composite score for high-impact practices, selectiv- ity ranking, and Carnegie Classification for the 101 institutions in the sample (correlations of p ≤ .01 were considered significant). Four-year graduation rate was significant and highly positively correlated with six-year graduation rate (0.945), and it was negatively correlated with selectivity ranking (−0.493). Six-year graduation rate also had a significant negative correlation with selectivity ranking (−0.577). The institution composite score of high- impact practice offerings had a moderate, positive correlation with core curriculum (0.565). No others were significant.

766 S. R. JOHNSON AND F. K. STAGE

Ta b le

3. Sp ea rm

an co rr el at io n s fo r al l in st it ut io n s.

N = 10 1

Fo ur -Y ea r

G ra d ua ti on

Ra te

Si x- Ye ar

G ra d ua ti on

Ra te

Fr es h m an

Se m in ar

W ri ti n g

C or e

C ur ri cu lu m

Le ar n in g

C om

m un

it y

G ro up

W or k

St ud

en t

Re se ar ch

St ud

y A b ro ad

Se rv ic e

Le ar n in g

C ap st on

e In te rn sh ip

C om

p os it e

Sc or e

Se le ct iv it y

C ar n eg ie

Fo ur -Y ea r G ra d ua ti on

Ra te

1. 00 0

Si x- Ye ar

G ra d ua ti on

Ra te

.9 45 *

1. 00 0

Fr es h m an

Se m in ar

− .2 50

− .2 52

1. 00 0

W ri ti n g

− .1 33

− .1 45

− .0 29

1. 00 0

C or e C ur ri cu lu m

.0 24

.0 49

.1 04

.1 31

1. 00 0

Le ar n in g C om

m un

it y

− .1 92

− .2 51

.0 56

− .0 93

.0 91

1. 00 0

G ro up

W or k

− .1 23

− .0 85

− .0 01

.0 43

.2 51

.1 81

1. 00 0

St ud

en t Re se ar ch

.0 35

.0 32

.0 33

.0 48

− .0 13

− .0 71

.1 67

1. 00 0

St ud

y A b ro ad

.0 89

.1 43

− .0 66

.0 73

.1 16

− .0 52

.0 26

.0 59

1. 00 0

Se rv ic e Le ar n in g

.0 01

.0 54

− .0 93

.1 56

.1 16

− .1 02

− .0 27

.2 31

.0 96

1. 00 0

C ap st on

e − .0 86

− .0 69

.0 14

.2 69

.2 30

.1 26

.2 48

.0 72

.1 96

.1 46

1. 00 0

In te rn sh ip

− .1 47

.1 21

− .0 37

.0 71

.2 86

.0 20

.1 58

.0 43

.2 87

.1 58

.2 38

1. 00 0

C om

p os it e Sc or e

− .1 70

− .1 65

.4 09

.2 94

.5 65 *

.3 05

.4 41

.2 56

.2 43

.2 23

.5 48

.4 61

1. 00 0

Se le ct iv it y

− .4 93 *

− .5 77 *

− .0 37

.1 50

− .0 26

.1 90

.0 96

.1 10

− .0 54

− .0 55

.1 24

.1 41

.0 70

1. 00 0

C ar n eg ie

− .2 08

− .3 08

.0 86

.1 25

− .0 31

.2 58

.0 39

− .0 88

− .0 81

− .0 31

.1 14

.0 93

.1 48

.2 72

1. 00 0

* p < .0 1.

THE JOURNAL OF HIGHER EDUCATION 767

Ta b le

4. Sp ea rm

an co rr el at io n s fo r m os t- se le ct iv e in st it ut io n s.

N = 16

Fo ur -Y ea r

G ra d ua ti on

Ra te

Si x- Ye ar

G ra d ua ti on

Ra te

Fr es h m an

Se m in ar

W ri ti n g

C or e

C ur ri cu lu m

Le ar n in g

C om

m un

it y

G ro up

W or k

St ud

en t

Re se ar ch

St ud

y A b ro ad

Se rv ic e

Le ar n in g

C ap st on

e In te rn sh ip

C om

p os it e

Sc or e

C ar n eg ie

Fo ur -Y ea r G ra d ua ti on

Ra te

1. 00 0

Si x- Ye ar

G ra d ua ti on

Ra te

.8 66 *

1. 00 0

Fr es h m an

Se m in ar

− .5 98 *

− .6 30

1. 00 0

W ri ti n g

.0 81

.0 27

− .2 38

1. 00 0

C or e C ur ri cu lu m

− .3 67

− .3 63

.4 07

.3 38

1. 00 0

Le ar n in g C om

m un

it y

− .4 91 *

− .5 74 *

.4 45

− .3 32

.1 76

1. 00 0

G ro up

W or k

− .8 32 *

− .6 66 *

.5 51

− .3 39

.1 36

.3 63

1. 00 0

St ud

en t Re se ar ch

.1 40

− .1 40

− .2 20

.1 72

.1 23

.0 31

− .2 23

1. 00 0

St ud

y A b ro ad

.1 40

.1 12

.3 46

.1 72

.1 23

.0 31

− .2 23

− .0 67

1. 00 0

Se rv ic e Le ar n in g

.3 14

.1 65

− .3 21

.2 51

.1 80

− .2 68

− .3 25

.6 32

− .0 97

1. 00 0

C ap st on

e − .2 53

− .2 10

.0 78

.2 58

− .2 46

− .1 09

.0 47

− .2 00

− .2 00

− .2 92

1. 00 0

In te rn sh ip

− .2 26

− .1 43

.0 92

− .1 26

− .2 71

.0 45

.3 73

− .0 97

− .0 97

− .1 42

.0 97

1. 00 0

C om

p os it e Sc or e

− .7 40 *

− .8 17 *

.8 66 *

.0 09

.4 76

.5 33

.6 12

.0 56

.2 27

− .0 71

.1 70

.2 07

1. 00 0

C ar n eg ie

− .5 76

− .5 75

.5 06

− .2 52

.1 80

.3 65

.5 36

− .0 97

− .0 97

− .1 42

.0 97

− .1 42

.5 40

1. 00 0

* p < .0 1.

768 S. R. JOHNSON AND F. K. STAGE

Spearman correlations for the variables are displayed for the most- selective institutions in Table 4. Four-year graduation rate was significant and highly positively correlated with six-year graduation rate (0.866). Four-year graduation rate was significantly and negatively correlated with the freshman seminar (−0.598), learning community (−0.491), group work (−0.832), and the institution’s composite score (−0.740). Six-year gradua- tion rate was significantly and negatively correlated with the learning community (−0.574) and group work (−0.666); it was also highly nega- tively correlated with composite score (−0.817). The freshman seminar and composite score were significantly and positively correlated among the most-selective institutions. Spearman correlations for the moderately selec- tive institutions are displayed in Table 5. Four and six-year graduation rates were significantly correlated for the least-selective institutions at a high level (0.942).

Table 6 displays the Spearman correlations for the study’s least-selective institutions. A moderate but significantly positive correlation (0.525) was found between four and six-year graduation rates. The relationship between graduation rates was lower among the least-selective institutions than among the most selective and moderately selective institutions. Students attending nonselective institutions may face a greater number of external variables or “pulls” away from their academic pursuits than students attending selective schools (Cabrera et al., 1992).

At least-selective institutions, six-year graduation rate was significantly and highly positively correlated with student research (0.737). This relation- ship was the single highest positive relationship between a high-impact practice and a graduation rate across all institutions in this study and deserves further attention by the higher education community.

Regression analysis

Table 7 displays the results of the regression analysis for four-year graduation rate. Independent variables in the first model were limited to the 10 high- impact practices. The coefficients for the freshman seminar (−0.045), cohort (learning community; −0.059), and internship (−0.084) were all significant at p < .05. While the adjusted R-squared for the first model was low (0.116) and little explanatory power can be derived from it, it was surprising that the independent variables all had either a negative association or no association with the outcome.

The second model for the four-year graduation rate included the 10 high- impact practices as well as the institutional and student body characteristics. The coefficients for freshman seminar (−0.043) and internship (−0.052) remained negative and significant in this model, but cohort did not. Offering a freshman seminar was significant and negative at p < .01 as was

THE JOURNAL OF HIGHER EDUCATION 769

Ta b le

5. Sp ea rm

an co rr el at io n s fo r m od

er at el y se le ct iv e in st it ut io n s.

N = 75

Fo ur -Y ea r

G ra d ua ti on

Ra te

Si x- Ye ar

G ra d ua ti on

Ra te

Fr es h m an

Se m in ar

W ri ti n g

C or e

C ur ri cu lu m

Le ar n in g

C om

m un

it y

G ro up

W or k

St ud

en t

Re se ar ch

St ud

y A b ro ad

Se rv ic e

Le ar n in g

C ap st on

e In te rn sh ip

C om

p os it e

Sc or e

C ar n eg ie

Fo ur -Y ea r G ra d ua ti on

Ra te

1. 00 0

Si x- Ye ar

G ra d ua ti on

Ra te

.9 42 *

1. 00 0

Fr es h m an

Se m in ar

− .3 05

− .3 28

1. 00 0

W ri ti n g

− .2 06

− .1 88

− .0 47

1. 00 0

C or e C ur ri cu lu m

.0 56

.0 84

.1 00

.1 24

1. 00 0

Le ar n in g C om

m un

it y

− .0 48

− .0 76

.0 01

− .0 27

.0 92

1. 00 0

G ro up

W or k

.0 36

.0 52

− .0 64

.1 36

.3 41

.1 07

1. 00 0

St ud

en t Re se ar ch

.1 00

.0 77

.1 38

− .0 13

.0 14

− .0 71

.1 62

1. 00 0

St ud

y A b ro ad

.0 99

.1 32

− .1 85

.0 68

.1 53

− .0 06

.0 51

.0 59

1. 00 0

Se rv ic e Le ar n in g

− .0 57

.0 03

− .0 06

.1 60

.0 50

− .0 60

.0 07

.2 45

.1 78

1. 00 0

C ap st on

e .0 08

.0 04

.0 35

.2 31

.3 03

.2 61

.3 34

.1 23

.2 17

.1 24

1. 00 0

In te rn sh ip

− .1 16

− .0 89

− .0 46

.0 38

.3 75

− .0 06

.0 86

.0 06

.3 57

.1 43

.1 51

1. 00 0

C om

p os it e Sc or e

− .1 22

− .1 33

.3 82

.2 92

.5 83

.3 31

.4 56

.3 02

.2 25

.2 31

.5 79

.4 31

1. 00 0

C ar n eg ie

− .0 45

− .0 98

.0 92

.1 23

− .0 68

.1 86

.0 10

− .0 28

.0 03

.1 06

.1 02

.1 24

.1 86

1. 00 0

* p < .0 1.

770 S. R. JOHNSON AND F. K. STAGE

Ta b le

6. Sp ea rm

an co rr el at io n s fo r le as t- se le ct iv e in st it ut io n s.

N = 10

Fo ur -Y ea r

G ra d ua ti on

Ra te

Si x- Ye ar

G ra d ua ti on

Ra te

Fr es h m an

Se m in ar

W ri ti n g

C or e

C ur ri cu lu m

Le ar n in g

C om

m un

it y

G ro up

W or k

St ud

en t

Re se ar ch

St ud

y A b ro ad

Se rv ic e

Le ar n in g

C ap st on

e In te rn sh ip

C om

p os it e

Sc or e

C ar n eg ie

Fo ur -Y ea r G ra d ua ti on

Ra te

1. 00 0

Si x- Ye ar

G ra d ua ti on

Ra te

.5 25 *

1. 00 0

Fr es h m an

Se m in ar

.0 98

.2 51

1. 00 0

W ri ti n g

.4 19

.4 41

.4 26

1. 00 0

C or e C ur ri cu lu m

− .0 75

− .1 31

− .1 81

− .1 65

1. 00 0

Le ar n in g C om

m un

it y

.5 02

− .2 46

− .0 73

− .3 33

− .0 49

1. 00 0

G ro up

W or k

.1 24

.0 00

− .3 05

− .1 92

− .2 86

.4 61

1. 00 0

St ud

en t Re se ar ch

.2 21

.7 37 *

− .3 01

.1 32

− .2 02

− .3 18

.3 68

1. 00 0

St ud

y A b ro ad

− .2 39

.1 76

.4 26

− .1 11

− .1 65

− .3 33

− .1 92

.1 32

1. 00 0

Se rv ic e Le ar n in g

.0 00

.0 00

− .3 67

.0 00

.6 11

.0 00

.0 00

.0 00

− .7 45

1. 00 0

C ap st on

e − .0 06

− .1 62

− .2 56

.2 56

.6 45

− .2 69

− .4 55

− .2 37

− .3 20

.6 02

1. 00 0

In te rn sh ip

.4 70

.3 46

− .1 19

.2 18

.3 25

.2 18

.2 52

.1 30

− .5 09

.4 88

.2 52

1. 00 0

C om

p os it e Sc or e

.4 34

.3 07

.3 02

.4 74

.5 48

− .1 06

.0 00

.0 92

− .2 96

.5 17

.4 75

.7 37

1. 00 0

C ar n eg ie

.3 77

− .4 03

− .0 57

.3 68

− .1 05

.3 32

− .2 23

− .5 43

− .4 30

.0 00

.4 18

.0 80

.1 08

1. 00 0

* p < .0 1.

THE JOURNAL OF HIGHER EDUCATION 771

federal student aid (−0.291), private student aid (1.333), and selectivity (−0.088). Including institutional variables increased explained variance by more than 40%.

The third model for the four-year graduation rate included institutional and student body characteristics as well as institution composite score and interaction of selectivity with composite score. The coefficients for federal student aid (−0.290), private student aid (1.225), and selectivity (−0.195) remained significant at p < .01. Composite score had a smaller significant, negative association (−0.035), while the interaction between selectivity and composite score had no association with four-year graduation rate. Replacing the 10 high-impact practices with composite variables in Model 3 resulted in a slightly lower adjusted R-squared than in Model 2.

Table 8 displays the results for six-year graduation rate. The same variables were included in the four and six-year analyses. The coefficients for freshman seminar (−0.038), cohort (−0.065), and internship (−0.067) were all signifi- cant and negative at p < .05 in Model 1. When institutional variables were added in Model 2, the freshman seminar (−0.037) was the only significant high-impact practice coefficient. Internship was no longer significant, which may indicate that offering internships for academic credit could slow stu- dents from completing a degree within 4 years but not hinder them from completing a degree within 6 years. An equivalency test on the coefficients

Table 7. Ordinary least squares (OLS) regression: four-year college graduation rates. N = 101 (1) (2) (3)

Constant 0.600** 0.817** 1.179** Early Career Freshman Seminar −0.045* −0.043** Variables Cohort −0.059* −0.023

Core Curriculum 0.031 0.006 Writing −0.028 −0.011

Upper Classmen Internship −0.084* −0.052* Variables Research 0.020 0.021

Study Abroad 0.037 0.021 Group Work −0.030 −0.021 Service Learning −0.004 −0.004 Capstone 0.002 0.014

Institutional Expenditures per Pupil 0.0004a 0.0101a

Variables Prop. FedAid −0.291** −0.290** Prop. PrivAid 1.333** 1.225** Prop. White 0.101 0.130 Selectivity −0.088** −0.195** Composite Score −0.035* Selectivity*Composite 0.006 Adjusted R2 .116 .545 .530

* p < .05. ** p < .01. aCoefficient is in $1,000 units. Prop. FedAid = Proportion of students receiving federal financial aid; Prop. PrivAid = Proportion of students receiving private financial aid; Prop. White = Proportion of student body that identifies as White.

772 S. R. JOHNSON AND F. K. STAGE

for the freshman seminar in the four and six-year models showed that they were not significantly different from one another.1

Coefficients for proportion of students with federal aid (−0.238), proportion of students with private student aid (1.39), and institutional selectivity (−0.083) were also significant at p < .01 in Model 2. Coefficients for federal aid (−0.227), private aid (1.245), and selectivity (−0.129) remained significant in Model 3. For the first time in any of the models, the proportion of students who identified as White or Caucasian was significant (0.119). The coefficient for the composite score, although significant in the four-year model, was not significant in the six-year model.

Adding five institutional variables from Model 1 to Model 2 increased the adjusted R-squared for four-year graduation rate by 42.9% and for the six- year graduation rate by 53.5%. Holding all else constant, an institution that was somewhat competitive had predicted four and six-year graduation rates that were 8% lower than those of an institution that was most or highly competitive. Additionally, increasing the percentage of a student body receiv- ing federal financial aid by 10% decreased the predicted four-year graduation rate by 2.9% and decreased the six-year graduation rate by 2.3%, holding all else constant. The proportion of students receiving private loans to help pay their tuition was the single positive, significant independent variable in any of the models. Projecting from the model, institutions that experienced a 1% increase in the proportion of their student body receiving private loans

Table 8. Ordinary least squares (OLS) regression: six-year college graduation rates. N = 101 (1) (2) (3)

Constant 0.762** 0.937** 0.810** Early Career Freshman Seminar −0.038* −0.037** Variables Cohort −0.065* −0.028

Core Curriculum 0.029 0.005 Writing −0.026 −0.009

Upper Classmen Internship −0.067* −0.037 Variables Research 0.013 0.014

Study Abroad 0.053 0.038 Group Work −0.010 −0.001 Service Learning 0.002 0.001 Capstone 0.000 0.009

Institutional Expenditures per Pupil 0.0009a 0.0016a

Variables Prop. FedAid −0.238** −0.227** Prop. PrivAid 1.39** 1.245** Prop. White 0.077 0.119* Selectivity −0.083** −0.129* Composite Score −0.016 Selectivity*Composite 0.002 Adjusted R2 .136 .671 .622

* p < .05. ** p < .01. aCoefficient is in $1,000 units. Prop. FedAid = Proportion of students receiving federal financial aid; Prop. PrivAid = Proportion of students receiving private financial aid; Prop. White = Proportion of student body that identifies as White.

THE JOURNAL OF HIGHER EDUCATION 773

would have a predicted increase in both four and six-year graduation rates of 1.3%. This finding is consistent with previous research that demonstrated that students who take on private debt to pay for college are more likely to complete college on time compared with their classmates who choose not to take out private loans, possibly because of increased work hours and other financial pressures to pay education expenses (Bound, Lovenheim, & Turner, 2010; Dynarski, 1999; Perna, 2008).

Model 2 held the greatest explanatory power for both the four-year out- come (adjusted R2 = .545) and the six-year outcome (adjusted R2 = .671). Results might be understood by considering institutional tradeoffs, costs, pedagogical priorities, and the populations of students at large public institu- tions, the focus of this study.

Given existing literature linking academic preparation and socioeconomic characteristics to college completion, the significance of institutional variable coefficients was unsurprising. However, the inability of individual high- impact practices to mitigate associations between institutional and student body characteristics was surprising. Previous literature on student engage- ment and persistence, combined with the size of the current study’s sample and the diversity of participating institutions led us to anticipate positive relationships from the high-impact practices. The regression models in this study did not demonstrate such associations.

Limitations

There were limitations to this study. The quantitative analyses performed have not determined causation between variables that were examined by this research, nor can they be used to make inferences about practices and outcomes at all colleges and universities. The selection of institutional variables in this study (Carnegie Classification, expenditures per pupil, student race, and student financial aid) was guided by existing literature. However, no study that includes a limited population can address the institutional factors that might influence time to degree or an institution’s implementation of high-impact practices.

The study did not include a post-hoc analysis to correct for multiple testing, such as the Bonferroni correction.2 Secondary data sets, such as IPEDS, are not value-neutral; they are submitted directly by institutions, and therefore, the researcher cannot detect or correct errors (Smith, 2008). Although Internet-based surveys are useful for wide distribution and quick results, they may be vulnerable to response bias because the respondents answer questions independently. Low response rates from bachelor’s, associ- ate degree, and least-selective institutions limited the conclusions that could be drawn from this study on these types of schools alone. Nevertheless, this study makes a contribution to the body of research on high-impact practices

774 S. R. JOHNSON AND F. K. STAGE

and college completion and provides insights into various types of institu- tional curricular priorities.

Discussion

This study examined whether the incorporation of high-impact practices into college curricula was associated with higher four and six-year graduation rates at large public institutions. This research was guided by theories of student engagement and by existing literature on college persistence and completion. Bivariate correlation matrices illustrated moderate to highly negative relation- ships between graduation rates and freshman seminars, learning communities, and group work as well as with the high-impact practice composite scores at the most-selective institutions. These relationships were not found in the entire analytic sample nor were they found in the analysis of moderately and least-selective institutions. At the least-selective institutions, student research was highly predictive of six-year graduation rates. No significant relationships were found between practices and graduation rates across Carnegie Classifications or among the moderately selective institutions.

Multiple regression analysis demonstrated that 8 of the 10 high-impact practices had no significant relationship with either four-year or six-year graduation rates. Internships had a slightly negative relationship with four- year graduation rates and no relationship with six-year rates, suggesting that the internship might add time to ultimate completion. Freshman seminars had a slightly negative relationship with both graduation rates; however, an equivalency test of the coefficients showed that the association did not significantly differ across the four and six-year models. Three institutional variables—selectivity, proportion of the student body receiving private stu- dent loans, and the proportion of the student body receiving federal financial aid—were all negative predictors of four and six-year graduation rates. The proportion of the student body receiving private loans had the single sig- nificantly positive relationship with graduation rates in the regression mod- els. All other variable relationships were not significant.

These findings do not support our original hypothesis that 1st-year semi- nars, writing requirements, learning communities, and service learning would be positive predictors of graduation rates. Given the large body of literature focused on practices that students experience early in their college career, we were surprised to find negative relationships between graduation rates and two early-career variables. Benefits from high-impact practices likely experienced by individuals at our participating institutions were not evident in institutional graduation rates. However, associations between student demographic characteristics and enrollment demonstrated in pre- vious literature were predictors of graduation rates in this study as well.

THE JOURNAL OF HIGHER EDUCATION 775

The slightly negative correlation between internships and four-year graduation rate, without any correlation with six-year graduation rate, may reflect career-oriented, institutional programs designed for students to graduate in their fifth or sixth year. If that is the case, then the finding complements previous research that students who finish four years of college without graduating have an increased likelihood of dropping out (Nora et al., 2005). But late-stage attrition may not occur if the reason for extending time to degree is a formal academic program, such as a co-op model.

The negative association between freshman seminars and both the four-year and six-year models is puzzling, particularly because the negative correlation to graduation rates was significant only at the most-selective institutions. This finding could be a signal of institutional trade-offs—that institutions investing heavily in requiring freshman seminars for all students (a large cost with 19,000 enrolled students) are less likely to spend on further engagement or guidance practices later in a student’s career. Secondly, required freshman seminars could be a signal of a school’s generally more rigorous expectations, which may lead some students to delay graduation to later years.

Decades of research on student persistence and completion have pointed to academic and social engagement as keys to success in college (Bean, 1980; Hu & McCormick, 2012; Tinto, 1993). High-impact practices have been connected to increased student engagement (Kuh, 2008) and have been praised as ben- eficial to both traditional and nontraditional college student populations (Sandeen, 2012). The current study did not question whether participation in high-impact practices led to greater student engagement. However, results indicated that engagement experienced from these practices alone was not necessarily an indicator of likely college completion or shorter time to degree at large public institutions. Additionally, the results suggest that conclusions drawn from research on achievement and enrollment of individual students are not easily translated to broader institutional outcomes.

One important finding from this study is that the quantity of practices offered on campus, measured by our composite score, was not related to graduation rates. Institutions planning to add high-impact practices to their curricula should make intentional decisions about which practices fit well on their campus and would be most beneficial to their students instead of focusing on quantity of offerings. Administrators at less-selective or non- selective institutions might consider expanding opportunities for student engagement during the final years of enrollment through research programs. The finding that student research was highly associated with graduation rates at the least selective institutions complements previous studies that have shown student research was one of only two high-impact practices associated with broad-reaching liberal arts learning outcomes (Kilgo et al., 2015).

776 S. R. JOHNSON AND F. K. STAGE

Nonselective institutions may find that student research provides greater benefits compared with other high-impact practices.

Administrators should also consider whether students receive adequate advising and personal support outside of academic activities. Results sug- gested that curricular requirements are not enough to ensure that students complete their college degrees on time. While most-selective institutions likely have mechanisms for academic support, they also have populations of students who will seek them. Moderately selective and less-selective institutions should actively encourage students by complementing academic activities with additional support systems for timely degree completion. Institutions might be encouraged to compare student outcomes prior to and after implementing a high-impact practice to determine whether it impacts their unique student populations.

In the years since the AAC&U introduced high-impact practices, few studies have examined all 10 together, their relationships to one another, and their relationships with college outcomes. Further research on all the practices, with larger samples sizes, should be completed to sufficiently inform the higher education community of their benefits. This study revealed a missing link between academically engaging activities and college completion at large public institutions. Research is needed on college completion and administrative practices, such as advising models, campus resources, and required activities that are outside the curriculum, which could provide evidence of intermediary variables between academic engage- ment and timely graduation.

Conclusion

This quantitative study provides insight into the relationships among the AAC&U’s 10 high-impact practices, time to degree, graduation rates, and institutional typologies. Findings from this study are important because they add a layer of complexity to the body of research on college student persis- tence and engagement. While some research has linked individual practices to engagement and learning outcomes, findings from this study question whether those benefits can be directly linked to timely college completion. Results also indicated that the current consensus about benefits of institu- tional adoption of high-impact practices may be misinformed. Our findings are important for both researchers and practitioners in the higher education community because advocacy for these practices is widespread, they can be costly to implement, and our knowledge about their relationships to institu- tional and student outcomes is limited.

THE JOURNAL OF HIGHER EDUCATION 777

Notes

1 Results of equivalency test provided upon request. 2 This study was not causal, nor did it seek to draw inferences about inclusion of high-impact

practices at all types of institutions. While a correction was not included for the purposes of exploring all the relationships present in the study’s participating institutions, all but one significant correlation included in Tables 2 through 5 would have remained significant if a Bonferroni correction had been applied. Specifically, the relationship between student research and six-year graduation rate among nonselective institutions was no longer significant after using the Bonferroni correction. One reason may be the small sample size. Results of the Bonferroni correction will be provided upon request.

References

American Federation of Teachers. (2003). Student persistence in college: More than counting caps and gowns (Item Number 36-0702). Washington, DC: AFT Higher Education.

Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press.

Astin, A. W. (1984). Student involvement: A developmental theory for higher education. Journal of College Student Personnel, 25, 297–308.

Astin, A. W. (1993). What matters in college? Four critical years revisited. San Francisco, CA: Jossey-Bass.

Bangert-Drowns, R. L., Hurley, M. M., & Wilkinson, B. (2004). The effects of school-based writing-to-learn interventions on academic achievement: A meta-analysis. Review of Educational Research, 74, 29–58. doi:10.3102/00346543074001029

Barron’s Profile of American Colleges. (2015). Retrieved from http://www.barronspac.com Bean, J. P. (1980). Dropouts and turnover. The synthesis and test of a causal model of student

attrition. Research in Higher Education, 12, 155–187. doi:10.1007/BF00976194 Berry, H. A., & Chisolm, L. A. (1999). Service-learning in higher education around the world:

An initial look. New York, NY: Ford Foundation and International Partnership for Service- Learning.

Bill & Melinda Gates Foundation. (2010). Completion by design (Concept paper). Seattle, WA: Bill & Melinda Gates Foundation. Retrieved from http://completionbydesign.org/sites/ default/files/site-uploads/mainsite/pdf/cbd_concept_paper.pdf

Binder, J. F., Baguley, T., Crook, C., & Miller, F. (2015). The academic value of internships: Benefits across disciplines and student backgrounds. Contemporary Educational Psychology, 41, 73–82. doi:10.1016/j.cedpsych.2014.12.001

Bok, D. (2008). Our underachieving colleges. Princeton, NJ: Princeton University Press. Bound, J., Lovenheim, M., & Turner, S. (2007). Understanding the decrease in college

completion rates and the increased time to the baccalaureate degree (Research Report 07-626). Ann Arbor, MI: Population Studies Center, University of Michigan.

Bound, J., Lovenheim, M., & Turner, S. (2010). Increasing time to baccalaureate degree in the United States. (Working Paper No. 15892). Cambridge, MA: National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w15892

Bowen, W. G., Chingos, M. M., & McPherson, M. S. (2009). Crossing the finish line: Completing college at America’s public universities. Princeton, NJ: Princeton University Press.

Brooks, R., Benton-Kupper, J., & Slayton, D. (2004). Curricular aims: Assessment of a university capstone course. Journal of General Education, 53, 275–287. doi:10.1353/ jge.2005.0001

778 S. R. JOHNSON AND F. K. STAGE

Buch, K., & Spaulding, S. (2008). A longitudinal assessment of an initial cohort in a psychology learning community. Teaching of Psychology, 35, 189–193. doi:10.1177/ 009862830803500306

Cabrera, A. F., Castaneda, M. B., Nora, A., & Hengstler, D. (1992). The convergence between two theories of college persistence. The Journal of Higher Education, 63, 143–164. doi:10.1080/00221546.1992.11778347

Cabrera, A., Nora, A., Crissman, J., & Terenzini, P. (2002). Collaborative learning: Its impact on college students’ development and diversity. Journal of College Student Development, 43, 20–34.

Carey, K. (2004). A matter of degrees: Improving graduation rates in four-year colleges and universities. Washington, DC: Education Trust.

Chok, N. (2010). Pearson’s versus Spearman’s and Kendall’s correlation coefficients for con- tinuous data. Pittsburgh, PA: University of Pittsburgh. Retrieved from http://d-scholarship. pitt.edu/8056/1/Chokns_etd2010.pdf

Cisero, C. A. (2006). Does reflective journal writing improve course performance? College Teaching, 54, 231–236. doi:10.3200/CTCH.54.2.231-236

Collier, P. (2000). The effects of completing a capstone course on student identity. Sociology of Education, 73, 285–299. doi:10.2307/2673235

Corrigan, M. E. (2003). Beyond access: Persistence challenges and the diversity of low income students. In J. E. King, E. L. Anderson, & M. E. Corrigan (Eds.), New directions for higher education (pp. 25–34). San Francisco, CA: Jossey-Bass.

DesJardins, S., Kim, D., & Rzonca, C. (2003). A nested analysis of factors affecting bachelor’s degree completion. Journal of College Student Retention, 4, 407–435. doi:10.2190/BGMR- 3CH7-4K50-B5G3

Douglass, J. A., & Zhao, C. (2013). Undergraduate research engagement at major U.S. research universities (Occasional Paper Series CSHE.14.13). Berkeley, CA: Center for Studies in Higher Education.

Dynarski, S. M. (1999). Does aid matter? Measuring the effect of student aid on college attendance and completion (Working Paper No. 7422). Cambridge, MA: National Bureau of Economic Research.

Gedikoglu, H., & Parcell, J. (2013, August). Implications of survey sampling design for missing data imputation. Paper presented at the annual meeting of the Agriculture and Applied Economics Association, Washington, DC.

Gilbert, B., Banks, J., Houser, J., Rhodes, S., & Lees, D. (2014). Student development in an experiential learning program. Journal of College Student Development, 55, 707–713. doi:10.1353/csd.2014.0072

Goener, C., & Snaith, S. (2003). Predicting graduation rates: An analysis of student and institutional factors at doctoral universities. Journal of College Student Retention, 5, 409– 420. doi:10.2190/LKJX-CL3H-1AJ5-WVPE

Gomez, S., Lush, D., & Clements, M. (2004). Work placements enhance the academic performance of bioscience undergraduates. Journal of Vocational Education and Training, 56, 373–385. doi:10.1080/13636820400200260

Hatch, D. K. (2013). Student engagement and the design of high-impact practices at community colleges (Unpublished doctoral dissertation). The University of Texas, Austin.

Horton, P. B., Fronk, R. H., & Walton, R. W. (1985). The effect of writing assignments on achievement in college general chemistry. Journal of Research in Science Teaching, 22, 535– 541. doi:10.1002/(ISSN)1098-2736

Hu, S., & McCormick, A. C. (2012). An engagement-based student typology and its relation- ship to college outcomes. Research in Higher Education, 53, 738–754. doi:10.1007/s11162- 012-9254-7

THE JOURNAL OF HIGHER EDUCATION 779

Ingraham, E. C., & Peterson, D. L. (2004). Assessing the impact of study abroad on student learning at Michigan State University. Frontiers, 10, 83–100.

Janz, J. (2004). The retention impact of the freshman seminar on students with varying pre- college academic performance (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses Database. (UMI No. 3141095)

Kilgo, C. A., Sheets, J. K., & Pascarella, E. T. (2015). The link between high-impact practices and student learning: Some longitudinal evidence. Higher Education, 69, 509–525. doi:10.1007/s10734-014-9788-z

Kinzie, J. (2013). Taking stock of capstones and integrative learning. Peer Review, 15(4), 27–30. Knight, W. (1994). Why the five-year (or longer) bachelor’s degree? An exploratory study of

time to degree attainment. New Orleans, LA: Association for Institutional Research. Kuh, G. (2008). High-impact educational practices: What they are, who has access to them,

and why they matter. Washington, DC: American Association of Colleges and Universities.

Laal, M., & Ghodsi, S. (2012). Benefits of collaborative learning. Social and Behavioral Sciences, 31, 486–490.

Largent, L., & Horinek, J. B. (2008). Community colleges and adult service-learners: Evaluating a first-year program to improve implementation. New Directions for Adult and Continuing Education, 2008(118), 37–47. doi:10.1002/ace.294

Lei, S. A., Kuestermeyer, B. N., & Westmeyer, K. A. (2010). Group composition affecting student interaction and achievement: Instructors’ perspectives. Journal of Instructional Psychology, 37, 317–325.

Mansfield, R. (2011). The effect of placement experience upon final-year results for surveying degree programmes. Studies in Higher Education, 36, 939–952. doi:10.1080/ 03075079.2010.486073

Maynes, N., Hatt, B., & Wideman, R. (2013). Service learning as a practicum experience in a pre-service education program. Canadian Journal of Higher Education, 43(1), 80–99.

Metzger, C. A. (2006). Study abroad programming: A 21st century retention strategy? College Student Affairs Journal, 25, 164–175.

National Commission on the Future of Higher Education. (2006). A test of leadership: Charting the future of U.S. higher education. Washington, DC: U.S. Department of Education. Retrieved from http://www2.ed.gov/about/bdscomm/list/hiedfuture/reports/ final-report.pdf

Nora, A. (2003). A theoretical and practical view of student adjustment and academic achievement. In W. Tierney & L. Hagedorn (Eds.), Increasing access to college: Extending possibilities for all students. Albany: State University of New York Press.

Nora, A., Barlow, L., & Crisp, G. (2005). Student persistence and degree attainment beyond the first year of college. In A. Seidman (Ed.), College student retention: Formula for student success (pp. 129–153). Westport, CT: Praeger.

Obama, B. (2009, February 24). Remarks of President Barack Obama address to joint session of Congress. Washington, DC: Obama White House.

Oseguera, L., & Hwang, J. (2014). Using large data sets to study college education trajectories. New Directions for Institutional Research, 158, 37–50. doi:10.1002/ir.20044

Padgett, R. D., Keup, J. R., & Pascarella, E. T. (2013). The impact of first-year seminars on college students’ life-long learning orientations. Journal of Student Affairs Research and Practice, 50, 133–151. doi:10.1515/jsarp-2013-0011

Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco, CA: Jossey-Bass.

Perna, L. W. (2008). Understanding high school students’ willingness to borrow to pay college prices. Research in Higher Education, 49, 589–606. doi:10.1007/s11162-008-9095-6

780 S. R. JOHNSON AND F. K. STAGE

Perna, L. W., & Titus, M. A. (2007). Understanding differences in the choice of college attended: The role of state public policies. Review of Higher Education, 27, 501–525. doi:10.1353/rhe.2004.0020

Porter, E. (2014, February 27). Why college supply matters. The New York Times. Retrieved from https://economix.blogs.nytimes.com/2014/02/27/why-college-supply-matters

Porter, S. R., & Swing, R. L. (2006). Understanding how first-year seminars affect persistence. Research in Higher Education, 47, 89–109. doi:10.1007/s11162-005-8153-6

Potts, G., & Schultz, B. (2008). The freshman seminar and academic success of at-risk students. College Student Journal, 42, 647–658.

Prentice, M., & Robison, G. (2010). Improving student learning outcomes with service learning (Report AACC-RB-10-1). Washington, DC: American Association of Community Colleges.

Rubin, D. (1987). An overview of multiple imputation. Cambridge, MA: American Statistical Association. Retrieved from https://ww2.amstat.org/sections/srms/Proceedings/papers/ 1988_016.pdf

Runyan, L. M. (2011). Undergraduate degree completion: A study of time and efficiency to degree (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses Database. (UMI No. 3515893)

Salisbury, M. H. (2011). The effect of study abroad on intercultural competence among undergraduate college students (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses Database. (UMI No. 3461229)

Sandeen, C. (2012). High-impact educational practices: What we can learn from the tradi- tional undergraduate setting. Continuing Higher Education Review, 76, 81–89.

Shapiro, D., Dundar, A., Wakhungu, P. K., Yuan, X., Nathan, A., & Hwang, Y. (2015). Completing college: A national view of student attainment rates—fall 2009 cohort (Signature Report No. 10). Herndon, VA: National Student Clearinghouse Research Center.

Smith, E. (2008). Using secondary data in educational and social research. New York, NY: McGraw Hill.

Spady, W. (1971). Dropouts from higher education: Toward an empirical model. Interchange, 2(3), 38–62. doi:10.1007/BF02282469

Stage, F. K. (1989). Motivation, academic and social integration, and the early dropout. American Educational Research Journal, 26, 385–402. doi:10.3102/00028312026003385

Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). Chicago, IL: University of Chicago Press.

Tos, A. (2015). Relationship between service-learning and student success: A mixed method study at a California community college (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses Database. (UMI No. 3701036)

U.S. Department of Education. (2014). 2012–2013 federal Pell Grant Program end-of-year report. Washington, DC: U.S. Department of Education. Retrieved from http://www2.ed. gov/finaid/prof/resources/data/pell-2012-13/pell-eoy-2012-13.html

U.S. Department of Education. (2015). Table 326.10. Graduation rates of first-time, full-time bachelor’s degree-seeking students at 4-year postsecondary institutions. Washington, DC: U. S. Department of Education. Retrieved from https://nces.ed.gov/programs/digest/d15/ tables/dt15_326.10.asp

Wolniak, G., & Engberg, M. (2015). The influence of ‘high-impact’ college experiences on early career outcomes. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.

THE JOURNAL OF HIGHER EDUCATION 781

Copyright of Journal of Higher Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.