Business Finance - Accounting APA assignment
Fire 2
EJ1403590.pdf
New articles in this journal are licensed under a Creative Commons Attribution 4.0 United States License.
This journal is published by Pitt Open Library Publishing.
34
This journal is supported by the Carnegie Project on the Education Doctorate: A Knowledge Forum on the EdD (CPED) cpedinitiative.org
impactinged.pitt.edu ISSN 2472-5889 (online) Vol. 8 No. 4 (2023) DOI 10.5195/ie.2023.296
Assessing Online Doctoral Student Research Competencies
John C. Gillham University of Findlay [email protected]
Nicole V. Schilling University of Findlay
ABSTRACT
The purpose of this study was to investigate online education doctoral students’ perceptions of their research
competencies. The researchers utilized the Scholar-Practitioner Research Development Scale (Rockinson-
Szapkiw, 2018) which consisted of 24 Likert-based items to assess research competency in five areas: attitude
toward or value of research, critical evaluation and application of research, research knowledge, research skills,
and research dissemination. For each of the five research competencies students reported their competency at
4.0 or above (on a scale of 1- 5) each year, with students most strongly agreeing with statements related to the
value of research, evaluation and application skills, and research knowledge. This study illuminates the
development of research competencies in online doctoral students. Recommendations include using the
Scholar-Practitioner Research Scale to assess program effectiveness, track program improvements, and
identify gaps in the curriculum.
KEYWORDS
EdD student development, Scholar-Practitioner Research Development Scale (SPRDS), research competencies, Carnegie Project on the Education Doctorate (CPED)
In 2006, Schulman et al. noted that “The problems of the
education doctorates [Ed.D.] are chronic and crippling. The purposes
of preparing scholars and practitioners are confused; as a result,
neither is done well” (p. 25). Schulman et al. (2006) were part of a
chorus of scholars (Andrews & Grogan, 2005; Archbald, 2008;
Deering, 1998; Grogan & Andrews, 2002; Levine, 2005; Malen &
Prestine, 2005; Murphy & Vriesenga, 2005; Osguthorpe & Wong,
1993; Toma, 2002; Townsend, 2002) who criticized Academia for
failing to distinguish the Ed.D. degree from the Ph.D. degree and
called for reform or redesign of the Ed.D. as a professional degree to
prepare educators for practice in the field. It was for these purposes
the Carnegie Foundation for the Advancement of Teaching initiated
the Carnegie Project on the Education Doctorate (CPED) in 2007
(Perry et al., 2015). Over time, the traditional Ph.D. model was
modified to meet the unique needs of the Ed.D. student, leading to
the concept of the scholar–practitioner (also described in the
literature as the scholarly practitioner or the practitioner-scholar).
LITERATURE REVIEW
Scholarship conducted since Shulman and colleagues’ (2006)
challenge provides a more robust conception of the role and
development of scholar-practitioners. Jenlink (2014) envisioned
scholar-practitioners as educational leaders who engage in research
as practice, view knowledge and practice as one, are situated in
critically oriented inquiry and noted “scholar–practitioner leadership
is grounded in a postmodern—post-positivist inquiry of leadership,
which seeks to blur boundaries in the knowledge-practice and
inquiry-practice relationship” (p. 8). Hochbein and Perry (2013)
stated that scholar-practitioners will utilize three habits of inquiry—
decipher, debate, and design—to solve their problems of practice. A
central focus of these habits is to utilize the body of research in the
field and apply it to a real-world context. Murakami-Ramalho et al.
(2013) propose scholarly practitioners need to understand and use
research to set goals, implement research design, and measure
growth. Developing research competencies begins with critically
consuming the literature (Slayton & Samkian, 2017), progressing to
learning research methods, and culminates in the defense and
publication of the dissertation (Baker & Pifer, 2014). Golde (2013)
conceptualizes the development of scholar-practitioners as occurring
in three stages: Entry and Integration, Integrating New Knowledge,
and Completion and Exit. Focusing on impact, one scholar notes that
“scholar–practitioners conjoin the strategies and knowledge gained
through meticulous academic endeavors with experiences and
knowledge...to form the basis of effective, change-centered practices”
(Bouck, 2011, p. 203). Rockinson-Szapkiw (2018) posited scholar-
practitioners should:
1. Develop a value of research, including an
understanding and appreciation of research as a means
to solve problems of practice and to advocate for social
Assessing Student Research Competencies
Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 8 No. 4 (2023) DOI 10.5195/ie.2023.296 35
justice and equity;
2. Obtain skills and knowledge to critically consume (e.g.,
information literacy), apply, and conduct research to
inform practice; and
3. Gain the ability to disseminate research to
professionals in the field to transform practice. (p. 2)
Therefore, the primary objectives of a scholar-practitioner are to
value, critically evaluate, and apply the necessary knowledge and
skills to disseminate research to solve problems in the field. At its
heart, “the scholar-practitioner’s goal is to bridge research, theory,
and practice” (Suss, 2015, p. 50).
As Ed.D. students begin to develop as scholar-practitioners,
they become aware of problems of practice and approach these
problems through the application of rigorous research methods. They
generate, analyze, and disseminate data among stakeholders to
produce change. Lasater et al. (2016) conducted a case study of the
perceptions of students and alumni from the CPED affiliated Ed.D.
program at University of Arkansas. Specifically, the researchers
sought to answer three questions:
1. How do current students and alumni describe their
experiences in the Educational Leadership online doctoral
program at University of Arkansas?
2. How do current students and alumni perceive the online
experience as impacting their professional practice?
3. How do current students and alumni describe the value of
the Educational Leadership program? (Methods, para. 1).
Utilizing a qualitative approach, the researchers distributed surveys
and conducted semi-structured interviews with students who were
enrolled in the program between 2010 and 2014. First among the
three findings, students perceived “the doctoral program led to a
change in their thinking” (Lasater et al., 2016, Findings, para. 1). One
participant reported their experiences changed the way they looked
at problems and how they approached problem-solving. They noted:
I think about “what does the literature say?” Those are
questions that are popping up in my head that didn’t pop up in
my head before. I definitely can read those stats parts of the
findings, when you read an article, now I am like, “oh, I think I
know what that means” (Doctoral Program Led to a Change in
Student Thinking, para. 2).
Another participant shared they have become more critical of data
distributed in the workplace and need proof that data is valid and
meaningful. These findings reflect graduates who value, critically
consume, and apply research to their professional context,
characteristics of scholar-practitioners. Other findings included
students valued the program design and the relationships they built
with faculty and their peers in the program (Lasater et al., 2016).
Perry and Zambo (2019) examined data gather from CPED-
influenced programs to investigate whether the CPED framework
was being used to develop the research capabilities of students. The
researchers specifically examined Inquiry as Practice, a CPED
design concept for scholar practitioner research development. The
examination was part of a broader study that sought to assess the
impact of the CPED framework upon member institutions’ doctoral
programs. The study investigated teaching inquiry to improve
practice in CPED-influenced programs, how students are socialized
in this method, the methodologies used by these programs, and the
skills and abilities that resulted from being taught inquiry.
For this qualitative study, online, open-ended surveys were sent
to individuals who were the primary contacts at CPED-influenced
programs. Fifty-three (60%) member institutions responded.
Participants were asked to provide student and faculty demographics,
to describe how they enacted the CPED framework, and to provide
syllabi to illustrate how the framework was enacted at their
institutions. A two-level analysis consisting of generating categories
and subcategories of elements of the CPED framework that were
enacted was used to analyze open-ended responses. Syllabi were
analyzed using content analysis until reaching saturation and
providing evidence for the findings. Data gathered to answer the
research questions allowed Perry and Zambo (2019) to better
understand how participating institutions “understand inquiry as a
form of practice for professional practitioners, how inquiry is taught,
and what graduates are expected to be able to know and do as a
result of having completed one of these programs” (p. 10).
Findings indicated participating CPED-influenced programs put
problems in the professional context at the center of doctoral and
research training. The practice of inquiry and reliance on the
research were understood to be essential skills for educational
leaders. Programs also placed an emphasis on the socialization of
students into their roles as scholar-practitioners. A focus on inquiry
was seen as an integral part of being a scholar-practitioner and was
present across the curriculum, in field experiences and in the
dissertation process. Participating programs utilized various research
frameworks and methodologies, with learning objectives centered on
the dissertation. Research methodologies tended to permeate the
curricula and increase in sophistication as students progressed
through their programs. Participating CPED-influenced programs
also utilized inquiry to develop scholar-practitioners who can do the
following:
think systematically, creatively, and reflectively; to blend their
own practical knowledge with the theories and ideas learned in
their program; to use literature to deeply understand problems
of practice; to analyze and critique policies, theory, and
research and to wisely consume literature and apply it to
develop solutions. (Perry & Zambo, 2019, p. 18)
Finally, participating CPED-influenced programs emphasized the
“creation and transformation of professional knowledge and practice”
(Perry & Zambo, 2019, p. 18), one of CPED’s Six Guiding Principles.
CPED-influenced programs foster students’ identities as scholar-
practitioners and students are inculcated with the belief that ongoing
professional learning must take place to renew professional practice
and programs.
Because many education doctoral students lack experience
conducting research, they may not feel competent in research or
developed an identity as a researcher (Caskey et al., 2020), and
scholars note developing an identity as a researcher for doctoral
students is a complicated problem (Labaree, 2003; Murakami-
Ramalho et al., 2013; Shulman et al., 2006). In their review of the
literature, Choi et al. (2019) investigated how doctoral students (both
Ph.D. and Ed.D.) develop their identities as scholars. The study was
built upon Gee’s (2000) theory of identity and Engeström’s (1987,
2001) cultural-historical activity theory. The researchers defined a
scholar’s identity as “as an individual’s felt or recognized abilities
allowing association with communities doing scholarship pertaining
to an academic discipline” (Choi et al., 2019, pp. 7-8). Choi et al.
(2019) sought to understand the conceptualizations of researcher
identity relevant to doctoral students, determine how doctoral
Gillham & Schilling
Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 8 No. 4 (2023) DOI 10.5195/ie.2023.296 36
students’ identities as scholars evolve, and determine the practical
implications for stakeholders based on the literature of how to
cultivate doctoral students’ identities as scholars.
After unstructured and structured phases of literature
identification, 62 potential research studies were identified. An
additional round of reviewing the exclusion/inclusion criteria resulted
in 36 empirical studies suitable for review. Independent coding by the
authors and subsequent discussions resulted in cogent codes and
findings.
The literature investigated in the Choi et al. (2019) study
revealed how doctoral students begin to see themselves as scholars
and become recognized as scholars by others. Multiple methods of
attaining scholarly identity were noted, including engaging with other
doctoral scholars, seeking and receiving feedback from the faculty,
engaging with undergraduate students and earning their respect,
garnering respect for their work from non-academics, participating in
affinity groups, and attending academic conferences. Student
identities as scholars increased as faculty shared their vulnerabilities
and students reflected on their research journey, noting the
challenges associated with producing research.
Choi et al. (2019) also noted how doctoral students developed a
sense of competence as scholar-practitioners. Choi et al. (2019)
identified 27 articles from the literature that indicated competence
and confidence often go hand-in-hand. As students develop research
competence, they also develop confidence and become more
invested in their identities as scholars. One method to promote
competence was participation in student-created writing groups for
faculty and peers.
An additional layer of challenge in developing scholar
practitioner research competencies is the increasing reliance on
online education as a medium for teaching the value and application
of research, as with the program in this study. For most students, the
dissertation represents the pinnacle of the research experience
during their preparation as scholar-practitioners and requires multiple
research competencies. Scholars have investigated what
dissertation chairs can do to support doctoral (Ed.D.) students as
they craft their dissertations in an online learning environment. In
their qualitative study, Burrington et al. (2020) conducted semi-
structured interviews with dissertation chairs they contacted through
social media. The six female and five male participants had a range
of 1.5 to 16 years’ experience working with dissertation students in
an online environment, with a mean of 5.5 years. Feedback was a
frequent theme in the findings, including the frequency, modes,
timelines, and effectiveness and relevance of feedback. Burrington et
al. (2020) noted “the importance of providing frequent feedback
through various modes of communication, emphasizing a tailored
approach to the students’ needs. Timely, thorough feedback was
supported, stressing effectiveness and relevancy, which was most
commonly achieved through one-on-one communication” (p. 1). The
researchers also noted the importance of the dissertation chair’s
expectations for doctoral scholarship; scholar-practitioners must
learn to represent themselves “in a scholarly voice, learning to
conduct original research, and learning to tell the story of that
research in a way that is consistent with a field of scholarship and
practice, as well as with the research methodology chosen”
(Burrington et al., 2020, pp. 7-8). The researchers found other
factors in effectively supporting online dissertation students,
including building a caring, trusting relationship, providing
individualized guidance, and balancing the needs of the student with
university’s requirements.
To assess the student’s development, the research competency
of the scholar-practitioner must be clearly examined using diverse
criteria (Perry, 2015; Rockinson-Szapkiw, 2018; Rolfe & Davies,
2009; Servage, 2009; Shulman et. al., 2006). The program in this
study sought to understand the development of its students’ research
competencies as a means of assessing its ability to produce scholar-
practitioners. To prepare students for their role as scholar-
practitioners, CPED-influenced and other doctoral programs need a
comprehensive, valid assessment of student growth to assess
program effectiveness and track program improvements. Such an
instrument might also serve as a guidepost for program faculty,
revealing areas of strength as well as areas of need in the curriculum.
PURPOSE AND CONTEXT OF THE STUDY
This study sought to assess the impact of a Doctor of Education
(Ed.D.) program on students’ perceptions of their ability to effectively
use research in the field. The purpose of this study was to explore
how online doctoral students, who are predominantly educational
leaders, perceive the way they value, understand, and can use
research. The program in this study prepares students to become
scholar-practitioners who solve problems of practice using research-
based strategies. Students apply the skills gained in their research
courses to their dissertations and professional context over a period
of three or more years. Across the curriculum, students are required
to use the literature to solve problems in their professional practice.
Graduates of the program take leadership roles in public education,
assume responsibility for the school districts in which they apply their
research skills, and serve as first responders in the gap between
scholarship and systemic change.
The setting for this study is a private residential liberal arts
institution located in the mid-western United States. Founded in 1882,
the institution offers 86 Bachelor’s degrees, 10 Master’s degrees,
five Doctoral degrees, and a variety of non-degree license and
certificate programs. Approximately one-third of the institution’s
4,829 students are enrolled in graduate programs, both online and
face-to-face. Doctor of Education degree requirements at this
institution include 60 course credits composed of core courses (24
credits), research and dissertation courses (21 credits), and electives
(15 credits). Students may choose from two elective strands,
Teaching and Learning or Administration. The Administrative strand
includes courses required by the state for a superintendent’s license.
Students may complete the program in as little as three years, with
no penalty for taking fewer courses each semester and extending
their timelines. To satisfy the residency requirement, students attend
three face-to-face Summer Institutes, each lasting three days. The
Summer Institute is an opportunity for students to present their
research, give and receive feedback to and from their peers, and
attend research training workshops.
A signature pedagogy of the program is the embedded
dissertation. Students begin working on the dissertation their first
semester in the program and continue to craft it as they progress
through the program. Several of the research courses are associated
with specific chapters of the dissertation so students apply what they
learn as they write the chapter in that course.
As a member of CPED, the program supports and promotes the
CPED Framework and the Guiding Principles for Program Design.
Assessing Student Research Competencies
Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 8 No. 4 (2023) DOI 10.5195/ie.2023.296 37
The expectation is for students to develop as scholar-practitioners
who will apply research to problems of practice (PoP) in their
professional contexts. CPED (2021) posits that:
Scholarly Practitioners blend practical wisdom with professional
skills and knowledge to name, frame, and solve problems of
practice. They use practical research and applied theories as
tools for change because they understand the importance of
equity and social justice. They disseminate their work in multiple
ways, and they have an obligation to resolve problems of
practice by collaborating with key stakeholders, including the
university, the educational institution, the community, and
individuals (Design-Concepts Upon Which to Build Programs,
para. 2).
Each core course in the program contains a key assessment that,
taken together, compose the comprehensive exams. For each key
assessment, students are required to analyze a PoP in their
professional context relative to the subject of the course (e.g., policy,
leadership, diversity, etc.), investigate the literature on their PoP,
then create a plan to solve the PoP using the research they have
identified.
Students in this program are primarily individuals working full
time in P12 education (teachers, principals, and superintendents),
with some students from other fields such as higher education,
health professions, and business. Students have an average age of
approximately 39.7 years, with an age range of 22 to 70 years.
Participants across the four years of the study ranged from one to
five years of involvement in the Doctor of Education program.
Offered entirely online (aside from the three-day Summer
Institutes), this program is representative of a larger trend in
education. Online graduate programs have witnessed consistent
growth since the year 2000. At that time, 2.2 million students were
enrolled in online graduate programs, a figure which increased to 3.1
million students by 2019 (NCES, 2021). Programs such as these
require doctoral students to develop research competencies in a
virtual context, thus presenting an addition layer of challenge to both
faculty and students. The informal assessments faculty might make
of research competencies in regularly scheduled face-to-face
settings are not possible in online, asynchronous courses such as
the one utilized in this program. In the same way, students’ self-
assessment of research competencies are done in isolation without
face-to-face context with peers and faculty, making the need for a
specific, objective assessment of research competencies even more
pressing.
Similar challenges may be prevalent among CPED member
institutions’ Doctor of Education programs, as 25% are offered
entirely online (CPED, 2020). While online enrollment in
undergraduate programs decreased during the COVID-19 pandemic,
graduate programs grew at higher-than-expected rates (Quality
Matters & Eduventures Research, 2021). Nearly 30% of graduate
students enrolled in private, not-for-profit institutions like the one in
this study are enrolled in exclusively online programs (NCES, 2020).
One study of Chief Online Officers at private universities found that
88% expect online enrollment to continue to increase (Quality
Matters & Eduventures Research, 2021).
The study was guided by the following research question: How
do scholar-practitioners perceive their research competency (value
of research, evaluation and application skills, research knowledge,
research skills, and research dissemination)?
RESEARCH METHODS
The researchers, who are online Doctor of Education program
faculty, in this study sought to explore students’ perceptions of their
research competency as scholar-practitioners through dissemination
of the Scholar-Practitioner Research Development Scale
(Rockinson-Szapkiw, 2018). The survey consisted of 24 Likert-based
items (strongly agree - 5, agree - 4, neutral - 3, disagree - 2, and
strongly disagree - 1) developed to access research competency in
five areas: value of research (six items), evaluation and application
skills (three items), research knowledge (six items), research skills
(six items), and research dissemination (three items). For example,
in respect to attitude or value of research, participants were asked to
indicate their level of agreement with each statement:
1. Acquiring research knowledge and skills during my program
is important.
2. Research can improve the lives of those served in my
professional practice.
3. Research can improve my professional practice.
4. Research is useful to solve complex problems I face in my
professional practice.
5. Research is important to promote equity and social justice in
my professional practice.
6. Disseminating my research to various audiences is important
to improve professional practice.
Rockinson-Szapkiw (2018) reported in her development of the
Scholar-Practitioner Research Development Scale that “evidence
from both an exploratory factor analysis and internal consistency
analysis demonstrate that the self-report scale has both validity and
reliability” (p. 20). The Cronbach’s alpha was .93 and each subscale
demonstrated good internal consistency (value of research = .88,
evaluation and application skills = .82, research knowledge = .85,
research skills = .90, and research dissemination = .93).
A link to the online survey was distributed electronically to all
Ed.D. students during the summer for four consecutive years (2018 -
2021). The primary use of the collected data was for program
evaluation and program improvement. In addition to the questions on
the Scholar Practitioner scale, four demographic questions were
asked: (1) years in the Ed.D. program, (2) gender, (3) professional
role, and (4) the number of years in their professional role. No
identifying data was collected, which protected student confidentiality
and encouraged truthful responses. However, due to the lack of
identifiable data, it was not feasible to track individual students or
cohorts longitudinally.
For each iteration of the survey, the data was downloaded into
an Excel spreadsheet. Means and standard deviations were
calculated for each of the 24 items and for the five constructs for
each of the four years.
RESULTS
A total of 129 students participated in the study over a four-year
period. A large majority of participants reported they worked in P12
education as teachers or administrators. Other participants worked in
fields such as higher education, health care, and business. Other
participant demographics can be found in Table 1.
Gillham & Schilling
Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 8 No. 4 (2023) DOI 10.5195/ie.2023.296 38
Table 1. Participant Demographics
Study
Year
Participants
(n)
Year in the
Program (%)
Gender Average Years in
Professional Role
2018 33 1 (28)
2 (34)
3 (22)
4 (16)
Female: 64%
Male: 36%
9.7
2019 33 1 (13)
2 (41)
3 (31)
4 (13)
5 (3)
Female: 61%
Male: 39%
10.3
2020 28 1 (18)
2 (36)
3 (32)
4 (14)
Female: 61%
Male: 39%
11.2
2021 35 1 (26)
2 (23)
3 (37)
4 (11)
5 (3)
Female: 54%
Male: 46%
10.4
Note. Percentages are rounded and may not equal 100.
To answer the research question, an initial descriptive analysis
of the data was conducted. Due to the lack of identifiable data, it was
not feasible to track individual students or cohorts longitudinally,
regardless patterns in the data emerged. Across all four years of the
survey, participants perceived their competency highest in value of
research, evaluation and application skills, and research knowledge,
respectively.
Value of Research
Participants perceived their highest competency in the value of
research construct in each of the four years of the study (M = 4.59,
4.65, 4.60, and 4.57 respectively). Within the value of research
construct, in the first year of the study, participants perceived their
competency to be equally highest (M = 4.71) for the items “Acquiring
research knowledge and skills during my program is important” and
“Research can improve the lives of those served in my professional
practice.” The item which participants reported their lowest perceived
competency (M = 4.49) was “Disseminating my research to various
audiences is important to improve professional practice.” In the
second year of the study, participants also perceived their
competency to be highest (M = 4.79) for the item “Acquiring research
knowledge and skills during my program is important.” The item
which participants reported their lowest perceived competency (M =
4.52) was “Research is important to promote equity and social justice
in my professional practice.” In the third year of the study,
participants perceived their competency to be highest (M = 4.79) for
the item “Acquiring research knowledge and skills during my
program is important.” The item which participants reported their
lowest perceived competency (M = 4.36) was “Disseminating my
research to various audiences is important to improve professional
practice.” In the fourth year of the study, participants perceived their
competency to be highest (M = 4.89) for the item “Acquiring research
knowledge and skills during my program is important.” The item
which participants reported their lowest perceived competency (M =
4.34) was “Research is important to promote equity and social justice
in my professional practice” (See Table 2).
Table 2. Participant Perceived Competency in Value of Research
Survey Item 2018 2019 2020 2021
M SD M SD M SD M SD
Acquiring research
knowledge and skills
during my program is
important.
4.71 0.46 4.79 0.48 4.79 0.42 4.89 0.32
Research can improve
the lives of those served
in my professional
practice.
4.71 0.57 4.70 0.53 4.71 0.46 4.54 0.70
Research can improve
my professional
practice.
4.63 0.55 4.76 0.44 4.75 0.52 4.69 0.58
Research is useful to
solve complex problems
I face in my professional
practice.
4.57 0.50 4.58 0.56 4.61 0.63 4.54 0.74
Research is important
to promote equity and
social justice in my
professional practice.
4.53 0.61 4.52 0.62 4.39 0.69 4.34 0.84
Disseminating my
research to various
audiences is important
to improve professional
practice.
4.49 0.70 4.55 0.62 4.36 0.73 4.43 0.65
Evaluation and Application Skills
Within the evaluation and application skills construct,
participants reported their highest perceived competency for the item
“I can identify scholarly resources to solve problems I encounter in
my professional practice” across all four years of the study (M = 4.65,
4.79, 4.61, and 4.71 respectively). In the first three years of the study,
participants reported their lowest perceived competency for the item
“I can apply theory to solve problems I encounter in my professional
practice” (M = 4.15, 4.36, and 4.29 respectively). In the fourth year of
the study, participants reported their lowest perceived competency
(M = 4.31) for the item “I can apply empirical research to solve
problems I encounter in my professional practice” (See Table 3).
Research Knowledge
Within the research knowledge construct, participants reported their
highest perceived competency for the item “I understand ethical
guidelines for research in my profession (e.g., obtain IRB approval,
do not harm participants)” across all four years of the study (M = 4.74,
4.81, 4.75, and 4.60 respectively). In the first two years of the study,
participants reported their lowest perceived competency for the item
“I understand analytic procedures to analyze data collected in my
professional practice” (M = 3.94 and 4.14 respectively). In the third
year of the study, participants reported their lowest perceived
competency (M = 4.04) for the item “I understand how theories and
paradigms are used to develop investigations to solve problems in
my professional practice.” In the fourth year of the study, participants
reported their lowest perceived competency (M = 4.20) for the item “I
understand how to engage in the research process, from
conceptualization to dissemination (e.g., communication to key
stakeholders), to address problems in my professional practice” (See
Table 4).
Assessing Student Research Competencies
Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 8 No. 4 (2023) DOI 10.5195/ie.2023.296 39
Table 3. Participant Perceived Competency in Evaluation and Application Skills
Survey Item 2018 2019 2020 2021
M SD M SD M SD M SD
I can apply empirical research to solve problems I encounter in my professional practice.
4.32 0.64 4.55 0.56 4.36 0.68 4.31 0.58
I can apply theory to solve problems I encounter in my professional practice.
4.15 0.74 4.36 0.74 4.29 0.66 4.37 0.60
I can identify scholarly resources to solve problems I encounter in my professional practice. 4.65 0.60 4.79 0.42 4.61 0.69 4.71 0.46
Table 4. Participant Perceived Competency in Research Knowledge
Survey Item 2018 2019 2020 2021
M SD M SD M SD M SD
I understand ethical guidelines for research in my profession (e.g., obtain IRB approval, do not
harm participants).
4.74 0.51 4.81 0.40 4.75 0.44 4.60 0.74
I understand how to formulate questions to investigate problems in my professional practice.
4.32 0.68 4.56 0.67 4.29 0.76 4.54 0.51
I understand research methods (e.g., quantitative, qualitative, and mixed) I can use to investigate
problems in my professional practice.
4.41 0.66 4.63 0.49 4.43 0.69 4.43 0.74
I understand analytic procedures to analyze data collected in my professional practice.
3.94 0.89 4.14 0.87 4.18 0.86 4.23 0.73
I understand how theories and paradigms are used to develop investigations to solve problems
in my professional practice.
4.00 0.92 4.39 0.72 4.04 0.84 4.31 0.72
I understand how to engage in the research process, from conceptualization to dissemination
(e.g., communication to key stakeholders), to address problems in my professional practice.
4.29 0.80 4.37 0.62 4.18 0.90 4.20 0.93
Table 5. Participant Perceived Competency in Research Skills
Survey Item 2018 2019 2020 2021
M SD M SD M SD M SD
I can design meaningful research investigations to address problems in my professional practice.
4.06 0.83 4.38 0.79 4.04 0.88 4.29 0.67
I can choose the appropriate method of inquiry (e.g., quantitative, qualitative, and mixed) to
address problems in my professional practice.
4.15 0.83 4.31 0.69 4.25 0.80 4.37 0.73
I can conduct rigorous research investigations to address problems in my professional practice.
4.12 0.89 4.22 0.83 4.07 0.86 4.03 0.92
I can interpret results from the data I analyze.
4.15 0.67 4.19 0.83 4.21 0.74 4.09 0.90
I can analyze data (e.g., quantitative, qualitative, and mixed) that I collect to address problems in
my professional practice.
4.06 0.86 4.22 0.79 4.18 0.77 3.94 0.91
I can develop investigation questions to examine problems in my professional practice. 4.33 0.60 4.39 0.76 4.32 0.67 4.40 0.65
Table 6. Participant Perceived Competency in Research Dissemination
Survey Item 2018 2019 2020 2021
M SD M SD M SD M SD
I can communicate (e.g., present, write) the results of research investigations I conduct to key
stakeholders.
4.18 0.85 4.44 0.72 4.25 0.70 4.17 0.86
I can discuss the results of research investigations in light of empirical and theoretical literature,
drawing connections between the practice and the knowledge of the profession.
4.12 0.89 4.31 0.74 4.00 0.90 4.17 0.82
I can communicate implications to improve practice based on the results of research
investigations I conduct.
4.27 0.72 4.38 0.79 4.14 0.85 4.29 0.83
Gillham & Schilling
Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 8 No. 4 (2023) DOI 10.5195/ie.2023.296 40
Research Skills
Within the research skills construct, in the first year of the study
participants perceived their competency to be equally highest (M =
4.15) for the items “I can choose the appropriate method of inquiry
(e.g., quantitative, qualitative, and mixed) to address problems in my
professional practice” and “I can interpret results from the data I
analyze.” In that same year, participants perceived their competency
to be equally lowest (M = 4.06) for the items “I can design meaningful
research investigations to address problems in my professional
practice” and “I can analyze data (e.g., quantitative, qualitative, and
mixed) that I collect to address problems in my professional practice.”
In the second, third, and fourth year of the study, participants
perceived their competency to be highest for the item “I can develop
investigation questions to examine problems in my professional
practice” (M = 4.39, 4.32, and 4.40 respectively). In the second year
of the study, the item which participants reported as their lowest
perceived competency (M = 4.19) was “I can interpret results from
the data I analyze.” In the third year of the study, participants
perceived their competency to be highest (M = 4.04) for the item “I
can design meaningful research investigations to address problems
in my professional practice.” In the fourth year of the study,
participants perceived their competency to be highest (M = 3.94) for
the item “I can analyze data (e.g., quantitative, qualitative, and mixed)
that I collect to address problems in my professional practice” (See
Table 5).
Research Dissemination
Within the research dissemination construct, participants
reported their lowest perceived competency for the item “I can
discuss the results of research investigations in light of empirical and
theoretical literature, drawing connections between the practice and
the knowledge of the profession” across all four years of the study (M
= 4.12, 4.31, 4.00, and 4.17 respectively). In the first year of the
study, participants perceived their competency to be highest (M =
4.27) for the item “I can communicate implications to improve
practice based on the results of research investigations I conduct.” In
the second year of the study, participants perceived their
competency to be highest (M = 4.44) for the item “I can communicate
(e.g., present, write) the results of research investigations I conduct
to key stakeholders.” In the third year of the study, participants
perceived their competency to be highest (M = 4.25) for the item “I
can communicate (e.g., present, write) the results of research
investigations I conduct to key stakeholders.” In the fourth year of the
study, participants perceived their competency to be highest (M =
4.29) for the item “I can communicate implications to improve
practice based on the results of research investigations I conduct”
(See Table 6).
CONCLUSIONS AND DISCUSSION
This study illuminates the development of research
competencies in online doctoral students. Participants in this study
are developing scholar-practitioner research competencies through
an online medium. Participants rated their competencies with a
weighted average of 4.0 or higher in all five constructs across all four
years. Even the weakest responses to an individual survey item (M =
3.94) barely fell below the 4.0 threshold. On average, students
strongly agreed or agreed with statements related to their research
competency in all five constructs. This may be attributed to the
emphasis placed in the program on using research to solve problems
of practice, as is evidenced in other CPED member programs (Perry
& Zambo, 2019). Another possible explanation for the high degree of
perceived competency is the expectation that students will emerge
from the program as a scholar-practitioner equipped to impact their
professional context, a factor found to impact doctoral student
development (Burrington et al., 2020).
Rockinson-Szapkiw (2018) posits scholar-practitioners have
three purposes, the first of which is “Develop a value of research,
including an understanding and appreciation of research as a means
to solve problems of practice” (p. 2). Students self-assessed their
competency to be highest in three categories: value of research,
evaluation and application skills, and research knowledge, aligning
with Rockinson-Szapkiw’s (2018) first purpose. This also supports
the findings of Perry and Zambo (2019) who found among CPED
member programs that application of research skills is an essential
element of preparing scholar-practitioners. These programs put
inquiry at the center of their curriculum, as does the program in this
study.
Rockinson-Szapkiw’s (2018) second and third purposes of
scholar-practitioners are to “Obtain skills and knowledge to critically
consume (e.g., information literacy), apply, and conduct research to
inform practice” and “Gain the ability to disseminate research to
professionals in the field to transform practice (p. 2).” These
purposes align with the research skill and research dissemination
competencies. In the last two years of the survey, participants most
strongly agreed with the statement “acquiring research knowledge
and skills during my program is important,” which suggests the
importance of research skills had been emphasized in the program
and, more importantly, internalized by the online doctoral students in
this study. Similarly, in Lasater et al.’s (2016) study, program
graduates approached problems and problem-solving differently and
became critical consumers of data.
Although the means were lower for research skills and research
dissemination than the other competencies, this may be explained by
the experience of participants in the study. Research value and
knowledge is emphasized from the first semester of the program;
however, the application of research skill and dissemination
culminate in a dissertation defense in the program’s third and final
year. Scholars have noted that acquiring research competencies in
doctorate programs begins with evaluating and consuming literature
(Slayton & Samkian, 2017), progresses to research methods, and
culminates in the dissertation defense and publication (Baker & Pifer,
2014). Between 49% and 61% of participants surveyed each year
were in their second year of the program, which may explain higher
perceived competency in the value of research and lower perceived
competency in research skill and research dissemination.
Additionally, as students progress through the program, they attend
more Summer Institutes and become more active in attending and
presenting at education research conferences. Choi et al. (2016)
found that participating in affinity groups and attending academic
conferences were both associated with development of a scholar
identity. These factors suggest why the skill to conduct and
disseminate research develops more slowly even as the value of
research, evaluation and application skills, and research knowledge
are developing throughout the program.
In conclusion, the findings indicate that participants in this study,
in accordance with Rockinson-Szapkiw (2018), are developing as
scholar-practitioners in their abilities to value research as a means to
Assessing Student Research Competencies
Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 8 No. 4 (2023) DOI 10.5195/ie.2023.296 41
solve problems of practice, obtain skills and knowledge to critically
consume, apply, and conduct research to inform practice, and gain
the ability to disseminate research to professionals in the field to
transform practice.
RECOMMENDATIONS AND CALLS FOR FUTURE RESEARCH
First, the researchers recommend using the Scholar-
Practitioner Research Scale to assess program effectiveness and
track program improvements. The scale represents a valid means for
doctoral faculty to assess whether students are emerging as scholar-
practitioners who can “bridge research, theory, and practice” (Suss,
2015, p. 50). Our data indicated students perceived their
competencies to be highest in value of research, suggesting the
program has been successful in fostering a belief in the important
role research plays in solving problems of practice. On the other
hand, students perceived research dissemination to be one of their
lowest competencies. However, growth in student perceptions of
dissemination of research over the last few iterations of the survey
reflect an increased focus on research dissemination in the program
as a result of the findings from earlier iterations of the Scholar-
Practitioner Research Scale. In response to data collected in the
earlier iterations of the survey, the program put a greater emphasis
on research dissemination earlier in the program, alumni shared how
they disseminated their research, and students were encouraged to
present at regional and national conferences, even during the
research-in-progress stage. Subsequently, student perceptions of
dissemination of research increased. Thus, the Scholar-Practitioner
Research Scale helped identify a need and empowered the program
to target and improve that research competency.
Golde (2013) conceptualizes the development of scholar-
practitioners as occurring in three stages: Entry and Integration,
Integrating New Knowledge, and Completion and Exit. It is
recommended that CPED-influenced Ed.D. program develop a
strategy for developing research competencies across all three
stages. Similarly, it is recommended that CPED-influenced
institutions and other institutions that prepare scholar-practitioners,
review individual items on the Scholar-Practitioner Research Scale to
identify possible gaps in the curriculum. For example, a Doctor of
Education program may emphasize using research to solve PoPs,
yet the item “I can identify scholarly resources to solve problems I
encounter in my professional practice” might lead faculty to question
the ways in which students are taught to access research, especially
after they graduate from the program.
The literature highlights the skills required of scholar-
practitioners, yet additional research is needed to determine the
impact of specific pedagogical practices and faculty behaviors upon
the development of individual research competencies. This is
particularly important for online doctoral education, a growing
segment of doctoral study.
Further research is required to determine the manner in which
scholar-practitioner research competencies are attained by students
and whether these competencies are attained sequentially,
concurrently, or in splintered fashion. Though Golde (2013)
conceptualized the development of scholar-practitioners as occurring
in three stages, the scholarship on doctoral students has yet to
determine if the growth and development of research competencies
are developed in a sequential, linear fashion or if development is
staggered and splintered. In other words, research competencies
may not be developed in a specific order or at a specific pace.
Perhaps students experience periods of rapid research competency
development and growth, interspersed with periods of relatively
slower growth. Likewise, some competencies may be developed
simultaneously. More research is needed to make these
determinations.
As Choi et al.’s (2019) literature review reveals, multiple studies
found a link between research competence and researcher
confidence. Further research is needed to determine if competencies
in particular skills is associated with researcher confidence and if
scholar-practitioner attitudes (e.g., value of research) are associated
with competence and confidence. Choi et al. (2019) also note that
specific activities such as participation in student-created writing
groups promote research competence, however additional research
is needed to determine how these might function in an online,
asynchronous environment, such as the one in this program.
While this study investigated the development of scholar-
practitioner competencies in an online Doctor of Education program,
it could not separate the impact of the program from the online
medium itself. Therefore, further investigation is needed to determine
the impact of an online learning environment on the development of
scholar-practitioner competencies.
Replication of this study using identifiable individuals and
groups is recommended to better understand the growth of research
competencies over time, including sequence and pace. A qualitative
study of student experiences while developing research
competencies would help illuminate this phenomenon and therefore
is also recommended.
REFERENCES
Andrews, R., & Grogan, M. (2005). Form should follow function: Removing the Ed.D. dissertation from the Ph.D. straight jacket. UCEA Review 46(2), 10–12.
Archbald, D. (2008). Research versus problem solving for the education leadership doctoral thesis: Implications for form and function. Educational Administration Quarterly, 44(5), 704–739.
Baker, V. L., & Pifer, M. J. (2014). Preparing for practice: Parallel processes of identity development in stage 3 of doctoral education. International Journal of Doctoral Studies, 9, 137–154. https://doi.org/10.28945/2041
Bouck, G. M. (2011). Scholar–practitioner identity, A liminal perspective. Scholar-Practitioner Quarterly, 5(2), 201–210.
Burrington, D., Madison, R. D., & Schmitt, A. (2020). Dissertation committee chairs’ current practices to support doctoral students in an online doctoral program. Online Journal of Distance Learning Administration, 23(3), 1–13.
Carnegie Project on the Education Doctorate. (2020). 2020 CPED Member Report. https://cped.memberclicks.net/assets/2020%20Member%20Report.pdf
Carnegie Project on the Education Doctorate. (2021). The CPED Framework. CPEDinitiative.org. https://www.cpedinitiative.org/the-framework
Caskey, M., Stevens, D., & Yeo, M. (2020). Examining doctoral student development of a researcher identity: Using the draw a researcher test. Impacting Education: Journal on Transforming Professional Practice, 5(1). https://doi.org/10.5195/ie.2020.92
Choi, Y. H., Bouwma-Gearhart, J., & Ermis, G. L. (2019, April 8). Identity
development in doctoral education: Literature reviews and implications [Paper presentation]. Annual Meeting of the American Educational Research Association, Toronto, CA.
Golde, C. M. (2013). Afterward: Mapping the transformation of the Ed.D student. In J. A. Perry & D. L. Carlson’s (Eds.), In their own words, A journey to the stewardship of the practice in education (pp. 139-148). Information Age.
Gillham & Schilling
Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 8 No. 4 (2023) DOI 10.5195/ie.2023.296 42
Hochbein, C., & Perry, J. A. (2013). The role of research in the professional doctorate. Planning and Changing 44(3/4), 181–195.
Jenlink, P. M. (2014). The cultural ecology of scholar-practitioner leaders: An ethnographic study of leadership. Education Leadership Review of
Doctoral Research, 1(1), 1–24.
Labaree, D. F. (2003). The peculiar problems of preparing educational researchers. Educational Researcher, 32(4), 13–22. https://doi.org/10.3102/0013189X032004013
Lasater, K., Bengston, E., & Murphy-Lee, M. (2016). An online CPED educational leadership program, Student perspectives on its value and influence on professional practice. Impacting Education, Journal on Transforming Professional Practice, 1(1). https://doi.org/10.5195/ie.2016.8
Levine, A. (2005). Educating school leaders. Columbia University Teachers’ College.
Malen, B., & Prestine, N. (2005). The case for revitalizing the dissertation. Review 46(2), 7–9.
Murakami-Ramalho, E., Militello, M., & Piert, J. (2013). A view from within: How doctoral students in educational administration develop research knowledge and identity. Studies in Higher Education, 38(2), 256–271. https://doi.org/10.1080/03075079.2011.578738
Murphy, J., & Vriesenga, M. (2005). Developing professionally anchored dissertations. School Leadership Review, 1(1), 33–57.
National Center for Education Statistics (2020). Number and percentage of students enrolled in degree-granting postsecondary institutions, by distance education participation, location of student, level of enrollment, and control and level of institution: Fall 2018 and fall 2019. https://nces.ed.gov/programs/digest/d20/tables/dt20_311.15.asp
National Center for Education Statistics (2021). Postbaccalaureate enrollment. https://nces.ed.gov/programs/coe/pdf/2021/chb_508c.pdf
Perry, J. A. (2015). The Carnegie Project on the Education Doctorate. Change: The Magazine of Higher Learning, 47(3), 56–61.
Perry, J. A., Zambo, D., & Wunder, S. (2015). Understanding how schools of education have redesigned the doctorate of education. Journal of School Public Relations 36(1), 58–85.
Perry, J. A., & Zambo, D. (2019, April 8). Developing educational leaders with the capacity to connect research and practice [Paper presentation]. Annual Meeting of the American Educational Research Association, Toronto, CA.
Quality Matters & Eduventures Research (2020). The changing landscape of online education (CHLOE) 4: Navigating the mainstream. https://www.qualitymatters.org/sites/default/files/research-docs- pdfs/CHLOE-4-Report-2020-Navigating-the-Mainstream.pdf
Rockinson-Szapkiw, A. (2018). The development and validation of the scholar– practitioner research development scale for students enrolled in professional doctoral programs. Journal of Applied Research in Higher Education, 10(4). 478–492. https://doi.org/10.1108/JARHE-01-2018-0011
Rolfe, G., & Davies, R. (2009). Second generation professional doctorates in nursing. International Journal of Nursing Studies, 46(9), 1265–1273.
Servage, L. (2009). Alternative and professional doctoral programs: What is driving the demand? Studies in Higher Education, 34(7), 765–779.
Shulman, L. S., Golde, C. M., Bueschel, A. C., & Garabedian, K. J. (2006). Reclaiming education’s doctorates: A critique and a proposal. Educational Researcher, 35(3), 25–32.
Slayton, J., & Samkian, A. (2017). Scaffolding learning for practitioner-scholars: The philosophy and design of a qualitative research methods course. Journal of Research on Leadership Education, 12(1), 51–71.
Suss, D. D. (2015). T4 MAP™: A scholar-practitioner model for performance improvement. Performance Improvement Quarterly, 27(4), 49–75. https://doi.org/10.1002/piq.21179
Toma, J. D. (2002, November 22). Legitimacy, differentiation, and the promise of the EdD in higher education [Paper presentation]. Annual Meeting of the Association for the Study of Higher Education, Sacramento, CA.
Townsend, B. (2002, November 22). Rethinking the EdD or what’s in a name [Paper presentation]. Annual Meeting of the Association for the Study of Higher Education, Sacramento, CA.
