week 6 Discussion 1
arita381181
Week6LiteratureReviews_1.pdf
SPECIAL ARTICLES
Where and How to Search for Evidence in the Education Literature: The WHEEL
Therese Poirier, PharmD, MPH and Erin Behnen, PharmD
School of Pharmacy, Southern Illinois University Edwardsville, Edwardsville, Illinois
Submitted November 21, 2013; accepted January 16, 2014; published May 15, 2014.
An awareness of how and where to search the education literature, and how to appraise it is essential to be a teacher scholar (an academic who takes a scholarly approach to teaching), to develop high quality education research, and to perform the scholarship of teaching and learning. Most pharmacy faculty scholars do not receive training in searching the education literature. Thus, a framework for searching the education literature is needed. The framework presented here on where and how to search for evidence in the education literature, referred to as the WHEEL for teaching, is meant to serve as a guide for faculty members in conducting comprehensive and exhaustive literature searches for the publica- tion of scholarship of teaching and learning projects, educational research, or approaching one’s teaching in a scholarly manner. Key resources to search and methods for searching the education literature are listed and described.
Keywords: education literature, searching, research, teaching
INTRODUCTION Pharmacy faculty members are faced with many
teaching challenges today including an emphasis on in- corporating active-learning strategies using instructional technology tools; measuring learning and outcomes through such things as progress examinations, objective structured clinical examinations, rubrics, and portfolios; and incorporating various instructional strategies such as team-based learning, high-fidelity simulations, and inter- professional education. In the recommendations from the American Association of Colleges of Pharmacy Task Force on Best Evidence Pharmacy Education, Hammer andcolleaguesrecommendedthataseriesofpapersinThe American Journal of Pharmaceutical Education (AJPE) on “how to’s” for the scholarship of teaching and learning (SoTL) and educational research is needed.
1 During the last few years, researchers have provided the academy with numerous key articles in the area of SoTL that guide the development of projects.2-5 What is lacking is a guide to searching the educational literature. Even if faculty members do not perform SoTL or educational research, faculty members should have a scholarly approach to teaching, which requires knowing how and where to search for education literature beyond the pharmacy dis-
cipline that supports teaching. Having an awareness of the levels of evaluation used to appraise the educational lit- erature is also important. The intent of this article is to guide faculty members in conducting a comprehensive and exhaustive literature search and in appraising the lit- erature in preparation for publication of scholarship of teaching and learning, for educational research projects, and in being a teacher scholar. These guidelines are sum- marized by the acronym WHEEL, which stands for where and how to search for evidence in the education literature. The analogy of a wheel is appropriate because as new information is learned and the search process is repeated, the wheel moves forward.
WHY TO SEARCH THE EDUCATION LITERATURE
Health professionals are familiar with the practice of evidence-based medicine for the best treatment of pa- tients. For this purpose, they identify a question, search the literature, appraise the evidence, apply the evidence, and evaluate the outcomes.6,7 This process is also appli- cable to teaching methods used to educate pharmacy stu- dents. In order for educators to take a scholarly approach to teaching, they must first review and be familiar with what is already known about different teaching methods. Evidence, including published literature as well as data collected internally, should be used whenever educators aredoing anythingfromtryinga newactive-learningtech- nique in the classroom to revising an entire curriculum.
Corresponding Author: Therese Poirier, PharmD, School of Pharmacy, Southern Illinois University Edwardsville, Edwardsville, IL 62025. Tel: 618-650-5155. Fax: 618-650- 5152. E-mail: [email protected]
American Journal of Pharmaceutical Education 2014; 78 (4) Article 70.
1
Searching the literature is a key component in this pro- cess. Reviewing the evidence prior to implementing a change can help to inform educators about potential pitfalls and best practices. A comprehensive literature search should be conducted to review all potentially valu- able information. Assistance with how and where to search for education-related literature is useful as most pharmacy faculty scholars do not receive this training.
Published literature provides background support to indicate what is known about a specific area. It also assists in identifying the needs or unknowns in the particular area of study. Reviewing the background support and data then allows educators and researchers to formulate their spe- cific ideas or research in an evidence-based manner and provides focus to support a unique project that would have a great impact in the classroom and could add to the lit- erature. Literature from other disciplines or from higher education in general may have already evaluated a similar teaching method and what they found may be applicable to pharmacy education. Analyzing the existing evidence in light of the current proposed project is an important step for conducting research that matters.
8 Incomplete searches may lead to a biased publication and an inappro- priate interpretation of the impact of a project.9 It is im- portant to know what has been done in the past in order to build on established knowledge. Without first consulting published literature and data, researchers will be repeat- ing work already accomplished by other scholars.
WHERE TO SEARCH FOR EDUCATION LITERATURE
Because there is no one database that houses all ed- ucation literature, multiple searches within multiple sour- ces are necessary in order to find all pertinent literature. Common databases (listed in Table 1) include thousands of citations relevant to health education. Unfortunately, the databases may not always have appropriate subject headings for searching health education literature and not all health education resources are indexed. Multiple searches using various terminology and use of related citations may be helpful to ensure complete searches in databases. Foundations and organizations focusing on ed- ucation are good sources of information that may be use- ful to supplement literature searches in databases. Many of these are listed in Table 2. Search functions within these resources vary greatly. Ancestry searching, which involves reviewing the bibliographies from any useful citations found in these searches, should be used to iden- tify additional citations.
It is important to also manually search pharmacy- related education journals such as the American Journal of Pharmaceutical Education and Currents in Pharmacy
Teaching and Learning. Additionally, scanning other education-related journals and information from other health professions is essential as well. Databases may not include all education-related journals, and indexing of these journals, if they are included, may not be com- prehensive. Table 3 includes a list of key journals related to teaching and notes if they are currently indexed within the most commonly used databases. This may include only partial indexing of the journals (eg, indexing only 1 year of a journal’s articles). These journals may be added or deleted from the databases over time. The resources listed in the tables are not all-inclusive lists, but may be ideas for beginning searches.
10
Finally, experts in the field may be contacted and Web searches for grey literature (any literature that is not controlled by commercial publishers) may be conducted. Grey literature for health education research may include items such as dissertations, theses, committee reports, gov- ernment reports, technical reports, etc.11
HOW TO SEARCH THE EDUCATION LITERATURE
The systematic approach to searching for any type of literature also applies to searching for education literature. The steps to searching for education literature are summa- rized in Figure 1. The figure includes numbers to order the steps; however, it is a circle (wheel) because as new infor- mationislearned,theprocessmayberepeatedtoincorporate thenew information.The firststep in the process istoclearly define the question or the hypothesis and the scope of the question. If a team of scholars are looking to create a tool to assessaninterprofessionalexperienceincludingnursingand pharmacy students, they would need to think about specif- ically which aspect(s) of the interaction they are looking to assess. Are they looking to assess students’ learning, contri- butions, attitude, etc? Are they looking only at experiential learning vs classroom lectures, or a one-time event vs mul- tiple interactions? They should think about the participants, intervention, comparison, outcome, and inclusion and ex- clusion criteria when determining the scope of the query. Thisisfollowedbydeterminingessentialconceptsforwhich more information is needed. In the interprofessional experi- ence information, this may include understanding what is meant by interprofessional vs interdisciplinary.
Search terms should be developed, taking time to think about all possible related terms, different spellings, and synonyms to ensure a complete search. Related citation searching may be used to identify additional appropriate citations.Searchingtheterm“interprofessional”inMedline links to the Medical Subject Heading (MeSH) terms inter- professional relations or interdisciplinary communication. Searching the term “interdisciplinary team” links to the
American Journal of Pharmaceutical Education 2014; 78 (4) Article 70.
2
T a b le
1 . D a ta b a se s U se fu l in
R e se a rc h in g th e E d u c a ti o n L it e ra tu re
D a ta b a se
D e sc r ip ti o n
A v a il a b il it y
A c a d e m ic
S e a rc h P re m ie r
In d e x e s m o re
th a n 1 2 ,0 0 0 p e e r- re v ie w e d sc ie n c e jo u rn a ls
in a re a s su c h a s: b io lo g y , c h e m is tr y , p sy c h o lo g y .
A c c e ss
is b y su b sc ri p ti o n .
B ri ti sh
E d u c a ti o n In d e x (B E I) -
E D U C A T IO
N -L IN E
In d e x e s e d u c a ti o n jo u rn a ls p u b li sh e d in
th e U n it e d K in g d o m
in c lu d in g jo u rn a ls re la te d to
h e a lt h sc ie n c e e d u c a ti o n .
A c c e ss
is p a rt ia ll y fr e e ly
a v a il a b le . F u ll a c c e ss
is b y su b sc ri p ti o n . w w w .l e e d s. a c .u k /b e i/
C O L N /C O L N _ d e fa u lt .h tm
l T h e C u m u la ti v e In d e x to
N u rs in g a n d
A ll ie d H e a lt h L it e ra tu re
(C IN
A H L )
In d e x e s m o re
th a n 3 ,0 0 0 jo u rn a ls re la te d to
n u rs in g
a n d a ll ie d h e a lt h . N u rs in g d is se rt a ti o n s a n d c o n fe re n c e
p ro c e e d in g s a re
a ls o in c lu d e d .
A c c e ss
is b y su b sc ri p ti o n .
E d u c a ti o n A b st ra c ts
In d e x e s a lm
o st 7 0 0 jo u rn a ls re la te d to
e d u c a ti o n a n d a ss e ss m e n t.
A c c e ss
is b y su b sc ri p ti o n .
E d u c a ti o n R e se a rc h C o m p le te
In d e x e s 2 ,4 0 0 jo u rn a ls c o v e ri n g in fo rm
a ti o n fo r e d u c a to rs
a n d a d m in is tr a to rs .
A c c e ss
is b y su b sc ri p ti o n .
T h e E d u c a ti o n R e so u rc e In fo rm
a ti o n
C e n tr e (E R IC )
In d e x e s m a te ri a ls fr o m
th e U .S . D e p a rt m e n t o f E d u c a ti o n ,
th e In st it u te
o f E d u c a ti o n S c ie n c e s a n d e d u c a ti o n re la te d re so u rc e s.
A c c e ss
is fr e e ly
a v a il a b le
o r v ia
su b sc ri p ti o n . e ri c .e d .g o v
E d u c a ti o n a l A d m in is tr a ti o n A b st ra c ts
In d e x e s m a te ri a ls re la te d to
e d u c a ti o n a l a d m in is tr a ti o n , le a d e rs h ip ,
m a n a g e m e n t a n d re se a rc h .
A c c e ss
is b y su b sc ri p ti o n .
E d u c a ti o n a l R e se a rc h A b st ra c ts (E R A )
M o re
th a n 1 ,0 0 0 a b st ra c ts a re
a d d e d m o n th ly
in th e a re a
o f e d u c a ti o n re se a rc h .
A c c e ss
is b y su b sc ri p ti o n .
E m b a se
In c lu d e s a ll o f M e d li n e p lu s 2 ,0 0 0 a d d it io n a l jo u rn a ls a n d in c lu d e s
a b st ra c ts fr o m
c o n fe re n c e s re la te d to
b io m e d ic a l re se a rc h .
A c c e ss
is b y su b sc ri p ti o n .
G o o g le
S c h o la r
A w e b -c ra w le r th a t se a rc h e s o n li n e th e se s, b o o k s, jo u rn a ls ,
p ro fe ss io n a l so c ie ti e s a n d u n iv e rs it ie s fo r sc h o la rl y li te ra tu re .
A c c e ss
is fr e e ly
a v a il a b le . sc h o la r. g o o g le .c o m
In te rn a ti o n a l P h a rm
a c e u ti c a l
A b st ra c ts (I P A )
In d e x e s m o re
th a n 8 0 0 jo u rn a ls re la te d to
p h a rm
a c y ,
m e d ic in e , a n d h e a lt h .
A c c e ss
is b y su b sc ri p ti o n .
M e d li n e
In d e x e s m o re
th a n 5 ,0 0 0 b io m e d ic a l li te ra tu re
jo u rn a ls
A c c e ss
is fr e e ly
a v a il a b le
(P u b M e d ) o r v ia
c o m m e rc ia l v e n d o rs . w w w .n c b i. n lm
.n ih .
g o v /p u b m e d
P sy c IN
F O
In d e x e s m o re
th a n 2 ,5 0 0 jo u rn a ls in
th e a re a s o f b e h a v io ra l
a n d m e n ta l h e a lt h
A c c e ss
is b y su b sc ri p ti o n .
R e se a rc h a n d D e v e lo p m e n t
R e so u rc e B a se
A b ib li o g ra p h ic
d a ta b a se
fo c u si n g o n c o n ti n u in g e d u c a ti o n
in h e a lt h p ro fe ss io n s, in te rp ro fe ss io n a l li te ra tu re ,
a n d fa c u lt y d e v e lo p m e n t.
A c c e ss
is fr e e ly
a v a il a b le . w w w .r d rb .u to ro n to .c a /
S c o p u s
In d e x e s m o re
th a n 2 1 ,0 0 0 ti tl e s re la te d to
sc ie n c e ,
m e d ic in e , a n d a rt &
h u m a n it ie s.
A c c e ss
is b y su b sc ri p ti o n .
S o c io lo g ic a l A b st ra c ts
In d e x e s m o re
th a n 1 ,8 0 0 ti tl e s re la te d to
so c ia l
a n d b e h a v io ra l sc ie n c e s.
A c c e ss
is b y su b sc ri p ti o n .
American Journal of Pharmaceutical Education 2014; 78 (4) Article 70.
3
T a b le
2 . E d u c a ti o n R e so u rc e s A v a il a b le
W it h in
O rg a n iz a ti o n s a n d F o u n d a ti o n s
O r g a n iz a ti o n
D e sc r ip ti o n
W e b A d d r e ss
A m e ri c a n A ss o c ia ti o n o f C o ll e g e s o f P h a rm
a c y
E d u c a ti o n a l R e so u rc e s
In c lu d e s c u rr ic u lu m
a n d a ss e ss m e n t in fo rm
a ti o n
fo r p h a rm
a c y e d u c a ti o n .
h tt p :/ /w w w .a a c p .o rg /R E S O U R C E S /E D U C A T IO
N /
P a g e s/ d e fa u lt .a sp x
A ss o c ia ti o n o f A m e ri c a n C o ll e g e s a n d
U n iv e rs it ie s – V A L U E ru b ri c s
R u b ri c s to
a ss e ss
le a rn in g in
u n d e rg ra d u a te
e d u c a ti o n .
In c lu d e s ru b ri c s fo r a re a s su c h a s: c ri ti c a l th in k in g ,
o ra l c o m m u n ic a ti o n , w ri tt e n c o m m u n ic a ti o n , e tc .
h tt p :/ /w w w .a a c u .o rg /v a lu e /r u b ri c s/ in d e x _ p .c fm
A ss o c ia ti o n fo r M e d ic a l E d u c a ti o n in
E u ro p e
P ro d u c e s g u id e s o n to p ic s re la te d to
h e a lt h c a re
p ro fe ss io n e d u c a ti o n .
w w w .a m e e .o rg
B e st E v id e n c e fo r M e d ic a l E d u c a ti o n
C o ll a b o ra ti o n
P ro d u c e s sy st e m a ti c re v ie w s o f m e d ic a l e d u c a ti o n .
h tt p :/ /w w w .b e m e c o ll a b o ra ti o n .o rg /
C a m p b e ll C o ll a b o ra ti o n
In c lu d e s p re se n ta ti o n s o n h o t to p ic s in
e d u c a ti o n
a n d li n k s to
e d u c a ti o n re so u rc e s.
h tt p :/ /w w w .c a m p b e ll c o ll a b o ra ti o n .o rg /r e so u rc e s/
li n k s/ li n k s_ e d u c a ti o n .p h p
C a rn e g ie
F o u n d a ti o n fo r th e A d v a n c e m e n t
o f T e a c h in g
P u b li c a ti o n s a n d p re se n ta ti o n s re la te d
to te a c h in g a n d le a rn in g .
h tt p :/ /w w w .c a rn e g ie fo u n d a ti o n .o rg
C e n tr e fo r th e A d v a n c e m e n t o f
In te rp ro fe ss io n a l E d u c a ti o n
P u b li c a ti o n s a n d p re se n ta ti o n s re la te d
to in te rp ro fe ss io n a l e d u c a ti o n .
h tt p :/ /w w w .c a ip e .o rg .u k /
C e n tr e fo r H e a lt h In fo rm
a ti c s a n d M u lt ip ro fe ss io n a l
E d u c a ti o n
R e se a rc h in
h e a lt h in fo rm
a ti o n a n d q u a li ty
m a n a g e m e n t.
w w w .c h im
e .u c l. a c .u k
C e n te r fo r H e a lt h S c ie n c e In te rp ro fe ss io n a l
E d u c a ti o n , R e se a rc h a n d P ra c ti c e
F a c u lt y d e v e lo p m e n t, to o ls a n d c u rr ic u la , a n d re so u rc e s
a n d p u b li c a ti o n s re la te d to
in te rp ro fe ss io n a l e d u c a ti o n
fr o m
th e U n iv e rs it y o f W a sh in g to n .
h tt p :/ /c o ll a b o ra te .u w .e d u /
H e a lt h R e so u rc e s a n d S e rv ic e s A d m in is tr a ti o n
(H R S A ) C o o rd in a ti n g C e n te r fo r In te rp ro fe ss io n a l
E d u c a ti o n a n d C o ll a b o ra ti v e P ra c ti c e
W e b in a rs
a n d fu n d in g o p p o rt u n it ie s fo r te a c h in g h e a lt h
c e n te rs , g e ri a tr ic s, d iv e rs it y , p u b li c h e a lt h , m e d ic in e ,
m e n ta l h e a lt h , a n d in te rp ro fe ss io n a l e d u c a ti o n .
h tt p :/ /b h p r. h rs a .g o v /g ra n ts /
in te rp ro fe ss io n a l/ in d e x .h tm
l
H ig h e r E d u c a ti o n A c a d e m y
T e a c h in g a n d le a rn in g re so u rc e s fo r h e a lt h e d u c a ti o n .
h tt p :/ /w w w .h e a c a d e m y .a c .u k /r e so u rc e s
In te rp ro fe ss io n a l E d u c a ti o n C o ll a b o ra ti v e
In te rp ro fe ss io n a l e d u c a ti o n re so u rc e s.
h tt p s: // ip e c o ll a b o ra ti v e .o rg
In st it u te
o f E d u c a ti o n S c ie n c e s –
W h a t W o rk s C le a ri n g h o u se
S u m m a ri z e s st u d ie s e v a lu a ti n g th e e ff ic a c y
o f e d u c a ti o n p ra c ti c e s.
h tt p :/ /i e s. e d .g o v /n c e e /w w c
In st it u te
fo r In te rn a ti o n a l M e d ic a l E d u c a ti o n
G lo b a l m in im
u m
e ss e n ti a l re q u ir e m e n ts
fo r m e d ic a l e d u c a ti o n .
h tt p :/ /w w w .i im
e .o rg
In te rn e t R e so u rc e s fo r H ig h e r E d u c a ti o n
O u tc o m e s A ss e ss m e n t
L in k s to
m u lt ip le
o n li n e re so u rc e s re la te d
to te a c h in g a n d le a rn in g b y c a te g o ri e s.
h tt p :/ /w w w 2 .a c s. n c su .e d u /U
P A /
a rc h iv e s/ a ss m t/ re so u rc e .h tm
Jo si a h M a c y Jr
F o u n d a ti o n
S u p p o rt fo r a n d p u b li c a ti o n s to
im p ro v e h e a lt h
p ro fe ss io n a l e d u c a ti o n .
h tt p :/ /m
a c y fo u n d a ti o n .o rg
N a ti o n a l E d u c a ti o n A ss o c ia ti o n H ig h e r E d u c a ti o n
B e st P ra c ti c e s
B e st p ra c ti c e s in
te a c h in g a n d le a rn in g
in h ig h e r e d u c a ti o n .
h tt p :/ /w w w .n e a .o rg /h o m e /3 3 5 0 8 .h tm
American Journal of Pharmaceutical Education 2014; 78 (4) Article 70.
4
Table 3. Key Teaching and Learning Journals for Educators in the Health Professions and Their Inclusion in Popular Databasesa
Databases (x indicates the journal is indexed in the database)
Medline CINAHL Embase Scopus ERIC IPA
Allied Health/Other Health Professions Advances in Health Sciences Education X X X X American Journal of Health Education X X X Evaluation and the Health Profession X Focus on Health Professional Education Health Education Research X X X X International Journal of Health Promotion and Education
X X X
Journal of Allied Health X X X Journal of Health Professions Education Journal of Nursing, Allied Health & Health Education
Dental Medicine Journal of Dental Education X X X
Educational Psychology Journal of Educational Psychology X
Instructional Technology International Journal of Instructional Media Journal of Information Technology Education X X Journal of Technology and Teacher Education T.H.E. Journal (Technological Horizons in Education)
X
Interprofessional Health Education Journal of Interprofessional Care X X X Journal of Research in Interprofessional Practice and Education
Medicine Academic Medicine X X X BMC Medical Education X X Education for Primary Care X X X Education for Health X X X Ethics and Medicine X Journal of The International Association of Medical Science Educators
Journal of Medical Ethics X X X Medical Education X X X Medical Teacher X X X Postgraduate Medical Journal X X X Teaching and Learning in Medicine X X
Nursing Journal of Nursing Education X X X Journal of Professional Nursing X X X Nurse Educator X X X
Pharmacy American Journal of Pharmaceutical Education X X X X Currents in Pharmacy Teaching and Learning X X Journal of Pharmacy Teaching X Pharmacy Education X X X
(Continued)
American Journal of Pharmaceutical Education 2014; 78 (4) Article 70.
5
MeSH term “patientcare teams.” Definitions forthe MeSH terms shouldbe reviewedto make surethat thechosenterm is the most appropriate for the search. Subject trees may also be reviewed to identify sub-themes. Multiple combi- nations of MeSH terms and keywords should be used.
Oncethekeyconcepts areidentified, resourcesshould be identified in which to complete searches. Searches should begin in tertiary resources (resources that provide an overview and summarize information from other re- sources) followed by searches in databases.
11,12 All re- sources likely to cover the essential concepts should be included in the search. Databases differ in the types of jour- nals and publication dates for the journals they index. Brief descriptions of the content of several databases are included in Table1. When searching for information about evaluating interprofessional interactions between nursing and phar- macy students, any resources that would have information about nursing, pharmacy, and interprofessional assessment shouldbeaccessed.Medline,TheCumulativeIndextoNurs- ing and Allied Health Literature (CINAHL), and education databases would be appropriate resources in which to begin. Ontheotherhand,itwouldlikelybeawasteoftimetosearch a database such as PsychINFO for this type of literature.
A Web search for information especially from ap- propriate government agencies, organizations, and foun- dations would help to ensure a complete search. For assessment of interprofessional interactions, there are
multiple organizations listed in Table 2 that may be ap- propriate to search and may also include searching ac- creditation standards for each discipline included for requirements for interprofessional education. If the inter- action involved managing patients with diabetes, a search for organizations related to diabetes management and ed- ucation should also be conducted. Searches should not be limited only to pharmacy literature, but should include interprofessional interactions between any health fields.
These searches should be supplemented by manual searching of appropriate journals that may not have been indexed in the databases searched or may not have been completely indexed, and ancestry searching of useful cita- tions. Once all of the information found is organized and appraised, areas where information is lacking should be identified; the process should be repeated to identify and fill in any gaps.
APPRAISING THE EVIDENCE The final step in the process is to manage the infor-
mation found by appraising the evidence and determining what is relevant to the project. The concept of using the “best evidence” is recommended.1,8 This concept has parallels to the Cochrane Collaboration, which has a mis- sion to gather the best evidence for medical practice. The Best Evidence Medical Education Collaboration and the Campbell Collaboration are 2 groups that strive for
Table 3. (Continued)
Databases (x indicates the journal is indexed in the database)
Medline CINAHL Embase Scopus ERIC IPA
Science Journal of College Science Teaching X Journal of Research in Science Teaching X X Journal of Science Education X
Other Higher Education Academic Exchange quarterly Assessment & Evaluation in Higher Education X X Assessment in Education: Principles, Policy and Practice
X X
College Student Journal X Educational Research Quarterly X Journal of Adult Education X Journal of Educational Research X X Journal of Faculty Development X Journal of Higher Education X X Journal of Instructional Development Journal of the Scholarship of Teaching and Learning X Practical Assessment, Research and Evaluation (PARE)
X X
a Indexing information was searched January 10, 2014 for complete and partial indexing of the resource. Journals currently indexed within databases and completeness of indexing may change.
American Journal of Pharmaceutical Education 2014; 78 (4) Article 70.
6
the systematic review of studies in teaching and education domains.1 The BEME Collaboration suggests appraising the evidence in medical education using the QUESTS criteria.8 Details of the QUESTS dimensions are provided in Table 4. The use of the educational information re- trieved from the research literature should be selected based on the highest quality of evidence in design, utility, extent, strength, target, and setting. Guidelines for evalu- ating educational manuscripts have been published.8,13
Following appraisal of the current literature, areas of addi- tional research should be identified for researchers to target their education projects. Learning to search for high-quality research in education and being able to appraise the quality is critical for the enhancement of SoTL projects as well as in using a scholarly approach to teaching.
The analysis of the design and methodology of high- quality education literature is not dissimilar from that of basic science or clinical science literature evaluation. Kirk- patrick described the different levels of evaluation that can be used in SoTL or in evaluating effectiveness of teaching in terms of a hierarchy.
14 These 4 levels from lowest to highest include: evaluation of reaction such as satisfaction, evaluation of learning such as knowledge or skills ac- quired, evaluation of behavior such as changes brought to the real world, and evaluation of results such as impact of behavior changes on society, outcomes, or costs. As the scholar progresses in this hierarchy, the level of evidence for the educational research is strengthened. Ideally, re- searchers in education should be striving to identify strategies that have impacts on the higher levels of eval- uation so that these strategies may be reproduced. Higher levels of evidence also would be more useful for enhanc- ing teaching. In searching the education literature, more of the evidence found is often at the lower levels of eval- uations, ie, levels 1 and 2, in contrast to evidence published in the basic science or clinical science literature.
SUMMARY In the same way that pharmacy educators take an
evidence-based approach to research and to caring for patients, it is important for them to take a scholarly ap- proach to teaching. It is essential to effectively search for and use evidence to improve teaching, to identify where
Table 4. QUESTS Method for Evaluating Evidence in Educational Practice1,4,a
Letter Meaning
Q Quality of the evidence from a design standpoint U Utility or the degree a method can be transferred
and adopted E Extent or amount of evidence S Strength of the evidence statistically T Target or question addressed and
how it was measured S Setting of the evidence in both context
and population a Reprinted from reference # 1.
Figure 1. WHEEL for Searching the Education Literature. (*Involve experts as the question is defined and again as methodology is developed.)
American Journal of Pharmaceutical Education 2014; 78 (4) Article 70.
7
there are needs for further research in teaching and learn- ing, and to enhance the application and integration of SoTL. Ideas should be defined and search terms should be identified. A comprehensive search should be con- ducted in all appropriate resources. Additionally, pub- lished literature outside of pharmacy education may be useful in guiding applications to pharmacy. All informa- tion found should be appraised to determine its value in improving pharmacy education.
REFERENCES 1. Hammer D, Sauer K, Fielding D, Skau K. White paper on best evidence pharmacy education (BEPE). Am J Pharm Educ. 2004; 68(1):Article 24. 2. Draugalis J, Coons S, Plaza C. Best practices for survey research reports: a synopsis for authors and reviewers. Am J Pharm Educ. 2008;71(1):Article 11. 3. Poirier T, Crouch M, MacKinnon G, Mehvar R, Monk-Tutor M. Updated guidelines for manscripts describing instructional design and assessment: the IDEAS format. Am J Pharm Educ. 2009;73(3):Article 55. 4. Anderson C. Presenting and evaluating qualitative research. Am J Pharm Educ. 2010;74(8):Article 141. 5. Roederer M, Marciniak M, O’Connor S, Eckel S. An integrated approach to research and manuscript development. Am J Health-Syst Pharm. 2013;70(14):1211-1218.
6. Rosenberg W, Donald A. Evidence-based medicine: an approach to clinical problem-solving. BMJ. 1995;310(6987): 1122-1126. 7. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd ed. Edinburgh: Churchill Livingstone; 2000. 8. Harden RM, Grant J, Buckley G, Hart IR. BEME guide no 1: best evidence medical education. Med Teach. l999;21(6): 553-562. 9. Kemtes KA, Smith K, Donohue B, Alvarez KM, Carpin KM, Sinchak J. A systematic approach to conducting educational literature searches. J Contin High Educ. 2003;51(2):26-32. 10. Cashin W, Clegg V. Periodicals related to college teaching. IDEA Paper No. 28. Idea Education. www. http://ideaedu.org/research-and- papers/idea-papers/idea-paper-no-28. Accessed April 25, 2014. 11. Haig A, Dozier M. BEME guide no 3: systematic searching for evidence in medical education – part 1: sources of information. Med Teach. 2003;25(4):352-363. 12. Haig A, Dozier M. BEME guide no 3: systematic searching for evidence in medical education – part 2: constructing searches. Med Teach. 2003;25(5):463-484. 13. Education Group for Guidelines on Evaluation. Guidelines for evaluating papers on educational interventions. BMJ. 1999;318: 1265-1267. 14. Kirkpatrick DL, Kirkpatrick JD. Transferring Learning to Behavior: Using the Four Levels to Improve Performance. San Francisco: Berrett-Koehler; 2005.
American Journal of Pharmaceutical Education 2014; 78 (4) Article 70.
8
Copyright of American Journal of Pharmaceutical Education is the property of American Association of Colleges of Pharmacy and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
