D

INSTRUCTIONAL DESIGN AND ASSESSMENT

A Course Assessment Process for Curricular Quality Improvement

Steven L. Peterson, PhD, Kristina M. Wittstrom, MEd, and Mark J. Smith, MA

College of Pharmacy, University of New Mexico

Submitted March 6, 2011; accepted June 21, 2011; published October 10, 2011.

Objective. To describe a systematic assessment process that provides continuous improvement in the curriculum, supports faculty development, and enhances student learning outcomes. Design. Teams of faculty members, students, and course instructors conducted course assessments, which consisted of monitoring the delivered instruction for agreement with planned content and course integration within the curriculum, and providing ongoing feedback for improving course content, course management, faculty teaching, and student learning experiences. Assessment. Areas needing improvement were identified and appropriate changes were made. Im- provements were achieved in course policy standardization, course integration within the curriculum, faculty teaching behaviors, and student experiences. Conclusion. The curriculum assessment plan provides a structured method of monitoring and de- livering continuous quality improvement.

Keywords: curriculum assessment, quality improvement, evaluation, learning outcomes, faculty development, curricular improvement

INTRODUCTION The 2007 Accreditation Council for Pharmacy Ed-

ucation (ACPE) standards include new and specific re- quirements for the development, implementation, and management of curriculum. These standards require an orderly and systematic review of curricularstructure, con- tent, processes, and learning outcomes, and they charge faculty members with the development, organization, de- livery, and improvement of the curriculum.

1 The University of New Mexico College of Pharmacy Curriculum Commit- tee is responsible for the developmentof outcomes and com- petencies,curricular revisions, and continuous monitoring of the curriculum to ensure that course content, learning activities, instructional methods, and appropriate program assessments are coordinated to generate desired student learning outcomes.

Assessmentmodelsand methodsusedin otherhigher education programs often are not applicable to pharmacy education for several reasons.

2 First, pharmacy course struc- ture is different from the large-lecture, single-instructor format used elsewhere in higher education, eg, a profes- sional course may have 1 course coordinator but be taught by multiple instructors. A second difference is that the phar-

macy curriculum is standardized and carefully structured with limited opportunities for students to select their courses or instructors. Students move through the curriculum in co- horts, each exposed to the same learning experiences and opportunities, with the exception of elective courses. A third difference between pharmacy and other higher edu- cation programs is the integrated structure of core courses within the pharmacy curriculum. Courses are integrated both horizontally, with reinforcement and collaboration between courses across each year of the professional pro- gram, andvertically,requiringstudentstosuccessfullycom- plete each level of the curriculum in order to progress. Thesedifferencesrequireacomprehensivecurriculardesign and assessment program focusing on the entire integrated pharmacy curriculum in addition to individual courses.

Several reports in the literature describe assessment efforts at colleges and schools of pharmacy, such as devel- opinganassessmentoffice,

3creatingacultureofassessment,4

and conductingcurricularevaluationandrefinement.5 There also are published comprehensive reviews that outline the rationale and basic principles for assessment in pharmacy education.6,7However,nomethodforcomparingtheplanned pharmacy curriculum to the delivered instruction was found in the literature.

Our goal was to develop a systematic assessment plan that would provide continuous improvement in the curric- ulum while supporting faculty development and student learning outcomes. We define curriculum to include con- tent, student learning outcomes, instructional methods and

Corresponding Author: Kristina Wittstrom, College of Pharmacy, MSC 09 5360, 1 University of New Mexico, The University of New Mexico, Albuquerque, NM 87131-0001. Tel: 505-272-3661. Fax: 505-272-6749. E-mail: kwittstrom@ salud.unm.edu

American Journal of Pharmaceutical Education 2011; 75 (8) Article 157.

1

approaches, teaching and learning processes, and student, instructor, and program assessment.6,8 Ongoing program assessment also should assist faculty members in making informed changes in course structures and instructional methods. Information collected should provide evidence of successes as well as suggest specific steps, resources, and support needed for improvements. Table 1 provides a summary of effective program assessment principles.

The curriculum assessment and development plan was initially created for 3 reasons: to comply with ACPE Standards 2007

1 to ensure course integration and sequenc- ing; and to respond to major revisions in both the PharmD curriculum and the college’s professional competencies and identify content gaps and duplication within and between courses. The 3 Professional Competencies and Outcome Expectations listed in Standard 121 were used to synthesize the 2004 Center for the Advancement of Pharma- ceutical Education Educational Outcomes9 into 30 profes- sional competencies. Although the Curriculum Committee workedwithindividualcoursecoordinatorstodeveloplearn- ingoutcomessupportingthenewprofessionalcompetencies, confirmation that the delivered course content corresponded to the intended content was needed. Our main objective for thispaperwastoprovideanexampleofacomparisonofthe planned curriculum to the delivered curriculum.

DESIGN A curriculum committee was formed consisting of

representative members of the educational community: students, faculty members, member of the college admin- istration (assistant dean for curriculum), and members of the pharmacy community. Faculty representation on the committee included an equal number of members from the pharmaceutical science and pharmacy practice depart- ments. Course coordinators of multi-instructor courses were encouraged to serve on the committee to facilitate

the sequencing and integration of the curriculum and help disseminate knowledge of course management policies and procedures where it would be most effective.

The3coursecoordinatorsforthePharmaceuticalCare Laboratory (PCL) courses were included as these courses arekey to reinforcementand integration ofkey conceptsand skillsacrosstheentirecurriculum.Servingonthecurriculum committee gave the course coordinators insight into the con- tent of concurrent courses as well as immediate access to other course coordinators, which greatly facilitated inte- gration of curriculum content throughout the program.

Two students from each of the first 3 years of the curriculum were selected from a list of volunteers based on theirmaturityand motivationto participate in thetime- intensive activity of curriculum committee work. Two students were chosen from each class to ensure class rep- resentation at all committee meetings. Students in the fourth year were not included because of conflicts with their experiential schedules.

The student committee members were encouraged to share responsibilities and discuss curricular issues with each other. As the courses progressed, students provided ongoing feedback to their peers and functioned as a con- duit of feedback from their peers to the curriculum com- mittee. Students were encouraged to provide critical but objective reviews of courses and instructors.Anynegative feedback from students was presented to faculty members by the committee leadership as constructive criticism with suggestions for improvement to protect the confidentiality of the students.

Practicing pharmacists representing both community and hospital practices were invited to participate. These members provided perspective on the knowledge, skills, and values most essential for the daily practice of phar- macy.

10 Members who also served as student preceptors provided an external view of common student shortcom- ings and offered practical solutions to identify and resolve deficiencies in the curriculum that may have led to these shortcomings.

All committee members were required to attend an orientation during which they were instructed in the poli- cies, procedures, and confidentiality expectations of the committee. A reference manual containing all critical ma- terials and explanations of all policies, procedures, and activities was provided asa guide to the orientation session. The training and the manual were intended to promote consistency among members and improve the reliability and validity of course and instructor evaluations.

Assessment Process The process of assessment was multidimensional,

including reviews by students and teaching peers and

Table 1. Effective Principles and Characteristics of a Pharmacy Curriculum Assessment Program6

Integrated into culture Ongoing and sustained Based on appropriate learning outcomes Reflects learning as multidimensional and integrated Considers experiences leading to outcomes Involves representatives from across the educational

community Part of several practices to promote change Used in reports to external stakeholders Undertaken in receptive, supportive, enabling environment Basis for funding/re-allocation decisions Directed by competent, trustworthy individuals Regularly evaluated

American Journal of Pharmaceutical Education 2011; 75 (8) Article 157.

2

self-reviews by course coordinators (Figure 1). Each as- sessment provided documentation of effective activities and recommendations for course improvement as well as verification that the delivered course content matched the course learning outcomes in a manner that supported the relevant professional competencies.11 Appropriate re- sources were provided or developed to make necessary changes to improve the course.

Each core course in the doctor of pharmacy curricu- lum was assigned to an assessment team consisting of a faculty team leader and 2 students from the curriculum committee. While most teams had only 3 members, addi- tionalfacultymemberscouldbeassignedtoteamsconduct- ing comprehensive course reviews. Whenever possible, pharmacy practice faculty members are assigned to pharma- ceutical science courses and vice versa to facilitate faculty- wide comprehension of the entire curriculum and to enhance content integration and sequencing.

The team leader was expected to attend the initial meeting of the course to assess presentation of the course content, syllabus, schedule, and course management ex- planations. New courses (those with a new instructor or coordinator) or those with recent managerial or instruc- tional difficulties underwent a comprehensive course re- view. The team leader or another faculty member attended 1ormore classmeetingseachweek.Anabbreviated course review was conducted for well-established courses with

few managerial or instructional concerns. These abbrevi- ated review courses were audited by the team leader every 2 to 3 weeks.

The 2 student committee members enrolled in the course were assigned to the course team. Students were a vital link in the assessment process, as they actually experienced the entire course and could provide the most informed opinions and insights. They were instructed to report their own observations and opinions and to com- ment on the general opinions of the class about the course. Students in each course collaborated to submit a single evaluation for that course.

Course coordinators prepared self-assessments of their roleinthecourseincludinganevaluationoftheireffectiveness in course management; reviews of the teaching contributions of other course instructors; a summary of student successes and failures; and recommendations to improve the course.

Mid-semester feedback about the course progress was solicited with an open-ended, extra-credit question(s) on a quiz or examination. A question or 2 was presented on a separate page of the examination and no student identi- fiers were included on that page to ensure anonymity. All students received a bonus of 1 or 2 points on their quiz or examination score even though there was no requirement to respond. Prior to analysis, the course coordinator tabulated the responses for changes in the course presentation that needed to be made immediately or in the future. A report on the mid-semester feedback activities was included in the course coordinator’s self-assessment.

At the end of the semester, the course coordinator and student members of the team forwarded their completed assessments to the team leader, who then prepared a single report synthesizing all comments and observations and in- cluding the University-required, general course/instructor evaluations. A summary of recommendations for course improvementwasmadebeforethefullreportwassubmitted to the curriculum committee 4 weeks after the end of the semester.

The curriculum committee held a comprehensive, semester-based review of assessments for all courses. Com- mittee members were provided a copy of the synthesized course evaluation, and each team leader was given 10 to 15 minutes to present the course overview. Course coordi- nators were encouraged to attend and participate in the general discussion, during which issues were addressed, explained,orclarified,andfinalrecommendationsforcourse improvement were made by the committee. The chair of the curriculum committee prepared the final course evalua- tion incorporating committee discussion and recommenda- tions. The final course evaluation was forwarded to the course coordinator and archived on Share Point (Microsoft Corporation,Redmond,WA).Thecommittee chairprepared

Figure 1. Curricular assessment process designed and imple- mented by the University of New Mexico College of Pharmacy Curriculum Committee.

American Journal of Pharmaceutical Education 2011; 75 (8) Article 157.

3

a brief summary of each course for presentation to the college’s executive administration.

The final stage of the course review cycle was to use the archived course review to develop materials for the next course offering (Figure 1). After preparing a draft syllabus and course schedule, the course coordinator met with a member of the curriculum committee to examine the recommendations made during the previous course review and to discuss changes for course improvement. The recommendations pertained to course content, struc- ture, management, or instructional strategies. Assistance in implementing changes to the course was provided as needed. The final course syllabus and schedule were reviewed and approved by the chair of the curriculum committee to ensure that all relevant recommendations had been implemented. The assessment program was integrated into core course preparation and is recognized as an ongoing quality-improvement effort.

11

EVALUATION AND ASSESSMENT A standardized course evaluation form was com-

pleted by each course coordinator and all team members for all core courses beginning in fall 2007 through spring 2010. All participants were instructed on how to use the form to promote consistency among evaluators and across semesters. The form was designed to evaluate courses from a variety of perspectives over the semester. The open-ended items initially included:

(1) Course policies and procedures: completeness of course syllabi, use of standardized syllabi format, and compliance with policies and procedures;

(2) Course content and relationship to learning out- comes: content and competencies match, learn- ing objectives addressed;

(3) Integration within the curriculum: appropriate placement within vertical integration, appropri- ate sequencing and horizontal integration with concurrent courses when appropriate;

(4) Skills: identifies that knowledge and skills are developed, practiced, and assessed;

(5) Student assessment: types and number of as- sessments linked to learning objectives, student performance, and advancement;

(6) Course coordinator performance review: course management skills;

(7) Summary of individual faculty teaching re- views: summarized from teaching evaluation forms and student instructor evaluations;

(8) Recommendations: specific recommendations and suggested changes for course improvement.

After2completereviewcycles(academicyears2007- 2008 and 2008-2009) of all courses, we determined that

there were no longer outstanding issues with the first 3 items on the course evaluation form. These items were removed in fall 2009 and replaced with 2 new items of interest to the college:

(1) Active learning: describe active-learning tech- niques observed

(2) Key assessments and key artifacts: specific ex- aminations or learning activities that serve as demonstration of a competency

Each instructor was observed and a review of teach- ing skills and ability was prepared. The teaching evalua- tion was included as part of the course review but also could be used as a standalone teaching review for purposes of annual performance, promotion, and tenure. Instructors were rated on verbal and visual presentations; handouts and assignments; student interactions; content; teaching techniques,andexaminationitems.Instructorperformance on each item was rated using a 5-point Likert-type scale for quantitative data with open-ended comments for a more qualitative assessment. All feedback was offered in a pos- itiveandconstructivemannerusingaplus/deltaapproach

12

(method for identifying items that are being done well and itemsthatcouldbeimproved) aimed atcoaching-andskill- development opportunities for teaching faculty members.

At the time of the assessment, the college had 28 core classroom courses, 23 of which were considered “mature” courses as they had been offered consistently for more than 3 years. The remaining 5 courses were more recent developments added to the core curriculum to meet Stan- dards 2007.

1 The content of the courses was mapped and included specific competencies of the revised curriculum.

The assessment-feedback loop brought about subtle changes to many areas of the curriculum. As areas that needed improvement were identified, solutions were de- veloped, implemented and tracked through the assessment process. Effective solutions required a greater level of co- ordination among many individuals than had occurred pre- viously, including the course coordinator, department chair, administration, other faculty and instructors, and students. The case study seen in Table 2 illustrates the extent of these individuals’ involvement needed to make effective changes.

Faculty input and involvement was solicited and con- cerns were addressed during the development process. While the majority of faculty members recognized the benefits of the plan, a few were skeptical, their main con- cerns being loss of autonomy, fear of centralized control, and erosion of academic freedom. The plan was presented as one of continuous quality improvement with faculty participation and feedback. Recalcitrant faculty members were asked to allow an audit of their course with the initial feedback provided for their review and discussion before dissemination. The annual course review then provided

American Journal of Pharmaceutical Education 2011; 75 (8) Article 157.

4

faculty members an opportunity to discuss and consider any revisions or changes proposed for implementation. Once they realized that the structured course design and coordinated curricular content presentation facilitated their instruction, their resistance abated.

As a result of the assessment process, faculty involve- ment with the curriculum increased. Attending a lecture in pharmacology, a pharmacy practice instructor identified and corrected outdated information presented on a drug

class. Duplications of instruction were noticed, leading to collaboration among instructors in course preparations. Solutions were attempted, successes or failures realized, and needed corrections made. Examples of corrections in- cluded relocating calculations from the first-year pharma- ceutical care laboratories to a single dedicated course and integrating additional review of the Top 200 drugs across 2 years of care laboratory with reinforcement in the Mecha- nism of Drug Action courses.

Table 2. Application of a Course Assessment Process in a Self-Care Therapeutics Course in the Doctor of Pharmacy Curriculum

Self-Care Therapeutics (Second-Year Pharmacy Course)

Assessment Findings and Recommendations Corrective Actions Taken

Course coordinator was overscheduled and unable to devote sufficient time to course development and management.

Curriculum committee chair met with department chair. The course was transferred to a receptive junior faculty member.

Course content did not match course learning outcomes and did not address professional competencies assigned to the course.

The new course coordinator received one-on-one coaching in matching content to learning outcomes. Competencies were integrated into the course, and a new syllabus and course schedule were drafted.

Significant portion of the course included “orphan” prescription products.

Prescription products were moved and integrated into the pharmacotherapeutics course content.

Course had 30 meetings and 18 instructors. Content lacked cohesiveness and continuity.

Recommendation made to reduce guest instructors to a maximum of 9. The revised course has 5 guest lecturers: 3 experienced faculty members and 2 outside specialists.

Examination questions lacked consistency and were mostly knowledge-level multiple choice.

Course coordinator received one-on-one instruction on how to link examination questions to learning outcomes and how to develop items at the problem-solving level.

Course should serve as an orientation to pharmacotherapeutics, providing students an opportunity to learn to integrate basic foundational sciences into solving patient-care problems.

Case studies were added to provide orientation to patient care. Course coordinator met with pharmacotherapeutics instructors to review examination item development.

Course content and schedule were not integrated with the PCL.

Course coordinator met with PCL coordinator to arrange a compatible schedule and develop learning activities to reinforce self-care concepts.

Course contained no active-learning exercises, only lectures and examinations.

Group projects were added, a brief research paper was assigned, and activities (pair-share, brief group activities) were incorporated into class time.

Results of Assessment Process d Junior faculty member has developed instructional skills – including syllabus development, content-learning objective-assessment integration, effective examination writing, and coordination of activities with other faculty members.

d Course is fully integrated into the curriculum as a key component supporting the professional competencies. d The course provides appropriate sequencing, supporting student learning outcomes. d Student course evaluations show strong course management and integration of materials. Examinations reflect content with no surprises.

d Students enter pharmacotherapeutics with critical-thinking skills to be successful. The most recent group to complete the revised Self-Care Therapeutics course is the first group in which all students passed Pharmacotherapeutics I on the first attempt.

Abbreviations: PCL 5 pharmaceutical care laboratory.

American Journal of Pharmaceutical Education 2011; 75 (8) Article 157.

5

Faculty participation in curriculum committee meet- ings continued to be encouraged. Non-members were wel- comedtopresentpetitionsfornewcourses,suggestchanges, or simply to sit in on committee meetings. Course coordi- nators were encouraged to attend the final course evaluation at the semester review of the curriculum, and their partici- pation in the final review served as a summative evaluation of the course.

Students became more positively involved in the cur- riculum. Students were full participating members of the curriculum committee, and their input was actively sought and their suggestions implemented as appropriate. For example, student comments regarding the amount of work for a single semester credit prompted a more in-depth re- view of course demands, leading to an increase in course credits.

The development of standardized syllabi provided students clarity on what was expected, allowing them to take more responsibility for their own learning. As a result, thenumberofstudentissuesrelatedtocoursemanagement, while not tracked, seemed to decline. Immediate and ob- jective student feedback was often given directly to mem- bers of the curriculum committee with confidence that it would be adequately addressed. One example involved a guest instructor who was editing his PowerPoint slides as he was presenting. After being notified by a student about this practice, a committee member subsequently ob- served and documented the inappropriate activity. Details of this event were then addressed in the course review and recommendations to discontinue this practice were made.

After implementation of the assessment process, many preceptors who had supervised and instructed stu- dents on introductory and advanced pharmacy practice experiences (IPPEs and APPEs) for several years reported that students were better prepared than in previous years. While comments were received from all areas of profes- sionalpracticeexperiences, theadvancedgroupofstudents generated the most feedback. Clinical preceptors com- mented that these students were better prepared and more ready to contribute during their APPEs. Community prac- tice preceptors reported a decrease in the need to coach students through basic concepts and tasks.

DISCUSSION The assessment process proved to be useful in that it

supported student learning by focusing on the educational program. Progress toward course integration, scaffolding, and sequencing was satisfactory and ongoing. Faculty in- struction also seemed to improve. While part of the im- provement may be attributable to the increase in faculty observation (Hawthorne effect), some is a result of the ready availabilityofresourcestodoabetterjob.Facultymembers

self-identified areas needing improvement and indepen- dently sought assistance in making changes. Communica- tion between the Pharmacy Practice and Pharmaceutical Sciences improved, and instructors from both groups attended courses sequenced with their own and then worked with each other to improve the overall educa- tional experience.

Changes made by course instructors in response to the feedback supported the appropriateness of curriculum changes. Students reported they felt well-prepared for the professional practice experiences and that the classroom and laboratory exercises are appropriate for their experi- ential learning needs. Studentsattaina deeper understand- ing of the material encountered in historically difficult classes when prerequisite material is presented to them in the appropriate learning sequence.

While improvement in student performance may be a result of many factors, no changes were made to the ad- missions process, core course requirements, or instructional faculty members. Because the only intentional variance in student instruction was the implementation of the curricular assessment plan, the plan was credited with having a signif- icant impact on student performance.

Course syllabi were standardized to provide consistent informationtostudentsaboutuniversityandcollegepolicies and procedures,and course goals and objectives incorporate and support the appropriate professional competencies of the college. The assessments confirmed that curricular course placement supported integration between courses, and sequencing appeared to be appropriate. The delivered content for mature courses matched the intended content, while newer courses are making progress in matching intended and delivered content

The assessment and feedback process implemented as part of the quality improvement plan for the curriculum has been successful. All participants — students, faculty and preceptors — reported improvement in and progress toward successfully integrating the classroom curriculum while optimizing student learning outcomes. Involvement of the curriculum committee in providing proactive solu- tions to identified issues is recognized and becoming established within the college.

The success of this curriculum development and as- sessment plan depended largely on 4 strategies. Courses are reviewed by an integrated team that includes faculty members, students,and thecourse coordinator;thereview included examination of both horizontal and vertical con- tributions to the PharmD curriculum; information from annual course reviews was constructively used in subse- quent course planning, management and instruction; and the willingness of faculty members and administration to embrace change was respectfully nurtured.

American Journal of Pharmaceutical Education 2011; 75 (8) Article 157.

6

CONCLUSIONS Thispaper presents acomprehensive curriculum devel-

opment plan in which instructional content was monitored. The curricular assessment model met its established goals. By design, the course assessment program was systematic, ongoing, and comprehensive. Implemented annually in each course by a team of faculty members and students, it incorporated multiple perspectives and specific feed- back for maintaining and improving quality within and across courses. A key component of this program was the annual course review, which completes the assessment cycle and ensures implementation of recommendations. As carried out in our setting, assessment resulted in mean- ingful improvements not only in teaching and learning but also in preparing student pharmacists for pharmacy practice experiences and for professional practice after graduation.

REFERENCES 1. Accreditation Council for Pharmacy Education. Accreditation Standards and Guidelines for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree (“Standards 2007”). http://www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf. Accessed August 20, 2011. 2. Kogan JR, Shea JA. Course evaluation in medical education. Teach Teach Educ. 2007;23:251-264.

3. Duncan-Hewitt W, Jungnickel P, Evans RL. Development of an office of teaching, learning and assessment in a pharmacy school. Am J Pharm Educ. 2007;71(2):Article 35. 4. Piascik P, Bird E. Creating and sustaining a culture of assessment. Am J Pharm Educ. 2008;72(5):Article 97. 5. Ho SSS, Kember D, Lau CBS, Au Yeung MYM, Leung DYP, Chow MSS. An outcomes-based approach to curriculum development in pharmacy. Am J Pharm Educ. 2009;73(1):Article 14. 6. Abate MA, Stamatakis MK, Haggett RR. Excellence in curriculum development and assessment. Am J Pharm Educ. 2003;67(3):Article 89. 7. Anderson HM, Anaya G, Bird E, Moore DL. A review of educational assessment. Am J Pharm Educ. 2005;69(1):Article 12. 8. Lattuca LR, Stark JS. Shaping the College Curriculum: Academic Plans in Context. 2nd ed. San Francisco: John Wiley & Sons; 2009. 9. 2004 CAPE Educational Outcomes, Supplements, American Association of Colleges of Pharmacy. http://www.aacp.org/ resources/education/Pages/CAPEEducationalOutcomes.aspx. Accessed August 20, 2011. 10. Perrier DG, Winslade N, Pugsley J, Lavack L. Designing a pharmaceutical care curriculum. Am J Pharm Educ. 1995;59(2):113-125. 11. Banta TW. Moving assessment forward: enabling conditions and stumbling blocks. In: Gray PJ, Banta TW, eds. The Campus-Level Impact of Assessment: Problems, Progress, and Possibilities: New Directions for Higher Education #100. San Francisco: Jossey-Bass; 1997: 79-91. 12. Helmiski L, Koberna S. Total quality in instruction: a systems approach. In: Roberts HV, ed. Academic Initiatives in Total Quality for Higher Education. Milwaukee, WI: ASQC Quality Press;1995: 309-362.

American Journal of Pharmaceutical Education 2011; 75 (8) Article 157.

7

Copyright of American Journal of Pharmaceutical Education is the property of American Association of

Colleges of Pharmacy and its content may not be copied or emailed to multiple sites or posted to a listserv

without the copyright holder's express written permission. However, users may print, download, or email

articles for individual use.