2 DISCUSSION QUESTIONS

ASSESSMENT650.pdf

Home >Education homework help >2 DISCUSSION QUESTIONS

UPdate Progress, Trends, and Practices in Higher Education

Strategic Planning and Decision Making in Higher Education: What Gets Attention and What Doesn’t Michael C. Choban, Gary M. Choban, David Choban

W HY IS THE IMPLEMENTATION OF LEARNING OUTCOMES ASSESSMENT IN

higher education proceeding so grudgingly? And when assessment results are attained, why are they so rarely integrated into the strategic planning process?

These questions remain prominent in spite of over a decade of intensive effort on the part of educational accrediting agencies to facilitate outcomes assessment. The American As- sociation of State Colleges and Universities (2006) notes, “Seasoned observers have pointed out the irony of the academy, as an institution dedicated to discerning the truth through evidence, being so seemingly resistant to measuring quality through evidence. It is an irony that puzzles—and frustrates—a widening circle of stakeholders (p. 3).”

One widely held notion is that the major impediment is technical—finding the right measures or the right data collection and analysis methods. However, the use of ef- fective evaluation methodologies in other fields makes this notion questionable. It is all the more so when we recognize that many of the very methodologies used effectively in fields outside higher education were developed by educators and are often effec- tively applied by educators in their own professional disciplines. To be sure, techni- cal hurdles remain, but the existing general knowledge base on research methods, along with the more recent availability of education-specific tools such as electronic portfolios, standardized testing services, survey services, and assessment management software, makes the technical deficit explanation of education’s slow and sporadic progress dubious.

This article addresses these puzzling questions and suggests that motivational rather than methodological factors present the primary challenge. Our analysis is based on observations we have made at colleges and universities across the United States while working with faculty and administrators on assessment implementation strategies.

ARTICLES Strategic Planning and Decision 1 Making in Higher Education: What Gets Attention and What Doesn’t Michael C. Choban, Gary M. Choban, David Choban

Editor’s Notes 3 Trudy W. Banta

Curiosity: A Link to Assessing 5 Lifelong Learning Keston H. Fulcher

The Collegiate Learning 10 Assessment: A Critical Perspective Mark D. Shermis

COLUMN Assessment Measures 8 Gary R. Pike

March–April 2008 Volume 20, Number 2

Assessment

CONTENTS

Published online in Wiley InterScience (www.interscience.wiley.com)

Call for Contributions The editor welcomes short articles and news items for Assessment Update. Guidelines follow for those who would like to contribute articles on outcomes assessment in higher education.

• Content: Please send an account of your experience with assessment in higher educa- tion. Include concrete examples of practice and results.

• Audience: Assessment Update readers are academic administrators, campus assessment practitioners, institutional researchers, and faculty from a variety of fields. All types of institutions are represented in the readership.

• Style: A report, essay, news story, or letter to the editor would be welcome. Limited references can be printed; however, extensive tables cannot be included.

• Format: In addition to standard manuscripts, news may be contributed via letter, tele- phone, or fax (317) 274-4651. The standard manuscript format is a 60-space line with 25 lines per page. Word is preferred. Articles may be sent to <[email protected]> as a Microsoft Word attachment.

• Length: Articles should be four to eight typed, double-spaced pages (1,000– 2,000 words). Short news items and content for the Book Review section may be 100–500 words in length. Annotations of recent publications for the Resources feature should be about 50–100 words long.

• Copyright: Articles shall not have been registered for copyright or published elsewhere prior to publication in Assessment Update.

• Deadlines: Each issue is typically planned four months before its publication. Future deadlines for submitting articles are June 1 (September–October 2008 issue), August 1 (November–December 2008 issue), and October 1 (January–February 2009 issue).

Please address mailed contributions and comments to Trudy W. Banta, Editor, Assessment Update, Rm. 140 Administration Bldg., 355 N. Lansing St., Indianapolis, IN 46202-2896. �

If It Affects Personnel, It Matters

There is a relationship between the effect a decision is expected to have on an institution’s personnel and the amount of attention paid to the decision; and of course, what people ex- pect is based largely on what has been experienced in the past. Decisions that have had clearly perceptible and relatively immediate effects on personnel typically receive much attention. Decisions that have had no effect, nebulous effects, or substantially delayed ef- fects on personnel typically receive considerably less attention. With this observation in mind, institutional decisions can be usefully classified into three broad categories: (1) de- cisions that are expected to influence employees’ job convenience, comfort, or status; (2) decisions that are expected to affect institution, program, or individual security or sur- vival; and (3) decisions that are not expected to affect the lives of personnel significantly.

The consequences of decisions in the first category—convenience, comfort, and status issues—tend to be relatively immediate and perceptible to personnel. Hence, such deci- sions tend to generate considerable interest and attention. Decisions involving office space, work schedules, room assignments, or travel privileges are examples in this category. The consequences of decisions in the second category—security and survival issues—also tend to be relatively immediate and are poignantly perceptible to faculty and staff. Hence, de- cisions in this category tend to play an inordinately dominant role in institutional decision processes and planning. Decisions about budgets, fundraising, accreditation, student ad- missions and retention, or tenure and promotion are examples in this category.

Decisions related to program outcomes typically have had little effect on the lives of faculty and staff. Thus, consideration of the quality of program outcomes tends to be ne- glected in strategic planning. While considerable time and attention are currently devoted to assessment issues, observations suggest that the focus of attention is usually more on de- veloping strategies for satisfying accrediting agencies than on improving learning outcomes.

Assessment Update Progress, Trends, and Practices in Higher Education March–April 2008 Volume 20, Number 2

Editor Trudy W. Banta, professor of higher education and senior advisor to the chancellor, Indiana University– Purdue University Indianapolis (IUPUI)

Managing Editor Karen Elaine Black, director of program review, IUPUI

Assistant Editor Frances W. Oblander, educational consultant

Book Review Editor Susan Kahn, director of institutional effectiveness, IUPUI

Consulting Editors Peter T. Ewell, National Center for Higher Education Management Systems Thomas Anthony Angelo, Victoria University of Wellington, Wellington, New Zealand T. Dary Erwin, James Madison University Cecilia L. Lopez, Harold Washington College Marcia Mentkowski, Alverno College Jeffrey A. Seybert, Johnson County Community College Peter J. Gray, United States Naval Academy Gary R. Pike, IUPUI

Assessment Update: Progress, Trends, and Prac- tices in Higher Education (Print ISSN 1041-6099; on- line ISSN 1536-0725 at Wiley Interscience, www. interscience.wiley.com) is published bimonthly by Wiley Subscription Services, Inc., A Wiley Company, at Jossey-Bass, 989 Market St., San Francisco, CA 94103- 1741. Periodicals Postage Paid at San Francisco, CA, and additional mailing offices. Individual subscriptions are $127 per year (institutional $162). Back issues are available in limited supply at $29 per issue. To order, phone toll-free (888) 378-2537 or fax (888) 481-2665. Visit our Web site at www.josseybass.com. Postmaster: Send address changes to Assessment Update, Jossey- Bass, 989 Market St., San Francisco, CA 94103-1741.

Copyright © 2008 Wiley Periodicals, Inc., A Wiley Com- pany. All rights reserved. Reproduction or translation of any part of this work beyond that permitted by Section 107 or 108 of the 1976 United States Copyright Act without permission of the copyright owner is unlawful. Requests for permission or further information should be addressed to the Permissions Department, c/o John Wiley & Sons, Inc., 111 River St., Hoboken, NJ 07030; (201) 748- 8789, fax (201) 748-6326, www.wiley.com/go/ permissions.

(continued on page 13)

The Logic of Strategic Planning

Logically speaking, learning outcomes are the criteria against which strategic plans should be evaluated, given that effective student learning rests at the core of an edu- cational institution’s mission. Security, sur- vival, and employee convenience issues, though important, are logically subordinate; they are means to the end. What this means in practice is that decisions about survival or convenience issues should always be considered in light of their impact on out- comes. Strategic plans should be aimed at producing the best student learning in the most cost-effective way, not the most cost- effective program in the most convenient way. Adhering to this principle entails more than merely expressing concern for student learning; it means deliberate monitoring of program outcomes as adjustments in the delivery process are made. Strategic plan- ning is concerned with formulating a plan of action designed to maximize program effectiveness. But strategic planning with- out knowledge of program outcomes is an exercise in futility.

Decisions about student retention and admissions, which are motivated primar- ily by survival concerns, illustrate this point. Unilateral efforts to improve reten- tion could, for example, lead to inadver- tent grade inflation as the faculty strives to minimize failures. Over time, such eroded standards might in turn result in a significant decline in learning outcomes. Hence, without simultaneous attention to program outcomes, short-term improve- ments in retention might be attained at the expense of long-term program quality. Over time, the decline in program quality might come full circle and negatively af- fect admissions and perhaps retention.

Similarly, changes in admission strate- gies, also motivated primarily by survival concerns, could affect program outcomes either positively or negatively. As the com- position of the student body changes as a result of changing admission strategies,

program outcomes may also change unless the program is appropriately adjusted. If such changes in outcomes go undetected, the need to make adjustments in the pro- gram in order to prevent or correct nega- tive outcomes is likely to go unaddressed.

In fact, at many academic institutions today, the verbal comments of personnel and the written text of the institution’s marketing literature promulgate a focus on student learning, while strategic plan- ning and decisions tend to be dominated primarily by program security and sur- vival issues. In other words, while logic and the institutional mission mandate a focus on student learning, the contingen- cies of survival bearing on faculty and staff still favor a focus on process (input) variables. There is still a relative eco- nomic advantage to focusing on inputs rather than outputs; impressive facilities and faculty credentials still sell better than nebulous outcomes. The prestige of both the institution and the faculty is still based more on inputs than outputs; thus, there is a vested interest in maintaining a focus on inputs. Evaluation of faculty for tenure and promotion is still focused more on teaching practices (inputs) than on the learning outcomes produced, thus provid- ing an incentive for remaining focused on inputs. Bluntly stated, the quality of learn- ing outcomes produced has had relatively little impact on the lives of program fac- ulty and staff and therefore still receives relatively little serious attention in strate- gic planning. Clearly, the contingencies of survival in higher education are rapidly changing (Commission on Higher Educa- tion, 2006), and some educators are be- coming very concerned; but rank-and-file faculty and staff have for the most part not yet been affected by the quality of the learning outcomes they produce.

Strategic Planning and Mission

The process of strategic planning has its roots in the world of business. In that set-

ting too, there is much focus on process (input) variables—the structures, actions, and resources needed. But unlike educa- tional institutions, for-profit businesses also have a clear, well-understood, and well-documented outcome that bears on company and employee well-being: fi- nancial gain. The criterion against which the effectiveness of a strategic plan is measured is tacitly understood by all and readily available for examination. Profit is the ultimate mission.

When strategic planning is applied in educational institutions, the process is often truncated by the absence of clearly defined and reliably documented outcomes. Goals are set, alternative action plans are dis- cussed, and a hypothesized best course of action is selected; but effectiveness of the selected plan is seldom evaluated against educational outcomes as defined by the in- stitutional mission. If the implemented strategic plan is evaluated at all, the crite- ria of success are likely to be process vari- ables in the survival category, such as financial health, student enrollments, and student retention, rather than the outcomes of student learning or community impact. If the business model of strategic planning is to make any sense in education, educa- tors will first have to develop the ability to clearly define and systematically monitor educational outcomes and come to under- stand the intrinsic relationship between strategic plans and program outcomes. In business, financial health is a valid criterion of success; in education, it is not.

The current tendency to focus on insti- tutional process rather than institutional mission is established at the highest level at most institutions. The success or failure of presidents at institutions of higher learning tends to be judged more in terms of the financial health of the institution than the degree of success at mission attainment, which is usually largely un- known. If this is so, it means that presi- dents have still more incentive to pursue highly visible process developments that

Strategic Planning and Decision Making in Higher Education (continued from page 2)

attract attention and financial support rather than more subtle and gradual learn- ing improvements.

Traditionally, an institution with a suc- cessful financial program could invest in impressive program inputs—equipment, buildings, prestigious faculty—and offer them to the public as evidence of institu- tional quality. Impressive inputs were and, for the most part, still are widely accepted as evidence of an effective program. While public expectations and standards of evaluation are changing, the effects of these changes have not yet significantly affected the behavior of many educators in regard to strategic planning. Currently, at many institutions, learning outcomes assessment is done in a perfunctory man- ner in response to accreditation require- ments and has little or no real effect on significant institutional decisions.

Accrediting agencies have put in place consequences associated with engaging in the process of outcomes assessment. But this is not tantamount to having conse- quences associated with the quality of the learning outcomes attained. The former requires only that faculty and staff go through the motions of doing outcomes assessment; the latter requires that faculty and staff actually pursue improvements in learning outcomes. This distinction is subtle but critical. As long as concern over accreditation continues to be the pri- mary motivation behind outcomes assess- ment, outcome results will likely remain on the margins of faculty and staff atten- tion. When the quality of attained learn- ing outcomes begins to affect the lives of program faculty and staff, outcomes as- sessment will take center stage and strate- gic planning will focus on student learning as the criterion of success.

Growing Public Demand for Evidence of Effectiveness

Higher education institutions are currently experiencing a growing demand from the public for evidence of the quality of pro- gram outcomes and a concomitant recog-

nition that traditionally touted process in- dicators of quality are inadequate (Com- mission on Higher Education, 2006). Encouragingly, a growing number of fac- ulty and administrators also is beginning to recognize these changing conditions and the inadequacy of the status quo (Board of Directors of the Association of American Colleges and Universities, 2006). The press for evidence of program effectiveness is likely to continue. As the public’s press for evidence of effective- ness continues and becomes more fo- cused, the current practice of merely expressing concern for student learning while pursuing process development will become increasingly inadequate. Institu- tional survival will become more depen- dent on evidence of program effectiveness in terms of learning outcomes and com- munity impact. In the past, there were few significant consequences associated with the quality of outcomes, and educators responded accordingly. In the not-too- distant future, the consequences associ- ated with the quality of outcomes may well be profound.

Addressing Learning Outcomes: An Immediate Challenge

In the emerging environment of public demand for evidence of program effec- tiveness, failing to consider learning out- comes as an important factor in the strategic planning process is likely to be- come increasing costly for higher educa- tion institutions. Administrators who recognize this shift in survival contingen- cies and are among the first to start inte- grating learning outcomes in the strategic planning process will likely realize sig- nificant benefits from their initiative. The immediate challenge is to find ways to make the quality of program outcomes matter in the sense that the outcomes achieved have clear and immediate con- sequences for faculty and staff. Consistent with good behavior management practice, consequences should be such that they reinforce outcome improvements and

achievements rather than punish deficien- cies. Administrators will need to have the foresight to recognize that in the near fu- ture, consumers will associate significant consequences with program outcomes, so immediate attention to outcomes is im- perative. Faculty will have to recognize that pursuing improvement in learning outcomes now is ultimately in their best interest. Institutional resource allocation, faculty and staff evaluation, and public recognition of attained outcomes suggest themselves as possible opportunities to apply outcome-based incentives. Virtually all faculty and staff say they are passion- ately concerned about student learning outcomes, but until they are encouraged to act systematically on those alleged con- cerns, the connection between strategic planning and institutional mission will re- main fictitious and the health of our insti- tutions will be in jeopardy. �

References American Association of State Colleges

and Universities. “Value-added assess- ment: Accountability's new frontier.” Perspectives, Spring 2006, 1–16.

Board of Directors of the Association of American Colleges and Universities. “Academic Freedom and Educational Responsibility.” Liberal Education, Spring 2006, 92(2), 6–13.

U.S. Department of Education. A Test of Leadership: Charting the Future of U.S. Higher Education. Washington, D.C., 2006.

Michael C. Choban is associate pro- fessor of psychology at West Virginia Wesleyan College in Buckhannon, West Virginia. Gary M. Choban is chief technology officer and David Choban is director of professional services at Nuventive in Allison Park, Pennsylvania.