a paper

The Sociological Roots of Utilization-Focused Evaluation

Michael Quinn Patton1

Published online: 5 June 2015 # Springer Science+Business Media New York 2015

Abstract Utilization-focused evaluation is widely recognized as one of the most influential approaches in the evaluation profession. What is not widely recognized is that the approach is fundamentally applied sociology. Six particular sociological specializations have influenced my evaluation perspective and practice: applied soci- ology, sociology of knowledge, diffusion of innovations, sociological perspectives on power and conflict, organizational sociology, and qualitative methods. International and comparative sociology has also been important in my evaluation career and practice.

Keywords Program evaluation . Sociology of knowledge . Diffusion of innovations .

Power. Organizational sociology. Qualitative methods . Utilization-focused evaluation .

Developmental evaluation

Utilization-Focused Evaluation is now in its 4th edition (Patton 2008). The book has been used in over 500 universities worldwide in the 35 years since the first edition appeared (Patton 1978). More than 10,000 evaluators, both practicing and aspiring, have participated in training workshops on utilization-focused evaluation sponsored by the American Evaluation Association, The Evaluators’ Institute, the International Program in Development Evaluation Training, and state, national, and international organizations and evaluation associations around the world. Essentials of Utilization- Focused Evaluation was published in Patton (2012) with special attention to evaluation in complex dynamic systems. Utilization-focused evaluation is widely recognized as one of the most influential approaches in the evaluation profession. What is not widely recognized is that the approach is fundamentally applied sociology.

First I’ll briefly describe utilization-focused evaluation (U-FE) then examine its sociological roots.

Am Soc (2015) 46:457–462 DOI 10.1007/s12108-015-9275-8

* Michael Quinn Patton [email protected]

1 Utilization-Focused Evaluation, Saint Paul, MN, USA

What is Utilization-Focused Evaluation?

U-FE been begins with the premise that evaluations should be judged by their utility and actual use; therefore, evaluators should facilitate the evaluation process and design any evaluation with careful consideration of how everything that is done, from beginning to end, will affect use. Use concerns how real people in the real world apply evaluation findings and experience the evaluation process. Therefore, the focus in utilization-focused evaluation is on intended use by intended users. Since no evaluation can be value-free, utilization-focused evaluation answers the question of whose values will frame the evaluation by working with clearly identified, primary intended users who have the responsibility to apply evaluation findings and implement recommendations.

Utilization-focused evaluation is highly personal and situational. The evaluation facilitator develops a working relationship with intended users to help them determine the kind of evaluation they need. This requires negotiation in which the evaluator offers a menu of possibilities within the framework of established evaluation standards and principles. Utilization-focused evaluation is a process for making decisions about an evaluation’s priority purpose and design in collaboration with an identified group of primary users focusing on their intended uses of the evaluation.

A sociology of use undergirds and informs utilization-focused evaluation: Intended users are more likely to use evaluations if they understand and feel ownership of the evaluation process and findings; they are more likely to understand and feel ownership if they’ve been actively involved; by actively involving primary intended users, the evaluator is training users in use, preparing the groundwork for use, and reinforcing the intended utility of the evaluation every step along the way.

Sociological Roots and Foundations of U-FE

Studying sociology, from undergraduate work at the University of Cincinnati through my doctorate at the University of Wisconsin, Madison, gave me a solid grounding in quantitative methods, theory construction, philosophy of science, and sociology of knowledge, all of which have influenced my evaluation thinking and practice. Sub- stantively, six particular sociological specializations have influenced my evaluation perspective: (1) applied sociology grounded in the sociological imagination; (2) soci- ology of knowledge, especially social constructionism; (3) diffusion of innovations; (4) sociological perspectives on power and conflict; (5) organizational sociology, which examines how people behave in institutions and organizations, much of which transfers to behavior in social change programs and interventions, the primary arena of evalu- ation studies; and (6) qualitative sociology.

Applied Sociology Grounded in the Sociological Imagination

Sociologiast Edward Suchman (1967) began his pioneering text on evaluation research with the observation that Bone of the most appealing ideas of our century is the notion that science can be put to work to provide solutions to social problems^ (p. 1). In talking over the years with sociologist-trained evaluators, a common theme is the

458 Am Soc (2015) 46:457–462

inspiration of C. Wright Mills who articulated a vision of applied sociology contribut- ing to illumination of critical public issues. U-FE explicitly positions evaluation use as a core sociological issue of the information age, identifiable as such because it Binvolves a crisis in institutional arrangements^ (Mills 1961:8–9). Sociology’s pioneers shared a commitment to analyzing and illuminating societal problems: Durkem, Weber, Marx, Sorokin, Veblen, and Ward, among others. They offered inspiration, as did those who followed with specific applied sociology contributions: Paul F. Lazarsfeld, Robert K. Merton, James Coleman, Amitai Etzioni, Al Lee, and Alva and Gunnar Myrdal, to name but a few prominent examples. Sociologists who pioneered evaluation research as an arena for applied sociology include Carol Weiss, Edward Suchman, Peter Rossi, Howard Davis, and Susan Salasin. These applied sociologists, whose work illuminated public issues, contributed to public policy formulation, and laid the foundations for the field of evaluation research, were exceptions in a discipline Mills attacked as mired between the extremes of abstracted empiricism and theoretical irrelevance. As has been reiterated periodically by scholars throughout history, we surely stand on the shoulders of giants (Bernard of Chartres, 12th century; Sir Isaac Newton, 17th century). These giants of applied sociology have both inspired and fertilized my evaluation career, independent consulting practice, and scholarship.

Sociology of Knowledge, Especially Social Constructionism

Thomas Theorem: BIf men define situations as real, they are real in their consequences^

Sociologists William I. Thomas and Dorothy Swaine Thomas

(1928, p. 572).

This basic social psychological theorem, that what is perceived as real is real in its consequences, has had enormous influence as a distinct perspective informing evalu- ation research. A constructionist evaluator expects that different stakeholders involved in a program (e.g., staff, clients, families of clients, administrators, funders) will have different experiences and perceptions of the program, all of which deserve attention and all of which are experienced as real. The constructionist evaluator captures these different perspectives through open-ended interviews and observations, and then ex- amines the implications of different perceptions (or multiple "realities"). The construc- tionist evaluator compares various program participants’ perceptions with each other and with those of funders or program staff. The analysis could include interpreting the effects of difference perceptions and experiences on attainment of stated program goals. For example, welfare recipients who perceive the program as an opportunity to get out of poverty may have different experiences and outcomes from those who view the program as merely meeting a requirement to keep receiving welfare payments. The inquiry would include looking at what shapes these different experiences and percep- tions as reported by participants themselves. The sociology of knowledge classic, The Social Construction of Reality, by Berger and Luckmann (1967), is the foundation for

Am Soc (2015) 46:457–462 459

valuing the diverse perspectives of multiple stakeholders in program evaluation gener- ally and U-FE specifically.

Diffusion of Innovations

The diffusion of innovations literature, a major focus of inquiry in rural sociology, informed my first empirical inquiry into evaluation use, an inquiry that was the basis for the first edition of Utilization-Focused Evaluation (Patton 1978). We basically did case studies of evaluations to find out what characteristics were associated with use. Sociologist Everett Rogers’ influential work on diffusion of innovations offered a conceptual framework for thinking about and studying evaluation use that prevails to this day.

Sociological Perspectives on Power and Conflict

Another root sociological influence has been a theory of power that I have found instructive in helping me appreciate what evaluation offers stakeholders and intended users. Under- standing this has helped me explain to intended users how and why their involvement in a utilization-focused evaluation is in their own best interest. It provides a basis for under- standing how knowledge is power and led me to the following premise: Use of evaluation will occur in direct proportion to its power-enhancing capability. Power-enhancing capa- bility is determined as follows: The power of evaluation varies directly with the degree to which the findings reduce the uncertainty of action for specific stakeholders. This view of the relationship between evaluation and power is derived from the classic organizational theories of Michael Crozier (1964) and James Thompson (1967).

For example, Thompson (1967) theorized that organizations are open systems that need resources and materials from outside and that Bwith this conception the central problem for complex organizations is one of coping with uncertainty^ (p. 13). He found that assessment and evaluation are used by organizations as mechanisms for reducing uncertainty and enhancing their control over the multitude of contingencies with which they are faced. Information for prediction is information for control—thus the power of evaluation. To be power laden, information must be relevant and in a form that is understandable to users. Crozier (1964) recognized this qualifier in linking power to reduced uncertainty: BOne should be precise and specify relevant uncertainty. People and organizations will care only about what they can recognize as affecting them and, in turn, what is possibly within their control^ (p. 158).

Reducing uncertainty is not a concern unique to managers and administrators. Policy- makers, social innovators, philanthropic funders, and other users of evaluation findings for decision-making want to act with as much certainty as the data allow. Utilization-focused evaluation, in particular, brings together multiple stakeholders with diverse perspectives to ensure the relevance, credibility, and utility of the results, including making judgments about program effectiveness, learning key lessons, accountability to funders and the public, and illuminating the future development of new interventions.

Organizational Sociology

Organizational sociology laid the foundation for attention to the application of systems thinking and complexity concepts to evaluation, and the identification of what I’ve

460 Am Soc (2015) 46:457–462

called developmental evaluation as a specific niche in utilization-focused evaluation (Patton 2011). Developmental evaluation supports innovation development to guide adaptation to emergent and dynamic realities in complex environments. Innovations can take the form of new projects, programs, products, organizational changes, policy reforms, and system interventions as well as innovations in science and technology, areas in which my sociology mentor, Jerald Hage (1970, 1999, 2011), has contributed significantly. A complex system is characterized by a large number of interacting and interdependent elements in which there is no central control; self-organizing and emergent behaviors based on sophisticated information processing generate learning, evolution, and development. Complex environments for social interventions and inno- vations are those in which one is uncertain about what to do to solve problems and key stakeholders are in conflict about how to proceed. Informed by systems thinking and sensitive to complex nonlinear dynamics, developmental evaluation supports social innovation and adaptive management. Evaluation processes include asking evaluative questions, applying evaluation logic, and gathering real-time data to inform ongoing decision making and adaptations. The evaluator is often part of a development team whose members collaborate to conceptualize, design, and test new approaches in a long-term, ongoing process of continuous development, adaptation, and experimenta- tion, keenly sensitive to unintended results and side effects. The evaluator’s primary function in the team is to infuse team discussions with evaluative questions, thinking, and data and to facilitate systematic data-based reflection and decision making in the developmental process. Developmental evaluation as a distinct niche emerged in response to one of my client’s questions and needs

Qualitative Sociology

My book on Qualitative Research and Evaluation Methods (Patton 2015) is now in its fourth edition. Using qualitative methods, grounded in qualitative sociology and the field methods of sociologists like Howard Becker and William Foote Whyte, has become a mainstay of utilization-focused evaluation. Qualitative evaluation methods include in-depth interviewing, external and internal (participant) observation, document and discourse analysis, focus groups, case studies, and mixed methods. Indeed, qual- itative evaluation and in-depth case studies were utilization-focused methodological responses to the kinds of evaluation questions stakeholders were asking and the criteria they applied to judge quality of findings: contextual understanding, in-depth analysis, and cross-case comparisons.

International Sociology

In the past few years, I’ve had extraordinary opportunities to be part of evaluation conferences and professional associations throughout the world. Evaluation has be- come a global profession and 2015 has been designated by the United Nations as the International Year of Evaluation. Our colleagues from around the world are infusing the profession with new energy as we learn different ways of looking at what we do and face challenges about how to translate some of what we do for the rest of the world. I find myself challenged to think more deeply about the cultural values and biases of Western approaches to evaluation and to include attention to cultural diversity and

Am Soc (2015) 46:457–462 461

sensitivity as important factors affecting evaluation quality. Sociology provides the foundation for such inquiries and sensitivities.

References

Berger, P., & Luckman, T. (1967). The social construction of reality: a treatise in the sociology of knowledge. Garden City, NY: Anchor.

Crozier, M. (1964). The bureaucratic phenomenon. Chicago: University of Chicago Press. Hage, J. (1970). Social Change in Complex Organizations. New York: Random House Hage, J. (1999). Organizational innovation. Hanover, NH: Dartmouth Pub Co. Hage, J. (2011). Restoring the innovative edge: driving the evolution of science and technology. Palo Alto,

CA: Stanford Business Books. Mills, C. W. (1961). The sociological imagination. New York: Grove. Patton, M. Q. (1978). Utilization-focused evaluation (1st ed.). Beverly Hills: Sage. Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks: Sage. Patton, M. Q. (2011). Developmental evaluation: applying complexity concepts to enhance innovation and

use. New York: Guilford Press. Patton, M. Q. (2012) Essentials of utilization-focused evaluation. Thousand Oaks, CA: Sage Publications. Patton, M. Q. (2015) Qualitative Research and Evaluation Methods, 4th ed. Thousand Oaks, CA: Sage

Publications. Suchman, E. (1967). Evaluative research. New York: Russell Sage. Thompson, J. D. (1967). Organizations in action. New York: McGraw-Hill. Thomas, W. I., & Thomas, D. (1928). The child in America. New York: Knopf.

462 Am Soc (2015) 46:457–462