a paper

Book Reviews

Book Reviews

Michael Quinn Patton (2012). Essentials of Utilization-focused Evaluation. Thousand Oaks, CA: Sage. 461 pp. $61.00 (paperback), ISBN: 9781412977418.

Reviewed by: Donna Podems, Centre for Research on Evaluation, Science and Technology (CREST), Stellenbosch University, Stellenbosch, South Africa DOI: 10.1177/1098214013496627

As an evaluation practitioner who draws heavily on Patton’s utilization approach, an academic who

lives outside the proverbial Ivory Tower, and a former student of Michael Patton, I avidly read his

book, Essentials of Utilization-Focused Evaluation.

For readers unfamiliar with utilization-focused evaluation (U-FE), I present a short history. The

research on evaluation use on which U-FE was based began in the early 1970s. The first edition of

U-FE was published in 1978; the second, revised edition was published in 1986; a third, expanded

edition appeared in 1997; and the fourth edition appeared in 2008. Each new edition drew upon

emerging research on evaluation and knowledge use, case examples of U-FE in the field, and

advances in the theory and practice of evaluation in general. The profession of evaluation has grown

tremendously in the four decades since Patton conducted his initial research on evaluation use. Espe-

cially noteworthy has been the growth and recognition of evaluation globally. Essentials of

Utilization-Focused Evaluation is Patton’s first attempt to condense the content into a shortened ver-

sion, and it incorporates recent changes in the evaluation field.

While I initially read Essentials of Utilization-Focused Evaluation (hereafter referred to as Essen-

tials) to see what the essentials were according to Patton, I reviewed the book with three specific inter-

ests. First, I wanted to understand how well this book addressed past criticisms of his other U-FE

books, and, related to this, how this book adds value to the previous ones. Second, I was interested

in how useful this book will be for practitioners who want to apply the approach. Is it a companion

guide to the fourth edition or does it stand on it its own? The third focus was to review the book’s

potential to provide students and others new to the approach with a firm foundation in U-FE.

While this review is informed mainly by my interpretation and experience of Essentials, I also

engaged several key informants who have read the book. These practicing evaluators, teachers and

students of evaluation, provided rich insights and ideas, which influenced this review and, hopefully,

resulted in a more holistic perspective.

A place to start the review is to explore the similarities with his other U-FE books. Patton’s

charming style persists with his usual mastery of writing. He engages the reader by mixing aca-

demic concepts and terminology with more commonly used nonacademic words and sayings

(e.g., ‘‘stuff happens’’). He also peppers the text with humor (e.g., ‘‘big hairy goal,’’ a term

found in other U-FE books), at times writing as if he is speaking directly to you (‘‘I am glad

you asked’’), and often relates nonevaluation history, facts, and stories to introduce evaluation

topics (such as the use of the Janus symbol, which I return to later). This style is appropriate for

all levels of evaluators. In addition, as a practitioner, I especially appreciate the candor with

which Patton writes. As with his other books, new and novice evaluators need to be aware that

American Journal of Evaluation 2014, Vol. 35(1) 154-159 ª The Author(s) 2013 Reprints and permission: sagepub.com/journalsPermissions.nav aje.sagepub.com

at University of North Carolina at Chapel Hill on February 4, 2016aje.sagepub.comDownloaded from

what is often presented so well by Patton, and therefore appears to be quite simple, is often

more complicated in practice.

Some diehard U-FE fans may recognize some of the charts, stories, and examples, and have

moments of deja vu. However, Patton has interspersed his Essentials text with new examples and

stories, which add to the overall feel of the book. While his use of exhibits and charts is not new,

he has provided several new ones that help clarify and update elements of U-FE.

What one will notice almost immediately is that the number of U-FE steps has increased, and the

book is organized in a different way, with a chapter devoted to each step. There is a bit of irony in the

fact that the ‘‘essentials’’ of U-FE include five additional steps. The U-FE process originally had

5 steps, grew to 12, and Essentials presents 17 exhaustive (and somewhat exhausting) steps. Further,

while many of the steps with which a U-FE user is familiar are described somewhat differently, the

essence and most of the content remain the same. As Patton notes in his preface, ‘‘Despite a lot that

is new, though presented in fewer pages, the basic message remains unchanged’’. The following

sections explain some of the key differences in Essentials.

Comparison of U-FE’s (2008) 12 Steps With Essential’s 17 Steps

Several of the added steps are newly introduced to the U-FE process, but most are present in the

2008 U-FE process, subsumed within other steps. For example, in the 12-step framework, Step 5

is presented for identification of primary intended uses. In the 17-step framework, it involves two

steps: identifying intended uses of the evaluation findings (Step 5) and identifying intended uses

of the evaluation process (Step 6). This change reflects the increasing importance of process use

as distinct from findings use, and Essentials makes that distinction.

A second example is found in Step 6 in the 12-step framework, ‘‘focusing the evaluation.’’ The

17-step framework divides that step into three: prioritizing evaluation questions, checking that fun-

damental issues are addressed, and theory of change work. These more distinct steps are still central

to the giant step of focusing the evaluation. A third example is Step 7 in the 12-step framework,

designing the evaluation. That step also becomes two in the 17-step framework: negotiating methods

and reviewing methods debates (as part of that negotiation).

Finally, the 12-step framework had steps on data analysis (Step 10) and facilitating use (Step 11).

The 17-step framework revises these two steps into three: data presented for user engagement, report

produced, and follow-up with users to facilitate use. Implicit in the 12 steps, and a major explicit

emphasis of the 17 steps, is that U-FE is not a set of linear steps for which you finish one and then

move on to the next. Rather, it is a matter of moving back and forth through the steps.

What Is Brand New

Some new additions to the U-FE process appear to result from Patton’s evolving thinking on his the-

ory and how to present it, such as the explicitly iterative U-FE process. Other changes are influenced

by the evolving thinking in evaluation, such as how the concept of complexity influences evaluation,

and some changes are in response to criticism of his 2008 book and its theory.

The overall graphic of the 17 steps introduces the reader to the U-FE process as soon as he or she

opens the book, and the ‘‘interludes’’ between chapters emphasize that U-FE is nonlinear and itera-

tive. Patton provides checklists in each chapter and cautions the reader about the process being nei-

ther sequential nor linear. He explains, ‘‘There are interconnections among the steps, feedback

loops, and the complex dynamics that affect any emergent system . . . the U-FE steps don’t just get completed and then you move on’’ (pp. 59–60). In typical Patton style, he provides a historical sym-

bol, Janus, to emphasize his point. Janus is the Roman god of ‘‘gates, doors, beginnings, endings and

time.’’ For Patton, this symbol represents U-FE’s nonlinear, interactive, dynamic, and complex

Book Reviews 155

at University of North Carolina at Chapel Hill on February 4, 2016aje.sagepub.comDownloaded from

nature. The symbol is used throughout the book to serve as an ongoing reminder of U-FE’s approach

(pp. xviii–xix).

Patton has added sections on exploring the context for U-FE, the global use of evaluation, and

how U-FE addresses and embraces complexity and systems thinking. While I was encouraged to see

the book acknowledge terms such as M&E (monitoring and evaluation), his interpretation and

description of M&E could have been expanded to include a more comprehensive understanding. For

example, his short entry on M&E in the table, ‘‘Alternative Ways of Focusing Evaluations’’ (p. 185),

describes it simply as ‘‘integrating monitoring and evaluation.’’ However, other descriptions and

understandings exist. Providing additional details or references on M&E would have added value

for evaluators who work in the international context, where the M&E term dominates the field of

evaluation.

The book offers practical exercises at the end of each chapter. These are not exercises for which

one can go back and find the answer in the book, but rather applications that require a person to

reflect and think. This is in keeping with the U-FE approach, which provides a framework requiring

reflection, critical thinking, and flexibility in order to apply it appropriately.

Some of Patton’s key changes appear to result from his reflection and engagement with past criti-

cisms, particularly the length of the 2008 U-FE book. When holding Essentials, one gets the first

indication that Patton has attempted to address the past critique that the fourth edition of U-FE was

too long (for reading) and too heavy (to travel with). Essentials is almost 200 pages shorter than

U-FE (Patton, 2008). If the goal was to bring down page numbers without losing the essence needed

to implement U-FE, Patton appears to have succeeded. At the same time, if the intent was a book

with only the bare essentials, this book falls a bit . . . long. I asked Patton about the length of Essentials, and he replied:

I actually set out to write a 100-page primer. That was my intent. Really. But I couldn’t do it. Once I

made the complex dynamic nature of U-FE a central emphasis, then revised the steps from 12 to 17

to better capture the nuances of the highly interactive process, and then added new exhibits and cartoons,

and . . . Let me just admit it. Brevity is not my strong suit.

An additional criticism relating to the 2008 edition is that one gets lost in the U-FE process. Pat-

ton tackles this in Essentials. On the inside cover of the book, as well as at the back, there is a map

that illustrates the U-FE approach and its 17 steps. By providing this, Patton addresses a key com-

plaint regarding the recent versions of his U-FE books, which did not clearly depict the approach.

This accessible map allows the reader to move confidently between the U-FE steps and the corre-

sponding chapters. Thus, placing this map on the inside front cover and at the book’s end offers two

immediate benefits: an accessible overview of the approach and a clear reference that can guide

learning about, or applying, the approach.

One further criticism of the book relates not to Patton’s writing or his intellectual content, but

rather to the printing. The blue printed charts and cartoons throughout the book are slightly blurry

and sometimes hard to read.

Useful to Whom?

For an evaluation book that is premised on use, it seems only appropriate to conclude by addressing

the book’s usefulness to different levels of evaluators, and for different users. Anyone new to eva-

luation would benefit from reading Chapters 1–4, which prepare novice evaluators for what they

need to consider before, and eventually when they enter, the first evaluation meeting.

For more experienced evaluators and those familiar with the 2008 U-FE process, this book offers

a refined approach that streamlines the U-FE process, despite the additional steps. At the same time,

I found it useful as a companion guide to the longer 2008 book, which provides more examples and a

156 American Journal of Evaluation 35(1)

at University of North Carolina at Chapel Hill on February 4, 2016aje.sagepub.comDownloaded from

wider exploration of evaluation in general. Further, this book is particularly useful for those who rely

on self-learning; Essentials is almost like a private lecture just for you, compliments of Michael

Patton.

Essentials provides an updated and refined U-FE process that reflects the evolving nature of the

evaluation field and evaluation use, addresses past criticisms, and provides a process that is more

easily followed, iterative as it is. The 17 steps, rather than making the process more complicated,

make Essentials more useful as either a stand-alone book or a companion to the 2008 U-FE. In

writing this book, Patton appeared to follow the wisdom of Albert Einstein, who said, ‘‘Make every-

thing as simple as possible, but not simpler.’’ Patton has done this with Essentials of utilization-

focused evaluation.

Reference

Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage.

King, J. A., & Stevahn, L. (2013). Interactive Evaluation Practice: Mastering the Interpersonal Dynamics of Program Evaluation. Thousand Oaks, CA: Sage. 456 pp. $40.00 (paperback), ISBN 9780761926733.

Reviewed by: Melanie Hwalek, SPEC Associates, Detroit, MI, USA DOI: 10.1177/1098214013504023

How refreshing! An easy-to-read, scholarly book that is rich in both theory and practical advice, and intensely focused on the interpersonal aspect of evaluation practice. Interactive Evaluation Practice:

Mastering the Interpersonal Dynamics of Program Evaluation is the first book I have found in the

evaluation literature that takes an in-depth look at why and how evaluators should be intentional and

take responsibility for ‘‘engaging people in making decisions, taking action and reflecting while con-

ducting an evaluation study’’ (p. 174).

King and Stevahn provide a thorough analysis of the interpersonal dynamics involved in

conducting a program evaluation, with particular emphasis on the facilitative roles of the evaluator.

The book is grounded in social psychological theory and backed by research on evaluation use. It

provides a plethora of best practice examples, templates, and frameworks that show how evaluators

can enhance their evaluation practice by paying particular attention to interpersonal dynamics.

Throughout the book, King and Stevahn use delightful images and analogies that make it fun to read.

Whoever thought of an evaluation being an emergent piece of art? Whoever considered the evaluator

as a performer in a ‘‘theatrical drama of the evaluation process’’ (p. 19)?

Structured in three parts, the book is meant to be studied as well as read, moving from the

general to the specific, from theory to case studies, and from principles to the practicality of

tables, templates, and guided reflection. Part 1 introduces the concept of interactive evaluation

practice (IEP), frames IEP principles within the basic inquiry tasks that evaluators perform, and

grounds IEP within the research on social interdependence theory and evaluation use. Part 2

offers evaluators wide-ranging frameworks and examples for several evaluation concerns: (a) the

kinds of conversations evaluators facilitate; (b) the structuring of interactions to facilitate excel-

lence in evaluation design, implementation, interpretation, and reporting; (c) the constructive

management of conflict; and (d) strategies for handling the unexpected. Part 3 offers three

detailed cases for reflection, with each exemplifying one of three different types of evaluator

Book Reviews 157

at University of North Carolina at Chapel Hill on February 4, 2016aje.sagepub.comDownloaded from