Individual Reflection, select concepts or topics that resonated with you during this course and commit to learning more about themManagement Accounting Skills

96 Harvard Business Review | October 2009 | hbr.org

Ju d

e B

u ff

u m

| by Andrew Likierman

Performance Measurement

The Five Traps of

1524 Oct09 Likierman layout.indd 961524 Oct09 Likierman layout.indd 96 9/4/09 12:38:55 PM9/4/09 12:38:55 PM

hbr.org | October 2009 | Harvard Business Review 97

IIN AN EPISODE of Frasier, the television sitcom that follows the fortunes of a Seattle-based psychoana-lyst, the eponymous hero’s brother gloomily sum-marizes a task ahead: “Diffi cult and boring – my favorite combination.” If this is your reaction to the challenge of improving the measurement of your organization’s performance, you are not alone. In my experience, most senior executives fi nd it an onerous if not threatening task. Thus they leave it to people who may not be natural judges of per- formance but are fl uent in the language of spread- sheets. The inevitable result is a mass of numbers and comparisons that provide little insight into a company’s performance and may even lead to decisions that hurt it. That’s a big problem in the current recession, because the margin for error is virtually nonexistent.

So how should executives take ownership of per- formance assessment? They need to fi nd measures, qualitative as well as quantitative, that look past this year’s budget and previous results to determine how the company will fare against its competitors in the future. They need to move beyond a few simple, easy-to-game metrics and embrace an array of more sophisticated ones. And they need to keep people on their toes and make sure that today’s measures are not about yesterday’s business model.

In the following pages I present what I’ve found to be the fi ve most common traps in measuring per- formance and illustrate how some organizations have managed to avoid them. My prescriptions aren’t exhaustive, but they’ll provide a good start. In any event, they can help you steal a march on rivals who are caught in the same old traps.

1524 Oct09 Likierman layout.indd 971524 Oct09 Likierman layout.indd 97 9/4/09 12:39:06 PM9/4/09 12:39:06 PM

The Five Traps of Performance Measurement

98 Harvard Business Review | October 2009 | hbr.org

Measuring Against Yourself The papers for the next regular performance as- sessment are on your desk, their thicket of numbers awaiting you. What are those numbers? Most likely, comparisons of current results with a plan or a budget. If that’s

the case, you’re at grave risk of falling into the fi rst trap of performance measurement: looking only at your own company. You may be doing better than the plan, but are you beating the competi- tion? And what if the estimates you’re seeing were manipulated?

To measure how well you’re doing, you need information about the benchmarks that matter most – the ones outside the organization. They will help you defi ne competitive priorities and connect executive compensation to relative rather than ab- solute performance – meaning you’ll reward senior executives for doing better than everyone else.

The trouble is that comparisons with your com- petitors can’t easily be made in real time – which is precisely why so many companies fall back on mea- surements against the previous year’s plans and

budgets. You have to be creative about how you fi nd the relevant data or some proxy for them.

One way is to ask your cus- tomers. Enterprise, the car-rental company, uses the Enterprise Service Quality Index, which measures customers’ repeat pur- chase intentions. Each branch of the company telephones a random sample of customers and asks whether they will use Enterprise again. When the index goes up, the company is gaining market share; when it falls, cus- tomers are taking their business elsewhere. The branches post re- sults within two weeks, put them next to profi tability numbers on monthly financial statements, and factor them into criteria for promotion (thus aligning sales goals and incentives).

Of course you have to make sure you don’t annoy your cus-

tomers as you gather data. Think about how res- taurant managers seek feedback about the quality of their service: Most oft en they interrupt diners’ conversations to ask if everything is OK; sometimes they deliver a questionnaire with the bill. Either ap- proach can be irritating. Danny Meyer, the founder of New York’s Union Square Hospitality Group, gets the information unobtrusively, through simple ob- servation. If people dining together in one of his restaurants are looking at one another, the service is probably working. If they’re all looking around the room, they may be wowed by the architecture, but it’s far more likely that the service is slow.

Another way to get data is to go to professionals outside your company. When Marc Eff ron, the vice president of talent management for Avon Products, was trying to determine whether his company was doing a good job of fi nding and developing man- agers, he came up with the idea of creating a net- work of talent management professionals. Started in 2007, the New Talent Management Network has more than 1,200 members, for whom it conducts original research and provides a library of resources and best practices.

Looking Backward Along with budget fig- ures, your performance assessment package al- most certainly includes comparisons between this year and last. If so, watch out for the second trap, which is to focus on the past. Beating last year’s numbers is not the

point; a performance measurement system needs to tell you whether the decisions you’re making now are going to help you in the coming months.

Look for measures that lead rather than lag the profi ts in your business. The U.S. health insurer Hu- mana, recognizing that its most expensive patients are the really sick ones (a few years back the com- pany found that the sickest 10% accounted for 80% of its costs), off ers customers incentives for early screening. If it can get more customers into early or even preemptive treatment than other companies can, it will outperform rivals in the future.

The quality of managerial decision making is another leading indicator of success. Boards must assess top executives’ wisdom and willingness to listen. Qualitative, subjective judgments based on independent directors’ own experience with an ex-

Most senior executives fi nd » performance measurement diffi cult if not threatening, and they’re reluctant to engage with it in a meaningful way. As a result, companies routinely fall into fi ve traps.

Specifi cally, they use them- » selves rather than competitors as benchmarks, focus on past indicators of success, overvalue numbers at the expense of quali- tative measures, set easy-to-game metrics, and cling to systems that have outlived their usefulness.

This article addresses each of » the traps in turn, offering advice about how to avoid them and ex- amples of organizations that have successfully done so.

trap

trap

IN BRIEF IDEA

1524 Oct09 Likierman layout.indd 981524 Oct09 Likierman layout.indd 98 9/4/09 12:39:13 PM9/4/09 12:39:13 PM

hbr.org | October 2009 | Harvard Business Review 99

ecutive are usually more revealing than a formal analysis of the executive’s track record (an unreli- able predictor of success, especially for a CEO) or his or her division’s fi nancial performance. (See

“Evaluating the CEO,” by Stephen P. Kaufman, HBR October 2008.)

It may sound trite, but how the company pre- sents itself in offi cial communications oft en signals the management style of top executives. In August 2006 the Economist reported that Arijit Chatterjee and Donald Hambrick, of Pennsylvania State Uni- versity, had devised a narcissism index on which to rate 105 company bosses, based on the prominence of the CEO’s photo in the annual report, his or her prominence in press releases, the frequency of the fi rst person singular in interviews with the CEO, and his or her compensation rela- tive to that of the fi rm’s second-highest- paid executive.

Finally, you need to look not only at what you and others are doing but also at what you aren’t doing. The managers of one European investment bank told me that they measure performance by the outcomes of deals they’ve turned down as well as by the outcomes of deals they’ve won. If the ones they’ve rejected turn out to be lem- ons, those rejections count as successes. This kind of analysis seems obvious once stated, but I’ve noticed a persistent bias in all of us to focus on what we do over what we don’t do. Good management is about making choices, so a decision not to do something should be analyzed as closely as a decision to do something.

Putting Your Faith in Numbers Good or bad, the metrics in your performance as- sessment package all come as numbers. The problem is that numbers- driven managers often end up producing reams of low-quality data. Think about how companies

collect feedback on service from their customers. It’s well known to statisticians that if you want eval- uation forms to tell the real story, the anonymity of the respondents must be protected. Yet out of a desire to gather as much information as possible at points of contact, companies routinely ask custom-

ers to include personal data, and in many cases the employees who provided the service watch them fi ll out the forms. How surprised should you be if your employees hand in consistently favorable forms that they themselves collected? Bad assess- ments have a tendency to mysteriously disappear.

Numbers-driven companies also gravitate to- ward the most popular measures. If they’re looking to compare themselves with other companies, they feel they should use whatever measures others use. The question of what measure is the right one gets lost. Take Frederick Reichheld’s widely used Net

Promoter Score, which measures the likelihood that customers will recommend a product or ser- vice. The NPS is a useful indicator only if recom- mendations play the dominant role in a purchase decision; as its critics point out, customers’ propen- sity to switch in response to recommendations var- ies from industry to industry, so an NPS is probably more important to, say, a baby-food manufacturer than to an electricity supplier.

Similar issues arise about the much touted link between employee satisfaction and profi tability. The Employee-Customer-Profi t Chain pioneered by Sears suggests that more-satisfi ed employees pro- duce more-satisfi ed customers, who in turn deliver higher profi ts. If that’s true, the path is clear: Keep your employees content and watch those profi ts soar. But employees may be satisfi ed mainly be- cause they like their colleagues (think lawyers) or because they’re highly paid and deferred to (think investment bankers). Or they may actually enjoy what they do, but their customers value price above the quality of service (think budget airlines).

A particular bugbear of mine is the application of fi nancial metrics to nonfi nancial activities. Anxious to justify themselves rather than be outsourced, many service functions (such as IT, HR, and legal) try to devise a return on investment number to

trap

At one investment bank, if the deals that managers have rejected turn out to be lemons, those rejections count as successes.

1524 Oct09 Likierman layout.indd 991524 Oct09 Likierman layout.indd 99 9/4/09 12:39:20 PM9/4/09 12:39:20 PM

The Five Traps of Performance Measurement

100 Harvard Business Review | October 2009 | hbr.org

help their cause. Indeed, ROI is oft en described as the holy grail of measurement – a revealing meta- phor, with its implication of an almost certainly doomed search.

Suppose an HR manager undertakes to assign an ROI number to an executive training program. Typically, he or she would ask program participants to identify a benefi t, assign a dollar value to it, and estimate the probability that the benefi t came from the program. So a benefi t that is worth $70,000 and has a 50% probability of being linked to the pro- gram means a program benefi t of $35,000. If the program cost $25,000, the net benefi t is $10,000 – a 40% ROI.

Think about this for a minute. How on earth can the presumed causal link be justifi ed? By a state- ment like “I learned a production algorithm at the program and then applied it”? Assessing any seri- ous executive program requires a much more so- phisticated and qualitative approach. First you have to specify ahead of time the needs of the program’s stakeholders – participants, line managers, and sponsors – and make sure that the syllabus meets your organizational and talent-management objec- tives. Once the program has ended, you have to look beyond immediate evaluations to at least six months aft er participants return to the workplace; their personal feedback should be incorporated in the next annual company performance review. At the soft drinks company Britvic, HR assesses its ex- ecutive coaching program by tracking coachees for a year aft erward, comparing their career trajecto- ries with those of people who didn’t get coached.

Gaming Your Metrics In 2002 a leaked internal memo from associates at Cliff ord Chance, one of the world’s largest law firms, contended that pressure to deliver bill- able hours had encour- aged its lawyers to pad their numbers and cre- ated an incentive to allo-

cate to senior associates work that could be done by less expensive junior associates.

Lawyers aren’t the only ones: A number of prom- inent companies have been caught trying to manip- ulate their numbers. Since 2004 Royal Dutch Shell has paid $470 million to settle lawsuits relating to its overstatement of reserves. Morgan Stanley was reportedly willing to lose €20 million on a securi-

ties trade for the Finnish government just before closing its books for 2004 in order to improve its position in the league table for global equity capital market rankings.

You can’t prevent people from gaming numbers, no matter how outstanding your organization. The moment you choose to manage by a metric, you invite your managers to manipulate it. Metrics are only proxies for performance. Someone who has learned how to optimize a metric without actually having to perform will oft en do just that. To create an eff ective performance measurement system, you have to work with that fact rather than resort to wishful thinking and denial.

It helps to diversify your metrics, because it’s a lot harder to game several of them at once. Clif- ford Chance replaced its single metric of billable hours with seven criteria on which to base bonuses: respect and mentoring, quality of work, excellence in client service, integrity, contribution to the com- munity, commitment to diversity, and contribution to the fi rm as an institution. Metrics should have varying sources (colleagues, bosses, customers) and time frames. Mehrdad Baghai and coauthors de- scribed in “Performance Measures: Calibrating for Growth” (Journal of Business Strategy, July–August 1999) how the Japanese telecommunications com- pany Soft Bank measured performance along three time horizons. Horizon 1 covered actions relevant to extending and defending core businesses, and metrics were based on current income and cash fl ow statements. Horizon 2 covered actions taken to build emerging businesses; metrics came from sales and marketing numbers. Horizon 3 covered creating opportunities for new businesses; success was measured through the attainment of preestab- lished milestones. Multiple levels like those make gaming far more complicated and far less likely to succeed.

You can also vary the boundaries of your mea- surement, by defi ning responsibility more narrowly or by broadening it. To reduce delays in gate-closing time, Southwest Airlines, which had traditionally applied a metric only to gate agents, extended it to include the whole ground team – ticketing staff , gate staff , and loaders – so that everyone had an incentive to cooperate.

Finally, you should loosen the link between meeting budgets and performance; far too many bonuses are awarded on that basis. Managers may either pad their budgets to make meeting them easier or pare them down too far to impress their bosses. Both practices can destroy value. Some com-

trap

1524 Oct09 Likierman layout.indd 1001524 Oct09 Likierman layout.indd 100 9/4/09 12:39:27 PM9/4/09 12:39:27 PM

hbr.org | October 2009 | Harvard Business Review 101

panies get around the problem by giving manag- ers leeway. The offi ce supplier Staples, for example, lets them exceed their budgets if they can demon- strate that doing so will lead to improved service for customers. When I was a CFO, I off ered scope for budget revisions during the year, usually in months three and six. Another way of providing budget fl ex- ibility is to set ranges rather than specifi c numbers as targets.

Sticking to Your Numbers Too Long As the saying goes, you manage what you mea- sure. Unfortunately, per- formance assessment systems seldom evolve as fast as businesses do. Smaller and growing companies are especially likely to fall into this

trap. In the earliest stages, performance is all about survival, cash resources, and growth. Comparisons are to last week, last month, and last year. But as the business matures, the focus has to move to profi t and the comparisons to competitors.

It’s easy to spot the need for change aft er things have gone wrong, but how can you evaluate your measures before they fail you? The answer is to be very precise about what you want to assess, be explicit about what metrics are as- sessing it, and make sure that everyone is clear about both.

In looking for a measure of customer satisfaction, the British law fi rm Addle- shaw Booth (now Addleshaw Goddard) discovered from a survey that its clients valued responsiveness most, followed by proactiveness and commercial- mindedness. Most fi rms would interpret this fi nd- ing to mean they needed to be as quick as possible. Addleshaw Booth’s managers dug deeper into the data to understand more exactly what “responsive- ness” meant. What they found was that they needed to diff erentiate between clients. “One size does not fi t all,” an employee told me. “Being responsive for some clients means coming back to them in two hours; for others, it’s 10 minutes.”

The point is that if you specify the indicator precisely and loudly, everyone can more easily see when it’s not fi t for the purpose. The credit-rating agencies have come under attack because they gave

AAA ratings to so many borrowers who turned out to be bad risks. The agencies have argued in their own defense that lenders misunderstood what the ratings meant. The AAA rating, they claim, was awarded on the basis of borrowers’ credit records, and it described the likelihood of default under normal market conditions; it did not factor in what might happen in the event of a massive shock to the fi nancial system. Reasonable as this explanation may be, it is no consolation to those who thought they knew what the magic AAA represented.

• • •

Why do organizations that excel in so many other ways fall into these traps? Because the people man- aging performance frameworks are generally not experts in performance measurement. Finance managers are profi cient at tracking expenses, moni- toring risks, and raising capital, but they seldom have a grasp of how operating realities connect with performance. They are precisely the people who strive to reduce judgments to a single ROI number. The people who understand performance are line managers – who, of course, are crippled by confl icts of interest.

A really good assessment system must bring fi - nance and line managers into some kind of mean- ingful dialogue that allows the company to benefi t from both the relative independence of the former and the expertise of the latter. This sounds straight- forward enough, but as anyone who’s ever worked in a real business knows, actually doing it is a rather tall order. Then again, who says the CEO’s job is supposed to be easy?

Andrew Likierman ([email protected]) is the dean of London Business School, a nonexecutive director of Barclays Bank, and the chairman of the UK’s National Audit Offi ce.

Reprint R0910L To order, see page 143.

The moment you choose to manage by a metric, you invite your managers to manipulate it.

trap

1524 Oct09 Likierman layout.indd 1011524 Oct09 Likierman layout.indd 101 9/4/09 12:39:36 PM9/4/09 12:39:36 PM

Copyright 2009 Harvard Business Publishing. All Rights Reserved. Additional restrictions may apply including the use of this content as assigned course material. Please consult your institution's librarian about any restrictions that might apply under the license with your institution. For more information and teaching resources from Harvard Business Publishing including Harvard Business School Cases, eLearning products, and business simulations please visit hbsp.harvard.edu.