Web Security

The Best Laid Plans of Mice and Men Raccoon and Dog

raccoon@swcp.com

DOI: 10.1145/2579281.2579286 http://doi.acm.org/10.1145/2579281.2579286

INTRODUCTION

“Planning” is a murky, kitchen-sink term for activities as diverse as project management, estimation, budgeting, scheduling, requirements gathering, architecting, designing, feature selection, prioritizing, and commitment. This diversity of uses should suggest that planning is called on to do too much and cannot possibly work as well as one might want. Empirically, planning is hard to do well and is fraught with delusions and abuses. Daniel Kahneman defines the “planning fallacy” as the tendency of people and organizations to underestimate the difficulty of tasks.

In this essay, we elaborate on the limits of planning. Planning does have value, but the downsides must also be kept in perspective. We are concerned with goal-oriented plans (such as implementing features and fixing bugs), rather than schedule- oriented plans (such as time boxes and iterations). We are concerned with tasks that are larger than some minimum size, say 40 man-hours, which in our experience is where plans start to go seriously awry.

In the first section, we argue that all plans are inherently flawed. Because of unknownness (uncertainty, change, error, irrationality, and so on), all plans start out flawed. Then psychologically and

ACM SIGSOFT Software Engineering Notes Page 7 March 2014 Volume 39 Number 2

socially, people prefer bad plans, which makes everything worse. Then politically, gatekeepers distort reality, which makes everything worse, again.

In the second section, we argue that planning is bad for everyone. Plans are rationalizations or prayers, which foster and disguise wishful thinking. Loss matters more than gain, so planning transmogrifies all change into loss. Even though all plans are flawed and must change, everyone converts plans into predictions and estimates into promises; they just do. The need for both stability and change at the same time increases everyone’s schizophrenia. Also, everyone uses plans as weapons, whenever it suits their purposes, which exacerbates conflicts between managers and developers.

In the third section, we argue that planning is useful and natural but hard to do well. Advantages to planning include that people do it naturally, it fosters useful progress for some tasks, and it adds important delay for other tasks. But, planning is not enough. The real problems for projects are setting expectations and driving progress, both of which planning actually does poorly. A reasonable goal for planning is to be less wrong. Unfortunately, nobody wants to be less wrong; everybody just wants to be right. Making plans without getting carried away is simply beyond human capacity.

ALL PLANS ARE FLAWED We explain the flaws of planning in three steps. First, because of

unknownness, all plans and all estimates start out inherently flawed. Second, people prefer bad plans, so they make choices that make their plans worse. Third, people play politics, so they make choices that make their plans worse, again.

Much Remains Unknown, so All Plans Start Out Flawed People are limited by their understandings. They cannot plan for

what they do not know, beyond say adding contingency factors. In the essay “Unknownness,” we argued that many aspects of software projects are unknown, uncertain, erroneous, destined to change, volatile, irrational, or arbitrary. People know neither the true states of their projects nor what it will take to complete them. REs know neither exactly what users will need nor what the requirements will be, so they cannot accurately do discovery or write requirements. SEs know neither the existing code base nor what code they will write, so they cannot accurately plan architectures or estimate tasks.

Unknowns and errors occur in two ways: what you don’t know (whether you are aware of it or not) and what you think you know, but you don’t. Many classical planners presume that the future is predictable, that everyone is rational, that everything is known, and so on, which are obviously untrue. These planners can neither predict the future nor divine any other source of unknownness. It works the other way around: plans are dissolved by unknownness.

Everyone Prefers Bad Plans, which Makes Everything Worse For psychological and social reasons, everyone (including users,

analysts, developers, and managers) deliberately prefers bad plans, in spite of themselves, even when they have the best of intentions. George Akerlof notes that many people make bad short-term decisions about procrastination, debt, drinking, and dieting, even when they understand the long-term consequences. (Wait, page 155) Freedman notes that people deliberately seek bad advice. (Wrong, page 69) Gilbert writes that people accept bad advice and reject good advice. (Stumbling on Happiness, page 235+)

Preference for Resonant: David Freedman writes, “Good expert advice will be at odds with [what] draws us to it.” (Wrong, 81) People prefer “resonant” advice, which echoes what they’ve believed all along. (Wrong, 83) Expert advice that sounds good is clear-cut, doubt-free, universal, upbeat, actionable, palatable, dramatic, tells stories, uses numbers, and applies retroactively. (Wrong, pages 75 to 80) Of course, resonance has nothing to do with the usefulness of the advice.

Preference for Simple: People prefer simple advice, even when they suspect it is wrong. (Wrong, pages 69 and 75) Nobody wants to hear a complicated plan, because like a Rube Goldberg contraption, it will probably be complicated in all the wrong ways. Freedman notes, “Were there simple truths to be had, we would have [found] them long ago.” (Wrong, page 81)

Complicated Advice is Bad, Too: Silver notes that complex models can be wrong, too. (Signal and Noise, page 225) Complicated models and plans are more realistic, because real problems and real solutions tend to be complicated, but that does not mean they are complicated in the right ways. All models have limits and everyone is skeptical of complexity.

Preference for the Familiar and Near: Shelling writes that, “There is a tendency in our planning to confuse the unfamiliar with the improbable.” (Signal and Noise, page 419) And conversely, people confuse the familiar with the probable. Kahneman notes that people often overrate the likelihood of events closer to us in time and space, which is called the “availability heuristic.” (Signal and Noise, page 424) The military refers to this as “fighting the last war.”

Focus Illusion: Joseph Hallinan writes that people are prone to exaggerate the importance of any single factor. (Make Mistakes, page 206) Freedman wrote that over 3,000 factors influence body weight, but experts will talk about the importance of individual factors (like blueberries or exercise) as if only one or two things mattered. (Wrong, page 81) A myriad factors influence every software project, but enthusiasts will argue that individual factors, like Haskell or quality or CMMI or TDD or Scrum or Cloud are the keys to success. Focus causes people to not attempt anything beyond what they understand.

In “Quality Wars: Open Source Versus Proprietary Software,” Diomidis Spinellis studied four operating systems (Solaris, Microsoft Research, Linux, and BSD) to compare open and closed-source projects. Spinellis concluded that while there were significant differences, all processes produced remarkably similar results. (Making Software, page 288)

Few factors make much difference in a complex project, because any factor that one might emphasize matters only a little bit. Overdoing any one factor necessarily draws attention and effort away from other important factors. (Wrong, page 83)

On the other hand, it is impossible for anyone to consciously balance 3,000 factors to manage their weight. It is impossible for anyone to remember and act on every fact about their software projects and processes. Enthusiasts who draw attention to individual factors can help to systematically improve practice over time. Keystone habits (such as unit testing) can improve other habits as well. However, your keystone habit may not work for me, and in the scheme of things, it probably doesn’t matter.

Restraint Bias: People are naturally optimistic. They think they can handle more than they can. (Happy Brain, page 118) They naturally make and commit to plans that are beyond their abilities without realizing it. Everyone underestimates the difficulty of

ACM SIGSOFT Software Engineering Notes Page 8 March 2014 Volume 39 Number 2

what they are trying to do. (Obvious, page 252) “Our beliefs tend to be simplistic and optimistic.” (Wrong, page 81)

Projection Bias: Wanting or intending something is much easier than following through and doing it. People think they will use a gift card or stop smoking, but they won’t. (Make Mistakes, pages 203 and 204) Wanting or intending to follow a perfect process to create a perfect app (or advising someone else to do so) is much easier than actually doing so.

Peer Pressure: People want to please; they want to be as “can do” as possible; they rarely discuss worst case scenarios, in part because there is tremendous social pressure to avoid such topics. Worst case scenarios may occasionally be realistic, but in industry, those who talk about the worst case are usually considered Eeyores and nobody loves an Eeyore. Cassandra wasn’t any fun, either. Comments about worst cases are often interpreted by bosses, colleagues, and clients as statements that you are uncooperative and don’t even want to try.

Brain scans show that succumbing to peer pressure to accept an obvious lie feels like perceiving the truth. It is not a conscious or rational decision; it is not deception; succumbing to peer pressure is the unconscious and psychological accepting of the social truth. Because of the social truth, everyone who warned about the dangers of banking deregulation in the late 1990s and the early 2000s was ignored, until the world economy crashed.

Rational Bias: Silver notes that it can be rational to be biased or even deliberately wrong about predictions. (Signal and Noise, pages 197 to 200) Apparently, the Weather Channel overstates the chance of rain when the probability is low, possibly to avoid being blamed for ruined picnics. (Signal and Noise, pages 135 to 136) Many project estimates are deliberately padded to provide a cushion for error, which is the same thing.

In securities, unknown analysts often make wild predictions, hoping for press, while reputable analysts make predictions that stay with the pack. (Signal and Noise, page 199) Young software engineers often make exaggerated claims to stand out or get attention. They are probably not even aware of their own exaggerations. (We were like that ourselves, once.) While such behavior may run counter to the ideals of professionalism, getting attention is a basic need that will never go away.

Gatekeepers Play Politics, which Makes Everything Worse, Again

Aristotle argued that “man is the political animal.” We would add that planning is merely politics by other means. Planning is (at least in part) a tool for organizing people. Larger projects have more need to set expectations, commitments, and so on, so planning matters more. The planning community has long understood that politics screws up all planning. They even named some of the problems.

The Dark Side of Planning The “dark side of planning” is a euphemism for the abuse of

power by managers and gatekeepers. Planning is often a fig leaf for doing what the most authoritative person in the room wants. People often defer to status, to “the highest paid person’s opinion,” also known as the “HiPPO.” (Little Bets, page 75) This becomes clear when projects get sidetracked or “captured” by managers who have more authority.

Managers have many ways to get what they want. When managers only respond to answers they like and ignore answers they dislike, it sends a message. Over time, managers teach their

subordinates to spin the truth and subordinates figure out what their managers want to hear. This is insidious, because even though the manager trained his people to distort the truth, he can always shrug and say, “I never told them to lie.” Everyone tells others what they want to hear to some extent, but whim can drown out truth.

Whenever managers or executives negotiate with developers, power imbalances will undermine the truth. Whenever managers negotiate for faster or cheaper, they are implicitly telling developers to underestimate their next project. When executives or managers say, “I want an aggressive plan,” they often mean or are interpreted to mean, “Lie to me.” To keep in good graces, analysts and developers must tell their bosses what they want to hear.

Executives and managers know that pushing subordinates or laying down challenges can inspire great effort. It can also inspire people to say what others want to hear. Apple employees had long joked about Steve Jobs and his “reality distortion zone.” Sometimes his challenges worked (the iPhone) and sometimes they didn’t (the external antenna), but Jobs got the decisions he wanted.

The Authorization Imperative The “authorization imperative” or “strategic misrepresentation”

occurs when planners need permission before their projects may proceed.

Gatekeeping is a heuristic that ensures that plans meet minimum standards. But, mistakes and abuses happen. Some planners with important projects (who are bad at playing the game or do not get along with the gatekeeper) will consistently lose approval. Other planners with bad projects (who excel at gaming the system or are pals with the gatekeeper) will consistently win approval.

As part of doling out approval, gatekeepers often play games like “guess the right number.” Managers cause problems when they pressure subordinates and planners to over-promise or underestimate to get approval.

Other gatekeepers instinctively respond to an estimate by haggling. “I think I can do this task in 20 hours.” “Can you do it in 10?” “Well, I can try.” Each person walks away with a different expectation. Hagglers may not realize that they adversely affect results, because they think in terms of negotiation or optimization rather than miscommunication or abuse.

Power is Seductive Gatekeeping can be a force for evil. Gatekeepers embody

distrust, gamesmanship, power, control, authority, and the right to say, “No.” While people are often skeptical of and frustrated by the game, gatekeepers force everyone to play. Managers and gatekeepers could reduce the politics in projects and organizations, but they won’t, because that would reduce their personal authority. After all, many managers clawed their way up the hierarchy specifically to wield that kind of power.

We are sure that most gatekeepers believe that they are beyond the petty abuses of power. We are also sure that all gatekeepers do it. Most people are deluded about how good they actually are. Professionals in golf, psychology, securities analysis, and medicine all believe that they are better than they are. (Make Mistakes, pages 169 to 171) Even the managers we most admire have abused their authority on occasion, usually in late afternoons when they were tired.

We are not arguing to eliminate managers or gatekeepers. Organizations need managers and gatekeepers, because employees

ACM SIGSOFT Software Engineering Notes Page 9 March 2014 Volume 39 Number 2

must do more than just whatever they feel like. However, we are arguing for an appreciation of the limits of power and the costs of the abuses of power. Though this initial subjective estimate needs further research and refinement, we estimate that half of all planning and estimation errors result from the biases and abuses of managers and gatekeepers.

When gatekeepers squelch all forward progress, companies need their cowboys or wild geese to actually get some useful work done, and organizations know it. Grace Hopper famously said, “It is easier to ask forgiveness than to ask for permission.” Grace Hopper was a hardcore cowboy, a gatekeeper’s worst nightmare, someone who could get something done when everyone else said, “No.”

PLANNING EXACERBATES HUMAN NATURE Planning exacerbates many problems for people. Planning is a

fig leaf that helps people to hide their rationalizations and wishful thinking. Planning increases the feeling that all change is loss, which increases stubbornness. Planning increases the need for both change and stability, at the same time, and this bifurcation increases schizophrenia. And, programmers and managers use plans as weapons, which can inflame their conflicts. Planning causes many more problems than anyone ever admits.

Plans Are Fig Leaves for Rationalization and Prayer “Rationalization [is] the act of causing something to be or seem

reasonable.” (Stumbling on Happiness, page 163) John Cacioppo and William Patrick write “We humans often use words and logic merely to rationalize our primitive emotions and prior expectations.” (Loneliness, page 245) All rationales are sales pitches. Fred Brooks writes that design reports are rationalizations. (Design of Design, page 157) The brain works very hard to rationalize. (Stumbling on Happiness, page 192)

We believe that most software plans are primarily rationalizations for what the planners wanted to do anyways, rather than well-considered evaluations or comparisons of alternate solutions. For example, many designs are elaborate euphemisms for “I want to use technology x,” where x is python, mobile, cloud, analytics, or the buzzword du jour. It is well known that people can rationalize anything, including war and environmental destruction, especially for their own short-term benefit, such as profit. Rationales must never be naively accepted as meaningful.

Illusion of Rationality: The “illusion of rationality” means taking an unproven model as reality, often a memo or a plan. Everyone uses this illusion instinctively every time he sets a coffee cup on a table or sits in a chair. The brain builds models that suggest that the tables and chairs are safe to use, based on minimal evidence, such as looking like a table or a chair at first glance. The brain creates illusions for larger plans, too. Robert McNamara believed that his high-level plans and systems analysis would win the Vietnam War. (Little Bets, page 25)

First, People Use Their Distorted Realities to Make Plans: Selection or attention or confirmation bias is the tendency for people to notice facts that support their biases and desires and to ignore facts that contradict them. Daniel Gilbert writes, “The eye [agrees] to look for what the brain wants.” (Stumbling on Happiness, page 183) People create plans that describe what they want to happen, more than what anyone should actually expect to happen.

Then, People Use Plans to Distort Their Realities: People use plans and estimates to distort their own perceptions. They look for

what is in the plan and ignore what is actually happening. (Happy Brain, page 34) Planning narrows the scope of attention. People counting basketball passes will fail to notice someone in a gorilla suit wandering across the court.

One might think that planning would help people to distinguish between what they do and don’t understand. But, planning also hides what people don’t understand from themselves and from everyone else. Disguise happens because one shifts from arguing about what should be done, which is partly known, to arguing about what is in the plan, which may be fully known, regardless of whether the plan means anything useful. Plans are like Zaphod’s sunglasses, which “at the first hint of trouble, they turn totally black and thus prevent you from seeing anything that might alarm you.” In other words, like a good story, planning causes suspension of disbelief.

Plans are Self-Serving Prayers: Too often, saying or writing the correct words and repeating them convincingly to managers and clients matters more than actually expressing anything useful about the future. “With this sacrifice of our time and effort to make this plan, we beseech thee oh Lord, make the users happy, let the app will be delivered on time and budget, and no bugs. Amen.” Like the lottery, plans enable people to delude themselves and believe that their hopes and desires will come true.

Loss Matters More than Gain, so Planning Makes All Change Feel Like Loss

Businesspeople talk about gain when they talk about maximizing profit or increasing market share. Engineers talk about gain when they talk about new features. Politicians and government workers often strive to minimize negative results to avoid angering voters. Safety and quality engineers strive to minimize loss. In general, users want both to maximize gain and to minimize loss. They want apps that have useful, new features as well as fewer bugs and lower cost.

Statisticians more or less define loss as negative gain (loss ≈ −gain), and when using appropriate sigmoid and scaling factors, these two concepts seem mathematically fungible. However cognitively, people treat gain and loss very differently, even irreconcilably. Everyone undervalues gain and overvalues loss in at least three ways. Neutral terms don’t exist, so we expect that gain and loss will never be reconciled.

Value: Losing $100 is about twice as painful as finding $100 is pleasurable. People are much more interested in avoiding a surcharge than in getting a discount, when the base price is the same. Users and product managers will value a feature differently, by two to one, depending on whether they presumed it would be there versus whether they presumed that they would have to pay extra to get it.

The difference between half full (positive) and half empty (negative) also shows up in perception mismatches between developers and clients. We have seen the following conversation endlessly repeated. “I just got feature x working. Let me show you.” “But, that button is gray; it was supposed to be blue.” “We’ll fix that later.” “But, gray means we cannot use it.” “Why are you hung up on that? This is cool. It’s what you asked for. You should be happy.” “I don’t care.” Everyone interprets the same murky situations differently, even irreconcilably, because of their differing perceptions of gain and loss. They simply do not understand each other. Whiny customers are not always right (and can be very annoying), but their perceptions are their perceptions.

ACM SIGSOFT Software Engineering Notes Page 10 March 2014 Volume 39 Number 2

Also, developers cannot reliably inspect, test, or evaluate their own code, because they cannot reconcile gain and loss within themselves.

Risk: Consider the following choices. You are a medical expert brought in to make a decision during a deadly viral epidemic. Would you rather “save 1/3 of the victims” or “take a 1 in 3 gamble that everyone will be saved.” You are later brought in to make a decision during a deadly bacterial epidemic. Would you rather “let 2/3 of the victims die” or “take a 2 in 3 gamble that everyone will die.” These two decisions are mathematically identical, but more than half of people answer differently. (Happy Brain, page 39) When people face loss, they prefer to take big risks to avoid the worst and will gamble to save everyone. When people face gains, they prefer to cling to sure gains and will prefer to guarantee saving the smaller number of people. (Make Mistakes, pages 94 to 95)

Software users and product managers perceive “getting some of what they want” and “giving up some of what they want” very differently, even when it is the same question. Your project is behind schedule. Would you rather “finish half of the remaining features” or would you rather “take a 50-50 gamble that you can finish all the features?” Would you rather “cancel half of the remaining features” or “take a 50-50 gamble that all remaining features will fail?” Most people will answer these questions differently, even though they are mathematically the same. How a question is framed can matter more than the question itself.

People Prefer to Err through Inaction: (Make Mistakes, page 53) Studies show that two-thirds of the time when test takers have doubts about an answer, changing it would improve their scores. Studies also show that test takers believe that changing a right answer into a wrong answer will feel twice as bad as changing a wrong answer into a right answer will feel good. Even though changing answers would probably improve their scores, few people actually do so, because their fear of regret outweighs the anticipated reward of a better score. (Make Mistakes, page 52) Even when people are told to change, they don’t. (Make Mistakes, page 52)

In software, changing requirements or architectures during development entails risk and people are often more comfortable allowing a poor decision to fail without making changes. Few people want the responsibility or regret if the changes go awry. In addition, students “remember sticking with the first answer as being a better strategy than it actually was.” (Make Mistakes, page 55) Plans enable people to be even wronger, because plans absolve them from the need to act responsibly as their circumstances change. On the other hand, Gilbert notes that people regret inaction more than action, after the fact. (Stumbling on Happiness, page 197)

All Change is Loss: Whenever a developer realizes that a requirement or feature will be hard to deliver, or a developer finds a better solution, then he or she may offer to change the plan. But, unless the change provides at least twice the replacement value for the same cost, most people will perceive it as a loss or as an unwelcome risk. Tradeoffs that produce at least twice the value for the same cost probably don’t exist, because those sorts of tradeoffs were probably already in the original plan. So once a reasonable plan is set, all change is loss.

Everyone Clings to the Past, so All Planning Causes Schizophrenia

All Plans and Estimates Become Promises that Cannot Change:

Everyone says things like planning is not to predict the future (Planning XP, page 2) and “estimates are not commitments” (Planning XP, page 58), but nobody means it. Everyone knows that they should treat plans and estimates with suspicion, but they cannot. Everyone, absolutely everyone, especially a power-that-be like a manager or client, converts plans into predictions and estimates into promises. They do so for well-known psychological reasons.

Anchoring: People cling to the first numbers they hear. Write down one irrelevant number, like a phone number or birth date. Then make an unrelated decision, like the price of a yard-sale item or the hours to complete a software task. The numbers will be correlated. (Make Mistakes, page 102) The first offer in a price negotiation determines the result. (Make Mistakes, page 107) Once a plan is suggested, everyone begins to treat it as a promise. Once an estimate is given, it becomes a commitment. Anchoring is universal and insidious. Gilbert notes that multitasking increases anchoring. (Stumbling on Happiness, page 150)

Participation Attachment: People overvalue their own work. (Upside of Irrationality, page 94) Daniel Ariely calls attachment the “IKEA” effect (Upside of Irrationality, pages 83 to 106) and the “Not Invented Here” or “NIH” effect. (Upside of Irrationality, pages 107 to 122) In the case of origami, amateurs valued their own work nearly as much as they valued the work of other professionals. (Upside of Irrationality, page 94) Furthermore, “We mistakenly think that others love our work as much as we do.” (Upside of Irrationality, page 99) Things that are harder to attain, we buy into more. (Upside of Irrationality, page 89) The more we participate in planning, and the harder the project, the more we will believe that our plan is good and believe that everyone else agrees.

Advertising Attachment: When people see an ad, they imagine themselves driving that car or eating that cereal. They slowly attach themselves to the product. Advertising proves that people attach easily. So, the more often managers repeat the plan and explain how everyone will benefit, the more everyone will buy into it, regardless of its merits.

Sunk Cost: Planning is a sunk cost and people have a hard time walking away from sunk costs. The cost of planning is not insignificant, but is rarely much of a problem. Yet, people who make or pay for plans will be loath to let them go. (Being Wrong, page 194)

Plans Enable Everyone to Cling to the Past: People love consistency and certainty. (Wrong, pages 68+; Happy Brain, page 17) So, once people make a decision, they often simply refuse change. The problem is that planning gives everyone concrete visions from the past to cling to. When a plan exists, explicit action is required to make changes. If people didn’t have a plan, they would have nothing to get hung up on that prevented change.

Schizophrenia: Planning leads to a bifurcation of reality. People need to believe in the plan or else they will not commit to it. But, plans are invariably flawed and must evolve. So, plans must both change and remain the same, at the same time, which is impossible and causes duress for everyone. In other words, planning causes schizophrenia.

Conflicts between the need for change and the need for stability are universal to the human condition. Everybody wants both. In most societies, the political spectrum splits fairly evenly between liberals who want change and conservatives who want stability. Even small groups, such as the eight participants in Biosphere 2,

ACM SIGSOFT Software Engineering Notes Page 11 March 2014 Volume 39 Number 2

split into two factions, one group arguing for change and the other for stability.

A Weapon that Escalates Conflict There are always power imbalances between managers and

developers, so there will always be conflicts. Everyone uses plans as weapons, in their own ways, whenever it suits their purposes.

A Stick to Hit With Beck and Fowler write “Plans can be used as a stick to beat

people with.” (Planning XP, page 2) Too often, managers use planning as a weapon to justify micromanagement, rather than as a pragmatic tool for getting better day-to-day results. When a manager deems that a plan is inadequate or that the plan is not being followed well enough, he may feel justified in stopping by your desk every hour or two to check up on things, to nag, to harass, to intimidate. There are better ways to handle these problems, but weak managers instinctively lash out.

Exaggerating Power Imbalances: Plans exaggerate power imbalances in hierarchies. Developers become more accountable for lower-level processes and decisions and lose authority, while managers sidestep accountability and gain authority. By passing off responsibility, managers gain political position. They just adore plans.

Laying Blame: Planning helps managers to scapegoat programmers. When people look for the source of errors, they look down at the last person to blame. (Make Mistakes, page 191) At the demise of one dot-com, the CEO gave a speech where he complained, “The software engineers failed to implement my vision.” This was after they added video ads to the site. Executives were proud of the increased revenue per ad, but the slow and annoying videos drove users away. The CEO and sales team didn’t have to reconsider any of their own decisions, because they already knew who to blame.

A Shield to Hide Behind Beck and Fowler also write that planning is an illusion to

conceal trouble. (Planning XP, page 2) When your boss stops by your desk every hour for a status update and you are stressing out, what do you do? You say, “I have a plan and I am following the plan,” even when you are lost at sea. Getting your boss out of your office, right now, matters most of all. You also want your manager to grow up, even though you know that will never happen. After he leaves, you can return to posting on Facebook or polishing your resume. Managers do this to their bosses, too, arguing that they have a plan and are following it.

Dodging Blame: Following the plan often seems like a strong defense. When everything is falling apart, people like being able to say, “We followed the plan,” which really means, “It’s not our fault.” But, getting defensive bodes ill for everyone. Defensiveness means that communication has broken down and that people are failing to be honest, responsive, and cooperative. When things go wrong, planners want to dodge blame, so they lie. The plan diverges from reality. (Planning XP, page 5)

From the developer’s point of view, when you don’t know how to solve a problem, planning feels like being set up. Whenever managers force developers to make plans, they suspect that managers will complain when anything deviates from those plans, which will assuredly happen. So, developers often react defensively to all planning.

THE GOOD NEWS AND THE BAD NEWS

In spite of what we have written so far, we believe that planning is good for all software projects, within reasonable limits. The problem is that people cannot keep their expectations about planning modest and real. Planning is a tool to be less wrong, but nobody wants to hear that. People just want to be right.

Planning is Inevitable and Helpful In Planning Extreme Programming, Kent Beck and Martin

Fowler write about the benefits of planning for Agile projects. In Managing the Software Process, Watts Humphrey writes about the benefits of planning for CMM projects. Here, we describe more benefits from psychology and sociology.

Setting Goals is Human Nature: David DiSalvo writes that the brain is a prediction machine. (Happy Brain, page 16) People spend about 12% of their time (about 2 hours every day) thinking about the future. (Wait, page 123; Stumbling on Happiness, page 17) George Loewenstein wrote, “The drive toward goal establishment and goal completion is ‘hard wired.’” (Upside of Irrationality, page 81). People make plans, because they do so anyways and they are comfortable doing so.

Countering Procrastination: Planning counters the bad habit of procrastination by giving people concrete and useful tasks to do now, as well as some sense that they are making effective progress toward goals. (Wait, pages 147 to 196)

Fighting the Flinch: The flinch effect occurs because people can react much faster than they can understand, which is useful when responding to lions and tigers and bears, but is not so good for making software. “Even in the case of quite unfamiliar tasks, people seem to prefer to act than to reflect.” (Make Mistakes, page 176) Planning encourages reflection and deliberation.

Stepping Back to See the Big Picture: Task saturation is the tendency for people to obsess over small tasks. (Make Mistakes, page 77) Ken Jacobson noted, “In many situations, the path of least resistance is to simply do another low-level task.” Small tasks are more knowable and tractable, which makes them easier to understand and get excited about. Planning can return attention to the bigger, harder issues that matter.

Adding Delay before Deciding: Planning helps people to take time before making important decisions, like what they want and what matters most. People who never take time to address these questions should never expect to answer them well. Delay provides an opportunity to make a better gestalt assessment of a situation.

Adding Delay before Acting: Planning both adds delay before real work starts and provides a rationale for waiting. When REs and SEs need feedback on prototypes, they should wait as long as they can before proceeding, to get as much feedback as possible. Additionally, Partnoy writes that you want to put off what doesn’t need to be done. (Wait, pages 147 to 196) Partnoy argues that in many situations, if you cannot assess a situation correctly, you will fail anyways, so you shouldn’t even start. (Wait, pages 63 to 80) When you are unsure what should be done, planning helps to increase delay.

Exploring More Options: Ancillary issues may matter more than the resulting plans. For example, planning may help people explore and appreciate a broader range of options, so that when things do go wrong, they have a broader palette of responses at hand. Planning allows (or forces) developers to spend time to consider their implementation options before jumping to coding conclusions. This may be the essence of Carl von Clausewitz’s

ACM SIGSOFT Software Engineering Notes Page 12 March 2014 Volume 39 Number 2

maxim, “Plans are of little importance, but planning is essential.” Control: Control is good for everyone’s mental and physical

health. In one study, nursing home residents were given a house plant for six months; twice as many residents died when the plant was cared for by the staff (30%) as when the plant was cared for by the resident (15%). (Stumbling on Happiness, pages 22 and 23) Planners make more choices and probably feel more in control of their destiny, which helps their mental and physical health.

Illusion of Control and Agency: The illusion of control provides the same psychological benefits as actual control. People will even make themselves the agent of control when none exists, for example gamblers may believe that their system will eventually win them big money. (Happy Brain, page 65) Similarly, managers and architects may come to believe that they personally cause their projects to succeed, going beyond overconfidence or ego.

Big Bets are Fun: People are fascinated by bold efforts, like putting a man on the moon. Small, incremental work can feel tedious, boring, and slow. Planning can enable bigger, more interesting, more daring bets, even though they entail greater risks.

Note that none of these advantages concern being right or perfect. They all concern doing better or doing something anyways regardless of the outcome. Also note that some are rationalizations for delay and others merely provide artificial peace of mind.

Less Wrong More and more software engineers seem aware that planning

has limits and that planning should occur only within those limits. Both Barry Boehm (“Architecture: How Much and When”) and George Fairbanks (Just Enough Architecture) recommend that before normal development starts, regardless of their circumstances, everyone should “plan as best they can, but no more.” REs and SEs naturally want to start projects with the best priors that they can get, because starting with better approximations to the real requirements, architectures, and source codes helps to minimize the learning and revision (i.e. the hard work) that they will have to do later. A wide spectrum of initial plans and priors can be useful, depending on what is and isn’t known.

50-50: When you really don’t know what will happen, Agile processes can start right away, with empty projects and no significant planning. The empty app is like Bayes starting with 50- 50, because it could evolve into anything, though it may take a lot of work to get there. 50-50 is appropriate for projects with new hires, new technologies, or volatile or unknown requirements. When projects have experienced developers, well-understood technologies, and stable, known requirements, odds are that somebody can find a better prior. Any fixed project could be used as the default starting point, but the empty project is the obvious one.

Yesterday: In weather, one fairly accurate model is that today will resemble yesterday. Likewise in software, one project often effectively approximates the next. Many square-root functions use the previous result as the seed for the next, because within apps, inputs to sqrt() are usually similar. Before maintenance, an app often closely resembles what the app will be after maintenance, which makes the existing app an excellent prior. Companies often specialize in specific markets or “product lines,” so that one project will resemble another. When Microsoft decided to market their SQL server, they purchased the source code from Sybase to get a head start. Large companies often purchase smaller companies with established products to enter new markets. Google

bought Android to compete with other smart-phone OSs. Of course, when a weather front moves in, then all bets may be off about tomorrow’s weather. Likewise, when your next project uses a new technology or has a new client, then all bets may be off.

BDUF: When you have a good idea of what your project will become and you believe that you can plan better than copying an existing project, then “big design up-front” may help. BDUF covers a wide spectrum of degrees of effort in its own right. Some Agile processes devote the first iteration to planning and architecture. In the Mythical Man-Month, Fred Brooks suggests devoting 17% (1/6) of the project to initial planning. In “Architecture: How Much and When?,” Barry Boehm suggests that initial planning should take from 5% to 37% of the total project effort, depending on the size of the project. (Making Software, page 179) How much BDUF helps is a function of how well you can predict the future, how stable your requirements and technologies are, how experienced your developers are in the tasks at hand, and so on.

BDUF is not about getting the architecture right, but about getting it less wrong than copying another architecture. Likewise, copying another architecture may be less wrong than starting with no architecture. The precondition for choosing BDUF as the prior is that you must subjectively expect that you can plan better than you can copy an existing project. The key word is “expect,” as opposed to “hope” or “guess” or “demand.” Likewise, the precondition for choosing a specific existing project (yesterday) as the prior is that you must subjectively expect that it is more representative than the empty project.

The Right Amount of Planning: Fairbanks encourages “sufficiently precise” planning. (Just Enough Architecture, page 298) When developers plan a lot up front and changes or errors occur, then planning effort will be wasted (and probably a lot of developer effort, too). Conversely, when developers fail to plan where they can, they will waste development effort, later. The optimal solution is the less wrong one in between. The greater the unknowns, the looser the planning must be.

But You Can’t Handle the Truth Nobody wants to be less wrong. Nobody wants to admit that

they could be wrong. Nobody ever says in a project bid, “We are less wrong than our competitors.” Nobody ever says, “Formal methods are less wrong than ad hoc development.” Everybody just wants to be right.

And there is good reason. Duncan Watts noted that only one future will emerge, so people really do want to predict it. (Obvious, page 161) “When we think about the future, we care about what will actually happen.” (Obvious, page 145)

Many people believe that planning is the essence of success. Benjamin Franklin, Winston Churchill, and the Sphinx have all stated something like, “If you fail to plan, your plan will fail.” This attitude is so pervasive that many people believe that results cannot turn out any better than they can plan. Selection bias reinforces this conclusion whenever someone remembers the situations where their plans worked and forgets the situations where their plans failed.

Planning Causes Failure But, caring about what will happen and planning to produce

good results are not enough. Plans that go awry are legion. Market-Garden (also known as “A Bridge Too Far”) was a

wartime disaster. The U.S. invasion of Iraq was much worse.

ACM SIGSOFT Software Engineering Notes Page 13 March 2014 Volume 39 Number 2

Before being ejected into space, Ford said to Arthur, “My plan rather involved being on the other side of the airlock.” Watts notes that in the 1990s, both Sony (with mini-discs) and Apple (with the iPod) had great strategies, but only one worked. (Obvious, pages 178 to 180)

Hewlett-Packard planned cool software for Palm-based tablet computers, which failed. Microsoft planned cool software called Metro, which failed. Customers refused to buy either one. These companies have enormous resources and obsessive proclivities for planning. As a practical matter, if teams at Hewlett-Packard and Microsoft cannot make and carry out effective plans, then you don’t stand a chance with your projects, either.

The Standish Chaos Reports claim that most software projects are late. The list of our own failures is too long and humiliating to recount, but for example this essay is over two years late and many of our own programs never got finished. Based on the observed facts, we conclude that planning in RE and SE causes many more failures than successes.

Having a plan is not enough; you need a good plan. So, why not just get a good plan? That is the crux. Everyone would if they could. But for all the reasons listed in the first section (unknownness, preference for bad plans, and politics), people only make bad plans. It is much easier to state what you want to happen, than to infer what will happen. All plans are flawed and must evolve to remain useful.

There is No Alternative We suspect that a few well-intended souls will take this essay

and say things like, “Now that we know about the limits of planning, we should make a plan to avoid those limits.” or “We need to characterize what we don’t know so that we can find out specifically what we need to learn.” These intensions are admirable, but deluded. They might do a bit better, but probably not much.

Either way they will make a new plan and then for all the reasons listed in the second section (anchoring, attachment, fig leaf for rationalization and wishful thinking) they will come to believe that their plan predicts the future. And because change feels like loss and everyone clings to the past, they will not allow the plan to change. And because they tried so hard, they will become even more schizophrenic.

All planning leads people astray. Rational people should not naïvely hope that their plans are perfect and they should be skeptical of all plans. But, most REs and SEs are all too human and unfortunately they irrationally delude themselves.

CONCLUSION Our analysis contradicts the classic approach that says, “just

make a plan,” which was epitomized by Fred Brooks in “Calling the Shot” (Mythical Man-Month, pages 89 and 90) and continues today in processes like CMMI, IEEE 12207, and RUP. We note that the PMI has ditched goal-driven processes and adopted Agile and that others are replacing RUP with AUP and DAD. Planning is a tool, not a solution or an ideal or an end. Neither a hammer nor a blueprint can guarantee that a building will serve its occupants. Software plans cannot guarantee good results, either.

We were tempted to call this essay “Planning Considered Harmful,” but that would overstate the problem. Many benefits of planning do exist, but they are murky and indirect. Since plans can cause so much harm, they should all be used with much more care than ever seems to occur. As Robert Burns noted centuries ago,

even the best laid plans gang aft agley.

NOTES Threats to Validity: Much of this essay is based on

contemporary research from behavioral economics, psychology, and sociology. We have applied numerous observations from psychological and sociological studies out of context. Further, software geeks probably differ from average people in many ways. However, we believe that the principles apply broadly and our uses are in the ballpark.

Risk versus Loss: In this essay, we treat risk and loss as synonyms, even though there are differences. The term loss is often used in the context of making choices, while the term risk is often used in the context of achieving specific goals. Government and business workers seem to prefer the term risk. For example, in Just Enough Architecture, Fairbanks uses the term risk consistently. In this essay, we use the term loss, which seems more common in decision theory.

Acknowledgements: Many friends and colleagues contributed ideas and reviews to this essay. We especially want to thank Mina Yamashita and Anthony J. Giancola.

BIBLIOGRAPHY [1] Douglas Adams, “The Hitchhiker’s Guide to the Galaxy,” TV-series, 1981, BBC. [2] Dan Ariely, Predictably Irrational, 2010, Harper. [3] Dan Ariely, The Upside of Irrationality: The Unexpected Benefits of Defying Logic, 2011, Harper. [4] Kent Beck and Martin Fowler, Planning Extreme Programming, 2000, Addison Wesley. [5] Barry Boehm, “Architecture: How Much and When?” in Andy Oram and Greg Wilson (editors), Making Software, 2010, O’Reilly. [6] Frederick P. Brooks, The Design of Design, 2010, Addison Wesley. [7] Frederick P. Brooks, The Mythical Man-Month, 1975, Addison Wesley. [8] Robert Burns, “To a Mouse,” 1785, wikipedia.com. [9] John Cacioppo and William Patrick, Loneliness: Human Nature and the Need for Social Connection, 2009, Norton. [10] David DiSalvo, What Makes Your Brain Happy and Why You Should Do the Opposite, 2011, Prometheus. [11] George H. Fairbanks, Just Enough Architecture: A Risk-Driven Approach, 2010, Marshall and Brainerd. [12] Daniel Gilbert, Stumbling on Happiness, 2006, Vintage. [13] Watts S. Humphrey, Managing the Software Process, 1989, Addison Wesley. [14] Michael Lewis, Moneyball, 2004, Norton. [15] Andy Oram and Greg Wilson (editors), Making Software, 2010, O’Reilly. [16] Frank Partnoy, Wait: the Art and Science of Delay, 2012, Public Affairs. [17] Raccoon and Dog, “Unknownness,” in ACM Software Engineering Notes, volume 38, issue 5, pages 8 to 17, September 2013, ACM Press. [18] Kathryn Schulz, Being Wrong: Adventures in the Margin of Error, 2010, Harper Collins. [19] Nate Silver, The Signal and the Noise, 2012, Penguin [20] Peter Sims, Little Bets, Free Press, 2011. [21] Diomidis Spinellis, “Quality Wars: Open Source versus Proprietary Software,” in Andy Oram and Greg Wilson (editors), Making Software, 2010, O’Reilly. [22] Duncan J. Watts, Everything is Obvious Once You Know the Answer, 2011, Crown.

ACM SIGSOFT Software Engineering Notes Page 14 March 2014 Volume 39 Number 2