answer the topics and read the chapters

8

Inside an OLM

Postflight Reviews in the Israeli Air Force

In this chapter, we present an in-depth analysis of after-action reviews, the most frequently discussed type of organzational learning mechanism (OLM) in the literature on organizational learning. The literature on this OLM describes after-action reviews in numerous settings, including high-hazard plants (Carrol, 1995, 1998), Microsoft (Cusumano & Selby, 1995), British Petroleum (Gulliver, 1987), and the U.S. Army (Baird et al., 1997). These descriptions focus on the procedural aspects of the after-action reviews and the stages that are followed. They also focus on how to conduct after-action reviews that produce lessons learned that are valid and have an impact. Our objective in this chapter is to further demystify organizational learning by examining the learning dynamics in after-action reviews: What goes on beneath the surface of the procedure- following activities? How do participants think and feel? How do they learn from the review process? How does the Israel Defense Force Air Force (IDFAF) actually learn?

We chose to study postflight reviews owing to their reputation as a “best practice” in the Israel Defense Force and in Israel in general. Air Force officers are in high demand in executive courses as lecturers on postflight reviews.

The importance of postflight reviews in the working life of the air-crews is evident from their integration into the working routine of the squadron. Training days begin with a 30-minute briefing for the aircrews that are scheduled to fly. Occasionally, the briefing includes a review of relevant lessons learned from past missions. Next, the aircrews who fly in the same formation (fighting units of two to eight planes) attend short formation briefings facilitated by the formation leaders (“number ones”). Assignments to the number one position are not based on seniority, a unique practice of the IDFAF that is designed to accelerate the development of junior officers. This means that relatively junior pilots may lead and later debrief pilots who are their senior in rank and experience, a practice that has direct implications for the postflight reviews culture.

A typical mission lasts between 30 and 60 minutes and is followed by a 45-minute formation postflight review, facilitated by the formation leader. Typical training days consist of the cycle of formation briefing– mission–formation postflight review, repeated two or three times, with occasionally an additional nighttime cycle. Each day concludes with a 60-minute squadron level postflight review attended by all formation aircrews and representatives of the relevant support units. This review session is facilitated by the squadron commander, one of the two deputies, or a veteran formation leader.

Formation postflight reviews are conducted in various designated locations in the squadron. These are equipped with VCR and TV screens for reviewing flight films recorded by cockpit-mounted VCRs. Morning briefings and daily squadron postflight reviews are conducted in a central briefing room equipped with maps, VCRs, and overhanging TV screens. Postflight reviews account for 40% to 50% of training and operational time on flying days, a proportion that is reduced in wartime owing to heavier workload and the requirement for longer rest periods. Aircrews spend more time on reviewing their missions than in executing them. This time allocation explains one pilot's assertion that in regard to skill improvement, “flying without reviewing is equivalent to not flying.”

Formation and daily squadron postflight reviews focus on different issues. Formation postflight reviews focus on what happened during the mission and how each pilot flew in combat exercise. The entire group reviews each pilot's video record meticulously. Particular attention is paid to errors, rule infractions, inappropriate actions that resulted in suboptimal outcomes, causes of error, potential remedies, and alternative actions that should have been taken. Daily squadron level post-flight reviews deal more with output: mission accomplishment and the functioning of the formations as units. They focus on errors that are generally relevant due to their pervasiveness, excessive risk, or rule infractions that require disciplinary action. Based on these criteria, pilots select their own film segments for review in the daily postflight reviews. In practice, formation postflight reviews focus on learning, and daily postflight reviews focus on knowledge dissemination and social control functions, as will be elaborated later in this chapter.

The chapter is based on observations of debriefings and on in-depth interviews with 13 pilots and navigators, a representative sample of the aircrews of a squadron of F-16 two-seat combat airplane. All the interviews began with the open-ended question, “How would you describe the postflight reviews to a person who has not seen one?” The answers to this question provided a wealth of information on the nature of the postflight reviews and the functions that they fulfill. Ostensibly, the function of postflight reviews is to improve pilots’ and navigators’ performance by learning from their experience. To our surprise and the surprise of officers who read the final report, the reviews fulfill a variety of additional functions or purposes. These additional functions of the postflight review mean that learning went beyond the detection and correction of error by individual pilots.  Table 8.1  presents verbatim quotes from the interview protocols that illustrate how the functions were expressed by the pilots and navigators.

 LEARNING FUNCTIONS

Postflight reviews fulfill three basic functions: learning functions, social control functions, and psychological functions. Altogether there are seven specific learning functions that we divided into three process and four output functions. The four output functions are improving individual performance, improving formation performance, improving training methods, and developing the doctrine and standard operating procedures of the squadron and of the IDFAF.

The three process functions are the construction of a valid mental model of the pilots’ actions during the mission, learning from others, and learning from failure. We labeled them “process functions” because they facilitate the accomplishment of the output functions. They form the core of the learning process in the postflight review, so understanding how they operate can shed considerable light on how this OLM works.

Table 8.1 Postflight Review Functions

Several researchers (Kim, 1993; Senge, 1990; Wack, 1985) suggest that organizational learning is concerned with changing managers’ shared mental models. Lessons learned based on distorted mental models are not likely to be valid. Owing to the speed with which events take place in air combat, aircrews sometimes form distorted mental models of what happened in the air. The postflight reviews help to improve mental models by three means: the removal of time and other distracting pressures, the VCR records, and the input from other participants, who help to correct misinterpretations of the meaning of the objective events.

Because postflight reviews are carried out in group settings, it is possible to benefit both from the input of informed colleagues, as was noted before, and from reviews of their performance. Some researchers have defined learning in general and organizational learning in particular as the detection and correction of error (Argyris & Schön, 1996). Other researchers dispute this definition as too narrow and suggest that learning from success is as important as learning from failure (Ellis et al., 1999). Echoing the facilitative effect of the costs of error on organizational learning (see  Chapter 5 ), virtually every pilot described the essence of the postflight reviews by a triple mantra-like slogan: “What happened? What went wrong? How can we do better next time?” Thus, although postflight reviews are held after every mission irrespective of its success or failure, the attention of participants is still focused on failure.

 SOCIAL AND CONTROL FUNCTIONS

Like all OLMs, postflight reviews are social activities that take place within formal organizations. Therefore, they fulfill social and control functions that go beyond learning itself. These functions include disciplining and culpability fixing, socialization, monitoring performance, and communicating commander's intent. The daily postflight review offers the squadron's commanders an opportunity to monitor the performance of individual pilots and navigators—including their behavior in the postflight review itself. It enables commanders to note repeated performance problems, breaches of safety regulations, and other impediments to the squadron's functioning.

When pilots break a rule, particularly safety regulations, they are reprimanded, or worse, by the squadron commander in the daily post-flight reviews. Errors of judgment are not punished, but public disclosure of error is difficult for two reasons. On the emotional level, there is loss of face. On the cognitive level, there is the requirement to supply a plausible explanation. Pilots are expected to identify their errors, try to explain them in a reasonable fashion, and suggest alternative courses of action. “Owning up to an error,” said one pilot, “and coming up with a satisfactory explanation for 40 other people is a considerable intellectual effort.”

Disciplining and culpability fixing involve the inherent conflict between punishing subpar performance and tolerating error for the purpose of learning. This function has to be fulfilled with tact: Overdoing the control function can cause irreparable damage to the psychological safety essential to learning. Strict monitoring and holding people accountable may entail a punitive stance that is inconsistent with a nonevaluative and safe atmosphere that promotes learning (Edmondson, 1999; Schein, 1993). The squadron's commander recognized this conflict:

If I charge a fine for every error, people will stop disclosing their errors—it's as simple as that. And if I avoid disciplining altogether, they will get the message that “everything goes.” That's why the postflight review is an intricate business that must be handled with judgment and care.

The commander's dilemma is partly mitigated by the fact that the pilots recognize the legitimacy of disciplinary action. As one pilot said, “Flying is hazardous, and without a disciplining framework, there are bound to be crashes.”

An interesting by-product of the disciplining and culpability-fixing function is that pilots learn to “fly the postflight review.” In other words, they learn to fly in a manner that will pass the scrutiny of others. Another interesting effect is that it raises the vigilance of participants in the formation postflight reviews. No one wants to gloss over bad performance in the supportive surrounding of the formation review, only to be caught in front of the entire squadron in the more formal daily review.

The postflight review is an arena for socializing pilots to the learning norms that contribute to its effectiveness. This process begins in flight school and continues throughout active service until it becomes second nature to the pilots. Interestingly, this intense socialization influences the pilots’ behavior in all spheres of life. For example, one pilot noted that

I, and in my opinion every pilot, learned to debrief everything that I do and goes wrong. If I put the baby to sleep and he keeps crying or if I fall from the surfboard, I automatically ask myself: What happened? What went wrong? How can I do better next time?

We believe that control, disciplining, and culpability fixing can be carried out simultaneously with learning only because the learning norms and the practice of debriefing are so deeply ingrained among the pilots and navigators. We return to this point when we discuss how psychological safety is achieved in the highly critical milieu of the post-flight review.

Daily postflight reviews engage the squadron's command with groups of pilots and navigators, including those who are only part-time in the squadron. This provides the commanders with an opportunity to exercise leadership and to communicate their intents. The postflight reviews provide a forum in which they can explicitly make their agenda known by pronouncing their policies and objectives. It also offers opportunities for making this agenda known implicitly through their behavior and reactions to the information revealed by the VCRs and to the behavior of the other participants during the post-flight reviews.

 PSYCHOLOGICAL FUNCTIONS

Psychological functions concern the effects on the participants of participating in postflight reviews. They include resilience building, bonding, involving, recognition and reward, and social comparison. Participation in postflight reviews helps build resilience by teaching participants to endure the intense stress experienced when their errors are exposed in public. Participation in the postflight review also strengthens the emotional ties of pilots and navigators to the squadron and to their colleagues. One interviewee referred to them as “the gathering of the tribe around the campfire.” Because of their reputation as a best practice and their importance in the Air Force, postflight reviews have become a kind of bonding ritual. In addition to sharing their errors with each other, participants affirm their membership in an elite group, in an elite unit, and in a unique process at the vanguard of the defense of the State of Israel. The postflight reviews are also a conduit through which participants can be directly involved in changing routines and procedures of the squadron and the Air Force at large.

Postflight reviews involve the two most important activities of the flight squadrons: flying and debriefing. Behavior in the postflight review, not just performance in combat, is a source of social recognition and reward—or the opposite. Pilots and navigators are expected to behave nondefensively and to make apt comments and suggestions. These behaviors influence the position of participants in the formal and informal social order in the squadron. Similar to every group of people who engage in comparable tasks or occupy comparable roles in organizations, participants in postflight reviews compare their performance to those of other participants (Festinger, 1954).

The various functions of the postflight reviews reveal the dynamics of learning in them. Learning is principally driven by the high costs of error: injury, death, or capture. In general, postflight reviews are designed to produce improvements in four areas: personal performance, squadron performance, doctrine, and training methods. To achieve these improvements, participants are concerned with three issues: What happened (the construction of a valid mental model of the mission); what went wrong (the detection of error); and what should be done to prevent or correct the error in the future (drawing of lessons learned).

The VCRs provide objective input for answering the first two questions. Input of fellow navigators and pilots contributes to the validity and utility of the answers to the last two questions. Monitoring and disciplining provide additional motivation for learning. Communicating the commander's intent sets specific goals for learning. Socialization helps to internalize norms of behavior that facilitate learning (see below).

The psychological functions support learning in two ways. Resilience building, as well as recognition and reward, helps participants struggle with the threat and loss of face generated by the public exposure of failure, social comparison, and culpability fixing. Bonding supports socialization and generates organizational commitment and involvement, which motivate pilots to draw lessons regarding doctrine and training methods, and not just in regard to themselves.

 PSYCHOLOGICAL SAFETY AND LEARNING CULTURE IN A HIGH-THREAT ENVIRONMENT

Socialization and bonding point to the cultural aspects of the postflight reviews that, together with the psychological aspects of participation, are key for a full understanding of the dynamics of learning in the review process. In Chapter 4 , we noted that psychological safety is essential for productive learning. The high degree of transparency produced by the VCR records, the rigorous search for errors, and the requirement for accounting for one's own errors in a plausible fashion seemingly create an environment that is highly threatening, particularly in the daily reviews. Indeed, interviewees referred to the difficulty of participating in the postflight review:

Every pilot will tell you that as soon as we make a bad mistake, the humiliation of this being seen by everybody at the daily postflight review jumps right up in front of our eyes.

Occasionally the postflight review simply kills you: “Why did you make this error? Here you were shot down. Here you screwed up the formation. Here you flew below some limit. There you broke some safety rule. Here I was shot down because of you.”

Sometimes nothing goes your way, just as on other occasions everything does go your way. That is why postflight reviews often require you to be strong.

How do pilots and navigators learn productively in the postflight reviews in spite of this difficulty?

The answer lies in three factors: the postflight review culture, the psychological safety that it engenders, and the participants’ long process of socialization into the postflight review culture. Pilots and navigators refer to a unique “debriefing culture” that is essential for rigorous inquiry. Analysis of the interview protocols and observation of formation and daily reviews showed that this culture consists of the five norms specified by the cultural facet of the multi-facet model: inquiry, integrity, issue orientation, transparency, and accountability (see Table 8.2 ).

Ed Schein, who studied organizational cultures (Schein, 1985), suggested that in addition to norms and values, organizational cultures consist of basic assumptions that are taken for granted by the organization members. The postflight reviews are based on a number of such deeply and commonly held assumptions:

· Personal experience, particularly if it is publicly reconstructed, provides an opportunity to learn and improve.

· Errors of judgment, but not rule infractions, are inevitable and represent opportunities for learning.

· Learning from experience is best accomplished by those who have participated in the experience.

· Learning is best done in group settings because individuals can benefit from sharing their knowledge with one another.

· Individual and group learning should be shared by other member units in the organization.

The cultural norm of issue orientation, coupled with a belief that errors are inevitable, reduces the loss of face generated by public exposure of error. All participants in the review make errors, and each one in turn is critiqued and has errors exposed. Furthermore, because the norms of inquiry, integrity, issue orientation, transparency, and accountability are so deeply ingrained, the loss of face generated by breaking them is greater than that which is generated by the public exposure of error. Finally, the VCRs ensure that criticism is based on hard evidence, thereby protecting participants from arbitrary criticism and enabling them to refute unjust criticisms.

Table 8.2 Postflight Review Values

Trust is an important component of psychological safety ( Chapter 4 ). Two types of trust that contribute to psychological safety are present in the postflight reviews. The first is trust in other people. Owing to the norm of issue orientation, participants can trust that the criticism of their colleagues is not motivated by blame seeking or malice. The second is trust in the system. The postflight review process is trustworthy because it is fair (all participants are subjected to the same treatment), issue oriented, and evidence based.

The psychological safety present in the postflight reviews differs from the sense of security engendered by the ability to cover up one's errors and by the presence of colleagues who are willing to collude in this venture. The latter is dysfunctional for learning and likely to produce a lingering apprehension of having to pay for both the original error and the subsequent cover-up.

Pilots and navigators undergo a long process of socialization in which they internalize the assumptions and behavioral norms of the postflight review culture. This process begins right from flight school:

When a cadet begins to tell stories during debriefing, the instructor cuts him short: “Stop the stories and come to the ‘match point.’ Why did you fail?”

This long process of socialization selects out those individuals who cannot withstand the criticism. Those who survive acquire the resilience to withstand failure and to accept criticism in the service of learning.

 WHAT CAN MANAGERS LEARN FROM POSTFLIGHT REVIEWS?

According to conventional wisdom, it is impossible to draw general lessons from unique cases. The uniqueness of the postflight reviews is salient on many dimensions. Looking just at the most prominent ones, the fact that the squadron is an elite military unit sets it apart, not just from non-Israeli military and civilian organizations but from most units in the Israel Defense Force. Pilots and navigators are a highly selective professional population who undergo a long process of deliberate socialization to the postflight review learning culture. The task structure of the squadron is unusually conducive to learning, first because of the feasibility of VCRs to collect high-quality data and second because learning (training and, more generally, preparing for the next operation or war) is a major task of the military. Finally, both cost of error and proximity to the core task of the organization are high.

Nevertheless, the case of the postflight reviews offers some lessons for managers and consultants who wish to improve processes of learning in a certain organization:

 

1. Similar to all best practice examples, the case sets a standard for high-quality learning. Managers of any system can use these standards to evaluate the frankness of the interchanges among participants and the rigor in which the detection and correction of error is pursued in their own OLMs.

2. It pays to invest in collecting good data for learning through technical means and by instituting the proper culture and policies. The Israeli Air Force places enormous effort and resources in obtaining such data. Culture and psychological safety are all the more important for obtaining good data when technical means such as VCRs are not feasible, as is the case in most situations. Good data can also be collected by instituting policies that require careful documentation and retention of data throughout the duration of projects or in conjunction with task performance.

3. It is important to explicitly set precise performance objectives and clear standards for satisfactory performance. In the post-flight reviews, performance standards are set by the mission's objectives. Thus, pilots said that preflight briefings, in which objectives are stated, are actually part of the postflight debriefing.

4. Learning questions are important. Even with all of the technical tools for creating transparency, learning is stimulated by good thought-provoking questions. In the postflight reviews, there are always four questions. The first three are posed explicitly by participants: What happened? What went wrong? How can we do better next time? The first question leads to a better mental model of the reality. The second question frames the learning. The third question generates “actionable” knowledge to be put into practice. The fourth question is, why did the situation go wrong? This question leads to a satisfactory explanation, or theory, that constitutes new knowledge.

5. After-action reviews provide an opportunity to exercise leadership, announce objectives, and bolster policies that are designed to achieve them.

6. The psychological safety necessary for productive learning can be fostered, even in a highly exposed and threatening environment. This kind of safety is not based on mutual protection from criticism but rather on fairness, trust in the system, and trust in others.

 

Finally, the most important lesson is that introducing or improving learning processes is challenging—but doable. On one hand, it requires much more than developing a vision or pronouncing a mission that includes organizational learning. On the other hand, it does not require the transformation of the organization to a “learning organization.” The postflight reviews are cultural islands in which pilots and navigators work on improving the core task of their organization. The Air Force has tried but never succeeded in disseminating this process to support units and noncombat flight squadrons. However, the Air Force never puts resources into these dissemination efforts that are comparable to those invested in postflight reviews in the combat squadrons.

Everyone with whom we spoke—pilots, navigators, and high-ranking commanders—all agreed that the Air Force reaps great benefit from the postflight reviews. It continually generates and applies cutting-edge knowledge and skill that make the Israel Air Force a top performer in its field. It has done this by sharply focusing efforts on developing an appropriate OLM and a learning culture in the area that is most important to the organization's success. This is a lesson that is certainly applicable to any organization.

________________

AUTHOR’S NOTE: This chapter is based on Ron, Lipshitz, and Popper (in press).