Development of Strategic Intelligence Analysis and Products
iega13l3
hedley2.pdf
This article was downloaded by: [American Public University System] On: 02 December 2012, At: 21:53 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
International Journal of Intelligence and CounterIntelligence Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/ujic20
Learning from Intelligence Failures John Hollister Hedley Version of record first published: 23 Feb 2007.
To cite this article: John Hollister Hedley (2005): Learning from Intelligence Failures, International Journal of Intelligence and CounterIntelligence, 18:3, 435-450
To link to this article: http://dx.doi.org/10.1080/08850600590945416
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
JOHN HOLLISTER HEDLEY
Learning from Intelligence Failures
The Russians have a host of sayings, and one that seems pertinent goes like this: ‘‘If you see a Bulgarian on the street, beat him up. He will know why.’’ Given the enormously tragic events of 11 September 2001, and the dismaying absence of weapons of mass destruction in post-invasion Iraq, any Central Intelligence Agency (CIA) officer seen on the street in Langley, Virginia, could be pummeled and would likely be someone who would know why. For nearly four years it has been hard to go wrong by criticizing the Agency.
But putting this pummeling—verbal pummeling, thankfully—into perspective is possible and desirable. It is part and parcel of the ‘‘intelligence school of hard knocks.’’ It can be put into perspective with four simple observations:
. Allegations of intelligence failure are inevitable.
. This is true in large part because, in intelligence, failures are inevitable.
. Intelligence organizations do learn (as well as suffer) from the allegations and the failures.
. Even though it is impossible to learn once and for all how to prevent the recurrence of something inevitable, the ratio of success to failure probably can be improved.
Dr. John Hollister Hedley, during a career of more than thirty years with the Central Intelligence Agency, edited the President’s Daily Brief, briefed President George H. W. Bush at the White House, served as Managing Editor of the National Intelligence Daily, and was Chairman of the CIA’s Publications Review Board. He has also taught intelligence at Georgetown University, Washington, D.C., and is now an independent consultant on security matters. An earlier version of this article was prepared for the annual meeting of the International Studies Association, Montreal, Canada, March 2004.
International Journal of Intelligence and CounterIntelligence, 18: 435–450, 2005
Copyright # Taylor & Francis Inc. ISSN: 0885-0607 print=1521-0561 online
DOI: 10.1080/08850600590945416
AND COUNTERINTELLIGENCE VOLUME 18, NUMBER 3 435
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
ALLEGATIONS OF INTELLIGENCE FAILURE ARE INEVITABLE
From the United States’s perspective, anytime something in the world happens that Washington didn’t want to have happen will be seen by some as an intelligence failure. Anything that catches the U.S. by surprise and is bad news is deemed an intelligence failure. This is not to be cynical, but realistic. However much insistence that it really is not reasonable to expect intelligence to predict or prevent any and all surprises, the reality is that this is what defines an intelligence failure. An unrealized prediction is a failure, especially when those who fund and monitor the performance of intelligence declare it to be.
Thus, allegations of intelligence failure are a ‘‘given,’’ if for no other reason than the fact that politicians and public servants abhor being caught off guard, especially by a development that falls within their purview. And journalists relish the fact that politicians, public servants, and governmental institutions get caught off guard and thus are vulnerable to public embarrassment. Doubtless, certain allegations of intelligence failure are unfair in that they derive from ignorance of the facts and from unrealistic expectations, but each of these conditions may also be a ‘‘given.’’ Unfair or not, allegations of intelligence failure can be expected in a robust democratic political system involving the ever-volatile mix of free speech, politicians, and the news media.
This is not to be cavalier or even flippant about charges of intelligence failure, but to put them in perspective. But at least three serious concerns are warranted about the levying of such charges: (1) how often they reflect an inexcusable lack of understanding—even the most basic grasp—of the very nature of an intelligence assessment; (2) the tendency—indeed the haste—to attribute malfeasance to those who err in an assessment, ridiculing them as professional incompetents or demonizing them for political pandering; and (3) that responses to allegations of failure will invariably be labeled by those who disagree with a policy or dislike an outcome as being an attempted ‘‘cover-up.’’ Yet, the fact remains that allegations of intelligence failures are the cost of doing business in a free and open society. And because of the far-reaching ramifications that intelligence judgments can have, questions surrounding what intelligence did or did not indicate will continue to be placed under public, political, and professional microscopes.
IN INTELLIGENCE, FAILURES ARE INEVITABLE
The statement that intelligence failures are inevitable has the certainty of a law of physics. No one person or organization can be right on all subjects at all times, nor is it reasonable or realistic to have such an expectation— either in making an intelligence assessment or in passing judgment on it
436 JOHN HOLLISTER HEDLEY
INTERNATIONAL JOURNAL OF INTELLIGENCE
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
after the fact. To do their job well, intelligence analysts must be willing to take risks. No matter how incomplete, inadequate, uncertain, or contradictory the information on which a judgment must be made, the judgment is nevertheless expected and must be made. Making it necessarily entails a recognition of the risk that the judgment can miss the mark. This should go without saying, but it doesn’t. (In intelligence work, having enough evidence to indict is common; but there’s almost never enough to convict.) Would that all those who would take to task the makers of an erroneous assessment be somewhat more understanding of the intellectual risks that had to be taken, and the unavoidability of taking them, but they aren’t. The expectation after the fact is that the analysts should have gotten it right.
Those who are caught up in the doing or studying of intelligence work should not be startled or dismayed about this. It should be expected and accepted, pure and simple, as a cost of doing business in a free country. There is no need to fall on one’s sword or spend time lamenting the unfairness of the allegations. They are a fact of life, and in many cases deserved. For as long as foreign policy surprises occur, and for as long as open societies have free news media and ambitious politicians, allegations of intelligence failure are likely to be made, virtually as being part of the process. Perhaps ‘‘process’’ is too dignified a term, but failures and the attendant allegations—the ‘‘school of hard knocks’’—is one from which lessons not only can be, but are learned.
The study of intelligence failures is perhaps the most academically advanced field in the study of intelligence,1 launched perhaps by Roberta Wohlstetter’s analysis of the U.S. failure to predict the Japanese attack on Pearl Harbor2—a failure which, more than any other factor, set in motion the post–World War II U.S. intelligence system. Impressive scholarly inquiries since then have probed the phenomena of intelligence failures and concluded that intelligence failures cannot be avoided.3 More than twenty- five years ago, for example, the scholar Richard Betts asserted that intelligence failures are not only inevitable, they are natural.4 The CIA’s experience surely is not unique in bearing out that conclusion.
THE LEARNING LESSONS
That the CIA’s experience exemplifies the contention that intelligence organizations can at least sometimes learn (as well as suffer) from both the allegations and the failures perhaps cannot be proved conclusively. It is impossible to comprehensively catalog in public the CIA’s record of failed assessments, the slings and arrows aimed at them, and the Agency’s responses in the way of corrective action (or action of corrective intent). Given the necessarily secret nature of intelligence work, too little can be
LEARNING FROM INTELLIGENCE FAILURES 437
AND COUNTERINTELLIGENCE VOLUME 18, NUMBER 3
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
known and cited of the record of assessments—how many, and how they stood the test of time—to determine a batting average. But several known episodes provide an illustrative sampling.
The U.S. intelligence system as a whole, and the CIA as a central clearing house for intelligence information, grew out of the colossal failure of pre– World War II intelligence organizations to provide warning of Japan’s 7 December 1941 attack on Pearl Harbor. In 1950, the nascent CIA was less than three years old when it failed to provide a clear warning of North Korea’s attack on South Korea. What followed illustrates a fairly common response: failure begets funds. More than once the phenomenon has been of failures being followed by increases in organizational size and funding. This happens when Congress, bent on demonstrating decisive corrective action, bestows money—as if a larger, more expensive organization would surely be less likely to fail.
Illustrative of this phenomenon, the war that followed the surprise invasion of South Korea was to the CIA what the launching of ‘‘Sputnik’’ was to the National Aeronautics and Space Administration (NASA). In the four years after the Soviet Union first put a satellite in space, NASA went from 900 to 33,000 employees; in the five years following the Communist invasion of South Korea, the CIA grew from 5,000 to 15,000 employees. Inevitably, agencies shuffle personnel, and bureaucrats reorganize. Intelligence failures may cost some jobs and stall some careers, but they also create jobs and launch careers as missions—and expectations—expand. A common consequence of an intelligence failure is to add missions and increase demands.
THINKING ‘‘RATIONALLY’’
Bureaucratically, the response to the warning failure on South Korea was to create the Office of National Estimates (ONE). The U.S., though hardly unique in recognizing the importance of estimative intelligence, became the first country to institutionalize it in a permanent bureaucracy. The National Intelligence Estimate (NIE) became, in the U.S. experience, a truly national product, drawing upon the analytical resources and reflecting the considered judgment of the many organizations making up the U.S. Intelligence Community (IC). Perhaps the most distinctive characteristic of that community is the degree to which it is organized for the systematic production of national estimative intelligence.5
Thus, a much larger CIA addressed the 1962 Cuban missile crisis. The head of the ONE was a former professor of European history, Sherman Kent, whose 30-year career in intelligence would earn him a reputation as perhaps America’s foremost practitioner of the analytic craft. Kent was directly involved in producing the estimate that, on 19 September 1962,
438 JOHN HOLLISTER HEDLEY
INTERNATIONAL JOURNAL OF INTELLIGENCE
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
pronounced the conclusion that the Soviet Union would be unlikely to introduce strategic offensive weapons into Cuba. Less than a month later, when the photographic evidence of 14 October became available, the assessment was proved wrong. Looking back, Kent himself analyzed the intelligence failure:
How could we have misjudged? . . .The short answer is that, lacking direct evidence, we went to the next best thing, namely information which might indicate the true course of developments.6
Kent and his colleagues could do no less. They did have the luxury of waiting for more and better data. ‘‘In brooding over an imponderable,’’ he later reflected, ‘‘there is a strong temptation to make no estimate at all . . . or to go for the worst case.’’7 But unfolding events and the need for policy decisions on how to tackle them would not allow for making no intelligence assessment at all. And, in the case of the NIE on the Soviet military buildup in Cuba, Kent and his estimators’ reading of the indicators led them off the mark. ‘‘We missed the Soviet decision to put the missiles into Cuba,’’ he reasoned, ‘‘because we could not believe that [Soviet leader Nikita] Khrushchev could make such a mistake.’’8
A lesson taught by this failure (one that unfortunately seems to require repeated relearning) is the need to be careful and skeptical in assuming that the object of an assessment is a rational actor according to the Western way of thinking. Some students of the Cuban missile crisis (Kent notably among them) ironically assert that the NIE actually was correct—in the sense that sending the missiles to Cuba was indeed a major error, one that a rational actor would have eschewed. In fact, this ‘‘irrational’’ act and the consequent humiliating Soviet withdrawal of the missiles contributed to Khrushchev’s ouster in 1964. But at the end of the day, an insufficient U.S. understanding of Khrushchev’s psychology and worldview resulted in the CIA’s intelligence assessment that Moscow’s sending its missiles would be an irrational move and thus not to be expected. Getting out of a ‘‘Western’’ mindset continues to be difficult, but is something CIA analysts acknowledge as being critically important in assessing the motives and policies of non-Western leaders in particular.9
Later, the Arab–Israeli wars of 1967 and 1973 illustrated that events in that troubled and tension-filled region have been as nettlesome for intelligence assessments as for policy initiatives. The two military conflicts provide interesting contrasts in successful and unsuccessful intelligence efforts to inform crisis decisionmaking. In 1967, the CIA and the rest of the U.S. Intelligence Community provided a valuable warning function. Although its intelligence analysis ran counter to views initially held by senior policymakers, President Lyndon B. Johnson and his National
LEARNING FROM INTELLIGENCE FAILURES 439
AND COUNTERINTELLIGENCE VOLUME 18, NUMBER 3
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
Security team ultimately adopted a policy based on intelligence analyses that alerted them to Arab troop movements, the thinking behind Egyptian plans regarding the Gulf of Aqaba, the likelihood of potential Soviet intervention in the support of the Arabs, and Israel’s ability to defeat a combination of Arab military forces.
By contrast, the U.S. Intelligence Community was as unanimously wrong in 1973 as it had been correct in 1967, concluding in 1973—as late as the night preceding the Egyptian attack across the Suez Canal—that the Arabs would not attack. The result was that the joint Egyptian–Syrian assaults along the Suez Canal and the Golan Heights surprised U.S. policymakers as well as those of Israel. What was different? Part of the problem was the same ‘‘rational actor’’ assumption that caused the embarrassing misreading of Khrushchev in the 1962 Cuban missile estimate. The CIA’s current intelligence analysis in one instance flatly stated, for example, that ‘‘For Egypt, a military initiative makes little sense at this critical juncture of President [Anwar] Sadat’s reorientation of domestic and foreign policies. Another round of hostilities would almost certainly destroy Sadat’s painstaking efforts to invigorate the economy and would run counter to his current efforts to build a united Arab political front. . . . For the normally cautious Syrian President, a military adventure now would be suicidal, and he has said so.’’10
By 1973 the military balance in the Middle East had shifted in Israel’s favor, so that intelligence analysts in Washington and in Tel Aviv believed Arab military inferiority would militate against an attack on Israel. U.S. analysts failed to explore the possibility that Arab leaders might decide to go to war—even at the risk of losing—if they believed they could thereby attain certain political objectives. On the eve of the war, the CIA reported, for example, that ‘‘The exercise and alert activities underway in Egypt may be on a somewhat larger scale and more realistic than previous exercises, but they do not appear to be preparations for a military offensive against Israel. . . . Tel Aviv assesses the Egyptian activity as normal, large-scale maneuvers and has not alerted its forces.’’11
So why this intelligence failure, this misreading of the intelligence indications as the outbreak of hostilities approached? A reflection thirty years after the fact suggests several reasons:
. Accepting statements, and their implications, at face value (taking statements by Egyptian President Anwar Sadat and Syrian President Asad about Israel’s military superiority as indicating that they would not initiate a war);
. Failing to look for or explore (or be trained for) denial and deception;
. Placing blind faith in an intelligence liaison service’s judgment—that of Israeli intelligence; and
. Failing to second-guess Israeli judgments dismissing the significance of Arab military preparations.
440 JOHN HOLLISTER HEDLEY
INTERNATIONAL JOURNAL OF INTELLIGENCE
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
Also figuring in this 1973 failure were cultural biases that led U.S. analysts to conclude that the Arabs could not have recovered so soon from their humiliating defeat in 1967, and could not have devised and concealed such an elaborate war plan. ‘‘If Arabs could not stand in line to get on a bus, how could they plan to cross the Suez Canal in the face of massive Israeli defensive lines on the other side?’’12
THE POST-MORTEM MOVEMENT
Many thoughtful, self-initiated attempts have been launched by the U.S. intelligence establishment to determine why analytical reasoning goes awry. To probe why U.S. intelligence did not adequately anticipate significant events, and to identify measures which might improve future performance, the U.S. Intelligence Community Staff in the early 1970s created a post- mortem program. The staff commissioned a Product Review Division to conduct the studies, reviewing the vast bulk of reporting on which published intelligence was based, reviewing all the published intelligence itself, and interviewing as many as possible of the parties involved in preparing relevant finished intelligence—including a representative selection of supervisors and high-level recipients of the intelligence. Seven topics were examined: the Arab–Israeli War of 1973; coverage of the anti- Salvador Allende coup in Chile in September 1973; India’s nuclear explosion of May 1974; the West Bank (a sequel to the Arab–Israeli post- mortem that had been promised the Director of Central Intelligence; the coup in Cyprus in July 1974; Egyptian military capabilities (concerning a last-minute change in a 1975 NIE to warn of an Arab attack that did not occur); and the seizure of the U.S. ship Mayaguez by Cambodia’s new Communist regime in 1975.13
The post-mortems produced some tangible organizational and procedural improvements in the way the Intelligence Community conducted its business. A horizontal network was set up to connect all the intelligence operations centers with similar centers in the White House, State Department, and Pentagon—the sharing of information being a problem as old as intelligence, this was no small accomplishment. The communications system for moving vital all-source information from the field to the President and other senior officials via the operations centers was overhauled. A new form of estimative warning paper also was created, the Alert Memorandum. But the post-mortem program lapsed in the mid- 1970s for a variety of reasons: some of its principal sponsors moved on; some intelligence officers saw it as an unnecessary exercise in self- flagellation; and it became embroiled in the House of Representatives Intelligence Oversight Committee’s effort to exploit public controversy over the CIA’s real and alleged abuses over the course of two decades.
LEARNING FROM INTELLIGENCE FAILURES 441
AND COUNTERINTELLIGENCE VOLUME 18, NUMBER 3
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
A constitutional standoff between Congressman Otis G. Pike (D., New York), the committee chairman, and President Richard M. Nixon over the President’s refusal to grant Congress permission to release classified information in the post-mortem on the Arab–Israeli War of 1973 was, in the eyes of a participant who bore a large responsibility for the program, the last straw.14 Once seized upon as a political football, the program was over.
INTO DANGEROUS TERRITORY
Perceptions of intelligence failure, of course, were not. Later in the same decade, what was widely regarded as the CIA’s failure to foresee the fall of the Shah of Iran and the Islamic revolution in 1978–1979 that turned Iran into a radical and hostile theocratic state served as a wake-up call for U.S. intelligence, which had focused its attention primarily on world Communism and the Soviet threat. The failure to provide adequate warning about the Iranian revolution helped spawn a restructuring of the CIA’s intelligence directorate, a reordering of intelligence priorities, and a significant reorientation of emphasis and organizational approaches to collection and analysis. It accelerated an institutional response to concerns about regional instability, the forces of nationalism, and the growth of terrorism. Political, economic, and military experts were integrated into a new organizational scheme aimed at enhancing interdisciplinary research and analysis. A new Office of Global Issues would tackle ‘‘transnational’’ topics.15 Thought was given to how the U.S. Intelligence Community’s formal Indications and Warnings (I&W) system, geared to signs of hostile military action, might somehow develop a political I&W system that would establish indicators that could broaden the concept of warning to include impending political instability.16
But the Intelligence Community’s failure to predict the Soviet invasion of Afghanistan soon brought new controversy. The IC had, in fact, accurately estimated the advantages and disadvantages of intervention. But, once again, the IC, having assessed that the disadvantages outweighed the advantages, concluded that the Soviets would act rationally, in accordance with U.S. perceptions of Soviet self-interest. The CIA’s Douglas MacEachin, who was directly involved in the assessments, recalls that one of the dark humor jokes circulating around the Agency after the invasion was that the analysts had gotten it right, and the Soviets got it wrong.17
The fact that those monitoring Soviet preparations evaluated them, not in terms of what the Soviets might intend, but in terms of how they fit with what U.S. analysts expected, illustrates, according to MacEachin, what those who study the history of intelligence performance will recognize as probably the most recurrent trap for analysts:
442 JOHN HOLLISTER HEDLEY
INTERNATIONAL JOURNAL OF INTELLIGENCE
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
Once having constructed an intellectual model of how the variables are likely to play out, each new piece of information is weighed in accordance with the components of that model. Evidence that does not fit is far more likely to be explained away than used to question the model’s validity. In this case, the actions taken (military preparations) were not used to interpret intentions so much as the conclusions about intentions were used to interpret the actions.18
REVIEWING THE JUDGMENTS
The CIA’s failure to predict the Soviet invasion of Afghanistan was grist for the mill of a renewed internal effort, this time in the early 1980s, to determine the causes of ‘‘instances when the intelligence community did not adequately anticipate significant events on the world scene,’’ and to identify ways to improve future performance.19 This time, the Director and Deputy Director of Central Intelligence appointed a Senior Review Panel to produce a study ‘‘on the quality of intelligence judgments preceding significant historical failures over the last twenty years or so.’’20 The cases nominated for review (to which the panel added an unspecified number of others ‘‘which have had major consequences for US interests’’) covered twelve subjects:
. The Likelihood of North Vietnam Intervention in South Vietnam
. The Likelihood of All-Out Soviet Support of Hanoi
. Cuba
. The Sino-Soviet Split
. The First Chinese Nuclear Test
. The Soviet ALFA-Class Submarine
. Libya
. The OPEC Price Increase of December 1973
. Ethiopia
. Afghanistan
. Iran
. Nicaragua—The Nature of Somoza’s Opposition21
REVIEW METHODOLOGY AND CONCLUSIONS
The panel’s approach was to concentrate on CIA and IC publications in the two or three years preceding each ‘‘critical or transforming outcome.’’ It examined the extent to which the main lines of analytical and estimative judgments were supplemented by the use of alternative hypotheses and speculative analyses going beyond developments strongly supported by the evidence. The panel looked for conjectures about lesser probabilities, hoping to judge whether ‘‘more speculative approaches might have proved in the end more realistic and succeeded in alerting the policy community
LEARNING FROM INTELLIGENCE FAILURES 443
AND COUNTERINTELLIGENCE VOLUME 18, NUMBER 3
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
earlier to outcome potentials largely ignored at the time.’’22 While noting that in recent years analytical quality and intelligence processes had improved dramatically (except in the length of national estimates), the panel found that a recurrent problem in the estimates that failed was that of recognizing qualitative change in situations in which trend continuity and precedent proved to be of marginal value, if not counterproductive. In the panel’s view, ‘‘the major factor in the failed estimates was overly cautious, overly conservative, single-outcome forecasting. . . .For the most part, they rested on the prevailing wisdom of the time and were reinforced by professional assessments of the available evidence.’’23
According to the internal CIA panel, which completed its report in December 1983, ‘‘this addiction to single-outcome forecasting’’ reinforces ‘‘some of the worst analytical hazards—status quo bias and a prejudice toward continuity of previous trends, ‘playing it safe,’ mirror-imaging, and predispositions toward consensus intelligence.’’24 The Senior Review Panel described the practice as being ‘‘compounded by what the British call ‘perseveration’ (a tendency for judgments made in the early stages of a developing situation to be allowed to affect later appraisals and an unreadiness to alter earlier views even when evidence requiring them to be revised becomes available.’’25
The CIA panel that conducted this study more than twenty years ago concluded that the central problems are how to deal with inevitable uncertainty, how to manage both greater and lesser probabilities concurrently, and how to cope with discontinuity and apparently unlikely outcomes. Clearly, such problems remain. In 1983, the recommendation was to increase sensitivities on the part of middle-level managers and analysts alike that these are in fact real problems and that failure to deal adequately with them will invite repeated failures. Specifically, the panel suggested that where there is general agreement on likely outcomes, national estimates should include an ‘‘alternative outcomes’’ section that would briefly spell out lesser probabilities and other possible developments not fully supported by the evidence.26
RESPONDING TO FAILURES, AGAIN
Whether or not this presentational approach was tried, intelligence failures and furors have obviously persisted. The fall of the Berlin Wall in 1989, followed rapidly by the collapse of the Soviet empire and the formal dissolution of the USSR in 1991, brought allegations that intelligence had failed to forecast the most significant international development of the second half of the twentieth century. Never mind that CIA analysts took Mikhail Gorbachev’s domestic initiatives seriously and looked ahead to describe their likely impact, concluding that the consequences might well
444 JOHN HOLLISTER HEDLEY
INTERNATIONAL JOURNAL OF INTELLIGENCE
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
not be what he hoped for and quite possibly be beyond his control. Never mind that, in 1991, a CIA analysis, aptly titled ‘‘The Soviet Cauldron,’’ warned that matters were coming to a head in Moscow some months before the coup attempt that wrote the USSR’s final chapter. Given the fact that there was no clear forecast of a total collapse, along with a timetable (which no other expert or groups of experts provided), the notion lingers that U.S. intelligence was caught entirely off guard.27
Sensitive to these allegations and smarting from other warning failures in the 1990s—such as the late realization that Iraq intended to invade Kuwait; a test of a North Korean missile displaying unexpected range and technology; and the surprise of India’s nuclear test—leaders of the CIA’s analytic directorate called for unprecedented emphasis on training in analytic tradecraft to sharpen analytic skills. Douglas MacEachin, as head of the intelligence directorate, especially insisted that analysts beware of fitting information into an existing model rather than using data to reassess the premises of the model itself.28 John McLaughlin (later to rise to Deputy Director, then Acting Director of Central Intelligence), head of the CIA’s intelligence directorate from 1997 to 2000, established the Sherman Kent School of Intelligence Analysis, the Agency’s most intensive effort to learn from the lessons of the past.29 McLaughlin also established a Senior Analyst Service to provide a career track by which analysts could advance to senior grades on the basis of expertise and performance alone, rather than by opting for staff and managerial assignments.30
These intensified efforts to find ways to instill lessons learned from off-the- mark assessments in the CIA’s first fifty years could not—obviously— prevent further intelligence surprises and erroneous estimates. The Islamist suicide attacks on the World Trade Center and the Pentagon of 11 September 2001, and the apparent absence of weapons of mass destruction in post- invas ion Iraq, are only the most monumental in their consequences, occasioning multiple, protracted inquiries. The findings were taken seriously and reviewed carefully, and legislation was passed in December 2004 to drastically revamp the Intelligence Community’s structure. Nevertheless, the following observations remain relevant:
(a) An appalling lack of understanding persists about the very nature of making intelligence assessments. In the United States at least, any researcher can easily find out as much about the CIA as about the Bureau of the Budget, the Bureau of Land Management, or a host of other governmental institutions that are much less under a perennial public microscope and much less written about than the CIA. Experienced journalists, as well as those involved in the extensive oversight of intelligence, should know more and reveal more in their public commentary about how intelligence assessments are done, and what the
LEARNING FROM INTELLIGENCE FAILURES 445
AND COUNTERINTELLIGENCE VOLUME 18, NUMBER 3
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
inherent limitations are. A better grasp should be possible of the fact that the grist for the intelligence analyst’s mill is a mix of often incomplete and frequently contradictory fragments of information—in one case, a dearth of data, and in another, an overwhelming volume. If lamenting allegations of failure is pointless—and, in the main, it is—lamenting the ignorance about much of what intelligence is, what it can and cannot do, how it operates, and why it is valued by those who need and use it, regardless of political party or presidential administration, may not be.
(b) Lessons learned do not stay learned. Good analysis is based on good data, and good use of that data. The perennial problem of intelligence support to policy is the need to attempt good analysis even in those instances in which there is no good data. As the previously enumerated failures indicate, the net assessments arrived at by deduction, induction, inference, or intuition cannot, in the final analysis, escape human nature. Assessments are subject to turnover in personnel, who bring varying degrees of experience; changing circumstances and the pressures of time and events; and various other human factors that together can make what seems to be clear- headed reasoning look otherwise in retrospect.
(c) Some lessons will be drawn and acted upon. Institutional responses to the events of 11 September 2001, have been extraordinarily far- reaching. Expanding and reorienting the mission of the Federal Bureau of Investigation; establishing a Cabinet-level Department of Homeland Security; and creating a Terrorist Threat Integration Center were only the preliminaries to the almost total restructuring of the IC. Surely other attempts will be made to apply the learned lessons—in organization and procedure, and in substantive emphasis within the ranks of analysts and the training they will receive. The result may actually be improved intelligence collection and assessments. But the safest bet is that neither external nor internal reforms can somehow ‘‘fix’’ intelligence assessments in a way that prevents mistakes. Renewed, absolutely genuine efforts can be made to do better and to learn certain lessons once and for all . But the safest prediction is that future intelligence judgments still will not be infallible.
(d) Intelligence can indeed suffer in the process of publicly probing failures in order to better understand and learn from them. The downside is the disruption and distraction caused by the many inquiries and protracted internal and external soul-searching. Stuart Cohen, who was Acting Chairman of the National Intelligence Council when the 2002 National Intelligence Estimate on Iraq’s weapons of mass production was published, has noted that
Confronting allegations about the quality of the U.S. intelligence performance has forced senior intelligence officials to spend much of
446 JOHN HOLLISTER HEDLEY
INTERNATIONAL JOURNAL OF INTELLIGENCE
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
their time looking backward. I worry about the opportunities lost because of this preoccupation, but also that analysts laboring under a barrage of allegations will become more and more disinclined to make judgments that go beyond ironclad evidence—a scarce commodity in our business. If this is allowed to happen, the nation will be poorly served and ultimately much less secure.31
The disruption and distraction can be part of the ‘‘process,’’ and one of the costs of doing business in a democracy. But the danger of avoiding intellectual risks, of trying above all not to be wrong, can take a greater toll. Playing it safe can dilute and devalue the intelligence product.
IMPROVING THE RATIO OF SUCCESS TO FAILURE
Improving the batting average of success is both a serious, continuing quest and a fervent hope. Though it is impossible to learn once and for all how to prevent the recurrence of something that is inevitable, the hope is that the ratio of success to failure will improve. There is always room for improvement. But achieving once and for all a general public understanding of what intelligence is and isn’t; what it can and cannot do; and what is reasonable to expect of it is perhaps equally impossible. To expect intelligence to predict or prevent any and all surprises is not reasonable. But it is unreasonable to expect that intelligence analysts will not make mistakes, just as policymakers have and will. Another hope is that the level of public understanding also will improve.
Finally, even as the CIA is expected to ‘‘roll with the punches,’’ any presidential administration, regardless of party, should recognize that taking the blows that intelligence judgments may deal to its policy hopes is in its self-interest and that of the country. Getting bad news about how policy efforts are faring is an indispensable and ultimately constructive part of the process. The Intelligence Community does not exist merely to steal secrets abroad, but to make brutally honest assessments, independent of a policy agenda about the information it gathers. For the IC, this means resisting inevitable political pressures from the agenda-bearers of any presidential administration. It means ‘‘telling truth to power,’’ having the backbone to offer unwelcome assessments when they are judged to be the most professional, accurate, and objective assessments possible.
Of course, the reality is that the recipients of unwelcome assessments will not always greet them with such reasonableness. For those who must render the assessments, no guarantee is possible that the ratio of hitting the mark to missing it will change dramatically, best efforts notwithstanding. The problem of preventing intelligence misjudgments remains unsolved, because uncertainty itself is the problem. The seemingly unknowable is compounded by fragmentary and contradictory pieces of information from
LEARNING FROM INTELLIGENCE FAILURES 447
AND COUNTERINTELLIGENCE VOLUME 18, NUMBER 3
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
sources of questionable reliability. Having to make the assessment anyway, because some reading of a situation is needed, is the problem as well as the challenge. Writing forty years ago about the particular challenge of crafting a National Intelligence Estimate in the U.S., Sherman Kent said it well:
IfNIEs could be confined to statements of indisputable fact, the taskwould be safe and easy. Of course the result could not then be called an estimate. By definition, estimating is an excursion out beyond established fact into the unknown—a venture in which the estimator gets such aid and comfort as he can from analogy, extrapolation, logic, and judgment. In the nature of things he will upon occasion end up with a conclusion which time will prove to be wrong. To recognize this as inevitable does not mean that we estimators are reconciled to our inadequacy; it only means we fully realize that we are engaged in a hazardous occupation.32
There still is no assurance that accurate conclusions will be drawn from the typically incomplete and ambiguous information from which intelligence assessments must be made. Intellectual risks must be taken. Failures will occur. Invective and censure will follow. The ‘‘school of hard knocks’’ will remain in session.
REFERENCES 1 Woodrow J. Kuhns provides an excellent discussion of the subject in the chapter entitled ‘‘Intelligence Failures: Forecasting and the Lessons of Epistemology,’’ in Richard K. Betts and Thomas G. Mahnken, eds., Paradoxes of Strategic Intelligence: Essays in Honor of Michael I. Handel (London: Frank Cass, 2003), pp. 80–97.
2 Roberta Wohlstetter, Pearl Harbor: Warning and Decision (Palo Alto, CA: Stanford University Press, 1962).
3 See, for example, Woodrow J. Kuhns, ‘‘Intelligence Failures’’; Michael Handel, ‘‘The Yom Kippur War and the Inevitability of Surprise,’’ International Studies Quarterly, Vol. 21, No. 3, September 1977, pp. 461–502; Richard K. Betts, ‘‘Analysis, War and Decision: Why Intelligence Failures Are Inevitable,’’ World Politics, Vol. XXXI, No. 1, October 1978 reprinted with permission in Studies in Intelligence, Vol. 23, No. 3, Fall 1979, and Christopher Brady, ‘‘Intelligence Failures: Plus ca Change. . .,’’ Intelligence and National Security, Vol. 4, No. 8, October 1993, p. 86.
4 Richard K. Betts, ‘‘Analysis, War and Decision,’’ op. cit., p. 54.
5 For a brief overview, see the introduction to Donald P. Steury, ed., Intentions and Capabilities: Estimates on Soviet Strategic Forces, 1950–1983 (Washington, DC: Center for the Study of Intelligence, 1996). For a detailed discussion of National Intelligence Estimates and the estimative process, see Harold P. Ford, Estimative Intelligence: The Purposes and Problems of National Intelligence Estimating, 2nd ed. (Lanham, MD: University Press of America, 1993).
448 JOHN HOLLISTER HEDLEY
INTERNATIONAL JOURNAL OF INTELLIGENCE
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
6 Sherman Kent, ‘‘A Crucial Estimate Relived,’’ originally appearing in the classified Spring 1964 issue of CIA’s internal journal, Studies in Intelligence, republished in Studies in Intelligence, Vol. 35, No. 4, Winter 1991, p. 67.
7 Ibid., p. 68.
8 Ibid., p. 70.
9 Peter Clement, ‘‘The Cuban Missile Crisis,’’ in Fifty Years of Informing Policy, published in 2002 by the Center for the Study of Intelligence, commemorating the 50th anniversary of CIA’s Directorate of Intelligence, p. 92. One of the best books available on this assumption of irrational action is Paul Berman’s Terror and Liberalism (New York: W. W. Norton, 2003).
10 Central Intelligence Bulletin, 6 October 1973, quoted by Emile A. Nakhleh, ‘‘The June 1967 and October 1973 Arab-Israeli Wars,’’ in Fifty Years of Informing Policy, p. 206.
11 Central Intelligence Bulletin, 5 October 1973, quoted in Fifty Years of Informing Policy, p. 206.
12 Emile A. Nakhleh, Fifty Years of Informing Policy, pp. 205–207.
13 Richard Shryock, ‘‘The Intelligence Community Post-Mortem Program, 1973– 1975,’’ Studies in Intelligence, Vol. 21, No. 3, Fall 1997, pp. 15–22.
14 Ibid., p. 27.
15 John Hollister Hedley, ‘‘The DI: A History of Service,’’ in Fifty Years of Informing Policy, pp. 14–15.
16 Abram N. Shulsky and Gary J. Schmitt, Silent Warfare, 3rd ed. (Washington, DC, Brassey’s, 2002), pp. 59–60.
17 Douglas J. MacEachin, Predicting the Soviet Invasion of Afghanistan: The Intelligence Community’s Record, monograph published by the Center for the Study of Intelligence, 2002, p. 46.
18 Ibid.
19 Willis C. Armstrong, William Leonhart, William J. McCaffrey, and Herbert C. Rothenberg, ‘‘The Hazards of Single-Outcome Forecasting,’’ in H. Bradford Westerfield, ed., Inside CIA’s Private World: Declassified Articles from the Agency’s Internal Journal, 1955–1992 (New Haven: Yale University Press, 1995), p. 238. This article, originally classified ‘‘Secret,’’ first appeared in Studies in Intelligence, Vol. 28, No. 3, Fall 1984.
20 Ibid.
21 Ibid., pp. 238–239.
22 Ibid., p. 239.
23 Ibid., p. 241.
24 Ibid.
25 Ibid., pp 241–242.
26 Ibid. pp. 242–243.
27 Douglas MacEachin rebuts this notion in CIA Assessments of the Soviet Union: The Record Versus the Charges (Washington, DC: Central Intelligence Agency, 1996), as does Kirsten Lundberg in CIA and the Fall of the Soviet Empire: The
LEARNING FROM INTELLIGENCE FAILURES 449
AND COUNTERINTELLIGENCE VOLUME 18, NUMBER 3
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
Politics of Getting It Right, Case Study C16-94-1251.0 for the Intelligence and Policy Project, John F. Kennedy School of Government (Cambridge, MA: Harvard University, 1994). For examples of CIA’s analysis of the Soviet Union in its last three years, and for an essay on that period, see Benjamin B. Fischer, ed., At Cold War’s End: U.S. Intelligence on the Soviet Union and Eastern Europe, 1989–1991 (Washington, DC: Central Intelligence Agency, 1999). For essays surveying in greater depth how the CIA analyzed Soviet political, military, and economic affairs throughout the Cold War, see Gerald K. Haines and Robert E. Leggett, eds., Watching the Bear: Essays on CIA’s Analysis of the Soviet Union (Washington, DC: Central Intelligence Agency, 2003).
28 Douglas MacEachin, Predicting the Soviet Invasion of Afghanistan, p. 46, and his forward to Richards J. Heuer, Jr., Psychology of Intelligence Analysis (Washington, DC: Central Intelligence Agency, 1999).
29 See Stephen Marrin, ‘‘CIA’s Kent School: Improving Training for New Analysts,’’ International Journal of Intelligence and CounterIntelligence, Vol. 16, No. 4, Winter 2003–2004, pp. 609–637.
30 John Hollister Hedley, Fifty Years of Informing Policy, pp. 15–16.
31 ‘‘Myths About Intelligence,’’ The Washington Post, 28 November 2003, p. A41.
32 Studies in Intelligence, Vol. 35, No. 4, op. cit., p. 65.
450 JOHN HOLLISTER HEDLEY
INTERNATIONAL JOURNAL OF INTELLIGENCE
D ow
nl oa
de d
by [
A m
er ic
an P
ub lic
U ni
ve rs
ity S
ys te
m ]
at 2
1: 53
0 2
D ec
em be
r 20
12
