valuation Packet

Lowenthal (2012) suggests a couple of reasons for why this might be:

· First, intelligence consumers keep the intelligence community busy. Intelligence analysts and their managers rarely feel they have time to withdraw from their day-to-day work to assess the effectiveness of that work. We have already mentioned several times how analysts working in fast-paced, tactical environments are often hard-pressed managing their time to produce intelligence products, as it is, leaving little to no opportunity to learn from one’s mistakes or successes.

· Second, members of the intelligence community tend to treat their products in a take-it-or-leave-it manner. If a decision-maker chooses poorly based on a less than perfect intelligence product, then the fault falls to him or her for failing to make wise choices. Intelligence analysts take pride in their technical competence and have a tendency to resist being audited or having their work reviewed. As a consequence, self-reflection seems to occur as a reaction to a failure rather than as part of a standard operating procedure.

Intelligence Analysis: Process or Project?

We often seem to straddle the line between thinking about intelligence analysis as a process and as a project. Seeing it as a process allows us to take a more organic approach, emphasizing what Clark (2007) would call its ‘social nature’.  Intelligence as a process gives primacy place to its character as a fluid, continuous activity underlying ongoing operations. It also allows us to fall into the trap of thinking that the way in which we work has no end point, excusing our professed inability to stop and assess the quality of our work. Many aspects of intelligence analysis, however, might benefit better from approaching it as a field of practice in which we manage projects.

The Project Management Institute defines a project as “a temporary endeavor undertaken to create a unique product, service, or result” (2008, p. 5). Projects are understood in contrast to processes, which are ongoing, repetitive efforts following an organization’s standard procedures. Projects are distinct from processes not only because projects result in something new, but because they involve the uncertainties of creating something new.

In criminal intelligence analysis, we already often deal with cases in our everyday lives that have specific starting points and specific ending points concluding with a specific product – hopefully a conviction in our case. Despite their resemblances, it would be fair to say that no two cases are exactly alike, which means every case results in ‘a unique product, service, or result’. When we think of cases as the result of finished projects, then we might think of criminal intelligence analysis – in so far as it contributes to cases – as projects themselves.

The Project Management Institute states that recording experiences should be a priority for any organization that manages the uncertainties inherent in creating a new product. Lessons learned and historical information contribute to an organization’s knowledge base. A lessons learned record should include all the documents related to an intelligence analysis operation. Records should outline each major step in the operation and state the thought processes behind any major decisions. Since all intelligence analysis involves risk and uncertainty, the record should state areas were key assumptions were made and what steps were taken to manage risk.

· A lessons learned body of knowledge is more than just a dossier of documents describing what happened. It also includes assessments of the product’s overall effectiveness, areas of unforeseen successes, and recommendations for how challenges might be overcome if faced in the future.

On the first point, it should include feedback from end users on how well final products met their needs. On the second and third points, all analysts experience variances in the quality of their work. Performing quality control requires identifying the reasons behind these variances, articulating corrective actions taken, and measuring their performances. Ideally, this activity will take place throughout the intelligence cycle, but at a minimum they should be performed at the end of a substantial project or case.

Lessons Learned Initiatives

The intelligence community as a whole has seen a shift in its composition that has affected the level of technical expertise in its ranks (Lowenthal, 2012). At the national level, the Cold War spurred large investments in a robust intelligence collection and analysis capacity. With its conclusion in the 1990s, many older, experienced intelligence professionals retired from the community and were not replaced thanks to budget cuts correlating to a reduction in defense spending in general.

After the attacks of September 11, 2001, the intelligence community received a rapid injection of funding that in some respects surpassed its ability to absorb. Young analysts were hired, sped through abbreviated training programs, and set to work on intelligence problems for which they were often ill prepared. They had access to only a limited numbers of mentors, most of who had spent their careers addressing traditional threats through traditional means that could not manage the level of complexity new threats involved.

· Meanwhile, intelligence reforms focused on strengthening intelligence collection capabilities and cross-agency information sharing, leaping over the need to advance the field of intelligence analysis. The DNI eventually identified this omission and has since sought to build a lessons learned knowledge base that can be re-invested into the intelligence community, but levels of investment in terms of time, funds, and personnel seem to remain a low priority.

A few examples in the intelligence community do exist for proactively assessing the quality of one’s work and building a body of knowledge based on an analysis of past experience.

For instance, the U.S. Army operates a Center for Army Lessons Learned (CALL). CALL issues a wide array of succinct handbooks for military commanders on dealing with challenges that influence their mission success in the contemporary operational environment. CALL relies on a rotating group of professionals with operational experience in various areas deemed critical for mission success. These professionals have worked on the ground in environments where key challenges have been identified.

In between these field experiences, they are called upon to meet with others who work under similar conditions, to interview them on what went well (and what went poorly), and to process this information into suggestions for the future. A review of the CALL website reveals papers on supporting Agribusiness in Afghanistan, paying informants in a deployed environment, responding to disasters, and even setting up a lessons learned program at the unit level. The intention is to keep commanders informed about new trends that affect their operations and to increase the overall quality of their decision-making. The other branches of the military have their own, similar versions.

In 2004, the Department of Justice established the Criminal Intelligence Coordination Council (CICC)to implement 28 recommendations stated in the National Criminal Intelligence Sharing Plan. The CICC’s stated mission is to support the development of criminal intelligence standards for information sharing, technologies, and training. The CICC operates the National Criminal Intelligence Resource Center (NCIRC), but it is only available on secure networks used within criminal intelligence organizations. The emphasis of the CICC and the NCIRC does not appear heavily focused on the development of criminal intelligence analytical tradecraft, echoing the persistent need Heuer and Pherson describe for supporting increased professional competence of intelligence analysts at all echelons.

The most dynamic area of analysis about intelligence analysis may be the academic research sector. Independent universities, government-funded think tanks, and non-governmental organizations have made active contributions to the public’s understanding of intelligence analysis and have increased the intelligence community’s awareness of best practices by leaps and bounds.

· An example of such work, made available for your review as this week’s supplementary material, is a short book published by the Joint Military Intelligence College to educate competitive intelligence professionals about intelligence analysis lessons learned from the national and military intelligence sectors (Krizan, 1999).

Though it predates post-9/11 intelligence reforms and so it does not address common intelligence practices exactly as we have addressed them in this course, it does provide an example of information sharing across intelligence disciplines at the macro-level.

During Week 2 we looked Hansen’s article on the future of US intelligence (2004), which was published in the International Journal of Intelligence and Counterintelligence – a very useful resource of tracking trends in research about the field of intelligence and intelligence analysis.

The journal Intelligence and National Security covers historical reviews of intelligence issues, while theJournal of Homeland Security and Emergency Management focuses on reporting innovations in field operations and cutting edge academic research primarily focused on the state and national levels. As students of intelligence analysis, we should develop an awareness of other journals and the work of specific researchers on areas of our interest that appear in journals not specific to intelligence.

Victory!

Welcome to the final chapter in the story of our 8-week exploration of criminal intelligence analysis. We have talked about tools and techniques used to process and exploit intelligence information in order to produce intelligence products. And we have seen how many of these activities fit within the broader scope of the intelligence analysis enterprise. Finally, we have applied these discussions toward adapted real-world scenario.

Now it is time to pause a moment, to organize our materials, and to make a record of our successes and failures, our challenges and our learning, so that those who come after us will have the advantage of our insights and experience.

Scenario

Last week, Chief Heaton authorized a raid against El Movi based on your recommendations. The action led to the successful apprehension of its targets without anyone getting hurt. While some of the members of the organization not present where direct action occurred did escape, Chief Heaton is confident that over time they will surface again. For now, prosecutions are underway. But perhaps more importantly, El Movi is no longer a functioning organization and its victims are safe. 

Chief Heaton has been impressed with the quality of your work overall on this case. He started out somewhat skeptical about your use of statistics, complicated computer programs, and structured analytic techniques, but now he sees the value of them. He wants you to close out your work by producing your own assessment of your efforts so the TPD can retain a record of your experience.

He wants you and your office to keep such records from now on so that he can use them to justify more investment in the criminal intelligence analysis section and you can use them to train new criminal intelligence analysts coming on board to the organization for the first time.

While this final report will describe each of the major steps taken in the El Movi case, the emphasis is on assessing your experience working through the case. Begin the report with a timeline of all the major events in the case, ensuring you indicate where in the timeline your products fall. For each of your products, briefly respond to the following questions:

· What was the most helpful about the technique I used to create this product, and why?

· What was the biggest hindrance of the technique I used to create this product, and why?

· What other technique or aspect of another technique would have helped me overcome this hindrance to create a better product, and why?

Close out your report by finishing the following sentences about your work on the case as a whole:

· The moment when I was the most engaged, excited, and involved in the case was when…

· The moment when I was the most distanced, disengaged, and uninvolved in the case was when…

· I would like to say about my experience as a criminal intelligence analysis on this case that…

· The one piece of advice I would give the next person who works on a similar case using similar techniques is…