423 Week 5: Formal source selection

21 Defense AT&L: November–December 2015

Getting the Best Value in a Source Selection? Brian Schultz n David Dotson

“Price is what you pay. Value is what you get.” —Warren Buffett

Schultz is a professor of Program Management, at the Defense Acquisition University (DAU) Capital Northeast Region, Fort Belvoir, Virginia. Dotson is a professor of Contract Management at the DAU Mid-Atlantic Region in California, Maryland.

C ompetition in acquisition is an important topic and has been since the Department of Defense (DoD) started acquiring systems from the defense industry. The key premise is that DoD will get greater value for the price paid as a result of competition. Some studies suggest savings in the 15 percent to 25 percent range and even greater under some conditions as a result of competition. However, greater value is not always tied to

lower prices or cost savings. Greater value can be realized through a superior technical solution as part of a trade-off of price and other factors in a source selection.

This article addresses our thoughts in the best value discussion, including the use of the Lowest Price Technically Acceptable (LPTA) method. According to the Government Accountability Office’s Report 14-884 (Factors DOD Considers When Choosing Best Value Processes Are Consistent with Guidance for Selected Acquisitions) dated July 2014, use of LPTA on contracts valued at more than $25 million has increased 10 percent while full trade-offs decreased by 9 percent over the last few years. This increased use of LPTA has generated a lot of discussion and interest from industry. The following is just a small sample of some recent news articles and blog titles that discuss use of LPTAs versus the trade-off for best value source selections:

Defense AT&L: November–December 2015 22

• “Five Factors Plaguing Pentagon Procurement,” American Enterprise Institute, Dec. 13, 2013.

• “Executive Slams ‘Lowest Price, Technically Acceptable’ Acquisition Regimes” National Defense Magazine, Jan. 16, 2014.

• “Eliminate LPTA and staff augmentation contracting other than for commodity services with minimal mission impact,” Defense Business Board Report, July 24, 2014.

• “LPTA Contracts Stifle Government Innovation,” Informa- tionWeek, Oct. 25, 2013.

• “Is the Government Starting to hate LPTA too?” Washing- ton Technology, June 7, 2013.

• “Too Much Competition can Reduce Incentives for Inno- vation,” Lexington Institute, April 22, 2013.

• “Best Value or LPTA? One Size Does Not Fit All in Acqui- sition,” IntegrityMatters.com, Feb. 27, 2013.

• “Pentagon launching new review of LPTA contracts,” Federalnewsradio.com, Dec. 8, 2014.

The March-April 2015 issue of Defense AT&L magazine in- cluded articles on the use of LPTA, one (“Getting ‘Best Value’ for the Warfighter and the Taxpayer”) by Frank Kendall, Under Secretary of Defense for Acquisition, Technology, and Logistics and another article providing an industry perspective, (“Low- est Price Technically Acceptable Overrated, Overused?”) by Will Goodman. Both articles highlighted the concern that LPTA should be used only in limited cases. But they differed on the approach that should be used in assessing the suitability of an LPTA source selection (value gained from performance above an established minimum versus assessing the long-term im- pacts on capability areas and changing the Federal Acquisition Regulation [FAR]).

Goodman suggested that looking at the acquisition as a single transaction is deficient. He suggested that contract- ing officers should be directed to assess capability area outcomes over the course of multiple solicitations in de- termining whether LPTA should be used. “To address the deficiency, the FAR should point out that LPTA is ideal for commodities and commoditized services and note that con- tracting officers should consider the possible and probable long-term impacts of an LPTA source selection on the ca- pability area addressed by the solicitation.”

While it is an innovative idea, we don’t believe the FAR sug- gestion above is practical. Program level contracting offi- cers are not typically in a position to do this and should not be expected to make this kind of portfolio-wide assessment. Furthermore, it would be difficult, even for senior leaders who oversee large portfolios of programs, to tie one or even multiple actions to a future mission area or mission capa- bility outcome. While forecasting and planning for future capability needs is an important and ongoing effort, it could be very difficult to determine long-term impacts based on the use of a source-selection method on any given contract. It would also raise questions about whether DoD is unnec- essarily paying more for a product or service because of

future outcomes that are difficult to forecast and perhaps even more difficult to measure.

Complexity and Risk Framework We suggest an alternative approach to choosing a source- selection method based on a complexity and risk framework intended for use by the government acquisition team. It would involve an integrated, multifunctional effort in close coordina- tion with the contracting officer. This assessment would go beyond just looking at performance requirements, to assessing the following programmatic areas:

• Organizational landscape • Mission and operational interfaces • Industrial capabilities • Deliverables or outcomes • Risk and opportunity management

We would argue that this level and depth of assessment will provide a more stable foundation for making informed deci- sions about source-selection strategies through the life of the acquisition program.

The proposed framework is based on our past experiences in pre-Request for Proposal planning efforts and is consistent with the current statutory, regulatory and policy guidance. Note that while risk and risk management are well defined in DoD, this is not the case for complexity. The FAR and DoD guidance are clear that both complexity and risk of the re- quirements are relevant in assessing the relative importance of price and nonprice factors (e.g., technical) and value above a minimum performance level. This framework can help DoD teams assess both complexity and risk.

The following is a brief summary of each assessment area:

Organizational Landscape: The overall health and stability of an acquisition organization can have significant implica- tions for its ability to plan, develop and execute acquisition efforts. Organizational issues can directly affect the abil- ity to oversee and manage certain efforts (e.g., complex trade-off decisions), requiring greater contractor technical competence and increasing overall risk to the program. For example, less stable organizations with management chal- lenges may need additional time and assistance to develop a performance work statement in response to a services acquisition requirement.

Such an unstable acquisition organization may be better served with more senior-level services support that can react quickly to changes in the environment and deliver advisory services with less oversight. Use of an LPTA in this example may not result in a good outcome as organizational risk and complexity are high.

Sample questions that should be considered for organizational landscape include: Is the program office staffed adequately?

23 Defense AT&L: November–December 2015

Is there high turnover in key personnel? Is there strong budget and advocacy support from stakeholders and/or sponsors? Is there strong alignment with the agency strategic plan? What is the availability of key organic resources to plan and manage the contractual efforts, both pre- and post-award? Has the organization accomplished its goals effectively?

Mission and Operational Interfaces: The operational environ- ment of the system or organization can result in added risks and complexities. For example, some systems must integrate with complex communications and information networks that involve specialized technical expertise in order to be effective. These operational environments can rapidly change as new threats emerge and new technologies are introduced.

For example, we both worked on airborne command and control system programs that had to integrate into multiple networks and required interoperability with joint and coalition partners, some with very different system configurations and exchange requirements. This added great complexity to the program in order to support current operations and planned future capability upgrades.

Program teams should address how this system or program or service fits into the bigger picture. What other system in- terfaces, networks and information exchange requirements are needed? How well understood and stable are the require- ments? What is the operational or organizational urgency? What kinds of emerging threats or new contingency opera- tions are relevant?

Industrial Capabilities: The ability and track record of industry (including suppliers) in the relevant domains should be as- sessed as early as possible. Previous execution issues, poor incumbent performance, poor financial performance, and lack

of qualified suppliers are examples of indicators for possible high complexity and risk. Market research plays an important role in gathering information needed for this assessment.

For example, consider a complex air traffic control radar sys- tem that has matured to the point where it is considered a production commodity. This system capability has multiple companies whose product meets the mandated requirements. While the system design is fairly complex, the industrial capa- bility could be rated as having low complexity and risk due to the nature of industry’s ability to deliver this product and the track record of previous contracts.

Sample questions to be considered include: What capabili- ties does industry offer to meet the stated needs? What is the past performance of previous contracts for similar ef- forts? What is the industry confidence level in estimating its costs to perform the work? What is the financial health and commitment to cover any overruns of interested companies? How much unique domain knowledge of the agency chal- lenges is necessary to execute the contract? How will this knowledge be obtained? Is it the contractor’s responsibility to have in-house expertise or will the government provide the support needed to ensure adequate knowledge? Note that, in some cases, the government team may need to dedicate staff to guide contractor efforts to ensure the product or service is tailored appropriately to meet the agency need. A good example of this is configuring software to meet a unique or- ganizational requirement.

Deliverables and Outcomes: The product or service outcome, clarity and scope of requirements, and the amount of develop- ment involved should be assessed as well as the determination of the value of (and what we are willing to pay for) increased levels of technical performance.

Several questions should come to mind in assessing the deliv- erables and expected outcomes of the acquisition: What is the expected outcome and how is success defined? What prod- ucts and deliverables are required and what are the accep- tance criteria or acceptable quality standards for them? How much new development work is required? Is the product a commercial item (or commercial service offering)? If a legacy effort, what issues and challenges were encountered in previ- ous contracts and how difficult were they to overcome? What is the value of performance above the minimum threshold? This is a question recently posed by Kendall. If a services ac- quisition, what skill levels and expertise are anticipated? What domain knowledge of the product or mission area is required?

Risk and Opportunity Management: After assessing the above, teams should identify risks to successful contract performance. Starting the analysis with the previous areas is recommended because doing so can assist in identifying risk areas and their probability and consequences. It is im- portant to continuously manage risks—meaning actions will be taken to mitigate risks, and the organization will

“The bitterness of poor quality remains long after

the sweetness of low price is forgotten.”

—Benjamin Franklin

Defense AT&L: November–December 2015 24

As shown in Figure 1, high com- plexity and risk suggest that a trade-off source selection should be considered. Moderate com- plexity and risk suggest that ei- ther a trade-off or a combination approach could be considered. Finally, low complexity and risk align with the LPTA approach. This proposed, notional scale also maps to the best value continuum associated with the relative im- portance of price and nonprice factors and incorporates the re- sults of both complexity and risk into the assessment.

Closing Thoughts As Franklin noted centuries ago, low price is not always the best deal, especially if either the quality or outcome of our acquisition is at stake. Source selection and use of an appropri- ate method to achieve best value is an important decision that should involve a deliberate process to ensure we have thought through what we need and how best to get it. A framework for assessing program complexity and risk may prove useful for some in making an informed decision about the source- selection method. The real value of using an assessment meth- odology similar to this is not an absolute answer but rather the critical thinking that supports good acquisition outcomes, both for DoD and industry—and ultimately for the taxpayer. T h e a u th o r s ca n b e co n ta c te d a t b r i a n . s c h u l t z @ d a u . m i l a n d [email protected].

implement a process to govern the risk management ef- forts throughout the contract. This often occurs as part of a risk management review board that meets periodically to review risk management efforts. Opportunity management should be assessed for initiatives that can reduce costs and add greater value to the effort within the scope of the exist- ing contract.

The culmination of this effort results in an integrated assess- ment of low, moderate or high complexity and risk based on the answers to relevant questions in each area. A roll-up of the areas to one color rating is a judgment call based on the rela- tive importance of the different areas that are relevant for the program. The notional rating scale shows green as low, yellow as moderate, and red as high complexity and risk.

Greater Importance of Price Lesser

Lesser Importance of Nonprice or Cost Factors (e.g., Technical) Greater

LPTA CA TO* TO TO TO CA*

Figure 1. Integrated Assessment Rating Scale

*Multiple approaches could be appropriate with moderate levels of risk and complexity LPTA: Lowest Price Technically Acceptable CA: Combination Approach TO: Trade-off Assessment scale not absolute—tailor to program circumstances

MDAP/MAIS Program Manager Changes With the assistance of the Office of the Secretary of De- fense, Defense AT&L magazine publishes the names of in- coming and outgoing program managers for major defense acquisition programs (MDAPs) and major automated infor- mation system (MAIS) programs. This announcement lists all such recent changes of leadership, for both civilian and military program managers.

Army COL James C. Mills relieved COL Gary D. Stephens as project manager for the Precision Fires Rocket and Missile Systems (PFRMS) on July 15.

COL William D. Jackson relieved COL Thomas H. Todd as project manager for Utility Helicopters (UH) on July 15.

COL Shane N. Fullmer relieved COL John Cavedo, Jr. as project manager for Joint Program Office, Joint Light Tactical Vehicles (JPO JLTV) on July 31.

COL Gregory H. Coile relieved COL Edward J. Swanson as project manager for the Warfighter Information Network- Tactical (WIN-T) on July 15.

COL Donald W Hurst, III relieved COL Sandra L. Vann- Olejasz as project manager for DoD Biometrics on July 16.

Air Force Col. Michael Harm relieved Col. Scott Owens as program manager for the Theater Battle Control Systems program on Aug. 3.

Col. Darien Hammett relieved Col. Carlin Heimann as pro- gram manager for the RQ-4 Global Hawk System program on July 5.

Col. Steven Whitney relieved Brig. Gen. William Cooley as program manager for the Global Positioning System Pro- gram on July 8.