new product forecasting
Nikhil_1806
assgn8pdfhandout3.pdf
THE INTERNATIONAL JOURNAL OF APPLIED FORECASTINGFall 2018 Issue 51
THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING
5 8 17 24 36 42
A Best-Practice Guide for Sales Forecasters A Blueprint for a Forecasting Support System: Part 2
Warning Signs for Forecasting Consumer-Induced Shortages The State of New-Product Forecasting
The Emerging and Long-Term Future of AI Deep Learning for Forecasting: Current Trends and Challenges
FORESIGHT Fall 201824
INTRODUCTION
New-product forecasting has long been a problematic and challenging area facing forecasters. More than 30 years ago, Gert Assmus (1984) portrayed new-product forecasting as “one of the most difficult and critical management tasks” due to the lack of sales history and absence of experience on consumer acceptance, trade support, and competi- tive reactions to the new product. Recent studies continue to characterize new- product forecasting as fraught with chal- lenges, with Mike Gilliland (2013) calling it “perhaps the most difficult and thank- less of all forecasting endeavors.”
Yet, with companies endlessly launch- ing new products, guidance on how to approach this challenge remains a priori- ty. One premise is that it should be distin- guished from those practices associated with forecasting existing products (Kahn, 2011). In addition, it requires a mix of art and science (Kelleher, 2013).
This paper extends our understanding of new-product forecasting practices by drawing upon the findings of our recently completed survey of companies conduct- ed in collaboration between our respec- tive institutions: the SAS Institute Inc. and Virginia Commonwealth University’s School of Business (the SAS-VCU Survey). We also present what we believe to be an emerging best practice. Our article concludes with suggestions for continued research as well as managerial
prescriptions regarding how a company should go about establishing a meaning- ful new-product forecasting process.
THE STUDY OF NEW-PRODUCT FORECASTING PRACTICE
Many studies of new-product forecast- ing focus on application of forecasting techniques such as diffusion models and on the accuracy and possible bias in new- product forecasts. A handful of studies have addressed organizational practices on new-product forecasting. One of us (Kahn, 2002) reported results of a sur- vey of 168 North American practitioners from a cross section of industries, and Jain (2017) recently reported findings of a survey based on 791 participants from multiple industries, predominantly in North America.
The 2018 SAS-VCU Survey Our 2018 survey was administered to SAS customers and company represen- tatives in attendance at a forecasting conference at VCU. Out of a qualified sample of approximately 1,000 individu- als, the survey response rate was 101 or 10%, comparable to published industry survey response rates. A broad cross sec- tion of industries was represented, with 30% from consumer packaged-goods companies, 11% from financial services, 10% from pharmaceuticals, 7% from telecommunications, and the remaining industries each 5% or less. Similar to the aforementioned studies, the majority of respondents were from North America
The State of New-Product Forecasting KENNETH B. KAHN AND CHARLES W. CHASE
PREVIEW As the authors observe, new-product forecasting has long been a problematic and challenging area. In this article, they assess the state of the practice from their recent survey completed by 100 companies and describe emerging approaches that are showing promise for significantly upgrading forecasting performance for these products. They offer several prescriptions for how a company should go about establishing an effective new-product forecasting process.
https://foresight.forecasters.org FORESIGHT 25
(79%), with 14% from Europe. Over 60% of the survey respondents came from companies with 5,000 or more employ- ees. The majority of respondents were directors/managers (53%), followed by analysts (24%) and senior management (13%).
The Survey Questions - What techniques should be used to
forecast new products? - What level of achieved forecast accu-
racy should be expected? - Is there a bias in new-product forecasts? - Who should be involved in the new-
product forecasting effort? - What software/technology should be
employed? - What comprises a good new-product
forecasting effort?
To help see where we stand today, we review here the prior literature and the current study findings about these six issues.
WHAT TECHNIQUES ARE USED TO FORECAST NEW PRODUCTS?
Much of what has been published on new- product forecasting is “technique-cen- tric.” Some papers have advocated the use of diffusion modeling, which depicts the increases in the number of adopters of a new product over time and the continued development of the product’s life cycle. Unfortunately, diffusion models have been shown to have a poor track record in predicting actual sales when compared to other methods and probably require more statistical sophistication than many practitioners possess. They don’t seem to have been widely used in industrial/ business settings (Golder & Tellis, 1998). Additionally, they have been misapplied at the individual product level when they were generally intended to predict the growth of a product category (Mahajan & colleagues, 1990).
In an analysis of 76 industrial firms, Lynn and colleagues (1999) found that judg- mental techniques and market research were favored over statistical techniques.
They concluded that successful high-tech industrial projects relied more on inter- nal expert judgment and brainstorm- ing, compared to unsuccessful high-tech industrial projects. Successful low-tech industrial projects tended to rely more on the traditional market research methods of one-on-one interviews with sales- people, surveys of buyers’ intentions, and formal surveys of customers.
Studying technique preferences across different types of new products, Kahn
■ We have recently completed a survey of companies conducted in a collaboration between our respective institutions: the SAS Institute Inc. and Virginia Commonwealth University’s School of Business (the SAS-VCU Survey). Responses were received by a good cross section of 100 companies. The questions posed:
What techniques should be used to forecast new products?
What level of achieved forecast accuracy should be expected?
Is there a bias in new-product forecasts?
Who should be involved in the new-product forecasting effort?
What software/technology should be employed?
What comprises a good new-product forecasting effort?
■ We found that new-product forecasting remains relatively unchanged in the past 20 years, with heavy reliance on judgment and customer/market research and use of spreadsheets for technology. Forecast accuracy has shown no trend toward improvement, and forecasts suffer from optimism bias.
■ New analytical procedures such as structured analogies and new technologies, including data mining and artificial intelligence, are showing promise for improving the new-product forecasting process.
Key Points
FORESIGHT Fall 201826
(2002) also found a clear preference for judgment and market research tech- niques, with jury of executive opinion, sales-force composite method, and like- product analysis (aka forecasting by anal- ogy) representing the three most com- mon qualitative techniques. Customer/ market research, which may be quantita- tive or qualitative in nature, was a fourth preference. The statistical techniques in use were relatively unsophisticated, including like-product analysis, moving averages, sales-force composite forecasts, and trend line analysis.
The 2017 Jain survey found that the most used models were
• Survey (23%), which attempts to cap- ture the future intentions of consum- ers to buy a product,
• Analog/Like-Product Analysis (22%), which uses the pattern of a similar product(s) launched in the past,
• Market Penetration (20%), which bases the forecast on market size, propor- tioned by estimates of penetration, purchase size, and frequency,
• Prediction Markets (17%), which draw on the wisdom of crowds, both inside and outside the company, and
• Delphi (13%), a structured group tech- nique where the coordinator receives unbiased input from a wide range of experts, both within and outside the company. Often after several rounds of feedback, there is an effort to achieve a consensus on the forecast.
The SAS-VCU 2018 survey asked respon- dents to indicate technique usage across four general categories: Judgment, Customer/Market Research, Time-Series Forecasting, and Statistical Techniques other than Time-Series Forecasting. Results in Figure 1 concur with prior studies, showing a strong preference for judgmental forecasting, with 78% of respondents characterizing judgment as always or often used, 71% of respondents indicating that customer research was always or often used, and less than 50% reporting that time-series and other sta- tistical techniques were always or often used.
WHAT FORECAST ACCURACY SHOULD BE EXPECTED?
One of the first studies of new-product forecast accuracy was from Shelley and Wheeler (1991), who reported that the ratio of actual sales to forecasted sales was .79 one year after introduction; that is, on average, actuals were about 20% below the forecast.
Kahn (2002) examined forecast accuracy by breaking new products down into sev- eral categories and reporting for each the forecast-accuracy metric equal to 100% minus the Mean Absolute Percentage Error (Hawitt, 2010).
• Cost reductions correspond to a change in price, but not the visual characteris- tics of the product, in order to provide a competitive advantage (72% accuracy).
• Product improvements are enhancements that
improve the product’s form or function (65%). • Line exten- sions are addi- tional items in the same product cate- gory under the
Figure 1. 2018 Survey Results for Types of Techniques Used to Forecast New Products
https://foresight.forecasters.org FORESIGHT 27
same brand name that pro- vide some distinct features that the original product (or set of products) doesn’t have (63%).
• New markets are exist- ing products brought into new markets with minimal changes to the product (54%).
• New-to-the-company is a new category entry for the com- pany, but the product is not new to the consumer (47%).
• New-to-the-world products are technological innovations that cre- ate a completely new market (40%).
Other surveys, including the 2018 SAS-VCU survey, found similar results regarding forecast accuracy as shown in Table 1. As summarized, it appears that new-product forecast accuracy has not improved over the past 16 years.
When compared to reported forecast accuracy for existing products, that of new-product forecasts is significantly worse. For example, McCarthy and col- leagues (2006) reported forecast accuracy of 86% (14% MAPE) for existing items at the product-line level in a 3-month to 24-month time horizon. Expectations for new-product forecast accuracy should therefore be set lower in comparison to those for ongoing products.
Gartner and Thomas (1993) argue that new-product forecast accuracy is a func- tion of industry marketing experience, resources, data, market turbulence, and technique breadth. With accuracy having not shown dramatic improvement over time, we can infer that companies are being challenged by these factors. Also, forecast accuracy may not be improving because markets are growing increasingly complex and turbulent. Interestingly, companies in the 2018 survey were asked to list ongoing new-product forecasting challenges (via an open-ended question), putting low forecast accuracy at the top, followed by lack of pertinent data, lack of tools, lack of process, and lack of market- ing experience.
Table 1 indicates that forecast accuracy is better for cost reductions and product improvements, which are indicative of incremental, minor product innova- tion, versus new-to-the-company/new category entries and new-to-the-world, which are more innovative, if not radi- cal, products. So, type of innovation does influence new-product forecasting accu- racy—likely as a consequence of market and technology uncertainty connected with more innovative products.
Another explanation may be that replace- ment products, like cost reductions and product improvements, can be forecast with statistical methods by linking the replacement series to the original series, as is possible in many software pro- grams; higher accuracy would be achieved through the use of quantitative methods. This was reported by Bill Tonetti (2006), who demonstrated how data for predeces- sor products can be used to successfully forecast semi-new products, or products that are extensions and modifications of existing products.
IS THERE A BIAS IN NEW-PRODUCT FORECASTS?
Tyzoon Tyebjee (1987) found an upward bias when forecasting during the new- product planning process. Moreover, this optimism bias was greater among those most deeply involved in the planning. Shelley and Wheeler (1993) and Kahn (2009) also report biases of optimism in new-product forecasts. One compelling reason for the optimism bias is that the
Table 1. New-Product Forecast Accuracy Reported by Select Studies, 2002 to 2018
FORESIGHT Fall 201828
forecasts must be high enough to achieve management approval and secure invest- ment. To counter such bias, it is essen- tial that there be transparency into how the new-product forecast is generated, including its underlying assumptions.
The SAS-VCU survey included a question about forecast bias. Seventy percent of the companies surveyed measured forecast bias and reported their belief about the
direction—overforecasting (bias of opti- mism) or underforecasting. As shown in Figure 2, almost three-quarters of these firms felt that new-product forecasts in their company were usually too high, the remaining equally divided between too low and no bias.
WHO SHOULD BE INVOLVED IN THE NEW-PRODUCT
FORECASTING EFFORT?
New-product forecasting is tradition- ally a mission of marketing and market research departments. Kahn (2002) found that sales and forecasting depart- ments also play a key role. Table 2 shows the results of the 2018 survey, indicat- ing that the departments of forecasting, marketing, and sales continue to have the major roles. Because of the predominant involvement of these (and other) depart- ments, new-product forecasting should be considered a cross-functional process.
The scale used was a five-point scale from “Does Not Participate” to “Highly Involved.” Results are percent of respon- dents who rated the respective depart- ment as very involved [4] or highly involved [5].
Related to the question of department involvement is the organizational mecha- nism for how new-product forecasts are
generated, with 45% of respondents reporting that they employ a cross- functional team, 24% use a single department, 9% rely on management as the source, and 18% allow individual departments to generate their own new- product forecasts. The latter category is not recommended, as this organiza- tional mechanism runs the risk of mul- tiple forecasts for the same new product. The nature of new-product forecasting should favor use of a cross-functional team, though this process can be time consuming due to the significant effort needed to get to consensus, coupled with the need to get multiple depart- ments to commit personnel who will actively engage with the team. This may be why some companies prefer to use a single department.
Figure 2. New-Product Forecast Bias, 2018 SAS-VCU Study
Table 2. Departments Described as Highly Involved in New- Product Forecasting
https://foresight.forecasters.org FORESIGHT 29
WHAT SOFTWARE IS EMPLOYED IN NEW-PRODUCT FORECASTING?
What technologies can be used to fore- cast new products? As shown in Figure 3, there appears to be a strong preference for spreadsheets: the 2018 study found that 80% of respondents indicated they are always or often used in this endeavor. A lesser majority of respondents report use of a demand forecasting and planning solution (54%) and statistical forecasting software (50%).
Canitz (2016) also found that companies rely extensively on spreadsheets, which he feels impedes an organization’s ability to consistently generate accurate forecasts and support best-in-class supply-chain processes. Spreadsheets, he observes, lack the advanced capabilities to support multiple forecasting methods, manage- ment by exception, multiple levels of aggregation, global consensus planning, visual metrics, and “what-if ” scenario analysis, to name a few. A computer tech- nology challenge facing companies is that many legacy enterprise resource planning (ERP) systems were not designed with new-product forecasting capabilities.
WHAT CONSTITUTES A SATISFACTORY PRACTICE?
Kahn (2002) found that, compared to companies dissatisfied with their new- product forecasting efforts, satisfied com- panies had greater involvement by R&D, market research, and sales forecasting
and a greater tendency to employ statisti- cal models such as regression and simu- lation for product improvements, line extensions, and market extensions. The SAS-VCU survey also found that satisfac- tion was related to the use of technology: satisfied companies tended to rely more on market-research techniques, statisti- cal techniques, demand forecasting and planning solutions, new-product plan- ning/life-cycle management solutions, and supply-chain management solutions.
Our survey also found that satisfaction was related to which business units were involved in the process; for example, a greater percentage of satisfied companies used a single department, while almost a quarter of unsatisfied companies used multiple individual departments.
It is important for companies to accept that new-product forecasts will have sig- nificant errors and hence focus on devel- oping forecast scenarios under different assumptions and planning contingencies for expected errors (Kahn, 2016). New- product forecasting is more about deriv- ing a forecast that allows the organization to plan properly (a meaningful forecast) than it is about accuracy.
NEW APPROACHES TO NEW-PRODUCT FORECASTING?
Key realities of new-product forecast- ing are minimal data, limited analytic capabilities, a general uncertainty sur- rounding the product’s launch, and the
Figure 3. Technology Used in New-Product Forecasting
FORESIGHT Fall 201830
ever-changing marketplace. One proposal has been to add “structure” to the use of like-product analysis, in which the forecast is based on the launch behavior of similar products. Green and Armstrong (2007) observe that forecasters often use analo- gies when forecasting, but in an unstruc- tured manner. They propose a structured judgmental procedure whereby experts list analogies, rate their similarity to the target, and match outcomes with possible target outcomes. An administrator would then derive a forecast from the informa- tion. Structured judgment incorporates statistical analysis of historical data along with judgment. Companies are finding that structured-analogy forecasts tend to be more accurate than unaided analogies, particularly in situations where experts have direct experience with their closest analogy.
Structured analogies can be enhanced through data mining techniques, which can more effectively identify candidate products that have similar attributes and characteristics. These candidates are “fil- tered” to yield a set of surrogate series that has the most similar statistical properties to the unknown new-product series. Since each process step requires both statistical and judgmental analysis supported by technology, a formal approach to com- bining statistics and domain expertise (judgment) can be applied to significantly enhance the structured-judgment pro- cedure. There is anecdotal evidence that these enhancements have the potential to improve forecast accuracy and also reduce—from days to hours—the time needed to create a new-product forecast.
Several large consumer-packaged-goods companies are automating the structured- judgment procedure using machine-learn- ing algorithms (e.g., neural networks, gradient boosting, and ensemble random forest models) to forecast new-product launches. These additional enhancements to the process have resulted in boosting accuracy as well as further reducing time
to create new-product forecasts, with some reporting reductions from hours to minutes.
Another augmentation to structured judgment can come by way of integrating structured and unstructured data using sentiment analysis. Sentiment analysis incorporates text mining techniques that collect unstructured data from social media, the Internet, and other sources, and examine what people are saying about the new product in real time. For example, once a large enough sample is collected, analysis of unstructured data would determine if consumers like the new product, liked the supporting pro- motion, can find the product on-shelf, and favor the product’s quality. Sentiment analysis also can be employed to capture what consumers might be say- ing about new-product iterations—for example, perhaps customers would like a different flavor, a new color, a different package size, etc. All of this information allows companies to adjust the launch forecast sooner in the cycle to assure suc- cessful marketing planning, merchandis- ing, and product management.
CONCLUSIONS AND RECOMMENDATIONS
Our review of the literature and com- parison of practices over the past 16 years suggests that practices for new-product forecasting remain relatively unchanged, with heavy reliance on judgment, custom- er/market research, and use of spread- sheets for technology. Forecast accuracy has shown no trend toward improvement, and forecasts suffer from optimism bias.
We believe that companies should view the new-product forecasting process as inherently cross-functional and create a cross-functional team that will enable collaboration across departments.
We believe there is value to analytics and technology in supporting a success- ful new-product forecasting effort. Our
Key realities of new-product forecasting are minimal data, limited analytic capabilities, a general uncertainty surrounding the product’s launch, and the ever- changing marketplace.
https://foresight.forecasters.org FORESIGHT 31
Kenneth B. Kahn is Professor of Marketing and Senior Associate Dean at Virginia Commonwealth University. He is author of the 2014 book New Product Forecasting: An Applied Approach, as well as numerous articles on new-product forecasting and product innovation.
SAS-VCU survey highlights the benefits of statistical forecasting techniques in conjunction with judgment. This cor- responds to the recent literature review by Goodwin and colleagues (2014), who advocate the use of multiple complemen- tary methods. Application of structured judgment aided by data mining is very promising. And demand forecasting and planning solutions should serve compa- nies better than reliance on spreadsheets alone.
That said, new-product forecasting is an organizational process in which transpar- ency is essential to avoid forecasting bias and achieve effective collaboration across the organization.
REFERENCES Assmus, G. (1984). New Product Forecasting, Journal of Forecasting, 3(2), 121-138.
Canitz, H. (2016). Overcoming Barriers to Improving Forecast Capabilities, Foresight, 41 (Spring), 26-34.
Canitz, H. (2016). The Forecasting Conundrum, Logility, August 23, https://www.logil- i t y. c o m / b l o g / h a n k - c a n i t z / a u g u s t - 2 0 1 6 / the-forecasting-conundrum.
Gartner, W.B. & Thomas, R.J. (1993). Factors Affecting New Product Forecasting Accuracy in New Firms, Journal of Product Innovation Management, 10(1), 35-52.
Gilliland, M. (2013). Worst Practices in New Product Forecasting, Journal of Business Forecasting, (Winter 2012-2013), 31-34.
Golder, P.N. & Tellis, G.J. (1998). Beyond Diffusion: An Affordability Model of the Growth of New Consumer Durables, Journal of Forecasting, 17 (3-4), 259-280.
Goodwin, P., Meeran, S. & Dyussekeneva, K. (2014). The Challenges of Pre-Launch Forecasting of Adoption Time Series for New Durable Products, International Journal of Forecasting, 30(4), 1082-1097.
Green, K.C. & Armstrong, J.S. (2007). Structured Analogies for Forecasting, International Journal of Forecasting, 23(3), 365–376.
Hawitt, D. (2010), Should You Report Forecast Error or Forecast Accuracy?, Foresight, Issue 18 (Summer 2010), 46.
Jain, C.L. (2017). Benchmarking New Product Forecasting and Planning – Research Report 17, Great Neck, New York: Institute for Business Forecasting.
Jain, C.L. (2007). Benchmarking New Product Forecasting, Journal of Business Forecasting, (Winter 2006-2007), 28-29.
Kahn, K.B. (2016). Solving the Problems of New Product Forecasting, Business Horizons, 57(5), 607-615.
Kahn, K.B. (2011). Product Planning Essentials (2nd Edition), Thousand Oaks, CA: M.E. Sharpe, 267.
Kahn, K.B. (2009). Identifying the Biases in New Product Forecasting, Journal of Business Forecasting, (Spring 2009), 34-37.
Kahn, K.B. (2002). An Exploratory Investigation of New Product Forecasting Practices, Journal of Product Innovation Management, 19(2), 133-143.
Kelleher, M. (2013). The Art and Science of New Product Forecasting, Journal of Business Forecasting, (Winter 2012-2013), 17-19.
Lynn, G.S., Schnaars, S.P. & Skov, R.B. (1999). Survey of New Product Forecasting Practices in Industrial High Technology and Low Technology Businesses, Industrial Marketing Management, 28, 565-571.
Mahajan, V., Muller, E. & Bass, F.M. (1990). New Product Diffusion Models in Marketing: A Review and Directions for Research, Journal of Marketing, 54 (1), 1-26.
McCarthy, T.M., Davis, D.F., Golicic, S.L. & Mentzer, J.T. (2006). The Evolution of Sales Forecasting Management: A 20-Year Longitudinal Study of Forecasting Practice, Journal of Forecasting, 25, 303-324.
Shelley, C.J. & Wheeler, D.R. (1991). New Product Forecasting Horizons and Accuracy, Review of Business, 12(4), 13-18.
Tonetti, B. (2006). Tips for Forecasting Semi-New Products, Foresight, Issue 4 (June), 53-56.
Tyebjee, T.T. (1987). Behavioral Biases in New Product Forecasting, International Journal of Forecasting, 3, 393-404.
Charles W. Chase is Executive Industry Consultant at the SAS Institute in Raleigh, North Carolina, where he serves as the principal solutions architect for demand forecasting & planning solutions. Charlies is author of Next Generation Demand Management: People, Process, Analytics and
Technology (2016), of Demand-Driven Forecasting: A Structured Approach to Forecasting (2nd Ed 2013), and co-author with Lora Cecere of Bricks Matter: The Role of Supply Chains in Building Market-Driven Differentiation (2013).
