When Numbers Don't Count: Building Qualitative Research Into Business Planning And Short-Term Forecasting

Monday, 04 February 2013 10:05


  • How should we assess the relevance and accuracy of any historical data, quantitative and qualitative, that may be used as a basis for making assumptions about future performance and future events?
  • To what extent should contextual (or external) factors, and assumptions about their potential impact, be used when developing forecasts for strategic (internal) response purposes?
  • What are the structural factors that have allowed certain misconceptions about forecasting to impact on how it is used, both quantitatively and qualitatively?
  • Do behavioural factors impact the forecasting process and the expectations of that process?
  • How can we practically combine quantitative and quantitative elements into a forecasting system with a short time horizon?

Forecasters often claim that short-term forecasting], with its limited time horizon, allows the identification of enough key, essentially quantitative, indicators to produce acceptable results. This is in comparison to forecasting long-term economic trends confronted with too many uncertain variables.


This report argues that there been an over zealous use of quantitative methods by forecasters and planners for short-term projections.We need to evaluate how more adaptive and dynamic planning systems, using a combination of both quantitative and qualitative information can challenge orthodoxy.


The traditional annual planning and budgeting cycle is insufficiently flexible to meet the demands of today’s rapidly changing and uncertain environment, and therefore a greater balance of quantitative and qualitative methods need to be introduced into short-term forecasting.


A framework is proposed bringing together qualitative and quantitative factors into play through an Adaptive Planning System (APS).


History examines the impact of past events. 


Proximity to an event having occurred in relation to a contemporary standpoint is no guarantee that an objective interpretation can be made. This concept of recency is misused and often exploited by stakeholders. It can be argued the more recent the event, the less likely researchers will have access to all the “facts”. The effects of many events do not become apparent until much later (months, years, decades and longer), impacted by lengthy gestation periods before unforeseen and unintended consequences manifest themselves. The evidence base can also be “contaminated” by “the Loss of History” – data that is either deemed to be not worthy of recording, is physically misplaced, re-written, deleted or classified– and by seeding of “fake” data, not to mention “sensitive dependence on initial conditions” – also known as chaos.


Challenged by accuracy of interpretation of even recent events – how can we expect to extrapolate with any certainty or accuracy into the future, short or long term. Indeed, any futures exercise begins with all the participants relating their memories of the recent pass to generate alternative futures, as people have very different versions of history. Short term “endemic myopia” occurs as many stakeholders are still able to influence, not only the interpretation of recent past events but subjectively influence how their motives and actions are interpreted in relation to the “foreseeable” future. If historical distortion can occur in the short-term then planning and the forecasts upon which it is based can be as error prone as long-term forecasts and where small differences to initial steering conditions can make large differences to final outcomes.


In addition to the impact of “short-term” historical distortion we also address how this exacerbates the fault lines in traditional short term planning and forecasting processes. Issues include: the cumbersome nature of the annual planning cycle, processes not being sufficiently adaptive to the demands of rapidly changing environments, forcing the fit of fragmented and insufficient data into probabilistic distributions, how fixed periodic planning cycles can inhibit adapting to short-term threats, the lack of flexibility in traditional decision making mechanisms when refinement is required, insufficient scenario planning integration, the tendency for  “knee-jerk” reactions without reference to the underlying causes of greater than expected variances, in turn exposing the organisation to the effects of unintended consequences.


An alternative combination of methods and processes is proposed, so that  plans can be developed, modified and iterated faster, help speed up analytical feedback, allow for greater emphasis on both qualitative and quantitative frameworks, examine recent time series data from both contextual and strategic standpoints – leading to more dynamic and plausible outcomes.


To some readers certain positions taken in this paper may appear polemical.


Such an approach is deliberate in that it is the author’s aim to stimulate debate in an area which has lacked a “philosophical” dimension in deference to the hegemony of the use of quantitative approaches for short-term forecasting.

Order your copy now

Electronic copy £495.00 plus VAT and admin.

When Numbers Dont Count £594.00 Add to Cart

About Bruce Garvey

Bruce Garvey is a founding partner of Strategy Foresight Partnership LLP, which deals with problem structuring for highly complex issues occussing under genuine uncertainty, using innovative methodologies for a wide range of public and private sector clients. This followed an earlier career with a major multi-national in a number of staff and operational posts in the UK, Europe and Middle East. He recently stepped down from a 6-year tenure as Chairman of Cass Entrepreneurs Network (CEN).



Executive Summary




1.1: Planning, Budgeting and Forecasting 1.2: Contextual versus Strategic 1.3: Quantitative and Qualitative States




2.1: The role of historical data in forecasting 2.2: Forecasting misconceptions – quantitative, qualitative and otherwise

       2.2.1: Quantitative

       2.2.2: A case in point – a study of the financial sector

       2.2.3: Qualitative

       2.2.4: Or otherwise – Statistics, damned statistics ……and causality

       2.2.5: Or otherwise – Handling the experts


2.3: Behavioural Factors – Individual and Organisational 2.4: Complexity and unbridled connectivity




3.1: Defining an Adaptive Planning System




4.1: A basic framework for incorporating quantitative based information 4.2: The integration of qualitative processes into the forecasting system 4.3: Applying a decision method 4.4: Managerial responsibility for planning and forecasting 4.5: A proposed process framework for incorporating quantitative and qualitative elements into a dynamic short-term rolling forecast system

      4.5.1: Base Framework

      4.5.2: The Adaptive Process in action