The Evolution of Forecasts _for a sports drink_ by Rabia06


									The Evolution of Forecasts
   (for a sports drink)

           David Heath
Department of Mathematical Sciences
    Carnegie Mellon University

Note: some details have been altered!
My charge (as I interpreted it):
 Discuss the use and usefulness of
  mathematics to solve a real problem
 Present a specific, real example in which
  relatively simple mathematics helped make
  something better
 Convey a bit of the “flavor” of doing
  mathematics motivated by applications
The story:
 Client contacted Peter Jackson and me
 Client made and distributed a “sports drink”
 Several years before they had run out,
  resulting in:
    Lost sales

    Angry customers (potentially lost!)

 They were in a bind
An earlier solution
 They had studied the variability of
  consumption (related to weather, etc.)
 They built a simple model and adopted a
  very conservative (i.e., lots of inventory)
  inventory policy.
 Since then, consumption had grown
 They couldn’t find enough warehouse space
  to hold all of the inventory
    Why the problem was difficult
   Most sales are in June, July and August
   Production facility had maximum production rate
    which could produce (each month) perhaps 12
    percent of annual sales
   The policy was to be sure that at the beginning of
    each month there would be enough product in
    inventory to cover the next 1.6 months’ forecast
    demand. This meant that on July 1 they needed to
    have more than 50% of a year’s sales in inventory
   This meant that on June 1 they needed lots of
    inventory (because of their limited production
What they wanted from us
   Find a way to make things better: increase
    “product availability” and/or decrease cost
What we needed to do
 Understand their current forecasting
 Determine whether current forecasts were
  “best possible”, or, if could be improved,
  improve them
 Build a model to allow computation of
  system performance. This model should be
  compatible with their current production
  planning techniques

Lots of warehouses, months, product sizes, product flavors
Their production planning
   Each month:
      start with current inventory

         Of each type

         In each region

         In each month

      Obtain forecasts for demands

   Use linear programming to solve for optimal
    production levels for this month for each plant.
    Solution is for this period and “all” future periods.
   Carry out the current month of the plan; move to
    next month (i.e., go back to the top of this page)
We first examined their forecasts
   How can you tell if forecasts are bad?
     Good forecasts should be a martingale

     What’s a martingale?

        Example: fortune of a gambler playing
         a fair game.
        Example: Conditional expected values
         of a future (unknown) quantity given
         more and more information.
One simple property:
 Martingales have “uncorrelated increments”
 You can perform simple tests (regression,
  for example) to determine whether observed
  increments (i.e., changes in forecasts for a
  given product, region, and date) are
 If they’re correlated, you can use the
  correlations to improve the forecasts
We checked our data and found
 Forecast changes had fairly high negative
 That meant that if the forecasts were raised,
  it was likely that later they would be
 This meant that the forecasts were far from
The forecasts were constructed by:
 Initially, forecasts (for the sales in some future
  month) were simply set equal to a multiple
  (like 1.06) of last year’s demand.
 As a given month grew closer, the sales
  estimates for that month were updated
 Here’s an example of how the forecasts varied
  over time (and how forecasts eventually
  became “past data”)
              Actual          Forecast             Trivial forecast
Delivery      March   April    May       June   July     August   Sept

March 1 20            25       40        60     100      90       40
April 1       22 A 26          42        63     100      90       40
May 1         22A     24 A     41        62     102      90       40
June 1        22 A 24 A        45 A      66     110      92       40
July 1        22 A 24 A        45 A      65 A   108      91       41
Aug 1         22 A 24 A        45 A      65 A   110 A    92       42
Sept 1        22 A 24 A        45 A      65 A   110 A    88 A     40
Since the forecasts looked bad
 We decided to find out how they were done
 I asked to meet the person who made the
 It turned out that forecasts came from the
  sales department.
 The job of forecasting was given to the
  newest employee in the sales department!
 (We asked why. They said that this gave
  new employees a good introduction to
We went to the client
 Our advice: You have to fix the forecasting
 We found that others in the firm already had
  a better forecasting technique.
 Remember that there were several plants
  (and warehouses). If one plant were short
  inventory, product could be shipped to it by
   The firm had tried to understand how many trucks
    they needed to handle these shipments.
   For this they needed forecasts of demand.
   The produced forecasts as follows:
      For each “customer” (a “customer” might be a
       chain of supermarkets, for example)
         They knew when the customer had last

          received a shipment.
         Based on season, temperatures, previous

          sales etc., they could estimate when this
          customer would reorder (and reorder, …)
Adding these expected orders
 provided forecasts of future demand.
 We checked to see whether the successive
  forecasts looked like martingales
  (regression analysis again) and found that
  they did (i.e., we couldn’t improve the
 But we weren’t done!
What should we recommend to the client?

    Client’s questions:
      How much money were we talking about
      If the benefits were small, then the risks
       (of having a new model) and the costs of
       implementation might outweigh the
      How should inventory levels be set?
To answer these questions
 We needed to have a simulation. (The
  production process was far too complex for
  us to build analytic models.)
 To run the simulation we needed to be able
  to generate forecasts which behaved “like”
  the observed data (from the “store by store
  customer” analysis).
 We wanted to be sure that stock-outs would
  be infrequent, and costs would be low.
A simulation process
We need to produce “simulated
forecasts” to drive the simulation
 These simulated forecasts should be the
  results of a good estimation procedure
 The forecasts should resemble, as closely as
  possible, the observed forecast behavior.
 So we analyzed the “good” forecasts to
  build a model for generating random
  variables to use in simulations
              Recall the forecast and actual values
Delivery      March   April   May    June   July      August   Sept

March 1 20            25      40     60     100       90       40
April 1       22 A 26         42     63     100       90       40
May 1         22A     24 A    41     62     102       90       40
June 1        22 A 24 A       45 A   66     110       92       40
July 1        22 A 24 A       45 A   65 A   108       91       41
Aug 1         22 A 24 A       45 A   65 A   110 A     92       42
Sept 1        22 A 24 A       45 A   65 A   110 A     88 A     40
How can we construct these?
 How do the forecasts evolve? To
  understand this, take differences
 Want to compare two forecasts for several
  months of demand at two successive
  forecast dates.
 Example:
 Months out:      1 2 3           4   5
    March        20 25 40 60 100
    April        22 26 42 63 100
    Difference: 2      1     2      3  0
For each month
   We look at changes (from each month to the next)
    of the forecasts for each future month.
   This gives us a sequence of vectors. The number
    of components that change from month to month
    is, in our example, 4.
   The first such vector is (2,1,2,3), giving the
    changes, from March 1 to April 1, of the forecast
    values (and one actual value) for each month.
 This gets us a set of vectors (or curves).
 On average (over all curves) the changes
  should be 0. (This is the martingale
 We’d like to generate new curves which
  have essentially the same variance-
  covariance structure as the data. So we
  estimate this VCV structure and generate
  samples for use in the simulation.
The first 4 differences:
   Mar – April       2, 1, 2, 3
   Apr – May         -2, -1, -1, 2
   May – June         4, 4, 8, 2
   June – July       -1, -2, -1, -1
   I’ll call these vectors “wiggles” (They’re the
    “wiggles” we’ve observed in past forecast
   In practice we’d get many such vectors, compute
    their covariance, and then choose our random
    variables for a simulation to have a Normal
    distribution with mean 0 and VCV equal to that of
    the vectors above.
 We use “Principal Components”
 Principal components analysis finds
  “fundamental wiggles”, based on the data.
 These “fundamental wiggles”, expressed as
  vectors, are orthogonal.
 The fundamental wiggles vary in size; we
  arrange them in decreasing order of size (so
  “biggest first”)
 We sometimes, to simplify the model, decide
  how many of these wiggles to keep (say M)
  and keep only wiggles 1,…M
    How to get the simulated demands
    and demand forecasts
   We assume we have a current forecast (vector) for the
    demand in each future period.
   To get the next forecast vector we
    Report the first entry in the forecast vector as the
    current realized demand. We then “slide the remaining
    forecasts to the left “one place”.
   For each “fundamental wiggle” we add (the wiggle),
    multiplied by an independent standard normal random
    variable, to our forecast vector
   The result is our new forecast vector.
   We feed these forecasts to the simulation
    routine and get
The final presentation
   Peter Jackson and I presented our work
      Audience:

         client’s project team members

         The head of that group

         The head of that division

      I presented the theory and model, and
       Peter presented the simulation results and
       our recommendations
The room was quiet
 Everybody seemed quite attentive
    They were participants in the project

    The “higher-ups” were present

 At the end of Peter’s presentation he
  asserted, “Based on this analysis, we
  believe you can achieve better product
  availability and, at the same time, save
  about $11,000,000 a year.”
There was a long moment of silence
 Nobody spoke
 Finally, the most senior manager present
  said, “That’s almost a million a month. We
  have to do it!”
 Then there was applause …
 (Reports of how things went in later years
  were positive. But, as usual, we didn’t get
 You can find the slides of this talk, as well
  as the published article, at
 By the way, we didn’t actually use the data
  in the say I described. We found that
  modelling changes in the logarithm of the
  data worked better. But it required some
  additional mathematics to make the
  resulting forecasts martingales.

To top