USE OF HIDDEN MARKOV MODELS TO ASSESS PREDICTABILITY ON ANNUAL TO

							     USE OF HIDDEN MARKOV MODELS TO ASSESS
 PREDICTABILITY ON ANNUAL TO DECADAL TIME SCALES


                          Rick Katz

       Institute for Mathematics Applied to Geosciences
           National Center for Atmospheric Research
                        Boulder, CO USA

       Joint work with Yongku Kim & Balaji Rajagopalan


Home page: www.isse.ucar.edu/staff/katz/

Talk: www.isse.ucar.edu/staff/katz/docs/pdf/rwkhmm.pdf
                           OUTLINE



(1) Background


(2) Generalized Linear Model (GLM) Weather Generator


(3) Treatment of Overdispersion Phenomenon


(4) Hidden Markov Model (HMM) for Climate Regimes


(5) Extensions
                         (1) Background


• Shifts in Climate Regimes in Argentine Pampas

-- Evidence?

-- Statistical models
  Deterministic trend vs. regime shift

-- Hidden Markov model

  Mixture of distributions (hidden states)
  Markov chain (persistence of hidden states)
• Pampas Region of Argentina


-- Marked wet season
 Southern Hemisphere summer

-- ENSO teleconnections

 Wetter than normal during El Niño events

 Smaller temperature range during El Niño events

 (i. e., higher minimum & lower maximum temperature)
                  (2) GLM Weather Generator


• Generalized Linear Modeling Approach

-- Furrer & Katz (2007) [following Chandler et al., Stern & Coe]

-- R functions glm & lm


• Daily Precipitation Occurrence

-- Markov chain model (two-state, first-order)

   Jt = 1 if wet day,     Jt = 0 if dry day

 Annual cycle in transition probabilities
 Also dependence on ENSO state (Zt , monthly mean)
-- Probability of precipitation occurrence      pt = Pr{Jt = 1}

 ln[pt / (1 − pt)] = β0 + β1 Jt−1               (Markov dependence)


       + β2 cos(2πt /T) + β3 sin(2πt /T)        (Single annual cycle)


       + β4 Zt                                  (Single ENSO effect)


       + β5 Jt−1 cos(2πt /T) + β6 Jt−1 sin(2πt /T) (Two annual cycles)


       + β7 Jt−1 Zt                          (Different ENSO effect)
      (3) Treatment of Overdispersion Phenomenon


• Aggregated Climate Variables

-- Introduce as additional covariates in GLM weather generator

-- Seasonal total precipitation & mean max. & min. temperature

• Need to Smooth Covariates

-- Rationale
 “Signal” (trends or regimes) vs. “noise” (sampling error)


-- Smoothing technique (LOESS)
 R function loess
-- Precipitation Occurrence

PtS time series of summer (S) smoothed (LOESS) total precipitation

PtW time series of winter (W) smoothed (LOESS) total precipitation

It = 1 if summer, It = 0 otherwise (indicator variable)

Omit ENSO covariate

  ln[pt / (1 − pt)] = β0 + β1 Jt−1               (Markov dependence)

        + β2 cos(2πt /T) + β3 sin(2πt /T)        (Single annual cycle)

        + β4 Jt−1 cos(2πt /T) + β5 Jt−1 sin(2πt /T) (Two annual cycles)

        + β6 It PtS + β7 (1 − It ) PtW       (Aggregated covariates)
Pergamino: Annual Total Precipitation
  (4) Hidden Markov Model (HMM) for Climate Regimes

• Seasonal or Annual Climate Time Series {Pt , t = 1, 2, . . ., T }

-- Hidden states {Ht , t = 1, 2, . . .}
  For example, Ht = 1 or 2

-- Model conditional distribution given hidden states (Pt │Ht = i)
  Assume Pt’s conditionally independent given Ht’s


-- Ht finite-state, first-order Markov chain (two or more states)

-- Hidden Markov dependence for Ht can induce “regime-like”
  behavior in Pt (even though Pt’s conditionally independent)
• Parameter Estimation

-- Expectation-Maximization (EM) algorithm


• Hidden State Identification

-- “Local” decoding

   Pr{Ht = i │P1, P2 , . . ., PT }, t = 1, 2, . . .


-- “Global” decoding

   Pr{H1 = h1, H2 = h2, . . ., HT = hT │P1, P2 , . . ., PT }

 maximize over {h1, h2, . . ., hT} using Viterbi algorithm
Local decoding
                            (5) Extensions


• Statistical Downscaling

-- GLM weather generator (daily time scale)

-- HMM as covariate (seasonal time scale)
 Replace aggregated climate variable


• Non-Homogeneous HMM (NHMM)

-- Observed covariates as well as hidden state (e.g., ENSO)