Conclusion by dffhrtcv3


									Chapter 7


   Extratropical cyclones are fundamental to the everyday weather of the midlatitudes.
They provide essential rainfall for human activities such as agriculture, but can also cause
large amounts of damage by their strong winds and heavy precipitation. It is therefore very
important that these cyclones are predicted as accurately and as far in advance as possible
by numerical weather prediction (NWP) models. The aim of this thesis is to explore the
prediction of extratropical cyclones by NWP using the objective feature tracking program
   In the past studies of the prediction of extratropical cyclones have mainly focused on
individual cyclones or cyclone simulations. There have been some statistical studies, but
these have used manual or semi-automated methods to identify and track the cyclones. As
a result these studies have been limited due to the large amount of work involved. The
work of this thesis is the first to use a fully automated method of identification and tracking
to investigate the prediction and predictability of extratropical cyclones by modern NWP
   In chapters 4 and 5 the prediction and predictability of extratropical cyclones was
explored using the ERA40 re-analysis system to construct forecasts for different observing
systems. The methodology (and consequently the results) of chapter 4 was limited by the
dataset, but in chapter 5 an alternative approach is developed and implemented, which has
provided detailed information about the prediction of cyclones. In chapter 6 the method
was used to evaluate the ECMWF and NCEP ensemble prediction systems (EPS). The work
of this thesis has also involved the development of a TRACK Internet Service (chapter 3)
to help with the large amount of data processing required for the predictability analysis.

7. Conclusion                                                                            170

   This chapter continues by returning to the aims discussed in chapter 1. In section 7.1 we
describe how each of these aims has been addressed and discuss the main conclusions. This
is followed by a discussion of potential future work in section 7.2 and some final remarks
are given in section 7.3.

7.1     Aims Addressed

   A description of how the aims of this thesis have been addressed will now be given.

   1. Investigate the prediction and predictability of extratropical cyclones by modern NWP.
   This is the main objective of this thesis and it was addressed throughout chapters 4, 5
and 6. The results show that current numerical models are able to predict the position of
the cyclones with reasonable (or often high) accuracy, but the amplitude of the cyclones
is somewhat more difficult to predict. Estimates of potential predictability also suggest
that the prediction of intensity could be improved considerably more than the position via
changes to the model. The difference in the ensemble spread and ensemble mean error, for
the intensity of cyclones, further highlights the difficulty. Further analysis of the position
error, shows that errors in propagation speed play a larger role than errors in direction.
There is small bias for forecasted cyclones to propagate too slowly and preliminary results
indicate that this may correspond to a small bias for the forecasted intensity to be too
   We believe that the main reason for the lower levels of skill in cyclone intensity and
propagation speed than that of the direction, is due to errors in the storms vertical struc-
ture. In chapter 2 we described the process of baroclinic instability, which is the primary
mechanism by which extratropical cyclones develop. A vertical tilt is critical to the growth
of extratropical cyclones, as it allows upper and lower level disturbances to interact in the
feed-back process of self-development. The propagation speed of storms also depends on
the interaction of upper and lower level disturbances. As discussed in chapter 2, baroclinic
instability can be described in terms of the vertical coupling of edge waves at the surface
and at the tropopause. The two waves will initially be moving at different speeds, but once
the two waves become phase-locked together they will propagate at the same speed. The
correspondence between an overprediction of intensity and underprediction of propagation
7. Conclusion                                                                          171

speed seems feasible, since the surface wave will initially be moving faster than the mean
flow until it couples with the upper level wave. This will reduce the propagation speed
and the disturbance will intensify. To summarise, errors in the storms vertical structure
will cause errors in both the intensity and propagation speed of the storm. In contrast the
direction of the storm will be mainly determined by the steering level winds and will not
be significantly affected by errors in the vertical structure.
   How could the prediction of cyclone intensity and propagation speed be improved?
The most obvious answer is probably a higher vertical resolution of observations. This
is discussed in more detail for aim 2, but perhaps the current observing network does
not provide sufficient observations to properly capture the vertical structure of storms.
Horizontal and vertical model resolution also appears to be important for predicting the
growth of storms. This is particularly true in the northern hemisphere (NH), where the
growth and development of storms is perhaps more difficult to predict because of the
variable surface boundary conditions of orography and sea surface temperatures. NH storms
also have significantly greater growth rates than SH storms (Hoskins and Hodges, 2002,
2005) and higher model resolutions may therefore be required to accurately capture their
growth. Data assimilation will also be important. Methods such as 4DVAR, which fit a
model trajectory to observations taken throughout the assimilation window, may improve
the prediction of baroclinic systems.
   The position of extratropical cyclones is predicted better than the intensity, but which
is more important? For intense cyclones the position is probably more important, since
knowing where a storm is likely to strike and cause damage is more important than know-
ing exactly how much damage it will cause. However, it is necessary to know that the
storm is going to be intense. A forecast of an intense storm, which is perfect in terms of
position, but gives no indication that the storm will be intense would not be very helpful.
Any necessary warnings and precautions would not be taken and in this case some error in
position could be compromised for an indication of the severity of the storm.

   2. Explore the impact different types of observation have on the prediction of extratrop-
ical cyclones.
   The impact that different types of observations have on the prediction of storms was
explored in chapters 4 and 5. In the NH the terrestrial system (surface and upper air
7. Conclusion                                                                             172

observations) had a higher level of skill than the satellite system. This is probably because
the satellite system does not give a high enough vertical resolution of observations to
accurately predict the storms development. This idea was confirmed when errors in growth
of the cyclones were considered, which were significantly larger for the satellite system. In
the SH the satellite system had the higher level of skill than the terrestrial system, but
this is to be expected since a majority of observations in the SH are satellite. The satellite
system predicted the growth of cyclones better in the SH than the NH, which indicates
that a high vertical resolution of observations is more important for the prediction of NH
storms than SH storms.
   In the future it is expected that the observing network will become more satellite based
with less radiosondes. The results of this thesis suggest that this may be detrimental to the
prediction of cyclones. Increasing radiosonde and other upper air observations could po-
tentially improve the prediction of cyclones, particularly in terms of their intensity. Higher
quality and resolution of satellite observation could also be beneficial. As well as increas-
ing observations in general, the use of targeted observations in which areas where storm
development is expected could have a large impact on forecast skill (Leutbecher et al., 2002).

   3. Evaluate the prediction of extratropical cyclones by EPS.
   In chapter 6 the prediction of extratropical cyclones by the ECMWF and NCEP EPS
was evaluated. Overall the ECMWF EPS had a slightly higher level of performance than
the NCEP EPS, but the results suggest that this is more to do with the model and data
assimilation system than the perturbation methodology. This agrees with the results of
the Buizza et al. (2005) study, which used the more conventional methods of ensemble
verification discussed in 2.4.5. The large number of differences between EPS of different
centres (e.g. resolution, model, data assimilation methods) has made it difficult to compare
the different perturbation methodologies using both the storm tracking approach of this
thesis and with the more conventional methods of Buizza et al. (2005). This really highlights
the importance of comparing the different perturbation methods using a single model and
assimilation system discussed in chapters 2 and 6.
   Although it has not been possible to draw any conclusions concerning the different
perturbation methodologies, the results of chapter 6 have illustrated a number of benefits
an ensemble forecast can offer over a single deterministic forecast. Firstly we find the high
7. Conclusion                                                                             173

level of skill of the best ensemble member very encouraging. Clearly an ensemble forecast
will always have a best ensemble member, but it is the difference in skill between this best
ensemble member and that of the average member we find encouraging. By day 5 of the
forecast it is about 3.5 days better for the intensity of the storms! One of the properties of
a perfect ensemble forecast must be that one of the ensemble members provides a perfect
forecast (Buizza, 1997). The skill of the best ensemble member therefore gives an indication
of how close to satisfying this criterion an EPS is. It also suggests that the errors in the
initial state are being sampled effectively. These arguments do however take a theoretical
viewpoint. From a practical perspective, the main question is the potential to identify the
best ensemble member at some useful time. Preliminary results suggest this may be very
   A second benefit offered by an EPS is the potential for some members to give very early
indications of storms. The results show that some members may provide an indication of
a storm as much as 7 days in advance. This is encouraging; however, the reliability of
the prediction of a storm by one or a few ensemble members also needs to be considered.
The reliability of an ensemble has not been addressed in this thesis, but will be considered
in future work (see section 7.2). The final benefit of an EPS, the results of this thesis
have illustrated, is the measure of the predictability of the atmosphere provided by the
ensemble spread. For the position of storms, the ECMWF ensemble mean error is very
close to the ensemble spread. This has a very useful practical application, since the spread
of the ensemble can give the forecaster an indication of the accuracy of the ensemble mean
forecast. Unfortunately the same is not true for the intensity of storms; the ensemble spread
is less than the ensemble mean error.
   An interesting result of this thesis is the superior quality of the ECMWF control forecast
to the perturbed members. It is   2   to 1 day better throughout the forecast. Unfortunately
data limitations have stopped us from determining if this is also the case for the NCEP
EPS. The question of whether the control forecast should have such an advantage over the
perturbed members is one of current debate. Palmer et al. (2005) argue that even for a per-
fect EPS, the control forecast will be better than the perturbed members on average. This
will certainly be true for the earlier part of the ECMWF ensemble forecast (see chapter
6) and the perturbed physics may also give the control forecast an advantage later in the
forecast. The control forecast has been produced from the best estimate of the initial state
7. Conclusion                                                                           174

and forecast model and may therefore be expected to have a higher level of skill than the
perturbed members. For an ensemble system based only on initial condition perturbations
it would perhaps be expected that the error of the control forecast would converge to that
of the perturbed members at higher forecast times. When we investigated whether the
ensemble member that was best for the first day or two of the forecast was also best at
the end of the forecast, we found that the error converged quickly to that of the average
perturbed member. Would this also then be expected for the control forecast? The answer
to this question is perhaps yet to be determined, but we believe that the difference in
skill between the control and the perturbed members should be considered as part of the
evaluation of an EPS. If the control forecast has a consistently higher level of performance
than the perturbed members it should perhaps be weighted accordingly in the calculation
of the ensemble mean.

   4. Develop a TRACK Internet Service to allow distributed datasets to be diagnosed with
distributed computing.
   In chapter 3 an Internet Service was developed to allow users to run the TRACK
program from a web browser with NCEP re-analysis and EPS data. These datasets are
both archived in the USA. The service also allows a list of multiple jobs to be submitted
to the Condor pool in ESSC, so that each job can be run on a different computer. The
Internet Service was used to compute the storm tracks from the NCEP EPS data that were
analysed in chapter 6. This drastically reduced both the time taken to process the data
and the amount of data that needed to be stored locally.
   In the future it is hoped that this service and services like it will be used as a tool
for other areas of scientific research. The TRACK Internet Service has already been used
by scientists from the US Navy to study past storms using the NCEP re-analysis data.
EScience methodologies, such as those used by the Internet Service, could also be very
useful for operational NWP. Condor is ideal for ensemble prediction, since each ensemble
member can be run on a different computer. The protocol OPeNDAP used to access the
remote data currently offers limited security. If these issues were addressed, in the future
protocols such as OPeNDAP could be very useful for providing operational data to scientific
7. Conclusion                                                                             175

7.2     Future Work

   Throughout this thesis the results have been limited by the size of the data sample.
The analysis methodology requires considerably larger data samples than standard verifi-
cation methods. In the future it is hoped that larger data samples will allow more extensive
diagnostics to be produced. The diagnostics presented in this thesis have been for entire
hemispheres. Larger datasets would allow more regional analysis to be performed to de-
termine differences in predictive skill and predictability. For example data dense areas,
and regions downstream of these data dense areas, will be expected to have higher levels
of forecast skill. There may also be differences in skill for storms travelling mainly over
ocean (such as those in the SH) and storms travelling over land. Larger datasets would
also allow the prediction of storms at different stages of their lifecyle to be considered.
The results of this thesis include storms at different stages of development at different lead
times. Since the results show that growth of cyclones is difficult to predict, storms which
are more developed in the initial conditions of the forecast may be predicted better by
forecast models.
   The observing system experiments of chapters 4 and 5 were performed using a 3DVAR
data assimilation system. Using a more advanced 4DVAR assimilation system could po-
tentially improve the prediction of the storms considerably. In future work we hope to
repeat the observing system experiments using 4DVAR and ECMWF’s Interim Reanalysis
system. It is expected that this may improve the prediction of the growth of baroclinic sys-
tems considerably. Throughout this thesis the results concerning the intensity of cyclones
have used the T42 filtered values. It is possible to perform the tracking using the filtered
fields and then obtain the actual storm intensities from the unfiltered fields. In the future
this will be investigated to see if it has any impact on the results concerning the intensity
of cyclones. We also plan to explore the vertical tilts of the predicted storms. This will
hopefully confirm our belief that the large errors in the predicted intensities of the cyclones
is due to errors in the vertical structure.
   In chapter 6 we discuss how the limitations of the NCEP EPS dataset allow only
a preliminary comparison of the two ensemble systems to be performed In future work
we hope to perform a more complete comparison of the two systems and other ensemble
systems by making use of the data provided by the THORPEX Interactive Grand Global
7. Conclusion                                                                            176

Ensemble (TIGGE) project (see THORPEX, 2005, for more details of the project). One
of the objectives of this project is to provide a central resource of operational ensemble
forecast data available within the research environment. It is hoped that this dataset
might allow us to evaluate the different perturbation methodologies more effectively. The
impact that ECMWF’s stochastic physics scheme has on the storms could be investigated
and the difference in skill between the control forecast and perturbed members of other
EPS could be explored. Multimodel ensemble systems could also be studied. This thesis
has not addressed the reliability of EPS. In future work we plan to address this issue by
exploring how probabilistic scoring methods, such as the Brier Skill Score (see chapter 2),
can be extended to storm tracks.
   The storm predictability diagnostics of this thesis were produced from only those fore-
cast and analysis tracks that match and therefore provide a rather optimistic measure of
forecast skill (see chapter 5). It is clearly not possible to produce error diagnostics for
unmatched tracks, but in the future we would like to investigate the nature of these tracks
a little more. We expect that the unmatched tracks will mainly correspond to fairly weak
disturbances generated by the forecast model at high lead times. It would, for example, be
possible to generate statistics from the unmatched tracks, of mean intensity and average
lead time at which the forecast storms are generated, to obtain this type of information.
   In this thesis the storm tracking methodology has been applied to extratropical cyclones.
It could in theory be applied to any transient weather system such as polar lows, tropical
cyclones and easterly waves. The method has been used already to study some individual
tropical cyclones. Figure 7.1 shows the tracks and intensities of hurricane Katrina predicted
by the ECMWF EPS. The predictions are shown for three different forecast start times
using matching criterion (iii) of chapters 5 and 6. The ECMWF analysis track and intensity
is also shown for each of the forecast start times. Hurricane Katrina formed as a tropical
storm in the Southern Bahamas. It then travelled across the central Bahamas, gaining
intensity until it reached Florida on the evening of 25th August 2005 as a category 1
hurricane. The storm then moved west towards the Gulf of Mexico. Katrina reduced in
strength to a tropical storm briefly, but quickly regained intensity reaching category 5 over
the Gulf of Mexico. The storm hit Grand Isle, Louisiana on the 29th August as a category
4 hurricane. It then moved slightly east of New Orleans before moving north.
   Figures 7.1 (a) and (b) shows the forecast started at 1200 UTC 22nd August 2005,
7. Conclusion                                                                                                                          177

                                                                          22           Ensemble Members
                                                                          18           Analysis

                                                   Intensity (10-5 s-1)
                                                                               0   1     2    3    4      5    6      7   8   9   10
                                                                                              Forecast Lead Time (days)
                           (a)                                                                            (b)

                                                   Intensity (10-5 s-1)

                                                                               0   1     2    3    4      5    6      7   8   9   10
                                                                                              Forecast Lead Time (days)

                           (c)                                                                            (d)

                                                   Intensity (10-5 s-1)

                                                                               0   1     2    3    4      5    6      7   8   9   10
                                                                                              Forecast Lead Time (days)

                           (e)                                                                            (f)

Figure 7.1: ECMWF ensemble forecasts of hurricane Katrina. The tracks and intensities, as a function of
forecast lead time, of the analysed storm and storm predicted by the ensemble members started at 1200
UTC 22nd Aug 2005 (a, b), 0000 UTC 25th Aug 2005 (c, d) and 00UTC 27th Aug 2005 (e, f) are shown.
The mean track and mean intensity of the ensemble members is also shown.
7. Conclusion                                                                           178

for which 32 perturbed ensemble members matched. The analysis track begins at 0000
UTC 20th August 2005. All of the ensemble members lie to the right of the analysed track
and the mean track does not offer any advantage over the control forecast at this time.
A majority of the ensemble members (including the control) do not predict the hurricane
force of the cyclone. A few give an indication very late into the forecast, but 1 ensemble
member does predict the rapid growth of the cyclone. Figures 7.1 (c) and (d) show the
forecast started 0000 UTC 25th August 2005, by which time the cyclone has almost reached
hurricane status. For this forecast 40 perturbed members match. Again there is a bias for
the ensemble member tracks to lie to the right of the analysed track. However, for this
forecast the mean track is clearly better than the control track at higher forecast times. A
large number of the ensemble members (including the control) predict the strength of the
cyclone, but once again the majority do not capture the rapid growth (see mean intensity).
Figures 7.1 (e) and (f) shows the forecast started 0000 UTC 27th August, for which all 50
perturbed ensemble members match. For this forecast the tracks are evenly spaced about
the analysed track resulting in an excellent mean track prediction, which is again better
than the control forecast at higher lead times. The intensity of the storm is predicted
better by the ensemble forecast than the earlier forecasts, but the rapid growth of the
cyclone is still underestimated. In future work a statistical analysis of tropical cyclones
could potentially be performed. This would require larger data samples than those used
for the extratropical cyclone analysis, since far smaller numbers of tropical cyclones occur
in a given season than extratropical cyclones.
   The TRACK Internet Service could be developed considerably in the future. Other
datasets provided via OPeNDAP could be included. These need not necessarily even be
atmospheric datasets. They could for example be ocean datasets, since the TRACK pro-
gram can also be used to track ocean eddies (Hodges, 1999b). At the moment it is possible
to download storm tracks and plot them in a web browser. Since this is rather limited,
the service could be extended to allow statistics to be generated from the computed storm
tracks. Statistics such as those in this thesis or those of Hodges (1996) could potentially
be generated from a web browser. The service currently runs on computers within ESSC
using Condor, which limits the number of users we are able to offer the service to. The
service could potentially be extended so that the user was able to run jobs on the National
Grid Service (NGS, GOSC, 2006). This is the UK’s largest operational grid with clusters
7. Conclusion                                                                             179

of computers located at Rutherford Appleton Laboratory (RAL) and the Universities of
Manchester, Oxford, Leeds, Sheffield and York. The use of NGS would allow a much larger
number of users and would enable larger amounts of data to be processed with TRACK.

7.3     Final Remarks

   The work of this thesis has demonstrated a new approach to forecast verification. Whilst
standard measures such as Root Mean Square Error (rmse) and Anomaly Correlation Coef-
ficient (acc) are very easy to apply and do provide a measure of forecast skill, their use from
a practical weather forecasters view may be rather limited. In contrast the storm tracking
approach of this thesis provides detailed information about the prediction of cyclones that
could be very useful to a weather forecaster, but the approach is not as easy to apply. Far
larger data samples are required and the computation of the diagnostics is more compu-
tationally expensive. Neither the storm tracking approach or the standard approaches are
perfect. All verification methods have biases and limitations that need to be considered
(see discussion in chapters 2 and 5). It is suggested that in the future operational fore-
cast centres should consider using a storm tracking verification approach in addition to the
standard approaches.

To top