Completing the Forecast
Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts
Board on Atmospheric Sciences and Climate
Background
Sponsor: NOAA-NWS Focus: estimation and communication of uncertainty in weather, hydrological, and short-term climate forecasts Process: 5 meetings, in Boulder and D.C. (including at AMS broadcast and WAF conference), with almost 50 presenters and other contributors
The study recognizes the diverse roles of participants in the weather and climate enterprise, as described in the Fair Weather report, and diverse needs of forecast users. Recommendations are primarily to NWS, but potentially also to other components of NOAA, other government agencies, and nongovernmental sectors (e.g., commercial meteorologists, media, weather risk management institutions, private companies).
Task
Provide guidance on how to identify and characterize needs for uncertainty information among various users of forecasts. Support this with examples of needs.
Identify limitations in current methods for estimating and validating forecast uncertainty, relating these limitations to users’ needs and recommend improvements or new methods and approaches. Identify sources of misunderstanding in communicating forecast uncertainty, including vulnerabilities dependent on the means of communication used, and recommend improvements in the ways used to communicate forecast uncertainty.
Recognizing the breadth and depth of this task, NWS advised the committee at its opening meeting to "teach us how to fish as opposed to giving us a fish."
Committee
RAYMOND J. BAN (Chair), The Weather Channel, Inc., Atlanta, GA JOHN T. ANDREW, California Department of Water Resources, Sacramento BARBARA G. BROWN, National Center for Atmospheric Research, Boulder, CO DAVID CHANGNON, Northern Illinois University, DeKalb, IL KONSTANTINE GEORGAKAKOS, Hydrologic Research Center, San Diego, CA JAMES HANSEN, Massachusetts Institute of Technology, Cambridge RONALD. N. KEENER, JR., Duke Energy, Charlotte, NC UPMANU LALL, Columbia University, New York, NY CLIFFORD F. MASS, University of Washington, Seattle REBECCA E. MORSS, National Center for Atmospheric Research, Boulder, CO ROBERT T. RYAN, NBC4, Washington, DC ELKE U. WEBER, Columbia University, New York, NY
NRC Staff
PAUL CUTLER, Study Director LEAH PROBST, Research Associate ROB GREENWAY, Senior Program Assistant
Review Process
Review Oversight
George Frederick, Vaisala, Inc. Kuo-Nan Liou, University of California, Los Angeles
Reviewers
James Block, DTN-Meteorlogix, Minneapolis, Minnesota David Budescu, University of Illinois, Champaign Simon Chang, Naval Research Laboratory, Monterey, California Holly Hartmann, University of Arizona, Tucson Kathryn Laskey, George Mason University, Fairfax, Virginia Daniel P. Loucks, Cornell University, Ithaca, NY Tom Skilling, WGN-TV News, Chicago, Illinois Mort Webster, University of North Carolina at Chapel Hill Lawrence Wilson, Meteorological Service of Canada, Quebec
Report Contents
Summary 1 Introduction 2 Uncertainty in Decision Making 3 Estimating and Validating Uncertainty 4 Communicating Forecast Uncertainty 5 Overarching Recommendations
Background
Uncertainty is a fundamental characteristic of weather, seasonal climate, and hydrological prediction, and no forecast is complete without a description of its uncertainty.
Nonetheless, for decades, users of weather, seasonal climate, and hydrological (collectively called “hydrometeorological”) forecasts have been conditioned to receive incomplete information about the certainty or likelihood of a particular event.
Background
Uncertainty was included in 19th Century forecasts Determinism in the 20th Century due to
progression of the science toward exactness numerical weather prediction satellite observations and models media
This legacy remains strong
An Opportunity
NWS and others in the Enterprise have recognized the need to view uncertainty as a fundamental part of forecasts. By partnering with other segments of the Enterprise to understand user needs, generate relevant and rich informational products, and utilize effective communication vehicles, NWS can take a leading role in the transition to widespread, effective incorporation of uncertainty information into hydrometeorological predictions.
Overarching Recommendations
There are nine overarching recommendations, all with equal priority. Detailed recommendations appear in Chapters 2, 3, and 4 that add further specificity and breadth. All recommendations should be considered in the context of NOAA’s Policy on Partnerships in the Provision of Environmental Information.
Recommendation 1
The entire Enterprise should take responsibility for providing products that effectively communicate forecast uncertainty information. NWS should take a leadership role in this effort.
Recommendation 2
NWS should improve its product development process by collaborating with users and partners in the Enterprise from the outset and engaging and using social and behavioral science expertise.
Recommendation 3
All sectors and professional organizations of the Enterprise should cooperate in educational initiatives that will improve communication and use of uncertainty information. In particular, 1) hydrometeorological curricula should include understanding and communication of risk and uncertainty, 2) ongoing training of forecasters should expose them to the latest tools in these areas, and 3) forecast providers should help users, especially members of the public, understand the value of uncertainty information and work with users to help them effectively incorporate this information into their decisions.
Recommendation 4
NWS should develop and maintain the ability to produce objective uncertainty information from the global to the regional scale.
Recommendation 5
To ensure widespread use of uncertainty information, NWS should make all raw and post-processed probabilistic products easily accessible to the Enterprise at full spatial and temporal resolution. Sufficient computer and communications resources should be acquired to ensure effective access by external users and NWS personnel.
Recommendation 6
NWS should expand verification of its uncertainty products and make this information easily available to all users in near real time. A variety of verification measures and approaches (measuring multiple aspects of forecast quality that are relevant for users) should be used to appropriately represent the complexity and dimensionality of the verification problem. Verification statistics should be computed for meaningful subsets of the forecasts (e.g., by season, region) and should be presented in formats that are understandable by forecast users. Archival verification information on probabilistic forecasts, including model-generated and objectively generated forecasts and verifying observations, should be accessible so users can produce their own evaluation of the forecasts.
Recommendation 7
To enhance development of new methods in estimation, communication, and use of forecast uncertainty information throughout the Enterprise, and to foster and maintain collaboration, confidence, and goodwill with Enterprise partners, NWS should more effectively use testbeds by involving all sectors of the Enterprise.
Recommendation 8
The committee endorses the recommendation by the NRC “Fair Weather” report to establish an independent advisory committee and encourages NOAA to bring its evaluation of the recommendation to a speedy and positive conclusion.
Recommendation 9
NWS should dedicate executive attention to coordinating the estimation and communication of uncertainty information within NWS and with Enterprise partners.
Key points not to forget
Broad applicability within NOAA and other agencies This will take time (turn the aircraft carrier) Let’s get started now NWS evolution - time is right to incorporate uncertainty information into process Uncertainty is another component of improving the forecast – it’s not detached from the primary mission of forecast improvement Uncertainty is not an add-on – it is in the nature of the product