What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting

TLDR: Three distinct types of goodness are identified in this paper: the correspondence between forecasters’ judgments and their forecasts, the correspondencebetween the forecasts and the matching observations, and the incremental economic and/or other benefits realized by decision makers through the use of the forecasts (type 3 goodness, or value).

Abstract: Differences of opinion exist among forecasters—and between forecasters and users—regarding the meaning of the phrase “good (bad) weather forecasts.” These differences of opinion are fueled by a lack of clarity and/or understanding concerning the nature of goodness in weather forecasting. This lack of clarity and understanding complicates the processes of formulating and evaluating weather forecasts and undermines their ultimate usefulness. Three distinct types of goodness are identified in this paper: 1) the correspondence between forecasters’ judgments and their forecasts (type 1 goodness, or consistency), 2) the correspondence between the forecasts and the matching observations (type 2 goodness, or quality), and 3) the incremental economic and/or other benefits realized by decision makers through the use of the forecasts (type 3 goodness, or value). Each type of goodness is defined and described in some detail. In addition, issues related to the measurement of consistency, quality, and value ar...