Effective Scoring Rules for Probabilistic Forecasts

This paper studies the use of a scoring rule for the elicitation of forecasts in the form of probability distributions and for the subsequent evaluation of such forecasts. Given a metric (distance function) on a space of probability distributions, a scoring rule is said to be effective if the forecaster's expected score is a strictly decreasing function of the distance between the elicited and “true” distributions. Two simple, well-known rules (the spherical and the quadratic) are shown to be effective with respect to suitable metrics. Examples and a practical application (in Foreign Exchange rate forecasting) are also provided.