Weighted scoringRules: Emphasizing Particular Outcomes When Evaluating Probabilistic Forecasts
Dublin Core
Title
Weighted scoringRules: Emphasizing Particular Outcomes When Evaluating Probabilistic Forecasts
Subject
: forecast evaluation, probabilistic forecasting, proper scoring rules, R
Description
When predicting future events, it is common to issue forecasts that are probabilistic,
in the form of probability distributions over the range of possible outcomes. Such forecasts
can be evaluated using proper scoring rules. Proper scoring rules condense forecast performance into a single numerical value, allowing competing forecasters to be ranked and
compared. To facilitate the use of scoring rules in practical applications, the scoringRules
package in R provides popular scoring rules for a wide range of forecast distributions. This
paper discusses an extension to the scoringRules package that additionally permits the
implementation of popular weighted scoring rules. Weighted scoring rules allow particular
outcomes to be targeted during forecast evaluation, recognizing that certain outcomes are
often of more interest than others when assessing forecast quality. This introduces the
potential for very flexible, user-oriented evaluation of probabilistic forecasts. We discuss
the theory underlying weighted scoring rules, and describe how they can readily be implemented in practice using scoringRules. Functionality is available for weighted versions
of several popular scoring rules, including the logarithmic score, the continuous ranked
probability score, and the energy score. Two case studies are presented to demonstrate
this, whereby weighted scoring rules are applied to univariate and multivariate probabilistic forecasts in the fields of meteorology and economic
in the form of probability distributions over the range of possible outcomes. Such forecasts
can be evaluated using proper scoring rules. Proper scoring rules condense forecast performance into a single numerical value, allowing competing forecasters to be ranked and
compared. To facilitate the use of scoring rules in practical applications, the scoringRules
package in R provides popular scoring rules for a wide range of forecast distributions. This
paper discusses an extension to the scoringRules package that additionally permits the
implementation of popular weighted scoring rules. Weighted scoring rules allow particular
outcomes to be targeted during forecast evaluation, recognizing that certain outcomes are
often of more interest than others when assessing forecast quality. This introduces the
potential for very flexible, user-oriented evaluation of probabilistic forecasts. We discuss
the theory underlying weighted scoring rules, and describe how they can readily be implemented in practice using scoringRules. Functionality is available for weighted versions
of several popular scoring rules, including the logarithmic score, the continuous ranked
probability score, and the energy score. Two case studies are presented to demonstrate
this, whereby weighted scoring rules are applied to univariate and multivariate probabilistic forecasts in the fields of meteorology and economic
Creator
Sam Allen
Source
https://www.jstatsoft.org/article/view/v110i08
Publisher
ETH Zürich
Date
August 2024
Contributor
Fajar bagus W
Format
PDF
Language
English
Type
Text
Files
Collection
Citation
Sam Allen, “Weighted scoringRules: Emphasizing Particular Outcomes When Evaluating Probabilistic Forecasts,” Repository Horizon University Indonesia, accessed April 4, 2025, https://repository.horizon.ac.id/items/show/8344.