Zurück zur Übersicht

Calibration tests for count data

Publikationsart Peer-reviewed
Publikationsform Originalbeitrag (peer-reviewed)
Publikationsdatum 2014
Autor/in Wei Wei, Held Leonhard,
Projekt Statistical methods for spatio-temporal modelling and prediction of infectious diseases
Alle Daten anzeigen

Originalbeitrag (peer-reviewed)

Zeitschrift TEST
Volume (Issue) 23(4)
Seite(n) 787 - 805
Titel der Proceedings TEST
DOI 10.1007/s11749-014-0380-8


Calibration, the statistical consistency of forecast distributions and observations, is a central requirement for probabilistic predictions. Calibration of continuous forecasts has been widely discussed, and significance tests are commonly used to detect whether a prediction model is miscalibrated. However, calibration tests for discrete forecasts are rare, especially for distributions with unlimited support. In this paper, we propose two types of calibration tests for count data: tests based on conditional exceedance probabilities and tests based on proper scoring rules. For the latter, three scoring rules are considered: the ranked probability score, the logarithmic score and the Dawid-Sebastiani score. Simulation studies show that all the different tests have good control of the type I error rate and sufficient power under miscalibration. As an illustration, we apply the methodology to weekly data on meningoccocal disease incidence in Germany, 2001--2006. The results show that the test approach is powerful in detecting miscalibrated forecasts.