This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Sandroni, A. Articles by Vohra, R. V. Search for Related Content

Calibration with Many Checking Rules

Kellogg School of Management, MEDS Department, Northwestern University, 2001 Sheridan Road, Evanston, Illinois 60208
Davidson Faculty of Industrial Engineering and Management, Technion, Haifa 32000, Israel
Department of Managerial Economics and Decision Sciences, Kellogg School of Management, Northwestern University, Evanston, IL 60208
sandroni{at}kellogg.northwestern.edu
rann{at}ie.technion.ac.il
r-vohra{at}nwu.edu

Each period an outcome (out of finitely many possibilities) is observed. For simplicity assume two possible outcomes, a and b. Each period, a forecaster announces the probability of a occurring next period based on the past.

Consider an arbitrary subsequence of periods (e.g., odd periods, even periods, all periods in which b is observed, etc.). Given an integer n, divide any such subsequence into associated subsubsequences in which the forecast for a is between [i /n, i+1/n), i {0, 1, ..., n}.

We compare the forecasts and the outcomes (realized next period) separately in each of these subsubsequences. Given any countable partition of [0,1] and any countable collection of subsequences, we construct a forecasting scheme such that for all infinite strings of data, the long-run average forecast for a matches the long-run frequency of realized a's.

A good Christian must beware of mathematicians and those soothsayers who make predictions by unholy methods, especially when their predictions come true, lest they ensnare the soul through association with demons. St. Augustine, De genesis ad litteram, Book II.

Key Words: Forecast; probabilistic forecast; calibration
History: Received: August 30, 2000; revision received: September 10, 2001;revision received: May 14, 2002;revision received: May 13, 2002;

This article has been cited by other articles:

				W. Olszewski and A. Sandroni Manipulability of comparative tests PNAS, March 31, 2009; 106(13): 5029 - 5034. [Abstract] [Full Text] [PDF]

				W. Olszewski and A. Sandroni Strategic Manipulation of Empirical Tests Mathematics of Operations Research, February 1, 2009; 34(1): 57 - 70. [Abstract] [PDF]