Prequential Analysis

"Prequential" is a portmanteau word for predictive sequential--a broad statistical methodology founded on a view of data (like Mark Twain's view of history) as "just one darned thing after another". This tutorial will outline the basic concepts of prequential analysis, and some of its properties and applications. Suppose we have been sold a method that purports to learn projectible regularities in a data-sequence. Prequential analysis assesses how well this works by contrasting its one-step ahead forecasts with realised outcomes. Like cross-validation, by never using an observation to contribute to its own forecast this procedure avoids over-optimistic "substitution bias". But, unlike cross-validation, it also avoids indirect substitution bias, since no two observations can each contribute to the other's forecast. This leads to more reliable performance, even when there is no natural ordering in the data. It also allows the development of a fruitful and elegant general theory. There are close connexions with martingale theory, wih the theory of online learning with expert advice, and with game-theoretic probability. Many traditional statistical concepts, such as consistency and efficiency, as well as practical techniques such as model selection and inference from misspecified models, take on an interesting new aspect when reconsidered from the prequential point of view.