Validation in Statistics and Machine Learning - Abstract
Schiffner, Julia
In the recent years in different fields like statistics, machine learning or pattern recognition an increasing amount of so called local classification methods has been developed. Local approaches to classification are not new. Well-known examples are the k nearest neighbors method and classification and regression trees (CART). Moreover, localized versions of nearly all standard classification techniques like e.g. linear and Fisher discriminant analysis, logistic regression as well as boosting are available.
The bias-variance decomposition of prediction error is a useful concept to gain deeper insight into the behavior of learning algorithms. It was originally introduced for quadratic loss functions, but since in classification the misclassification rate is usually of interest, generalizations to zero-one loss have been developed in the last 15 years (e.g. James, 2003).
We employ the bias-variance decomposition in order to gain deeper insight into how local approaches work and which properties they have. We illustrate the decomposition on some toy examples. Moreover, we show results of a benchmark study where based on artificial and real-world data sets selected local and global classification methods are analyzed.
The results support our intuition that local methods exhibit lower bias compared to global counterparts and thus may improve the misclassification rate.
- G. M. James. Variance and bias for general loss functions. Machine Learning, 51(2):115-135, May 2003.