Base Model Combination Algorithm for Resolving Tied Predictions for K-Nearest Neighbor OVA Ensemble Models

Model aggregation is the process of constructing several base models that are then combined into a single model for prediction. Ensemble classification has been studied by many researchers and found to provide significant performance improvements over single models. This paper presents a new base model combination algorithm for K-nearest neighbor (KNN) ensemble models based on One-Versus-All (OVA) classification. The proposed algorithm uses two decision functions to determine the best prediction among the many predictions provided by the base models. It is demonstrated in this paper that tied or conflicting predictions can be effectively resolved when a probabilistic function and a distance function are used by a combination algorithm for OVA KNN base model predictions. The resolution of tied predictions leads to improvements in predictive performance.