PROJECT TITLE :

Global Negative Correlation Learning A Unified Framework for Global Optimization of Ensemble Models

ABSTRACT:

The field of Machine Learning makes extensive use of ensembles as an approach, and the diversity that exists within ensembles is typically cited as the primary factor in the success of these approaches. The majority of these methods either involve data sampling or involve modifying the structure of the constituent models in order to encourage diversity within the ensemble. Despite this, there is a family of ensemble models in which diversity is explicitly promoted in the error function of the individuals. In these models, the individuals make up the ensemble. Within this group of techniques, the negative correlation learning (NCL) ensemble framework is most likely the most well-known algorithm. This article performs an analysis of NCL and finds that the framework actually minimizes the combination of errors made by the individuals that make up the ensemble rather than attempting to minimize the residuals made by the ensemble as a whole. We propose a new ensemble framework and give it the name global negative correlation learning (GNCL). This framework places the emphasis not on the individual fitness of the components of the ensemble but rather on the optimization of the global ensemble as a whole. Under the assumption of fixed basis functions, an analytical solution for the parameters of base regressors based on the NCL framework and the proposed global error function is also provided (although the general framework could also be instantiated for Neural Networks with nonfixed basis functions). Extensive testing with different types of data sets, including regression and classification, are used to evaluate the proposed ensemble framework. It has been demonstrated, through comparisons with other cutting-edge ensemble methods, that GNCL produces the highest overall level of performance.