Generalisation Power Analysis for finding a stable set of features using evolutionary computation feature selection algorithms

Evolutionary Computations (EC) are powerful techniques for feature selection tasks however, they reach different solutions in each run, and this is known as the stability issue. Existing solutions to finding a stable subset of features when using an EC algorithm include aggregation and frequency-based methods. These methods may return feature subsets that achieve weak or inconsistent classification performance when utilised to build classifiers, and this limitation is known as ‘lack of generalisation power’. To address this limitation, this paper proposes a novel algorithm called Generalisation Power Analysis (GPA) that measures the performance of feature subsets in terms of generalisation power and hence evaluates their ability to achieve optimal or near-optimal accuracy over multiple classifiers. GPA has been designed to work with the stochastic nature of EC algorithms. Experiments with eleven benchmark datasets revealed that the proposed GPA approach consistently outperformed alternative methods in finding subsets that achieved high generalisation power. Although GPA requires relatively higher computation time compared to alternative approaches as it embeds multiple classifiers, the advantages of using GPA during feature selection outweigh this limitation since the outcome will be a robust prediction model that has been developed using a subset of features that are not biased towards a specific classifier.