Exploiting diversity for optimizing margin distribution in ensemble learning
journal contributionposted on 16.06.2017, 11:08 by Qinghua Hu, Leijun Li, Xiangqian Wu, Gerald Schaefer, Daren Yu
Margin distribution is acknowledged as an important factor for improving the generalization performance of classifiers. In this paper, we propose a novel ensemble learning algorithm named Double Rotation Margin Forest (DRMF), that aims to improve the margin distribution of the combined system over the training set. We utilise random rotation to produce diverse base classifiers, and optimize the margin distribution to exploit the diversity for producing an optimal ensemble. We demonstrate that diverse base classifiers are beneficial in deriving large-margin ensembles, and that therefore our proposed technique will lead to good generalization performance. We examine our method on an extensive set of benchmark classification tasks. The experimental results confirm that DRMF outperforms other classical ensemble algorithms such as Bagging, AdaBoostM1 and Rotation Forest. The success of DRMF is explained from the viewpoints of margin distribution and diversity.
This work is supported by the National Program on Key Basic Research Project under Grant 2013CB329304, National Natural Science Foundation of China under Grants 61222210, 61170107, 61073125, 61350004 and 11078010, the Program for New Century Excellent Talents in University (No. NCET-12-0399), and the Fundamental Research Funds for the Central Universities (Grant No. HIT.NSRIF.2013091 and HIT.HSS.201407).
- Computer Science