Recursive elimination–election algorithms for wrapper feature selection

Abstract For classification tasks in machine learning, this paper proposes a brand-new wrapper feature selection algorithm prototype named recursive elimination–election (REE), which is conceived in a simple but exquisite structure inspired by the recursion technique in computer science. Prevalent metaheuristic methods such as differential evolution (DE), particle swarm optimization (PSO), etc., from evolutionary computation (EC) and swarm intelligence (SI) communities have recently been widely applied to feature selection research, but suffer from severe drawbacks including but not limited to low-efficient binary representation transformation, poor population diversity, excessive control parameter adjustments and sophisticated mechanisms. Instead, REE is organically constructed with an ordinary subset representation of feature indexes, simple operators, getting rid of extra control parameters. Specifically, REE is assembled of two basic recursive sub-algorithms, i.e., recursive random bisection elimination (RRBE) and recursive greedy binary election (RGBE), which somewhat embody the idea of “divide-and-conquer”. By inspecting smaller and potential feature subsets in recursive ways, better subsets are returned automatically. A comprehensive experimental study was conducted on 14 UCI and ASU benchmark datasets with feature sizes ranging from dozens to thousands by using REE together with 6 state-of-the-art metaheuristic algorithms for comparison. The results show that the proposed REE has competitive search ability for feature selection problems, and it is especially prominent in handling high-dimensional datasets. Therefore, REE is promising to become a wrapper feature selection search paradigm with low solution cost and high efficiency.