Locally alignment based manifold learning for simultaneous feature selection and extraction in classification problems

Dimensionality reduction is an important step in increasing the performance of machine learning algorithms while decreasing the processing time. From feature reduction approaches, feature extraction is aimed to achieve better data representation while feature selection is meant to discard redundant features. The main contribution of this manuscript is to propose a hybrid feature selection and extraction approach which performs these tasks in a unified framework, simultaneously. Also, the proposed approach aims to maintain the manifold of data using locally alignment constrains. The main goal of these suggestions is to discriminate different classes while reducing the redundancy of the samples specially for the imbalanced data classification problems. The proposed approach optimizes a hybrid objective function that tries to both increase the between class discrimination and decrease the within class distribution while minimizing the information loss. Also, we have embedded a weighting factor into the objective function to achieve a basis for feature importance measurement and feature selection. To evaluate the proposed approach, a multiclass linear SVM classifier is applied on the reduced data in a k-fold cross-validation scheme, and the accuracy, as well as F-score, is used as the performance measure. Comparisons of the proposed method with some recent approaches on 12 datasets of UCI, show the superiority of the proposed method over previously proposed approaches. Also, Friedman and Nemenyi tests are applied which show significant improvement in the proposed approach.