LbR: A New Regression Architecture for Automated Feature Engineering

In recent years, machine learning has developed rapidly and has been widely applied in many fields, such as finance and medical treatment. Many studies have shown that feature engineering is the most important part of machine learning and the most creative part of data science. However, in the traditional feature engineering step, it often requires the participation of experienced domain experts and is very time-consuming. Therefore, automatic feature engineering technology arises, aiming at improving the performance of the model by automatically generating high informative features without expert domain knowledge. However, in these methods, new features are generated by pre-defining a set of identical operators on datasets, ignoring the diversity of data sets. So there is room for improvement in performance. In this paper, we proposed a method named LbR (Label based Regression), which can fully mine correlations between feature pairs and then select feature pairs with high discrimination to generate informative features. We conducted many experiments to show that LbR has better performance and efficiency than other methods in different data sets and machine learning models.