Neural Architecture Search Based on Tabu Search and Evolutionary Algorithm

Most existing optimization methods for neural architecture search (NAS), including evolutionary algorithms, reinforcement learning and gradient-based approaches, have not employed memory strategies explicitly, which may lack of efficiency when searching neural architectures. To solve this issue, we propose a new NAS approach by using an evolutionary algorithm which employs a tabu mechanism to help to improve the search efficiency. To be more specific, the individuals of parent population are selected by tournament selection and tabu list. The tournament selection select parent population according to the accuracy of each individual. And the tabu mechanism builds a tabu list to record the chosen operations in the last previous search process, which employs a search memory mechanism to improve the efficiency explicitly. To confirm the superior performance of our approach, a well-designed surrogate model is used to accelerate the process of performance evaluation on CIFAR-10. The comprehensive experimental results show that the proposed method can reach to 2.48% error rate with about 2 GPU days, which demonstrates the superiority of the suggested method.