HWOA: A hybrid whale optimization algorithm with a novel local minima avoidance method for multi-level thresholding color image segmentation

Abstract Traditional methods to address color image segmentation work efficiently for bi-level thresholding. However, for multi-level thresholding, traditional methods suffer from time complexity that increases exponentially with the increasing number of threshold levels. To overcome this problem, in this paper, a new approach is proposed to tackle multi-threshold color image segmentation by employing the Otsu method as an objective function. This approach is based on a hybrid of the whale optimization algorithm (WOA) with a novel method called the local minima avoidance method (LMAM), abbreviated as HWOA. LMAM avoids local minima by updating the whale either within the search space of the problem or between two whales selected randomly from the population-based on a certain probability. HWOA is validated on ten color images taken from the Berkeley University Dataset by measuring the objective values, peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), features similarity index (FSIM), and CPU time, and compared with a number of the well-known robust meta-heuristic algorithms: the sine–cosine algorithm (SCA), WOA, modified salp swarm algorithm (MSSA), improved marine predators algorithm (IMPA), modified Cuckoo Search (CS) using McCulloch’s algorithm (CSMC), and equilibrium optimizer (EO). The experimental results show that HWOA is superior to all the other algorithms in terms of PSNR, FSIM, and objective values, and is competitive in terms of SSIM.