Smartphone-Based Recognition Aid of Upward Staircases with People for the Visually Impaired

Many visually impaired individuals often use upward staircases to move to other floors, but it is difficult for them to find distant upward staircases. Several assistive systems have been proposed in the past, and the recent trends are smartphone-based systems. This paper described a CNN-based recognition method of upward staircases. The recognition method was a key technology for our smartphone-based assistive system. In the method, GoogLeNet models were used as CNNs. Two types of image data augmentation were used beforehand. One was the data augmentation based on the Affine transformation, and the other was the data augmentation based on the Cutout technique, where square- and human-type masks were arranged in the grid positions of images. These masks were able to emulate situations where upward staircases were partially occluded by persons. These data augmentation produced four image datasets and therefore four GoogLeNet models, which were applied to the 560 images of actual 28 environments. There were upward staircases in the 14 environments, and not in the other 14 environments. The recognition accuracy was evaluated by F-measures. When the square-type masks were used, the maximum F-measure was 0.95.