A mobile intelligent guide system for visually impaired pedestrian

Traditionally, tactile walking surface indicators (TWSIs) have been used as guide tools for visually impaired pedestrians, but the bumpy bricks also bring bad experiences to other users on the road, such as the elderly, people in wheelchairs, babies in strollers and ladies wearing high heels. In this paper, we propose an intelligent guide system based on a smartphone. Videos containing information about tactile paving captured by the phone camera are processed and analyzed by the system, and guide messages are sent to the user in the form of sound or vibration. In our system, the MobileNet model fine-tuned by transfer learning is used to perform feature extraction on overlapping grids. Single Shot MultiBox Detector (SSD) is then used for TWSI detection. Finally, the user’s position is determined by the Score Voting algorithm, and corresponding guide information is given. In order to further improve the real-time performance of the system, we quantize the model to compress it while ensuring accuracy. The results of experiments on real tactile paving show that our system has high accuracy and real-time performance. With our system, bumpy tactile paving bricks can be replaced with flat stickers or paint with TWSI patterns. This is comforting for other road users, and it will be easy to set up and keep up. Moreover, the types of patterns can be extended for further applications.