Shape Tracking of Flexible Morphing Matters from Depth Images

The development and use of soft or flexible structural matters across various research domains have drastically increased in recent decades. Its flexible, compliant nature and interactive safety have made it a preferred candidate compared to its rigid bodied counterparts. However, the lack of robust soft robot detection and localization techniques has constrained its feedback control system, limiting its application. This paper proposes a novel depth sensor-based detection and tracking algorithm adaptive to shape morphing robots. The detection algorithm first employs optimal iterative threshold segmentation on the depth image to remove background and detect occlusions. Blob detection and polygon approximation using Fourier descriptor techniques are then utilized to detect and extract the contours of the shape morphing soft robots. Finally, using the pixel coordinates obtained from the detection algorithm, transformation is applied from the pixel coordinate system to the world coordinate system on the depth image to achieve motion tracking in 3D space. Qualitative and quantitative assessments prove that the detection algorithm is robust and accurate in tracking shape morphing soft robots.