Eating and drinking gesture spotting and recognition using a novel adaptive segmentation technique and a gesture discrepancy measure

Abstract Despite the increasing developments on human activity recognition using wearable technology, there are still many open challenges in spotting and recognising sporadic gestures. As opposed to activities, which exhibit continuous behaviour, the difficulty of spotting gestures lies in their rather sparse nature. This paper proposes a novel solution to spot and recognise a set of similar eating and drinking gestures from continuous inertial data streams. First, potential segments containing an eating or a drinking gesture are found using a Crossings-based Adaptive Segmentation Technique (CAST). Second, further to the long-established range of features employed in previous human activities recognition research work, a gesture discrepancy measure is proposed to improve the classification performance of the system. At the final step, a range of state-of-the-art classification models is employed for evaluation. Various conclusions can be drawn from the results obtained. First, given the 100% recall achieved at the segmentation step, the CAST can be considered a reliable segmentation technique for spotting drinking and eating gestures which may be employed in future gesture spotting work. Second, the addition of gesture discrepancy as a feature descriptor consistently improves the classification performance of the system. Third, the reliability of the food and drink intake monitoring approach proposed in this work finds support on the out-performance of previous similar work.