Deep Touch: Sensing Press Gestures from Touch Image Sequences

Capacitive touch sensors capture a sequence of images of a finger’s interaction with a surface that contain information about its contact shape, posture, and biomechanical structure. These images are typically reduced to two-dimensional points, with the remaining data discarded—restricting the expressivity that can be captured to discriminate a user’s touch intent. We develop a deep touch hypothesis that (1) the human finger performs richer expressions on a touch surface than simple pointing; (2) such expressions are manifested in touch sensor image sequences due to finger-surface biomechanics; and (3) modern neural networks are capable of discriminating touch gestures using these sequences. In particular, a press gesture based on an increase in a finger’s force can be sensed without additional hardware, and reliably discriminated from other common expressions. This work demonstrates that combining capacitive touch sensing with modern neural network algorithms is a practical direction to improve the usability and expressivity of touch-based user interfaces.