A real-time silent speech system for voice restoration after total laryngectomy

Abstract Background and aim Individuals who have lost their voice following a laryngectomy as a treatment for cancer will inevitably struggle with their daily communication. Unfortunately, the current methods for speaking after laryngectomy all have limitations, either because of the poor acoustics generated by these methods or because they are potentially harmful. The aim of this work is thus to explore an alternative method for post-laryngectomy voice restoration in which the movement of the intact articulators is captured and then converted into audible speech using machine learning techniques. Materials and methods To demonstrate the feasibility of speech generation from captured articulator movement, 6 healthy adults were recruited. For each subject, both the speech acoustics and the subject's articulator movements were recorded simultaneously. Articulator movements were captured using a technique known as permanent magnet articulography (PMA), in which small magnets are attached to the articulators (typically tongue and lips) and the magnetic field generated by the magnets is captured with sensors located close to the mouth. Deep artificial neural networks were then used to model the mapping between the sensor data and the speech acoustics, thus, enabling the synthesis of speech from captured articulatory data. Results The proposed silent speech system is able to generate speech that sounds natural, resembles the subject's own voice and is fairly intelligible (up to 92% intelligibility for some speakers on a phonetically-rich corpus). Conclusions With further research, the proposed system could in future be a real option to restore lost voice after laryngectomy.