Automatic Speech-Lip Synchronization System for 3D Animation

In the 3D animation field, the quality of productions is continuously increasing. It is a very active market with a high level of competitiveness where modest companies, in terms of budget, must reach a balance between the resources they can apply and the economic investment in a given production. Consequently, the automation of manual design processes, which are normally highly time-consuming, has become a crucial research topic for 3D animation studios. The work we are describing here presents one of these automatic tools, specifically focused on the synchronization of the speech and the lip movement of the characters, a process that is called lipsync . We have developed a very robust and accurate speech recognition module that together with a knowledge-based system, autonomously provides lipsync results. Additionally, the system has been integrated in the production plan of a 3D animation company, leading to drastic operator time reductions.