User Feedback and Ranking in-a-Loop: Towards Self-Adaptive Dialogue Systems

Accurate skill retrieval is a key factor for the success of modern conversational AI agents. The major challenges lie in the ambiguity in human spoken language and the wide spectrum of candidate skills. In this paper, we make the first attempt to attack the problem by implementing a user feedback enhanced reranking strategy, and propose a self-adaptive dialogue system (AdaDial) for conversational AI agents. In AdaDial, we consider estimating user feedback and adjusting ranking strategy into a "closed-loop". In particular, we propose a scalable schema for user feedback estimation and a feedback enhanced reranking model with customized feature encoding, target attention based feature assembling, and multi-task learning. As a result, AdaDial achieves self-adaptivity at both individual- and system-levels. Online experimental results demonstrate that AdaDial could not only retrieve desired skills for different users in different scenarios, but also correct its regular strategy according to negative feedback. AdaDial has been deployed on a large-scale conversational AI agents with tens of millions daily queries, and is bringing continued positive impacts on user experience.