XiaoIce Band: A Melody And Arrangement Generation Framework For Pop Music

Authors:
Hongyuan Zhu USTC
Qi Liu USTC
Nicholas Jing Yuan Microsoft
Chuan Qin USTC
Jiawei Li Soochow University
Kun Zhang USTC
Guang Zhou Microsoft
Furu Wei Microsoft
Yuanchun Xu Microsoft
Enhong Chen USTC

Introduction:

The authors present a focused study on pop music generation. They propose an end-to-end melody and arrangement generation framework, called XiaoIce Band, which generates a melody track with several accompany tracks played by several types of instruments. They also propose a Multi-Instrument Co-Arrangement Model (MICA) using multi-task learning for multi-track music arrangement.

Abstract:

With the development of knowledge of music composition and the recent increase in demand, an increasing number of companies and research institutes have begun to study the automatic generation of music. However, previous models have limitations when applying to song generation, which requires both the melody and arrangement. Besides, many critical factors related to the quality of a song such as chord progression and rhythm patterns are not well addressed. In particular, the problem of how to ensure the harmony of multi-track music is still underexplored. To this end, we present a focused study on pop music generation, in which we take both chord and rhythm influence of melody generation and the harmony of music arrangement into consideration. We propose an end-to-end melody and arrangement generation framework, called XiaoIce Band, which generates a melody track with several accompany tracks played by several types of instruments. Specifically, we devise a Chord based Rhythm and Melody Cross-Generation Model (CRMCG) to generate melody with chord progressions. Then, we propose a Multi-Instrument Co-Arrangement Model (MICA) using multi-task learning for multi-track music arrangement. Finally, we conduct extensive experiments on a real-world dataset, where the results demonstrate the effectiveness of XiaoIce Band.

You may want to know: