Using machine learning to evaluate the fidelity of heavy equipment acoustic simulations

Abstract Audio quality is an important consideration when creating acoustic simulations. However, there has been a longstanding tradeoff in evaluating the quality or fidelity of sound: subjective listening tests are time consuming and expensive, but objective measures often fail to capture the nuances of human perception. The research presented here seeks to address this problem by investigating the use of machine learning to evaluate the fidelity of heavy equipment acoustic simulations. The developed models are designed to classify sounds into one of two categories, based on whether the audio is “natural” sounding or “artificial” sounding. Two distinct datasets are presented. The first, made up of a library of compressed recordings of heavy equipment, is used primarily for developmental purposes. The second, made up of a library of simulated audio clips, is used to test performance for the intended purpose of evaluating simulated audio fidelity. Several common algorithms are compared and various audio features considered in developing the machine learning models. The final model consists of a logistic regression algorithm and uses the input features loudness, sharpness, roughness, fluctuation strength, and Mel-frequency cepstral coefficients. The developed models accurately predict human perceptions of audio fidelity, achieving approximately 98% accuracy for both datasets. The accuracies achieved provide evidence that machine learning models could potentially supplant listening tests, although limitations including the scope of the dataset and the small number of listening test participants necessitate further validation.