Working Paper: ‘It’s Written All over Your Face’—Applying Neural Networks to Explain Consumer Behavior and Characteristics Based on Facial Information

Facial images of consumers, collected on web-based platforms that are typified by a high degree of personal exchange (e.g., social networks) or service-dominated interaction (e.g., financial advisory services), constitute a capable resource to explain consumer matching (e.g., network formation) or characteristics (e.g., demographics). Consumer researchers and marketing practitioners alike, however, lack an inclusive understanding on how facial images can be analyzed and utilized to derive conclusive insights on consumer matching and characteristics. In light of technological progress in computer vision, material requirements in consumer research, and projected implications for marketing practice, this research is inspired by the following two objectives. First, this research aims to examine consumer matching based on facial information. Second, this research aims to investigate the predictability of consumer characteristics based on facial information. Thereby, this research benefits from exclusive high-dimensional interaction, transaction, profile, and image data of approx. 640,000 users of a social network. From a theoretical perspective, this research provides the theoretical antecedents and implementation provisions for the computational processing and analysis of facial images in consumer research, fortifies research on consumer matching, and, finally, advances research on the predictability of consumer characteristics based on facial information. From a practical perspective, this research assists both business strategists and software developers in improving approaches and implementation provisions for consumer matching and the predictability of consumer characteristics as well as in developing prospective use cases with regard to the practical implementation. Finally, this research is anticipated to inspire a future societal dialogue on the utilization of private data in computer-mediated environments and to inform the general public on the potential applications of facial data in various spheres of human habitat.