Multiple users' emotion recognition: Improving performance by joint modeling of affective reactions

This paper studies emotion recognition in the context of collaboration. When people are interacting with each other they tend to reach a similar emotional state through mechanisms like empathy and emotion contagion. We thus investigated if participants' emotions could be determined from the affective reactions and behaviors of their partner. Two types of emotional expressions were studied: physiological reactions and speech. Results show that emotions could be recognized with similar performance when employing affective features from the self or the partner. In addition, performance was improved when combining self and partner information. The results demonstrate that in social situations an emotion recognition model should include information about partners.