Multimodal Machine Learning for Social Interaction with Ageing Individuals

2021 
Multimodal machine learning (MMML) is a vibrant multi-disciplinary research field which addresses some of the original goals of artificial intelligence by integrating and modelling multiple communicative modalities, including linguistic, acoustic and visual messages. With the goal of better understanding and modelling behaviours of ageing individuals, this research field brings some unique challenges for multimodal researchers given the heterogeneity of the data and the contingency often found between modalities. In this chapter, we identify four key challenges necessary to enable multimodal machine learning for ageing individuals: (1) multimodal, this modelling task includes multiple relevant modalities which need to be represented, aligned and fused; (2) high variability, this modelling problem expresses high variability given the many social contexts, large space of actions and the possible physical or cognitive impairment; (3) sparse and noisy resources, this modelling challenge addresses unreliable sensory data and the limitation and sparseness of resources that are specific for the special user group of ageing individuals; and (4) concept drift, where two types of drift were identified, namely on the group level and on the individual level (the former refers to the fact that the target group of usage is not fully known at the moment of development of according interfaces given that it is yet to age, and the latter refers to the fact that ageing may lead to drifting behaviour and interaction preferences throughout the ongoing ageing effect). These four challenges come together when we build an evaluation plan that enables, at the same time, the strategy to include the broader machine learning community in this effort. This research agenda will enable more effective and robust modelling technologies as well as development of socially competent and culture-aware embodied conversational agents for elderly care.
    • Correction
    • Source
    • Cite
    • Save
    • Machine Reading By IdeaReader
    24
    References
    0
    Citations
    NaN
    KQI
    []