Mining and Validating Social Media for COVID-19-Related Human Behaviors between January and July 2020; An Infodemiology Study (Preprint)

BACKGROUND Health authorities can minimize the impact of an emergent infectious disease outbreak through effective and timely risk communication, which can build trust and adherence to subsequent behavioral messaging. Monitoring the psychological impacts of an outbreak, as well as public adherence to such messaging is also important for minimizing long-term effects of an outbreak. OBJECTIVE We used social media data to identify human behaviors relevant to COVID-19 transmission, and the perceived impacts of COVID-19 on individuals, as a first step toward real-time monitoring of public perceptions to inform public health communications. METHODS We develop a coding schema for 6 categories and 11 subcategories, which includes both a wide number of behaviors, as well codes focused on the impacts of the pandemic (e.g., economic and mental health impacts). We use this to develop training data and develop supervised learning classifiers for classes with sufficient labels. Classifiers that perform adequately are applied to our remaining corpus and temporal and geospatial trends are assessed. We compare the classified patterns to ground truth mobility data and actual COVID-19 confirmed cases to assess the signal achieved here. RESULTS We apply our labeling schema to ~7200 tweets. The worst performing classifiers have F1 scores of only 0.18-0.28 when trying to identify tweets about monitoring symptoms and testing. Classifiers about social distancing, however, are much stronger, with F1 scores of 0.64-0.66. We applied the social distancing classifiers to over 228 million tweets. We show temporal patterns consistent with real-world events, and show correlations of up to -0.5 between social distancing signals on Twitter and ground truth mobility throughout the United States. CONCLUSIONS Behaviors discussed on Twitter are exceptionally varied. Twitter can provide useful information for parameterizing models that incorporate human behavior, as well as informing public health communication strategies by describing awareness of and compliance with suggested behaviors.