Semantic multimedia-enhanced spatio-temporal queries in a crowdsourced environment

Social networks, such as Facebook, Instagram, and Twitter, provide intuitive ways to share a variety of information including geotagged multimedia data within users' communities of interest (COI) or publicly in real-time. Real-time geotagged multimedia data can provide semantics to conventional spatial queries in order to enrich the user navigation experience. In this paper, we introduce a mechanism to process spatio-temporal queries by leveraging geotagged multimedia data such as images, audio, video, and text, in order to add semantics to the conventional queries. Our framework collects, stores, and spatially tags multimedia data shared by users through social networks or through our developed mobile application. The system then uses such data in order to enhance the conventional routing services by resolving existing usability issues and by providing semantics to the routes in terms of enriched points of interest while taking dynamic road conditions into account. A proof of concept of the system will be demonstrated with the following spatio-temporal queries on road networks: 1) multimedia-enhanced shortest path queries; 2) multimedia-enhanced k-nearest neighbor queries; and 3) multimedia-enhanced range queries. Finally, a novel technique for finding lost individuals using geotagged multimedia data is also introduced. The results are tailor-made to the users' smartphone bandwidth and resolution requirements