Detecting Social Signals in User-shared Images for Connection Discovery using Deep Learning

With the advance of mobile technology and social media, image sharing has become part of our daily lives. For many applications, such as follower/followee recommendation, shared images are an excellent source to discover connections among users who shared them. Shared images on social media are like invitations for user interactions, such as comment, like, and more. Social signals are in those images, and those signals can be objects that interest related users such that they will start to interact. Conventionally, connections among users are discovered through recognizing objects among those shared images, such as a using convolutional neural network (CNN) to extract features that are sensitive to object recognition. However, social signals are not limited to object, they can be colour, textual, or even a concept that may not be captured effectively by conventional CNN. This paper proposes a CNN-based analytic framework to detect social signals among users. The CNN is optimized using a triplet network with user-shared images, and the relationships among users who upload them. It is observed that images from 2 users with a connection have a shorter distance after encoding, than 2 users without a connection. A framework is implemented, which is verified with over 1.7 million images by over 2000 users from two image-oriented social networks, Skyrock and Flickr. It is proven that the proposed analytic framework shows an up to 89% improvement on approaches using object recognition for follower/followee recommendation. To the best of our knowledge, this paper is the first to propose an analytic framework to detect social signals from visual features for connection discovery.