Algorithm for detecting anomalous hosts based on group activity evolution

Abstract Network behavior analysis is an active and challenging research direction in anomalous network traffic detection. With rapidly-increasing popularity of online applications, network traffic volume has grown exponentially. In the meantime, network communication is becoming more complex. Consequently, new interaction patterns of network behaviors, named group activity, need to be investigated. Group activities can be generated by various hosts over the network and changes in group activities usually cannot be captured by traditional anomaly detection methods. Currently, the primary limitation of traditional flow-based methods for network traffic analysis is the lack of a mechanism that studies the social relationship in the interaction patterns between hosts. Besides, existing approaches fail to detect group activities. To overcome these limitations, the paper aims to detect anomalous hosts based on the profile of group activity evolution. We propose a mathematical method to quantify and detect anomalous group activities based on the stability of group activity evolution, which fully captures both the temporal and structural characteristics of network evolution. This anomaly detection algorithm is called GAP (Group Activity Profile), which is a powerful supplement and extension of the traditional method of network behavior analysis. The main contributions of this paper are: (1) the paper introduces a new perspective of the group activity based on network evolution and network behavior anomaly detection; (2) the algorithm is suitable for measuring the changes in group activity evolution and determining whether the current evolution relatively conforms to or deviates from the normal evolution; and (3) this work defines the baseline of the group activity evolution by applying historical characteristics that can significantly reduce the false positive rate, and detect anomalous hosts accurately. The results of experiments conducted on real datasets demonstrate that GAP is capable of detecting anomalous host more effectively than traditional methods. GAP is free of parameters and achieves high scalability, which can effectively identify group activities as well as accurately detect anomalous hosts over time.