A Cognitive Unsupervised Clustering for Detecting Cyber Attacks

It has always been a challenge to make meaning out of unstructured data. In the field of network intrusion detection, the availability of structured, labeled datasets is limited. The need of unsupervised learning from unlabeled datasets is of vital significance, yet there is little breakthrough achieved in the research community. Most approaches adhere to techniques that are over-exhaustive in terms of resources and do not yield satisfactory results; hence, human analysts must re-examine all the events for intrusion attempts. This study makes an effort to find an approach of making sense out of unstructured, unlabeled data, in a way that helps the human analysts to disregard a major portion of the network dataset that contains regular traffic and isolates the finite time-windows that have been subjected to potential attacks, utilizing the concepts of cognitive science, complexity analysis, and statistical higher-order feature learning. In this research, use statistical higher-order features from network flows to classify the network traffic into flows containing normal traffic and flows subject to attacks, using unsupervised k-means clustering and variance fractal dimension trajectory-based complexity analysis. We validate our algorithm on the UNSW dataset and compared our results with traditional unsupervised clustering. The proposed model was able to detect errors with the accuracy of 87.27%.