Design and Implementation of Information Dissemination Simulation Algorithm in Large-Scale Complex Network Based on Spark

Due to the rapid development of the Internet and online social media, information in social networks spreads fast and it is difficult to control. To discover the laws of information dissemination in social networks, it requires a large number of propagation computing experiments in complex networks. The network propagation experiment based on the SIR model is widely used in disease spreading and information dissemination research. Currently, it is difficult to conduct ultra-large-scale network propagation calculations due to hardware, software and some other reasons. However, the information dissemination on Internet has the characteristics of large scale, large amount of information and fast speed of transmission. Smaller-scale network communication experiments were designed based on the abstraction method, but they gradually failed to meet the growing demand for computing. This paper uses Spark to implement a very large-scale network propagation computational experimental algorithm. The performance of the algorithm and the Nepidemix single-core propagation computing component are compared to show its advantages and disadvantages. The algorithm can carry out experiments of information dissemination with millions of nodes if there are enough cluster computing resources. And it is easy to get started with development. Based on this, it can efficiently conduct information dissemination simulation in large-scale complex networks. It laid the foundation for other studies such as the dissemination, monitoring, forecasting and intervention of public opinion.