DeepPlace: Deep reinforcement learning for adaptive flow rule placement in Software-Defined IoT Networks

Abstract In this paper, we propose a novel and adaptive flow rule placement system based on deep reinforcement learning, namely DeepPlace, in Software-Defined Internet of Things (SDIoT) networks. DeepPlace can provide a fine-grained traffic analysis capability while assuring QoS of traffic flows and proactively avoiding the flow-table overflow issue in the data plane. Specifically, we first investigate the traffic forwarding process in an SDIoT network, i.e., routing and flow rule placement tasks. We design a cost function for the routing to set up traffic flow paths in the data plane. Next, we propose an adaptive flow rule placement approach to maximize the number of match-fields in a flow rule at SDN switches. To deal with the dynamics of IoT traffic flows, we model the system operation by using the Markov decision process (MDP) with a continuous action space and formulate its optimization problem. Subsequently, we develop a deep deterministic policy gradient-based algorithm to help the system obtain the optimal policy. The evaluation results demonstrate that DeepPlace can efficiently maintain a significant number of match-fields in a flow rule, i.e., approximately 86% of the maximum level, while minimizing the QoS violation ratio of traffic flows, i.e., 6.7%, in a highly dynamic traffic scenario, which outperforms three other existing solutions, i.e., FlowMan, FlowStat, and DeepMatch.