Stochastic Maintenance Schedules of Active Distribution Networks based on Monte-Carlo Tree search

The integration of volatile distributed energy resources (DERs) brings new challenges for the active distribution network maintenance scheduling (DN-MS). Conventionally, the DN-MS is formulated as a deterministic optimization model without considering the uncertainties of DERs. In this paper, the DN-MS is formulated as a multistage stochastic optimization problem, which is cast as a stochastic mixed-integer nonlinear programming model. It aims to reduce the total maintenance cost constrained by the reliability indices. To capture the operational characteristics of active distribution networks, the uncertainties of DERs and post-outage operation strategies of switching devices are incorporated into the model. In general, this type of model is intractable and mainly solved by heuristic search methods with low efficiency. Recently, Monte-Carlo tree search (MCTS) is emerging as a scalable and promising reinforcement learning approach. We propose a stochastic MCTS solution to this problem. In the tree search procedure, a sample average approximation technique is developed to estimate multistage maintenance costs considering uncertainties. To speed up the MCTS, the complicated constraints of the original problem are transformed to penalty or heuristics functions. This approach can asymptotically approximate the optimum with promising computation efficiency. Numerical test results demonstrate the superiority of the proposed method over benchmark methods.