Distributed Volt-VAR Optimization based on Multi-Agent Deep Reinforcement Learning

In this paper, we propose a multi-agent deep reinforcement learning (DRL) based approach to solve the distributed Volt-VAR optimization (VVO) problem in distribution networks by considering the uncertainty of system load and renewable power generation. We formulate the distributed VVO problem as a non-cooperative Markov game. Specifically, we split the distribution network into multiple regions that are controlled by a group of networked agents. We consider the statuses/ratios of switchable capacitor banks (SCBs), the tap position of voltage regulators (VRs), and the reactive power of inverter-based distributed generators (DGs) as the control variables. The objective is to minimize the total system loss while maintaining bus voltages within a normal operating range. To solve the problem, a multi-agent trust region policy optimization (MATRPO) based approach is applied to learn a set of decentralized policies. Simulation results on a modified IEEE-34 test system show that the proposed approach can successfully learn a set of high-quality decentralized policies for the agents to collaboratively reduce the system loss and regulate the bus voltages. The simulation results also demonstrate the superiority of the proposed approach over independent learning and the MADDPG method.