Prioritized Replay Dueling DDQN Based Grid-Edge Control of Community Energy Storage System

This paper develops a new prioritized replay dueling DDQN (PRD-DDQN) method for grid-edge control of community energy storage system with good robustness to uncertainties and fast convergence speed while achieving good control performance. The control problem is first formulated as a Markov decision process (MDP) considering the current time interval, state of charge of the CESS, the price signals, and the pre-trading power between MG and CESS/utility grid. Unlike the double deep Q-network-based method to solve this MDP, the proposed PRD-DDQN endows the agent with a powerful capability of learning by interacting with a more complex environment. This is due to the collaboration of the dueling structure and prioritized replay policy based on the sum-tree. As a result, the control accuracy, model robustness, and algorithm convergence speed are significantly enhanced. Besides, the proposed algorithm supports minute-level and multi-agent parallel control. Comparison results with a deterministic model-based method and other deep reinforcement learning-based methods demonstrate the effectiveness and superiority of the proposed approach.