Double Deep Reinforcement Learning-Based Energy Management for a Parallel Hybrid Electric Vehicle with Engine Start-Stop Strategy

To optimize the fuel economy of parallel hybrid electric vehicles (HEVs), improve the working conditions of the engine, and promote the application of deep reinforcement learning (DRL) in the field of energy management strategies (EMSs), this paper firstly proposed a Deep Reinforcement Learning-based EMS combined with an engine start-stop strategy. In addition, considering both the engine and the transmission are controlled components, this paper developed a novel Double Deep Reinforcement Learning (DDRL)-based EMS, which uses Deep Q-Network (DQN) to learning the gear-shifting strategy and uses Deep Deterministic Policy Gradient (DDPG) to control the engine throttle, and the DDRL-based EMS realizes the multi-objective synchronization control by intelligent algorithms. After offline training, the simulation result of the online test shows that the fuel consumption gaps of the proposed DRL-based and DDRL-based EMS are -0.55% and 2.33% compared to that of the deterministic dynamic programming (DDP)-based EMS by overcoming some inherent flaws of DDP, respectively. Moreover, the computational efficiency is improved significantly, and the average output time per action is 0.91ms. Therefore, the control strategy that combines learning-based and rule-based control and the further multi-objective control strategies can achieve good optimization effects and application efficiency.