Maintenance optimisation of multicomponent systems using hierarchical coordinated reinforcement learning

Abstract The Markov decision process (MDP) is a widely used method to optimise the maintenance of multicomponent systems, which can provide a system-level maintenance action at each decision point to address various dependences among components. However, MDP suffers from the “curse of dimensionality” and can only process small-scale systems. This paper develops a hierarchical coordinated reinforcement learning (HCRL) algorithm to optimise the maintenance of large-scale multicomponent systems. Both parameters of agents and the coordination relationship among agents are designed based on system characteristics. Furthermore, the hierarchical structure of agents is established according to the structural importance measures of components. The effectiveness of the proposed HCRL algorithm is validated using two maintenance optimisation problems, one on a natural gas plant system and the other using a 12-component series system under dependant competing risks. Results show that the proposed HCRL outperforms methods in two recently published papers and other benchmark approaches including the new emerging deep reinforcement learning.