Timed Coloured Petri Net Simulation Model for Reinforcement Learning in the Context of Production Systems

This paper shows that Petri net simulation models are a suitable basis for using Reinforcement Learning to generate a supervisory control unit in industrial production systems. By modelling control unit functions as transitions and state information as places and tokens, a timed coloured Petri net model describing a material flow system can be constructed. The transitions form the action space, while the places and tokens correspond to the observation space of the Reinforcement Learning agent. To confirm the suitability of the timed coloured Petri net model a case study of a simple production facility was conducted. Existing python packages were expanded to provide additional timed Petri net functionality. Then a Q-Learning agent was trained on the Petri net simulation model to perform a simple task. The case study showed that Petri nets provide a suitable model type for training a Reinforcement Learning algorithm and are capable of modelling all relevant components of a material flow system.