Using generative adversarial networks to evaluate robustness of reinforcement learning agents against uncertainties

Abstract This paper describes the process of creating uncertainty-infused synthetic profiles of building performance. The synthetic profiles are utilized as a resource for evaluating the response of trained machine learning models to unseen events. Applications of the introduced method can benefit researchers and practitioners who train data-driven building models on limited historical data and is particularly useful when a physics-based model of the building is unavailable. As an original contribution, we propose a conditional deep convolutional Generative Adversarial Network (GAN) for projecting multi-dimensional time-series profiles of building performance. The proposed GAN reflects climate and operation variations into the synthetic building performance profiles, while preserving the internal consistency within the generated data. To ensure high quality synthetic profiles, this study validates the plausibility of generated data through qualitative (visualization) and quantitative (Pearson correlation, Wasserstein distance) assessments. Synthetic profiles are fed to a trained reinforcement learning model and a rule-based controller to compare their performances in the presence of uncertainty. Results show that with limited training data, a reinforcement learning model's response can be fairly sensitive to uncertainties and disturbances, insofar, some advantages over rule-based controllers may be overestimated. To ensure the reproducibility of the presented results, this study is conducted on open data and models are shared as open source.