Covariate Shift Adaptation for Adversarially Robust Classifier

We show that adaptive batch normalization (BN) technique that involves re-estimating the BN parameters during inference, can significantly improve the robustness of adversarially trained models for any random perturbations, including the Gaussian noise. This simple finding enables us to transform an adversarially trained model into a randomized smoothing classifier to provide certified robustness for l2 norm. Moreover, we achieve l2 certified robustness even for adversarially trained models, learned using l∞-bounded adversaries. Further, adaptive BN significantly improves robustness against common corruptions, without any detrimental effect on their performance against adversarial attacks. This enables us to achieve both adversarial and corruption robustness using the same classifier.