Reducing Adversarial Vulnerability through Adaptive Training Batch Size

Reducing Adversarial Vulnerability through Adaptive Training Batch Size

adversarial examples, batch normalization, fixup initialization, batch size variation

Neural networks possess an ability to generalize well to data distribution, to an extent that they are capable of fitting to a randomly labeled data. But they are also known to be extremely sensitive to adversarial examples. Batch Normalization (BatchNorm), very commonly part of deep learning architecture, has been found to increase adversarial vulnerability. Fixup Initialization (Fixup Init) has been shown as an alternative to BatchNorm, which can considerably strengthen the networks against adversarial examples. This robustness can be improved further by employing smaller batch size in training. The latter, however, comes with a tradeoff in the form of a significant increase of training time (up to ten times longer when reducing batch size from the default 128 to 8 for ResNet-56). In this paper, we propose a workaround to this problem by
starting the training with a small batch size and gradually increase it to larger ones during training. We empirically show that our proposal can still improve adversarial robustness (up to 5.73%) of ResNet-56 with Fixup Init and default batch size of 128. At the same time, our proposal keeps the training time considerably shorter (only 4 times longer, instead of 10 times).

Ken Sasongko, Adila Alfa Krisnadhi, Mohamad Ivan Fanany

http://dx.doi.org/10.21609/jiki.v14i1.907

Faculty of Computer Science Universitas Indonesia

2021-02-28

Sri Wahyuni

e-ISSN : 2502-9274 printed ISSN : 2088-7051

PDF

English

Text

Jurnal Ilmu Komputer dan Informasi (Journal of Computer Science and Information)