1 Introduction
Deep neural networks are shown to be vulnerable to adversarial attacks, where the natural data is perturbed with humanimperceptible, carefully crafted noises [9, 15, 30]. To mitigate this pitfall, extensive efforts have been devoted to adversarial defense mechanisms, where the main focus has been on specialized adversarial learning algorithms [9, 19], loss/regularization functions [13, 39], as well as image preprocessing [32, 26, 33, 36]. Yet, there is an orthogonal dimension that few studies have explored: the intrinsic influence of network architecture on network resilience to adversarial perturbations. Although the importance of architectures in adversarial robustness has emerged in the experiments of several previous work [28, 34, 19, 7], more comprehensive study on the role of network architectures in robustness remains needed.
In this work, we take the first step to systematically understand adversarial robustness of neural networks from an architectural perspective. Specifically, we aim to answer the following questions:

[leftmargin=*]

What kind of network architecture patterns is crucial for adversarial robustness?

Given a budget of model capacity, how to allocate the parameters of the architecture to efficiently improve the network robustness?

What is the statistical indicator for robust network architectures?
It is nontrivial to answer the above questions, since we need to train a massive number of networks with different architectures and evaluate their robustness to gain insights, which, however, is exceedingly timeconsuming, especially when adversarial training is used. Thanks to the method of oneshot neural architecture search (NAS), it becomes more accessible to evaluate robustness among a large number of architectures. Specifically, we first train a supernet for once, which subsumes a wide range of architectures as subnetworks, such as ResNet [11] and DenseNet [12]
. Then we sample architectures from the supernet and finetune the candidate architectures for a few epoches to obtain their robust accuracy under adversarial attacks. We further conduct extensive analysis on the obtained architectures and have gained a number of insights to the above questions:
1) We present a statistical analysis on architectures from our cellbased search space, and discover a strong correlation between the density of the architecture and the adversarial accuracy. This indicates that densely connected pattern can significantly improve the network robustness.
2) We restrict the number of parameters under three different computational budgets, namely small, medium, and large. Our experimental results suggest that adding convolution operations to direct edges is more effective to improve model robustness, especially for small computational budgets.
3) We further release the cellbased constraint and produce studies on cellfree search space. For this setting, we find that the distance of flow of solution procedure matrix between clean data and adversarial data can be a good indicator of network robustness.
By adopting these observations, we search and design a family of robust architectures, called RobNets. Extensive experiments on popular benchmarks, including CIFAR [14], SVHN [21], and TinyImageNet [16], indicate that RobNets achieve a remarkable performance over widely used architectures. Our studies advocate that future work on network robustness could concentrate more on the intrinsic effect of network architectures.
2 Related Work
Adversarial Attack and Defence. Deep neural networks (NNs) can be easily fooled by adversarial examples [9, 30], where effective attacks are proposed such as FGSM [9], BIM [15], C&W [5], DeepFool [20], MIFGSM [8], and PGD [19]. Extensive efforts have been proposed to enhance the robustness of NNs, including preprocessing techniques [32, 4, 26, 36], feature denoising [33], regularization [39, 13], adding unlabeled data [6, 27], model ensemble [31, 22], where adversarial training [9, 19] turns out to be the most effective and standard method for improving robustness. A few empirical attempts on robustness of existing network architectures have been made [28, 34, 7], but no convincing guidelines or conclusions have yet been achieved.
Neural Architecture Search.
Neural architecture search (NAS) aims to automatically design network architectures to replace conventional handcrafted ones. Representative techniques include reinforcement learning
[41, 42, 40, 10, 2], evolution [25, 29] and surrogate model [17], which have been widely adopted to the search process. However, these methods usually incur very high computational cost. Other efforts [18, 24, 3] utilize the weight sharing mechanism to reduce the costs of evaluating each searched candidate architecture. Towards a fast and scalable search algorithm, our investigation here is based on the oneshot NAS [3].3 Robust Neural Architecture Search
3.1 Preliminary
In this section, we briefly introduce the concept of oneshot NAS and adversarial training for the ease of better understanding of our further analysis.
OneShot NAS. The primary goal of NAS [42, 17, 18, 3] is to search for computation cells and use them as the basic building unit to construct a whole network. The architecture of each cell is a combination of operations chosen from a predefined operation space. In oneshot NAS, we construct a supernet to contain every possible architecture in the search space. We only need to train the supernet for once and then at evaluation time, we can obtain various architectures by selectively zero out the operations in the supernet. The weights of these architectures are directly inherited from the supernet, suggesting that weights are shared across models. The evaluated performance of these oneshot trained networks can be used to rank architectures in the search space, since there is a nearmonotonic correlation between oneshot trained and standalone trained network accuracies. We refer readers to [3] for more details of this order preservation property.
In oneshot NAS, the oneshot trained networks are typically only used to rank architectures and the bestperforming architecture is retrained from scratch after the search. In our work, however, we do not aim to get one single architecture with the highest robust accuracy, but to study the effect of different architectures in network robustness. Thus, we do not involve retraining stage in our method but fully utilize the property of accuracy order preservation in oneshot NAS.
Robustness to Adversarial Examples. Network robustness refers to how network is resistant to adversarial inputs. The problem of defending against bounded adversarial perturbations can be formulated as follows:
(1) 
where defines the set of allowed perturbed inputs within distance, denotes the model and denotes the data distribution. One promising way to improve network robustness is adversarial training. [19] proposed to use Projected Gradient Descent (PGD) to generate adversarial examples and augment data during training, which shows significant improvements of network robustness. In our study, we focus on adversarial attacks bounded by norm and use PGD adversarial training to obtain robust networks of different architectures.
3.2 Robust Architecture Search Framework
We now describe the core components of our robust architecture search framework. Our work is based on conventional oneshot architecture search methods [3, 18], with certain modifications to facilitate adversarial training and further analysis. We introduce them in detail accordingly.
Search Space. Following [42, 17, 18, 3], we search for computation cell as the basic building unit to construct the whole network architecture. Each cell is represented as a directed acyclic graph consisting of nodes. Each node corresponds to a intermediate feature map . Each edge represents a transformation chosen from a predefined operation pool containing candidate operations (see Fig. 2(a)). The intermediate node is computed based on all of its predecessors: . The overall inputs of the cell are the outputs of previous two cells and the output of the cell is obtained by applying concatenation to all the intermediate nodes. For the ease of notation, we introduce architecture parameter to represent candidate architectures in the search space. Each architecture corresponds to a specific architecture parameter . For edge of an architecture, the transformation can then be represented as . We refer direct edges to those and refer skip edges to those .
The main differences in search space between our work and conventional NAS lie in two aspects: 1) We shrink the total number of candidate operations in , remaining only: separable convolution, identity, and zero. This helps to lift the burden of adversarial training, while remaining powerful candidate architectures in the search space [35]. 2) We do not restrict the maximal number of operations between two intermediate nodes to be one (i.e., could contain at most operations). As shown in Fig. 2, such design encourages us to explore a larger space with more variants of network architectures, where some classical humandesigned architectures can emerge such as ResNet and DenseNet.
Robust Search Algorithm. We develop our robust search algorithm based on the oneshot NAS approach [3]. Specifically, we set all the elements in architecture parameter as 1 to obtain a supernet containing all possible architectures. During the training phase of the supernet, for each batch of training data, we randomly sample a candidate architecture from the supernet (by arbitrary setting some of the elements in to 0). This path dropout technique is incorporated to decouple the coadaptation of candidate architectures [3]. We then employ the minmax formulation in Eq. (1) to generate adversarial examples with respect to the sampled subnetwork, and perform adversarial training to minimize the adversarial loss. Such mechanism ensures adversarial examples generated during training are not specific to one architecture. We also provide the pseudo code of our robust search algorithm in the Appendix.
Robustness Evaluation. Once obtaining the supernet after robust training, we can collect candidate architectures by random sampling from the supernet and inheriting weights. Rather than direct evaluating the network on validation dataset as vanilla NAS methods, we find that finetuning the sampled network with adversarial training for only a few epochs can significantly improve the performance, which is also observed in [1]. The intuition behind finetuning is that while the training scheduler tries to inflate the robustness of each architecture, it yet needs to maintain the overall performance of all candidate architectures. The adversarial accuracy before and after finetuning for 1,000 randomly sampled candidate architectures is illustrated in Fig. 3. It can be clearly seen that the robustness performance has been largely improved.
After finetuning, we evaluate each candidate architecture on validation samples that are adversarially perturbed by the whitebox PGD adversary. We regard the adversarial accuracy as the indicator of the network robustness.
3.3 Analysis of CellBased Architectures
Having set up the robust search framework, we would like to seek for answers for the first question raised in Sec. 3.1, that what kind of architecture patterns is crucial for adversarial robustness. We first conduct analysis of model robustness for cellbased architectures by following a typical setting in NAS methods [42, 18], where the architectures between different cells are shared.
Statistical Results. In cellbased setting, we adopt robust architecture search on CIFAR10. We set the number of intermediate nodes for each cell as . Recall that we have 2 nonzero operations and 2 input nodes, so the total number of edges in the search space is 14. This results in a search space with the total complexity , where each architecture parameter is . For the training of the supernet, we choose 7step PGD adversarial training with step size. After training the supernet, we randomly sample 1,000 architectures from the supernet and finetune each of them for 3 epochs.
is separable between robust and nonrobust networks, which demonstrates the architecture has an influence on network robustness. (b) Values of weights of the trained linear classifier. We observe that almost all of the weight values are positive, indicating that there is a strong correlation between architecture density and adversarial accuracy.
We plot the histogram of adversarial accuracy of these 1,000 architectures in Fig. 3. As the figure shows, although most of the architectures achieve relatively high robustness (with % robust accuracy), there also exist a large number of architectures suffering from poor robustness (far lower from the average 50.3%). This motivates us to consider whether there exist some shared features among the robust networks.
To better visualize how distinguishable the architectures are, we first sort the 1,000 architectures with respect to the robust accuracy. Next, top 300 architectures are selected with a label of and last 300 architectures with label of . Finally, tSNE helps us to depict the corresponding to each architecture. We visualize the lowdimensional embedding of 600 in Fig. 4. As shown, the embedding of architecture parameter is separable between robust and nonrobust networks, which clearly demonstrates that architecture has an influence on network robustness.
This finding naturally raises a question: Which paths are crucial to network robustness in architectures? A straightforward idea is that we train a classifier which takes the architecture parameter as input and predicts whether the architecture is robust to adversarial attacks. In this case, the weights that correspond to crucial paths are expected to have larger values. We use the 600 architectures introduced above and their corresponding labels to train a classifier. Surprisingly, we find out that even a linear classifier^{1}^{1}1https://scikitlearn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html fits the data well (the training accuracy of these 600 data points is 98.7%). The results are illustrated in Fig. 4. The figure reveals that almost all of the weight values are positive, which indicates a strong relationship between how denser one architecture is wired and how robust it is under adversarial attacks. To further explore the relationship, we perform an analysis of the correlation between adversarial accuracy and the density of the architecture. We define the architecture density as the number of connected edges over the total number of all possible edges in the architecture, which can be expressed as:
(2) 
We illustrate the result in Fig. 5, which shows that there is a strong correlation between architecture density and adversarial accuracy. We posit that through adversarial training, densely connected patterns in the network are more beneficial against adversarial features and learn to be resistant to them. This gives us the answer to the first question in Sec. 1: Densely connected pattern can benefit the network robustness.
3.4 Architecture Strategy under Budget
It has been observed in many previous studies [28, 19] that, within the same family of architectures, increasing the number of parameters of the network would bring improvement of robustness. This is because such procedure will promote the model capacity, and thus can benefit the network robustness. However, if we are given a fixed total number of parameters (or we refer to as a computational budget), how to obtain architectures that are more robust under the limited constraint? In this section, we concentrate on how the pattern of an architecture influences robustness when given different fixed computational budgets. One advantage of our robust architecture search space for this study is that, the number of parameters of a network is positively correlated to the number of convolution operations in the architecture.
We first analyze the number of convolution operations with respect to network robustness, using the 1,000 architectures obtained in Sec. 3.3. The results are illustrated in Fig. 6. With the number of convolution operations increases, the adversarial accuracy improves steadily. We also plot the statistics for the number of convolutions on skip edges and direct edges, respectively. The results declare that convolutions on direct edges contribute more on adversarial accuracy than those on skip edges. This inspires us to dig deeper on the effect of the convolutions on direct edges for different computational budgets.
We consider three different computational budgets: small, medium and large. Since the maximum number of convolution operations in the cellbased setting is 14, we set the total number of convolutions smaller than 7 as small budget, between 8 and 10 as medium and larger than 11 as large. For each of the budget, we randomly sample 100 architectures, evaluate their adversarial accuracy following Sec. 3 and calculate the proportion of convolutions on direct edges among all convolutions. As illustrated in Fig. 6, the adversarial accuracy has clear boundaries between different budgets. Furthermore, for small and medium budget, the proportion of direct convolutions has a positive correlation to adversarial accuracy. This indicates that for smaller computational budget, adding convolutions to direct edges can efficiently improve the network robustness. We also note that this phenomenon is not obvious for the large setting. We speculate that for architectures within the large budget, densely connected patterns dominate the contributions of network robustness. With the above results, we conclude: Under small computational budget, adding convolution operations to direct edges is more effective to improve model robustness.
3.5 Towards a Larger Search Space
Relax the CellBased Constraint. In previous sections, we obtain several valuable observations for the cellbased setting. One natural question to ask is: What if we relax the constraint and permit all the cells in the network to have different architectures? Moreover, what can be the indicator for the network robustness in this cellfree setting? In this section, we relax the cellbased constraint and conduct studies on a larger architecture search space. The relaxation of the constraint raises an explosion of the complexity of the search space: for a network consisting of cells, the total complexity increases to . The exponential complexity makes the architecture search much more difficult to proceed.
Feature Flow Guided Search. To address the above challenges, here we propose a feature flow guided search scheme. Our approach is inspired by TRADES [39]
, which involves a loss function minimizing the KL divergence of the logit distribution between an adversarial example and its corresponding clean data. The value of this loss function can be utilized as a measurement of the gap between network robustness and its clean accuracy. Instead of focusing on the final output of a network, we consider the feature flow between the intermediate cells of a network. Specifically, we calculate the Gramian Matrix across each cell, denoted as flow of solution procedure (FSP) matrix
[37]. The FSP matrix for the th cell is calculated as:(3) 
where denotes the input feature map of the cell and denotes the output feature map. For a given network, we can calculate the distance of FSP matrix between adversarial example and clean data for each cell of the network:
(4) 
We sample 50 architectures finetuning for the cellfree search space and evaluate the gap of clean accuracy and adversarial accuracy for each architecture. We also calculate the FSP matrix distance for each cell of the network and show representative results in Fig. 7 (complete results are provided in Appendix). We can observe that for the cells in a deeper position of the network, the FSP distance has a positive correlation with the gap between network robustness and clean accuracy. This gives us the answer to the third question in Sec. 1: A robust network has a lower FSP matrix loss in the deeper cells of network.
By observing this phenomenon, we can easily adopt FSP matrix loss to filter out the nonrobust architectures with high loss values, which efficiently reduces the complexity of the search space. Thus, after the sampling process from supernet in cellfree setting, we first calculate FSP matrix loss for each architecture and reject those with high loss values. We then perform finetuning to get final robustness.
4 Experiments
In this section, we empirically evaluate the adversarial robustness of the proposed RobNet family. Following the guidance of our three findings in Sec. 3, we train and select a set of representative RobNet models for evaluation. We focus on bounded attacks and compare the RobNet family with stateoftheart humandesigned models.
Models  Model Size  Natural Acc.  FGSM  PGD  PGD  DeepFool  MIFGSM 

ResNet18  11.17M  78.38%  49.81%  45.60%  45.10%  47.64%  45.23% 
ResNet50  23.52M  79.15%  51.46%  45.84%  45.35%  49.18%  45.53% 
WideResNet2810  36.48M  86.43%  53.57%  47.10%  46.90%  51.23%  47.04% 
DenseNet121  6.95M  82.72%  54.14%  47.93%  47.46%  51.70%  48.19% 
RobNetsmall  4.41M  78.05%  53.93%  48.32%  48.07%  52.96%  48.98% 
RobNetmedium  5.66M  78.33%  54.55%  49.13%  48.96%  53.32%  49.34% 
RobNetlarge  6.89M  78.57%  54.98%  49.44%  49.24%  53.85%  49.92% 
RobNetlargev2  33.42M  85.69%  57.18%  50.53%  50.26%  55.45%  50.87% 
RobNetfree  5.49M  82.79%  58.38%  52.74%  52.57%  57.24%  52.95% 
4.1 Experimental Setup
Implementation Details. As described in Sec. 3, we use both cellbased and cellfree searching algorithm to select out a set of RobNet architectures, respectively. The robust search is performed only on CIFAR10, where we use PGD adversarial training with attack iterations and a step size of (0.01). For evaluation on other datasets, we directly transfer the RobNet architectures searched on CIFAR10.
Specifically, we first follow the cellbased robust search framework to obtain architectures that exhibit densely connected patterns. Considering the strategy under budget, we further generate three cellbased architectures that all follow more convolution operations on direct edges, but with different computational budgets. We refer to the three selected architectures as RobNetsmall, RobNetmedium, and RobNetlarge. Furthermore, we leverage FSP guided search described in Sec. 3.5 to efficiently generate cellfree robust architectures and select one representative model for evaluation, which is referred to as RobNetfree. Note that we are not selecting the best architecture, as the searching space is too large to allow us to do so. Instead, we follow the proposed algorithm to select representative architectures and study their robustness under adversarial attacks. More details of the selecting process and visualizations of the representative RobNet architectures can be found in Appendix.
We compare RobNet with widely used humandesigned architectures, including ResNet [11], WideResNet [38], and DenseNet [12]. All models are adversarially trained using PGD with attack steps and a step size of (0.01). We follow the training procedure as in [19] and keep other hyperparameters the same for all models.
Datasets & Evaluation. We first perform an extensive study on CIFAR10 to validate the effectiveness of RobNet against blackbox and whitebox attacks. We then extend the results to other datasets such as SVHN, CIFAR100, and TinyImageNet. Finally, we show the benefits from RobNet are orthogonal to existing techniques. We provide additional supporting results, as well as detailed training procedure and hyperparameters in Appendix.
4.2 Whitebox Attacks
Main Results. We show the results against various whitebox attacks in Table 1. We choose stateoftheart network architectures that are widely used in adversarial literature [19, 33, 39] for comparison. As illustrated in the table, all the selected models from RobNet family can consistently achieve higher robust accuracy under different whitebox attacks, compared to other models.
The strongest adversary in our whitebox setting is the PGD attacker with 100 attack iterations (i.e., PGD). When zoom in to the results, we can observe that by only changing architecture, RobNet can improve the previous arts under whitebox attacks by 5.1% from 47.5% to 52.6%.
The Effect of Dense Connections. Table 1 also reveals interesting yet important findings on dense connections. ResNet and its wide version (WideResNet) are most frequently used architectures in adversarial training [19, 33, 39]. Interestingly however, it turns out that the rarely used DenseNet model is more robust than WideResNet, even with much fewer parameters. Such observation are wellaligned with our previous study: densely connected pattern largely benefits the model robustness. Since RobNet family explicitly reveals such patterns during robust architecture search, they turn out to be consistently robust.
The Effect of Parameter Numbers. Inspired by the finding of computational budget, we seek to quantify the robustness of RobNets with different parameter numbers. We compare three models with different sizes obtained by cellbased search (i.e., RobNetsmall, RobNetmedium, and RobNetlarge). As Table 1 reports, with larger computational budgets, network robustness is consistently higher, which is well aligned with our arguments.
We note that the model sizes of RobNets are consistently smaller than other widely adopted network architectures. Yet, the natural accuracy of RobNet model is unsatisfying when compared to WideResNet. To further study the influence of network parameters, we extend the RobNetlarge model to have similar size as WideResNet by increasing the number of channels and stacked cells, while maintaining the same architecture within each cell. We refer to this new model as RobNetlargev2. It turns out that by increasing the model size, not only can the robustness be strengthened, the natural accuracy can also be significantly improved.
The Effect of Feature Flow Guided Search. When releasing the cellbased constraints during robust searching, RobNet models can be even more robust. We confirm it by comparing the results of RobNetfree, which is obtained using FSP Guided Search as mentioned in Sec. 3.5, with other cellbased RobNet models. Remarkably, RobNetfree achieves higher robust accuracy with 6 fewer parameter numbers when compared to RobNetlargev2 model.
Models  FGSM  PGD 

ResNet18  56.29%  54.28% 
ResNet50  58.12%  55.89% 
WideResNet2810  58.11%  55.68% 
DenseNet121  61.87%  59.34% 
RobNetlarge  61.92%  59.58% 
RobNetfree  65.06%  63.17% 
4.3 Blackbox Attacks
We further verify the robustness of RobNet family under blackbox attacks. We follow common settings in literature [23, 19, 36] and apply transferbased blackbox attacks. We train a copy of the victim network using the same training settings, and apply FGSM and PGD attacks on the copy network to generate adversarial examples. Note that we only consider the strongest transferbased attacks, i.e., we use whitebox attacks on the independently trained copy to generate blackbox examples.
The results are shown in Table 2. They reveal that both cellbased and cellfree RobNet models are more robust under transferbased attacks. Note that here the source model has the same architecture as the target model, which makes the blackbox adversary stronger [19]. We also study the transfer between different architectures, and provide corresponding results in the Appendix.
Models  SVHN  CIFAR100  TinyImageNet 

ResNet18  46.08%  22.01%  16.96% 
ResNet50  47.23%  22.38%  19.12% 
RobNetlarge  51.26%  23.19%  19.90% 
RobNetfree  55.59%  23.87%  20.87% 
4.4 Transferability to More Datasets
So far, our robust searching has only been performed and evaluated on CIFAR10. However, the idea of robust neural architecture search is much more powerful: we directly use the RobNet family searched on CIFAR10 to apply on other datasets, and demonstrate their effectiveness. Such benefits come from the natural advantage of NAS that the searched architectures can generalize to other datasets [42, 40].
We evaluate RobNet on SVHN, CIFAR100, and TinyImageNet under whitebox attacks, and set attack parameters as follows: total perturbation of (0.031), step size of
(0.01), and with 100 total attack iterations. The training procedure is similar to that on CIFAR10, where we use 7 steps PGD for adversarial training. We keep all the training hyperparameters the same for all models.
Table 3 shows the performance of RobNet on the three datasets and compares them with commonly used architectures. The table reveals the following results. First, it verifies the effectiveness of RobNet family: they consistently outperform other baselines under strong whitebox attacks. Furthermore, the gains across different datasets are different. RobNets provide about 2% gain on CIFAR100 and TinyImageNet, and yield 10% gain on SVHN.
Models  Natural Acc.  PGD 

ResNet18  78.38%  45.10% 
ResNet18 + Denoise  78.75%  45.82% 
RobNetlarge  78.57%  49.24% 
RobNetlarge + Denoise  84.03%  49.97% 
4.5 Boosting Existing Techniques
As RobNet improves model robustness from the aspect of network architecture, it can be seamlessly incorporated with existing techniques to further boost adversarial robustness. To verify this advantage, we select feature denoising technique [33] which operates by adding several denoising blocks in the network. We report the results in Table 4. As shown, the denoising module improves both clean and robust accuracy of RobNet, showing their complementariness. Moreover, when compared to ResNet18, RobNet can better harness the power of feature denoising, gaining a larger improvement gap, especially on clean accuracy.
5 Conclusion
We proposed a robust architecture search framework, which leverages oneshot NAS to understand the influence of network architectures against adversarial attacks. Our study revealed several valuable observations on designing robust network architectures. Based on the observations, we discovered RobNet, a family of robust architectures that are resistant to attacks. Extensive experiments validated the significance of RobNet, yielding the intrinsic effect of architectures on network resilience to adversarial attacks.
References
 [1] Anonymous. Deeper insights into weight sharing in neural architecture search. Submitted to ICLR, 2020. under review.
 [2] Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. Designing neural network architectures using reinforcement learning. In ICLR, 2017.
 [3] Gabriel Bender, PieterJan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoc Le. Understanding and simplifying oneshot architecture search. In ICML, 2018.
 [4] Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow. Thermometer encoding: One hot way to resist adversarial examples. In ICLR, 2018.
 [5] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), 2017.
 [6] Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi. Unlabeled data improves adversarial robustness. arXiv preprint arXiv:1905.13736, 2019.
 [7] Ekin D Cubuk, Barret Zoph, Samuel S Schoenholz, and Quoc V Le. Intriguing properties of adversarial examples. arXiv preprint arXiv:1711.02846, 2017.
 [8] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Xiaolin Hu, Jianguo Li, and Jun Zhu. Boosting adversarial attacks with momentum. In CVPR, 2018.
 [9] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
 [10] Minghao Guo, Zhao Zhong, Wei Wu, Dahua Lin, and Junjie Yan. Irlas: Inverse reinforcement learning for architecture search. In CVPR, 2019.
 [11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
 [12] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.
 [13] Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv:1803.06373, 2018.
 [14] Alex Krizhevsky et al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
 [15] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In ICLR Workshop, 2017.
 [16] FeiFei Li, Andrej Karpathy, and Justin Johnson. Tiny imagenet visual recognition challenge.
 [17] Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, LiJia Li, Li FeiFei, Alan Yuille, Jonathan Huang, and Kevin Murphy. Progressive neural architecture search. In ECCV, 2018.
 [18] Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. In ICLR, 2019.

[19]
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and
Adrian Vladu.
Towards deep learning models resistant to adversarial attacks.
In ICLR, 2018.  [20] SeyedMohsen MoosaviDezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In CVPR, 2016.
 [21] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
 [22] Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In ICML, 2019.
 [23] Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to blackbox attacks using adversarial samples. arXiv preprint arXiv:1605.07277, 2016.
 [24] JuanManuel PerezRua, Moez Baccouche, and Stephane Pateux. Efficient progressive neural architecture search. In BMVC, 2018.
 [25] Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le. Regularized evolution for image classifier architecture search. In AAAI, 2019.
 [26] Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In ICLR, 2018.
 [27] Robert Stanforth, Alhussein Fawzi, Pushmeet Kohli, et al. Are labels required for improving adversarial robustness? arXiv preprint arXiv:1905.13725, 2019.
 [28] Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, PinYu Chen, and Yupeng Gao. Is robustness the cost of accuracy?–a comprehensive study on the robustness of 18 deep image classification models. In ECCV, 2018.

[29]
Masanori Suganuma, Shinichi Shirakawa, and Tomoharu Nagao.
A genetic programming approach to designing convolutional neural network architectures.
InProceedings of the Genetic and Evolutionary Computation Conference
, pages 497–504. ACM, 2017.  [30] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
 [31] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
 [32] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In ICLR, 2018.
 [33] Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In CVPR, 2019.
 [34] Cihang Xie and Alan Yuille. Intriguing properties of adversarial training. arXiv preprint arXiv:1906.03787, 2019.
 [35] Saining Xie, Alexander Kirillov, Ross Girshick, and Kaiming He. Exploring randomly wired neural networks for image recognition. In ICCV, 2019.

[36]
Yuzhe Yang, Guo Zhang, Dina Katabi, and Zhi Xu.
MENet: Towards effective adversarial robustness with matrix estimation.
In ICML, 2019. 
[37]
Junho Yim, Donggyu Joo, Jihoon Bae, and Junmo Kim.
A gift from knowledge distillation: Fast optimization, network minimization and transfer learning.
In CVPR, 2017.  [38] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
 [39] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled tradeoff between robustness and accuracy. In ICML, 2019.
 [40] Zhao Zhong, Junjie Yan, Wei Wu, Jing Shao, and ChengLin Liu. Practical blockwise neural network architecture generation. In CVPR, 2018.
 [41] Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. In ICLR, 2017.
 [42] Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In CVPR, 2017.
Appendix A Details of Robust Architecture Search
We provide details of our robust architecture search algorithm. The pseudo code is illustrated in Algorithm 1.
Appendix B Details of Adversarial Training
We further provide training details of PGDbased adversarial training for each individual architecture. We summarize our training hyperparameters in Table 5. We follow the standard data augmentation scheme as in [11]
to do zeropadding with 4 pixels on each side, and then random crop back to the original image size. We then randomly flip the images horizontally and normalize them into
. We use the same training settings for CIFAR10 and CIFAR100.CIFAR  SVHN  TinyImageNet  
Optimizer  SGD  SGD  SGD  
Momentum  0.9  0.9  0.9  
Epochs  200  200  90  
LR  0.1  0.01  0.1  
LR decay 



Appendix C Complete Results of FSP Matrix Loss
We provide additional results for the correlation of FSP matrix distance along with the performance gap between clean accuracy and adversarial accuracy in cellfree setting. Results for several cells have been shown in the main paper. Here we provide results for additional cells in Fig. 8.
As can be observed from the figure, for cells in deeper positions of the network, the FSP distance has a positive correlation with the gap between network robustness and its clean accuracy, which indicates that a robust network has a lower FSP matrix loss in the deeper cells of the network.
Appendix D Visualization of RobNets
In this section, we first describe the details of how we select architectures of RobNet family. Further, we visualize several representative RobNet architectures.
In cellbased setting, we first filter out the architectures with architecture density . Then we only consider the architectures which have a portion of direct convolutions larger than . For each of the computational budget, we sample 50 architectures from the supernet following the process described above and finetune them for 3 epochs to get the adversarial accuracy. We select architecture with best performance under each budget, and refer them as RobNetsmall, RobNetmedium and RobNetlarge, respectively.
In cellfree setting, we first randomly sample 300 architectures from the supernet, and calculate the average FSP matrix distance for last 10 cells of each sampled network. Following the finding of FSP matrix loss as indicator, we reject those architectures whose average distance is larger than a threshold. In our experiments, we set the threshold to be , which leads to 68 remaining architectures. Finally, we finetune each of them for 3 epochs and select the architecture with the highest adversarial accuracy, which is named as RobNetfree.
We visualize several representative architectures of RobNet family in Fig. 9.
TargetSource  ResNet18  ResNet50 


RobNetlarge  RobNetfree  
ResNet18  54.28%  54.49%  56.44%  57.19%  55.57%  59.37%  
ResNet50  56.24%  55.89%  56.38%  58.31%  57.22%  60.19%  
WideResNet2810  57.89%  57.96%  55.68%  58.41%  59.08%  60.74%  
DenseNet121  61.42%  61.96%  60.28%  59.34%  60.03%  59.96%  
RobNetlarge  59.63%  59.82%  59.72%  60.03%  59.58%  60.73%  
RobNetfree  66.64%  66.09%  65.05%  64.40%  63.35%  63.17% 
Appendix E Additional Blackbox Attack Results
We provide additional results on transferbased blackbox attacks on CIFAR10, across different network architectures. The blackbox adversarial examples are generated from an independently trained copy of the network, by using whitebox attack on the victim network. We apply PGDbased blackbox attacks with 100 iterations across different architectures, and report the result in Table 6. All models are adversarially trained using PGD with steps.
In the table, we highlight the best result of each column in bold, which corresponds to the most robust model against blackbox adversarial examples generated from one specific source network. We also underline the empirical lower bound for each network, which corresponds to the lowest accuracy of each row.
As the table reveals, RobNetfree model achieves the highest robust accuracy under transferbased attacks from different sources. Furthermore, the most powerful blackbox adversarial examples for each network (i.e., the underlined value) are from source network that uses the same architecture as the target network. Finally, by comparing the transferability between two network architectures (e.g., RobNetfree ResNet18 & ResNet18 RobNetfree), we can observe the following phenomena. First, our RobNet models are more robust against blackbox attacks transferred from other models. Moreover, our RobNet models can generate stronger adversarial examples for blackbox attacks compared with other widely used models.
Appendix F Additional Whitebox Attack Results
As common in recent literature [33, 36, 39], strongest possible attack should be considered when evaluating the adversarial robustness. Therefore, we further strengthen the adversary and vary the attack iterations from 7 to 1000. We show the results in Fig. 10, where RobNet family outperforms other networks, even facing the strong adversary. Specifically, compared to stateoftheart models, RobNetlarge and RobNetfree can gain and improvement, respectively. We also observe that the attacker performance diminishes with 5001000 attack iterations.
Comments
There are no comments yet.