{"id":713494,"date":"2020-12-22T09:32:02","date_gmt":"2020-12-22T17:32:02","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=713494"},"modified":"2020-12-22T09:54:30","modified_gmt":"2020-12-22T17:54:30","slug":"unadversarial-examples-designing-objects-for-robust-vision","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/unadversarial-examples-designing-objects-for-robust-vision\/","title":{"rendered":"Unadversarial examples: Designing objects for robust vision"},"content":{"rendered":"\n
\"An<\/figure>\n\n\n\n

Editor\u2019s note: This post and its research are the result of the collaborative efforts of our team\u2014MIT PhD students <\/em>Andrew Ilyas<\/em> (opens in new tab)<\/span><\/a> and <\/em>Logan Engstrom<\/em> (opens in new tab)<\/span><\/a>, Senior Researcher <\/em>Sai Vemprala<\/em><\/a>, MIT professor<\/em> Aleksander Madry<\/em> (opens in new tab)<\/span><\/a>, and Partner Research Manager <\/em>Ashish Kapoor<\/em><\/a>.<\/em><\/p>\n\n\n\n

Many of the items and objects we use in our daily lives were designed with people in mind. In October, the Reserve Bank of Australia put out into the world its redesigned $100 banknote (opens in new tab)<\/span><\/a>. Some design elements remained the same\u2014such as color and size, characteristics people use to tell the difference between notes\u2014while others changed. New security features to help protect against fraud were added as were raised bumps for people who are blind or have low vision. Good design enables intended audiences to easily acquire information and act on it.<\/p>\n\n\n\n

Read Paper<\/a>                        Code & Materials (opens in new tab)<\/span><\/a> <\/p>\n\n\n\n

Modern computer vision systems take similar cues\u2014floor markings direct a robot\u2019s course, boxes in a warehouse signal a forklift to move them, and stop signs alert a self-driving car to, well, stop. The neural networks underlying these systems might understand the features that we as humans find helpful, but they might also understand different features even better. In scenarios in which system operators and designers have a level of control over the target objects, what if we designed the objects in a way that makes them more detectable, even under conditions that normally break such systems, such as bad weather or variations in lighting?<\/p>\n\n\n\n

We introduce a framework that exploits computer vision systems\u2019 well-known sensitivity to perturbations of their inputs to create robust, <\/em>or unadversarial, objects<\/em>\u2014that is, objects that are optimized specifically for better performance and robustness of vision models. Instead of using perturbations to get neural networks to wrongly classify objects, as is the case with adversarial examples, we use them to encourage the neural network to correctly classify the objects we care about with high confidence.<\/p>\n\n\n\n

\"An
Figure 1: Optimizing objects for pre-trained neural networks rather than only optimizing the networks themselves can significantly boost performance and robustness on computer vision tasks. Above, a human-designed jet and a jet modified with a texture optimized for easier model detection are correctly classified under normal weather conditions; only the modified jet is correctly classified in the presence of fog or dust.<\/figcaption><\/figure>\n\n\n\n
\n\t
\n\t\t
\n\t\t\t\t\t\tPublication<\/span>\n\t\t\tUnadversarial Examples: Designing Objects for Robust Vision<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

We show that such optimization of objects for vision systems significantly improves the performance and robustness of these systems, even to unforeseen data shifts and corruptions. An example of this is demonstrated above in Figure 1, where we modify a jet with a pattern optimized to enable image classifiers to more robustly recognize the jet under various weather conditions: while both the original jet and its unadversarial counterpart are correctly classified in normal conditions, only the unadversarial jet is recognized when corruptions like fog or dust are added. We present the details of this research in our paper \u201cUnadversarial Examples: Designing Objects for Robust Vision<\/a>.\u201d<\/p>\n\n\n\n

Why design objects for neural networks?<\/h2>\n\n\n\n

The fragility of computer vision systems makes reliability and safety a real concern when deploying these systems in the real world. For example, a self-driving car\u2019s stop-sign detection system might be severely affected in the presence of intense weather conditions such as snow or fog. While techniques such as data augmentation, domain randomization, and robust training might seem to improve the performance of such systems, they don\u2019t typically generalize well to corrupted or otherwise unfamiliar data that these systems face when deployed.<\/p>\n\n\n\n\n\t

\n\t\t\n\n\t\t

\n\t\tSpotlight: Event Series<\/span>\n\t<\/p>\n\t\n\t

\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"Research\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

Microsoft Research Forum<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

Join us for a continuous exchange of ideas about research in the era of general AI. Watch the first four episodes on demand.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tWatch on-demand\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n\n

We were motivated to find another approach by scenarios in which system designers and operators not only have control of the neural network itself, but also have some degree of control over the objects they want their model to recognize or detect\u2014for example, a company that operates drones for delivery or transportation. These drones fly from place to place, and an important task for the system is landing safely at the target locations. Human operators may manage the landing pads at these locations, as well as the design of the system, presenting an opportunity to improve the system\u2019s ability to detect the landing pad by modifying the pad itself. <\/p>\n\n\n\n

Designing robust objects for vision<\/h2>\n\n\n\n

Our starting point in designing robust objects for vision is the observation that modern vision models suffer from a severe input sensitivity that can, in particular, be exploited to generate so-called adversarial examples (opens in new tab)<\/span><\/a>: imperceptible perturbations of the input of a vision model that break it. Adversarial examples can potentially be used to intentionally cause system failures; researchers and practitioners use these examples to train systems that are more robust to such attacks. These perturbations are typically constructed by solving the following optimization problem, which maximizes the loss of a machine learning model with respect to the input:<\/p>\n\n\n\n

\\(\\delta_{adv} = \\arg\\max_{\\delta \\in \\Delta} L(\\theta; x + \\delta, y),\\)<\/p>\n\n\n\n

<\/p>\n\n\n\n

where \\(\\theta\\) is the set of model parameters; \\(x\\) is a natural image; \\(y\\) is the corresponding correct label; \\(L\\) is the loss function used to train \\(\\theta\\) (for example, cross-entropy loss in classification contexts); and \\(\\Delta\\) is a class of permissible perturbations. In our work, we aim to convert this unusually large input sensitivity from a weakness into a strength. That is, instead of creating misleading inputs, as shown in the above equation, we demonstrate how to optimize inputs that bolster performance, resulting in these unadversarial examples, or robust objects. This is done by simply solving the following optimization problem:<\/p>\n\n\n\n

\\(\\delta_{unadv} = \\arg\\min_{\\delta \\in \\Delta} L(\\theta; x + \\delta, y).\\)<\/p>\n\n\n\n

In our research, we explore two ways of designing robust objects: via an unadversarial patch applied to the object or by unadversarially altering the texture of the object (Figure 2). Both ways require the above optimization algorithm to iteratively optimize the patch or texture with \\(\\Delta\\) being the set of perturbations spanning the patch or texture. Note that we start with a randomly initialized patch or texture.<\/p>\n\n\n\n

\"At
Figure 2: An example of an unadversarial patch (left) placed on a toy jet and an unadversarial texture (right) implicit in the design of a jet.<\/figcaption><\/figure><\/div>\n\n\n\n