{"id":713494,"date":"2020-12-22T09:32:02","date_gmt":"2020-12-22T17:32:02","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=713494"},"modified":"2020-12-22T09:54:30","modified_gmt":"2020-12-22T17:54:30","slug":"unadversarial-examples-designing-objects-for-robust-vision","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/unadversarial-examples-designing-objects-for-robust-vision\/","title":{"rendered":"Unadversarial examples: Designing objects for robust vision"},"content":{"rendered":"\n
\"An<\/figure>\n\n\n\n

Editor\u2019s note: This post and its research are the result of the collaborative efforts of our team\u2014MIT PhD students <\/em>Andrew Ilyas<\/em><\/a> and <\/em>Logan Engstrom<\/em><\/a>, Senior Researcher <\/em>Sai Vemprala<\/em><\/a>, MIT professor<\/em> Aleksander Madry<\/em><\/a>, and Partner Research Manager <\/em>Ashish Kapoor<\/em><\/a>.<\/em><\/p>\n\n\n\n

Many of the items and objects we use in our daily lives were designed with people in mind. In October, the Reserve Bank of Australia put out into the world its redesigned $100 banknote (opens in new tab)<\/span><\/a>. Some design elements remained the same\u2014such as color and size, characteristics people use to tell the difference between notes\u2014while others changed. New security features to help protect against fraud were added as were raised bumps for people who are blind or have low vision. Good design enables intended audiences to easily acquire information and act on it.<\/p>\n\n\n\n

Read Paper (opens in new tab)<\/span><\/a>                        Code & Materials (opens in new tab)<\/span><\/a> <\/p>\n\n\n\n

Modern computer vision systems take similar cues\u2014floor markings direct a robot\u2019s course, boxes in a warehouse signal a forklift to move them, and stop signs alert a self-driving car to, well, stop. The neural networks underlying these systems might understand the features that we as humans find helpful, but they might also understand different features even better. In scenarios in which system operators and designers have a level of control over the target objects, what if we designed the objects in a way that makes them more detectable, even under conditions that normally break such systems, such as bad weather or variations in lighting?<\/p>\n\n\n\n

We introduce a framework that exploits computer vision systems\u2019 well-known sensitivity to perturbations of their inputs to create robust, <\/em>or unadversarial, objects<\/em>\u2014that is, objects that are optimized specifically for better performance and robustness of vision models. Instead of using perturbations to get neural networks to wrongly classify objects, as is the case with adversarial examples, we use them to encourage the neural network to correctly classify the objects we care about with high confidence.<\/p>\n\n\n\n

\"An
Figure 1: Optimizing objects for pre-trained neural networks rather than only optimizing the networks themselves can significantly boost performance and robustness on computer vision tasks. Above, a human-designed jet and a jet modified with a texture optimized for easier model detection are correctly classified under normal weather conditions; only the modified jet is correctly classified in the presence of fog or dust.<\/figcaption><\/figure>\n\n\n\n
\n\t