{"id":439581,"date":"2017-11-13T08:59:30","date_gmt":"2017-11-13T16:59:30","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=439581"},"modified":"2017-11-27T14:56:50","modified_gmt":"2017-11-27T22:56:50","slug":"neural-fuzzing","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/neural-fuzzing\/","title":{"rendered":"Neural fuzzing: applying DNN to software security testing"},"content":{"rendered":"<div id=\"attachment_440193\" style=\"width: 370px\" class=\"wp-caption alignleft\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-440193\" class=\"wp-image-440193\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Springfield-Group-001-800x1000.jpg\" alt=\"\" width=\"360\" height=\"450\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Springfield-Group-001-800x1000.jpg 800w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Springfield-Group-001-800x1000-240x300.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Springfield-Group-001-800x1000-768x960.jpg 768w\" sizes=\"(max-width: 360px) 100vw, 360px\" \/><p id=\"caption-attachment-440193\" class=\"wp-caption-text\">William Blum, Principal Research Engineering Lead. (Photography by Scott Eklund\/Red Box Pictures)<\/p><\/div>\n<p>Microsoft researchers have developed a new method for discovering software security vulnerabilities that uses machine learning and deep neural networks to help the system root out bugs better by learning from past experience. This new research project, called <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/not-all-bytes-are-equal-neural-byte-sieve-for-fuzzing\/\">neural fuzzing<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, is designed to augment traditional fuzzing techniques, and early experiments have demonstrated promising results.<\/p>\n<p>Software security testing is a hard task that is traditionally done by security experts through costly and targeted code audits, or by using very specialized and complex security tools to detect and assess vulnerabilities in code. We recently released a tool, called <a href=\"https:\/\/www.microsoft.com\/en-us\/security-risk-detection\/\">Microsoft Security Risk Detection<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, that significantly simplifies security testing and does not require you to be an expert in security in order to root out software bugs. The Azure-based tool is available to Windows users and in preview for Linux users.<\/p>\n<p><strong>Fuzz testing<\/strong><br \/>\nThe key technology underpinning Microsoft Security Risk Detection is fuzz testing<em>, <\/em>or fuzzing. It\u2019s a program analysis technique that looks for inputs causing error conditions that have a high chance of being exploitable, such as buffer overflows, memory access violations and null pointer dereferences.<\/p>\n<p>Fuzzers come in different categories:<\/p>\n<ul>\n<li><strong>Blackbox fuzzers<\/strong>, also called \u201cdumb fuzzers,\u201d rely solely on the sample input files to generate new inputs.<\/li>\n<li><strong>Whitebox fuzzers<\/strong> analyze the target program either statically or dynamically to guide the search for new inputs aimed at exploring as many code paths as possible.<\/li>\n<li><strong>Greybox fuzzers, <\/strong>just like blackbox fuzzers, don\u2019t have any knowledge of the structure of the target program, but make use of a feedback loop to guide their search based on observed behavior from previous executions of the program.<\/li>\n<\/ul>\n<div id=\"attachment_439602\" style=\"width: 857px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-439602\" class=\"wp-image-439602 size-full\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-1-screenshot.png\" alt=\"\" width=\"847\" height=\"398\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-1-screenshot.png 847w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-1-screenshot-300x141.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-1-screenshot-768x361.png 768w\" sizes=\"(max-width: 847px) 100vw, 847px\" \/><p id=\"caption-attachment-439602\" class=\"wp-caption-text\">Figure 1 &#8211; Crashes reported by AFL. Experimental support in MSRD<\/p><\/div>\n<p><strong>Neural fuzzing<br \/>\n<\/strong>Earlier this year, Microsoft researchers including myself, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/risin\/\">Rishabh Singh<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, and Mohit Rajpal, began a research project looking at ways to improve fuzzing techniques using machine learning and deep neural networks. Specifically, we wanted to see what a machine learning model could learn if we were to insert a deep neural network into the feedback loop of a greybox fuzzer.<\/p>\n<p>For our initial experiment, we looked at whether we could learn over time by observing past fuzzing iterations of an existing fuzzer.<\/p>\n<p>We applied our methods to a type of greybox fuzzer called American fuzzy lop, or AFL.<\/p>\n<p>We tried four different types of neural networks and ran the experiment on four target programs, using parsers for four different file formats: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.gnu.org\/software\/binutils\/\">ELF<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/mupdf.com\/\">PDF<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.libpng.org\/pub\/png\/\">PNG<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.xmlsoft.org\/\">XML<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/p>\n<p>The results were very encouraging\u2014we saw signi\ufb01cant improvements over traditional AFL in terms of code coverage, unique code paths and crashes for the four input formats.<\/p>\n<ul>\n<li>The AFL system using deep neural networks based on the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/en.wikipedia.org\/wiki\/Long_short-term_memory\">Long short-term memory<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (LSTM) neural network model gives around 10 percent improvement in code coverage over traditional AFL for two files parsers: ELF and PNG.<\/li>\n<li>When looking at unique code paths, neural AFL discovered more unique paths than traditional AFL for all parsers except PDF. For the PNG parser, after 24 hours of fuzzing it found twice as many unique code paths as traditional AFL.<\/li>\n<\/ul>\n<div id=\"attachment_439605\" style=\"width: 419px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-439605\" class=\"wp-image-439605 size-full\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-1-graph.png\" alt=\"\" width=\"409\" height=\"390\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-1-graph.png 409w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-1-graph-300x286.png 300w\" sizes=\"(max-width: 409px) 100vw, 409px\" \/><p id=\"caption-attachment-439605\" class=\"wp-caption-text\">Figure 2 &#8211; Input gain over time (in hours) for the libpng file parser.<\/p><\/div>\n<ul>\n<li>A good way to evaluate fuzzers is to compare the number of crashes reported. For the ELF file parser, neural AFL reported more than 20 crashes whereas traditional AFL did not report any. This is astonishing given that neural AFL was trained on AFL itself. We also observed more crashes being reported for text-based file formats like XML, where neural AFL could find 38 percent more crashes than traditional AFL. For PDF, traditional AFL did overall better than neural AFL in terms of new code paths found. However, neither system reported any crashes.<\/li>\n<\/ul>\n<div id=\"attachment_439614\" style=\"width: 816px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-439614\" class=\"wp-image-439614 size-full\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-2-combined.jpg\" alt=\"\" width=\"806\" height=\"390\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-2-combined.jpg 806w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-2-combined-300x145.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Figure-2-combined-768x372.jpg 768w\" sizes=\"(max-width: 806px) 100vw, 806px\" \/><p id=\"caption-attachment-439614\" class=\"wp-caption-text\">Figure 3 \u2013 Reported crashes over time (in hours) for readelf (left) and libxml (right).<\/p><\/div>\n<p>Overall, using neural fuzzing outperformed traditional AFL in every instance except the PDF case, where we suspect the large size of the PDF files incurs noticeable overhead when querying the neural model.<\/p>\n<p>In general, we believe our neural fuzzing approach yields a novel way to perform greybox fuzzing that is simple, efficient and generic.<\/p>\n<ul>\n<li><strong>Simple<\/strong>: The search is not based on sophisticated hand-crafted heuristics &#8212; the system learns a strategy from an existing fuzzer. We just give it sequences of bytes and let it figure out all sorts of features and automatically generalize from them to predict which types of inputs are more important than others and where the fuzzer\u2019s attention should be focused.<\/li>\n<li><strong>Efficient<\/strong>: In our AFL experiment, in the first 24 hours we explored significantly more unique code paths than traditional AFL. For some parsers we even report crashes not already reported by AFL.<\/li>\n<li><strong>Generic<\/strong>: Although we\u2019ve tested it only on AFL, our approach could be applied to any fuzzer, including blackbox and random fuzzers.<\/li>\n<\/ul>\n<p>We believe our neural fuzzing research project is just scratching the surface of what can be achieved using deep neural networks for fuzzing. Right now, our model only learns fuzzing locations, but we could also use it to learn other fuzzing parameters such as the type of mutation or strategy to apply. We are also considering online versions of our machine learning model, in which the fuzzer constantly learns from ongoing fuzzing iterations.<\/p>\n<p><em><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/wiblum\/\">William Blum<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> leads the engineering team for Microsoft Security Risk Detection. <\/em><\/p>\n<p><strong>Related<\/strong>:<\/p>\n<ul>\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/not-all-bytes-are-equal-neural-byte-sieve-for-fuzzing\/\">Not all bytes are equal: Neural byte sieve for fuzzing<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/security-risk-detection\/\">Learn more about Microsoft Security Risk Detection<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/blogs.microsoft.com\/ai\/2017\/07\/21\/ai-for-security-microsoft-security-risk-detection-makes-debut\/\">AI for security: Microsoft Security Risk Detection makes debut<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Microsoft researchers have developed a new method for discovering software security vulnerabilities that uses machine learning and deep neural networks to help the system root out bugs better by learning from past experience. This new research project, called neural fuzzing, is designed to augment traditional fuzzing techniques, and early experiments have demonstrated promising results. Software [&hellip;]<\/p>\n","protected":false},"author":36509,"featured_media":442878,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"categories":[194489],"tags":[],"research-area":[13558],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-439581","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-security","msr-research-area-security-privacy-cryptography","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[308699],"related-events":[],"related-researchers":[{"type":"user_nicename","value":"wiblum","user_id":34834,"display_name":"William Blum","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/wiblum\/\" aria-label=\"Visit the profile page for William Blum\">William Blum<\/a>","is_active":false,"last_first":"Blum, William","people_section":0,"alias":"wiblum"}],"msr_type":"Post","featured_image_thumbnail":"<img width=\"507\" height=\"280\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Springfield-Group-001-option-1-507x280.jpg\" class=\"img-object-cover\" alt=\"Microsoft&#039;s Springfield group. (Photography by Scott Eklund\/Red Box Pictures)\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Springfield-Group-001-option-1-507x280.jpg 507w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Springfield-Group-001-option-1-507x280-300x166.jpg 300w\" sizes=\"(max-width: 507px) 100vw, 507px\" \/>","byline":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/wiblum\/\" title=\"Go to researcher profile for William Blum\" aria-label=\"Go to researcher profile for William Blum\" data-bi-type=\"byline author\" data-bi-cN=\"William Blum\">William Blum<\/a>","formattedDate":"November 13, 2017","formattedExcerpt":"Microsoft researchers have developed a new method for discovering software security vulnerabilities that uses machine learning and deep neural networks to help the system root out bugs better by learning from past experience. This new research project, called neural fuzzing, is designed to augment traditional&hellip;","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/439581"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/36509"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=439581"}],"version-history":[{"count":19,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/439581\/revisions"}],"predecessor-version":[{"id":440226,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/439581\/revisions\/440226"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/442878"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=439581"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=439581"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=439581"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=439581"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=439581"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=439581"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=439581"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=439581"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=439581"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=439581"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=439581"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}