<\/p>\n
Adam Shostack here. <\/span><\/p>\n
<\/p>\n
I said recently that I wanted to talk more about what I do. The core of what I do is help Microsoft\u2019s product teams analyze the security of their designs by threat modeling. \u00a0\u00a0So I\u2019m very concerned about how well we threat model, and how to help folks I work with do it better.\u00a0\u00a0 I\u2019d like to start that by talking about some of the things that make the design analysis process difficult, then what we\u2019ve done to address those things.\u00a0 As each team starts a new product cycle, they have to decide how much time to spend on the tasks that are involved in security.\u00a0 There\u2019s competition for the time and attention of various people within a product team.\u00a0 Human nature is that if a \u00a0process is easy or rewarding, people will spend time on it.\u00a0 If it\u2019s not, they\u2019ll do as little of it as they can get away with.\u00a0 So the process evolves, because, unlike Dr No<\/a>, we want to be aligned with what our product groups and customers want<\/p>\n
There have been a lot of variants of things called \u201cthreat modeling processes\u201d at Microsoft, and a lot more in the wide world.\u00a0\u00a0 People sometimes want to argue because they think Microsoft uses the term \u201cthreat modeling\u201d differently than the rest of the world.\u00a0 This is only a little accurate.\u00a0 There is a community which uses questions like \u201cwhat\u2019s your threat model\u201d to mean \u201cwhich attackers are you trying to stop?\u201d\u00a0 Microsoft uses threat model to mean \u201cwhich attacks are you trying to stop?\u201d\u00a0 There are other communities whose use is more like ours.\u00a0 In this paragraph, I\u2019m attempting to mitigate a denial of service threat, where prescriptivists<\/a> try to drag us into a long discussion of how we\u2019re using words.)\u00a0\u00a0 The processes I\u2019m critiquing here are the versions of threat modeling that are presented in Writing Secure Code<\/a>, Threat Modeling<\/a><\/i>, and The Security Development Lifecycle<\/i><\/a> books.<\/p>\n
Of course, that wasn\u2019t the intent, but it often was the effect.<\/p>\n
The Disconnected Process<\/b><\/p>\n
Another set of problems is that threat modeling can feel disconnected from the development process.\u00a0 The extreme programming folks are fond of only doing what they need to do to ship, and Microsoft shipped code without threat models for a long time.\u00a0 The further something is from the process of building code, the less likely it is to be complete and up to date.\u00a0 That problem was made worse because there weren\u2019t a lot of people who would say \u201clet me see the threat model for that.\u201d\u00a0 \u00a0So there wasn\u2019t a lot of pressure to keep threat models up to date, even if teams had done a good job up front with them.\u00a0 There may be more pressure with other specs which are used by a broader set of people during development.<\/p>\n
Validation<\/b><\/p>\n
Once a team had started threat modeling, they had trouble knowing if they were doing a good job.\u00a0 Had they done enough?\u00a0 Was their threat model a good representation of the work they had done, or were planning to do?\u00a0 When we asked people to draw diagrams, we didn\u2019t tell them when they could stop, or what details didn\u2019t matter.\u00a0 When we asked them to brainstorm about threats, we didn\u2019t guide them as to how many they should find.\u00a0 When they found threats, what were they supposed to do about them?\u00a0 This was easier when there was an expert in the room to provide advice on how to mitigate the threat effectively. \u00a0\u00a0How should they track them?\u00a0 \u00a0Threats aren\u2019t quite bugs\u2014you can never remove a threat, only mitigate it.\u00a0 So perhaps it didn\u2019t make sense to track them like that, but that left threats in a limbo.<\/p>\n
“Return on Investment”<\/b><\/p>\n
\u00a0 The time invested often didn\u2019t seem like it was paying off.\u00a0 Sometimes it really didn\u2019t pay off.\u00a0\u00a0\u00a0 (David LeBlanc makes this point forcefully in \u201cThreat Modeling the Bold Button is Boring<\/a>\u201d) Sometimes it just felt that way\u2014Larry Osterman made that point, unintentionally in \u201cThreat Modeling Again, Presenting the PlaySound Threat Model<\/a>,\u201d where he said \u201cLet’s look at a slightly more interesting case where threat modeling exposes an issue.\u201d\u00a0 Youch!\u00a0 But as I wrote in a comment on that post, \u201cWhat you’ve been doing here is walking through a lot of possibilities.\u00a0 Some of those turn out to be uninteresting, and we learn something.\u00a0 Others (as we’ve discussed in email) were pretty clearly uninteresting\u201d\u00a0 It can be important to walk through those possibilities so we know they\u2019re uninteresting.\u00a0 Of course, we\u2019d like to reduce the time it takes to look at each uninteresting issue.<\/p>\n
Larry Osterman lays out some other reasons threat modeling is hard in a blog post: http:\/\/blogs.msdn.com\/larryosterman\/archive\/2007\/08\/30\/threat-modeling-once-again.aspx<\/a><\/p>\n
\nOne thing that was realized very early on is that our early efforts at threat modeling were quite ad-hoc.\u00a0We sat in a room and said “Hmm, what might the bad guys do to attack our product?” It turns out that this isn’t actually a BAD way of going about threat modeling, and if that’s all you do, you’re way better off than you were if you’d done nothing.<\/p>\n
<\/p>\n
Why doesn’t it work?\u00a0 There are a couple of reasons:<\/p>\n
It takes a special mindset to think like a bad guy.\u00a0 Not everyone can switch into that mindset.\u00a0 For instance, I can’t think of the number of times I had to tell developers on my team “It doesn’t matter that you’ve checked the value on the client, you still need to check it on the server because the client that’s talking to your server might not be your code.”.<\/p>\n
Developers tend to think in terms of what a customer needs.\u00a0 But many times, the things that make things really cool for a customer provide a superhighway for the bad guy to attack your code.<\/p>\n
It’s ad-hoc.\u00a0 Microsoft asks every single developer and program manager to threat model (because they’re the ones who know what the code is doing).\u00a0 Unfortunately that means that they’re not experts on threat modeling. Providing structure helps avoid mistakes.<\/p>\n<\/blockquote>\n
With all these problems, we still threat model, because it pays dividends.\u00a0 In the next posts, I\u2019ll talk about what we\u2019ve done to improve things, what the process looks like now, and perhaps a bit about what it might look like either in the future, or adopted by other organizations.<\/p>\n
<\/div>\n","protected":false},"excerpt":{"rendered":"Adam Shostack here. I said recently that I wanted to talk more about what I do. The core of what I do is help Microsoft\u2019s product teams analyze the security of their designs by threat modeling. \u00a0\u00a0So I\u2019m very concerned about how well we threat model, and how to help folks I work […]<\/p>\n","protected":false},"author":61,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ms_queue_id":[],"ep_exclude_from_search":false,"_classifai_error":"","_classifai_text_to_speech_error":"","footnotes":""},"content-type":[3663],"topic":[3687],"products":[],"threat-intelligence":[3732],"tags":[],"coauthors":[1939],"class_list":["post-1909","post","type-post","status-publish","format-standard","hentry","content-type-research","topic-threat-intelligence","threat-intelligence-influence-operations"],"yoast_head":"\n
The Trouble with Threat Modeling | Microsoft Security Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Trouble with Threat Modeling | Microsoft Security Blog\" \/>\n<meta property=\"og:description\" content=\" Adam Shostack here. I said recently that I wanted to talk more about what I do. The core of what I do is help Microsoft\u2019s product teams analyze the security of their designs by threat modeling. \u00a0\u00a0So I\u2019m very concerned about how well we threat model, and how to help folks I work […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Security Blog\" \/>\n<meta property=\"article:published_time\" content=\"2007-09-26T15:11:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-05-16T06:11:04+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-microsoft_logo_element.png\" \/>\n\t<meta property=\"og:image:width\" content=\"512\" \/>\n\t<meta property=\"og:image:height\" content=\"512\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"SDL Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"SDL Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/\"},\"author\":[{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/author\/sdlteam\/\",\"@type\":\"Person\",\"@name\":\"SDL Team\"}],\"headline\":\"The Trouble with Threat Modeling\",\"datePublished\":\"2007-09-26T15:11:00+00:00\",\"dateModified\":\"2023-05-16T06:11:04+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/\"},\"wordCount\":1748,\"publisher\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/\",\"name\":\"The Trouble with Threat Modeling | Microsoft Security Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#website\"},\"datePublished\":\"2007-09-26T15:11:00+00:00\",\"dateModified\":\"2023-05-16T06:11:04+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Trouble with Threat Modeling\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#website\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/\",\"name\":\"Microsoft Security Blog\",\"description\":\"Expert coverage of cybersecurity topics\",\"publisher\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization\",\"name\":\"Microsoft Security Blog\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-cropped-microsoft_logo_element.png\",\"contentUrl\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-cropped-microsoft_logo_element.png\",\"width\":512,\"height\":512,\"caption\":\"Microsoft Security Blog\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The Trouble with Threat Modeling | Microsoft Security Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/","og_locale":"en_US","og_type":"article","og_title":"The Trouble with Threat Modeling | Microsoft Security Blog","og_description":" Adam Shostack here. I said recently that I wanted to talk more about what I do. The core of what I do is help Microsoft\u2019s product teams analyze the security of their designs by threat modeling. \u00a0\u00a0So I\u2019m very concerned about how well we threat model, and how to help folks I work […]","og_url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/","og_site_name":"Microsoft Security Blog","article_published_time":"2007-09-26T15:11:00+00:00","article_modified_time":"2023-05-16T06:11:04+00:00","og_image":[{"width":512,"height":512,"url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-microsoft_logo_element.png","type":"image\/png"}],"author":"SDL Team","twitter_card":"summary_large_image","twitter_misc":{"Written by":"SDL Team","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/#article","isPartOf":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/"},"author":[{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/author\/sdlteam\/","@type":"Person","@name":"SDL Team"}],"headline":"The Trouble with Threat Modeling","datePublished":"2007-09-26T15:11:00+00:00","dateModified":"2023-05-16T06:11:04+00:00","mainEntityOfPage":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/"},"wordCount":1748,"publisher":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/","url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/","name":"The Trouble with Threat Modeling | Microsoft Security Blog","isPartOf":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#website"},"datePublished":"2007-09-26T15:11:00+00:00","dateModified":"2023-05-16T06:11:04+00:00","breadcrumb":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2007\/09\/26\/the-trouble-with-threat-modeling\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/"},{"@type":"ListItem","position":2,"name":"The Trouble with Threat Modeling"}]},{"@type":"WebSite","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#website","url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/","name":"Microsoft Security Blog","description":"Expert coverage of cybersecurity topics","publisher":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization","name":"Microsoft Security Blog","url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-cropped-microsoft_logo_element.png","contentUrl":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-cropped-microsoft_logo_element.png","width":512,"height":512,"caption":"Microsoft Security Blog"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/logo\/image\/"}}]}},"msxcm_display_generated_audio":false,"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Microsoft Security Blog","distributor_original_site_url":"https:\/\/www.microsoft.com\/en-us\/security\/blog","push-errors":false,"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/posts\/1909","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/users\/61"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/comments?post=1909"}],"version-history":[{"count":0,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/posts\/1909\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/media?parent=1909"}],"wp:term":[{"taxonomy":"content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/content-type?post=1909"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/topic?post=1909"},{"taxonomy":"products","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/products?post=1909"},{"taxonomy":"threat-intelligence","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/threat-intelligence?post=1909"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/tags?post=1909"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/coauthors?post=1909"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}