<\/p>\n
Adam Shostack here. <\/span><\/p>\n <\/p>\n I said recently that I wanted to talk more about what I do. The core of what I do is help Microsoft\u2019s product teams analyze the security of their designs by threat modeling. \u00a0\u00a0So I\u2019m very concerned about how well we threat model, and how to help folks I work with do it better.\u00a0\u00a0 I\u2019d like to start that by talking about some of the things that make the design analysis process difficult, then what we\u2019ve done to address those things.\u00a0 As each team starts a new product cycle, they have to decide how much time to spend on the tasks that are involved in security.\u00a0 There\u2019s competition for the time and attention of various people within a product team.\u00a0 Human nature is that if a \u00a0process is easy or rewarding, people will spend time on it.\u00a0 If it\u2019s not, they\u2019ll do as little of it as they can get away with.\u00a0 So the process evolves, because, unlike Dr No<\/a>, we want to be aligned with what our product groups and customers want<\/p>\n There have been a lot of variants of things called \u201cthreat modeling processes\u201d at Microsoft, and a lot more in the wide world.\u00a0\u00a0 People sometimes want to argue because they think Microsoft uses the term \u201cthreat modeling\u201d differently than the rest of the world.\u00a0 This is only a little accurate.\u00a0 There is a community which uses questions like \u201cwhat\u2019s your threat model\u201d to mean \u201cwhich attackers are you trying to stop?\u201d\u00a0 Microsoft uses threat model to mean \u201cwhich attacks are you trying to stop?\u201d\u00a0 There are other communities whose use is more like ours.\u00a0 In this paragraph, I\u2019m attempting to mitigate a denial of service threat, where prescriptivists<\/a> try to drag us into a long discussion of how we\u2019re using words.)\u00a0\u00a0 The processes I\u2019m critiquing here are the versions of threat modeling that are presented in Writing Secure Code<\/a>, Threat Modeling<\/a><\/i>, and The Security Development Lifecycle<\/i><\/a> books.<\/p>\n In this first post of a series on threat modeling, I\u2019m going to talk a lot about problems we had in the past.\u00a0 In the next posts, I\u2019ll talk about what the process looks like today, and why we\u2019ve made the changes we\u2019ve made.\u00a0\u00a0 I want to be really clear that I\u2019m not critiquing the people who have been threat modeling, or their work.\u00a0 A lot of people have put a tremendous amount of work in, and gotten some good results.\u00a0 There are all sorts of issues that our customers will never experience because of that work. \u00a0I am critiquing the processes, \u00a0saying we can do better, in places we are doing better, and I intend to ensure we continue to do better.<\/p>\n We ask feature teams to participate in threat modeling, rather than having a central team of security experts develop threat models.\u00a0 There\u2019s a large trade-off associated with this choice.\u00a0 The benefit is that everyone thinks about security early.\u00a0 The cost is that we have to be very prescriptive in how we advise people to approach the problem.\u00a0 Some people are great at \u201cthink like an attacker,\u201d but others have trouble.\u00a0\u00a0 Even for the people who are good at it, putting a \u00a0process in place is great for coverage, assurance and reproducibility.\u00a0 But the experts don\u2019t expose the cracks in a process in the same way as asking everyone to participate.<\/p>\n Getting Started<\/b><\/p>\n The first problem with \u2018the threat modeling process\u2019 is that there are a lot of processes.\u00a0 \u00a0People, eager to threat model, had a number of TM processes to choose from, which led to confusion.\u00a0 If you\u2019re a security expert, you might be able to select the right process.\u00a0 If you\u2019re not, judging and analyzing the processes might be a lot like analyzing cancer treatments.\u00a0 Drugs?\u00a0 Radiation?\u00a0 Surgery?\u00a0 It\u2019s scary, complex, and the wrong choice might lead to a lot of unnecessary pain.\u00a0\u00a0 You want expert advice, and you want the experts to agree.<\/p>\n Most of the threat modeling processes previously taught at Microsoft were long and complex, having as many as 11 steps.\u00a0 That\u2019s a lot of steps to remember.\u00a0 There are steps which are much easier if you\u2019re an expert who understands the process.\u00a0 For example, \u2018asset enumeration.\u2019\u00a0 Let\u2019s say you\u2019re threat modeling the GDI graphics library.\u00a0 What are the assets that GDI owns?\u00a0 A security expert might be able to answer the question, but anyone else will come to a screeching halt, and be unable to judge if they can skip this step and come back to it.\u00a0 (I\u2019ll come back to the effects of this in a later post.)<\/p>\n I wasn\u2019t around when the processes were created, and I don\u2019t think there\u2019s a lot of value in digging deeply into precisely how it got where it is.\u00a0 I believe the core issue is that people tried to bring proven techniques to a large audience, and didn\u2019t catch some of the problems as the audience changed from experts to novices.<\/p>\n The final problem people ran into as they tried to get started was an overload of jargon, and terms imported from security.\u00a0 We toss around terms like repudiation as if everyone should know what it means, and sometimes implied they\u2019re stupid if they don\u2019t.\u00a0 (Repudiation is claiming that you didn\u2019t do something.\u00a0 For example, \u201cI didn\u2019t write that email!,\u201d \u201cI don\u2019t know what got into me last night!\u201d\u00a0 You can repudiate something you really did, and you can repudiate something you didn\u2019t do.)\u00a0 Using jargon sent several unfortunate messages:<\/p>\n Of course, that wasn\u2019t the intent, but it often was the effect.<\/p>\n The Disconnected Process<\/b><\/p>\n Another set of problems is that threat modeling can feel disconnected from the development process.\u00a0 The extreme programming folks are fond of only doing what they need to do to ship, and Microsoft shipped code without threat models for a long time.\u00a0 The further something is from the process of building code, the less likely it is to be complete and up to date.\u00a0 That problem was made worse because there weren\u2019t a lot of people who would say \u201clet me see the threat model for that.\u201d\u00a0 \u00a0So there wasn\u2019t a lot of pressure to keep threat models up to date, even if teams had done a good job up front with them.\u00a0 There may be more pressure with other specs which are used by a broader set of people during development.<\/p>\n Validation<\/b><\/p>\n Once a team had started threat modeling, they had trouble knowing if they were doing a good job.\u00a0 Had they done enough?\u00a0 Was their threat model a good representation of the work they had done, or were planning to do?\u00a0 When we asked people to draw diagrams, we didn\u2019t tell them when they could stop, or what details didn\u2019t matter.\u00a0 When we asked them to brainstorm about threats, we didn\u2019t guide them as to how many they should find.\u00a0 When they found threats, what were they supposed to do about them?\u00a0 This was easier when there was an expert in the room to provide advice on how to mitigate the threat effectively. \u00a0\u00a0How should they track them?\u00a0 \u00a0Threats aren\u2019t quite bugs\u2014you can never remove a threat, only mitigate it.\u00a0 So perhaps it didn\u2019t make sense to track them like that, but that left threats in a limbo.<\/p>\n “Return on Investment”<\/b><\/p>\n \u00a0 The time invested often didn\u2019t seem like it was paying off.\u00a0 Sometimes it really didn\u2019t pay off.\u00a0\u00a0\u00a0 (David LeBlanc makes this point forcefully in \u201cThreat Modeling the Bold Button is Boring<\/a>\u201d) Sometimes it just felt that way\u2014Larry Osterman made that point, unintentionally in \u201cThreat Modeling Again, Presenting the PlaySound Threat Model<\/a>,\u201d where he said \u201cLet’s look at a slightly more interesting case where threat modeling exposes an issue.\u201d\u00a0 Youch!\u00a0 But as I wrote in a comment on that post, \u201cWhat you’ve been doing here is walking through a lot of possibilities.\u00a0 Some of those turn out to be uninteresting, and we learn something.\u00a0 Others (as we’ve discussed in email) were pretty clearly uninteresting\u201d\u00a0 It can be important to walk through those possibilities so we know they\u2019re uninteresting.\u00a0 Of course, we\u2019d like to reduce the time it takes to look at each uninteresting issue.<\/p>\n Other Problems<\/b><\/p>\n\n