![\"Saleema](\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/02\/Research_Podcast_Seleema_Site_1400x788-1.png\")
<\/p>\n<\/p>\n
There\u2019s an old adage that says if you fail to plan, you plan to fail. But when it comes to AI, Dr. Saleema Amershi (opens in new tab)<\/span><\/a>, a principal researcher in the Adaptive Systems and Interaction group (opens in new tab)<\/span><\/a> at Microsoft Research, contends that if you plan to fail, you\u2019re actually more likely to succeed! She\u2019s an advocate of calling failure what it is, getting ahead of it in the AI development cycle and making end-users a part of the process.<\/p>\n Today, Dr. Amershi talks about life at the intersection of AI and HCI and does a little AI myth-busting. She also gives us an overview of what \u2013 and who \u2013 it takes to build responsible AI systems and reveals how a personal desire to make her own life easier may make your life easier too.<\/p>\n Saleema\u00a0Amershi:\u00a0We\u2019re trying to make sure we think carefully and thoughtfully about how to build these systems, so in that sense we might need to slow things down, but we\u2019re also trying to push the boundaries, right?\u00a0That means like coming up with new techniques so we can then accelerate our progress. What are the new methodologies and techniques and tools we need to build so that we can still make rapid progress, but do so carefully?<\/p>\n Host:\u00a0<\/b>You\u2019re listening to the Microsoft Research Podcast, a show that brings you closer to the cutting-edge of technology research and the scientists behind it. I\u2019m your host, Gretchen Huizinga.<\/b><\/p>\n Host: There\u2019s an old adage that says if you fail to plan, you plan to fail. But when it comes to AI, Dr.\u00a0<\/b>Saleema<\/b>\u00a0<\/b>Amershi<\/b>, a principal researcher in the Adaptive Systems and Interaction group at Microsoft Research, contends that if you plan to fail, you\u2019re actually more likely to succeed! She\u2019s an advocate of calling failure what it is, getting ahead of it in the AI development cycle and making end-users a part of the process.<\/b><\/p>\n Today, Dr.\u00a0<\/b>Amershi<\/b>\u00a0talks about life at the intersection of AI and HCI<\/b>,<\/b>\u00a0and does a little AI myth-busting. She also gives us an overview of what \u2013 and who \u2013 it takes to build responsible AI systems<\/b>,<\/b>\u00a0and reveals how a personal desire to make her own life easier may make your life easier too. That and much more on this episode of the Microsoft Research Podcast.<\/b><\/p>\n (music plays)\u00a0<\/i><\/b><\/p>\n Host:\u00a0<\/b>Saleema<\/b>\u00a0<\/b>Amershi<\/b>, welcome to the podcast.<\/b><\/p>\n Saleema\u00a0Amershi: Thanks. Thanks for having me.<\/p>\n Host: You work in a group called the Adaptive Systems and Interaction Group and you\u2019re a principal researcher there. Tell us, in broad strokes, what you do for a living and why you do it? What gets you up in the morning?<\/b><\/p>\n Saleema\u00a0Amershi: Sure. So I work at the intersection of human computer interaction and artificial intelligence. And so I create technologies and techniques to help people both build and use AI systems. So that means, you know, I think a lot about developers and data scientists and the tools that they can use to create our AI systems, but also the end-users who ultimately will use and interact with AI systems in their everyday lives. And so I think about how we can make these systems,\u00a0and people interacting with them,\u00a0more efficient and effective. And then more recently, I\u2019ve\u00a0started to think a lot more about responsible AI and how we can help people\u00a0interact with these things safely and responsibly so that they can trust them and use them to help them in their everyday lives.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: And so, you know,\u00a0if I think about what wakes me up in the morning, it\u2019s this responsible AI stuff. It\u2019s both exciting and terrifying, you know? There\u2019s so much potential for AI to help people and society, but also a lot of potential for harm,\u00a0so that gives me a lot of work to do.<\/p>\n Host: Well, before we get specific, I want to get a little geeky, from an academic point of view, and talk about methodology. I don\u2019t talk about it that much on the show.\u00a0<\/b>And\u00a0<\/b>I think it would be interesting to kind of take a quick look at the tools that you use to equip ML software engineers with the tools they\u2019ll use.<\/b><\/p>\n Saleema\u00a0Amershi:\u00a0Mmm-hmmm.\u00a0So a lot of my work involves designing and building new interactive AI systems. So that means I end up using both design tools and prototyping tools as well as, you know,\u00a0development tools and machine learning tools to build AI models. And I think, you know, to build these systems effectively,\u00a0you really need to understand some of,\u00a0like,\u00a0how these AI and machine learning systems work, you know? You need to understand what knobs are available to give to people so that they can interact with them more effectively. So I think a lot of my work involves actually like building and developing systems. And then, in terms of methodologies, I use both qualitative and quantitative methods. You know, I really like to understand the needs of people before I start building things. That\u2019s really the, you know,\u00a0user centered\u00a0design approach, right? Where you really, like\u2026\u00a0all your decisions are based on user needs.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: And often times for that,\u00a0you use a lot of qualitative methods, interviewing techniques and surveys to get that sort of rich feedback. But at the same time, I come from a math background so I really like to see numbers and hard evidence,\u00a0so I also do a lot of quantitative research,\u00a0so controlled studies where we,\u00a0you know,\u00a0statistically compare things so I can, you know,\u00a0trust my own work and understand whether or not the things I build actually help\u00a0people.\u00a0I think there\u00a0are benefits and limitations to all these methods. You know, if you use quantitative techniques,\u00a0you really have to control a lot of variables and that means you can really only answer very narrow questions.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: Right. So yes, you can get statistical significance and numbers, but you don\u2019t really understand,\u00a0qualitatively, like,\u00a0why this is happening and why is this working better for\u00a0people or not?\u00a0And\u00a0I think I really like to use both methods in all the work I do. I like\u00a0to have a\u00a0quantitative and a\u00a0qualitative perspective because they really feed into each other and\u00a0that\u2019s how you can really understand\u00a0why things are working or not.<\/p>\n Host: Let\u2019s get into your work right now. There\u2019s a lot of discussion today about ethics in AI. I think it\u2019s because we\u2019re starting to see some of the ramifications of these systems that we\u2019re putting out in the real world. And Microsoft has actually been a leader in this space, so I want to talk about several threads of research you\u2019ve undertaken and the implications of your findings. And I want to start by setting the stage and operationalizing the term Responsible AI, or RAI. What is RAI, and what does it look like IRL, in the real world?<\/b><\/p>\n Saleema\u00a0Amershi: I\u2019m a really practical person,\u00a0so to me,\u00a0Responsible AI is really all about the how, right? Like,\u00a0how do we design, develop and deploy these systems that are fair, reliable, safe and trustworthy and all those things. And to do this, I really believe that we need to think of Responsible AI as a set of socio-technical problems, okay? So that means that we need to go beyond just data and models and making those better. We have to think about the people who are ultimately going to be interacting with these systems.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: Even if you collect a really huge, diverse data set and your models are tuned appropriately, if a person can\u2019t effectively understand the AI or, you know, take over control when it inevitably fails, that can also cause problems. So I think when we\u00a0create\u00a0responsible\u00a0AI\u00a0systems we need to think about these systems responsibly, which, you know, opens up many challenges, but also new opportunities.<\/p>\n Host: Right. I like to say that failure has a lot of faces and not all of them are ugly. You take it a step further and say that Responsible AI requires planning to fail! Why?<\/b><\/p>\n Saleema\u00a0Amershi:\u00a0Mmm-hmm.\u00a0Yeah, so this is something I\u2019ve been thinking about a lot lately and some types of failures are really inherent and unavoidable in AI. So yes, we should be doing our due diligence and trying to make sure we deploy systems that are, you know, as error-free as possible, that we\u2019ve debugged them carefully\u2026\u00a0we need to do all that work. But we also need to recognize that we can never get rid of all of these errors. And that\u2019s by design. So\u2026<\/p>\n Host: Wait\u2026<\/b><\/p>\n Saleema\u00a0Amershi:\u00a0\u2026let me give you\u2026<\/p>\n Host: Wait\u2026!<\/b><\/p>\n Saleema\u00a0Amershi: I can give you an example.<\/p>\n Host: Yeah, please<\/b>!<\/b><\/p>\n Saleema\u00a0Amershi: So imagine, you know, you have a facial recognition system that\u2019s used for access control. By design, an algorithm will be tuned to optimize for some metric. So maybe you try to optimize for precision or recall, which really, like,\u00a0affects the amount of false positive and false negative errors you can have.<\/p>\n Host: Okay.<\/b><\/p>\n Saleema\u00a0Amershi: You can never really get rid of all of those errors because, by definition, an AI model is a simplification of the world. You can never fully capture the world. So AI algorithms are designed to do the best\u00a0under\u00a0the circumstances. And that means it\u2019ll sacrifice parts of the input space to try to get something that\u2019s optimized accordingly. And so that means you will definitely have some false positives and false negatives. The algorithm will try to determine a model that generalizes\u00a0well to\u00a0new data and may sacrifice parts of the input space.<\/p>\n Host: Okay.<\/b><\/p>\n Saleema\u00a0Amershi: And so the choice of parameters you use,\u00a0or thresholds you use,\u00a0is really important and you really need to think about the user scenario there. So if you get that choice wrong, that\u2019s going to be costly to the users. So in the facial recognition scenario, false positives are much worse than false negatives, right? If somebody accesses your account, that can be much more costly than if you have trouble accessing it yourself.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: So if you don\u2019t get that right, that\u2019s a failure. In the same sense, you can\u2019t avoid all false positives and false negatives,\u00a0so you need to ensure that you give people mechanisms to not only understand when those are happening, but also to override the system,\u00a0or take over control when they inevitably happen. So if you can\u2019t get into your system, provide another means of accessing your system that doesn\u2019t rely on that technology. So anticipating those failures, as designers and developers, if you recognize these common AI failures and make sure you design interfaces and your systems to help people address those failures,\u00a0that\u2019s how we can work towards creating responsible AI systems.<\/p>\n Host: Go back to the \u201cplanning for failure\u201d then<\/b>,<\/b>\u00a0in the design part of that. What does that mean?<\/b><\/p>\n Saleema\u00a0Amershi: So that\u2019s about, I would say, enumerating the common types of AI failures. So false positives, false negatives, being uncertain, being only partially correct, but also, again, thinking of AI systems as socio-technical problems, right? So that means going beyond just the\u00a0model errors themselves, but places where the system can fail in terms of how the user is able to interact with them.<\/p>\n Host: Okay.<\/b><\/p>\n Saleema\u00a0Amershi: So again, like the mismatched expectations issue, right? I would consider that a failure. If a person has higher expectations of the system than it\u2019s capable of, that\u2019s a failure, right?<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: Because if a doctor is relying on a system to\u00a0make\u00a0clinical recommendations\u00a0about\u00a0their patients, if they think that the system is smarter than it\u00a0actually\u00a0is,\u00a0they may over-rely on it and that can result in harms.<\/p>\n Host: So what\u2019s the mitigation there because you\u2019ve got the system that\u2019s going to fail\u2026 is it more educating up front?<\/b><\/p>\n Saleema\u00a0Amershi: This is where we can get creative as an industry. For some of these,\u00a0we often go to, let\u2019s just give the people all the documentation, right? Like list out everything, but nobody reads documentation, right?<\/p>\n Host: I was just going to say, I don\u2019t even read the apps.<\/b><\/p>\n Saleema\u00a0Amershi: Exactly, right?\u00a0And so what are other ways that we can sort of expose the capabilities and limitations of these systems?\u00a0We\u2019ve done a lot of work in this space and trying to understand what is effective for this and other types of failures. So showing examples of the variety of things that an AI system can do effectively is a way of giving people an understanding of their capabilities.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: Giving people controls to turn the knobs themselves. That gives them an understanding, but it also makes them part of the process and so they\u2019re sort of more accepting when these things inevitably break\u00a0down and are more willing to interact with them,\u00a0and continue to interact with them,\u00a0because they are part of that process.<\/p>\n (music plays)<\/i><\/b><\/p>\n Host: Let\u2019s zoom in and talk about Responsible AI, writ large, and kind of following up on some of the points that you just made about planning for failure and understanding that these things fail. I know that there are some myths out there about it and some hurdles that\u00a0<\/b>people have to overcome, both on the making side and the using side. So talk sort of generally about what we\u2019re facing here, and what we need to do to build Responsible AI. What are the necessary building blocks?<\/b><\/p>\n Saleema\u00a0Amershi: Yeah, this is a great question and\u00a0it\u2019s\u00a0something we\u2019ve really started to look into recently. We\u2019ve started to do some preliminary work to understand sort of the challenges people face in trying to build responsible AI systems.<\/p>\n Host: Yeah.<\/b><\/p>\n Saleema\u00a0Amershi: And also the perceptions that people have about responsible AI issues around like bias and fairness and, you know, some of the things that we hear about in the news. And some of what we\u2019ve been finding is really interesting. There\u2019s AIs now that are being used to making hiring decisions or recommendations. So imagine you\u2019re using an AI system to make recommendations about who to hire for a technology job,\u00a0or an engineering job. You know, we\u2019ve seen in the news, sometimes\u00a0these systems can be biased and so maybe your system is biased against recommending women for engineering positions. If you ask people about this\u00a0fairness and bias issue a lot of people will come and say, you know, well this is just reality. If we try to fix this, we\u2019re actually just adding our own biases and who are we to change reality?<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: And this goes back to, you know, what I was saying earlier, which is that,\u00a0this is not reality, right? Like AI models are,\u00a0by definition,\u00a0a simplification of the world. Like we cannot represent the world and all the factors that will impact whether or not a person will be a good hire. We can\u2019t represent all of that.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: And so,\u00a0while it might be true that there\u2019s, you know, gender disparity in technology, it\u2019s incorrect to say that this is reality. Another thing that we hear is that, you know, this is just math, right? AI is math,\u00a0so it can\u2019t be wrong, you know? And yes, it\u2019s true,\u00a0like the math is designed to do what you tell it to do, but again, going back to what I was saying earlier, the AI is designed to do the best under the circumstances.\u00a0And because you can\u2019t fully capture the real world, that means that it will try to, you know, minimize errors. Not eliminate them, but minimize them, right? And so even if the math is doing what it\u2019s supposed to, you know, it\u2019s operating over data and that data can be limited, how you show that\u00a0information\u00a0to the user can cause problems and failures. And so the math might not be wrong, but the AI system,\u00a0overall,\u00a0could still be wrong.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi:\u00a0I think we should think about these humans and machines as having different error regions, right? So yes, each of these systems will have uncertainty, but they\u2019ll be different. And so ideally,\u00a0those uncertainty regions don\u2019t overlap\u00a0that\u00a0much, right? And then you can complement each other effectively.<\/p>\n Host: Right, right, right.<\/b><\/p>\n Saleema\u00a0Amershi: And it\u2019s true because AI systems can see things that people can\u2019t, right? And vice versa. And so I think that\u2019s the way we should be looking at these and,\u00a0you know,\u00a0thinking about what is that overlapping region and ensuring that people understand sort of the limitations of those systems and where it might fail versus where you might fail.<\/p>\n Host: What are we facing on the making end of this? We\u2019re talking here about how these systems operate with users. Are there any challenges that we face as developers?<\/b><\/p>\n Saleema\u00a0Amershi: I think this is where a lot of the work needs to be done. We have to, I think, re-think how we go about building these systems.\u00a0I\u2019m a firm believer that building responsible AI systems requires an interdisciplinary approach. And so for example, a lot of times we put a lot of our emphasis and resources towards building better models and getting better data, but again, you know, these are socio-technical systems. So we also have to think about how those systems will be deployed in the world and who it\u2019s going to affect. Who are the different stakeholders? What are the implications to those different stakeholders when things go wrong?\u00a0And I think that really requires a user-centered approach.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: And so we should be leveraging, you know, the skills and expertise of, for example, our user researchers and designers who having the training to sort of understand the needs of people. And that understanding should be driving all of our AI decisions, including,\u00a0you know,\u00a0what algorithms to use. If you need a system that can provide an explanation to a user so that they can make an appropriate decision, you\u2019re going to need an interpretable model.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: So that\u2019s going to affect the choice of algorithm you use. Understanding the users is going to affect the parameters you choose,\u00a0the data you collect, right? All of those decisions should be driven by the scenario. And I think sometimes we do it the reverse way in the industry, right?<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: We build technologies and then sort of stick an interface on it and hope it works for people.\u00a0But I think,\u00a0to do it responsibly, we need to do the reverse.<\/p>\n Host: So there\u2019s a level of opacity I think in terms of form factor when we deliver an AI system. When I think of what I get now, it\u2019s a laptop, it\u2019s a phone\u2026 but inside of that is where these AI systems are being deployed and so I don\u2019t necessarily differentiate between my phone that used to do what it did and now my phone\u00a0<\/b>that<\/b>\u00a0now I can talk to. Or my phone that looks at my face and says okay, Gretchen, you can come in. How do you think about design on those things that alert users that they\u2019re now dealing with AI and how do you educate about that?<\/b><\/p>\n Saleema\u00a0Amershi: AI systems are fundamentally different than our traditional computing systems and I think our practitioners and our product teams are sort of\u00a0really\u00a0struggling to design effective systems because of those differences. What we know about how to design effective traditional computing systems, like making sure your\u00a0interfaces are consistent, or your systems behave\u00a0consistently,\u00a0so people know how to use them,\u00a0know what to expect.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: That\u2019s something that\u2019s inherently very difficult for AI systems because they can operate differently in subtly different contexts\u00a0or from one user to the next.\u00a0How do we design for that? Where like I think we need to come up with a lot more guidance to help our teams understand what is the effective way of designing these systems so people can interact with them effectively.<\/p>\n Host:\u00a0<\/b>Right.\u00a0<\/b>Bill Buxton, design guru here at Microsoft Research, was on the podcast and he talked a lot about the need for great design from the get-go and spoke about who gets to make decisions about how something is designed.\u00a0<\/b>And we\u2019re not just talking about the form factor<\/b>, w<\/b>e\u2019re talking about the entire package.\u00a0<\/b>And so<\/b>,<\/b>\u00a0with these new AI systems, how can we bring new people to the table that really need to speak to the design up front into that traditional system?<\/b><\/p>\n Saleema\u00a0Amershi: Yeah, this is something I think about a lot and it requires really a cultural shift, right? It\u2019s about recognizing and understanding the skills and expertise that each of these different disciplines brings to the table and how they can complement each other in order to create responsible systems. That is just something that requires education. It requires trying it out, right? Like,\u00a0hey, if you do bring people in earlier, you\u2019ll likely create a better system because now your decisions are driven by what\u2019s actually going to be useful for people. I\u2019m actually really hopeful right now. I joined MSR about\u00a0seven\u00a0years ago in the machine learning group and I think things were much different back then, right? You know, it was much harder for me to explain,\u00a0like,\u00a0what I brought to the table for machine learning, you know, and why I was even there, you know? I was the first like HCI hire in the machine learning group, and it took me\u00a0a while before people really understood what the complementary skills that I brought to the table that helped, you know, make these things effective.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: And I think people are more open to that now so even though it requires this shift in our industry, I\u2019m hopeful that it will happen.<\/p>\n Host: Right. Well, and Bill talked quite a bit about the fact that the arguments that you hear are\u00a0<\/b>\u201c<\/b>we don\u2019t have enough time, we don\u2019t have enough money, we need to ship<\/b>\u2026\u201d<\/b>\u00a0There are constraints that are inherent in that cycle.<\/b><\/p>\n Saleema\u00a0Amershi: There are ways to plan for this, right? Like reserving some resources for dealing with responsible AI issues, you know. If you know they\u2019re going to be there,\u00a0so if you reserve some resources for that then you don\u2019t run into this problem of we don\u2019t have enough time or resources.\u00a0I think another thing that helps is calling these issues failures. I use the word failures really intentionally because we used to call responsible AI issues, you know,\u00a0issues\u00a0or\u00a0problems, but just that term will put it lower in the priority stack.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: Right? But if you call it a failure, like if people might get harmed or\u00a0if\u00a0there\u2019s a bias against people, like that\u2019s a failure. That\u2019s something that\u2019s a showstopper, right?\u00a0You\u2019re not going to deploy something that has a failure.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: So I think it\u2019s important to talk about it that way so\u00a0that\u00a0we ensure that we\u2019re actually prioritizing fixing them.<\/p>\n Host: AI brings both new challenges and new opportunities in innovation in user interface and user experience. You address\u00a0<\/b>this<\/b>\u00a0issue in a paper that proposes guidelines for human-AI interaction from a research perspective, a development perspective and a user perspective, which I think is cool. Talk about the genesis of this work and the findings that you presented at a paper in CHI this year.<\/b><\/p>\n Saleema\u00a0Amershi: So we created the guidelines because we were really seeing that our product teams and practitioners were struggling to design for AI. And I spoke about this a bit earlier, which is, you know,\u00a0this fact that AI systems are fundamentally different from traditional computing systems. So they\u2019re going to be inconsistent. They\u2019re going to be error prone. So what we know about designing for traditional systems doesn\u2019t always work. This is evidenced by just the failures that we see every day in the news, you know, that range from\u00a0like\u00a0funny,\u00a0like,\u00a0auto-completion errors to\u00a0like\u00a0really harmful errors.\u00a0And at the same time, there\u2019s actually been a lot of research advances over the last like twenty years around how to develop these types of AI systems effectively. And for me, that was somewhat frustrating,\u00a0seeing sort of people struggle, and missing out on some of what\u2019s been going on in the academic community or industrial circles, and I felt\u00a0that\u00a0there was a lot of sort of reinventing the wheel and wasted time. And that\u2019s, you know, partly because guidance is really scattered across many different industrial circles. Guidance that is available hasn\u2019t always been evaluated in a wide variety of scenarios,\u00a0so if you know something works well for like a bot,\u00a0how do you know it\u2019s going to work well for any other AI system? And sometimes guidance is presented at very different altitudes. Like you can have really high-level guidance like,\u00a0make sure these things are, you know, trustworthy, but like how do you do that, right?<\/p>\n Host: What does that even mean?<\/b><\/p>\n Saleema\u00a0Amershi: Exactly, right? Like so,\u00a0we wanted to provide guidance that was more actionable, right?<\/p>\n Host: Sure.<\/b><\/p>\n Saleema\u00a0Amershi: So we got together with a large group of people and said like, hey, let\u2019s just try to synthesize what we know across the industry\u00a0and\u00a0come up with guidance that\u2019s clear, actionable and that we know is relevant to a wide variety of scenarios. So we went through this iterative process. We collected guidance and best practice recommendations from a wide variety of sources. I think we had like two hundred to begin with, and then we iteratively grouped them, revised them, tested them with real practitioners to understand if they were really something that people could detect and notice in interfaces,\u00a0and that\u2019s how we developed them. We tried to take a\u00a0really\u00a0rigorous and systematic approach so that we could, you know,\u00a0feel confident in recommending these as things that we know are tried and true. And I think that helps both, you know, researchers, developers, and end-users, you know. It helps practitioners design better systems.\u00a0That gives end-users better systems to use. And I think it also helps accelerate research because,\u00a0like I said,\u00a0you know,\u00a0I felt that we were sort of reinventing the wheel and I think,\u00a0by synthesizing this work,\u00a0it can kind of reveal where the real gaps in our knowledge are and so we can try to,\u00a0then,\u00a0really push the boundaries into what we don\u2019t know.<\/p>\n Host: Who\u2019s this for? I mean, when you say guidelines for human-AI interaction, who\u2019s your audience?<\/b><\/p>\n Saleema\u00a0Amershi: I would say primarily practitioners. So product teams.\u00a0Like, we want them to, you know, have the knowledge to create good systems. But also researchers. Like I said, you know, I want to really advance the field.\u00a0Here\u2019s what we know now, let\u2019s figure out and improve the situation and do research in areas that we have less knowledge.<\/p>\n Host: You currently chair a really interesting working group at Microsoft on human-AI interaction and collaboration<\/b>,<\/b>\u00a0and it\u2019s a part of Microsoft\u2019s\u00a0<\/b>Aether<\/b>\u00a0effort, which is a company-wide initiative around\u00a0<\/b>R<\/b>esponsible AI. And I know there\u2019s a lot of Venn diagram overlap with this and what we\u2019ve been talking about up to this point, but I want you to drill in a little bit on why the issue of human-AI collaboration warrants an actual group<\/b>,<\/b>\u00a0or task force, maybe<\/b>,<\/b>\u00a0is a good word, and who\u2019s involved?<\/b><\/p>\n Saleema\u00a0Amershi:\u00a0Like you said,\u00a0you know,\u00a0much of what I\u2019ve talked about today,\u00a0actually came out of this working group. So\u00a0the,\u00a0you know, the studies about people\u2019s perceptions around responsible AI failures, the guidelines work\u2026\u00a0all came out of this\u00a0Human-AI\u00a0Interaction and\u00a0Collaboration working group, which we call the HAIC working group. And, you know, just like\u00a0you know,\u00a0any new set of problems, this is a new space, right? Like we don\u2019t know how to do human-AI interaction and collaboration well yet. And so we need people to be really trying to push the boundaries of this area, you know? People with the right expertise in order to make advances.\u00a0So that includes, you know, coming up with new best practices and techniques, but also even just advancing the state-of-the-art in terms of the methodologies we use. That\u2019s something\u00a0that\u00a0we\u2019ve been looking into recently is that a lot of our techniques and methodologies for building traditional systems don\u2019t work that well. So, you know,\u00a0we have,\u00a0in design, we have prototyping techniques like Wizard of Oz techniques, for example, that\u2019s often used to do early prototyping and testing. But that\u2019s really hard with AI systems because it\u2019s hard to mimic the different ways an AI system will behave. But that means,\u00a0if you don\u2019t have that in place,\u00a0then you need to take a dependency on a model, which means that you can\u2019t sort of test your interface early,\u00a0and that causes problems, right?<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi:\u00a0So we really need experts, people who understand human-computer interaction, user research methodologies and AI systems to think deeply about new methodologies to enable rapid prototyping and iteration and other methodologies for evaluation, testing and building AI systems.<\/p>\n Host:\u00a0<\/b>Let me drill in a little bit there<\/b>. T<\/b>echnology advances\u00a0<\/b>at a<\/b>\u00a0very rapid pace now<\/b>,\u00a0<\/b>maybe faster than it ever has<\/b>,<\/b>\u00a0and you\u2019re a group who are a little bit putting the brak<\/b>e<\/b>s on and saying, wait, we need to make sure this isn\u2019t going to harm anybod<\/b>y.\u00a0<\/b>How much influence do you have among the various groups of people that are putting this technology out?<\/b><\/p>\n Saleema\u00a0Amershi: So I would characterize it not necessarily that we\u2019re putting the brakes on things. Yes,\u00a0we\u2019re trying to make sure we think carefully and thoughtfully about how to build these systems, so in that sense we might need to slow things down, but we\u2019re also trying to push the boundaries, right?\u00a0That means like coming up with new techniques so we can then accelerate our progress.\u00a0What are the new methodologies and techniques and tools we need to build so that\u00a0we can still make rapid progress, but do so\u00a0carefully?\u00a0So I kind of see us as,\u00a0you know,\u00a0partially pressing the brakes, but partially, you know, pressing the accelerator!<\/p>\n (music plays)<\/i><\/b><\/p>\n Host: Well,\u00a0<\/b>Saleema<\/b>, we\u2019ve reached the part of the podcast where I ask all my guests the same thing: what keeps you up at night? And obviously, a lot of your work is based on what keeps all of us up at night. I\u2019m glad you\u2019re doing it. Ethical AI is a huge issue here and you\u2019re tackling it head on. But that said, not everyone is going to comply with best practices, so what kinds of things can be done to mitigate the undesirable consequences of these powerful tools and ensure that I don\u2019t lose any sleep at night?<\/b><\/p>\n Saleema\u00a0Amershi: Yeah, you know, I mean, this space is,\u00a0like I said earlier, it\u2019s both exciting and terrifying for me, you know? And I really believe that to create ethical and responsible systems,\u00a0we really need a diverse set of perspectives at the table, right? This is both at the macro level and the micro level. You know, the macro level, in terms of coming up with policies, like I think we need policies around this, but that needs to be driven both by industries and government agencies working together, right? If you have just one of those entities making decisions,\u00a0sometimes you\u2019ll come up with things that just don\u2019t work.<\/p>\n Host: Right.<\/b><\/p>\n Saleema\u00a0Amershi: And then at the micro level,\u00a0you know,\u00a0building\u00a0individual products. I really feel that we need people with not only diverse backgrounds in terms of, you know, race and ethnicity and gender, but also different skill sets, you know? People with different experiences, different tools and methodologies that you use. How do we enable these people to work together? And I think that\u2019s going to require a cultural shift, which is hard to do, but, you know,\u00a0I\u2019m trying my best! You know, the\u00a0Aether\u00a0working group and HAIC,\u00a0Human-AI\u00a0Interaction and\u00a0Collaboration, this is sort of what we are really trying to do.<\/p>\n Host:\u00a0<\/b>Microsoft<\/b>\u00a0Research is not a monolith. People come from all over here. Different backgrounds and life experiences and unique personal stories. So tell us yours,\u00a0<\/b>Saleema<\/b>. What got you started in computer science and what landed you here at Microsoft Research?<\/b><\/p>\n Saleema\u00a0Amershi: I did my PhD at the University of Washington, which is really just across the lake from Redmond here, and so that made it really easy to collaborate with MSR and there was just so many interesting people to work with.\u00a0And so I ended up doing three internships here at MSR. I just kept coming back. And even,\u00a0like,\u00a0when I wasn\u2019t doing internships, I would collaborate with people. I just always loved it. I loved the energy, the breadth of experience and expertise,\u00a0and everyone is just willing to talk to you and work together,\u00a0and I sort of always\u00a0knew that I wanted to come back here. And so after grad school, you know, I applied, came here.\u00a0When I first started people would ask me if I was, you know, back for another internship.<\/p>\n Host: You again?<\/b>!<\/b><\/p>\n Saleema\u00a0Amershi: No, I\u2019m really working here now!<\/p>\n Host: Well, rewind before University of Washington and your PhD. What got a young\u00a0<\/b>Saleema<\/b>\u00a0<\/b>Amershi<\/b>\u00a0interested? Where did you start? Are you from Washington\u00a0<\/b>s<\/b>tate<\/b>,\u00a0<\/b>e<\/b>tc<\/b>?<\/b><\/p>\n Saleema\u00a0Amershi: So I grew up in Vancouver, B.C.\u00a0Ummm,\u00a0yup!<\/p>\n Host: I did not know that.<\/b><\/p>\n Saleema\u00a0Amershi: Yeah, so I went to UBC for undergrad and my masters,\u00a0and I actually started off as a math major. In fact, I never thought I would go into computer science. You know, it wasn\u2019t like, computers were just sort of becoming common in high schools and I wasn\u2019t really exposed to it that much, but I really liked math.\u00a0That\u2019s\u00a0what I wanted to do. I wanted to be a math professor. And so when I started undergrad, at the time, you know,\u00a0you had to take computer science courses as part of your math major,\u00a0and that\u2019s,\u00a0I think,\u00a0when I got really exposed to computer science, which is really about putting math to work. You know, it\u2019s about making math do things. And do cool things, you know? And that\u2019s when I started transitioning to computer science. And then I started working at the Laboratory for Computational Intelligence at UBC, working on intelligent tutoring systems, right? That\u2019s where I got exposed to AI systems, and then, as I was building those systems, I found it was hard to do, you know? So I was trying to make my life easier, you know, by building better tools and that\u2019s kind of what let me to HCI and here I am working now\u00a0on\u00a0this intersection.<\/p>\n Host: What\u2019s one interesting thing<\/b>\u00a0\u2013<\/b>\u00a0a trait, a characteristic, a life event, a side quest<\/b>\u00a0\u2013\u00a0<\/b>that people might not know about you and maybe it\u2019s impacted your career as a researcher?<\/b><\/p>\n Saleema\u00a0Amershi: I think of myself as a pragmatist and most of what\u2019s driven my work, I would say, and driven the path that I took was trying to make my own life easier, you know? So when I was working with intelligent tutoring systems back at UBC, I remember at that time,\u00a0to build these things you would create,\u00a0like,\u00a0these giant belief networks that were hand-tuned by experts for just one system.\u00a0And that was like,\u00a0yes, it was powerful, but it was not scalable.\u00a0And that\u2019s what got me into machine learning, right? It\u2019s like okay, how do we do this without having to hand-tune all these things? So that\u2019s how I started using machine learning for intelligent tutoring systems. Then,\u00a0when I was\u00a0using machine learning, like I said, you know, the tools that we had for those for, you know, data collection and cleaning,\u00a0and understanding and debugging, were just really hard to use.\u00a0And it was really difficult. There was a steep learning curve.\u00a0So that\u2019s what got me into interactive machine learning\u00a0and HCI. You know, because I wanted to create\u00a0better tools for myself, you know?\u00a0So I could\u00a0build these things more efficiently and effectively. And then, you know,\u00a0at the same time, I\u2019m not, you know I build these systems, but I\u2019m not a user, right? So when I interact with these AI systems in my everyday life, you know, like social networking systems or recommender systems, it really frustrates me when I can\u2019t do what I want, you know? I can\u2019t steer these things the way I want.\u00a0And I think that\u2019s about giving everyday people the right controls and knobs in order to steer these things. I think there\u2019s a myth that people won\u2019t want to put in the time and effort to interact with these or steer their systems, but I don\u2019t think that\u2019s true, you know. I think, if people start to understand the benefits of doing so and if you give them easy controls,\u00a0they\u2019d be willing to do so. And so really it\u2019s about helping myself. You know, making my life easier, which in turn will help other people\u00a0build and use these things effectively.<\/p>\n Host: At the end of every show I give my guests a chance to encourage, inspire or even instruct our listeners pretty well in any way they see fit. Do you have any parting words? Any thoughts on what we might need\u00a0<\/b>from<\/i><\/b>\u00a0next gen researchers\u00a0<\/b>for<\/i><\/b>\u00a0next gen technologies?<\/b><\/p>\n Saleema\u00a0Amershi: Yeah, I really believe that there\u2019s so many interesting opportunities to work at the intersection of different fields. There\u2019s a lot of opportunities to bridge different communities, enable them to work together more effectively to create really novel solutions to our problems. So, you know,\u00a0what I would recommend for, you know,\u00a0the\u00a0students out there,\u00a0the next generation of researchers,\u00a0is to explore those opportunities and work across interdisciplinary boundaries, you know, train yourself in multiple different fields so you can understand problems that might be solved by bringing those together. I think that could really help change the world.<\/p>\n Host:\u00a0<\/b>Saleema<\/b>\u00a0<\/b>Amershi<\/b>, thank you so much for joining us today!<\/b><\/p>\n Saleema\u00a0Amershi: Thank you for having me!<\/p>\n (music plays)<\/i><\/b><\/p>\n To learn more about Dr.\u00a0<\/i><\/b>Saleema<\/i><\/b>\u00a0<\/i><\/b>Amershi<\/i><\/b>\u00a0and how researchers are working to make AI robust and responsible, visit Microsoft.com\/research<\/i><\/b><\/p>\n","protected":false},"excerpt":{"rendered":" There\u2019s an old adage that says if you fail to plan, you plan to fail. But when it comes to AI, Dr. Saleema Amershi, a principal researcher in the Adaptive Systems and Interaction group at Microsoft Research, contends that if you plan to fail, you\u2019re actually more likely to succeed! She\u2019s an advocate of calling failure what it is, getting ahead of it in the AI development cycle and making end-users a part of the process. Today, Dr. Amershi talks about life at the intersection of AI and HCI and does a little AI myth-busting. She also gives us an overview of what \u2013 and who \u2013 it takes to build responsible AI systems and reveals how a personal desire to make her own life easier may make your life easier too.<\/p>\n","protected":false},"author":39507,"featured_media":634926,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"https:\/\/player.blubrry.com\/id\/54626645\/","msr-podcast-episode":"105","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[240054],"tags":[],"research-area":[13556,13563,13554],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-632772","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-msr-podcast","msr-research-area-artificial-intelligence","msr-research-area-data-platform-analytics","msr-research-area-human-computer-interaction","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"https:\/\/player.blubrry.com\/id\/54626645\/","podcast_episode":"105","msr_research_lab":[199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144633],"related-projects":[],"related-events":[],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"Related:<\/h3>\n
\n
\nTranscript<\/h3>\n
![\"Saleema](\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/02\/Research_Podcast_Seleema_Site_1400x788-1-960x540.png\")
","byline":"","formattedDate":"February 5, 2020","formattedExcerpt":"There\u2019s an old adage that says if you fail to plan, you plan to fail. But when it comes to AI, Dr. Saleema Amershi, a principal researcher in the Adaptive Systems and Interaction group at Microsoft Research, contends that if you plan to fail, you\u2019re…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/632772"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/39507"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=632772"}],"version-history":[{"count":7,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/632772\/revisions"}],"predecessor-version":[{"id":886944,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/632772\/revisions\/886944"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/634926"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=632772"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=632772"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=632772"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=632772"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=632772"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=632772"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=632772"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=632772"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=632772"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=632772"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=632772"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}