![\"Headshot](\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/03\/Research_Podcast_BehnazArzani_Site_1400x788-1024x576.png\")
<\/h3>\n<\/h3>\nDr. Behnaz Arzani (opens in new tab)<\/span><\/a> is a senior researcher in the Mobility and Networking group (opens in new tab)<\/span><\/a> at MSR, and she feels your pain. At least, that is, if you\u2019re a network operator trying to troubleshoot an incident in a datacenter. Her research is all about getting networks to manage themselves, so your life is as pain-free as possible.<\/p>\n On today\u2019s podcast, Dr. Arzani tells us why it\u2019s so hard to identify and resolve networking problems and then explains how content-aware, or domain-customized, auto ML frameworks might help. She also tells us what she means when she says she wants to get humans out of the loop, and reveals how a competitive streak and a comment from her high school principal set her on the path to a career in high tech research.<\/p>\n Behnaz\u00a0Arzani:\u00a0Humans are great at innovating and building stuff, but when it comes to figuring out what went wrong, and how it went wrong, and fixing things, it\u2019s much better to have automation do that than humans do that because we take our sweet time with things. And we also don\u2019t have the mental power to process so much data that\u2019s out there all at once. Machines are much better at doing that.<\/p>\n Host:\u00a0<\/b>You\u2019re listening to the Microsoft Research Podcast, a show that brings you closer to the cutting-edge of technology research and the scientists behind it. I\u2019m your host, Gretchen Huizinga.<\/b><\/p>\n Host: Dr.\u00a0<\/b>Behnaz<\/b>\u00a0<\/b>Arzani<\/b>\u00a0is a senior researcher in the Mobility and Networking group at MSR,\u00a0<\/b>and she feels your pain<\/b>. At least, that is, if you\u2019re a network operator trying to troubleshoot an incident in a datacenter. Her research is all about getting networks to manage themselves, so your life is as pain-free as possible.<\/b><\/p>\n On today\u2019s podcast, Dr.\u00a0<\/b>Arzani<\/b>\u00a0tells us why it\u2019s so hard to identify and resolve networking problems and then explains how content-aware, or domain-customized, auto ML frameworks might help. She also tells us what she means when she says she wants to get humans out of the loop, and reveals how a competitive streak and a comment from her high school principal set her on the path to a career in high tech research.<\/b>\u00a0<\/b>That and much more on this episode of the Microsoft Research Podcast.<\/b><\/p>\n Host:\u00a0<\/b>Behnaz<\/b>\u00a0<\/b>Arzani<\/b>, welcome to the podcast.<\/b><\/p>\n Behnaz\u00a0Arzani: Hey, thanks for having me.<\/p>\n Host: I like to start situating.<\/b><\/p>\n Behnaz\u00a0Arzani: Mm-hmm.<\/p>\n Host: And you\u2019re a senior researcher at MSR and you work in Mobility and Networking.<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: But that falls under the bigger umbrella of\u00a0<\/b>S<\/b>ystems and\u00a0<\/b>N<\/b>etworking.<\/b><\/p>\n Behnaz\u00a0Arzani: Mm-hmm.<\/p>\n Host: So to kick off our conversation, give us the elevator pitch version of what you all are up to. What\u2019s the big goal of Mobility and Networking and how does it fit into the broader ecosystem of Systems and Networking?<\/b><\/p>\n Behnaz\u00a0Arzani: Right. So I guess Mobility and Networking, as the name suggests, goes into two parts of\u00a0mobility and\u00a0networking. So half of our groups are doing things like IoT research, edge research, things that have to do with mobile phones and things like that,\u00a0or,\u00a0like,\u00a0devices.\u00a0And what people like me do are more of the networking aspects. So every distributed system that is out there has an underlying network,\u00a0and so our job is to try\u00a0to\u00a0figure out how to operate those networks properly and how to make those networks work the best way possible.<\/p>\n Host: So what would you say the big<\/b>,<\/b>\u00a0audacious goal of Systems and Networking or Mobility and Networking is, sort of, writ large?<\/b><\/p>\n Behnaz\u00a0Arzani: So I think every person you ask is going to give you a different answer on that. My particular take on this is that we\u2019re the infrastructure behind a lot of the systems that you see out there, from the web that you access to the storage systems that you use to everything else,\u00a0and so our job is to make this as seamless as possible. So you shouldn\u2019t even know\u00a0that\u00a0these networks are there. You should just use it and expect that they work properly.<\/p>\n Host: Well let\u2019s talk about what gets\u00a0<\/b>you<\/i><\/b>\u00a0up in the morning\u2026<\/b><\/p>\n Behnaz\u00a0Arzani: Mm-hmm.<\/p>\n Host:<\/b>\u00a0\u2026and where you situate yourself as a researcher. What\u2019s your big goal,\u00a0<\/b>Behnaz<\/b>, as a scientist, and what do you want to be known for at the end of your career?<\/b><\/p>\n Behnaz\u00a0Arzani:\u00a0Yeah, so, when I think about this\u00a0\u2013\u00a0and this was mainly after doing many, many internships in Azure Networking and seeing what operators have to deal with every day\u00a0\u2013\u00a0and\u00a0to me it seems like the worthy goal,\u00a0or what I really want to achieve,\u00a0is something where the life of an operator, a network operator, is as painless as possible because it can get painful on days. Especially if there is, you know, something broke and they have to figure out what happened. It can be a nightmare and what I would like to see is that they don\u2019t have to do that.<\/p>\n Host: And we will get into how you\u2019re going about that shortly. Let\u2019s start with one of the things that is kind of interesting to me. Many people I\u2019ve talked to on this podcast emphasize the importance of keeping humans in the loop.<\/b><\/p>\n Behnaz\u00a0Arzani: Mm-hmm.<\/p>\n Host: But you suggest<\/b>,<\/b>\u00a0in some ways<\/b>, and<\/b>\u00a0for some problems<\/b>,<\/b>\u00a0we actually need to get humans out of the loop or at least you question why, after so many years, you still have so many humans trying to figure so much of this all out. So when you say, get humans out of the loop, what do you mean<\/b>,<\/b>\u00a0and then how does it play out in the work you\u2019re doing?<\/b><\/p>\n Behnaz\u00a0Arzani: Right. So I think, depends on what do you get the humans out of the loop\u00a0of<\/i>. I think, to me,\u00a0humans are great at innovating and building stuff, but when it comes to figuring out what went wrong, and how it went wrong, and fixing things, it\u2019s much better to have automation do that than humans do that because we take our sweet time with things. And we also don\u2019t have the mental power to process so much data that\u2019s out there all at once. Machines are much better at doing that.\u00a0And what I keep seeing is that we as humans are very, very inefficient and what that causes is that our customers, often, are in pain while humans are trying to figure things out like that,\u00a0and so why do we have to do that after so many years of having networks out there and I think it\u2019s because this particular problem is just a really, really hard problem to solve. And so I find that both exciting and hard and so it\u2019s a challenge that\u2019s worth pursuing.<\/p>\n Host: Hasn\u2019t it gotten more complicated? Rather than saying, you know, we should have had this figured out by now it\u2019s like well, the internet\u00a0<\/b>threw<\/b>\u00a0us a lot more problems. The cloud has thrown us a bigger problem. How would you answer that?<\/b><\/p>\n Behnaz\u00a0Arzani: A lot of this has to do with scale. So we just have more and more of things.<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: And the bigger things are\u00a0as\u00a0harder to handle, but also our processing capabilities have also increased so that\u2019s one piece of good news. The other thing is,\u00a0like,\u00a0if you think of things like cloud, yes, they did like throw us a curve ball in introducing something new, but also they added a little bit of structure. So when you think about like a cloud network,\u00a0it\u2019s much more symmetric and much easier to reason about compared to something like the internet, which is just a hodge-podge of devices connected to each other with arbitrary topologies. Like,\u00a0if you look at the stuff we did in 007, for example, what we used,\u00a0really,\u00a0was that the fixed structure and the nice structure that\u00a0cloud networks actually have.<\/p>\n Host: So that actually helped.<\/b><\/p>\n Behnaz\u00a0Arzani: That actually helps, yeah.<\/p>\n Host: Interesting. Okay, because, you know, you just think the bigger it is<\/b>,<\/b>\u00a0the more messy it is, but you\u2019re actually saying it\u2019s added a layer of structure\u2026<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah.<\/p>\n Host: \u2026to help iron out some of the problems. All right, well let\u2019s talk a little bit more in detail about those kinds of problems. Data center diagnosis is hard. There\u2019s lots of incidents, lots of different kinds of incidents, incidents with whole life cycles…<\/b>\u00a0<\/b>Why is this so hard and what are some specific research projects you have going to make it not so hard?<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah, so if you go back to what I was saying earlier, like, the network is really the underlying infrastructure of a lot of distributed systems. So there\u2019s a lot of dependency on a well-functioning network, but the problem is also that when something goes wrong, how do you know if it\u2019s the network that\u2019s problematic or if there\u2019s other layers of the infrastructure that may be problematic?<\/p>\n Host:\u00a0<\/b>M<\/b>m-h<\/b>m<\/b>m.<\/b><\/p>\n Behnaz\u00a0Arzani: A very simple example of this is,\u00a0all of the VMs we operate in Azure are dependent on our storage systems,\u00a0for example,\u00a0because they have a virtual hard drive that has access to those storage systems. So they have to go over the network, but there\u2019s also that storage system itself that can fail. It\u2019s also the VM virtual hard disk that might fail. Like there\u2019s a lot of different failure scenarios. The host might fail. The server might fail. Everything. And so what we see often is that, well the first step is to\u00a0just locate who the expert is that needs to look at this problem. And often it\u2019s the case that because there\u2019s so many different levels of expertise, like the storage people, the storage operator knows really well how storage works, but he may not know anything about how a network works, right?<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: And so he doesn\u2019t know how to look at this network and to determine whether this network is healthy or not.<\/p>\n Host: Mm-hmm.<\/b><\/p>\n Behnaz\u00a0Arzani: Right? So you really need a network operator to get engaged at that point, but at the same time, you need to first know that you need the network operator to engage at that point. It\u2019s kind of like a chicken and egg problem.<\/p>\n Host: Right. You don\u2019t know what you don\u2019t know.<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah, so the projects we\u2019re working on right now, in the case of the\u00a0storage example that I gave, I think in 2016,\u00a0we had a Net 4.0 project that dealt with that. Right now, we are looking at a project called Scouts, which, its goal is basically to say, if each individual team provides an abstraction that basically says, is it likely that my team is probably going to be responsible for this problem? So my expertise is needed, or not?<\/p>\n Host: Yeah.<\/b><\/p>\n Behnaz\u00a0Arzani: That way, at least, a storage operator, when it sees that the storage system is failing can know, oh, I need a networking expert\u2026<\/p>\n Host: Mm-hmm.<\/b><\/p>\n Behnaz\u00a0Arzani: \u2026or I need a host expert, an SDN expert. What type of expert do I need to help me\u2026<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: \u2026with figuring this out?<\/p>\n Host: So, the problem upstream is diagnosing where the problem is, and you want to do that quickly.<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: So that you can address the issue and the problem has been that with a human, it takes way too long to even figure out who to blame.<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host:<\/b>\u00a0So what is it that machines can do to help us out here?<\/b><\/p>\n Behnaz\u00a0Arzani: I think the observation, at least we had,\u00a0and there\u2019s a lot of work still remaining to be done, but the observation is well, we see enough examples as,\u00a0like,\u00a0if I\u2019m the networking team\u00a0in\u00a0Microsoft, we\u2019ve seen examples of failures happen in the past and we collect a lot of data from our own infrastructure. So the idea is, can we learn from past failures whether this is probably going to be caused by a networking problem or a physical networking problem, for example, and basically use machine learning to identify whether this problem is likely due to this team\u2019s infrastructure failing.<\/p>\n Host: So do these failures present themselves in a certain way that would be a pattern detection thing that would be really good for machines to work on?<\/b><\/p>\n Behnaz\u00a0Arzani: In certain cases, yes. So,\u00a0for example,\u00a0in the case of physical networking that turns out to be really true. It\u2019s more complicated when you have\u2026\u00a0for example, something like a software load balancer is a lot more complicated because it has a lot more dependencies and its failures are also more complex.<\/p>\n Host:\u00a0<\/b>Mm<\/b>-h<\/b>m<\/b>m.\u00a0<\/b><\/p>\n Behnaz\u00a0Arzani: So for certain teams this is easier, but the nice thing is also that for these teams these are often the first ones to get blamed anyway because all of the teams depend on them.<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: So it\u2019s kind of like a win-win situation. You might want to build similar things for the teams that you can build this for\u2026<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani:\u00a0\u2026and then hope that this would simplify the problem to an extent that it makes the life of operators easier.<\/p>\n Host: Okay. Well, talk a little bit more about the lifecycle of a problem.<\/b><\/p>\n Behnaz\u00a0Arzani: Mm-hmm.<\/p>\n Host: Or an incident we\u2019ll call it, because we all recognize there are incidents that are going to happen and there will be a lot of them\u2026<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: \u2026when you have this giant scale. What do you mean when you say lifecycle of an incident?<\/b><\/p>\n Behnaz\u00a0Arzani: Well, so an incident starts\u00a0when\u00a0some monitoring infrastructure picked up that some anomaly is happening, that something is not operating as it should.<\/p>\n Host: Okay.<\/b><\/p>\n Behnaz\u00a0Arzani: And so an incident is created. A lot of\u00a0the\u00a0times we also have automation that goes and checks and knows how to fix it,\u00a0so that\u2019s the good case.<\/p>\n Host: Sure.<\/b><\/p>\n Behnaz\u00a0Arzani: That\u2019s like the best-case scenario. But in some cases,\u00a0when automation also fails to solve the problem, we have humans that are called, basically, to try\u00a0to\u00a0resolve it. And basically, the first step that that human takes is to figure out who to call to help, and also they get together and try to figure out, okay, which part of the system went wrong, how do we fix it? And the first step is actually mitigating the problem, meaning, for example, if I have a software load balancer that\u2019s problematic, I\u2019ll redirect all of my traffic to a different software load balancer while I figure out what\u2019s going on with this load balancer, right? And then they go proceed to fix it and resolve the issue.<\/p>\n Host: I\u2019m having a visual of an ER doc<\/b>\u2026<\/b>\u00a0<\/b>Y<\/b>ou know, you triage and you say, you know, is he breathing? Is he bleeding? Start one, stop the other.<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: And then we can move on to what\u2019s really the problem.<\/b><\/p>\n Behnaz\u00a0Arzani: Exactly. Yeah, that\u2019s basically what happens.<\/p>\n (music plays)<\/i><\/b><\/p>\n Host: Well, let\u2019s talk a little bit more about automation for a second and, and this trend towards auto ML<\/b>,<\/b>\u00a0or automated machine learning. And it\u2019s one line of research that seems really promising, and there\u2019s some specific branches of it. You refer to them as content-aware ML, or domain-customized auto ML frameworks.<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: So talk somewhat generally about the work that\u2019s going on in ML and then tell us how you\u2019re instantiating it in the world of networks and distributed systems.<\/b><\/p>\n Behnaz\u00a0Arzani: Right. I mean, I think that this came up when I was the only one in our group which knew a little bit about networking and machine learning and I had thirty different teams in Azure asking me to build the machine learning model that does something, whatever that was. And it felt\u00a0like\u00a0the\u00a0pattern that I was going through each time was very, very similar. And so it felt like I should be able to replicate my brain somehow so that like I\u2019m not needed in that process. And I didn\u2019t know at the time, when I researched it, I found that auto ML is actually a thing in the machine learning communities. I didn\u2019t know that!\u00a0And then when I looked at those, what I found is that a lot of them try to do anything and everything, or they\u2019re customized to domains that are very, very popular. Things like video analytics, like natural language processing, things like that are always needed, not necessarily something for networking. So my friend and I,\u00a0Bita\u00a0Rouhani from Doug Burger\u2019s group, started to look at well, what happens if you just dump networking data into these systems? Like,\u00a0just let\u2019s see how well they do. And they did it abysmally bad. The state-of-the-art was like terrible. And so we looked at it and said okay, why is that the case?\u00a0And\u00a0what we found was that,\u00a0well,\u00a0there\u2019s simple domain customizations that we could do,\u00a0even on the input. Not anything to the machine learning, but just how we present the data that would significantly boost their accuracy. And so the idea was well, actually,\u00a0operators are really good at that part. Like they really know their data. They really know things about the data that the auto ML frameworks don\u2019t know. So is there a way to bridge this gap? Is there a way to provide that domain knowledge without\u00a0him\u00a0knowing anything about ML?\u00a0Maybe\u00a0like\u00a0somehow the auto ML framework knows\u00a0what\u00a0information it needs and queries for that information from the user and the user provides that information and then we use that to generate\u00a0a\u00a0more customized ML model as part of those auto ML frameworks.<\/p>\n Host: So this sounds a lot like Patrice Simard\u2019s work in machine teaching, which is similar to this domain-specific ML, right?<\/b><\/p>\n Behnaz\u00a0Arzani: Right. I mean, it\u2019s similar and yet different. I think the nice thing about networking is actually, even though the types of problems we tackle are very, very diverse, they fall into a very limited set of categories. Things like congestion control, like diagnosis, like traffic engineering, like I can count them on my hand how many broad problem topics\u2026<\/p>\n Host: Sure.<\/b><\/p>\n Behnaz\u00a0Arzani: \u2026we tackle. And so because of that, it\u2019s much easier to provide a networking-specific abstraction for these systems than it is for any generic problem. And again, like for example, a network has a specific structure. You always have underlying topology. Like there are things we know, right? Where, for generic problems,\u00a0we might not know those specific topics and I think that\u2019s where,\u00a0like,\u00a0our take on the problem is different in the sense that we want to exploit network-specific domains that you can quantify almost, right? Like you can use a structure for them as opposed to like a generic problem.<\/p>\n Host: So you\u2019re providing a domain expert in networking with machine learning tools and they don\u2019t necessarily have to be a machine learning expert to be able to use these tools.<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: To make the whole thing happen.<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah. And I might, I should preface\u00a0that\u00a0that we don\u2019t know how to do this. Like I\u2019m just\u00a0like\u00a0giving you the idea of, this is what we want to do. We don\u2019t know how to do it yet!<\/p>\n Host: Okay. Go in there a little bit. You don\u2019t know how to do this,\u00a0<\/b>so this is a like<\/b>\u2026<\/b><\/p>\n Behnaz\u00a0Arzani: An idea.<\/p>\n Host:\u00a0<\/b>Okay<\/b>\u2026<\/b>\u00a0<\/b>Where are you with the idea? How far have you pushe<\/b>d<\/b>\u00a0on it<\/b>?<\/b><\/p>\n Behnaz\u00a0Arzani: So what we did initially was just to verify this hypothesis that domain knowledge actually helps auto ML systems and we were successfully able to demonstrate that. What we\u2019re doing now is take one specific area in networking that\u2019s very, very well-structured, but yet rich in problems, specifically congestion control. So within congestion control,\u00a0you might have a lot of different problems. What is the best congestion control protocol for me to use at any given point in time, given different objective functions that I have? Or,\u00a0like, can I design an ML-based congestion control protocol? And a lot of different other questions that we have a whole list of. And our idea is well, how do we build a domain-customized auto ML framework for congestion control specifically?\u00a0So it\u2019s not even for networking, just for this very, very tiny domain within networking.\u00a0And we\u2019re exploring whether we can do that.<\/p>\n Host: Okay. Thank you for the word hypothesis. It<\/b>\u00a0was<\/b>\u00a0the one I was searching for and couldn\u2019t find five minutes ago. You have a paper that just got accepted at the conference for Network<\/b>ed<\/b>\u00a0Systems Design and Implementation,\u00a0<\/b>NSDI<\/b>, this year and you call it\u00a0<\/b>Private Eye<\/i><\/b>.<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: And it deals with scalable and privacy-preserving compromise detection in the cloud. What problem is this work addressing and what\u2019s promising about your approach to it?<\/b><\/p>\n Behnaz\u00a0Arzani: So the problem was, really, when we talked to Azure operators, one of the things they mentioned is, we have these really good compromise detection systems that are very, very effective that customers\u00a0can<\/i>\u00a0use, but they don\u2019t\u00a0want<\/i>\u00a0to use.<\/p>\n Host: Why?<\/b><\/p>\n Behnaz\u00a0Arzani: Or\u00a0I don\u2019t know if\u00a0like \u201cdon\u2019t want to use\u201d might be a strong word, it might be that they are hesitant to use. And the reason for that seems to be that they\u2019re concerned about their privacy, how much data they want to share with Microsoft, and also taking on a third-party code. So basically, Microsoft will have to maintain that compromise detection system for them and a lot of customers are\u00a0uncomfortable with that.<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: So we looked at this, and the idea was, well, we still need to protect all of our customers, even though they don\u2019t necessarily want to use these systems. So how do we do this without needing our customer\u2019s permission to do so? And the observation was \u2013 and this is not a new observation, a lot of researchers have made this observation in the past, which is \u2013 well, network behavior changes when a VM is compromised. So can we use that change to basically say whether a VM is likely to be compromised or not, and then go from there?\u00a0The other observation, which is unique to this paper was,\u00a0though,\u00a0that we do have these compromise detection systems that are very, very effective and they\u2019re running on at least our first-party VMs. And these are VMs that run things like Bing, like SQL, like services that we have, and some of our customers are also opting in to use them.\u00a0So what they do is provide a constant stream of detections of compromised VMs that they\u2019ve seen, and we can use those as sort of quote-unquote \u201clabels\u201d to learn, okay, this is what compromise looks like and this is what changes it induces in the network behavior of these VMs. Putting these two on top of each other, we were like okay, maybe we can do something that\u2019s privacy preserving compromise detection that operates at\u00a0data center\u00a0scale.<\/p>\n Host: Wow.<\/b><\/p>\n Behnaz\u00a0Arzani: And then scale is also a hard thing here so the paper goes into a lot of trouble of explaining, for example, how do I ensure that I can run at this massive scale without sacrificing too much on accuracy, without having to use things like IP addresses\u2026<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: \u2026with, right now, with GDPR are difficult to use because GDPR says that if a customer wants to, they can contact you and say that you have to delete this and you have twenty four hours to do so\u00a0and so on\u2026<\/p>\n Host: Wow. So this sounds like it is also in the sort of early stages of<\/b>,<\/b>\u00a0how might we do this<\/b>?<\/b><\/p>\n Behnaz\u00a0Arzani: Yes.\u00a0I mean, the paper basically goes and demonstrates that we can theoretically do this and my experience with the other Scout project kind of says\u00a0that\u00a0there\u2019s a whole nine yards between \u201cwe think we can do this\u201d and we deploy it and we say, oh, this came up\u2026<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: \u2026so we have to handle this, and this other thing\u2026 so\u00a0like\u00a0what I found actually very interesting is,\u00a0from research paper to actually deployment, things can change\u00a0a hundred and eighty\u00a0degrees. Like you might just completely change the approach you use just because new constraints come up in deployment that you hadn\u2019t thought about when you were doing the prototype version.<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: Which is basically what the paper usually is.<\/p>\n Host: But you\u2019re moving forward.<\/b><\/p>\n Behnaz\u00a0Arzani: Mm-hmm.<\/p>\n Host: And this paper is kind of the beginning of the exploration and\u00a0<\/b>then\u00a0<\/b>you\u2019re going to try to scale it up.<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: And see where it breaks.<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah.\u00a0It probably will break. I\u2019m going to be honest about that, but yeah.<\/p>\n Host:\u00a0<\/b>Yeah, but that\u2019s what research is about, right?\u00a0<\/b><\/p>\n (music plays)<\/i><\/b><\/p>\n Host:\u00a0<\/b>Well, you just referred to Scout again and let\u2019s preface this by saying collaboration is so essential to research now that Microsoft Research even has an award for it<\/b>!<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: And you recently won this, you and your team recently won it, for this project called Scout.<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah.<\/p>\n Host: So tell us about the project. What is Scout and why did it win MSR\u2019s collaboration award?<\/b><\/p>\n Behnaz\u00a0Arzani: Ahh, that\u2019s interesting. So, I started Scout two years ago, with an intern of mine who was from Princeton and we basically first started to think about okay, is this even a problem? Like the first step is,\u00a0like,\u00a0how bad is this problem and is this really a problem?<\/p>\n Host: Define this problem.<\/b><\/p>\n Behnaz\u00a0Arzani: Meaning,\u00a0is it really the case that people find it hard to blame a team for a problem? We did that investigation and said, yeah, apparently it is hard,\u00a0so let\u2019s try to solve this problem.<\/p>\n Host: Wait, wait, wait. So, for me it\u2019s easy to blame. Okay. Let\u2019s just like level-set here\u2026<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: You\u2019re saying it\u2019s\u00a0<\/b>hard<\/i><\/b>\u00a0to\u00a0<\/b>blame<\/b>\u00a0a team, or it\u2019s hard to\u00a0<\/b>prove<\/i><\/b>\u00a0the blame?<\/b><\/p>\n Behnaz\u00a0Arzani: Well, everybody points the finger at the other one.<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: So that\u2019s basically what happens. Because also like, you know, people have limited time,\u00a0so\u00a0they\u00a0do a superficial check and if everything seems healthy, it\u2019s like nope, not me\u2026<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: \u2026your turn. And so they keep passing the ball around until somebody figures out what\u2019s going on and that is very, very inefficient\u2026<\/p>\n Host: Okay.<\/b><\/p>\n Behnaz\u00a0Arzani: \u2026and we basically just demonstrated that that\u2019s the case.<\/p>\n Host: All right.<\/b><\/p>\n Behnaz\u00a0Arzani: Then we were like okay, how do we solve this problem? And so we went about at least doing a prototype version of the Scout, which is basically a paper we submitted and fine. But then we\u2019re like, okay, can we actually deploy this? And there\u2019s this really cool project in the Systems group going on called Resource Central, which has to do with a framework to deploy machine learning models in production. So that\u2019s where Ricardo\u00a0Bianchini\u00a0came in and said, well, we have this really cool framework, why don\u2019t you guys take advantage of this and use this to deploy the system? So they helped us to basically deploy the first version of a Scout and then the physical networking team in Azure was the first team that we targeted to build a Scout for and they helped us with insights and what they knew about the network. The data they collected helped us figure out okay, you did a great job here, you did a sucky job here, we hate you for it, and all of those different things. So like they provided us with really good feedback.<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: And this is an ongoing collaboration, so we found that Scouts do really well in certain cases, but they\u00a0suck\u00a0at\u00a0cases\u00a0where operators actually get angry about like\u2026<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: \u2026we can classify really, really hard problems. When it comes to the easy stuff,\u00a0we sometimes make mistakes. And it turns out, operators don\u2019t like it when you get things wrong that they would have gotten right.<\/p>\n Host: You can\u2019t have everything though. Come on guys. Well, here\u2019s\u00a0<\/b>a<\/b>\u00a0question though<\/b>:<\/b>\u00a0is the \u201ceasy stuff problem\u201d like the person who\u2019s proofreading a paper and gets all of the small print right, but the headline has a massive error in it?<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah, pretty much. Yeah. So like for example like one example we saw is, there was an incident where the title of the incident said,\u00a0\u201cArista\u00a0switch. This is experiencing problem.\u201d\u00a0So a switch is basically the purview of the physical networking team. It\u2019s actually saying this is what the problem is. Our very cool\u00a0Scout said, this is not a physical networking issue. And I was like, okay. Why? Turns out that that particular incident was a transient problem, so that meant that there was a blip. And that blip really didn\u2019t register in the monitoring data that we had. The machine learning model thought, things are fine! Nothing\u2019s bad!<\/p>\n Host: Interesting.<\/b><\/p>\n Behnaz\u00a0Arzani: But because we didn\u2019t have the contextual information \u2013 and this goes back to the need for context, right?<\/p>\n Host: Yeah.<\/b><\/p>\n Behnaz\u00a0Arzani: Like we didn\u2019t have the contextual information,\u00a0and\u00a0so we got that one wrong. And what we learned from that is, well, we need to have some form of contextual features as part of our feature set. Now if you look at our prototype version, this really didn\u2019t register to us as an important problem because our accuracy was so high.\u00a0We had\u00a0like\u00a098% true positive, 97% true negative, but in that 3% we had these very, very simple mistakes that operators are very unforgiving about because it\u2019s like, it\u2019s\u00a0saying it in the titl<\/i>e!<\/i><\/p>\n Host: So how do you fix that?<\/b><\/p>\n Behnaz\u00a0Arzani: Well, so it actually ends up being a relatively simple fix because again, like it\u2019s in the title. You just use information from the title as part of the features that you\u2019re using. For us, the original hypothesis was, use the monitoring data as God. Basically, what does the data show? But also,\u00a0it\u2019s a fact that, you know, there are some incidents\u00a0that turn out to be non- problems, but there\u2019s still an incident and somebody has to still go and look,\u00a0so it\u2019s important to basically have the context from the incident itself as well, as part of your feature set.<\/p>\n Host:\u00a0<\/b>Okay<\/b>. So I want to sort of weave back in some of the things we\u2019ve talked about<\/b>.<\/b>\u00a0<\/b>M<\/b>y understanding\u00a0<\/b>of<\/b>\u00a0what you want to accomplish here is a self-managing network using auto ML frameworks and having as little human slowdown in the process as possible.<\/b><\/p>\n Behnaz\u00a0Arzani: Right.<\/p>\n Host: You don\u2019t want humans completely out of the loop.<\/b><\/p>\n Behnaz\u00a0Arzani: Not yet. I don\u2019t think that\u2019s possible. I mean, ideally you would want to,\u00a0and I think that\u2019s like kind of the Holy Grail.<\/p>\n Host: Do you foresee a future where that would be possible?<\/b><\/p>\n Behnaz\u00a0Arzani:\u00a0As somebody that I really admire once told me, when you build systems you have to ask yourself, can it take me to Mars? And I think that\u2019s pretty much what we failed to do when building networks.<\/p>\n Host: Okay.<\/b><\/p>\n Behnaz\u00a0Arzani: At least recently, because there is,\u00a0maybe\u2026\u00a0I don\u2019t know, but when I look at a lot of the work I have done,\u00a0and a lot of the work my peers have done, I think we never really asked\u00a0ourselves that question, which is why we\u2019re in the mess we\u2019re in.<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: Maybe, over time, as we start to ask that question more,\u00a0will it take us to Mars?<\/p>\n Host: All right.<\/b><\/p>\n Behnaz\u00a0Arzani: Then\u2026 because you know, we are the same people that built actual things that took us to the moon.<\/p>\n Host: Right.<\/b><\/p>\n Behnaz\u00a0Arzani: And those did not need operators to manage them. Yeah\u2026<\/p>\n Host: Well, okay, but so having the contextual input for these systems to identify the stupid errors, can you do that and make that happen with a machine without having human context provided?<\/b><\/p>\n Behnaz\u00a0Arzani: The right answer to that question is, I don\u2019t know. These are things that we\u2019re experimenting with, that we\u2019re trying, but who knows?<\/p>\n Host: All right. Let\u2019s bring this all together. Your big goal is to get to networks that can manage themselves and we\u2019re not there yet, so what would you say are the big open problems in the field that<\/b>,<\/b>\u00a0if solved<\/b>,<\/b>\u00a0would get us closer to the network equivalent of self-driving cars?<\/b><\/p>\n Behnaz\u00a0Arzani: So I think there\u2019s a couple of things. One is,\u00a0what data do we actually need from the network to be able to do this? I think that\u2019s still an open problem. But the problem is not that we don\u2019t know what exact data we need, it\u2019s like, what data we need and how to efficiently collect it? Like how do we collect it without actually breaking the network while doing so?\u00a0I think that\u2019s like\u2026\u00a0There\u2019s a lot of work going on,\u00a0and we see paper after paper on this topic, but we really don\u2019t know what is the necessary and sufficient data set to be able to do this.<\/p>\n Host: Okay.<\/b><\/p>\n Behnaz\u00a0Arzani: That\u2019s one. The control loops that we need to able to then use this data to do self-driving networks and\u00a0self-driving \u2013\u00a0the equivalent of self-driving cars is, I don\u2019t think, there in place yet. And we don\u2019t even have the mechanisms to then implement that control loop yet. I also think that we\u2019ve been bogged down by just how to get the network to work in the first place,\u00a0and a lot of the papers that we see like, for example, the traffic engineering papers we see,\u00a0have\u00a0to do with that. And so I think it\u2019s just, we haven\u2019t had time yet to fully explore the other side of things.<\/p>\n Host: And networks themselves\u2026 like you started talking about Internet of Things<\/b>,\u00a0<\/b>and Donald\u00a0<\/b>Kossmann<\/b>\u00a0was recently on the podcast and talked about\u00a0<\/b>nine<\/b>\u00a0billion \u201cthings,\u201d as it were.\u00a0<\/b>And<\/b>\u00a0trying to think of how you even wrap your brain around how you would manage that kind of a network.<\/b><\/p>\n Behnaz\u00a0Arzani: Right. Luckily, that\u2019s out of my area of expertise. I work on data center networks. If I get that to work, I\u2019m happy. That I would talk to somebody else about!<\/p>\n Host: There\u2019s the finger pointing over there<\/b>!<\/b>\u00a0I\u2019m data centers. That\u2019s your business.\u00a0<\/b><\/p>\n Behnaz\u00a0Arzani:\u00a0Yup!<\/p>\n Host:\u00a0<\/b>Well, we\u2019ve talked about what gets you up in the morning<\/b>,<\/b>\u00a0and\u00a0<\/b>it\u2019s<\/b>\u00a0a lot that gets you up in the morning<\/b>!\u00a0<\/b>Now I want to know what keeps you up at night<\/b>.<\/b>\u00a0And I often joke with some researchers that their entire career is about what keeps them up at night, technically speaking. That said, is there anything about your work, outside the fact that it\u2019s important to get your work right, that keeps you up at night, metaphorically, and if so, how are you dealing with it?<\/b><\/p>\n Behnaz\u00a0Arzani:\u00a0So I guess\u00a0\u2013\u00a0and this is more recent\u2026\u00a0I don\u2019t think it\u2019s been the case for the past, like, whatever years is \u2013like\u00a0after we started actually deploying the Scout and getting it to be used in production, like, my worry is again, will it get us to Mars, for lack of a better word? In the sense of how trustworthy is it? When is it going to break again?\u00a0How long is it going to last as is? When is the next time that somebody\u2019s going to yell at me because I got a simple thing wrong? So I think reliability of machine learning systems for networking,\u00a0and how hands-free can they actually be,\u00a0is something that keeps me up at night because it seems to me, at least with our experience, that there\u2019s some level of hand holding that\u2019s needed over time. And that worries me because, what does that actually mean? Does it mean that you always need somebody babysitting these types of systems?\u00a0And that\u2019s not necessarily the best thing that you would want.<\/p>\n Host: Yeah. You got me thinking so deeply right now about the preferred future, and\u00a0<\/b>\u201c<\/b>humans out\/humans in<\/b>\u201d<\/b>\u00a0and if we could ever really get to a full representation of\u00a0<\/b>data center<\/b>\u00a0problems.<\/b><\/p>\n Behnaz\u00a0Arzani: Right. Who knows?<\/p>\n Host: That\u2019s why you\u2019re working here.<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah.<\/p>\n Host: Well it\u2019s story time,\u00a0<\/b>Behnaz<\/b>. Tell us about yourself. How did you get started in the high-tech life and how did you end up at Microsoft Research? I heard the word internships, plural, earlier on\u2026!<\/b><\/p>\n Behnaz\u00a0Arzani:\u00a0Yeah. Well.\u00a0I had a very messy way to getting where I\u2019m at. In high school I loved physics and I liked circuits and electrical systems so I went into electrical engineering as my bachelor\u2019s degree. And still,\u00a0I like electrical engineering\u00a0a lot<\/i>. Like, I was the circuits person, the analog circuits person\u2026 how do you analyze\u2026 at that time, they would teach us things about BJTs and such,\u00a0and then that was where I wanted to go,\u00a0and a friend of mine said, you\u2019ll never find a job in electrical engineering.<\/p>\n Host: Really?<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah,\u00a0especially in analog circuits, which I was particularly good at. And so I was like okay, what\u2019s the next thing I\u2019m good at? And that was probability and networking. And I\u2019m like, okay, that\u2019s what I\u2019m going to do. And actually,\u00a0the first few classes that I sat in \u2013 because there was an electrical engineering analog circuits class that was in parallel to the digital signal processing class that we had to take if we were in the communications major\u00a0\u2013 and I would sit in that class and I was like, I have to be in that other class! And then I started to fall in love with it. I was like, I really, really like networking. So I applied to a networking PhD, again in electrical engineering, and then\u00a0my advisor just left my school. So I had to find a new advisor and that was in the computer science department and that\u2019s how I became a computer scientist\u2026<\/p>\n Host: Wow.<\/b><\/p>\n Behnaz\u00a0Arzani: \u2026completely by accident!<\/p>\n Host: Circuitous route<\/b>!<\/b><\/p>\n Behnaz\u00a0Arzani: Yes. And then I loved it. So like most of it is like,\u00a0I accidentally stumbled into where I\u2019m at, and then I ended up falling in love with it.<\/p>\n Host: Where did it all start? I mean, where was this taking place and who were you working with?<\/b><\/p>\n Behnaz\u00a0Arzani: So I was at the University of Pennsylvania. I was working with\u00a0Roch\u00a0Guerin\u00a0when I started,\u00a0and then\u00a0Roch\u00a0left to become the chair of computer science at Washington, St. Louis. So then I moved to computer science to work with my new advisor, Boon\u00a0Thau\u00a0Loo, who is still there, and I did networking.\u00a0And then I think my first internship was in 2015.<\/p>\n Host: Here?<\/b><\/p>\n Behnaz\u00a0Arzani: In Azure networking.<\/p>\n Host: Oh, Azure, OK.<\/b><\/p>\n Behnaz\u00a0Arzani: So not Microsoft Research, no. And then again, I loved it so I came back for a second time and then I applied for a postdoc here. I did a postdoc and then I applied for full-time jobs,\u00a0and then the rest is history.<\/p>\n Host: So postdoc in Azure, or postdoc in Microsoft Research?<\/b><\/p>\n Behnaz\u00a0Arzani: Microsoft Research.<\/p>\n Host: Okay.<\/b><\/p>\n Behnaz\u00a0Arzani: I don\u2019t think Azure has postdocs. I actually fought for a postdoc in Azure and they said that we don\u2019t have such a thing!<\/p>\n Host: Right<\/b>?<\/b>\u00a0Tell me a little bit<\/b>,<\/b>\u00a0though<\/b>,<\/b>\u00a0about the back and forth between Azure and Microsoft Research.<\/b><\/p>\n Behnaz\u00a0Arzani: So the way it happens in MSR is very different than the way it happened in Azure. So when I was an Azure intern, I talked to Azure people every day, twenty-four hours a day. So I knew about all the problems that were going on. I knew what the people\u2019s pain points are because they were sitting next to me. Here at MSR, people come to us and say, I need this problem solved. Or we solve a problem like, hey, we solved this problem, do you actually need this that we did? And so it\u2019s very, very different, I would say, the dynamic of going from an idea at MSR to actually deploying it in production.<\/p>\n Host:\u00a0<\/b>M<\/b>m-h<\/b>m<\/b>m.<\/b><\/p>\n Behnaz\u00a0Arzani: And it\u2019s much, much harder than if you come up with the idea when you\u2019re sitting in Azure and deploying it in Azure, but it\u2019s amazing how easily it gets done. It\u2019s amazing, like, how fun the collaborations are\u00a0and so on.<\/p>\n Host: Right. From your position now, where do you see yourself in the future? Staying in research?<\/b><\/p>\n Behnaz\u00a0Arzani: I prefer to not think about that type of thing! I like to be the person who does things while they\u2019re fun and once they\u2019re not fun, stop doing them and move on to the next thing so I have no idea how to answer that question!<\/p>\n Host: All right. Well tell us something that we might not know about you. Maybe it impacted your life or career, a life defining moment or some personal characteristic\u2026 but\u00a0<\/b>maybe it\u2019s just something interesting that would give us some context about you outside the lab.<\/b><\/p>\n Behnaz\u00a0Arzani: Uh, okay. Not something I\u2019m really proud of, but I\u2019m a very, very competitive person. So I always attribute me getting to where I am to a friend of mine in high school,\u00a0where our principal would come and say, learn from this person. This person is great. And I was like, I can do better. And it\u2019s sad, but true, that the reason I\u2019m here is because of a competition with another person in high school. Otherwise, I would not get into college,\u00a0I think, I would not get to where I am.<\/p>\n Host: Okay, so let me clarify. There was an actual person that your principal said, be like that person<\/b>?<\/b><\/p>\n Behnaz\u00a0Arzani: And I was like no, I\u2019m going to be better than that person.<\/p>\n Host: Oh, my gosh. I\u2019d like to meet that principal. Well, before we go, I want to give you the opportunity to talk to some version of your grad school self. Assuming you\u2019d listen to you, what advice would you give yourself if you could go back and give yourself advice?<\/b><\/p>\n Behnaz\u00a0Arzani: Hmm. The advice I would think is, it\u2019s okay to be nitpicky. Like,\u00a0I think one thing that I found frustrating, as a PhD student, was how much one of my advisors,\u00a0Roch, wanted us to be very, very meticulous about making sure about every single detail about something before we made a conclusion. And it took a long time to do. It was a lot of pain. And I\u2019ve now learned to appreciate that. And so, what I would say is,\u00a0it\u2019s hard now, but it\u2019s such good advice.<\/p>\n Host:\u00a0<\/b>Behnaz<\/b>\u00a0<\/b>Arzani<\/b>, thank you for joining us today.<\/b><\/p>\n Behnaz\u00a0Arzani: Thank you.<\/p>\n Host: It\u2019s been so much fun<\/b>!<\/b><\/p>\n Behnaz\u00a0Arzani: Yeah, I know. Thanks.<\/p>\n (music plays)<\/i><\/b><\/p>\n To learn more about Dr.\u00a0<\/i><\/b>Behnaz<\/i><\/b>\u00a0<\/i><\/b>Arzani<\/i><\/b>\u00a0and the latest in networking research, visit Microsoft.com\/research<\/i><\/b><\/p>\n","protected":false},"excerpt":{"rendered":" Dr. Behnaz Arzani is a senior researcher in the Mobility and Networking group at MSR, and she feels your pain. At least, that is, if you\u2019re a network operator trying to troubleshoot an incident in a datacenter. Her research is all about getting networks to manage themselves, so your life is as pain-free as possible. On today\u2019s podcast, Dr. Arzani tells us why it\u2019s so hard to identify and resolve networking problems and then explains how content-aware, or domain-customized, auto ML frameworks might help. She also tells us what she means when she says she wants to get humans out of the loop, and reveals how a competitive streak and a comment from her high school principal set her on the path to a career in high tech research.<\/p>\n","protected":false},"author":37583,"featured_media":644052,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"https:\/\/player.blubrry.com\/id\/57428878","msr-podcast-episode":"111","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[240054],"tags":[],"research-area":[13547],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-644049","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-msr-podcast","msr-research-area-systems-and-networking","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"https:\/\/player.blubrry.com\/id\/57428878","podcast_episode":"111","msr_research_lab":[199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144899],"related-projects":[],"related-events":[],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"Related:<\/h3>\n
\n
\nTranscript<\/h3>\n
![\"Headshot](\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/03\/Research_Podcast_BehnazArzani_Site_1400x788-960x540.png\")
","byline":"","formattedDate":"March 18, 2020","formattedExcerpt":"Dr. Behnaz Arzani is a senior researcher in the Mobility and Networking group at MSR, and she feels your pain. At least, that is, if you\u2019re a network operator trying to troubleshoot an incident in a datacenter. Her research is all about getting networks to…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/644049"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/37583"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=644049"}],"version-history":[{"count":5,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/644049\/revisions"}],"predecessor-version":[{"id":668130,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/644049\/revisions\/668130"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/644052"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=644049"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=644049"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=644049"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=644049"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=644049"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=644049"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=644049"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=644049"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=644049"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=644049"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=644049"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}