GEHRKE: Welcome, Lex.

LEX STORY: Oh, thank you.

GEHRKE: Really great to have you here. Can you tell us a little bit about what you’re doing here at MSR (Microsoft Research) …

STORY: OK.

GEHRKE: … and how did you actually end up here?

STORY: Well, um, within MSR, I actually work in the hardware prototype, hardware development. I find solutions for the researchers, especially in the areas of developing hardware through various fabrication and industrial-like methods. I’m a model maker. My background is as an industrial designer and a product designer. So when I attended school initially, it was to pursue a science; it was [to] pursue chemistry.

GEHRKE: And you grew up in California?

STORY: I grew up in California. I was born in Inglewood, California, and I grew up in San Bernardino, California. Nothing really too exciting happening in San Bernardino, which is why I was compelled to find other avenues, especially to go seek out travel. To do things that I knew that I would be able to look back and say, yes, you’ve definitely done something that was beyond what was expected of you having grown up in San Bernardino.

GEHRKE: And you even had that drive during your high school, or …

STORY: Yeah, high school just didn’t feel like … I think it was the environment that I was growing up in; it didn’t feel as if they really wanted to foster exceptional growth. And I had a father who was … had multiple degrees, and he had a lot of adversity, and he had a lot of challenges. He was an African American. He was a World War II veteran. But he had attained degrees, graduate degrees, in various disciplines, and that included chemical engineering, mechanical engineering, and electrical engineering.

GEHRKE: Wow. All three of them?

STORY: Yes. And so he was … had instilled into us that, you know, education is a vehicle, and if you want to leave this small town, this is how you do it you. But you need to be a vessel. You need to absorb as much as you can from a vast array of disciplines. And not only was he a man of science; he was also an artist. So he always fostered that in us. He said, you know, explore, gain new skills, and the collection of those skills will make you greater overall. He’s not into this idea of being such a specialist. He says lasers are great, but lasers can be blind to what’s happening around them. He says you need to be a spotlight. And he says then you have a great effect on a large—vast, vast array of things instead of just being focused on one thing.

GEHRKE: So you grew up in this environment where the idea was to, sort of, take a holistic view and not, like, a myopic view …

STORY: Yes, yes, yes.

GEHRKE: And so what is the impact of that on you?

STORY: Well, as soon as I went into [LAUGHS] the Marine Corps, I said, now I can attain my education. And …

GEHRKE: So right after school, you went to the …

STORY: I went directly into the Marine Corps right after high school graduation.

GEHRKE: And you told me many times, this is not the Army, right?

STORY: No, it’s the Marine Corps. It’s a big differentiation between … they’re both in military service. However, the Marine Corps is very proud of its traditions, and they instill that in us during boot camp, your indoctrination. It is drilled upon you that you are not just an arm of the military might. You are a professional. You are representative. You will basically become a reflection of all these other Marines who came before you, and you will serve as a point of the young Marines who come after you. So that was drilled into us from day one of boot camp. It was … but it was very grueling. You know, that was the one aspect, and there was a physical aspect. And the Marine Corps boot camp is the longest of all the boot camps. It was, for me, it was 12 weeks of intensive, you know, training. So, you know, the indoctrination is very deep.

GEHRKE: And then so it’s your high school, and you, sort of, have this holistic thinking that you want to bring in.

STORY: Yes.

GEHRKE: And then you go to the Marines.

STORY: I go to the Marines. And the funny thing is that I finished my enlistment, and after my enlistment, I enroll in college, and I say, OK, great; that part of my … phase of life is over. However, I’m still active reserve, and the Desert Shield comes up. So I’m called back, and I said, OK, well, I can come back. I served in a role as an NBC instructor. “NBC” stands for nuclear, biological, chemical warfare. And one of the other roles that I had in the Marine Corps, I was also a nuke tech. That means I knew how to deploy artillery-delivered nuclear-capable warheads. So I had this very technical background mixed in with, like, this military, kind of, decorum. And so I served in Desert Shield, and then eventually that evolved into Operation Desert Storm, and once that was over, I was finally able to go back and actually finish my schooling.

GEHRKE: Mm-hmm. So you studied for a couple of years and then you served?

STORY: Oh, yes, yes.

GEHRKE: OK. OK.

STORY: I had done a four-year enlistment, and you have a period of years after your enlistment where you can be recalled, and it would take very little time for you to get wrapped up for training again to be operational.

GEHRKE: Well, that must be a big disruption right in the middle of your studies, and …

STORY: It was a disruption that …

GEHRKE: And thank you for your service.

STORY: [LAUGHS] Thank you. I appreciate that. It was a disruption, but it was a welcome disruption because, um, it was a job that I knew that I could do well. So I was willing to do it. And when I was ready for college again, it made me a little hungrier for it.

GEHRKE: And you’re already a little bit more mature than the average college student …

STORY: Oh, yes.

GEHRKE: … when you entered, and then now you’re coming back from your, sort of, second time.

STORY: I think it was very important for me to actually have that military experience because [through] that military experience, I had matured. And by the time I was attending college, I wasn’t approaching it as somebody who was in, you know, their teenage years and it’s still formative; you’re still trying to determine who you are as a person. The military had definitely shown me, you know, who I was as a person, and I actually had a few, you know, instances where I actually saw some very horrible things. If anything, being in a war zone during war time, it made me a pacifist, and I have … it increased my empathy. So if anything, there was a benefit from it. I saw some very horrible things, and I saw some amazing things come from human beings on both ends of the spectrum.

GEHRKE: And it’s probably something that’s influenced the rest of your life also in terms of where you went as your career, right?

STORY: Yes.

GEHRKE: So what were you studying, and then what were your next steps?

STORY: Well, I was studying chemistry.

GEHRKE: OK, so not only chemistry and mechanical engineering …

STORY: And then I went away, off to Desert Storm, and when I came back, I decided I didn’t want to study chemistry anymore. I was very interested in industrial design and graphic design, and as I was attending, at ArtCenter College of Design in Pasadena, California, there was this new discipline starting up, but it was only for graduates. It was a graduate program, and it was called computer-aided industrial design. And I said, wait a minute, what am I doing? This is something that I definitely want to do. So it was, like, right at the beginning of computer-generated imagery, and I had known about CAD in a very, very rudimentary form. My father, luckily, had introduced me to computers, so as I was growing up a child in the ’70s and the ’80s, we had computers in our home because my dad was actually building them. So his background and expertise—he was working for RCA; he was working for Northrop Grumman. So I was very familiar with those. 

GEHRKE: You built PCs at home, or what, what … ?

STORY: Oh, he built PCs. I learned to program. So I … 

GEHRKE: What was your first programming language?

STORY: Oh, it was BASIC …

GEHRKE: BASIC. OK, yup.

STORY: … of course. It was the only thing I could check out in the library that I could get up and running on. So I was surrounded with technology. While most kids went away, summer camp, I spent my summer in the garage with my father. He had metalworking equipment. I understood how to operate metal lathes. I learned how to weld. I learned how to rebuild internal combustion engines. So my childhood was very different from what most children had experienced during their summer break. And also at that time, he was working as a … in chemistry. So his job then, I would go with him and visit his job and watch him work in a lab environment. So it was very, very unique. But also the benefit of that is that being in a lab environment was connected to other sciences. So I got to see other departments. I got to see the geology department. I got to see … there was disease control in the same department that he was in. So I was exposed to all these things. So I was always very hungry and interested, and I was very familiar with sciences. So looking at going back into art school, I said, oh, I’m going to be an industrial designer, and I dabble in art. And I said, wait a minute. I can use technology, and I can create things, and I can guide machines. And that’s the CAM part, computer-aided machining. So I was very interested in that. And then having all of this computer-generated imagery knowledge, I did one of the most knuckleheaded things I could think of, and I went into video game development.

GEHRKE: Why is it knuckleheaded? I mean, it’s probably just the start of big video games.

STORY: Well, I mean … it wasn’t, it wasn’t a science anymore. It was just pursuit of art. And while I was working in video game development, it was fun. I mean, no doubt about it. And that’s how I eventually came to Microsoft, is the company I was working for was bought, purchased by Microsoft.

GEHRKE: But why is it only an art? I’m so curious about this because even computer games, right, there’s probably a lot of science about A/B testing, science of the infrastructure …

STORY: Because I was creating things strictly for the aesthetics.

GEHRKE: I see.

STORY: And I had the struggle in the back of my mind. It’s, like, why don’t we try to create things so that they’re believable, and there’s a break you have to make, and you have to say, is this entertaining? Because in the end, it’s entertainment. And I’ve always had a problem with that.

GEHRKE: It’s about storytelling though, right?

STORY: Yes, it is about storytelling. And that was one of the things that was always told to us: you’re storytellers. But eventually, it wasn’t practical, and I wanted to be impactful, and I couldn’t be impactful doing that. I could entertain you. Yeah, that’s great. It can add some levity to your life. But I was hungry for other things, so I took other jobs eventually. I thought I was going to have a full career with it, and I decided, no, this is not the time to do it.

GEHRKE: That’s a big decision though, right?

STORY: Oh, yeah. Yeah.

GEHRKE: Because, you know, you had a good job at a game company, and then you decided to …

STORY: But there was no, there was no real problem solving for me.

GEHRKE: I see. Mm-hmm.

STORY: And there was opportunity where there was a company, and they were using CAD, and they were running wax printers, and it was a jewel company. And I said, I can do jewelry.

GEHRKE: So what is a wax printer? Explain that.

STORY: Well, here’s … the idea is you can do investment casting.

GEHRKE: Yeah.

STORY: So if you’re creating all your jewelry with CAD, then you can be a jewelry designer and you can have something practical. The reason I took those jobs is because I wanted to learn more about metallurgy and metal casting. And I did that for a bit. And then, eventually, I—because of my computer-generated imagery background—I was able to find a gig with HoloLens. And so as I was working with HoloLens, I kept hearing about research, and they were like, oh yeah, look at this technology research created, and I go, where’s this research department? So I had entertained all these thoughts that maybe I should go and see if I can seek these guys out. And I did find them eventually. My previous manager, Patrick Therien, he brought me in, and I had an interview with him, and he asked me some really poignant questions. And he was a mechanical engineer by background. And I said, I really want to work here, and I need to show you that I can do the work. And he says, you don’t need to prove to me that you can do the work; you have to prove to me that you’re willing to figure it out.

GEHRKE: So how did you do that, or how did you show him?

STORY: I showed him a few examples. I came up with a couple of ideas, and then I demonstrated some solutions, and I was able to present those things to him during the interview. And so I came in as a vendor, and I said, well, if I apply myself, you know, rigorously enough, they’ll see the value in it. And, luckily, I caught the eye of …was it … Gavin [Jancke], and it was Patrick. And they all vouched for me, and they said, yeah, definitely, I have something that I can bring. And it’s always a challenge. The projects that come in, sometimes we don’t know what the solution is going to be, and we have to spend a lot of time thinking about how we’re going to approach it. And we also have to be able to approach it within the scope of what their project entails. They’re trying to prove a concept. They’re trying to publish. I want to make everything look like a car, a beautiful, svelte European designed … but that’s not always what’s asked. So I do have certain parameters I have to stay within, and it’s exciting, you know, to come up with these solutions. I’m generating a concept that in the end becomes a physical manifestation.

GEHRKE: Yeah, so how do you balance this? Because, I mean, from, you know, just listening to your story so far, which is really fascinating, is that there’s always this balance not only on the engineering side but also on the design and art side.

STORY: Yes!

GEHRKE: And then a researcher comes to you and says, I want x.

STORY: Yes, yes, yes. [LAUGHS]

GEHRKE: So how do you, how do you balance that?

STORY: It’s understanding my roles and responsibilities.

GEHRKE: OK.

STORY: It’s a tough conversation. It’s a conversation that I have often with my manager. Because in the end, I’m providing a service, and there are other outlets for me still. Every day, I draw. I have an exercise of drawing where I sit down for at least 45 minutes every day, and I put pen to paper because that is an outlet. I’m a voracious reader. I tackle things because—on a whim. It’s not necessarily that I’m going to become a master of it. So that’s why I attended culinary school. Culinary school fell into this whole curiosity with molecular gastronomy. And I said, wait a minute, I don’t want to be an old man …

GEHRKE: So culinary school is like really very, very in-depth understanding the chemistry of cooking. I mean, the way you understand it …

STORY: Yeah, the molecular gastronomy, the chemistry of cooking. Why does this happen? What is caramelization? What’s the Maillard effect?

GEHRKE: So it’s not about just the recipe for this cake, or so …

STORY: No … the one thing you learn in culinary school very quickly is recipes are inconsequential.

GEHRKE: Oh, really?

STORY: It’s technique.

GEHRKE: OK.

STORY: Because if I have a technique and I know what a roux is and what a roux is doing—and a roux is actually gelatinizing another liquid; it’s a carrier. Once you know these techniques and you can build on those techniques, recipes are irrelevant. Now, the only time recipes matter is when you’re dealing with specific ratios, but that’s still chemistry, and that’s only in baking. But everything else is all technique. I know how to break down the, you know, the connective tissue of a difficult cut of meat. I know what caramelization adds. I understand things like umami. So I look at things in a very, very different way than most people. I’m not like a casual cook, which drove me to go work for Cook’s Illustrated and America’s Test Kitchen, outside of Boston. Because it wasn’t so much about working in a kitchen; it was about exploration, a process. That all falls back into that maddening, you know, part of my personality … it’s like, what is the process? How can I improve that—how can I harness that process?

GEHRKE: So how was it to work there? Because I see food again as, sort of, this beautiful combination of engineering in some sense, creating the recipe. But then there’s also the art of it, right? The presentation.

STORY: Yes …

GEHRKE: And how do you actually put the different flavors together?

STORY: Well, a lot of that’s familiarity because it’s like chemistry. You have familiarity with reactions; you have familiarity and comparisons. So that all falls back into the science. Of course, when I plate it, that falls … I’m now borrowing on my aesthetics, my ability to create aesthetic things. So it fulfills all of those things. So, and that’s why I said, I don’t want to be an old man and say, oh, I wish I’d learned this. I wanted to attend school. I took a sabbatical, attended culinary school.

GEHRKE: So you took a sabbatical from Microsoft?

STORY: Oh, yes, when I was working in video games. Yeah.

GEHRKE: OK.

STORY: I took a sabbatical. I did that. And I was like, great. I got that out of the way. Who’s to say I don’t open a food truck?

GEHRKE: Yeah, I was just wondering, what else is on your bucket list, you know?

STORY: [LAUGHS] I definitely want to do the food truck eventually.

GEHRKE: OK, what would the food truck be about?

STORY: OK. My heritage, my background, is that I’m half Filipino and half French Creole Black.

GEHRKE: You also had a huge family. There’s probably a lot of really good cooking.

STORY: Oh, yeah. Well, I have stepbrothers and stepsisters from my Mexican stepmother, and she grew up cooking Mexican dishes. She was from the Sinaloa area of Mexico. And so I learned a lot of those things, very, very unique regional things that were from her area that you can’t find anywhere else.

GEHRKE: What’s an example? Now you’ve made me curious.

STORY: Capirotada. Capirotada is a Mexican bread pudding, and it utilizes a lot of very common techniques, but the ingredients are very specific to that region. So the preparation is very different. And I’ve had a lot of people actually come to me and say, I’ve never had capirotada like that. And then I have other people who say, that is exactly the way I had it. And by the way, my, you know, my family member was from the Sinaloa area. So, yeah, but from my Filipino heritage background, I would love to do something that is a fusion of Filipino foods. There’s a lot of great, great food like longganisa; there’s a pancit. There’s adobo. That’s actually adding vinegars to braised meats and getting really great results that way. It’s just a … but there’s a whole bevy of … but my idea eventually for a food truck, I’m going to keep that under wraps for now until I finally reveal it. Who’s, who’s to say when it happens.

GEHRKE: OK. Wow, that sounds super interesting. And so you bring all of these elements back into your job here at MSR in a way, because you’re saying, well, you have these different outlets for your art. But then you come here, and … what are some of the things that you’ve created over the last few years that you’re especially proud of?

STORY: Oh, phew … that would … Project Eclipse.

GEHRKE: Eclipse, uh-huh.

STORY: That’s the hyperlocal air-quality sensor.

GEHRKE: And this is actually something that was really deployed in cities …

STORY: Yes. It was deployed in Chicago …

GEHRKE: … so it had to be both aesthetically good and … to look nice, not only functional.

STORY: Well, it had not only … it had … first of all, I approached it from it has to be functional. But knowing that it was going to deploy, I had to design everything with a design for manufacturing method. So DFM—design for manufacturing—is from the ground up, I have to make sure that there are certain features as part of the design, and that is making sure I have draft angles because the idea is that eventually this is going to be a plastic-injected part.

GEHRKE: What is a draft angle?

STORY: A draft angle is so that a part can get pulled from a mold.

GEHRKE: OK …

STORY: If I build things with pure vertical walls, there’s too much even stress that the part will not actually extract from the mold. Every time you look at something that’s plastic injected, there’s something called the draft angle, where there’s actually a slight taper. It’s only 2 to 4 degrees, but it’s in there, and it needs to be in there; otherwise, you’re never going to get the part out of the mold. So I had to keep that in mind. So from the ground up, I had designed this thing—the end goal of this thing is for it to be reproduced in a production capacity. And so DFM was from day one. They came to me earlier, and they gave me a couple of parts that they had prototyped on a 3D printer. So I had to go through and actually re-engineer the entire design so that it would be able to hold the components, but …

GEHRKE: And to be waterproof and so on, right?

STORY: Well, waterproofing, that was another thing. We had a lot of iterations—and that was the other thing about research. Research is about iteration. It’s about failing and failing fast so that you can learn from it. Failure is not a four-lettered word. In research, we fail so that we use that as a steppingstone so that we can make discoveries and then succeed on that …

GEHRKE: We learn.

STORY: Yes, it’s a learning opportunity. As a matter of fact, the very first time we fail, I go to the whiteboard and write “FAIL” in big capital letters. It’s our very first one, and it’s our “First Attempt In Learning.” And that’s what I remember it as. It’s my big acronym. But it’s a great process. You know, we spin on a dime. Sometimes, we go, whoa, we went the wrong direction. But we learn from it, and it just makes us better.

GEHRKE: And sometimes you have to work under time pressure because, you know, there’s no …

STORY: There isn’t a single thing we don’t do in the world that isn’t under time pressure. Working in a restaurant … when I had to, as they say, grow my bones after culinary school, you work in a restaurant, and you gain that experience. And one of the …

GEHRKE: So in your sabbatical, you didn’t only go to culinary school; you actually worked in this restaurant, as well?

STORY: Oh, it’s required.

GEHRKE: It’s a requirement? OK.

STORY: Yeah, yeah, it’s a requirement that you understand, you familiarize yourself with the rigor. So one of the things we used to do is … there was a Denny’s next to LAX in Los Angeles. Because I was attending school in Pasadena. And I would go and sign up to be the fry cook at a Denny’s that doesn’t close. It’s 24 hours.

GEHRKE: Yup …

STORY: And these people would come in, these taxis would come in, and they need to eat, and they need to get in and out.

GEHRKE: As a student, I would go to Denny’s at absurd times …

STORY: Oh, my, it was like drinking from a fire hose. I was getting crushed every night. But after a while, you know, within two or three weeks, I was like a machine, you know. And it was just like, oh, that’s not a problem. Oh, I have five orders here of this. And I need to make sure those are separated from these orders. And you have this entire process, this organization that happens in the back of your mind, you know. And that’s part of it. I mean, every job I’ve ever had, there’s always going to be a time pressure.

GEHRKE: But it must be even more difficult in research because you’re not building like, you know, Denny’s, I think you can fry probably five or 10 different things. Whereas here, you know, everything is unique, and everything is different. And then you, you know, you learn and improve and fail.

STORY: Yes, yes. But, I mean, it’s … but it’s the same as dealing with customers. Everyone’s going to have a different need and a different … there’s something that everyone’s bringing unique to the table. And when I was working at Denny’s, you’re going to have the one person that’s going to make sure that, oh, they want something very, very specific on their order. It’s no different than I’m working with, you know, somebody I’m offering a service to in a research environment.

GEHRKE: Mm-hmm. Mm-hmm. That’s true. I hadn’t even thought about this. Next time when I go to a restaurant, I’ll be very careful with the special orders. [LAUGHTER]

STORY: That’s why I’m exceptionally kind to those people who work in restaurants because I’ve been on the other side of the line.

GEHRKE: So, so you have seen many sides, right? And you especially are working also across developers, PMs, researchers. How do you bridge all of these different gaps? Because all of these different disciplines come with a different history and different expectations, and you work across all of them.

STORY: There was something somebody said to me years ago, and he says, never be the smartest guy in the room. Because at that point, you stop learning. And I was very lucky enough to work with great people like Mike Sinclair, Bill Buxton—visionaries. And one of the things that was always impressed upon me was they really let you shine, and they stepped back, and then when you had your chance to shine, they would celebrate you. And when it was their time to shine, you step back and make sure that they overshined. So it’s being extremely receptive to every idea. There’s nothing, there’s no … what do they say? The only bad idea is the lack of …

GEHRKE: Not having any ideas …

STORY: … having any ideas.

GEHRKE: Right, right …

STORY: Yeah. So being extremely flexible, receptive, willing to try things that even though they are uncomfortable, that’s I think where people find the most success.

GEHRKE: That’s such great advice. Reflecting back on your super-interesting career and all the different things that you’ve seen and also always stretching the boundaries, what’s your advice for anybody to have a great career if somebody’s starting out or is even changing jobs?

STORY: Gee, that’s a tough one. Starting out or changing—I can tell you about how to change jobs. Changing jobs … strip yourself of your ego. Be willing to be the infant, but also be willing to know when you’re wrong, and be willing to have your mind changed. That’s about it.

GEHRKE: Such, such great advice.

STORY: Yeah.

GEHRKE: Thanks so much, Lex, for the great, great conversation.

STORY: Not a problem. You’re welcome.

[MUSIC]

To learn more about Lex and to see pictures of him as a child or from his time in the Marines, visit aka.ms/ResearcherStories.

[MUSIC FADES]

In this episode, I’m talking with Emre Kiciman, the senior principal research manager leading the AI for Industry research team at Microsoft Research Redmond. After completing a PhD in systems and networking in 2005, Emre began his career with Microsoft Research in the same area, studying reliability in large-scale internet services. Exposure to social data inspired him to refocus his research pursuits: his recent work in causal analysis—including DoWhy, a Python library for causal inference—is helping to connect the whats and whys in the abundance of data that exists. Meanwhile, his work with large language models is geared toward making AI systems more secure and maximizing their benefit to society. Here’s my conversation with Emre, beginning with some of his work at Microsoft Research and how he landed in computer science. 

GEHRKE: Welcome to What’s Your Story. So can you just tell us a little bit about what you do at MSR [Microsoft Research]?

KICIMAN: Sure. I work primarily on two areas at the moment, I guess. One is causal analysis, where we work on trying to answer cause-and-effect questions from data in a wide variety of domains, kind of, building that horizontal platform. And I work a lot recently, especially with this large language model focus, on the security of AI-driven systems: how do we make sure that these AI systems that we’re building are not opening up new vulnerabilities to attackers? 

GEHRKE: Super interesting. And maybe we can start out even before we go more in depth into that by, you know, how did you actually end up in computer science? I learned that you grew up in Berkeley. 

KICIMAN: Yeah, on average, I like to say.  

GEHRKE: On average? [LAUGHTER] 

KICIMAN: So I moved to the US with my parents when I was 2 years old, and we lived in El Cerrito, a small town just north of Berkeley. And then around middle school age, we moved to Piedmont, just south of Berkeley. So on average, yes, I grew up in Berkeley, and I did end up going there for college. And you asked about how I got into computer science. When I was probably around third or fourth grade, my dad, who was a civil engineer, decided that he wanted to start a business on the side, and he loved software engineering and wanted to build software to help automate a lot of the more cumbersome design tasks in the design of steel connections, and so he wrote … he bought a PC and brought it home and started working on his work. But then that was also my opportunity to learn what a computer was. 

GEHRKE: So that was your first computer? Was it an x86? 

KICIMAN: Yes, it was an IBM PC, the first x86, the one before the 286. And—it wasn’t the very original PC. It did have a CGA—color graphics adapter—so we could have four colors at once.  

GEHRKE: Nice. 

KICIMAN: And, yeah, that’s … it came with—luckily for me, I guess—it came with a BASIC manual. So reading that manual is how I learned how to program. 

GEHRKE: And this is the typical IBM white box with a monitor on top of it and a floppy drive, or how should I picture it? 

KICIMAN: Yeah, two floppy drives …  

GEHRKE: Two floppy drives? OK …  

KICIMAN: Two floppy drives, yeah, so you could copy from one to the other.  

GEHRKE: Five and a quarter or three and a half? 

KICIMAN: Five and a quarter, yeah, yeah. The loud, clickety-clack keyboard and, yeah, a nice monitor. So not the green and black; the one that could display the colors. And, yeah, had a lot of fun with programming. 

GEHRKE: So what were some of the first things that you wrote? 

KICIMAN: A lot of the first ones were just the examples from the book, the for loops, for example. But then after that, I started getting into some of the, you know, building, like, little mini painting tools. You know, you could move a cursor around the screen, click a button and paint to fill in a region, and then save the commands that you did to make graphics. Eventually, that actually turned into, like, a friend and I really enjoyed playing computer games, so we had in our mind we’re going to build a computer game. 

GEHRKE: Who doesn’t think that.  

KICIMAN: Of course, right? 

GEHRKE: Of course … 

KICIMAN: And so we had, like, a “choose your own adventure”–style program. I think we had maybe even four or five screens you could step through, right. And he was able to get some boxes, and we printed some manuals even. We had big plans, but then we didn’t know what to do, how to finish the game, how to get it out there, so … but we had a lot of fun.  

GEHRKE: Wow, that sounds amazing. 

KICIMAN: Really fond memories, yeah. 

GEHRKE: That sounds amazing. And then you went to Berkeley afterwards? Is that how you realized your passion, or how do you decide to study computer science?

KICIMAN: Yeah … so from that age, I was set on computing. I think my parents were a bit of a devil’s advocate. They wanted me to consider my options. So I did consider, like, mechanical engineering or industrial engineering in, like, maybe junior year of high school, but it never felt right. I went into computing, had a very smooth transition into Berkeley. They have a local program where students from the local high school can start to take college classes early. So I’d even started taking some computer classes and then just went right into my freshman year. 

GEHRKE: Sounds like a very smooth transition. Anything bumpy? Anything bumpy on the ride out there, or …?  

KICIMAN: Nothing really, nothing really bumpy. I had one general engineering class that somehow got on my schedule at 8 AM freshman year. 

GEHRKE: [LAUGHS] That’s a tough one.  

KICIMAN: That’s a tough one, yeah. And so there were a few weeks I didn’t attend class, and I knew there was a midterm coming up, so I show up. Because, you know, next week, there’s a midterm. I better figure out what they’re, what they’re learning. And I come in a couple minutes late because it’s, even though I’m intending to go, it’s still an 8 AM class. I show up a few minutes late, and everyone is heads down writing on pieces of paper. The whole room is quiet. And the TA gives me a packet and says, you might as well start now. “Oh no.” And I’m like freaking out. Like this is, this is a bad dream. [LAUGHS] And I’m flipping through … not only do I not know how to answer the questions; I don’t understand the questions, like the vocabulary. It’s only been three weeks. How did they learn so much? And then I noticed that it’s an open-book exam and I don’t have my book on top of it, like … but what I didn’t notice and what became apparent in about 20 minutes … the TA clapped his hands, and said, “All right, everyone, put it down. We’ll go over the answers now.” It was a practice. 

GEHRKE: Oh, lucky you. 

KICIMAN: Oh, my god, yes. So I did nothing but study for that exam for the next week and did fine on it. 

GEHRKE: So you didn’t have to drop the class or anything like that? 

KICIMAN: No, no, no. I studied enough that I did reasonably, you know, reasonably well.  

GEHRKE: At what point in time was it clear to you that you wanted to do a PhD or that you wanted to continue your studies? 

KICIMAN: I tried to explore a lot during my undergrad, so I did go off to industry for a summer internship. Super fun.  

GEHRKE: Where did you, where did you work?  

KICIMAN: It was Netscape. 

GEHRKE: Oh Netscape. 

KICIMAN: And it was a joint project with IBM. 

GEHRKE: Which year was that in? 

KICIMAN: This would have been ’90, around ’93.¹ 

GEHRKE: ’93 … OK, so the very early days of Netscape, actually. 

KICIMAN: Yeah, yeah. They were building Netscape Navigator 4, and the project I was on was Netscape Navigator for OS/2.  

GEHRKE: OK.

KICIMAN: IBM’s OS/2 had come out and was doing poorly against NT, and they wanted to raise its profile. And this team of 20 people were really just focused on getting this out there. And so I always thought of, you know—and I was an OS/2 user already, which is how I got onto that project. 

GEHRKE: OK … And how was the culture there, or …?  

KICIMAN: The culture, it’s what you would think of as a startup culture. You know, they gave out all their meals. There was lots of fun events. You know, dentists came into the parking lot like once a month or something like that. 

GEHRKE: Dentist?  

KICIMAN: There was, like, a yeah, it was, yeah, you know, everyone’s working too much at the office, so the company wanted to make things easy.  

GEHRKE: That sounds great. 

KICIMAN: But the next summer then, I did a research internship, a research assistantship, at Berkeley. I worked with Randy Katz and Eric Brewer and got into, you know, trying to understand cellphone networks and what they were thinking about, you know, cloud infrastructure for new cellular technologies. 

GEHRKE: And Eric Brewer, was he, at that point in time, already running Inktomi, or … ? 

KICIMAN: He was already running Inktomi. Yeah, yeah, he’d already started it. I don’t think it was public yet at the time, but maybe getting there.  

GEHRKE: OK. Well, this was right at the beginning when, like, all the, you know, cloud infrastructure was defined and, you know, a lot of the basics were set. So you did this internship then in your, after your junior year, the second one?  

KICIMAN: Yeah, after my junior year. It was then senior year, and it was time to apply for, you know, what’s going to come after college. And I knew it … after that assistantship at Berkeley, I knew I was going to go do a PhD. 

GEHRKE: So what is the thing about the internship that made you want to stay in research? 

KICIMAN: Oh, it’s just the … it gave a vision of the future. Like, we were playing with, like, you know, there were people in the lab playing with video over the internet and, you know, teleconferencing, and just seeing that, it felt like you were seeing into the future and diving deep technically across the stack in a way that the industry internship hadn’t done. And so that part of it and obviously lots of particulars. You know, lots of internships do go very deep in industry, as well, but that’s what struck me, is that, kind of, wanting to learn was the big driver.  

GEHRKE: And what excited you about systems as compared to something that’s more applications-oriented or more touching the user? I feel like systems you always have to have this, kind of, drive for infrastructure and for scale and for, you know, building the foundation as compared to, like, directly impacting the user. 

KICIMAN: I think the way I think about systems today—and I can’t remember what it was about systems then. I’d always done operating … like, operating systems was one of my first upper-division courses at Berkeley and everything. So, like, I certainly enjoyed it a lot. But the way I think about systems now—and I think I do bring systems thinking to a lot of the work I do, even in AI and responsible AI—is the way you structure software, it feels like you should be making a statement about what the underlying problem is, what is the component you should be building from an elegance or first-principles perspective. But really, it’s about the people who are going to be using and building and maintaining that system. You want to componentize it so that the teams who are going to be building the bigger thing can work independently, revise and update their software without having to coordinate every little thing. I think that’s where that systems thinking comes in for me, is what’s the right abstraction that’s going to decouple folks from each other. 

GEHRKE: That’s a really great analogy because the way it was once told to me was that systems is really about discovering the beauty in large software. Because once you touch the user, you, sort of, have to do whatever is necessary to, you know, make the user happy. But in the foundations, you should have simplicity; you should have ease; you should have elegance. Is that how you think about it? 

KICIMAN: I do think about those aspects, but it’s for a purpose. You know, you want the elegance and the simplicity so that you can have, you know, one team working on Layer 1 of the stack, another team working on Layer 2 of the stack, and you don’t want them to have to talk to each other every 10 minutes when they’re making any change to any line of code, right. And so thinking about, what is the more fundamental layer of abstraction that lets these people work on separate problems? That’s what’s important to me. And, of course, like, that then interplays with people’s interests and expertise. And as people’s expertise evolves, that might mean that that has implications for the design of your system.  

GEHRKE: And so you’re, OK, you’re an undergrad. You have done this research experience; you now apply. So now you go to grad school. Do you do anything fun between your undergrad and grad school? 

KICIMAN: No, I went straight in. 

GEHRKE: Right straight in?  

KICIMAN:  Right straight in. I did my PhD at Stanford. So I went, you know, a little way to school. 

GEHRKE: To a rival school, isn’t it? Isn’t it a big rival school? 

KICIMAN: To a rival school. Well, the undergrad school wins. I think that’s the general rule of thumb. But I did continue working with folks at Berkeley. So my adviser was also from Berkeley and so …  

GEHRKE: Who was your adviser? 

KICIMAN: My adviser was Armando Fox, …  

GEHRKE: OK, yeah. Mm-hmm.  

KICIMAN: … and we had a … 

GEHRKE: Recovery-oriented computing? 

KICIMAN: Yes, exactly. Recovery-oriented computing. And the other person on the recovery-oriented computing project …  

GEHRKE: Dave Patterson …  

KICIMAN: … was Dave Patterson, yeah. 

GEHRKE: So it was really a true, sort of, Stanford-Berkeley joint project in a way?

KICIMAN: Yes, yeah. And that was my PhD. The work I did then was the first work to apply machine learning to the problem of fault detection and diagnosis in large-scale systems. I worked with two large companies—one of them was Amazon; one of them was anonymous—to test out these ideas in more realistic settings. And then I did a lot of open-source work with J2EE to demonstrate how you can trace the behavior of a system and build up models of its behavior and detect anomalies. Funnily enough, I know this is going to sound a little alien to us now maybe in today’s world: Dave and Armando would not let me use the phrase “artificial intelligence” anywhere in my thesis because they were worried I would not be able to get a job. 

GEHRKE: I see. Because that was, sort of, one of … I mean, AI goes through these hype cycles and then, you know, the winters again, and so this was one of the winter times? 

KICIMAN: This was definitely a wintertime. I was able to use the phrase “machine learning” in the body of the thesis, but I had to make up something about statistical monitoring for the title. 

GEHRKE: So what is the actual final title of your thesis, if you remember it? 

KICIMAN: “Statistical monitoring for fault detection and diagnosis in large-scale internet services” or something like that. 

GEHRKE: Makes sense. 

KICIMAN: Yeah. 

GEHRKE: So you replaced AI with statistical modeling and then everything [turned out all right]? 

KICIMAN: Yes, yeah. Everything … then it didn’t sound too hype-y. 

GEHRKE: And then after your PhD, you went straight to MSR, is that right? 

KICIMAN: Yeah. I mean, so here I’m coming out of my PhD with a focus on academic-style research for large-scale systems. Kind of boxed myself in a little bit. No university has a large-scale internet service, and most large-scale internet service companies don’t have research arms. So Microsoft Research was actually the perfect fit for this work. And when I got here, I started diving in and actually expanding a little bit and thinking about what are the end-to-end reliability issues with our services. So assume that the back end is running well. What else could go wrong that’s going to get in the way of the user? So I had one project going on, wide area network reliability with David Maltz, and one project …  

GEHRKE: Who is now CVP in Azure.  

KICIMAN: Who’s now, yeah, leading Azure network—the head of Azure networking. And one project on how we can monitor the behavior of our JavaScript applications that were just starting to become big. Like around then is when, you know, the first 10,000-line, 100,000-line-of-code JavaScript applications [were] appearing, and we had no idea whether they were actually running correctly, right? They’re running on someone else’s browser and someone else’s operating system. We didn’t know.  

GEHRKE: A big one at that point in time, I think was Gmail, right? This was, sort of, a really big one. But did we have any big ones in Microsoft? 

KICIMAN: Gmail was the first big one in the industry. 

GEHRKE: Hotmail, was it also Java, based in JavaScript? 

KICIMAN: Hotmail was not initially JavaScript based. The biggest one at that time was our maps. Not Bing maps, but whatever we called it.  

GEHRKE: MSN maps, or …  

KICIMAN: Probably something like that, yeah, yeah.  

GEHRKE: I see. And so you applied your techniques to that code base and tried to find a lot of bugs? 

KICIMAN: Yeah, this project was—and this was about data gathering, right, so I’m still thinking about it from the perspective of how do I analyze data to tell me what’s going on. We had data for the wide area network, but these web applications, we didn’t have any. So I’m, like, I’m going to build this infrastructure, collect the data, so that in a couple years, I can analyze it. And so what I wrote was a proxy that sat on the side of the IAS server and just dynamically instrumented all the JavaScript that got shipped out. And the idea was that no one user was going to pay the cost of the instrumentation, but everyone would pay a little small percentage, and then you could collect it in the back end to get the full complete picture.  

GEHRKE: Right. It’s so interesting because, I mean, in those days, right, you still thought maybe in terms of years and so on, right. I mean, you’ve said, well, I instrumented, then maybe in a year, I have some data. And today it happens that I instrument, and tomorrow I have enough data to make a decision on an A/B test and so on, right. It was a very different time, right. And also, it was probably a defining time for Microsoft because we moved into online services, right. We moved into large-scale internet services. So it must have been exciting to be in the middle of all of this. 

KICIMAN: It really was. I mean, there was a lot of change happening both inside Microsoft and outside Microsoft. That’s when … soon after this is when social networking started to become big, right. You started seeing Facebook and Twitter show up, and search became a bigger deal for Microsoft when we started investing in Windows Live and then Bing, and that’s actually … my manager, Yi-Min Wang, actually joined up with Harry Shum to create the Internet Services Research Center with the specific focus of helping Bing. And so that also shifted my focus a little bit and so had me looking more at some of the social data that would, kind of, take my trajectory on a little bit further.

GEHRKE: Right. I mean, so you’re unique in that, you know, people very often, they come in here and, you know, they’re specialists in systems, and they branch out within systems a little bit and, you know, of course, move with time. Maybe now they do, you know, AI infrastructure. But you have really moved quite a bit, right. I mean, you did your PhD on systems … I mean, systems and AI really, the way I understand it. Then you worked here a little bit more on systems in wide area and large-scale systems. But then, you know, you really became also an expert in causality and looked at, sort of, the social side. And now you, of course, have started to move very deeply into LLMs. So rather than talking about the topics itself, how do you decide? How do you make these decisions? How do you … you know, you’re a world expert on x, and how do you, in some sense, throw it all away and go to y? Do you decide one day, “I’m interested in y“? Do you, sort of, shift over time a little bit? How do you do it? 

KICIMAN: I’ve done it, I think, two or maybe three times, depending on if you count now, and some transitions have gone better than others. I think my transition from systems to social data and computational social science, it was driven by a project that we did for search at the time. Shuo Chen, another researcher here at Microsoft Research, built a web application that lets you give very concrete feedback back to Windows Live. You could drag and drop the results around and say, this is what I wanted it to look like. And this made, you know, feedback much more actionable and helped really understand DSATs and where they’re coming from. DSAT being dissatisfactions. And I looked at that and I was like, I want to be able to move search results around and share with my friends. And I, kind of, poked at Shuo, you know, asked him if he would build this, and he said no. He said he’s busy. So eventually, I—because I knew something about JavaScript applications—decided to just drop things and spend six months building out this application. So I built out this social search application where you could drag and drop search results around, share it with your friends, and we put it out, actually. We got it deployed as an external service. We had maybe 10,000 people kick the tires.  

GEHRKE: Within Microsoft or …?  

KICIMAN: No, externally.  

GEHRKE: OK.  

KICIMAN: Yeah. There was a great headline that, like, Google then fast followed with a similar feature, and the headline was like, Google fast follows, basically, on Microsoft. Our PR folks were very excited about that. I say this all … I mean, it’s all history now. But certainly, it was fun at the time. But now we’re … I’m giving this demo, this talk, about this prototype that we built and what we’re learning about, you know, what’s in people’s way, what’s friction, what do they like and not like, etc. And I’m standing up and, you know, giving this presentation, this demo, and someone says, hey could you, could you go back to, you know, go back in the browser? On the bottom right corner, it says Mike did something on this search page; he edited some search results. Could you click on that? I want to know what he did. I’m like, OK, yeah, sure. I click on it. And [it’s like], OK, that’s great. That’s, that’s really interesting. And this happened multiple times. Like, in a formal presentation, for someone to interrupt you and ask a personal question just out of their own curiosity, that’s what showed me … that’s what got me really thinking deeply about the value of this social data and, like, why is it locked up in a very specific interface. What else could you do with this data if it’s so engaging, so fascinating, that people are willing to interrupt a speaker for some totally irrelevant, basically, question? And that’s when I switched to really trying to figure out what to do with social data. 

GEHRKE: I see. So it was this, kind of, really personal experience of people being so excited about that social interaction on the demos that you’re giving. 

KICIMAN: Exactly. They cared about their friends and what their friends did, and that was super clear.

GEHRKE: So, so coming back, let’s go there in a second, but coming back to the story that you told, you said you had 10,000 external users. 

KICIMAN: Yeah.

GEHRKE: So I’m still, you know, also always trying to learn what we can do better because we sometimes have prototypes that are incredibly valuable. They’re prototypes that have fans; they’re prototypes that, you know, the fans even want to contribute. But then somehow, we get stuck in the middle; and they don’t scale, and they don’t become a business. What happened with that?

KICIMAN: Yeah. 

GEHRKE: Also in [retrospect], … 

KICIMAN: In retrospect … 

GEHRKE: … what, what … should we have done something different, or did it live up to its potential? 

KICIMAN: I think we learned something. I think that there were a couple of things we learned. One was that, you know, every extra click that people wanted to do, you know, took the number of interactions down by, you know, an order of magnitude. So starring something and bringing it to the top, that was very popular. Dragging and dropping? Little bit less so. Dragging and dropping from one search to a different search? So maybe I’ll search for, you know, “Johannes,” find your homepage, and then drag and drop it to, like, people’s, you know, publications list to, like, keep an eye on or something. Like that, almost never. And people were very wary about editing the page. Like, what if I make a mistake? What if it’s just, just me, like, who wants this, and I’m messing up search for the rest of the world? And it’s like, no, no, it’s just your friends, like just you and your friends who are going to see this. And so we learned a lot about people’s mental models and, like, what stood in the way of, you know, interactions on the web. There were lots of challenges to doing this at scale. I mean, we needed, for example, a way of tracking users. We needed a way of very quickly, within 100 milliseconds, getting information about a user’s past edits to search pages into, you know, into memory if we were going to do this for real on Windows Live. And we just didn’t have the infrastructure.

GEHRKE: I see. And those problems were hard in those days. 

KICIMAN: Yeah. A prototype is fine. People, you know, will handle a little bit of latency if it’s a research prototype, but for everyday use, you need something more. 

GEHRKE: And there was no push to try it, to land it somehow, or what … ?  

KICIMAN: There were big pushes, but the infrastructure, it was really … 

GEHRKE: I see. It was really an infrastructure problem, then? 

KICIMAN: Yeah, yeah. 

GEHRKE: OK. Interesting because it sounds to me like, wow, there’s an exciting research problem there; now you need the infrastructure to try to make all of these things really, really fast. It’s always fascinating to see, you know, where things get stuck and how they, how they proceed. 

KICIMAN: Yeah, I think it’d be a lot easier to build that—from an infrastructure point of view—today. But, of course, then there’s lots of other questions, like is this really what, you know, the best thing to do. Like I mentioned, Google had this fast follow feature. They also removed it afterwards, as well.  

GEHRKE: OK. Yeah, hindsight is always, you know, twenty-twenty. So, OK, so you’re now starting to move into social computing, right, and trying to understand more about social interactions between users. How did you end up in causality, and then how did you make the switch to LLMs? And maybe even more about this; I mean, I understand here this was, sort of, this personal story that you really saw that, you know, the audience was really asking you about what’s happening here and that, sort of, motivated you. Was it always this personal drive, or was it always others who pulled you? And how did you make these switches? 

KICIMAN: I think the switch from systems into social, it was about trying to get closer to problems that really mattered to people. I really enjoy working on systems problems, but oftentimes, they feel like they’re in the back end. And so I wanted something where, you know, even if I’m not the domain expert working on something, I can feel like I’m making a contribution to that problem. The transition with social data then into causality and, um, and LLMs, that was a bit smoother. So working with social data, trying to understand what it meant and what it said about the world in aggregate, was super-fascinating problems. So much information is embedded in the digital traces that people leave behind. But it was really difficult for people to come to solid conclusions. So there was one conference I went to where almost every presentation that day gave some fascinating insight. This is how people make friendships. This is how, you know, we’re seeing, like, signs of disease spread in, you know, through real-world interactions as they’re in social data. Here’s how people spend their time. And then people would, and then people would close; their conclusion slide every time was, “And, of course, correlation is not causation, so anything could actually be happening.” Like, that is such, that is such a bummer. Like, beautiful theory, great understanding. You spent so much time. I feel like I got some insight. And then you pull the rug out and say, but maybe not. And I’d heard about this work on … that there was work on causal analysis and that there were certain conditions and ways to get actual learned causal relationships from data. So that’s the day I decided I’m going to go figure out what that is and how to apply it to social data for these types of questions. And I went out, and the first work there was a collaboration with Munmun De Choudhury, faculty at Georgia Tech, looking at online traces related to mental health and suicidal ideation and trying to understand what some of the factors were in a more, in a more solid and causal fashion. And so this really became, like, this was … this interest in computational social science really ended up branching out into two areas. One, obviously, I’m caring about, what can we learn about the world? Part of this is, of course, thinking deeply about the implications of AI on society, like what is it going to mean that we have this data for all of these, you know, societal challenges? And then causality. So the AI and its implications on society is what led towards the work on the security of AI systems and now security of AI as it relates to large language models. And then causality was the other branch that split off from there. Both of them really stemming from this desire to see that we have a positive impact with AI.

GEHRKE: So you mentioned that, you know, you were sitting in these talks and people are talking about the correlation, and now you finally have this new tool, which is causation. So what are some of the examples where, you know, with correlation you came out with answer A, but now causation gave you some better, some real deep insights? 

KICIMAN: I haven’t gone looking to refute studies, so … 

GEHRKE: I see. OK.  

KICIMAN: … but there are many well-known studies in the past where people have made mistakes because they didn’t account for the right confounding variables. Ronny Kohavi has a great list of these on one of his websites. But a fun one is a study that came out in the late ’90s on the influence of night lights on myopia in children. So this was a big splash. I think it made it to like Newsweek or 60 Minutes and stuff, that if you have night lights in the house, your kids are more likely to need glasses. And this was wrong. 

GEHRKE: My parents told me all the time, don’t read in bed, you know, with your flashlight because your eyes are going to get bad. 

KICIMAN: Yes.  

GEHRKE: That’s the story basically, right? 

KICIMAN: This was, yeah, the night lights that plug in the wall.  

GEHRKE: But that’s the …  

KICIMAN: That’s the idea, the same thing. 

GEHRKE: The same thing, right. 

KICIMAN: And so these people analyzed a bunch of data, and they found that there was a correlation, and they said that, you know, it’s a cause; you know, this is a cause. And the problem was that they didn’t account for the parents’ myopia. Apparently, parents who had myopia were more likely to install night lights. And then you have the genetic factor then actually causing the myopia. Very simple. But, you know, people have to replicate this study to, you know, to realize it was a mistake. Others were things like correlations, I think, around vitamin C have been reported repeatedly and then refuted in randomized control trials. But there’s many of these. Medicine, in particular, has a long history of false correlations leading people astray. 

GEHRKE: Do you have a story where here at Microsoft your work in causation had a really big impact? 

KICIMAN: You know, the one—it’s still ongoing—but one of the ones that I’m really excited about now, and thinking also from the broader societal impact lens, is a collaboration with Ranveer Chandra and his group. So with a close collaborator at MSR India, Amit Sharma, we’ve developed a connection between representation learning and underlying causal representation of the data-generating process that’s driving something. So if you imagine, like, we want to learn a classifier on an object, on an image, and we want that classifier to generalize to other settings, there’s lots of reasons why this can go wrong. You know, you have, you know, like a classic example is the question of, is this picture showing you a camel, or is it showing you a cow? The classifier is much more likely to look at the background, and if it’s green grass, it’s probably a cow. If it’s sandy desert, it’s probably a camel. But then you fail if you look at a camel in the zoo or a cow on a beach, right. So how do you make sure that you’re looking at the real features? People have developed algorithms for these. But no algorithm actually is robust across all the different kinds of distribution shifts that people see in the real world. Some algorithms work on these kinds of distribution shifts. Some algorithms work on those kinds of distribution shifts. And it was a bit of an interesting, I think, puzzle as to why. And so we realized that these distribution shifts, if you look at them from a causal perspective, you can see that the algorithms are actually imposing different statistical independence constraints. And you can read those statistical independence constraints off of a causal graph. And the reason that some algorithms worked well in some settings was that the underlying causal graph implied a different set of statistical independence constraints in that setting. And so that algorithm was the right one for that setting. If you have a different causal graph with different statistical independence constraints, the other algorithm was better. And so now you can see that no one algorithm is going to work well across all of them. So we built an adaptive algorithm that looks at the causal graph, picks the right statistical independencies, and applies them, and now what we’re doing with this algorithm is we’re applying it to satellite imagery to help us build a more generalizable, more robust model of carbon in farm fields so we can remotely sense and predict what the carbon level is in a field. And so, the early results …  

GEHRKE: And that’s important for what?

KICIMAN: And so this is important because soil is seen as a very promising method for sequestering carbon for a climate change perspective. And it’s also the more carbon there is … the higher your soil carbon, usually the healthier the soil is, as well. It’s able to absorb more water, so less flooding; your crops are more productive because of the microbial growth that’s happening. And so people want to adopt policies and methods that increase the soil carbon in the fields for all of these reasons. But measuring soil carbon is really intensive. You have to go sample it, take it off to a lab, and it’s too expensive for people to do regularly. And so if we can develop remote-sensing methods that are able to take a satellite image and, you know, really robustly predict what the real soil carbon measurement would be, that’s really game changing. That’s something that, you know, will help us evaluate policies and whether they’re working; help us evaluate, you know, what the right practices should be for a particular field. So I’m really excited about that.  

GEHRKE: That’s really exciting. You’d mentioned when we talked before that you’d benefited in your career from several good mentors. How do you think about mentoring, and what are the ways that you benefited from it? And how do you, you know, live that now in your daily life as you’re a mentor now to the next generation? 

KICIMAN: Yeah, the way I look at all the people—and there’s so many—who have, you know, given me a hand and advice and, you know, along the way, I often find I pick up on some attributes of my mentors, of a particular mentor, and find that it’s something that I want to emulate. So recognizing, you know, everyone is complicated and no one is perfect, but, you know, there’s so many ways that, you know, individuals get things right and trying to understand what it is that they’re doing right and how I can try and repeat that for, like, you said, the next generation, I think, is really, really important. It’s like one story, for example, around 2008, while I was still working on large-scale internet services, I was going around the company to, kind of, get a sense of, you know, what’s the current state of the reliability of our services and how we architect them and run them. And so I was talking to developers and architects and Ops folks around the company, and James Hamilton was a great mentor at that moment, helping me to connect, helping suggest questions that I might ask. 

GEHRKE: So he was working on SQL Server reliability, right, at that point in time or on Windows reliability? 

KICIMAN: He was already starting to move over into datacenter reliability. I think at the time, right before he moved over to the research side of things, I think he was one of the heads of the, of our enterprise email businesses, and then he came over to research to focus on, I think, datacenters in general. And, yeah, and he just donated so much of his time. He was so generous with, you know, reviewing this large report that I was writing and just helping me out with insights. That struck me as, like … he’s a very busy person. He’s doing all this stuff, and he’s spending, you know, I sent him an email with, you know, 15 pages, and he responds with feedback within a couple of hours every morning. That was astonishing to me, especially in hindsight, and so … but that kind of generosity of time and trying to help direct people’s work in a way that’s going to be most impactful for what they want to achieve, that’s something I try and emulate today. 

GEHRKE: So, so, you know, you’ve benefited from a lot of great mentors and you said you’re now also a mentor to others. Do you have any last piece of advice for any of our listeners? 

KICIMAN: I think it’s really important for people to find passion and joy in the work that they do and, at some point, do the work for the work’s sake. I think this will drive you through the challenges that you’ll inevitably face with any sort of project and give you the persistence that you need to really have the impact that you want to have. 

GEHRKE: Well, thanks for that advice. And thanks for being in What’s Your Story, Emre. 

KICIMAN: Thanks very much, Johannes. Great to be here.  

[MUSIC] 

To learn more about Emre or to see photos of Emre as a child in California, visit aka.ms/ResearcherStories. 

[MUSIC FADES] 

In this episode, I’m talking with Principal PM Manager Weishung Liu. Wei has used her love of storytelling and interest in people and their motivations to deliver meaningful products and customer experiences. This includes the creation of a successful line of Disney plush toys and contributions to the satellite internet system Starlink. With Microsoft, she helped develop Watch For, a real-time video analytics platform that has gone on to enhance gaming via streaming highlights and to support content moderation in products such as Xbox. Today, she’s facilitating connections and devising strategies to empower teams within Microsoft Research to maximize their reach. Here’s my conversation with Wei, beginning with her childhood in Silicon Valley.

JOHANNES GEHRKE: Hi, Wei. Welcome to What’s your Story. You’re our principal PM manager here in the lab, and we’ll talk in a little while about, you know, what you’re doing here right now, but maybe let’s start with, how did you actually end up in tech? Where did you grow up?

WEISHUNG LIU: Oh, wow. OK. So this is a very long, long and, like, nonlinear story about how I got into tech. So I grew up in Silicon Valley, which one would assume means just, like, oh, yes, you grew up in Silicon Valley; therefore, you must be in the STEM field, and therefore, you will be in tech for the rest of your life.

GEHRKE: Yep, that’s, sort of, a too familiar a story.

LIU: That’s a very linear story. And I totally actually wanted to rebel against that whole notion of going into tech. So I grew up in Silicon Valley and thought, like, man, I want to not do STEM.

GEHRKE: So did your parents want you to be either a doctor or engineer? Is that the … ?

LIU: Absolutely. It was either a doctor, engineer, or lawyer. So thankfully my sister went the PhD in psychology route, so she, kind of, checked that box for us. And so I was a little bit more free to pursue my very, very, very wide variety of interests. So a little bit of personal information about me. So I grew up a very sick child, and so I was hospitalized a lot. I was in the ER a lot. But that actually afforded me a lot of opportunities to be, sort of, an indoor-only child of reading and playing video games and all sorts of things that I would say, like, expanded my worldview. Like, it was just all sorts of different stories. Like, reading has stories; video games have stories.

GEHRKE: Tell us a story about reading and a story about video games. What …

LIU: Oh my goodness …

GEHRKE: … were your favorite set of books?

LIU: I was really interested in, like, historical fiction at the time. One book that I remember reading about—oh my gosh, it’s a very famous book, and I don’t remember the name anymore. However, it was about a young girl’s perspective of being, living in an internment camp, the Japanese internment camps, back during World War II, I believe, after Pearl Harbor.[1] And it was just kind of her diary and her perspective. It was almost like Diary of Anne Frank but from a Japanese American girl’s perspective instead. And I just loved, kind of, reading about different viewpoints and different eras and trying to understand, like, where do we overlap, how do things change over time, how does history repeat itself in some ways? And, and I love that. And then video games. So I was really into Japanese RPGs back in the day. So it’s funny. I started … my first console was a Mattel Intellivision II, and then it gradually went up to like Nintendo, Super Nintendo, all those, all those consoles. But I had a friend who I used to play RPGs with …

GEHRKE: So these were network RPGs or individual RPGs?

LIU: These were individual RPGs. This is, you know, when I was around 10, the internet appeared, so it probably dates me a little bit. Every time a new RPG came out like by—the company is now called Square Enix but back then it was called SquareSoft—or Nintendo like Zelda, he and I would immediately go out and buy the game or, you know, convince our parents at the time to buy the game, and then we would compete. So, like, this is not couch co-op; he was actually in Texas.

GEHRKE: Like long-distance co-op?

LIU: This is long-distance, long-distance gaming where we would compete to see who would beat the game first.

GEHRKE: Wow.

LIU: No, you’re not allowed to use walkthroughs. And he almost always beat me.

GEHRKE: But these games are like 60-hour, 80-hour games?

LIU: Yeah, like 60- or 80-hour games, but, like, you know, we got so good at them that, well, you had to figure out like how do you, kind of, bypass and get through the main quest as fast as possible. So that was always—

GEHRKE: So any of the side quests and things like that just … ?

LIU: Yeah, oh, yeah, no. So I’m actually a huge completionist, though, so I’d always go back after and do all the side quests to get, you know, we’ll just say “100 percent” achievement. I’m a little bit of an achievement machine that way. But so, like, that kind of stuff was always super fun for me. And so I spent so much of my time then—because I was, kind of, more homebound a lot—just exploring and being curious about things. And, and that got me into art and into design, and I thought, man, I’m going to be an architect someday because I love designing experiences, like spaces for people.

GEHRKE: You thought at that point in time like a real, like a building architect or an architect for like virtual worlds or so … ?

LIU: No, real, like a real physical space that people inhabit and experience. And so, like, I avoided as much STEM as I could in school. I couldn’t, just due to where I lived and grew up and the high school requirements that I had. But the minute I went to college, which happened to be at the University of Washington, which has a great architecture program, I was like, I’m never going to take another STEM class in my life.

GEHRKE: So you enrolled as an architecture major?

LIU: I enrolled as an architecture major, and I was like, I will do what we would call the “natural world” credits, which is kind of the STEM-like things. But I would intentionally find things that were not, like, hard science because I’m like, I’m never going to do this again. I’m never going to be in tech. All these people that are so obsessed with tech who, you know, went to MIT and Stanford, and I’m like, no, no, no, I’m going to be an architecture major.

GEHRKE: So you took, like, the physics for poets class or so …?

LIU: Stuff like that, right. [LAUGHS] Very, very similar. But I ended up just loving learning at school, which is very unsurprising. You know, I took, like, an Arabic poetry class. I took a French fairy tales class. And I just, kind of, explored college and all the things that it had to offer in terms of academics so much that I actually ended up deciding to get two degrees: one in industrial design, which is not too far away from architecture. Architecture is like with large spaces, like you build one building or design one building that lasts maybe 100 years. Industrial design, I, kind of, joke about it. It’s, you know, you design smaller form factors that sometimes, if they’re manufactured with plastics, last millions of years, [LAUGHS] and you build millions of them. But then I also ended up getting a degree in comparative religion, as well. Which it meant that, like, my schooling and my class schedules are always a little bit odd because I’d go from, you know, like, the industrial design shop down in our design building and like making things with my hands and working at the bandsaw, and then I’d, you know, rush to this other class where we have like very fascinating philosophical debates about various things in, sort of, the comparative religion space. And I’d write, you know, 10-page essays and … about all sorts of things. And, you know, there’s, like, the study of death is a great example and how different cultures react to death. But, you know, that was as far away from STEM [LAUGHS] as I could have possibly gone.

GEHRKE: Right. I was just thinking, can you maybe explain to our listeners a little bit who may come a little bit more from the STEM field traditionally, what do you study in comparative [religion], and what is the field like?

LIU: So for me, it was really just, like, I took a lot of classes just trying to understand people. I really … and it sounds, kind of, silly to say it that way, but religion is really formed and shaped by people. And so for me, like, the types of classes that I took were, sort of, like studying Western religion, studying Eastern religion, studying the philosophy of religion, like or even—and this still, I still think about it from time to time—how do you define religion? And just even … there’s still so many scholarly debates about how to define, like, what is a “pure” definition of religion, and nobody can really still identify that yet. Is it, you know, because then there’s this distinction of spiritualism and being religious versus something else or just completely made-up, you know, pseudoscience, whatever, right. People have this wide spectrum of things that they describe. But it’s really around learning about the different foundations of religion. And then people tend to specialize. You know, they might specialize in a particular area like Hinduism or, you know, broadly speaking, Eastern religions, or people will, you know, start focusing on Western religions. Or sometimes I think about a specific topic like the intersection of, for example, religion and death or religion and art or even, you know, religion and violence. And there’s a broad spectrum of things that people start specializing in. And it’s very, it’s, sort of, very much in the mind but very much in the heart of how you understand that.

GEHRKE: Yeah, I can see how it even connects to industrial design because there you also want to capture the heart …

LIU: Yes.

GEHRKE: … the hearts of people, right.

LIU: Yep. And that’s kind of how I, how I describe, you know, when people are like, why did you major in that? Like, what do you even do with that? Did you even think about what career you would have with that? I’m like, no, I just really wanted to learn, and I really wanted to understand people. And I felt like religion is one way to understand, sort of, like, sociologically how people think and get into that deep, like, that deep feeling of faith and where does it come from and how does it manifest and how does it motivate people to do things in life. And to your point, it’s very similar to industrial design because you’re, you know, we talk about design thinking and you have to really deeply understand the user and the people that you’re designing for in order to create something that really lasts, that matters to them. So that’s, kind of, my, at least my undergrad experience. And in a very, very brief way, I’ll just kind of walk through or at least tell you the very nonlinear path that I took to get to where I am here now at Microsoft Research. So like the day after I graduated from the University of Washington, I moved to Florida.

GEHRKE: And just as a question: so you graduated from the University of Washington—did you have like a plan, you know, this is like the career I want to have?

LIU: Oh no! So here’s the funny thing about design, and I hope that, you know, my other, the designers who might be watching or listening [LAUGHS] to this might not get upset—hopefully don’t get upset with me about this—is I love the design thinking aspect of design, like understanding why people do the things they do, what types of habits can you build with the products—physical products? I was very obsessed with physical, tangible things at the time. And then I learned through, like, internships and talking to other designers who were, you know, already in the field that that’s not what they do. That they don’t go and like, oh, let’s go talk to people and understand deeply what they do. Like, there’s other people that do that. OK, well, what do you do? Well, I work in, you know, CAD, or I work on SolidWorks, or I do Rhino, and I do surfacing. I’m like, OK, what else? Who decides what gets made? Oh, that’s like, you know, a product manager or product—oh, what’s that? Who? What? What does that even mean? Like, tell me more about that.

GEHRKE: So it’s like the dichotomy that you see even here in the company where the engineers have to, sort of, build the things, but the product managers are …

LIU: But someone else is …

GEHRKE: … in the middle

LIU: … someone else is, kind of, interpreting what the market and the users are saying, what the business is saying. And I was like, I like doing that because that’s more about understanding people and the business and the reason—the why. And so …

GEHRKE: Just before you go to your career, I mean, I must … I have to ask, what are some of the favorite things that you built during your undergrad? Because you said you really like to build physical things.

LIU: Oh my gosh!

GEHRKE: Maybe one or two things that you actually built …

LIU: Yeah …

GEHRKE: … that was, sort of, so fun.

LIU: So one of my projects was actually a Microsoft-sponsored project for one quarter, and all they showed up with—his name’s Steve Kaneko. He retired not too long ago from here. Steve showed up and said, I want you all to design a memory-sharing device.

GEHRKE: Interesting …

LIU: And that was it.

GEHRKE: So what is memory sharing? He didn’t define what that means?

LIU: He didn’t define it because as designers, that was our way of interpret—we had to interpret and understand what that meant for ourselves. And it was a very, very free-form exploration. And I thought … the place that I started from was … at the time, I was like, there’s like 6 or 7 billion people in the world. How many of them do I actually know? And then how many of them do I actually want to know or maybe I want to know better?

GEHRKE: To share a memory with …

LIU: To share my memories with, to share a part of me. Like, memories are …

GEHRKE: Pretty personal.

LIU: … who we are—or not who we are but parts of who we are—and drive who we become in some ways. And so I thought, you know, what would be cool is if you had a bracelet, and the bracelet were individual links, and each individual link was a photo, like a digital photo, very tiny digital photo, of something that you chose to share. And so, you know, I designed something at the time … like, the story I told was, like, well, you know, this woman who’s young decided to go to, you know, she’s taking the bus, and she put on her, like, “I wish to go to Paris” kind of theme, right. So she had a bunch of Parisian-looking things or something in that vein, right. And, you know, she gets on the bus and her bracelet vibrates. There’s, like, a haptic reaction from this bracelet. And that means that there’s someone else on the bus with this, you know, with a bracelet with their memories. It’s kind of an indicator that people want to share their stories with someone else. And, you know, wouldn’t it be great if, you know, this woman now sits down on the bus, because she sits next to the person who’s wearing it. Turns out to be an elderly woman who’s wearing, coincidentally, you know, her Paris bracelet, but it’s of her honeymoon of her deceased husband from many years ago. And, you know, like, think of the power of the stories that they could share with each other. That, you know, this woman, elderly woman, can share with, you know, this younger woman, who has aspirations to go, and the memories and the relationship that they can build from that. And so that was, kind of, my memory-sharing device at the time.

GEHRKE: I mean, it’s super interesting because, I mean, the way I think about this is that we have memory-sharing applications now like Facebook and Instagram and TikTok and so on, but they, the algorithm decides really …

LIU: Yes …

GEHRKE: … who to share it with and where and why to share it. Whereas here, it’s proximity, right? It somehow leads to this physical and personal connection afterwards, right? The connection is not like, OK, suddenly on my bracelet, her stories show up …

LIU: Yes …

GEHRKE: … but, you know, maybe we sit next to each other on the bus, and it vibrates, and then we start a conversation.

LIU: Exactly. It’s you own, you know, whatever content is on that you choose to have on your physical person, but you’re sharing yourself in a different way, and you’re sharing your memories and you’re sharing a moment. And it might just be a moment in time, right. It doesn’t have to be a long-lasting thing. That, you know, this elderly woman can say, hey, there’s this really great bistro that we tried on, you know, this particular street, and I hope it’s still there, because if you go, ask for this person or try this thing out and, like, what an incredible opportunity it is for this other woman, who, you know, maybe she does someday go to Paris and she does find it. And she thinks of that time, like, how grateful she was to have met, you know, this woman on the bus. And just for that brief whatever bus … however long that bus ride was, to have that connection, to learn something new about someone else, to share and receive a part of somebody else who you may never have known otherwise. And then that was, that was what I was thinking of, you know, in terms of a memory-sharing device was memory creates connections or it reinforces connections. So I guess very similarly to my people thing and being fascinated by people, like, this was my way of trying to connect people in a different way, in the space that they inhabit and not necessarily on their devices.

GEHRKE: And then what did Microsoft say to that? Was there like an end-of-quarter presentation?

LIU: Oh, yeah! There was a, there was a, you know, big old presentation. I can’t even remember which building we were at, but I think everybody was just like, wow, this is great. And that was it. [LAUGHTER]

GEHRKE: And that was it. It sounds like a really fascinating device.

LIU: Yeah, it was. And lots of people came up with all sorts of really cool things because everybody interpreted the, I’ll just say, the prompt differently, right.

GEHRKE: Right …

LIU: … And that was my interpretation of the prompt at the time.

GEHRKE: Well, super interesting.

LIU: Yeah.

GEHRKE: Coming back to, so OK, so you’ve done just a bunch of really amazing projects. You, sort of, it seems like you literally lived the notion of liberal education.

LIU: I did. I, like, even now I just love learning. I get my hands on all sorts of weird things. I picked up whittling as a random example.

GEHRKE: What is whittling? Do I even know what that is? [LAUGHS]

LIU: So whittling is basically carving shapes into wood. So … I’m also very accident prone, so there’s, like, lots of gloves I had to wear to protect my hands. But, you know, it was like, oh, I really just want to pick up whittling. And I literally did, you know. You can grab a stick and you can actually buy balsa wood that’s in a, in decent shape. But you can just start carving away at whatever … whatever you would like to form that piece of wood into, it can become that. So I made a cat, and then I made what I jokingly refer to as my fidget toy at home. It’s just a very smooth object. [LAUGHS]

GEHRKE: That you can hold and …

LIU: I just made it very round and smooth and you can just, kind of, like, rub it, and yeah, it’s …

GEHRKE: Super interesting.

LIU: … it’s … I pick up a lot of random things because it’s just fascinating to me. I learned a bunch of languages when I was in school. I learned Coptic when I was in school for no other reason than, hey, that sounds cool; you can read the Dead Sea Scrolls [LAUGHS] when you learn Coptic—OK!

GEHRKE: Wow. And so much, so important in today’s world, right, which is moving so fast, is a love for learning. And then especially directed in some areas.

LIU: Yeah.

GEHRKE: You know, that’s just really an awesome skill.

LIU: Yeah.

GEHRKE: And so you just graduated. You said you moved to Florida.

LIU: Oh, yes, yes. Yes. So, so about a month before this happened, right—it didn’t just spontaneously happen. A month before, I had a good friend from the architecture program who had said, hey, Wei, you know, I’m applying for this role in guest services at Disney. I was like, really? You can do that? And she’s like, yeah, yeah, yeah. So I was like, that sounds really cool. And I, you know, went to, like, the Disney careers site. I’m like one month or two months away from graduating. Still, like, not sure what I’m totally going to do because at that point, I’m like, I don’t think I want to be a designer because I don’t—the part that I love about it, the part that I have passion about, is not in the actual design of the object, but it’s about the understanding of why it needs to exist.

GEHRKE: The interconnection between the people and the design.

LIU: The people and the design, exactly. And so when I found, I found this, like, product development internship opportunity, and I was like, what does that even mean? That sounds cool. I get to …

GEHRKE: At Disney?

LIU: At Disney. And it was, like—and Disney’s tagline, the theme park merchandise’s tagline, was “creating tangible memories.” I was like, oh boy, this just checks all the boxes. So I applied, I interviewed, did a phone interview, and they hired me within 24 hours. They were like, we would like you to come. And I was like, I would absolutely love to move to Florida and work there. So, yeah, the day after I graduated from U-Dub, I drove all the way across the country from Seattle.

GEHRKE: You drove?

LIU: From Seattle with two cats.

GEHRKE: That must have been an interesting adventure by itself.

LIU: Oh, yes. With two cats in the car, let me tell you, it was fascinating. All the way to Florida, Orlando, Florida. And the day that I got there or, no, two days after I got there, I found out that I was going to be working in the toys area. So plush and dolls, which is, like, you can imagine just absolutely amazing. Making, like, stuffed toys that then—because my office was a mile down the road from Disney’s Animal Kingdom and therefore a couple miles away from Magic Kingdom or Hollywood Studios or EPCOT—I could actually go see, I’ll just say, the “fruits of my labor” instantly and not only that. See it bring joy to children.

GEHRKE: So what is the path? So you would design something, and how quickly would it then actually end up in the park? Or how did you, I mean, how did you start the job?

LIU: What did I do there? Yeah, yeah …

GEHRKE: Well, what’s the interface between the people and the design here?

LIU: Yeah … so, so, really, I didn’t actually do any design. There was an entire group called Disney Design Group that does all the designing there. And so what I did was I understood, what do we need to make and why? What memories are we—what tangible memories do we want to create for people? Why does it matter to them? In many ways, it’s, sort of, like, it’s still a business, right. You’re creating tangible memories to generate revenue and increase the bottom line for the company. But … so my role was to understand what trends were happening: what were the opportunities? What were guests doing in the parks? What types of things are guests looking for? What are we missing in our SKU lineup, or stock-keeping-unit lineup, and then in which merchandising areas do they need to happen? And so I, actually, as part of my internship, my manager said, hey, I let every intern every time they’re here come up with any idea they want, and you just have to see it from start to execution—in addition to all the other stuff that I worked on. I was like, sounds good. And I came up with this idea that I was like, you know, it would be cool … Uglydolls was really popular at the time. Designer toys were getting really popular from Kidrobot, which was kind of, like, there was this vinyl thing and you can—it was just decorative of all different art styles on the same canvas. And I was like, you know, what if we did that with Mickey, and then, you know, what if the story that we’re telling is, you know, just for the parks—Walt Disney World and Disneyland—that there were aliens or monsters coming to visit the park, but they wanted to blend in and fit in? Well, how would they do that? Well, they clearly see Mickey heads everywhere, and Mickey is very popular here clearly, and so they try to dress up like Mickey, but they don’t do it quite well. So they got the shape right, but everything else about them is a little bit different, and they all have their own unique personalities and …

GEHRKE: You can tell a story around them …

LIU: You can tell a story—see, it’s all about stories. And then it … I got buy-in from everybody there, like, all the way up to the VP. I had to get brand because I was messing with the brand icon. But, you know, it became an entire line called Mickey Monsters at Disney. I still have them all. There were two—then it went from plush; it became consumables, which are like edible things. It went into key chains. It went, it was super … it was … I probably went a little bit too hard, or I took the, I think, I took the assignment very seriously. [LAUGHS]

GEHRKE: Yep, yep. Well, it seemed to be a huge success, as well.

LIU: Yeah. It did really well in the time that it was there. We did a test, and I was really, really proud of it. But you know, my—what I did though is, you know, very concretely was I started with an idea. I, you know, convinced and aligned with lots of people in various disciplines that this is something that we should try and experiment on. You know, worked with the designers to really design what this could look like. You know, scoped out what types of fabrics because there’s all sorts of different textures out there. Working with, kind of, our sourcing team to understand, like, which vendors do we want to work with. And then typically, in the plush industry, manufacturing back in the day could happen—and in terms of supply chain, manufacturing, and then delivery of product—could take about six months.

GEHRKE: OK … 

LIU: And so when I was there, anything I worked on would, kind of, appear in six months, which is actually very cool. I mean, it’s not like software, where anything you work on is, you’re like boop, compile—oh look [there] it is. It depends on how fast your computer is. You know, it’s pretty instantaneous compared to six months to see the fruits of your labor. But it was a really, just such a great experience. And then seeing, you know, then going to the parks and seeing children with …

GEHRKE: Yeah, the stuff that you …

LIU: … the thing that I worked on, the thing that I had the idea on, and, like, them going like, Mom, I really want this.

GEHRKE: Right …

LIU: You know, we’re not really selling to the kids; we’re, kind of, selling to the parents.

GEHRKE: It’s a bit like this feeling that we can have here at Microsoft, right, if any of our ideas makes it into products …

LIU: Yup …

GEHRKE: … that are then used by 100 million people and hopefully bring them joy and connection.

LIU: Exactly. And that’s why, like, I just think Microsoft is great, because our portfolio is so broad, and so much of our work touches different parts of our lives. And I’ll even pick on, you know, like I have, you know, in my family, my daughter goes to school—clearly, obviously, she would go to school—but she used Flipgrid, now known as Flip, for a while. And I was like, hey, that’s cool. Like, she uses something that, you know, I don’t directly work on, but my company works on.

GEHRKE: Well, and you were involved with it through Watch For, right …

LIU: Yes, I was …

GEHRKE: … which did become the motivation for Flip.

LIU: Yep. Watch For, you know, helps to detect inappropriate content on Flip. And, you know, that’s super cool because now I’m like, oh, the work that I’m doing actually is directly impacting and helping people like my daughter and making a difference and, you know, keeping users safe from content that maybe we don’t want them to see. You know, other areas like Microsoft Word, I’m like, wow, this is a thing. Like, I’m at the company that makes the thing that I’ve used forever, and, you know, like, it’s just fascinating to see the types of things that we can touch here at Microsoft Research, for example. And how, you know, I, you know, Marie Kondo popularized the term “joy,” like, “sparking joy,” but …

GEHRKE: If you look at an item and if it doesn’t sparkle joy …

LIU: If it doesn’t spark joy, right …

GEHRKE: … then you know on which side it goes.

LIU: Exactly. But, but, you know, like, I’ve always felt like I want the things that I work on to create joy in people. And it was very obvious when you make toys that you see the joy on children’s faces with it. It’s a little bit different, but it’s so much more nuanced and rewarding when you also see, sort of, the products that, the types of things that we work on in research create joy. It’s, you know, it’s funny because I mentioned software is instantaneous in many ways, and then, you know, toys takes a little bit longer. But then, you know, in the types of research that we do, sometimes it takes a little bit longer than, a little bit longer [LAUGHS] …

GEHRKE: It takes years sometimes!

LIU: … than six months. Years to pay off. But, like, that return on that investment is so worth it. And, you know, I see that in, kind of, the work that lots of folks around MSR [Microsoft Research] do today. And knowing that even, sort of, the circles that I hang out in now do such crazy, cool, impactful things that help benefit the world. And, you know, it’s funny, like, never say never. I’m in tech and I love it, and I don’t have a STEM background. I didn’t get a STEM background. I didn’t get it, well, I don’t have a STEM degree. Like, I did not go—like, I can’t code my way out of a paper bag. But the fact that I can still be here and create impact and do meaningful work and, you know, work on things that create joy and positively impact society is, like, it speaks to me like stories speak to me.

GEHRKE: I mean, there’s so many elements that come together in what you’re saying. I mean, research is not a game of the person sitting in the lowly corner on her whiteboard, right? But it’s a team sport.

LIU: Yep.

GEHRKE: It requires many different people with many different skills, right? It requires the spark of ingenuity. It requires, you know, the deep scientific insight. It requires then the scaling and engineering. It requires the PM, right, to make actually the connection to the value, and the execution then requires the designer to actually create that joy with the user interface to seeing how it actually fits.

LIU: Exactly. And it’s fascinating that we sometimes talk about research being like a lonely journey. It can be, but it can also be such an empowering collaborative journey that you can build such incredible cool things when you bring people together—cross-disciplinary people together—to dream bigger and dream about new ideas and new ways of thinking. And, like, that’s why I also love talking to researchers here because they all have such unique perspectives and inner worlds and lives that are frankly so different from my own. And I think when they encounter me, they’re like, she’s very different from us, too.

GEHRKE: But I think these differences are our superpower, right, because …

LIU: Exactly. And that’s what brings us together.

GEHRKE: … they have to be bridged and that brings us together. Exactly. So how, I mean, if you think about Microsoft Research as over here. You’re here in Disney in Florida?

LIU: Yes, yes, yes. So …

GEHRKE: You had quite a few stops along the way.

LIU: I did have a lot of stops along the way.

GEHRKE: And very nonlinear also?

LIU: It was also very nonlinear. So Disney took me to the third, at the time, the third-largest toy company in the US, called JAKKS Pacific, where I worked on again, sort of, Disney-licensed and Mattel-licensed products, so “dress up and role play” toys is what we refer to them as. “Dress up” meaning, like, if you go to your local Target or Walmart or whatever, kind of, large store, they will have in their toy sections like dresses for Disney princesses, for example, or Disney fairies. Like, I worked on stuff like that, which is also very cool because, you know, usually around Halloween time here in the US is when I’m like, hey, I know that. And then that, kind of, took me to a video game accessory organization here in Woodinville.

GEHRKE: There’s the connection to tech starting to appear.

LIU: There’s a little bit connection of tech where I was like, I love video games! And I got to work on audio products there, as well, like headphones. And it was the first time I started working on things that, I’ll just say, had electrons running through them. So I had already worked on things that were, like, both soft lines—we refer to a soft line as bags and things that require, like, fabrics and textiles—and then I worked on hard lines, which were things that are more, things that are more physically rigid, like plastics. And so I was like, OK, well, I’ve worked on hard-lines-like stuff, and now I’m going to work on hard lines with electrons running through them. That’s kind of neat. And I learned all sorts of things about electricity. I was like, oh, this is weird and fascinating and circuits and … . And then I was like, well, this is cool, but … what else is there? And it took me to not a very well-known company in some circles, but a company called Fluke Corporation. Fluke is best known for its digital multimeters, and I worked there on their thermal imaging cameras. So it’s, for people who don’t know, it’s kind of like Predator vision. You can see what’s hot; you can see what’s not. It’s very cool. And Fluke spoke to me because their, you know, not only is their tagline “they keep your world up and running”; a lot of the things that Fluke does, especially when I heard stories from, like, electricians and technicians who use Fluke products, are like, this Fluke saved my life. I’m like, it did? What? And they’re like, you know, I was in a high-voltage situation, and I just wasn’t paying attention. I, you know, didn’t ground properly. And then there was an incident. But, you know, my multimeter survived, and more importantly, I survived. And you’re like, wow, like, that’s, that’s really cool. And so while I was at Fluke, they asked me if I wanted to work on a new IoT project. And I was like, I don’t even know what IoT is. “Internet of Things” … like, OK, well, you said “things” to me, and I like things. I like tangible things. Tell me more. And so that was, kind of, my first foray into things that had … of products with electrons on them with user interfaces and then also with software, like pure software, that were running on devices like your smartphones or your tablets or your computers. And so I started learning more about like, oh, what does software development look like? Oh, it’s a lot faster than hardware development. It’s kind of neat. And then that took me to SpaceX, of all places. It was super weird. Like, SpaceX was like, hey, do you want to come work in software here? I was like, but I’m not a rocket scientist. They’re like, you don’t need to be. I was like, huh, OK. And so I worked on Starlink before Starlink was a real thing. I worked on, kind of, the back-office systems for the ISP. I also worked on what we would refer to as our enterprise resource planning system that powers all of SpaceX. It’s called Warp Drive.

GEHRKE: That’s where you got all your software experience.

LIU: That’s where I learned all about software and working on complex systems, also monoliths and older systems, and how do you think about, you know, sometimes zero-fault tolerance systems and also, that also remain flexible for its users so they can move fast. And then from SpaceX, that took me to a startup called Likewise. It’s here in Bellevue. And then from the startup, I was like, I really like those people in Microsoft. I really want to work in research because they come up with all these cool ideas, and then they could do stuff with it. And I’m such an idea person, and maybe I’m pretty good at execution, but I love the idea side of things. And I discovered that over the course of my career, and that’s actually what brought me here to begin with.

GEHRKE: And that’s, sort of, your superpower that you bring now here. So if I think about a typical day, right, what do you do throughout, throughout your day? What is it, what is it to be a PM manager here at MSR?

LIU: So it’s funny because when I was just a PM and not a manager, I was more, kind of, figuring out, how do I make this product go? How do I make this product ship? How do I move things forward and empower organizations with the products that I—people and organizations on the planet to achieve more [with] what I’m working on? And now as a PM manager, I’m more empowering the people in my team to do that and thinking about uniquely like, who are they, what are their motivations, and then how do I help them grow, and then how do I help their products ship, and how do I help their teams cohere? And so really my day-to-day is so much less, like, being involved in the nitty-gritty details of any project at any point in time, but it’s really meeting with different people around Microsoft Research and just understanding, like, what’s going on and making sure that we’re executing on the impactful work that we want to move forward. You know, it’s boring to say it’s—it doesn’t sound very interesting. Like, mostly, it’s emails and meetings and talking, and, you know, talking to people one-on-one, occasionally writing documents and creating artifacts that matter. But more importantly, I would say it’s creating connections, helping uplift people, and making sure that they are moving and being empowered in the way that they feel that—to help them achieve more.

GEHRKE: That’s super interesting. Maybe in closing, do you have one piece of career advice for everybody, you know, anybody who’s listening? Because you have such an interesting nonlinear career, yet when you are at Disney you couldn’t probably … didn’t imagine that you would end up here at MSR, and you don’t know what, like, we had a little pre-discussion. You said you don’t know where you’re going to go next. So what’s your career advice for any listener?

LIU: I would say, you know, if you’re not sure, it’s OK to not be sure, and, you know, instead of asking yourself why, ask yourself why not. If you look at something and you’re like, hey, that job looks really cool, but I am so unqualified to do it for whatever reason you want to tell yourself, ask yourself why not. Even if it’s, you know, you’re going from toys to something in STEM, or, you know, I’m not a rocket scientist, but somehow, I can create value at SpaceX? Like, if you want to do it, ask yourself why not and try and see what happens. Because if you stop yourself at the start, before you even start trying, then you’re never going to find out what happens next.

[MUSIC]

GEHRKE: It’s just such an amazing note to end on. So thank you very much for the great conversation, Wei.

LIU: Yeah. Thanks, Johannes.

GEHRKE: To learn more about Wei or to see photos of her work and of her childhood in Silicon Valley, visit aka.ms/ResearcherStories (opens in new tab).

[MUSIC FADES]

GEHRKE: We’ve talked a little bit beforehand, but you’ve had this amazing career in tech. How did you actually … tell us a bit about how you grew up, and how did you end up in tech?

NICOLE FORSGREN: Yeah, it’s, you know, it’s, kind of, this ridiculous story. I grew up in a little farm town and ended up going to college and thought I would just be there a year or two because …

GEHRKE: Little farm town in?

FORSGREN: In Idaho.

GEHRKE: In Idaho?

FORSGREN: Yeah. So across the street from a potato field. My grandpa was a potato farmer. And where I’m from, girls—a lot of girls—go to college but not usually for very long. You kind of go to school to get married. And so I was majoring in psych and family science.

GEHRKE: So in high school, you didn’t know anything about or you weren’t excited about tech yet or anything?

FORSGREN: I don’t think I knew much about it, right.

GEHRKE: OK.

FORSGREN: So, I mean, we had a computer at home. My dad was a civil engineer. But I had only ever used it to write papers. I knew WordPerfect because back then WordPerfect was, kind of, the thing.

GEHRKE: Yeah, right. So it was a PC?

FORSGREN: It was a PC. It had, you know, reveal codes, which …

GEHRKE: It was DOS, just not Windows yet, right, or was it Windows?

FORSGREN: I think it was DOS back then. Yeah. And that was …

GEHRKE: And so connector interfaces and everything?

FORSGREN: Yeah,we had all the interfaces. And we had a typing class in high school, but that, that was it. And then I went on to college, and a couple of months into my freshman year, my dad, who I was really close with … so I grew up as a tomboy. I was always mountain biking up in the mountains. I was always just, like, dirty and, kind of, just playing around. I was very much a tomboy. Two days after Thanksgiving, he was in a snowmobile accident and in a coma, and suddenly, I sat back and I realized that, you know, all the things that I thought I was going to be able to rely on, like, you know, if I ever needed to fall back on my family or my dad or pay for things, for some reason, like, that was what stood out to me for some reason. You know, I was in college on a volleyball scholarship, and, like, I was paying for things and it was OK. But for some reason, that was what stood out at me—because I was the oldest of four children. The youngest was 11 years younger than me. So I was like, if anything happens, I think I just felt that responsibility, was that I was going to need to have money. And one of the girls on the practice team, I think in just a totally random comment, said that she could make money in her degree.

GEHRKE: Because you hadn’t chosen that as the …

FORSGREN: Yeah, it just, sort of, came … ooh, because when we went to college, it was, I was going to get married and so then I wouldn’t need money, right. Because I didn’t choose a degree for that. I want to say it was within a couple of days of my dad’s accident—so he was still in a coma; he was still hurt—and it was a very side comment she had made. And I remember hearing it, kind of, in the back, and I, kind of, perked up, and I was like, “What do you mean you can make money with your degree?” She said, “Oh, yeah, when I graduate”—with a two-year degree—”I can make $40,000 a year.”

GEHRKE: So she wasn’t at … was she at your university, or she was at a different …?

FORSGREN: So I went to a JC first because we don’t really go to college. And I was like, oh, $40,000. And I remember thinking, well, if I stay in psych and family science, to be able to make that kind of money, I would need to be a high school teacher, which requires a master’s degree. And so I talked to her just a little bit, and I said, “Well, what’s your degree?” She said computer information systems. I said, “Well, do you like it?” She said, “Yeah, it’s really cool. It’s computers.” I said, “Oh, and what can you make again?” She said about $40,000. I was like, “Cool.” And I just remember looking up that degree and going into finding, you know, I just found the counselors in that degree and telling them I wanted to change my major.

GEHRKE: Oh, wow, that’s a huge decision. I mean, first of all, did your father recover?

FORSGREN: So he ended up coming out of his coma within a couple of months, but he was brain damaged for the next few years. And he passed just after I graduated from college.

GEHRKE: I’m sorry to hear that. But then … so you went ahead and changed your major and went into …?

FORSGREN: I changed my major—without ever having a computer class …

GEHRKE: Right.

FORSGREN: … to CIS, which is, now I know, computers and business. That summer, I applied for an internship and got it. [LAUGHS]

GEHRKE: After your freshman year?

FORSGREN: After my freshman year.

GEHRKE: Wow. What kind of internship? Where was it?

FORSGREN: It was in mainframes for computer hospital systems. And so I showed up, and they thought I was full time, which I was not. I just interviewed well apparently. And so they threw me the manual and the documentation.

GEHRKE: What kind of mainframe was it? Do you remember what kind of machine it was?

FORSGREN: Yeah, it was AS/400s, and it was programing RPG and CLI, and it was for …

GEHRKE: Beautiful languages. [LAUGHS]

FORSGREN: They were. I loved it. And it was Siemens, um—SMS MedSeries 4 was the company, later acquired by Siemens. And it was a wonderful team, and I spent the first week reading the documentation and proofreading the documentation [LAUGHS], and then I just, kind of, dove in.

GEHRKE: So proofreading? You mean you discovered a bunch of bugs?

FORSGREN: There were a lot of typos, a couple of bugs, right. Because I just, kind of, had to figure it out as I went.

GEHRKE: Wow. So it’s not like somebody did preparing with you at the beginning? You were just handed the manual, and then you went with it?

FORSGREN: I was just handed the manual, and then I went with it. I had to …

GEHRKE: Wow, it’s an amazing self-starter.

FORSGREN: … figure out one or two things, and then I just dove in, and then I went back that next year and had my first RPG class. [LAUGHS]

GEHRKE: Wow. They were teaching RPG in college?

FORSGREN: Yeah. So this was … my freshman year was ’97-’98, and then my sophomore year was ’98-’99, so it was a JC, and they were, kind of, gearing up one or two classes in anticipation of the Y2K bug.

GEHRKE: Right.

FORSGREN: And so mainframes …

GEHRKE: Oh, I remember that. Yes …

FORSGREN: …and financial and hospital systems …

GEHRKE: Everybody thought the world was ending or at least some people were thinking the world would end.

FORSGREN: They were so worried about it. And so they had brought back one or two mainframe languages to help with the Y2K crisis. And so that summer, I did hospital systems. The next summer, I did, um, it was a company called Assist Cayenta, and it was, like, it was financial systems and ordering systems. That’s how I started my career, in, like, it was a version of mainframes.

GEHRKE: And then did you get that high-paying job afterwards?

FORSGREN: Yeah. So I had a full-time offer, but by then, I decided to go for my four-year degree. Transferred to Utah State, which was within a few hours from home. I still had to, kind of, stay closer to home because the family, and I wanted to, you know, make sure I was close for my dad. But then I did a six-month internship at IBM. I was a software engineer, and then they hired me full time when, again, I was a software engineer, so …

GEHRKE: But you basically finished the two-year college, then went to Utah State. Finished a four-year degree.

FORSGREN: Finished a four-year degree. Again, there I was … they called it business information systems—now it’s management information systems—at Utah State. It was a very technical degree, so I was doing network engineering. I was doing databases. I was doing, you know, C++. All my programing classes were all in the computer science department. When I got hired at IBM, they hired me as a software engineer. I was working on large-scale enterprise storage systems.

GEHRKE: Which language did you program in then?

FORSGREN: There, I was doing some C++. I was doing a little bit of C. I was doing some Java, and I was doing a little bit of their firmware.

GEHRKE: Wow.

FORSGREN: And then I eventually ended up managing some of their systems, so I did some Bash, and then eventually, I even got a hardware patent, so I—it was kind of everything! 

GEHRKE: Wow. So what is the patent about?

FORSGREN: It’s a way to, kind of, further obfuscate for cold boot attacks. This really, kind of, fantastic article came out showing that if you take compressed air and turn the can upside down, you can freeze some of the bits in chips.

GEHRKE: Right.

FORSGREN: And then if you rip the chip out, you can read what’s on the chip.

GEHRKE: Oh, wow.

FORSGREN: Because it freezes all the elect—

GEHRKE: Super interesting.

FORSGREN: … all the electrons on the chip. And we were like, well, this is ridiculous because it’s only frozen for 2 to 5 seconds. But then if you rip it out and you drop it in, like, liquid nitrogen, then you can read it for 2 to 5 minutes.

GEHRKE: Right.

FORSGREN: Like, well, again, this is ridiculous. But if you’ve entered in a password, then it’s stored in plaintext. Well, it’s not ridiculous if you get your way into a lab—and at the time, I was working in a large lab—because you’ve entered in the password for one computer, one of the servers, one of the stored servers, and you’ve destroyed the, you know, you’ve broken the disk for that one server, but the password is going to be the same for every single server in the lab.

GEHRKE: OK.

FORSGREN: And so we realized that this is a serious, you know, problem that’s a threat vector for a lab. And it’s pretty easy to get into labs, right. Like, you can just, you can follow anybody in. And so we wrote a patent that kind of further obfuscates and it hides where passwords are stored through malloc() calls.

GEHRKE: I see, so … I see. So the location of the password was then somewhat obfuscated and not so easy to find.

FORSGREN: Well, so the location of passwords but also additional plaintext strings and other strings are obfuscated through, kind of, throughout the pieces of, like, different areas of chips.

GEHRKE: This was motivated by the hardware but then implemented in software?

FORSGREN: Yeah.

GEHRKE: OK, wow, that’s super interesting.

FORSGREN: So part of hardware and part of software.

GEHRKE: I mean, just imagine this career, right, in … I guess in high school you’re playing volleyball?

FORSGREN: In high school and college, I was playing volleyball. Ended up getting hurt, so I didn’t play volleyball longer.

GEHRKE: OK. But then switched to, you know …

FORSGREN: Switched to tech.

GEHRKE: MIS, and …

FORSGREN: Yeah, switched to MIS.

GEHRKE: Patent.

FORSGREN: Patent—and shoutout to, you know, my coauthors on the patent, Ben Donie, Andreas Koster. They were great because we all just, kind of, ended up brainstorming one day. And it was this, kind of, windy path, but it was, I think, it was interesting because at the time, you know, I originally went into tech because I thought I would really need money but ended up falling in love with it, right. I’ve had a lot of fun along the way.

GEHRKE: So what did you fall in love with with tech? What really gets you going on tech?

FORSGREN: I love the fact that there are hard, interesting problems that you can solve lots of different ways. And if I can’t solve it initially and if I can’t solve it the first time, I can just kind of spin it around or pivot it in different ways and then just solve it again.

GEHRKE: So it’s this notion that you have this not only hard problems—because in math, you have lots of hard problems, as well—but here there’s the experimentation with it and the trial and error or …?

FORSGREN: The experimentation and the fact that it’s applied and the fact that I can build something and watch it work. Math I liked. And math, up to a point, I was pretty good at math, and I could kind of see how the equations were supposed to work. Computers and programs helped because I could really see how it was working.

GEHRKE: Yeah, I think that, I mean, I can so much relate to this because that’s how I fell in love with computing, because you have this machine here and it basically can do everything, right. And I mean, we will get later to AI. What we see with the current AI systems, right, you can see that, you know, it can be nearly intelligent or it is intelligent, right. And so it’s just amazing to have that kind of machine below you or with you to help you and to be able to train it and program it. And so, you were at IBM …

FORSGREN: So I was at IBM.

GEHRKE: … and now you’re at MSR (Microsoft Research).

FORSGREN: And now I’m at MSR.

GEHRKE: So what’s the bridge in between?

FORSGREN: Oh, so also, kind of, a windy path, but it’s interesting because as I look back, I guess it makes sense to me. So then I had an opportunity to go get a PhD, and I actually started doing, kind of, large data, like, NL, you know, natural language. You know, how do we want to think about, you know, analyzing sentiment analysis, analyzing, you know, those types of, you know, big questions like when are people lying, when are people not lying, machine learning problems. But I was also working at IBM at the time. Because I, like, continued to like, you know, working on systems and working on large problems. So I was doing both at the same time, and I ended up doing usability study … completely randomly. And …

GEHRKE: What was the study on? What did you study?

FORSGREN: So we did a usability study for sysadmins. And it was interesting because at the time, IBM was trying to build, like, a GUI for large-scale sysadmins. And so Todd Eischeid and I did the usability study. We wrote up the findings for IBM; we shared them. And, you know, we found that in some cases, they would use this frontend user interface and it was fine. And in some cases, they were like, “I don’t know,” right, “like I could click this button, but it really makes me nervous.” And we’re like, “It’s OK. It’s a sandbox. This is fake. You can click the button. I’m curious to see what the next step is.” And they were like, “It’s just so risky.” And it struck me as being, sort of, interesting because, like, there were just cases where risk and complexity really interfered with our ability to trust a system or to trust a GUI. And so we wrote this up and we shared it, and IBM was like, eh, you know, we used user-centered guidelines; we used, you know, user-centered design guidelines. We were like, but the same guidelines can’t work for complex systems and complex distributed systems that can work for laptops. And …

GEHRKE: Because in one way, I affect this one machine here, but in the other way, I affect this row of machines, and I know how many this actually is.

FORSGREN: Right. And they really wanted to see the command line interface and the backend data, and they really wanted to verify. And so you really had this difference between not just risk and complexity but also expertise. You know, there were some cases where you could hide complexity and other cases where it just wasn’t appropriate, right. And, you know, simultaneously, I’m working on these really, really large projects with IBM, and people are just, kind of, burning out. And I thought, you know, there has to be something to this, right. And there were kind of rumors as I was going to tech conferences still of, you know, this new-fangled way to make software and, kind of, reduce burden. And so I just pivoted. I changed my research project to start studying what now we know of as DevOps.

GEHRKE: Maybe we will get to this in a little bit, but I’m so impressed by, you know, you go to college, do A. Something happens. You do B, and don’t look back. Same thing here, right? You’re a successful developer at IBM. You have this event which says, wow, I should study this intensively, and you go and get a PhD, right. I mean, it’s just super impressive. How do you do that?

FORSGREN: It’s, I will say, it’s not always super straightforward, [LAUGHTER] but there are times when I, kind of, sit and I’m like, I’m getting one or two signals. I really want to take a half-step back and say, here’s an opportunity. Should I take the opportunity, or should I not? There have been one or two times where I’m not sure, you know, but there have been times where I’m like, I need to jump, and I can reevaluate in six months or a year, but I’m going to take this now. And I will say about 90 percent of the time I have been absolutely on.

GEHRKE: I find this so fascinating because we all are faced with different opportunities and chances in our lives. How do you evaluate this? Do you have, like, a checklist? Do you go back and ruminate and meditate on the mountain? Or what’s your method?

FORSGREN: I actually have a spreadsheet. [LAUGHS]

GEHRKE: OK.

FORSGREN: But I don’t always follow it. So what I like to do is identify, like, what are my criteria, which is, you know, what things are important to me.

GEHRKE: Yup.

FORSGREN: You know, some of my big moves—what’s important to me in the city? And then for each of those criteria … or when I am considering a new job—what things are important, and then how important are they?

GEHRKE: And you put, like, a risk score behind it or, like, a score?

FORSGREN: Either a risk score or an importance score. And then I’ll just, kind of, multiply it out. And, now, then I will go back and I’ll like …

GEHRKE: That sounds so amazingly systematic.

FORSGREN: … and then I’ll nudge all the numbers, and then sometimes, I’ll still change my mind because there’s something about your gut that says, I don’t know. But by identifying all those things first, it helps me think through it. Now, the spreadsheet, like, I’ve shared with folks, and it’s really interesting because just the exercise of saying what things are important to me for this decision and how important are they sometimes … not even doing like the math, right. Because it’s not real math. Let’s be real. Or it’s, like, very simplified math. Just identifying those things, sometimes just that exercise, people are like, “Oh, I know what my decision is now.”

GEHRKE: I think often just even thinking about this with that clarity, right, creates the resulting clarity then and think about all the factors.

FORSGREN: Right. And there have been times where I’ve completely changed what I thought I would do and, like, I can give an example. After I left academia, I was at a startup, and then following that, I started, like, my cute little baby startup company. I thought for sure I was going to go to a large consulting firm. I mean, I had them ranked. I thought I knew my choices, and I was like, no, let’s, let me think. Let me identify what order I should go in. And, like, it didn’t matter how I, kind of, rearranged all the numbers, starting my own company was at the top.

GEHRKE: Well, let’s go to that in your career.

FORSGREN: Yeah, so let’s, let’s catch up.

GEHRKE: Exactly, so you decide to go do your PhD?

FORSGREN: So I finished the PhD. I stay in academia for a while because I really like the research that I’m doing. It’s really interesting. I think I’m, kind of, on to something, and academia is a good place to do this, right. So I was a professor at Pepperdine for a few years. I go to Utah State for a few years. Again, like, what’s this opportunity? Pepperdine was a lovely place. The faculty were incredible. Malibu is gorgeous. Utah State, my alma mater, comes along, and they’re like, we would like to hire you, and we’ll create an opportunity for a new position. So I took that pivot. And now it’s a joint appointment to create an analytics program in the MIS department. I was also an accounting professor, because I have a master’s in accounting, so it was, kind of, this, like, perfect situation. I was there for, you know, two, three years, and I’m doing really, really well. A really strong path to tenure. Letter from the provost saying that things are looking really well. And I’m like, this is good, but it’s not great.

GEHRKE: You had a spreadsheet that said that basically or [LAUGHS] …

FORSGREN: This was in my gut.

GEHRKE: This is just “I had a feeling.”

FORSGREN: In my gut, I’m like, I’m doing really well. My research is going really well. We just hit the Wall Street Journal because we find early signal that DevOps—like now it’s being called DevOps—shows organizational impact. And so a few folks in industry were coming to me and they said, you know, “This is super relevant. You’re really changing how we’re doing things.” This was still, kind of, earlyish in this research program, and they said, “What if we create an opportunity for you where you could spend half of your time doing research and half of your time helping our company improve our engineering practices,” and I was like, I think I might do this. You know, at that point, I just decided to go for it, and it was a little company here in Seattle called Chef Software.

GEHRKE: It was actually here in Seattle?

FORSGREN: I was here in Seattle.

GEHRKE: Chef Software?

FORSGREN: Yeah,they did configuration management software. And also, that was a really interesting tie because—or, kind of, like pull, tug and pull because I was doing this research with their competitor called Puppet, and I went to Chef, and so I’m, kind of, like, managing these relationships really well or as well as I could. But that was also one of my first exposures to managing conflict at work and in professional relationships because this report, it was a research report, but it was done, kind of, through industry. So the main sponsor was Puppet, but I was working at Chef. And so how do I manage that? So I was at Chef for a year and a half, and then at the end of that year and a half, we, kind of, looked at each other and we were like, I think we’ve reached the end of this road because I had done about as much as I could do for this little startup. They had about 200, 250 people. They really didn’t need a researcher. They were doing me a solid. And then at the same time, we had, kind of, spun out a separate entity called DORA—DevOps Research and Assessment—and we said, so what should happen here? We had been doing research, the State of DevOps Report, but then so many companies were coming to us and they were saying, what if I want to have our own customized assessment? How could we do this? And, you know, I looked at a couple of my co-founders, you know, Jez Humble and Gene Kim, and I said, I know how to do this. I know the algorithm I would use. I know how I would build this out. I think we could do it, a SaaS company. I have a very low risk tolerance. I have never wanted to start a company. And I think, you know, to your prior question, like, how have I thought about doing this before? I actually looked at them, and I said, you know, who wants to be CEO? And they said, we think you should. And I said, eh, I’ll give you one year, and then I want you to tell me if you want me to continue doing this. And so we started this company. I mean, our first prototype, I drew pen and paper in the back of a notebook, and I showed it to Capital One, and we said, would you buy it if it looked like this? And then we just, kind of, iteratively built out pieces and pieces. And I was like copy-pasting pieces of it into reports as we went. And then at our offsite after one year, we read our results. I shared ARR and sales projections, and I said, OK, well, you have a first right of refusal, and do you want to renew? And they looked at me like what? And I’m like, no, it’s been 12 months. Do you want me to return again? And we just, kind of, decided if we wanted to. After that, things were going really well, but we were also growing and scaling really, really rapidly. Too fast for us to keep up as a bootstrapped company. And so, you know, I needed to figure out how to gain and acquire infrastructure. And so that would either have been external funding and hiring really rapidly or getting acquired, and so we … I approached a handful of companies, and Google acquired us.

GEHRKE: Wow. It’s an amazing story and must have also been a time of craziness and also fear of uncertainty. How, you know, what were some of your emotions during that time?

FORSGREN: It was. It was a lot. It was really interesting because it was, kind of, balancing also a few things, right. Because I had … I think some of it is also balancing identity, right. Am I a researcher, and how do I think about maintaining that identity and that credibility that I care so much about when the research and “publication path”—you know, kind of, in finger quotes—that I care so much about has shifted, right. Because now I’m doing a lot of applied research, and how do I think about that? Some of it was, am I an entrepreneur, and what does that look like, and what does, what does that credibility look like? How do I put out a product fast enough? How do I manage and maintain this company? How do I manage and maintain relationships with my employees, with my partners, you know, with this partner ecosystem that we’ve developed? And then when we get acquired by Google, what does that look like? How do you manage that growth path?

GEHRKE: And these are all very, very different skills than you had as a software developer or even as a professor.

FORSGREN: Especially as a professor. Right. And it was also a really, really wonderful time, I think, to grow and learn and iterate. And I’m really grateful for the partners that I had along the way, right. There were times that were bumpy, right. But I appreciate that we were really honest with each other, right. There were times when we disagreed, and there were times when I also said, like, I know this is the right thing. Please, you have to let me do this. And there were times when, you know, they had their expertise.

GEHRKE: And then, you know, you were at the company. The company gets sold to Google.

FORSGREN: Yep, company gets sold to Google.

GEHRKE: Now you’re at MSR.

FORSGREN: And then I was at Google for a little bit. And then I had this amazing opportunity to join GitHub. And I was really excited about that because that framing in that invitation, we were talking about, you know, what could it look like? You know, what would a perfect world be where I could return to something that looked like research and strategy? Because I realized there were pieces about research that I really, really loved. And there were also pieces about, I don’t want to say products—it’s not quite product—but it’s about strategy. What is it that I love doing in terms of, kind of, execution and identifying holes, and how does that feed back into the research projects that I want to do? And so when Microsoft first approached me, they said, you know, what would your ideal job look like? And I, kind of, laid that out, and they said, you know, well, let’s think this through. And so we started with GitHub because, you know, if you want to study developers, that’s an amazing place to start. So we did a couple of iterations with the Octoverse report that were really rewarding and then we said, you know, another good place, you know, another wonderful opportunity would be to think about developers and a Developer Experience Lab. And if there’s a research lab, where does that live? And we talked about it some more. You and I did, too.

GEHRKE: That’s how we started talking … exactly.

FORSGREN: Yeah, and we thought, you know, MSR really is the perfect place for that to live.

GEHRKE: And you’ve been at MSR now for a few years, and now we’re going through yet another change. I mean, you’ve been through many of these changes before. We talk about this current change. It seems like, again, coming back to this, you’ve been amazing of, like, when the environment shifts, finding out where to go. You know, you have your spreadsheets, you know, as one mechanism. Now we’re at another sea change with AI, right. And AI is clearly changing the way we write code, which is, sort of, the innermost loop right now with GitHub, with Copilot, but it’s probably going to make it much more into the inner and outer loop and the whole way we write code and the way we develop with low code. So, I mean, first of all, how do you think about that sea change, and how do you deal with your research group and, you know, yourself as an identity, again, as the world around you is having this massive change towards AI?

FORSGREN: You know, sometimes, I just laugh that the world is a circle, right. I’m really excited because, you know, we’ve come back around to getting to rethink what it means to do what we love, right. So I’m personally getting an opportunity to come back to be a developer again, right. What does it mean to dive back in and learn brand-new things again? Because AI is new for almost all of us. Even people who have been studying AI for 10 years are saying, like, so much of this is new, so much of this is something that we couldn’t have predicted. I’m getting to, you know, dive back in and play with new tools and new technologies, and that I really love. I also love that you mentioned that, you know, there’s the inner loop and there’s the outer loop. And so in terms of my team, I’m really, really excited about the team that I get to work with because …

GEHRKE: Do you want to explain maybe a little bit, just for the audience, inner and outer loop?

FORSGREN: Yeah, so inner and outer loop. So inner loop is, you know, the coding that we do, kind of, locally, so it can be writing the actual code, you know, local build, if you do local build. It can be debugging; it can be everything that’s like just right there on your screen as you’re writing your code. Now outer loop is everything that you do to get that code running in production, right. So it can be additional tests. It can be integration build. It can be, you know, everything out through release and deployment until we’re operating that code and then, like, continuing to operate that on our systems at scale.

GEHRKE: I mean, the way I saw this was so impressive when I joined Microsoft, of course, right. What it means to actually … that first of all, software development is a team sport, right.

FORSGREN: Yes.

GEHRKE: And also it’s not a team of like 11 people like soccer, where I grew up with, but it’s a team sport of, you know, potentially thousands of people, right, and that there’s an, actually, there’s a lot engineering systems around it. And it’s called software engineering for a good reason.

FORSGREN: Yeah.

GEHRKE: Because there’s systems around it that help us to scale to that many people contributing to a single outcome. And so the inner loop is, basically, my notion is that when I do my own little exercises with the ball and then the outer loop is when I actually, you know, do strategy with the whole team and see that I integrate well. Is that a reasonable analogy?

FORSGREN: Exactly. And so sometimes the joke is, you know, well it worked on my machine, right. That’s kind of inner loop. Yeah. And then outer loop is all of the orchestration, right. All of the architecture. Everything else that we need to make things, especially really large things and complex distributed systems, work at scale. Yeah.

GEHRKE: And so if you think about … how do you think AI would influence both the inner and outer loop? I mean, we see what’s coming out in terms of, you know, GitHub Copilot and even more capabilities. I mean, in the “[Sparks of Artificial General Intelligence]” paper, we describe that GPT-4 can actually write an application with close to a thousand lines of code, right. So how do you think AI’s actually going to influence developer productivity and software engineering as a whole?

FORSGREN: I think there are so many exciting ways to think through this, right. I love your point that there’s inner and there’s outer loop, right. So, yes, we absolutely have opportunities to think about how it influences the way we write code, but I think it also has so many opportunities downstream, right. How can it, how could we use it to improve our code bases, right? Can it identify technical debt and clean up some of our technical debt for us? Can it help us think about downstream incident management? Can it help us manage our servers and our systems? Can we use it to take a look at our code for us? Can we ask it, can you please help me improve the security posture of my code? Can you help me improve the performance of my code? Can you help me improve anything else in my code, right? Even that explicitly: can you say, is there anything I haven’t thought of in my code, and what should I do? We can ask it, you know, to proactively watch and monitor our systems’ performance for us and then proactively manage that for us, you know. As we look, you know, even further into the future, there may be opportunities for brand-new abstraction layers. What happens if we let LLMs, or invite LLMs, to execute for us and then reason about that code for us so that all we need to do is guide and direct it?

GEHRKE: We’ve been talking a little bit about code, but maybe in the future, code would be, sort of, this low-level abstraction like what we have right now with assembly. There are very few people who still optimize, let’s say, locks and database systems with the assembly code, but most people write at a much higher level of abstraction. So what do you see as this next level of abstractions that are coming?

FORSGREN: You know, I think …

GEHRKE: Is it language, basically, language interaction?

FORSGREN: I do think some will be language. You know, I really like that idea for at least a few things, for a couple of things I just mentioned, right. Like asking it to do things like please check for security; please improve the performance. Can you help me generate workloads? Can you help me run these type of canary tests, right?

GEHRKE: Like semantic linters with a very bigger … with much bigger capabilities.

FORSGREN: Exactly. I also think there’s an opportunity for graphical interfaces, and by that, I mean what if we create a diagram or a UML and then ask it to implement that in the best way possible. Or reverse it, right? When I think about when I was working in code bases, the best way I could get a feel of the code base was to code it, and then I could create this mental model. If we’re doing less coding, how can we create that mental model? I think there could be wonderful opportunities to ask or invite these LLMs to diagram some of these code bases for us.

GEHRKE: So interesting …

FORSGREN: You know, don’t just ask it to create documentation or explain it to us, because language can be somewhat limited, but we know that diagrams and pictures can be incredibly powerful.

GEHRKE: I see. Create the actual architecture diagram for us.

FORSGREN: Create an architecture diagram—and create it in two or three different ways to help us understand how different components interact, which can also help us understand where there may be redundancies in code. There’s a huge amount of technical debt out there, right. So I think that, kind of, opens up some really interesting ideas for what could be there. You know, there are also some wonderful horizons that we’re already approaching in terms of testing and exploratory testing and what that can mean for really improving the way our code works.

GEHRKE: It’s super exciting. I mean, I would love to be able to talk more about that with you. But let me ask one last question. I mean, you’ve had this absolutely stunning career. I mean, if you think about where you started out, you know, then developer, you know, professor, startup founder, you know, working in a big company, being in a restructuring of the company, for someone who’s starting out, what’s the career advice that you give for anybody who’s right now starting and going into tech, people who are at university or just graduating?

FORSGREN: I think there would be two, and I think they’re related. One would be assume that something can be figured out and that it’s not hard. I think that would probably be one of my best tricks. I didn’t find out until the end of college that girls were bad at math, which I am not, or that computers were hard. It really helped that for most of my life, my dad just, sort of, helped me rethink through things or repivot or retry lots of things. And so if it’s hard, that’s OK. It just might mean that you should just spin it on its head and try to take another look. So that would be the first one. And I think the second one is consider new opportunities and go ahead and take them. And if after six or nine or 12 months, it’s not the right opportunity, go ahead and change your mind. I would not be where I am today if I hadn’t taken one or two really incredible opportunities that seemed a little bananas at the time, and it just worked out.

GEHRKE: That’s amazing advice, especially also in something that tells us to A/B test even our own life. The only problem is that we have only a limited number of tests that we can do, but I mean, it clearly is an amazing story. Thank you so much for the conversation.

FORSGREN: Yeah, thank you.

GEHRKE: Thank you.

To learn more about Nicole’s work or to see photos of Nicole as a child in Idaho, visit aka.ms/ResearcherStories (opens in new tab).

IVAN TASHEV: So I’m born in a city called Yambol in Bulgaria, my origin country. The city [was] created 2,000 years B.C. and now sits on the two shores of the river called Tundzha. It always has been an important transportation and agricultural center in the entire region, and I grew up there in a family of two lecturers. My parents were teaching history. And they loved to travel. So everywhere I go, I had two excellent tourist guides with me: “This in this place happened at this and this and this in this year.”

GEHRKE: Were there quizzes afterwards? [LAUGHTER]

TASHEV: But it happened that I was more fond to engineering, technology, math, all of the devices. It just … mechanical things just fascinated me. When I read in a book about the parachutes, I decided that I will have to try this and jump into it from the second floor of a building with an umbrella to see how much it will slow me down. It didn’t.

GEHRKE: And how … did you get hurt?

TASHEV: Oh, I ended with a twisted ankle for quite a while.

GEHRKE: Oh, OK, good. Nothing more … worse. [LAUGHTER] So you were always hands on, that’s what you’re telling me, right? Always the experimenter?

TASHEV: Yep. So I was doing a lot of this stuff, but also I was very strong in math. It happened that I had good teachers in math, and going to those competitions of mathematical Olympiads was something I started since fifth grade. Pretty much every year, they were well organized on school, city, regional level, and I remember how in my sixth grade, I won the first place of a regional Olympiad, and the prize was an 8mm movie camera. That, I would say, changed my life. This is my hobby since then. I have been holding this, a movie camera of several generations, everywhere I go and travel. In Moscow, in Kyiv, in Venice. Everywhere my parents were traveling, I was shooting 8mm films, and I continue this till today. Today, I have much better equipment but also very powerful computers to do the processing. I produce three to five Blu-ray Discs pretty much every year. Performances of the choir or the dancing groups in the Bulgarian Cultural and Heritage Center of Seattle mostly.

GEHRKE: Wow, that’s fascinating. And was that hobby somehow connected to your, you know, entry into, you know, science and then actually doing a PhD and then actually going and, you know, going into audio, audio processing?

TASHEV: The mathematical high school I attended in my … in the city where I’m born was one of the fifth … one of the five strongest in the country, which means first, math every day, two days, twice; physics every day. Around ninth grade, at the end, we finished the entire high school curriculum and started to study differentials and integrals, something which is more towards the university math courses. But this means that I had no problems entering any of the, of the universities with mathematical exams. I didn’t even have to do that because I qualified in one year, my 11th grade, to become member of the Bulgarian national teams in … for the International Math Olympia and for International Physics Olympia. And they actually coincided, so I had to choose one, and I chose physics. And since then, I’m actually saying that math is the language of physics; physics is the language of engineering. And that kind of showed the tendency … so literally, I was 11th grade and I could literally point and choose any of the universities, and I decided to go and study electronic engineering in the Technical University of Sofia.

GEHRKE: And then how did you end up in the US?

TASHEV: So that’s another interesting story. I defended my … graduated from the university, defended my PhD thesis. It was something amazing.

GEHRKE: What was it on, actually?

TASHEV: It was a control system for a telescope. But not just for observation of celestial objects but for tracking and ranging the distance to a satellite. It’s literally one measurement. You shoot with the laser; it goes to the satellite, which is 60 centimeters in diameter; it returns back; and you measure the time with accuracy of 100 picoseconds. And this was part of studying how the Earth rotates, how the satellites move. The data … there were around 44 stations like this in the entire Earth, and the data were public and used by NASA for finalizing the models for the satellites, which later all became GPS; used by Russians to finalize the models for their GLONASS system; used by people who studied the precession and the rotation of the Earth. A lot of interesting PhD theses came from the data from the results of this device, including tides. For example, I found that Balkan Peninsula moves up and down 2 meters every day because of the tides. So the Earth is liquid inside, and there are tides under us in the same way as with the oceans.

GEHRKE: Oh, wow, super interesting. I actually just wanted to come back … so just to get the right kind of comparison for the, for the unit, and so picoseconds, right? Because I know what a nanosecond is because …

TASHEV: Nanoseconds is 1-0 minus ninth; picoseconds is 1-0 minus 12th.

GEHRKE: OK. Good, good. Just to put that in perspective.

TASHEV: Thank you, Johannes. To, to be exact. So this was the, the accuracy. The light goes 30 centimeters for that time. For one nanosecond. And we needed to go way shorter than that. But why this project was so fascinating for me … can you imagine this is 1988—people having Apple II on compatible computers playing with the joystick a very famous game when you have the crosshair in the space and you shoot with laser the satellites.

GEHRKE: [LAUGHS] Absolutely.

TASHEV: And I was sitting behind the ocular and moving a joystick and shooting at real satellites. [LAUGHS]

GEHRKE: Not with the goal to destroy them, of course.

TASHEV: No. The energy of the laser was one joule. You can put your hand in front. But very short and one nanosecond. So it can go and enter and you have the resolution to measure the distance.

GEHRKE: Super, super exciting.

TASHEV: And after that, I became assistant professor in the Technical University of Sofia. How I came to Microsoft is a consequence of that. So I was teaching data and signal processing, and the changes in Europe already started. Think about 1996. And then a friend of mine came back from a scientific institution from the former Eastern Germany, and he basically shared how much money West Germany has poured into the East German economy to change it, to bring it up to the standards, and that … it was, I think, 900 billion Deutsche Marks.

GEHRKE: But this was after the … 

TASHEV: After the changes. After, after basically the East and West Germany united. And then this was in the first nine years of the changes. And then we looked at each other in the eyes and said, wait a minute. If you model this as a first-order system, this is the time constant, and the process will finish after two times more of the time constant, and they will need another 900 billion Marks. You cannot imagine how exact became that prediction when East Germany will be on equal economically to the West Germany. But then we looked at each other’s eyes and said, what about Bulgaria? We don’t have West Bulgaria. And then this started to make me think that most probably there will be technical universal software, but in this economical crisis, there will be no money for research, none for development, for building skills, for going to conferences. And then pretty much around the same time, somebody said, hey, you know, Microsoft is coming here to hire. And I sent my résumé knowing that, OK, I’m an assistant professor. I can program. But that actually happened that I can program quite well, implementing all of those control systems for the telescope, etc., etc., and literally …

GEHRKE: And so there was a programming testing as part of the interview?

TASHEV: Oh, the interview questions were three or four people, one hour, asking programming questions. The opening was for a software engineer.

GEHRKE: Like on a whiteboard?

TASHEV: Like on a whiteboard. And then I got an email saying that, Ivan, we liked your performance. We want to bring you to Redmond for further interviews. I flew here in 1997. After the interviews, I returned to my hotel, and the offer was waiting for me on the reception.

GEHRKE: Wow, that’s fast.

TASHEV: So this is how we decided to move here in Redmond, and I started and went through two full shipping cycles of programs.

GEHRKE: So you didn’t start out in MSR (Microsoft Research), right?

TASHEV: Nope.

GEHRKE: Where were you first?

TASHEV: So, actually, I was lucky enough both products were version 1.0. One of them was COM+. This is the transactional server and the COM technology, which is the backbone of Windows.

GEHRKE: Was the component model being used at that point in time?

TASHEV: Component object model. Basically, creating an object, calling … getting the interface, and calling the methods there. And my experience with low-level programming on assembly language and microprocessor actually became here very handy. We shipped this as a part of Windows 2000. And the second product was the Microsoft Application Center 2000, which was, OK, cluster management system.

GEHRKE: But both of them had nothing to do with signal processing, right?

TASHEV: Nope. Except there were some load balancing in Application Center. But they had nothing to do with signal processing; just pure programming skills.

GEHRKE: Right.

TASHEV: And then in the year of 2000, there was the first TechFest, and I went to see it and said, wait a minute. There are PhDs in this company and they’re doing this amazing research? My place is here.

GEHRKE: And TechFest, maybe … do you want to explain briefly what TechFest is?

TASHEV: TechFest is an annual event when researchers from Microsoft Research go and show and demonstrate technologies they have created.

GEHRKE: So it used to be, like, in the Microsoft Conference Center.

TASHEV: It used to be in the Microsoft Conference Center …

GEHRKE: Like, a really big two-day event …

TASHEV: … and basically visited by 6, 7,000 Microsoft employees. And usually, Microsoft Research, all of the branches were showing around 150ish demos, and it was amazing. And that was the first such event. Pretty much …

GEHRKE: Oh, the very first time?

TASHEV: The very first TechFest. And pretty much not only me, but the rest of Microsoft Corporation learning that we do have a research organization. In short, in three months, I started in Microsoft Research.

GEHRKE: How did you get a job here then? How did that happen?

TASHEV: So … seriously, visiting TechFest made me to think seriously that I should return back to research, and I opened the career website with potential openings, and there were two suitable for me. One of them was in Rico Malvar’s Signal Processing Group …

GEHRKE: Oh, OK, yeah …

TASHEV: … and the other was in Communication, Collaboration, and Multimedia Group led by Anoop Gupta. So I sent my résumé to both of them. Anoop replied in 15 minutes; next week, I was on informational with him. When Rico replied, I already had an offer from Anoop to join the team. [LAUGHS]

GEHRKE: Got it. And that’s, that’s where your focus on communication came from then?

TASHEV: Yes. So our first project was RingCam.

GEHRKE: OK.

TASHEV: So it’s a 360-camera, eight-element microphone array in the base, and the purpose was to record the meetings, to do a, a meeting diarization, to have a 360 view, but also, based on the signal processing and face detection, to have a speaker view, separate camera for the whiteboard, diarization based on who is speaking based on the direction from the microphone array. Honestly, even today when you read our 2002 paper … Ross Cutler was creator of the 360 camera; I was doing the microphone array. Even today when you read our 2002 paper, you say, wow, that was something super exciting and super advanced.

GEHRKE: And that then you brought it all the way to shipping, right, and it became a Microsoft product?

TASHEV: So yes. At some point, it was actually monitored personally by Bill Gates, and at some point …

GEHRKE: So he was PMing it, basically, or …? [LAUGHS]

TASHEV: He basically was …

GEHRKE: He was just aware of it.

TASHEV: I personally stole the distributed meeting system in Bill Gates’ conference room.

GEHRKE: Wow.

TASHEV: We do have basically 360 images with Bill Gates attending a meeting. But anyway, it was believed that this is something important, and a product team was formed to make it a product. Ross Cutler left Microsoft Research and became architect of that team, and this is what became Microsoft RoundTable device. It was licensed to Polycom, and for many years was sold as Polycom [CX5000].

GEHRKE: Yeah, actually, I remember when I was in many meetings, they used to have exactly that device in the middle, and the nice thing was that even if somebody was remote, right, you could see all the people around the table and you got this, sort of, really nice view of who was next to whom and not sort of the transactional windows that you have right now in Teams. That’s a really interesting view.

TASHEV: So, as you can see, very exciting start. [LAUGHS] But then Anoop went and became Bill Gates’ technical assistant, and the signal processing people from his team were merged with Rico Malvar’s signal processing team, and this is how I continued to work on microphone arrays and the speech enhancement, and this is what I do till today.

GEHRKE: And you mentioned, like, amazing products from Microsoft like Kinect and so on, right. And so you were involved in the, like, audio processing layer of all of those, and they were actually then … part of it was designed here in this room?

TASHEV: Yep.

GEHRKE: So tell me a little bit more about how that happened.

TASHEV: You know, at the time, I was fascinated by a problem which was considered theoretically impossible: multichannel acoustic echo cancellation. There was a paper written in 1998 by the inventor of the acoustic echo cancellation from Bell Labs stating that stereo acoustic echo cancellation is not possible.

GEHRKE: And he proved it, or what does it mean? He just …

TASHEV: It’s very simple. You have two unknowns—the two impulse responses from the left and the right loudspeaker—and one equation; that’s the microphone signal. What I did was to circumvent this. When you start Kinect, you’ll hear some melodic signals, and this is the calibration. At least you know the relation between the two unknowns, and now you have one unknown, which is basically discovered using an adaptive filter, the classic acoustic echo cancellation. So technically, Kinect became the first device ever shipped with surround sound acoustic echo cancellation, the first device ever that could recognize human speech from 4 1/2 meters while the loudspeakers are blasting. And gamers are listening to very loud levels of their loudspeakers.

GEHRKE: So maybe just tell the audience a little bit, what does it mean to do acoustic echo cancellation? What is it actually good for, and what does it do?

TASHEV: So in general, speech enhancement is removing unwanted noises and sounds from the desired signal. Some of them we don’t know anything about, which is the surrounding noise. For some of them, we have a pretty good understanding. This is the sound from our own loudspeakers. So you send the signal to the loudspeakers and then try to estimate on the fly how much of it is captured by the microphone and subtract this estimation, and this is called acoustic echo cancellation. This is part of every single speakerphone. This is one of the oldest applications of the adaptive filtering.

GEHRKE: So would the right way to think about this is that noise cancellation is cancelling unwanted noise from the outside?

TASHEV: Unknown noises …

GEHRKE: … whereas acoustic echo cancellation is cancelling the own noise that actually comes …

TASHEV: … which we know about.

GEHRKE: Right, OK.

TASHEV: And that was an amazing work, but … it also started actually in TechFest. I designed this surround sound echo cancellation, and my target was … at the time, we had Windows Media Center. It was a device designed to stay in the media room and controlling all of those loudspeakers. And I made sure to bring all of the VPs of Windows and Windows Media Center, and then I noticed that I started repeatedly to see some faces which I didn’t invite—I didn’t know—but they came over and over and over. And after the meeting, after TechFest, a person called me and said, “Look, we are working on a thing which your technology fits very well,” and this is how I started to work for Kinect. And in the process of the work, I had to go and talk with industrial designers because of the design of the microphones, with electrical designers because of the circuitry and the requirements for identical microphone channels, and with the software team, which had to implement my algorithms, and this … actually, at some point, I had an office in their building and was literally embedded working with them day and night, especially at the end of the shipping cycle, of the shipping cycle when the device had to go out.

GEHRKE: And this was not a time when you could go, like, in the device and, you know, update software on the device or anything. The device would go out as is, right?

TASHEV: Actually, this was one of the first devices like that.

GEHRKE: Oh, it could?

TASHEV: Yup.

GEHRKE: Wow, I didn’t know that.

TASHEV: Already Kinects were manufactured. They are boxed; they are already distributed to the, to the stores. But there was a deadline when we had to provide the image when you connected Kinect to your Xbox and it has to be uploaded.

GEHRKE: But, no, I get that. But then once it was actually connected to the Xbox, you could still update the firmware on the …

TASHEV: Yes, yes.

GEHRKE: Oh, wow. That’s, that’s really cool. OK.

TASHEV: But it also has a deadline. So that was amazing. Literally left us, all of us, breathless. There are plenty of serious technological challenges to overcome. A lot of firsts as a technology is, basically, was brought to this device to make sure … and this is the audio. And next to us were the video people and the gaming people and the designers, and everybody was excited to be working like hell so we can basically bring this to the customers.

GEHRKE: Wow, that’s super exciting. I mean even just being involved in … I think that’s one of the really big things that is so much fun here at Microsoft, right, that you can get whatever you do in the hands of, you know, millions—if not hundreds of millions—of people, right. Coming, coming back to, you know, your work now in, in audio signal processing, and that whole field is also being revolutionized like many other fields right now with AI, right.

TASHEV: Absolutely.

GEHRKE: Photography, one of the other fields that you’re very passionate about, is also being revolutionized with AI, of course.

TASHEV: Also revolutionized.

GEHRKE: You know, in, in terms of changes that you’ve made in your career, how do you deal with such changes, and what were … you know, this is something where you have been an expert in a certain class of algorithms, and now suddenly it says there’s this completely new technology coming along, and we need to shift. How are you dealing with this? How did you deal with this, personally?

TASHEV: Let me put it in …

GEHRKE: In some sense, you’re becoming a little bit of a dinosaur in a little bit while …

TASHEV: Oh, not at all.

GEHRKE: That’s what I’m saying.

TASHEV: I wouldn’t be in research! [LAUGHS]

GEHRKE: Exactly. How did you overcome that?

TASHEV: So, first, each one of us was working and trying to produce better and better technology, and at the time, the signal processing, speech enhancement, most of the audio processing was based on statistical signal processing. You build statistical models, distributions, hidden Markov models, and get …

GEHRKE: Like speech recognition.

TASHEV: … certain improvements. Yep. And all of us started to sense that this set of tools we have started to saturate. And it was simple. We use the simple models we can derive. Let’s say speech is Gaussian distribution; noise is Gaussian distribution. You derive the suppression rule. But this is simplifying the reality. If you apply a more precise model of the speech signal distribution, then you cannot derive easily the suppression rule, for example, in the case of noise suppression. And it was literally hanging in the air that we have to find a way, a way to learn from data. And I have several papers, actually before the neural networks start to appear, that let’s get a big dataset and learn from the data this suppression rule.

GEHRKE: So a more data-driven approach already.

TASHEV: Data-driven approach. I have several papers from that, and by the way, they were not quite well accepted by my audio processing community. All of them are published on bordering conferences, not in the core conferences. I got those papers rejected. But then appeared neural networks. Not that they were something new. We had neural networks in ’80s, and they didn’t work well. The new … the miracle was that now we had an algorithm which allows us to train them. Literally, next year after the work of Geoff Hinton was published in the Implementation of Deep Learning, several things happened. At first, my colleagues in the speech research group started to do neural network–based speech recognition, and I, in my audio group, started to do neural network–based speech enhancement. This is the year of 2013 or 2014. We had the speech, neural network–based speech enhancement algorithm surpassing the existing statistical signal processing algorithm literally instantly. It was big. It was heavy. But better.

GEHRKE: When did the first of these ship? What … can you tell any interesting ship stories about this?

TASHEV: The first neural network–based speech enhancement algorithm was shipped in 2020 in Teams.

GEHRKE: OK, OK.

TASHEV: We had to work with that team for quite a while. Actually, four years took us to work with Teams to find … you see, here in the research, industrial research lab we have a little bit different perspective. It’s not just to make it work; it’s not just to make it a technology. That technology has to be shippable. It has to meet a lot of other requirements and limitations in memory and in CPU and in reliability. It’s one thing to publish a paper with very cool results with your limited dataset and completely different to throw this algorithm in the wild, where it can face everything. And this is why it cost us around four years before to ship the first prototype in Teams.

GEHRKE: That, that makes sense. And I think a lot of the infrastructure was also not there at that point in time early on, right, in terms of, you know, how do you upload a model to the client, even in terms of all the model profiling, you know, neural architecture search, quantization, and other tooling that now exists where you can take a model …

TASHEV: That’s correct.

GEHRKE: … and squeeze it on the right kind of computation for the …

TASHEV: That’s correct. And …

GEHRKE: So you did all of that manually, I guess, at that point in time.

TASHEV: Initially, yes. But new architectures arrived. The cloud. Wow, it was a savior. You can press a button; you can get a hundred or thousand machines. You can run in parallel multiple architectures. You can really select the optimal from every single standpoint. Actually, what we did is we ended up with a set of speech enhancement algorithms. Given computing power, we can tell you what is the best architecture for this, or if you want to hit up this improvement, I can tell you how much CPU you will need for that.

GEHRKE: Got it.

TASHEV: But that tradeoff is also something very typical for industrial research lab and not very well understood in academia.

GEHRKE: Makes sense. Let me, let me switch gears one last time, namely, I mean, you have made quite a few changes in your career, you know, throughout, right. You started as an assistant professor and then became, sort of, a core developer, then, you know, were a member of a signal processing group and now you’re, sort of, driving a lot of the audio processing research for the company. How do you deal with this change, and do you have any advice for our listeners on how to, you know, keep your career going, especially as the rate of change seems to be accelerating all the time?

TASHEV: So for 25 years in Microsoft Corporation, I have learned several rules I follow. The first is dealing with ambiguity. It is not just change in the technology but changes in the … of the teams and organizations, etc., etc. Simply put, there are things you cannot change. There are things you cannot hide. Just accept them and go on. And here comes the second rule. To succeed in Microsoft, you have to be laser focused on what you are doing. This is the thing you can change. Focus on the problems you have to solve, do your job, and be very good at it. This is the most important … those are the two most important rules I have used in my career in Microsoft.

GEHRKE: OK, super, super interesting, Ivan. Thank you very much for this amazing conversation.

TASHEV: Thank you for the invitation, Johannes.

GEHRKE: To learn more about Ivan’s work or to see photos of Ivan pursuing his passion for shooting film and video, visit aka.ms/ResearcherStories (opens in new tab).

DESNEY TAN: Born and raised in Singapore. Dad was an architect. Mom did everything, um, to run the family. When I turned 13, Mom and Dad came to me and they said, “Hey, would you like to try something new?” I said sure. You know, I had no idea what they, they were thinking. Two weeks later, they sent me to the US to study. Um, looking back, sometimes I flippantly claim I was just eating too much at home and so they had to send me away. [LAUGHTER] But actually, it was, you know, I think it was prescient on their part. They sort of looked at my path. They looked at the education system. They looked at the way I learned and the way I created and the way I, I acted, and they somewhat realized, I think, very early on that the US was a great … would, would be a great place for me to sort of flourish and, and sort of experiment and explore and, and grow.

GEHRKE: And so how did it work? You just went by yourself?

TAN: So I had an aunt and an uncle in Louisiana. Spent a couple of years in high school there. Um, sort of … fun, fun side story. They looked at … the high school looked at my math curriculum in Singapore, and they said, “Oh, he’s at least a year ahead.” So they skipped me a year ahead. And then through some weird miscalculations on their part, they actually ended up skipping me nearly two years ahead.

GEHRKE: Oh, wow.

TAN: And by the time we realized, I had already integrated into school, the courses were just fine, and so I ended up skipping a lot of years.

GEHRKE: So you ended up graduating then high school what …

TAN: Pretty early. I was 15.

GEHRKE: 15 …

TAN: Graduated from high school. Got to college. Had no idea what I wanted to do. What 15-year-old does? Um, ended up in liberal arts college, so University of Notre Dame. So, so I don’t know how Mom let me do this, but, you know, I got all my acceptance letters together. I said I don’t know anything about college. I don’t know where I want to go. I don’t know what I want to do. I’m going to toss all the letters up in the air, and the one that lands on top is the school I’m going to.

GEHRKE: And that’s what you did?

TAN: Yeah, that’s exactly what I did. Um, divine intervention, let’s call it. Notre Dame landed on top. You know, switched majors a bunch of times. I started off in aerospace, did chemical engineering, civil engineering. I was on the steps of becoming a priest until they sent me away. They said, “Hey, if it’s not a mission and a calling, go away and come back later.” And ended up with a computer engineering degree. You know, I had great mentors, you know, who looked out for me. I had a couple of guardian angels out there, you know, guided me along, and that, you know, that was just a wonderful breadth of education. Went back to the military for a couple of years. Uh, served there for a couple of years. Did a bunch of growing up.

GEHRKE: That, that’s quite a change, right, from being like in college and then going back to the military.

TAN: Yeah, yeah, it was a mandatory service in Singapore, and so I went back. Had a ton of fun. Learned a bunch of stuff about the world, about myself. I claim the military is one of the few organizations in the world that takes an 18-year-old and teaches them leadership, um, and teaches them about themselves and teaches them about how to push themselves and where the boundaries are. And so fairly accidentally, I, I got to benefit from all of that. At the end of that, I realized my computer engineering degree was, you know … I realized two things. One, my computer engineering degree was a little outdated by the time I got out of the military, and two, that I didn’t love being told what to do. [LAUGHS]

GEHRKE: [LAUGHS] OK.

TAN: So I came back. Uh, did grad school. I was at Carnegie Mellon. Ended up getting hooked up with a wonderful professor, Randy Pausch.

GEHRKE: “The Last Lecture,” right?

TAN: Who gave “The Last Lecture” in his last days. You know, learned a ton from him not only about academics and scholarship, but also about life and, um, and leadership.

GEHRKE: And so he was at the intersection of graphics and HCI, right, if I remember correctly?

TAN: That’s correct, yeah.

GEHRKE: So what is your PhD in?

TAN: My PhD was actually looking at, um, distributed displays in virtual reality. So how, how the human brain absorbs information and uses the world around us to be able to, um, interact with digital data and analog data.

GEHRKE: Early on in a really important field already.

TAN: Yeah, no, it was great. Spent a couple of years with NASA in the Jet Propulsion Lab doing autonomous navigation. This was the early days of, um, you know, AI and, and planning.

GEHRKE: So those aerospace engineering classes, were they actually useful?

TAN: They, you know, all the classes I took ended up coming back to be useful in a number of ways. And actually, um, you know, the diversity of viewpoints and the diversity of perspectives is something that sat very deeply in me. So anyways, you know, spent some time at NASA. Um, spent some time at Disney with the Imagineers building virtual reality theme parks. This was the late ’90s, early 2000s. So Disney at the time had all the destination theme parks: Disneyland, Disney World, places you would fly for a week and, and spend a week at. Their goal was really to build a theme park in a box that they could drop down into the urban centers, and the only way to get a theme park into a building was digital experiences. And so this was the very early days of VR. We were using, you know, million-dollar military-grade headsets. They were, you know, 18, 19 pounds.

GEHRKE: Wow.

TAN: Disney was one of the companies—and, you know, it’s sat with me for a very long time—that designs experiences for every single person on earth, right. So these headsets had to work on your 2-year-old. They had to work on your 102-year-old. They had to work on, you know, a person who spoke English, who read, who didn’t read, who didn’t speak English. You know, tall, short, large, small, all of it. And they did a wonderful job finding the core of what it means to be human and designing compelling experiences for all of us, um, and that was a ton of fun. We ended up deploying these facilities called DisneyQuest. There was one in Chicago; one in Orlando. They just closed them down a couple of years ago because actually all the VR rights have now migrated into the theme parks themselves.

GEHRKE: And it was actually a VR experience? You would go and sit …

TAN: It was a VR experience. They dropped them down. They had basically buildings. There were, you know, floors full of classic and new-age arcade games. And then there were VR experiences that you could run around in and, um, interact with.

GEHRKE: Interesting. I’ve never, I mean, I lived in Madison for four years, but I’ve never heard of that Quest experience. It seems to be a fun way to experience Disney … by not going to any of the, the theme parks.

TAN: It was super fun. Um, yeah, we, I personally got to work on a couple of rides. There was Pirates of the Caribbean.

GEHRKE: Oh, wow.

TAN: So you put on … a family would put on headsets and kind of run around, shooting pirates and what have you. And then the Aladdin ride was I thought one of the better rides.

GEHRKE: Oh, wow, yeah …

TAN: Where you sit on a magic carpet as you can imagine.

GEHRKE: Oh yeah. That sounds fun.

TAN: It was perfectly scripted for it. Um, anyways, ended up at Microsoft largely because entertainment technology while a lot of fun and while I learned a ton was, uh, strangely unsatisfying, and there was something in me and, you know, that was seeking human impact at scale in a much deeper and much more direct way. And so I thought I’d be here for three or four years largely to learn about the tech industry and how, you know, large pieces of software were deployed before going off and doing the impact work. And I’ve now been here for nearly 20 years.

GEHRKE: And where do you start? Did you start out right away at Microsoft Research, or were you first in a product group?

TAN: My career here has been a cycle of starting in Microsoft Research, incubating, failing, trying again. Failing again. You know, at some point, screaming “Eureka!” [LAUGHTER] and then doing my tours of duty through the product groups, commercializing … productizing, commercializing, you know, seeing it to at least robustness and sustainability if not impact and then coming back and doing it again. Um, and the thing that’s kept me here for so long is every time I’ve completed one of those cycles and thought I was done here, um, the company or the world in some cases would throw, you know, a bigger, thornier, juicier thing in front of me, and Microsoft has always been extremely encouraging, um, and supportive of, you know, taking on those challenges and really innovating and opening up all new, whole new opportunities.

GEHRKE: I mean this whole cycle that you’re talking about, right, of sort of starting out small at MSR (Microsoft Research), you know, having sort of the seed of an, of an idea and then growing it to a bigger project and at some point in time transitioning, transitioning it into, into the product group and actually really making it a business. So tell me about … you said you have done this, you know, a few times and, you know, once you were even highly successful. I’d love to learn more about this because I think it’s so inspiring for everybody to learn more about this.

TAN: Yeah. No, it’s been magical. I have to say before going into any of these stories that none of these paths were architected. As, as you well know, they never are. So actually my, my first experience was as an intern here, and, you know, I was a sort of brash, perhaps rash, intern. I was working on virtual reality, and in the evenings, I would meet with folks around the company to learn more, and I met with a team that was building out multi-monitor functionality in Windows NT. Prior to Windows NT, Windows computers had one and only one monitor, and they started to build the functionality to build multiple. As the brash grad student, you know, I, I had different thoughts about how this should be implemented and, you know, couldn’t convince anyone of it. And so in the evenings, I ended up starting just to build it. At the end of the internship, in addition to all the stuff I was doing, I said, “Hey, by the way, I’ve built this thing. You know, take it or leave it. Here you go.” And it ended up being the thing that was implemented in NT for a variety of reasons. That really got me hooked. Prior to that, I had imagined myself an academic, going back and, you know, being a professor somewhere in academia. And as soon as I saw, you know, the thing I did and that, you know, Microsoft actually polished up and made good in the real world…

GEHRKE: And shipping in millions and millions of desktops, right?

TAN: That’s right. There was no getting away from that.

GEHRKE: OK, right.

TAN: When I first got here, MSR had actually hired me thinking I’d work on virtual reality. And I got here and I said, hey, VR … I’ve just done a ton of VR. VR is probably 15 or 20 years out from being democratized and consumerized. I’m going to do something for a couple of years, and then I’ll come back to this. Um, so I got into computational neuroscience, looking at, um, sensors that scanned or sensed the brain and trying to figure out mental state of people. I had the imagination that this would be useful both for direct interaction but also for understanding human behavior and human actions a little bit better. We won’t go into that work, but, um, what happened with the productization of that was I went … this was at the time when Bill Gates was actually pushing very hard on tablet PCs and the stylus and the pen as an interesting input modality. The realization we had was, hey, we’ve got spatial temporal signal coming off the brain we’re trying to make sense of; the tablet guys had spatial temporal signal coming off a pen they were trying to make sense of in handwriting recognition. And so we went over and we said, hey, what interesting technological assets do you have that we can steal and use on the brain. Turns out they were more convincing than us. And, and so they said, hey, actually you’re right. The problems do look similar. What do you have that you could bring over? And so if you look at the handwriting recognition system even that stands today, it’s a big mess of a neural network, um, largely because that came out of interpreting neural signal that got transferred into the handwriting recognizer.

GEHRKE: I see.

TAN: And so I ended up spending two, maybe 2 1/2 years, working not only on the core recognition engine itself but also the entire interface that ran around the tablet PC and, you know, the tablet input panel.

GEHRKE: But that’s sort of an interesting realization, right. You came because you thought you would land Technology X for Application Y, but actually you land it for a very different application.

TAN: That’s right. And, and each cycle has had a little bit of that surprise and that serendipity, which we’ve now built into the way we do research. And, um, you sort of head down a path because it moves you forward as quickly as possible. But you keep your eyes peeled for the serendipitous detours and the, the discovery that comes out of that. Um, and I think that’s what makes Microsoft Research as an organization, um, so compelling and, and so productive, right, as … we, we do run very fast, but we have the freedoms and, you know, the flexibility really to take these windy paths and to take these detours and, and to go flip over, you know, rocks, some of which end up being, you know, dead ends.

GEHRKE: Right.

TAN: Others of which end up being extremely productive.

GEHRKE: Right. And so if you think about, let’s say, a junior person in the lab, right. They’re sort of looking at you and your career and saying, “Wow, what steps should I take to, you know, become as successful as Desney?” What, what advice would you give them, right? Because it seems like you have always had sort of MSR as sort of your rock, right. But then you jumped over the river a few times, but then came back and jumped over again. Came back.

TAN: First off, I, I don’t know that Desney has been so successful so much as, you know, the people around Desney have been extremely successful and Desney’s gotten to ride the wave. But, yeah, no, I mean every, everyone’s … you know, as I look around the table and the board, you know, everyone has a slightly different journey, and everyone has slightly different work styles and mindsets and personalities and risk tolerance and what have you. Um, so the first thing really is, is not to try to fully emulate anyone else. I always claim we’re, we’re kind of like machine learning models, right. We, we should be taking input data, positive and negative, and building our models of ourselves and our models of the world and really operating within that context. I think having a North Star, whether it’s implicit or explicit, has been extremely useful for myself and the people around me.

GEHRKE: By North Star, you mean like a philosophical North Star or technical North Star or North Star in what you want to be? What, what do you mean?

TAN: Yes, yes, yes. All of it.

GEHRKE: So tell me more about your personal North Star.

TAN: For, for, for us … for myself, it’s really been about human impact, right. Everything we do is centered on human impact. We do research because it’s part … it’s, it’s one of the steps towards achieving human impact. We productize because it’s one of the steps towards human impact. Our jobs are not ever done until we hit the point of human impact, and then they’re not quite done because there’s always more to be had. Um, so I think having that, you know, perhaps a value system, um, at least, you know, sort of grounds you really nicely and, and creates, I think, or can create a courage and a bravery to pursue, which I think is important. You know, different people do this differently, but I have been very lucky in my career to be surrounded by people that have been way, way, way better than myself, um, and, and extremely generous of their passions and their skills and their expertise and their time. You know, ask it and just about any successful person by whatever definition and I think they’ll tell you the same thing, that it’s the people around. And then being tolerant, maybe even seeking of, this windy path. You know, when I was early in the career, I always looked at successful people, or people I deemed to be successful, and it always felt like they had a goal, and it was a very nice straight line to get there, and they did all the right things and, and took all the right steps, and, um, and I don’t know anyone today that I deem to be successful that had a straight-line path and did all the right things.

GEHRKE: Yeah, and it’s often these setbacks in, you know, one’s career that actually give you often some of the best learnings because either of some things that you’ve sort of done structurally wrong or some things that, you know, you really need more experience and, and, you know, that setback gave you that experience. So, so one other question around this is also just around change, right. Because especially right now, we’re living in this time where maybe the rate of change especially in AI is kind of unprecedented. I mean, benchmarks are falling in like a quarter of the time than they would have thought to be lasting. You know, we all have played with ChatGPT. Just extrapolate that out a few more months, if not years, right. OpenAI is here talking about AGI. So how do you think about change for yourself and evolution and learning, and do you have any, any routines? How, how do you keep up with everything that’s going on?

TAN: Yeah, it’s, uh … good question. I guess the overarching philosophy, the approach that I’ve taken with my career, is that everything’s constantly in change. You know, the rate of change may vary, and the type of change and the, the mode of change might vary, but everything’s constantly changing, and so our jobs at any given point are to understand the context in the world, in the organization, with the people around you, and really be doing the best that you can at any given moment. And as that context changes, you kind of have to dynamically morph with it. I subscribe pretty fully to the Lean Startup model. So, you know, formulate hypotheses … and this is the research process really, right. Formulate hypotheses, test them as quickly as you can, learn from that, and then do it again, and rinse and repeat. And then … and, you know, you could sort of plot your path and steer your path through based on that. Um, and so we operate very much on that. As, as the world changes, we change. As, you know, the org changes, we change. And there’s a certain robustness that comes along with that. It’s not all roses, and obviously change is and uncertainty is, is a difficult context to operate in.

GEHRKE: And super interesting because it also speaks to some of the things that one should, um, sort of look out for when doing research, right. If you’re saying, well, I have these hypotheses and I want to quickly test them, right, if I’m in a field or if I work with data that I, you know, cannot really use, where the testing of an hypothesis will take months if not years to bring out, this might not be the best research direction. So how should I think about sort of research, the choice of research problems …

TAN: It’s a good question, yeah.

GEHRKE: … sort of with this, with this change in mind, right?

TAN: Yeah, yeah. Um, I don’t know. I, I’m, I guess … again, I’m brash on this. There are, there are very few problems and spaces that can’t be navigated, um, and so things that seem impossible at first glance are often navigable, you know, with a little bit or maybe sometimes a lot of creativity. Um, you know, if our jobs are to take Microsoft and the rest of the world to places that Microsoft and the rest of the world might not get itself to—hopefully positive places—then we’re going to have to do things in a way that is probably unnatural for Microsoft and the rest of the world, um, to get there. And the company and the organization, MSR, has been extremely supportive of that level of creativity.

GEHRKE: Can you give an example of that for …?

TAN: We had Cortana, which is our speech recognition and conversational engine. We didn’t really have a platform to deploy that on. At the same time, we saw a bunch of physicians, clinicians, struggling with burnout because they were seeing patients for less than half the time. They were spending more of their time sitting in front of the computer, documenting stuff, than they were seeing patients and treating patients. We said, hey, what if you put the two together? What if you sat in the room, listened to the doctor and the patient, and started to automatically generate the documentation? And in fact, if you did that, you could structure the data, which leads for better downstream analytics. Um, and if you did that, you could start to put machine learning and AI and smarts into the system, as well. That project, which was called EmpowerMD, led eventually—after a bunch of missteps and a bunch of learnings and a bunch of creativity—to a very deep partnership with Nuance, um, and creation of Dragon Ambient eXperience and the eventual acquisition thereof of that company. And, um, it’s just a wonderful product line. It’s, you know, kind of a neat way to think about data and intelligence and human augmentation and integration into otherwise messy, noisy human processes. Um, but yeah, you know, I think with enough creativity, um, you know, we’ve, we’ve bumped into very, very few brick walls.

GEHRKE: And what I love about the story is that it’s not about a specific technology choice, but it’s more about a really important problem, right.

TAN: That’s right. Yeah. If your problem is right and if your conviction is right about the value of the solution …

GEHRKE: Yeah.

TAN: …you build teams around it. You build processes around it. You’re creative in the way you execute. And, um, I’d say more times than not, we end up getting there.

GEHRKE: Yeah, well, I love that insight because it’s often much more valuable to solve an important problem than to land some deep technology on a problem that very few people care about …

TAN: I think that’s right.

GEHRKE: …and it seems like that’s what you have done here.

TAN: Yeah.

GEHRKE: Well, it was really great and inspiring to hear from you, Desney. Thanks so much for the conversation.

[OUTRO MUSIC]

TAN: Yeah, thanks for having me, Johannes.

GEHRKE: To learn more about Desney’s work or to see photos of Desney during his winding journey to Microsoft, visit aka.ms/ResearcherStories (opens in new tab).