{"id":306413,"date":"2009-10-26T09:00:28","date_gmt":"2009-10-26T16:00:28","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=306413"},"modified":"2016-10-16T19:09:33","modified_gmt":"2016-10-17T02:09:33","slug":"total-recall-how-to-have-it-all","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/total-recall-how-to-have-it-all\/","title":{"rendered":"Total Recall: How to Have It All"},"content":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research<\/p>\n

In September, pioneering computer-science researcher <\/em>Gordon Bell<\/em> (opens in new tab)<\/span><\/a> and his Microsoft Research colleague <\/em>Jim Gemmell<\/em> published <\/em>Total Recall: How the E-Memory Revolution Will Change Everything (opens in new tab)<\/span><\/a>, a book that summarizes nearly a decade of an effort to record digitally everything in Bell\u2019s life: What he did, what he saw, what he read, some of what he ate, what he felt\u2014his entire life experience. The book is written in Bell\u2019s voice, but the publication, like the <\/em>MyLifeBits<\/em> (opens in new tab)<\/span><\/a> project that generated it, is a true collaboration. Bell and Gemmell recently found time to discuss the genesis of the research\u2014and its implications for us all. <\/em><\/p>\n

Q: How is the book being received?<\/strong><\/p>\n

Gemmell:<\/strong> We\u2019re getting a really good reception. We\u2019ve heard from entrepreneurs who are inspired by it and from typical people who are excited and wanting to get started. We get every kind of reaction, from people who are scared of the concept, usually because they don\u2019t truly understand it, to people who are really excited.<\/p>\n

Q: Gordon, what do you recall as the first germ of the idea that eventually led to MyLifeBits?<\/strong><\/p>\n

\"Gordon

Gordon Bell (left) and Jim Gemmell<\/p><\/div>\n

Bell:<\/strong> It started from three points of view. Raj Reddy [of Carnegie Mellon University] asked for my books so he could scan them as part of his Million Book Project (opens in new tab)<\/span><\/a>. Meanwhile, I wanted to scan and get rid of all my paper; I\u2019ve had a long desire to have everything online. And [Microsoft Chairman] Bill Gates (opens in new tab)<\/span><\/a> stimulated the idea by saying that, someday, you\u2019ll be able to record everything you see and hear.<\/p>\n

I published a paper on storing everything in January of 2001. In a way, I thought I was done. I was saving everything, but it became clear that the problem was a search problem. You could save everything, and there was a huge value in saving everything, but the big problem was organizing and then searching\u2014in general, trying to provide value to the content.<\/p>\n

At that point, Jim Gray (opens in new tab)<\/span><\/a> and Jim Gemmell said, \u201cLet\u2019s create a serious database and work on the problem of organization and search.\u201d Roger Lueder came in to work on the database and to write code to make it work.<\/p>\n

Gemmell:<\/strong> Until that point, Gordon had been trying to capture everything he had already. Then, we started to make a concerted effort to capture more, to write software to record every Web page visited, to set up a telephone recording system in his office. We set up TV and radio recording\u2014all these tools to capture even more of someone\u2019s life.<\/p>\n

Bell:<\/strong> In 2003, what really kicked the project into gear was Lyndsay Williams\u2019 SenseCam (opens in new tab)<\/span><\/a>. We got one of the first ones, and we incorporated it into the recording. Now, we had an icon. It\u2019s a mixed blessing, because we don\u2019t record our daily activities with this camera, but that\u2019s what grabbed everybody\u2019s imagination.<\/p>\n

When you\u2019ve got a project, it\u2019s important to have a good name and to have something that\u2019s going to grab everybody. The MyLifeBits name was good; that got us a nice article (opens in new tab)<\/span><\/a> in New Scientist<\/em>. Then, in 2003, we got a further boost from the SenseCam. People said, \u201cThat\u2019s what it means to record everything you see,\u201d even though we don\u2019t really advocate doing that.<\/p>\n

Q: Jim, what prompted you to get involved?<\/strong><\/p>\n

Gemmell:<\/strong> Gordon and I were working closely together on telepresence. His going paperless was partly an attempt to be an efficient teleworker. He wanted his files available to him wherever he may be.<\/p>\n

Gordon was beginning to scan his photographs, and he said, \u201cI don\u2019t even remember what half these things are, and unless they\u2019re labeled, I can\u2019t really do anything with them.\u201d I said, \u201cWhy don\u2019t we try to write some software?\u201d<\/p>\n

Here, I had a factory that was producing data\u2014collecting, scanning, and committed to capturing everything. That\u2019s what made it interesting. It\u2019s the corpus you get that starts to make the whole project interesting. This is the epitome of problem-driven research\u2014an approach that both Jim Gray and Gordon advocate.<\/p>\n

Q: What\u2019s a real-life scenario in which the results of this project have proved invaluable to you?<\/strong><\/p>\n

\"total-recall\"Gemmell:<\/strong> What we\u2019ve done is a small foretaste of what\u2019s eventually going to be possible. But something as simple as writing a control that would record every Web page that we visited \u2026 It became something you come to rely on. If I\u2019ve ever seen a Web page, I\u2019ve got a copy of it, and I can find it again quickly because I only have to search my own corpus, not the entire Web. At first, it just seemed like something fun to do, but over time, it became something you presume is part of your technological arsenal.<\/p>\n

Gordon and I are treated in a different way by people because we have these electronic memories. People come to us saying, \u201cDon\u2019t you have such and such?\u201d Just the other day, I had my son saying, \u201cDo you have a copy of that essay I wrote several years ago?\u201d And I said, \u201cYes, of course.\u201d Within about a minute, I was able to e-mail that to him.<\/p>\n

Bell:<\/strong> Having everything gives you a feeling of security. You almost feel smug that you\u2019ve got it all, you\u2019ve got your life with you, you\u2019ve got a record of things. The other day, I had a bunch of receipts that came in, so I took a picture of all of them and threw them away. I submitted an expense report, and later I was asked for a receipt. so I got that picture and sent it away.<\/p>\n

On the health side, you get something important, whether it\u2019s an explanation of benefits, or immunizations, or allergies, or a hospital discharge. I once had to go into a hospital that didn\u2019t know me, and I had a lot of information that was critical. Everyone should go ahead and have this health information available when they travel, and Microsoft\u2019s HealthVault (opens in new tab)<\/span><\/a> is a fine place to keep it.<\/p>\n

Gemmell:<\/strong> People today have health records strewn around, at the general practitioner, at the lab, at the chiropractor, at the dentist\u2014all over the place. In the future, you\u2019ll have all that information at your fingertips. You\u2019ll have a quantitative record of your health. You\u2019ll have your blood pressure and your temperature, and, eventually, you\u2019ll have in-body sensors knowing what\u2019s going on with your blood. Our health data will be increasingly captured with all sorts of instrumentation. That will be a huge change.<\/p>\n

Bell:<\/strong> I encourage all my grandchildren to retain everything they do, all their homework, and take pictures of their artwork. The whole family is now geared to this, that there will be a time when this information will be useful. It certainly doesn\u2019t cost anything. The cost to keep an essay or to photograph a piece of artwork or take a little movie is so small that if there\u2019s any value at all, it\u2019s worth it.<\/p>\n

When you\u2019ve done this over a long time, you basically see that your enemy is a little scrap of paper. You\u2019ll write something down\u2014a phone number, a name\u2014and the chances are that if you ever do that, it\u2019s going to be used again. You might as well tell the computer, because you will need it later on. I went paperless in 2002, and, at that point, I had maybe a foot of stuff a year that I scanned. Now, it\u2019s down to three or four inches a year, because more information is coming in electronically, such as bills.<\/p>\n

Q: How much time does it take to maintain your database?<\/strong><\/p>\n

Gemmell:<\/strong> About the only time I spend is scanning paper. A lot of the research that we\u2019ve done or that we\u2019ve encouraged is in automatic methods of capturing information. There is so much that it\u2019s got to be automatic.<\/p>\n

Bell:<\/strong> Our rule is that the computer has to understand what we\u2019re doing, so we give it as much information as we can. That means labeling photographs. Increasingly, though, computers have good face-recognition software. The computer will understand more, and that makes it a lot easier and much more useful.<\/p>\n

Gemmell:<\/strong> When Gordon was scanning his old photos, he had to label them to get any value. But if he has a new photo and he was using his GPS, that\u2019s both location- and time-stamped. Between the time and the location and what\u2019s on his calendar\u2014and with face recognition getting better\u2014the amount of work has gone way down.<\/p>\n

Bell:<\/strong> Our feeling from the start has been that if it\u2019s going to take any time, people are not going to use it. Whatever you do has to almost immediately pay for itself in terms of time savings.<\/p>\n

Q: Gordon, what event occurred before the project began that you wish you had been able to capture?<\/strong><\/p>\n

Bell:<\/strong> I had been pretty good about keeping things, but my regret is that I threw anything<\/em> away. If I had known what I know now, I would have kept everything. I would have five to 10 times more stuff.<\/p>\n

I was lucky in that I was head of engineering at Digital, and they let me\u00a0have my files, so I have a lot of documents from the 12 years that I was head of engineering. That provided an interesting corpus. I don\u2019t have all the e-mails I had while running engineering, which would be nice. I should have kept all my e-mails.<\/p>\n

Q: How is your information shared?<\/strong><\/p>\n

Gemmell:<\/strong> We\u2019re not life bloggers. We\u2019re not interested in sharing all of our lives on the Web with the general public. We think that people who do that are pretty crazy, and the number of life-changing catastrophes of people appearing publically on the Web is staggering. We are life loggers<\/em>. It\u2019s intensely private and personal for us. We\u2019re very interested in research that makes things more safe, more secure, and private. We\u2019re interested in pursuing the notion of a Swiss data bank that can have plausibly deniable data storage that\u2019s secure and that makes it possible to deny things exist and keep it out of the reach of anybody but yourself.<\/p>\n

Q: What would it take for the Total Recall concept to attain critical mass? What would that mean?<\/strong><\/p>\n

Gemmell:<\/strong> It\u2019s going to attain critical mass. It\u2019s inevitable. The train has left the station. In fact, we were concerned about not getting the book published fast enough. We have new startups in all kinds of areas related to Total Recall. Some of the things we did when we began, that were a real nuisance to do with software, today are standard-issue features, like desktop search and recording of instant-messenger conversations. The iPod Nano now includes video recording and a pedometer. GPS in cellphones is getting more common. There are already all kinds of wearable health devices. Things are already rolling in this direction, no question about it.<\/p>\n

Bell:<\/strong> One of the most important things about having everything available is what it does for our lives. I can work anywhere; I just back up my computer.<\/p>\n

Gemmell:<\/strong> In the future, it will be the exception to not be able to recall something. Certainly, everything you ever read\u2014all your correspondence, books, and articles\u2014you\u2019ll be able to instantly search through. If I want to look at some topic, what are the things I\u2019ve dealt with? It will be instantly available to you. That will have a huge impact on learning, people\u2019s professional lives, and any hobby they\u2019re interested in.<\/p>\n

Bell:<\/strong> I downplay all the personal stuff\u2014the movies, the photos\u2014but a family is much more oriented to that. The ability to take photographs and music and videos and make productions of it is really important. We concentrate on the things we\u2019re doing in our work lives, and, in a way, we sometimes neglect the more personal uses. Most families recognize them: better tools for logging your life, easier ways to show a timeline.<\/p>\n

Gemmell:<\/strong> There\u2019s software that exists in research to do automatic summarization of content and automatically come up with stories. More of that software coming out is critical to the momentum of Total Recall. Software will come out to automatically classify stuff. And then there is the ease of use of simple things like backup and replication. It\u2019s too complicated to explain to my parents how to manage their information and keep it safe. It\u2019s got to be a no-brainer.<\/p>\n

Gordon, being our chief guinea pig in this, has gone through a lot of pain and hassle changing batteries, charging devices, and installing them. We\u2019re very excited about Microsoft\u2019s HealthVault, but some medical devices can be a pain. A lot of it is software, but some of it is hardware. As that gets better, as we improve usability and come up with these tools, then we go from the beginning that\u2019s already here to the full fruition of the Total Recall vision.<\/p>\n","protected":false},"excerpt":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research In September, pioneering computer-science researcher Gordon Bell and his Microsoft Research colleague Jim Gemmell published Total Recall: How the E-Memory Revolution Will Change Everything, a book that summarizes nearly a decade of an effort to record digitally everything in Bell\u2019s life: What he did, what he saw, what […]<\/p>\n","protected":false},"author":39507,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[194475,194460],"tags":[214796,214778,214787,214793,214784,197152,214790,214775,214781],"research-area":[13563,13555],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-306413","post","type-post","status-publish","format-standard","hentry","category-database-data-analytics-platforms","category-search-and-information-retrieval","tag-e-memory","tag-healthvault","tag-million-book-project","tag-mylifebits","tag-record-life-experiences","tag-sensecam","tag-telephone-recording-system","tag-total-recall","tag-total-recall-how-the-e-memory-revolution-will-change-everything","msr-research-area-data-platform-analytics","msr-research-area-search-information-retrieval","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[169803,169704],"related-events":[],"related-researchers":[],"msr_type":"Post","byline":"","formattedDate":"October 26, 2009","formattedExcerpt":"By Rob Knies, Managing Editor, Microsoft Research In September, pioneering computer-science researcher Gordon Bell and his Microsoft Research colleague Jim Gemmell published Total Recall: How the E-Memory Revolution Will Change Everything, a book that summarizes nearly a decade of an effort to record digitally everything…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/306413"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/39507"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=306413"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/306413\/revisions"}],"predecessor-version":[{"id":306428,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/306413\/revisions\/306428"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=306413"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=306413"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=306413"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=306413"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=306413"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=306413"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=306413"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=306413"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=306413"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=306413"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=306413"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}