{"id":212095,"date":"2016-01-25T19:33:26","date_gmt":"2016-01-25T19:33:26","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/project\/photo-story\/"},"modified":"2017-10-07T21:38:02","modified_gmt":"2017-10-08T04:38:02","slug":"photo-story","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/photo-story\/","title":{"rendered":"Photo Story"},"content":{"rendered":"<p>The capability of managing personal photos is becoming crucial. In this work, we have attempted to solve the following pain points for mobile users: 1) intelligent photo tagging, best photo selection, event segmentation and album naming, 2) speech recognition and user intent parsing of time, location, people attributes and objects, 3) search by arbitrary queries.<\/p>\n<div class=\"conM \">\n<p align=\"justify\">We first segment and categorize the unstructured photo streams into multiple semantic-related albums in an automatic way. Second, we analyze the photo content by image tagging techniques, and further suggest the\u00a0important time, location, semantic words in user albums with visual samples\u00a0according to\u00a0the photo distribution for each user.<\/p>\n<p align=\"justify\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-233119\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_tagging-1024x405.jpg\" alt=\"photostory_tagging\" width=\"1024\" height=\"405\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_tagging-1024x405.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_tagging-300x119.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_tagging-768x304.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_tagging.jpg 1494w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><br \/>\nThird, we learn a retrieval model that can simultaneously consider the relationship between the user input and the pre-defined vocabulary set by concept mapping, and the relationship between the vocabulary set and the photo visual content by deep learning techniques. We also consider the photo quality assessment to rank the images in the retrieval result. The whole system is shown as follows:<\/p>\n<p align=\"justify\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-233121\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_flow-chart-1024x253.jpg\" alt=\"photostory_flow-chart\" width=\"1024\" height=\"253\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_flow-chart-1024x253.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_flow-chart-300x74.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_flow-chart-768x189.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/01\/photostory_flow-chart.jpg 1541w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>The capability of managing personal photos is becoming crucial. In this work, we have attempted to solve the following pain points for mobile users: 1) intelligent photo tagging, best photo selection, event segmentation and album naming, 2) speech recognition and user intent parsing of time, location, people attributes and objects, 3) search by arbitrary queries. [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13562,13551,13555],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-212095","msr-project","type-msr-project","status-publish","hentry","msr-research-area-computer-vision","msr-research-area-graphics-and-multimedia","msr-research-area-search-information-retrieval","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"1\/25\/2016","related-publications":[272535,272583,272598,283529,310025,331619,215266,271170,441843,271377,441861,272517,442173,272523],"related-downloads":[],"related-videos":[],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Jianlong Fu","user_id":32260,"people_section":"Group 1","alias":"jianf"}],"msr_research_lab":[199560],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/212095"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/212095\/revisions"}],"predecessor-version":[{"id":388391,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/212095\/revisions\/388391"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=212095"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=212095"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=212095"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=212095"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=212095"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}