{"id":307442,"date":"2008-09-04T01:00:57","date_gmt":"2008-09-04T08:00:57","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=307442"},"modified":"2016-10-18T20:17:31","modified_gmt":"2016-10-19T03:17:31","slug":"autocollage-summarize-adventures-click","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/autocollage-summarize-adventures-click\/","title":{"rendered":"AutoCollage: Summarize Your Adventures with a Click"},"content":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research<\/em><\/p>\n

You\u2019ve got photos\u2014lots and lots of photos. Everybody does these days, thanks to the digital-photography revolution. Hundreds, thousands, a veritable treasure trove of image-based memories.<\/p>\n

Sometimes, though, you don\u2019t need thousands of photos. Sometimes, you just need that one representative shot to convey the fun you had during your day at the beach. But which one? The kids frolicking in their new swimsuits? The adults soaking in some rays? Your family\u2019s nonpareil sandcastle? To tell the story adequately, no single one will suffice. You need them all combined into a defining composite.<\/p>\n

Microsoft Research Cambridge<\/a> is at your service.<\/p>\n

AutoCollage<\/a>, an easy, novel framework for the automatic creation of representative collages from collections of photos, became available to the general public on Sept. 4. Utilizing a collection of sophisticated technological techniques, AutoCollage is simple to use, produces attractive imagery, and, perhaps most important, is a whole lot of fun.<\/p>\n

A free, 30-day trial version of the software is available worldwide<\/a>, and a full, unrestricted version can be purchased<\/a> in the United States and in European Union nations. AutoCollage is one of the first Microsoft Research products to be made available to consumers.<\/p>\n

\"This

This is the sort of visual summary provided by AutoCollage, a new product from Microsoft Research Cambridge.<\/p><\/div>\n

It works like this: AutoCollage\u2014which works with either Windows Vista<\/a> or Windows XP Service Pack 2 and above\u2014cuts out interesting parts of photos and combines them together, following natural features as boundaries between images. The selected pieces are sized similarly and assembled into a pleasing whole.<\/p>\n

\u201cThe most significant feature that differentiates AutoCollage is that it offers exceptionally sophisticated blending technology for photographs, powered by state-of-the-art computer-vision techniques,\u201d says Alisson Sol, development manager for the Incubation and Tech Transfer team at the England lab. \u201cIt\u2019s great that we can give everyone the opportunity to play with and use this compelling technology, and we\u2019re looking forward to seeing what collages they come up with.\u201d<\/p>\n

The application is a direct result of months of incubation efforts at Microsoft Research Cambridge.<\/p>\n

\u201cWhile the majority of the work undertaken at Microsoft Research is longer-term, pure research,\u201d says Mitch Goldberg, director of the Incubation, and Tech Transfer team, \u201ccompelling innovations are also brought to market through a mix of technology transfer into Microsoft products, licensing our technology, and creating new ventures.\u00a0 Cambridge Incubation is proud to\u00a0make AutoCollage available worldwide to trial as a Microsoft Research download and a full version available for purchase in the United Kingdom and the United States via the online Microsoft store.\u201d<\/p>\n

While AutoCollage might be simple and intuitive to use, behind the scenes, the most advanced technologies extant are doing all the heavy lifting.<\/p>\n

The most sophisticated tool of its kind available to consumers, the software combines object recognition, face detection, image blending, and other computer-vision and -graphics techniques to provide a seamless summary of the most interesting images within a group of photos.<\/p>\n

Just ask Carsten Rother. A researcher in the Machine Learning and Perception<\/a> group within Microsoft Research Cambridge, Rother expanded an earlier research project called Tapestry to the point that the Cambridge Incubation team became intrigued with the possibilities.<\/p>\n

\u201cPeople have a lot of images,\u201d Rother says, \u201cand the first goal was to ask, \u2018Can we create a representation of these images as compact as possible?\u2019 \u201d<\/p>\n

The answer, as you might have surmised, is a resounding yes. A pair of papers outlines the evolution of the project. Digital Tapestry<\/em><\/a>, written by Rother, Vladimir Kolmogorov, and Andrew Blake of Microsoft Research Cambridge, in conjunction with Sanjiv Kumar of Carnegie Mellon University, was presented during the Institute of Electrical and Electronics Engineers\u2019 Computer Vision and Pattern Recognition conference in 2005. A second, entitled AutoCollage<\/em><\/a>, written by Rother and Microsoft Research Cambridge colleagues Lucas Bordeaux<\/a>, Youssef Hamadi, and Blake, was featured in 2006 during the Association for Computing Machinery\u2019s annual conference on Computer Graphics and Interactive Techniques.<\/p>\n

\u201cWe\u2019ve tested tens of thousands of different collages in the course of our research,\u201d Rother says, \u201cand it\u2019s really exciting that the positive feedback we\u2019ve received from our user studies shows we\u2019ve answered these challenges successfully.\u201d<\/p>\n

One of AutoCollage\u2019s most impressive features is its ease of use. You simply point to a folder containing a collection of photos, click a button, and the system creates a collage, using the most representative images within the collection and artfully placing, via computer-vision techniques, the most interesting portions of those photos into a rectangular format.<\/p>\n

Once the collage is complete\u2014usually taking only a few seconds\u2014the resultant composite can be printed, e-mailed, or set as the background of a PC desktop. Many of those who have used the software find that one of the most useful ways to employ AutoCollage is to place a collage at the front of a collection of images, such as a photo album, thereby summarizing the contents to come.<\/p>\n

The program\u2019s creators suggest a range of 7-30 photos for optimal performance, with a default set at 12. A slider enables the user to adjust according to the scope of the collection. Set your desired number of images, press the button, and voil\u00e0. Wait till Mom sees this!<\/p>\n

In actuality, although the process might seem simple, there are significant technological achievements that combine to produce the final collage.<\/p>\n

\u201cYou have one big objective function that you want to optimize,\u201d he says. \u201cIf you have a lot of input images, what should be in there? The most important images should be in there. The images should be as different as possible.<\/p>\n

\u201cFrom each image, you take the most interesting part, what we call the \u2018region of interest.\u2019 Then, these images should be arranged in a nice way.\u201d<\/p>\n

\"AutoCollage

AutoCollage uses intelligent ordering to place the most interesting photos in a collection in the most pleasing configuration.<\/p><\/div>\n

AutoCollage also makes use of face-detection technology and clever rules of thumb born of experimentation, such as placing images that include sky at the top of a collage, where the sky isn\u2019t as jarring as if it appeared in the middle of the collage.<\/p>\n

Rother cites five steps the software uses to produce a collage.<\/p>\n

\u201cThe first task,\u201d he says, \u201cis to rank all the images, where the top-ranked image is the most likely to end up in the collage and the last one is the least interesting. If there are a lot of faces in a group shot, it\u2019s more likely to be in. If there are two duplicate images, then only one should be in.\u201d<\/p>\n

Next is an analysis of the top-ranked images, the detection of regions of interest.<\/p>\n

\u201cWhat is the prominent area?\u201d Rother asks. \u201cWe know that a face is likely to be interesting for the user to have in the collage. It\u2019s less important to have a lot of sky regions.<\/p>\n

\u201cThere are a lot of other internal features, like image contrast, that decide what is an interesting region. There has been a lot of research on that, and we exploit that body of knowledge.\u201d<\/p>\n

Then the most visually appealing portions of the top-ranked images are combined so the regions of interest don\u2019t overlap, a process called packing.<\/p>\n

\u201cAfter the packing,\u201d Rother continues, \u201cwe do a cut where we segment from each image the exact, not rectangular region. We use code used in other projects, such as GrabCut<\/a>, for image segmentation.\u201d<\/p>\n

\u201cThe segmentation has two objectives,\u201d he explains. \u201cOne is that it should not be a tiny fraction of the image, because you want to have each image to be as balanced as possible, each image being a relatively big portion. And you prefer sharper boundaries to high-contrast transitions, because it\u2019s very likely that those indicate a true object.<\/p>\n

\u201cWe don\u2019t have any object recognition right now in the system. We have it in the sense of face detection and sky detection, but we don\u2019t do generic object recognition to say: \u2018That\u2019s a car. That\u2019s a road.\u2019 But edges are likely candidates for these transitions. If there is an edge, we take it in the segmentation.\u201d<\/p>\n

That leaves one final step, the blending of the images to produce the final collage. This uses an existing technique called Poisson blending.<\/p>\n

As a whole it\u2019s an integrated procedure. All these things are invisible to the user, who simply waits a moment for the final result. AutoCollage is not computationally expensive, so if, by chance, the first collage is not satisfactory, you simply tweak the number of photos or the collection of photos and try again. If assistance is needed, users can consult a comprehensive help system included with the tool or get support in the Microsoft Research products forum.<\/p>\n

\u201cSeeing an AutoCollage of your own photos,\u201d said one participant in a user study, \u201cis a surprisingly emotive experience.\u201d<\/p>\n

The AutoCollage application, driven by the Microsoft Research Cambridge Incubation team, is a result of worldwide collaboration. Although much of the work was performed at Microsoft Research Cambridge\u2014with the Computer Vision, Incubation and Tech Transfer, Computer-Mediated Living, and Constraint Reasoning groups at that lab all making contributions\u2014Microsoft Research associates in Redmond<\/a> and Beijing<\/a> also played key roles.<\/p>\n

Andrew Herbert, managing director of Microsoft Research Cambridge, is delighted with the results.<\/p>\n

\u201cAutoCollage is a great example of some very innovative computer-science research from our Cambridge facility and partner labs in the United States and China,\u201d Herbert says. \u201cWe are furthering our commitment to transfer scientific innovations by providing consumers with access to technologies that are on the cutting edge of computer science.\u201d<\/p>\n

For Sol, the novel results provided by AutoCollage are validation enough.<\/p>\n

\u201cThis is the first application that I can explain to my Grandma,\u201d he smiles. \u201cShe can really use it and quickly produce an AutoCollage, click e-mail, and send it to us.<\/p>\n

\u201cThis is what I\u2019m really proud of: It is a small application. It doesn\u2019t fulfill a business need. It doesn\u2019t enhance your productivity. It enhances how happy you are with using computers.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research You\u2019ve got photos\u2014lots and lots of photos. Everybody does these days, thanks to the digital-photography revolution. Hundreds, thousands, a veritable treasure trove of image-based memories. Sometimes, though, you don\u2019t need thousands of photos. Sometimes, you just need that one representative shot to convey the fun you had during […]<\/p>\n","protected":false},"author":39507,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[194471,194480,194455],"tags":[215534,215540,215537,215543],"research-area":[13556,13562,13551],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-307442","post","type-post","status-publish","format-standard","hentry","category-computer-vision","category-graphics-and-multimedia","category-machine-learning","tag-autocollage","tag-digital-tapestry","tag-digital-photography","tag-grabcut","msr-research-area-artificial-intelligence","msr-research-area-computer-vision","msr-research-area-graphics-and-multimedia","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199560,199561,199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-events":[],"related-researchers":[],"msr_type":"Post","byline":"","formattedDate":"September 4, 2008","formattedExcerpt":"By Rob Knies, Managing Editor, Microsoft Research You\u2019ve got photos\u2014lots and lots of photos. Everybody does these days, thanks to the digital-photography revolution. Hundreds, thousands, a veritable treasure trove of image-based memories. Sometimes, though, you don\u2019t need thousands of photos. Sometimes, you just need that…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/307442"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/39507"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=307442"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/307442\/revisions"}],"predecessor-version":[{"id":308600,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/307442\/revisions\/308600"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=307442"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=307442"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=307442"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=307442"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=307442"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=307442"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=307442"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=307442"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=307442"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=307442"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=307442"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}