{"id":582631,"date":"2019-05-02T10:03:25","date_gmt":"2019-05-02T17:03:25","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=582631"},"modified":"2019-05-31T16:31:25","modified_gmt":"2019-05-31T23:31:25","slug":"thinking-with-data-tapping-into-the-potential-of-the-digital-pen","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/thinking-with-data-tapping-into-the-potential-of-the-digital-pen\/","title":{"rendered":"(Th)Inking with data \u2014 tapping into the potential of the digital pen"},"content":{"rendered":"
\"ActiveInk

(opens in new tab)<\/span><\/a> ActiveInk aims to capture the freedom of a traditional pen, making consuming information and externalizing thoughts about it a cohesive digital experience. The system\u2019s active ink turns pen strokes into a means of executing an intended action, such as highlighting, hiding, or cutting specific information.<\/p><\/div>\n

Whether at work or at home, people are regularly gathering and interpreting information to build their base of knowledge and gain a deeper understanding of the world around them. In this endeavor, they encounter data in many shapes and forms\u2014lines of text in books, timelines and charts in magazines and newspapers, photos in print and online. They\u2019re searching webpages and browsing interactive maps to identify good schools for their children or to find a great house to purchase; they\u2019re monitoring their medical and financial data to make informed decisions in their life.<\/p>\n

When making sense of information on paper, many people engage in active reading, crossing out parts of data visualizations that are irrelevant to their current analysis, identifying outliers in a scatterplot, or marking passages of interest. They may also jot down their hypotheses and interpretations directly on top of the data, such as drawing a correlation line. As people conduct more of their lives electronically, it’s natural to offer them a way to engage in these active reading activities on their computers with a digital pen.<\/p>\n

But it\u2019s not enough to try to replicate the pen-and-paper experience. In fact, it might not be possible, as even with hardware and software advances in the past decade, writing on a piece of glass doesn\u2019t feel the same as writing on paper. The digital experience doesn\u2019t match the sensory aspect\u2014the tactile, visual, and auditory feedback\u2014of the analog experience. Affordability and reliability are also among the many reasons why people might choose pen and paper over the digital option (opens in new tab)<\/span><\/a>. Instead of seeking to emulate the analog experience, our aspiration is to uncover the unique capabilities of the digital pen. Our research aims to give the digital pen superpowers<\/em> for marking documents in ways that go beyond what is possible with physical pen and paper.<\/p>\n

The power of active<\/em> ink<\/h3>\n

With the web-based system ActiveInk (opens in new tab)<\/span><\/a>, we\u2019re enabling users to seamlessly transition between exploring data on screen and externalizing their thoughts on screen using pen and touch. ActiveInk, inspired by such previous work as InkSeine (opens in new tab)<\/span><\/a>, allows the natural use of pen for active reading behaviors while supporting analytic actions on underlying data by activating any of the ink strokes laid on the screen. So marks serve not only as reminders of an action to be applied later\u2014whether that be extract, remove, enhance, or ignore that particular information\u2014they become the means to accomplish the action. It\u2019s active<\/em> ink. More than a set of strokes, it facilitates an interaction between users\u2019 thoughts and the underlying content.<\/p>\n

Let\u2019s say, for example, you\u2019re working with a map of the United States to identify those regions of the country where you might want to live and work. You decide to focus on only the East and West coasts. You can cross out or scribble across the Midwestern states\u2014your indication that the information is unnecessary\u2014and use the same markup to initiate the desired action, which in this case is to delete those states.<\/p>\n

With one pen, users can mark things freely in a variety of colors and stroke styles and execute or fix them later. Activating these annotations by tapping on them reveals a set of commands, such as highlight, remove, and label, and analytical functions, such as compute a regression. The digital pen can also be used as a precision input to cut off outliers in cluttered visual.<\/p>\n

The driving force<\/h3>\n

Switching between digesting data on screen and externalizing resulting thoughts and analysis with pen and paper has physical and cognitive costs, including time expended writing down enough notes to remember or retrieve later the chart or table you need, for example. At the heart of ActiveInk, which we\u2019re presenting at the ACM CHI Conference on Human Factors in Computing Systems (CHI) (opens in new tab)<\/span><\/a>, is three principles to minimize those costs:<\/p>\n

    \n
  1. Provide space to think<\/strong>. Sensemaking is not linear: People need to cross-reference different pieces of information from different files. They may need to go back to previously seen charts, look for new ones, or skim through a long text to identify a missing piece of information. Thinking often requires capturing fleeting thoughts and revisiting them multiple times before finalizing an analysis. ActiveInk provides users with an infinite canvas to do that. They have free reign to drop information in and reorganize it at will. Marks made to an individual piece of data stay with it as users move around to available parts of the canvas, never running out of room to work.<\/li>\n
  2. Empower the digital pen to both interact with data and allow for externalizing thoughts<\/strong>. Two interface strategies were developed and evaluated to operate on the infinite canvas:\n
      \n
    • In the prefix <\/strong><\/em>method<\/strong>, users select an action to be taken from a menu of options either with a finger on their non-dominant hand or the pen in their dominant hand, and the selected action is applied as the pen is used.<\/li>\n
    • In the postfix <\/strong><\/em>method<\/strong>, everything done with the digital pen results in a pen stroke that can be activated after the fact, as described above; the marks that result in the action remain as a means to recall or undo the action.<\/li>\n<\/ul>\n<\/li>\n
    • Avoid requiring memorization of gestures<\/strong>. The ways people annotate information is different, so instead of asking users to change the way they work, ActiveInk supports how they naturally use a pen.<\/li>\n<\/ol>\n

      <\/p>\n