Pen + Touch Interaction

Microsoft Research has long pioneered new techniques in digital pen input, including particularly its combination with multi-touch in a manner that reflects how people naturally use their hands.

For example, in the real world people hold (“touch”) a document with their non-preferred hand, and often frame a particular region of interest between thumb and forefinger. The pen (held, of course, in the preferred hand) then marks up the page.

Interfaces with such “Pen + Touch” capabilities are now deeply integrated with Windows and distinguish Microsoft’s innovative line of Surface products, including the Surface Pro tablet, Surface Studio drafting table, and the Surface Hub electronic whiteboard.

This example shows a common natural behavior while interacting with paper documents or notebooks in the real world. Here one hand ‘frames’ the portion of the page the individual is working on, while the other hand marks in reference to this area with the pen.

Another example is orienting a page with one hand while manipulating the writing implement with the other, as in this example of “crayon and touch” technology from everyday life.

Our work stems from a desire to understand and leverage the full capabilities of both hands.

…While simultaneously leveraging the mutual affordances of both Pen and Touch as input modalities.

This page collects a number of the people and publications that contribute to this important and still-emerging aspect of ‘modern’ interaction with devices.