Emerging Input Technologies for Always-Available Mobile Interaction

Miniaturizing our computers so we can carry them in our pockets has drastically changed the way we use technology. However, mobile computing is often peripheral to the act of operating in the real world, and the form factor of today’s mobile devices limits their seamless integration into real-world tasks. Interacting with a mobile phone, for example, demands both visual and manual focus. We describe our goal of creating always-available interaction, which allows us to transition between mobile computing and real-world tasks as efficiently as we can shift our visual attention. We assert that this could have the same magnitude of impact that mobile computing had on enabling tasks that were not possible with traditional desktop computers.

In this review, we survey and characterize the properties of sensors and input systems that may enable this shift to always-available computing. Following this, we briefly explore emerging output technologies, both visual and non-visual. We close with a discussion of the challenges that span various technologies, such as ambiguity, sensor fusion, gesture design, and cognitive interference, as well as the opportunities for high-impact research those challenges offer.