Escapement: A Tool for Interactive Prototyping with Video via Sensor-Mediated Abstraction of Time

We present Escapement, a video prototyping tool that introduces a powerful new concept for prototyping screen-based interfaces by flexibly mapping sensor values to dynamic playback control of videos. This recasts the time dimension of video mock-ups as sensor-mediated interaction.

This abstraction of time as interaction, which we dub video-escapement prototyping, empowers designers to rapidly explore and viscerally experience direct touch or sensor-mediated interactions across one or more device displays. Our system affords cross-device and bidirectional remote (tele-present) experiences via cloud-based state sharing across multiple devices. This makes Escapement especially potent for exploring multi-device, dual-screen, or remote-work interactions for screen-based applications.

We introduce the core concept of sensor-mediated abstraction of time for quickly generating video-based interactive prototypes of screen-based applications, share the results of observations of long-term usage of video-escapement techniques with experienced interaction designers, and articulate design choices for supporting a reflective, iterative, and open-ended creative design process.

Escapement is a prototyping tool that reifies video snippets as sensor-mediated interactive prototypes, for screen-based applications, and across one or more devices. The tool flexibly maps a variety of real-time sensor inputs (such as tilt, motion, or touch) to the time index of a pre-recorded video (or series of still images). This empowers designers to work directly with “time as a design material” in the prototyping process, exploring the feel of an interaction in response to sensor data and corresponding visual feedback.

Parameters & Strategies with Escapement: The tool provides designers access to various parameters of control, which allows for the manipulation of video with time abstracted out. Throughout the years the tool has been in use, we identified prototyping strategies in use by designers as they create video-escapement prototypes.