Collaborative Acceleration for Mixed Reality

A new generation of augmented reality (AR) devices, such as
the Microsoft HoloLens, promises a user experience known
as mixed reality (MR) that is more seamless, immersive, and
intelligent than earlier AR technologies. However, this new
experience comes with high computational costs, including
exceptionally low latency and high quality requirements.
While this cost could be offset in part through offloading,
we also observe an increasing availability of on-device, task-
specific accelerators. In this paper, we propose collaborative
acceleration, a collaborative technique that utilizes the unique
hardware accelerated capabilities of an MR device, in con-
junction with an edge node, to partition an application’s
core workflow according to the specific strengths of each
device. To better understand the workloads of next gener-
ation MR applications, we implement a concrete MR app
on the HoloLens: an assistive tool to visually aid users in
manipulating physical objects. Through our prototype, we
find that offloading a subset of the app’s workload to an edge
while also leveraging the strengths of the HoloLens delivers
accurate enough results at a low latency. Our work provides
an early glimpse into the system design challenges of MR,
potentially the first “killer application” of edge offloading.