Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers

With the widespread deployment of video analytics applications, edge compute servers are preferred for the analytics of the videos (for reasons of bandwidth and privacy). DNNs are deployed on the edge servers for inference, but when out in the field, they suffer from data drift where the live video data is significantly different than the training data. Continuous learning techniques are designed to handle data drift by periodically retraining the models on new data. Our work addresses the challenge of jointly supporting inference and retraining tasks on edge servers. The fundamental tradeoff that we address in our work is the tradeoff between the retrained model’s accuracy and the inference accuracy, because during retraining, resources are diverted away from inference tasks thereby lowering their accuracy. We design a scheduling solution Ekya that balances this tradeoff, across multiple video streams simultaneously, and optimizes for the inference accuracy over the retraining window’s duration.