Location Pairs: A Test Coverage Metric for Shared-Memory Concurrent Programs

We present a coverage metric targeted at shared memory concurrent programs: the Location Pairs (LP) coverage metric. The goals of this metric are (i) to measure how thoroughly a program has been tested from a concurrency standpoint, i.e., whether enough qualitatively different thread interleavings have been explored, and (ii) to guide testing towards unexplored concurrency scenarios. This metric was inspired by an access pattern known to lead to high-level concurrency errors. We built a tool for measuring LP coverage and used the LP metric for interactive debugging. We also compared LP coverage with other concurrency coverage metrics on Java benchmarks. Using mutations and manually-inserted bugs, we demonstrated that LP coverage corresponds better to concurrency errors, is a better measure of how well a program is exercised concurrency-wise by a test set, reaches saturation later than other coverage metrics, and is viable and useful as an interactive testing and debugging tool.