Increasingly, business processes require data-driven real-time feedback based on large quantities of log data and customer telemetry from multiple sources. The Logan Project takes a broad approach to understanding the specific needs of consumers of telemetry and log data, focusing on giving them better support for extracting the data they need, cleaning it, and creating queries against it.
To understand the needs of real users, we conduct both broad-based email surveys and in-person interviews to determine the major challenges that data scientists and consumers of telemetry data face. We work closely with internal users to understand requirements that tool-based automation need to support.
Based on these findings, we have implemented a proof-of-concept for visually building queries involving sequences of events, phrased as visual regular expressions. Our most recent research focuses on cohort comparison and data cleaning.
We are also exploring programming by example, with a prototype for filtering data streams and forming nested queries. We are just beginning to scratch the surface of ways to allow both programmers and non-programmers to extract, filter, and query telemetry data sets comfortably.
Learn more about event-oriented sequential analysis at the IEEE Vis 2016 Event Event (opens in new tab).
