Data Wrangling using Programming by Examples

Data Wrangling using Programming by Examples

Sumit Gulwani founded the PROSE research and engineering team at Microsoft that develops programming-by-example (PBE) APIs and ships them through multiple Microsoft products. PBE is a new frontier in AI wherein the computer programs itself---the user provides input-output examples and the computer synthesizes an intended script. This is significant because 99% of computer users do not know programming. Even for programmers, this can provide a 10-100x productivity increase for many task domains. A killer application of PBE is in the space of data cleaning/preparation since data scientists often spend up to 80% time wrangling data into a form suitable for learning models or drawing insights. In this video, Sumit illustrates how a data cleaning task, that Python programmers took an average of 30 minutes to…