This special edition of Research Focus highlights some of the 100+ papers from Microsoft Research that were accepted for publication at NeurIPS 2022 – the thirty-sixth annual Conference on Neural Information Processing Systems.
In this issue, we continue to feature some of our 100+ papers accepted at NeurIPS 2022.
Outstanding paper: Gradient Estimation with Discrete Stein Operators
Jiaxin Shi, Yuhao Zhou, Jessica Hwang, Michalis Titsias, Lester Mackey
Gradient estimation — approximating the gradient of an expectation with respect to the parameters of a distribution — is central to the solution of many machine learning problems. However, when the distribution is discrete, most common gradient estimators suffer from excessive variance. To improve the quality of gradient estimation, we introduce a variance reduction technique based on Stein operators for discrete distributions in our paper: Gradient Estimation with Discrete Stein Operators. We then use this technique to build flexible control variates for the REINFORCE leave-one-out estimator. Our control variates can be adapted online to minimize variance and do not require extra evaluations of the target function. In benchmark generative modeling tasks such as training binary variational autoencoders, our gradient estimator achieves substantially lower variance than state-of-the-art estimators with the same number of function evaluations.
Learning Modular Simulations for Homogeneous Systems
Jayesh K. Gupta, Sai Vemprala, Ashish Kapoor
Data-driven simulations have emerged as an effective alternative to building simulators from scratch, which involves complex handcrafting and expert knowledge. However, complex real-life systems are often decomposable into modular subsystems and are usually designed in such a manner for engineering tractability. Unfortunately, current data-driven methods tend to learn monolithic blocks in an end-to-end fashion, which often results in simulators that only work for a particular configuration and are not generalizable. In our paper: Learning Modular Simulations for Homogeneous Systems, we present a modular simulation framework for modeling homogeneous multibody systems, which combines ideas from graph neural networks and neural differential equations. We propose the Message-Passing Neural Ordinary Differential Equation (MP-NODE), a paradigm for modeling individual subsystems as neural ODEs along with spatio-temporal message-passing capability, which is then used to orchestrate full simulations. We evaluate our framework on a variety of systems and show that message passing allows coordination between multiple modules over time for accurate predictions and can also enable zero-shot generalization to new system configurations. Furthermore, we show that our models can be transferred to new system configurations with lower data requirement and training effort, compared to those trained from scratch.
Self-explaining deep models with logic rule reasoning
Seungeon Lee, Xiting Wang, Sungwon Han, Xiaoyuan Yi, Xing Xie, Meeyoung Cha
Deep learning has shown high predictive accuracy in a wide range of tasks, but its inner working mechanisms are obscured by complex model designs. This raises important questions about whether a deep model is ethical, trustworthy, or capable of performing as intended under various conditions.
In our paper, Self-explaining deep models with logic rule reasoning, we present SELOR, a framework for integrating self-explaining capabilities into a given deep model to achieve both high prediction performance and human precision. By “human precision”, we refer to the degree to which humans agree with the reasons models provide for their predictions. Human precision affects user trust and allows users to collaborate closely with the model. We demonstrate that logic rule explanations naturally satisfy human precision with the expressive power required for good predictive performance. We then show how to enable a deep model to predict and explain with logic rules. Our method does not require predefined logic rule sets or human annotations and can be learned efficiently and easily with widely-used deep learning modules in a differentiable way. Extensive experiments show that our method gives explanations closer to human decision logic than other methods while maintaining the performance of deep learning models.
3DB: A Framework for Debugging Computer Vision Models
Guillaume Leclerc, Hadi Salman, Andrew Ilyas, Sai Vemprala, Logan Engstrom, Vibhav Vineet, Kai Xiao, Pengchuan Zhang, Shibani Santurkar, Greg Yang, Ashish Kapoor, Aleksander Madry
Computer vision models such as classification and object detection are known to fail in many different ways. While the importance of recognizing and addressing such shortcomings is well understood, we lack a scalable and efficient means of identifying such failure cases. To address this issue, we introduce 3DB: an extendable, unified framework for testing and debugging vision models using photorealistic simulation. In our paper, 3DB: A Framework for Debugging Computer Vision Models (opens in new tab), we show how 3DB enables users to test vision models on synthetic images, where a diverse set of factors can be controlled. We demonstrate, through a wide range of use cases, that 3DB allows users to discover shortcomings in computer vision systems and gain insights into how models make decisions. 3DB captures and generalizes many robustness analyses from prior work and enables one to study their interplay. Finally, we find that the insights generated by the system transfer to the physical world. We are releasing 3DB as a library alongside a set of example analyses, guides, and documentation: https://3db.github.io/3db/ (opens in new tab).
Artificial intelligence 
Computer vision