MSRGamut

Abstract

Without good models and the right tools to interpret them, data scientists risk making decisions based on hidden biases, spurious correlations, and false generalizations. This has led to a rallying cry for model interpretability. Yet the concept of interpretability remains nebulous, such that researchers and tool designers lack actionable guidelines for how to incorporate interpretability into models and accompanying tools. Through an iterative design process with expert machine learning researchers and practitioners, we designed a visual analytics system, Gamut, to explore how interactive interfaces could better support model interpretation. Using Gamut as a probe, we investigated why and how professional data scientists interpret models, and how interface affordances can support data scientists in answering questions about model interpretability. Our investigation showed that interpretability is not a monolithic concept: data scientists have different reasons to interpret models and tailor explanations for specific audiences, often balancing competing concerns of simplicity and completeness. Participants also asked to use Gamut in their work, highlighting its potential to help data scientists understand their own data.

The GAMUT prototype that was used as the design probe is available here (opens in new tab).

Fred Hohman, the lead author for the paper, published a medium article (opens in new tab) describing this paper.

The basic concept of the application, shown below is to have separate, but linked visualizations of feature importance, global feature contributions, and instance level explanations.

Interacting with Gamut’s multiple coordinated views together.

This work was published in SIGCHI2019 under the following citation:

Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models (opens in new tab)
Fred Hohman, Andrew Head, Rich Caruana, Robert DeLine, Steven Drucker
ACM Conference on Human Factors in Computing Systems (CHI). Glasgow, UK, 2019.

General Additive Modelling software:

The models in the prototype are built by using the open source Interpret-ML library (opens in new tab), or with the PyGam package (opens in new tab).

Dataset Attribution:

The GAMut demonstration includes GAM (General Additive Models) built from several publicly available datasets:

• Ames Iowa Housing Dataset (opens in new tab): All datasets may be freely used in teaching without contacting the author or JSE for permission.
• Datasets from the R data repository: http://vincentarelbundock.github.io/Rdatasets/datasets.html
  • Boston Housing: http://vincentarelbundock.github.io/Rdatasets/doc/MASS/Boston.htmlHarrison, D. and Rubinfeld, D.L. (1978) Hedonic prices and the demand for clean air. J. Environ. Economics and Management 5, 81–102.
    Belsley D.A., Kuh, E. and Welsch, R.E. (1980) Regression Diagnostics. Identifying Influential Data and Sources of Collinearity. New York: Wiley.
  • PIMA: Diabetes in Pima Indian Women: (http://vincentarelbundock.github.io/Rdatasets/doc/MASS/Pima.tr.html)Smith, J. W., Everhart, J. E., Dickson, W. C., Knowler, W. C. and Johannes, R. S. (1988) Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the Symposium on Computer Applications in Medical Care (Washington, 1988), ed. R. A. Greenes, pp. 261–265. Los Alamitos, CA: IEEE Computer Society Press.
    Ripley, B.D. (1996) Pattern Recognition and Neural Networks. Cambridge: Cambridge University Press.
  • Diamonds: prices of 50000 round cut diamonds: http://vincentarelbundock.github.io/Rdatasets/doc/ggplot2/diamonds.html
    Diamond data obtained from AwesomeGems.com on July 28, 2005.
• Datasets from UCI Machine Learning Repository: https://archive.ics.uci.edu/ml/index.php (Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.)
  • Income: https://archive.ics.uci.edu/ml/datasets/Census+Income
  • Heart-disease: https://archive.ics.uci.edu/ml/datasets/Heart+Disease
    1. Hungarian Institute of Cardiology. Budapest: Andras Janosi, M.D.
    2. University Hospital, Zurich, Switzerland: William Steinbrunn, M.D.
    3. University Hospital, Basel, Switzerland: Matthias Pfisterer, M.D.
    4. V.A. Medical Center, Long Beach and Cleveland Clinic Foundation: Robert Detrano, M.D., Ph.D.
  • Red Wine Quality: https://archive.ics.uci.edu/ml/datasets/wine+quality
    Paulo Cortez, University of Minho, Guimarães, Portugal, http://www3.dsi.uminho.pt/pcortez (opens in new tab)
    A. Cerdeira, F. Almeida, T. Matos and J. Reis, Viticulture Commission of the Vinho Verde Region(CVRVV), Porto, Portugal
    @2009
  • Yacht Hydrodynamics: http://archive.ics.uci.edu/ml/datasets/yacht+hydrodynamics
    Ship Hydromechanics Laboratory, Maritime and Transport Technology Department, Technical University of Delft.
• Titanic Dataset: Paul Hendricks (2015). titanic: Titanic Passenger Survival Data Set. R package version 0.1.0.