skip to main content
research-article
Free Access

The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery.

Published:01 June 2018Publication History
Skip Abstract Section

Abstract

Supervised machine-learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world?

References

  1. Athey, S., Imbens, G. W. 2015 Machine-learning methods https://arxiv.org/abs/1504.01132v1 (see also ref. 7).Google ScholarGoogle Scholar
  2. Caruana, R., Kangarloo, H., Dionisio, J. D, Sinha, U., Johnson, D. 1999. Case-based explanation of non-case- based learning methods. In Proceedings of the American Medical Informatics Association (AMIA) Symposium: 212-215.Google ScholarGoogle Scholar
  3. Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., Elhadad, N. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21st Annual SIGKDD International Conference on Knowledge Discovery and Data Mining, 1721-1730. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Chang, J., Gerrish, S., Wang, C., Boyd-Graber, J. L., Blei, D. M. 2009. Reading tea leaves: how humans interpret topic models. In Proceedings of the 22nd International Conference on Neural Information Processing Systems (NIPS), 288-296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Doshi-Velez, F., Wallace, B., Adams, R. 2015. Graph- sparse lDA: a topic model with structured sparsity. In Proceedings of the 29th Association for the Advancement of Artificial Intelligence (AAAI) Conference, 2575-2581. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. FICO (Fair Isaac Corporation). 2011. Introduction to model builder scorecard; http://www.fico.com/en/latest-thinking/white-papers/introduction-to-model-builder-scorecard.Google ScholarGoogle Scholar
  7. Goodman, B., Flaxman, S. 2016. European Union regulations on algorithmic decision-making and a "right to explanation." https://arxiv.org/abs/1606.08813v3.Google ScholarGoogle Scholar
  8. Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., Baesens, B. 2011. An empirical evaluation of the comprehensibility of decision table, tree- and rule- based predictive models. Journal of Decision Support Systems 51(1), 141-154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Kim, B. 2015. Interactive and interpretable machine- learning models for human-machine collaboration. Ph.D. thesis. Massachusetts Institute of Technology.Google ScholarGoogle Scholar
  10. Kim, B., Rudin, C., Shah, J. A. 2014. The Bayesian case model: A generative approach for case-based reasoning and prototype classification. In Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS), volume 2, 1952-1960. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Kim, B., Glassman, E., Johnson, B., Shah, J. 2015. iBCM: Interactive Bayesian case model empowering humans via intuitive interaction. Massachusetts Institute of Technology, Cambridge, MA.Google ScholarGoogle Scholar
  12. Krening, S., Harrison, B., Feigh, K., Isbell, C., Riedl, M., Thomaz, A. 2017. Learning from explanations using sentiment and advice in RL. IEEE Transactions on Cognitive and Developmental Systems 9(1), 41-55.Google ScholarGoogle ScholarCross RefCross Ref
  13. Lipton, Z. C., Kale, D. C., Wetzel, R. 2016. Modeling missing data in clinical time series with RNNs. In Proceedings of Machine Learning for Healthcare.Google ScholarGoogle Scholar
  14. Liu, C., Rani, P., Sarkar, N. 2006. An empirical study of machine-learning techniques for affect recognition in human-robot interaction. Pattern Analysis and Applications 9(1): 58-69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Lou, Y., Caruana, R., Gehrke, J. 2012. Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 150-158. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Lou, Y., Caruana, R., Gehrke, J., Hooker, G. 2013. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 623-631. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Mahendran, A., Vedaldi, A. 2015. Understanding deep image representations by inverting them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1-9.Google ScholarGoogle ScholarCross RefCross Ref
  18. McAuley, J., Leskovec, J. 2013. Hidden factors and hidden topics: understanding rating dimensions with review text. In Proceedings of the 7th ACM Conference on Recommender Systems, 165-172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., Dean, J. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS), volume 2, 3111?3119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Mordvintsev, A., Olah, C., Tyka, M. 2015. Inceptionism: going deeper into neural networks. Google AI Blog; https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html.Google ScholarGoogle Scholar
  21. Mounk, Y. 2014. Is Harvard unfair to Asian-Americans? New York Times (Nov. 24); http://www.nytimes.com/2014/11/25/opinion/is-harvard-unfair-to-asian-americans.html.Google ScholarGoogle Scholar
  22. Pearl, J. 2009. Causality. Cambridge University Press.Google ScholarGoogle Scholar
  23. Ribeiro, M. T., Singh, S., Guestrin, C. 2016. "Why should I trust you?": explaining the predictions of any classifier. In Proceedings of the 22nd SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135-1144. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Ridgeway, G., Madigan, D., Richardson, T., O'Kane, J. 1998. Interpretable boosted naïve Bayes classification. In Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining: 101-104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Simonyan, K., Vedaldi, A., Zisserman, A. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. https://arxiv.org/abs/1312.6034 (see notes to refs 1, 7).Google ScholarGoogle Scholar
  26. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R. 2013. Intriguing properties of neural networks. https://arxiv.org/abs/1312.6199 (see refs 1, 7, 25).Google ScholarGoogle Scholar
  27. Tibshirani, R. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 58(1), 267-288.Google ScholarGoogle ScholarCross RefCross Ref
  28. Van der Maaten, L., Hinton, G. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 2579-2605.Google ScholarGoogle Scholar
  29. Wang, H.-X., Fratiglioni, L., Frisoni, G. B., Viitanen, M., Winblad, B. 1999. Smoking and the occurrence of Alzheimer's disease: cross-sectional and longitudinal data in a population-based study. American Journal of Epidemiology 149(7), 640-644.Google ScholarGoogle ScholarCross RefCross Ref
  30. Wang, Z., Freitas, N., Lanctot, M. 2016. Dueling network architectures for deep reinforcement learning. Proceedings of the 33rd International Conference on Machine Learning 48, 1995-2003. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery.
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image Queue
      Queue  Volume 16, Issue 3
      Machine Learning
      May-June 2018
      118 pages
      ISSN:1542-7730
      EISSN:1542-7749
      DOI:10.1145/3236386
      Issue’s Table of Contents

      Copyright © 2018 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 1 June 2018

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Popular
      • Editor picked

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format