Description
Electronic health records are an increasingly important resource for understanding the interactions between patient health, environment, and clinical decisions. In this paper we report an empirical study of predictive modeling of seven patient outcomes using three state-of-the-art machine learning methods. Our primary goal is to validate the models by interpreting the importance of predictors in the final models. Central to interpretation is the use of feature importance scores, which vary depending on the underlying methodology. In order to assess feature importance, we compared univariate statistical tests, information-theoretic measures, permutation testing, and normalized coefficients from multivariate logistic regression models. In general we found poor correlation between methods in their assessment of feature importance, even when their performance is comparable and relatively good. However, permutation tests applied to random forest and gradient boosting models showed the most agreement, and the importance scores matched the clinical interpretation most frequently.
Learning Objective: Intepretation of machine learning models
Measures of feature importance
Authors:
William La Cava (Presenter)
University of Pennsylvania
Christopher Bauer, Biomedical and Translational Infomratics Institute
Jason Moore, University of Pennsylvania
Sarah Pendergrass, Biomedical and Translational Infomratics Institute
Presentation Materials: