Synopsis of model interpretability (Part 2) — Understanding complex ML models

Based on my colleagues (Patrick Hall, Wen Phan and Sri) at H2O.ai recent article “Ideas on interpreting machine learning”), this is 2nd part of my synopsis.

… continued from part 1 (https://medium.com/@avkash_6547/synopsis-of-model-interpretability-part-1-seeing-your-data-and-applying-ml-69592ec2da16).....

Part 3: Understanding complex machine learning models

• Surrogate models:
• A simple model with globally interpretable attributes, used for explaining complex model, usually created by training a linear regression or decision tree on the original inputs and predictions of a complex model
• Enhance understanding because they provide insight into the internal mechanisms of complex models
• Enhance trust when their coefficients, variable importance, trends, and interactions are in line with human domain knowledge and reasonable expectations of modeled phenomena
• Local Interpretable Model-agnostic Explanation (LIME ):
• A technique for local interpretability helps explaining mainly nonlinear, non-monotonic models along with other kinds
• LIME is a prescribed method for building local surrogate models around single observations
• Requires a set of explainable records be found, simulated, or created and then used to explain machine learning algorithms make decisions for other, less well-understood records.
• Increases trust when the important variables and their linear trends around specific records conform to human domain knowledge and reasonable expectations of modeled phenomena
• These two packages LIME & eli5 implements several handy machine learning interpretability tools
• Maximum activation Analysis (MAA):
• Applies with trees and neural networks where low residuals for a certain tree are analogous to high-magnitude neuron output in a neural network
• It elucidates internal mechanisms of complex models by determining the parts of the response function that specific observations or groups of similar observations excite to the highest degree, either by high-magnitude output from neurons or by low residual output from trees
• It is local in scope because it illustrates how certain observations or groups of observations are treated by discernible aspects of a complex response function increases understanding and enhances understanding because it exposes the internal structures of complex models
• Sensitivity Analysis:
• These are the most important validation technique for machine learning models and investigates whether model behavior and outputs remain stable when data is intentionally perturbed or other changes are simulated in data
• It is global interpretation when a single, global surrogate model to ensure major interactions remain stable when data is lightly and purposely corrupted and it is local when determining if the important variables in a credit allocation decision remain stable for a given customer segment under macroeconomic stress testing
• It enhances understanding because it shows a model’s likely behavior and output in important situations, and how a model’s behavior and output may change over time and enhances trust when a model’s behavior and outputs remain stable when data is subtly and intentionally corrupted
• Variable Importance Measure:
• Typically seen in tree-based models, very model specific, display insights into even the average direction that a variable affects a response function and display the magnitude of a variable’s relationship with the response as compared to other variables used in the model
• For neural networks these are associated with the aggregated, absolute magnitude of model parameters for a given variable of interest
• Leave-One-Covariate Out (LOCO):
• It is a model agnostic idea and creates local interpretations for each row in a training or unlabeled score set by scoring the row of data once and then again for each input variable (e.g., covariate) in the row and in each additional scoring run, one input variable is set to missing, zero, its mean value, or another appropriate value for leaving it out of the prediction
• It is global in scope when creates global variable importance measures by estimating the mean change in accuracy for each variable over an entire data set and can even provide confidence intervals for these global estimates of variable importance and offers local variable importance measures for each row in a training set or in new data
• It increases understanding by showing most influential variables and their relative rank and increases
• Treeinterpreter (Tree interpreter):
• Treeinterpreter is model specific to decision trees and it decomposes decision tree and random forest predictions into bias (overall training data average) and component terms from each independent variable used in the model.
• Derived from the concept of Several average tree “The tree nearest on average to a given set off trees”, outputs a list of the bias and individual contributions for a variable in a given model or the contributions the input variables in a single record make to a single prediction
• It is global in scope when it represents average contributions of independent variables to an overall decision tree or random forest model prediction and it is local in scope when used to explain certain predictions
• It increases understanding by displaying average, ranked contributions of independent variables to the predictions of decision tree and random forest models

Thank you so much for reading! As always your feedback is welcome.