modelStudio: Interactive Studio with Explanations for ML Predictive Models

Machine learning predictive models are widely used in many areas of business and research. Their rising popularity is due to them being effective but often leads to problems with explaining their prediction. This has led to development of many Interpretable Machine Learning tools, e.g., DALEX (Biecek, 2018) R package, lime (Ribeiro, Singh, & Guestrin, 2016) and shap (Lundberg & Lee, 2017) Python packages and H2o.ai Driverless AI (Hall, Gill, Kurka, & Phan, 2017).


Introduction
Machine learning predictive models are widely used in many areas of business and research.Their rising popularity is due to them being effective but often leads to problems with explaining their prediction.This has led to development of many Interpretable Machine Learning tools, e.g., DALEX (Biecek, 2018) R package, lime (Ribeiro, Singh, & Guestrin, 2016) and shap (Lundberg & Lee, 2017) Python packages and H2o.ai Driverless AI (Hall, Gill, Kurka, & Phan, 2017).
Nowadays, we can see a huge demand for automation in many areas.This is how Automated Machine Learning and Automated Exploratory Data Analysis came to existence.AutoML (Truong et al., 2019) and AutoEDA (Staniak & Biecek, 2018) tools not only speed up the model development process but also often lead to new discoveries or higher quality of models.
There are tools for automation of the XAI process like modelDown (Romaszko, Tatarynowicz, Urbański, & Biecek, 2019) which produces static HTML site to compare and explain various models.Unfortunately, such tools are focused on global level explanations and deliver monotonous experience.

The modelStudio package
The modelStudio R package automates the process of model exploration.It generates advanced interactive and animated model explanations in the form of a serverless HTML site.It combines R (R Core Team, 2019) with D3.js (Bostock, 2016) to produce plots and descriptions for various local and global explanations.Tools for model exploration unite with tools for EDA to give a broad overview of the model behaviour.
The usage of modelStudio is meant to be intuitive and simple.The computation time needed to produce the output might not be short though.The main goal of this tool is to make model explaining more automated and achieve higher quality explanations by juxtaposition of complementary aspects of a model.
Comparing instance level explanations and model level explanations side by side adds wider context and allows for deeper understanding.modelStudio helps to study relations between various methods for model explanation like Break Down, SHAP, Partial Dependency Plots, Feature Importance, and others.

Key Features
The generated HTML site has many interactive features.One can choose which plots are displayed on the grid and change them at any given moment by clicking the X symbol.A drop down list may be used to pick the observation that will be considered for local explanation plots.One may manipulate plots having a variable-based dimension by selecting corresponding bars on the other plots.Mousing over the D symbol displays a description of the plot.Finally, mousing over lines and bars displays the tooltip.

Explanations
Seven possible plots to choose from are implemented.There are three local explanation plots, three global explanation plots and a feature density plot.
Local explanations are designed to better understand model behaviour around a single observation.
• Ceteris Paribus plot presents model responses around a single point in the feature space (Biecek, 2019).
Global explanations are designed to allow for better understanding of how the model works in general, for some population of interest.
• Partial Dependency plot presents averages from N number of Ceteris Paribus Profiles (Greenwell, 2017).
Detailed overview of these methods can be found in "Predictive Models: Explore, Explain, and Debug" (Biecek & Burzykowski, 2019).The last explanations are implemented in the ingredients (Biecek et al., 2019a) R package.

Conclusions
The modelStudio package is easy to use and its output is intuitive to explore.Automation is convenient and interactivity adds an another dimension to visualisations.All of this enhance explanation of machine learning predictive models.More features and examples can be found in the vignette: modelStudio -perks and features and on GitHub.

Figure 2 :
Figure 2: 1. Open in browser or save as HTML document or PNG image 2. Click on bars to choose which feature will be used for other plots 3. Mouse over the D symbol to display a description of the plot and click X to close the plot 4. Choose which observation will be used for local explanations 5. Mouse over lines and bars to display the tooltip 6. Click on the text to choose the plot 7. Interact with other elements like a slider