XAI: exPlainable Artificial Intelligence
By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH
Table of Contents
1. Black Box AI¶
- Black box AI: AI produces insights based on a data set, but the end-user doesn’t know how
- Many machine learning and deep learning models share ‘black box’ problem
- AI does not provide reasons behind the decision or prediction it makes
- The reliability of AI models may be questioned
- XAI which humans can understand the decisions or predictions made by the AI
Why XAI?
- XAI can be used to increase the interpretability of AI by enabling description of the expected outcome and potential bias of the model
- Depending on the AI performance, XAI results can be used in various ways:
- AI performance < Human performance
- XAI suggests improvement directions for AI models
- AI performance ≈ Human performance
- XAI identifies the principles behind AI model learning
- AI performance > Human performance
- XAI enables acquiring new knowledge from AI
- AI performance < Human performance
Model-Specific XAI
- Model-Specific XAI: only applicable to specific algorithms that provides explanations by using the intrinsic structure of the model
- Examples: Class Activation Mapping (CAM) & Gradient-CAM for Convolution Neural Network (CNN) models
Model-Agnostic XAI
- Model Agnostic XAI: applicable to any machine learning algorithms and work on the black box model
- Obtain explanations by perturbing and mutating the input data and obtaining sensitivity of the performance of theses mutations with respect to the original data performance
- Examples:
- SHapley Additive exPlanations (SHAP)
- Local Interpretable Model-agnostic Explanations (LIME)
2. SHapley Additive exPlanations (SHAP)¶
- SHAP: a game theoretic approach to explain the output of any machine learning model
- Compute feature importance on each predicted value using Shapley value
- According to the SHAP value, the contribution of each feature can be expressed as the degree of change in the overall performance when the contribution of that feature is excluded
- Unlike general permutation method, SHAP calculates the model influence by considering the dependencies between features
3. Local Interpretable Model-agnostic Explanations (LIME)¶
- LIME: which feature of the current data the model focused on and which features were used as the basis for prediction
- LIME partially masks the input data
- Then, the transformed input data is given to the model and the predicted value is obtained
- If the prediction result changed a lot, we can know that the masked part was important
- On the other hand, if the prediction result did not change much, we can know that the masked part was not very important
In [1]:
!pip install shap lime
In [2]:
from google.colab import drive
drive.mount('/content/drive')
Import Libraries
In [ ]:
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_squared_error
from sklearn.metrics import r2_score
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
Define Input Data
In [ ]:
df = pd.read_csv("/content/drive/MyDrive/kstp/data_files/datafile.csv")
normalized_df = (df-df.min())/(df.max()-df.min())
df = normalized_df
In [ ]:
df
In [ ]:
y = df['Weight'].values
In [ ]:
X = df.drop(["Weight"], axis = 1)
In [ ]:
print("There are {} possible descriptors:".format(len(X.columns)))
print(X.columns)
Split Train-Test Set and Train Random Forest Regressor
In [ ]:
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.30, random_state = 1)
rf = RandomForestRegressor(random_state = 1)
rf.fit(X_train, y_train)
Evaluate the Random Forest Regressor with Test Set
In [ ]:
y_pred = rf.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
print('RMSE = {:.3f} '.format(np.sqrt(mse)))
In [ ]:
r2 = r2_score(y_test, y_pred)
print('R2 = {:.3f} '.format(r2))
In [ ]:
fig, ax = plt.subplots(figsize=(10, 10))
ax.scatter(y_test, y_pred, edgecolors=(0, 0, 0))
ax.plot([y.min(), y.max()], [y.min(), y.max()], "r--", lw=4)
ax.set_xlabel("Measured")
ax.set_ylabel("Predicted")
plt.show()
4.2. SHAP¶
SHAP Implementation
In [ ]:
import shap
explainer = shap.TreeExplainer(rf)
shap_values = explainer.shap_values(X_test)
SHAP Value of a Test Data (Local Feature Importance)
In [ ]:
shap.bar_plot(shap_values[0],features = X_test.iloc[0,:] ,feature_names = X.columns )
Average of Absolute SHAP Values of Entire Test Data (Global Feature Importance)
In [ ]:
shap.summary_plot(shap_values, X_test, plot_type = "bar")
4.3. LIME¶
LIME Implementation
In [ ]:
from lime import lime_tabular
X_train = X_train.to_numpy()
X_test = X_test.to_numpy()
explainer = lime_tabular.LimeTabularExplainer(X_train,
mode = "regression",
feature_names = X.columns)
explainer
LIME Result
In [ ]:
explanation = explainer.explain_instance(X_test[0], rf.predict, num_features = len(X.columns))
explanation
explanation.show_in_notebook()
In [ ]:
with plt.style.context("ggplot"):
explanation.as_pyplot_figure()
Import Libraries
In [3]:
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.metrics import r2_score
import matplotlib.pyplot as plt
import tensorflow as tf
import pandas as pd
import numpy as np
Define Input Data
In [4]:
df = pd.read_csv("/content/drive/MyDrive/kstp/data_files/datafile.csv")
normalized_df = (df - df.min())/(df.max() - df.min())
df = normalized_df
In [5]:
df
Out[5]:
In [6]:
y = df['Weight'].values
In [7]:
X = df.drop(["Weight"], axis = 1)
In [8]:
print("There are {} possible descriptors:".format(len(X.columns)))
print(X.columns)
Split Train-Test Set
In [9]:
X_train, X_test, Y_train, Y_test = train_test_split(X, y, test_size = 0.30, random_state = 10)
In [10]:
X_train.shape
Out[10]:
Define ANN Model
In [11]:
model = tf.keras.models.Sequential([
tf.keras.Input(shape = (X_train.shape[1],)),
tf.keras.layers.Dense(1024, activation = 'relu'),
tf.keras.layers.Dense(128, activation = 'relu'),
tf.keras.layers.Dense(1)
])
In [12]:
model.summary()
In [13]:
model.compile(loss = tf.keras.losses.MeanSquaredError(),
optimizer = tf.keras.optimizers.Adam())
Train DNN Model
In [14]:
model.fit(X_train,
Y_train,
epochs = 150,
batch_size = 64,
validation_split = 0.1)
Out[14]:
Evaluate ANN Model
In [15]:
y_pred = model.predict(X_test)
mse = mean_squared_error(Y_test, y_pred)
print('RMSE = {:.3f} '.format(np.sqrt(mse)))
In [16]:
r2 = r2_score(Y_test, y_pred)
print('R2 = {:.3f} '.format(r2))
In [17]:
fig, ax = plt.subplots(figsize = (10, 10))
ax.scatter(Y_test, y_pred, edgecolors = (0, 0, 0))
ax.plot([y.min(), y.max()], [y.min(), y.max()], "r--", lw = 4)
ax.set_xlabel("Measured")
ax.set_ylabel("Predicted")
plt.show()
5.2. SHAP¶
SHAP Implementation
In [18]:
import shap
X_train = X_train.to_numpy()
X_test = X_test.to_numpy()
explainer_shap = shap.DeepExplainer(model = model, data = X_train)
shap_values = explainer_shap.shap_values(X_test)
SHAP Value of a Test Data (Local Feature Importance)
In [19]:
shap.bar_plot(shap_values[0][1], features = X_test[1], feature_names = X.columns )
Average of Absolute SHAP Values of Entire Test Data (Global Feature Importance)
In [20]:
shap.summary_plot(shap_values, X_test, feature_names = X.columns)
5.3. LIME¶
LIME Implementation
In [21]:
from lime import lime_tabular
explainer = lime_tabular.LimeTabularExplainer(X_train,
mode = "regression",
feature_names = X.columns)
explainer
Out[21]:
In [22]:
print("Prediction : ", model.predict(X_test[1].reshape(1,-1)))
print("Actual : ", [[Y_test[1]]])
explanation = explainer.explain_instance(X_test[1], model.predict, num_features = len(X.columns))
explanation
Out[22]:
LIME Result
In [23]:
explanation.show_in_notebook()
In [24]:
with plt.style.context("ggplot"):
explanation.as_pyplot_figure()
In [1]:
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')