Visual Diagnostics for a Model Explainer - Tools for the Assessment of LIME Explanations

.title[
# Visual Diagnostics for a Model Explainer - Tools for the Assessment of LIME Explanations
]
.author[
### Katherine Goode
]
.date[
### December 3, 2019
]

---

# Overview

1. Motivational Dataset

2. LIME

3. Motivation for Assessing LIME

4. Diagnostic Plots

5. Discussion and Current Work

---

# Motivational Dataset

---

# Bullet Matching

---

# Hamby Bullet Study Hamby et. al. (2009)

- Bullets from “known” and “unknown” gun barrels
- Sent to firearm examiners around the world
- Examiners asked to use the known bullets to identify which barrels the unknown bullets were fired from

---

# CSAFE and Hamby Bullets

- Center for Statistics and Applications in Forensic Evidence
- Has access to Hamby bullets
- Took high definition scans

---

# Automated Bullet Matching Algorithm

- Hare, Hofmann, and Carriquiry (2017): 
 - Extracted signatures from scans
 - Developed variables that measure similarity between signatures
 - Fit a random forest to automate bullet matching
 - 100% bullet matching accuracy with test set in paper (0.0039 land to land error)
 
.center[<img src="./figures/signatures.png" width=650>]

---

# Model Explanations

Random forest model:

- High accuracy - great!

- Interpretability - not so great

Importance of explanations:

- Understanding the model could help improve it

- Firearm examiners can assess model prediction

- Important to explain predictions to a jury

---

# Global Explanations

![](slides_files/figure-html/unnamed-chunk-1-1.png)

---

# Local Explanations

- Feature importance may vary on a local level

- What if we are interested a particular prediction?

---

# LIME

---

# Enter LIME

- Model explainer developed by computer scientists Ribeiro, Singh, and Guestrin (2016)

- Designed to assess if a black box predictive model is trustworthy

- Produces "explanations" for individual predictions

- Meaning:
  - **L**ocal 
  - **I**nterpretable
  - **M**odel-Agnostic 
  - **E**xplanations

---

# LIME Concept

---

# LIME Procedure

For **one** prediction of interest...

1. Data Simulation and Interpretable Transformation
  - Simulate data from the original data 
  - Apply a transformation that will allow for easily interpretable explanations
  
2. Fit an Interpretable Model:
  - Response = Black-box predictions from simulated data
  - Predictors = transformed simulated data
  - Weights = distances between simulated values and prediction of interest
  
3. Interpret the Explainer

---

# Interpretable Transformation

---

# LIME Explanations

Bullet 1, land 2 to bullet 2, land 2

---

# Motivation for Assessing LIME

---

# Quality of Explanation

- LIME uses an interpretable model to mimic the complex model

- Quality of the explanation depends on the approximation

---

# Input Options

- LIME has been implemented in Python and R

- Ridge regression used as the interpretable model

- Offer various implementation settings:

- Simulation method
  - Feature selection method
  - Computation of the weights
  - etc.
  
- Provide a default method

- Otherwise, no advice on how to adjust settings

---

# Diagnostic Plots

---

# Metric Comparison

---

# Top Feature Selected Comparison

---

# Discussion and Current Work

---

- Visual diagnostics allow for the assessment of 
  - local explanation
  - accuracy of local approximation
  - consistency across implementation options
  
- Currently working on
  - creating a set of visuals to assessing each step in the LIME procedure
  - formalize the assumptions made by LIME
  - developing an R package for the creation of these plots