An Overview of Visualization Techniques for Explainable Machine Learning

.title[
# <img src="figures/lime-sine.png" /> An Overview of Visualization Techniques for Explainable Machine Learning
]
.author[
### Katherine Goode - ISU Graphics Group - April 10, 2020
]

---

<style>

/* colors: #ffa41b, #000839, #005082, #00a8cc */
a, a > code {
  color: #ffa41b;
  text-decoration: none;
}

.remark-slide-content {
  background-color: #FFFFFF;
  border-top: 80px solid #005082;
  font-size: 20px;
  font-weight: 300;
  line-height: 1.5;
  padding: 1em 2em 1em 2em
}

.inverse {
  background-color: #000839;
  border-top: 80px solid #000839;
  text-shadow: none;
	background-position: 50% 75%;
  background-size: 150px;
}

.remark-slide-content > h1 {
  font-family: 'Skia';
  font-weight: normal;
  font-size: 45px;
  margin-top: -95px;
  margin-left: -00px;
  color: #FFFFFF;
}

.title-slide {
  background-color: #FFFFFF;
  border-top: 80px solid #FFFFFF;
  background-image: none;
}

.title-slide > h1  {
  color: #111111;
  font-size: 40px;
  text-shadow: none;
  font-weight: 400;
  text-align: left;
  margin-left: 15px;
  padding-top: 80px;
}
.title-slide > h2  {
  margin-top: -25px;
  padding-bottom: -20px;
  color: #111111;
  text-shadow: none;
  font-weight: 300;
  font-size: 35px;
  text-align: left;
  margin-left: 15px;
}
.title-slide > h3  {
  color: #111111;
  text-shadow: none;
  font-weight: 300;
  font-size: 25px;
  text-align: left;
  margin-left: 15px;
  margin-bottom: -30px;
}

body {
  font-family: 'Skia';
}

.remark-slide-number {
  font-size: 13pt;
  font-family: 'Skia';
  color: #272822;
  opacity: 1;
}

.inverse .remark-slide-number {
  font-size: 13pt;
  font-family: 'Skia';
  color: #FAFAFA;
  opacity: 1;
}

</style>

.left-code {
  color: #777;
  width: 40%;
  height: 92%;
  float: left;
}
.right-plot {
  width: 59%;
  float: right;
  padding-left: 1%;
}
</style>

# Machine Learning

Machine learning models may provide magical predictions,...

---

# Black Box Models

...but being able to explain how many machine learning models produce the predictions is not an easy task.

---

# The Importance of Explanability

---

# Literature on Explanability

**General trends I've noticed:**

- Many recent papers

- Often machine learning and computer science perspectives

- Lots of European authors
  - [General Data Protection Regulation](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation) (GDPR) implemented in 2018
  - [Goodman and Flaxman (2016)](https://arxiv.org/abs/1606.08813): "It is reasonable to suppose that any adequate explanation would, at a minimum, provide an account of how input features relate to predictions, allowing one to answer questions such as: Is the model more or less likely to recommend a loan if the applicant is a minority?"

**Key resources for this talk:**

- [Interpretable Machine Learning](https://christophm.github.io/interpretable-ml-book/) by [Christoph Molnar](https://www.compstat.statistik.uni-muenchen.de/people/molnar/)
- Corresponding R package: [iml](https://github.com/christophM/iml)
- [Computational statistics working group](https://www.compstat.statistik.uni-muenchen.de/) at Ludwig-Maximilians-University in Munich led by [Bernd Bischl](https://www.compstat.statistik.uni-muenchen.de/people/bischl/) ([mlr](https://mlr.mlr-org.com/) and [mlr3](https://mlr3.mlr-org.com/) R packages)

---

# The Plan...

**Setting the Stage**

- Definitions and Philosophical Aspects

**Methods**

- Model Agnostic

- Random Forest Specific

- Neural Network Specific

**Concluding Thoughts**

- Additional Methods and Resources

- A Cautionary Conclusion

---

# Definitions and Philosophical Aspects

---

# Explainability versus Interpretability

There are not agreed upon definitions...

.medium[**[Interpretable Machine Learning (Molnar 2020)](https://christophm.github.io/interpretable-ml-book/)**

- "I will use both the terms **interpretable** and **explainable** interchangeably"
- "I will use “**explanation**” for explanations of individual predictions."

**[Methods for Interpreting and Understanding Deep Neural Networks (Montavon, Samek, and Muller 2017)](https://arxiv.org/pdf/1706.07979.pdf)**

- "**post-hoc interpretability**, i.e. a trained model is given and our goal is to understand what the model predicts (e.g. categories) in terms what is readily interpretable (e.g. the input variables)"
- "Post-hoc interpretability should be contrasted to incorporating **interpretability directly** into the structure of the model..."
- "...when using the word “**understanding**”, we refer to a functional understanding of the model, in contrast to a lower-level mechanistic or algorithmic understanding of it."
- also distinguish between **interpretation** and **explanation**

**[The Mythos of Model Interpretability (Lipton 2017)](https://arxiv.org/pdf/1606.03490.pdf)**

- Paper dedicated to the philosophical discussion of what interpretability is in machine learning

**[Explaining Explanations: An Overview of Interpretability of Machine Learning (Gilpin et. al. 2019)](https://arxiv.org/pdf/1806.00069.pdf)**

- "We take the stance that **interpretability** alone is insufficient.
In order for humans to trust black-box methods, we need **explainability** – models that are able to summarize the reasons for neural network behavior, gain the trust of users, or produce insights about the causes of their decisions"
- Implies that you need both interpretability and explainability?
]

---

# Explainability versus Interpretability

My definitions (based on a conversation with **[Nick Street](https://tippie.uiowa.edu/people/nick-street)** (University of Iowa))...

.pull-left[
**Interpretability** = the ability to directly use the parameters of a model to understand the mechanism of how the model makes predictions

- a linear model coefficient: indicates the amount the response variable changes based on a change in the predictor variable
  
<br>

$$\hat{y}=\hat{\beta}_0+\hat{\beta}_1x_1+\cdots+\hat{\beta}_px_p $$
]

.pull-right[
**Explainability** = the ability to use the model in an indirect manner to understand the relationships in the data captured by the mode

- LIME: model agnostic method that uses a surrogate model

<div class="figure">
<img src="./figures/lime.png" alt="Figure from LIME paper (Ribeiro 2016)" width="276" />
<p class="caption">Figure from LIME paper (Ribeiro 2016)</p>
</div>

]

---

# Should we explain black-box models?

[**Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead**](https://arxiv.org/pdf/1811.10154.pdf) by [**Cynthia Rudin**](https://users.cs.duke.edu/~cynthia/):

- Debunks the “accuracy-interpretability trade-off” myth

- "Explanations must be wrong. They cannot have perfect fidelity with respect to the original model. If the explanation was completely faithful to what the original model computes, the explanation would equal the original model..."

- "...it is possible that the explanation leaves out so much information that it makes no sense."

- Rudin has worked on developing machine learning models with direct interpretability

---

# Model Agnostic Methods

---

# Overview of Model Agnostic Methods

- Can be applied to any model

- Convenient if comparing various types of predictive models

<br>

**Disadvantages**

- Must work with any model

]

.pull-right[
<div class="figure">
<img src="./figures/agnostic.png" alt="From Interpretable Machine Learning (Molnar)" width="150%" />
<p class="caption">From Interpretable Machine Learning (Molnar)</p>
</div>
]

---

# Washington D.C. Bike Rentals

Example data in Interpretable Machine Learning - can be accessed [here](https://github.com/christophM/interpretable-ml-book/blob/master/data/bike.RData)

```r
bike <- load("data/bike.RData")
data(bike)
# Fit a random forest
bike_mod = randomForest::randomForest(x = bike %>% dplyr::select(-cnt), y = bike$cnt)
```

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-0e48f4b55714dc4ba1f6" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-0e48f4b55714dc4ba1f6">{"x":{"filter":"none","vertical":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130","131","132","133","134","135","136","137","138","139","140","141","142","143","144","145","146","147","148","149","150","151","152","153","154","155","156","157","158","159","160","161","162","163","164","165","166","167","168","169","170","171","172","173","174","175","176","177","178","179","180","181","182","183","184","185","186","187","188","189","190","191","192","193","194","195","196","197","198","199","200","201","202","203","204","205","206","207","208","209","210","211","212","213","214","215","216","217","218","219","220","221","222","223","224","225","226","227","228","229","230","231","232","233","234","235","236","237","238","239","240","241","242","243","244","245","246","247","248","249","250","251","252","253","254","255","256","257","258","259","260","261","262","263","264","265","266","267","268","269","270","271","272","273","274","275","276","277","278","279","280","281","282","283","284","285","286","287","288","289","290","291","292","293","294","295","296","297","298","299","300","301","302","303","304","305","306","307","308","309","310","311","312","313","314","315","316","317","318","319","320","321","322","323","324","325","326","327","328","329","330","331","332","333","334","335","336","337","338","339","340","341","342","343","344","345","346","347","348","349","350","351","352","353","354","355","356","357","358","359","360","361","362","363","364","365","366","367","368","369","370","371","372","373","374","375","376","377","378","379","380","381","382","383","384","385","386","387","388","389","390","391","392","393","394","395","396","397","398","399","400","401","402","403","404","405","406","407","408","409","410","411","412","413","414","415","416","417","418","419","420","421","422","423","424","425","426","427","428","429","430","431","432","433","434","435","436","437","438","439","440","441","442","443","444","445","446","447","448","449","450","451","452","453","454","455","456","457","458","459","460","461","462","463","464","465","466","467","468","469","470","471","472","473","474","475","476","477","478","479","480","481","482","483","484","485","486","487","488","489","490","491","492","493","494","495","496","497","498","499","500","501","502","503","504","505","506","507","508","509","510","511","512","513","514","515","516","517","518","519","520","521","522","523","524","525","526","527","528","529","530","531","532","533","534","535","536","537","538","539","540","541","542","543","544","545","546","547","548","549","550","551","552","553","554","555","556","557","558","559","560","561","562","563","564","565","566","567","568","569","570","571","572","573","574","575","576","577","578","579","580","581","582","583","584","585","586","587","588","589","590","591","592","593","594","595","596","597","598","599","600","601","602","603","604","605","606","607","608","609","610","611","612","613","614","615","616","617","618","619","620","621","622","623","624","625","626","627","628","629","630","631","632","633","634","635","636","637","638","639","640","641","642","643","644","645","646","647","648","649","650","651","652","653","654","655","656","657","658","659","660","661","662","663","664","665","666","667","668","669","670","671","672","673","674","675","676","677","678","679","680","681","682","683","684","685","686","687","688","689","690","691","692","693","694","695","696","697","698","699","700","701","702","703","704","705","706","707","708","709","710","711","712","713","714","715","716","717","718","719","720","721","722","723","724","725","726","727","728","729","730","731"],["SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","SUMMER","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","FALL","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","WINTER","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING","SPRING"],["2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2011","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012","2012"],["JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","JAN","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","FEB","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","MAR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","APR","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","MAY","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUN","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","JUL","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","AUG","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","SEP","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","OKT","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","NOV","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ","DEZ"],["NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY","NO HOLIDAY"],["SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON","TUE","WED","THU","FRI","SAT","SUN","MON"],["NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY","NO WORKING DAY","WORKING DAY","WORKING DAY","WORKING DAY","NO WORKING DAY","NO WORKING DAY","WORKING DAY"],["MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","MISTY","GOOD","MISTY","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","GOOD","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","MISTY","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","MISTY","MISTY","GOOD","GOOD","MISTY","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","MISTY","MISTY","GOOD","GOOD","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","MISTY","MISTY","MISTY","GOOD","GOOD","RAIN/SNOW/STORM","GOOD","GOOD","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","MISTY","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","RAIN/SNOW/STORM","RAIN/SNOW/STORM","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","MISTY","MISTY","MISTY","MISTY","MISTY","MISTY","MISTY","MISTY","MISTY","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","MISTY","MISTY","GOOD","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","RAIN/SNOW/STORM","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","MISTY","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","RAIN/SNOW/STORM","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","RAIN/SNOW/STORM","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","MISTY","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","MISTY","MISTY","GOOD","GOOD","MISTY","GOOD","MISTY","RAIN/SNOW/STORM","GOOD","GOOD","MISTY","GOOD","MISTY","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","MISTY","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","RAIN/SNOW/STORM","MISTY","GOOD","GOOD","MISTY","GOOD","MISTY","GOOD","MISTY","MISTY","GOOD","MISTY","GOOD","MISTY","MISTY","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","MISTY","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","MISTY","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","MISTY","RAIN/SNOW/STORM","MISTY","MISTY","GOOD","GOOD","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","MISTY","RAIN/SNOW/STORM","MISTY","MISTY","MISTY","GOOD","MISTY","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","MISTY","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","GOOD","GOOD","MISTY","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","GOOD","GOOD","MISTY","MISTY","MISTY","MISTY","MISTY","MISTY","GOOD","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","MISTY","MISTY","GOOD","GOOD","MISTY","MISTY","RAIN/SNOW/STORM","MISTY","MISTY","MISTY","GOOD","MISTY"],[8.175849,9.083466,1.229108,1.4,2.666979,1.604356,1.236534,-0.244999999999999,-1.498349,-0.910849000000001,-0.0527230000000003,0.118169,-0.244999999999999,-0.439109999999999,2.966651,2.888349,0.264151,2.183349,5.732178,4.298349,0.342499999999999,-5.2208712,-3.4634801,-3.4226089,2.503466,2.2225,1.165,1.563466,1.236534,2.176534,0.499150999999999,1.032178,4.22,0.786979000000001,1.931288,2.966651,5.434151,4.768349,2.379151,-1.665199,-1.215644,0.887277000000001,2.4575,6.876534,11.505,4.506089,6.958267,12.484151,16.518349,10.760849,5.405199,6.256651,0.564434,2.421733,5.895644,9.124356,5.2775,8.143466,11.141831,4.533349,7.745,1.321651,4.298349,10.055849,9.696534,4.301733,5.7475,5.904151,10.287277,6.876534,7.470849,10.064356,7.285199,6.917377,9.165199,11.505,17.38,14.2075,7.6275,12.230445,12.758349,8.306979,5.395,4.415849,4.494151,3.893021,4.424356,6.2175,6.1,4.611651,6.1,6.805,9.781651,18.946651,11.465849,10.369151,12.5625,7.784151,8.0975,12.053349,19.995644,15.6175,11.3875,13.9725,12.993349,12.249151,13.463349,16.0875,15.774151,19.965,13.580849,7.823349,13.62,19.338349,20.513349,21.688349,21.14,21.0225,15.97,14.2075,13.228349,17.810849,20.983349,11.465849,13.580849,14.520849,16.44,16.831651,17.0275,17.0275,17.4975,17.145,16.0875,16.479151,18.4375,19.1425,18.398349,17.85,16.949151,17.223349,20.3175,20.395849,21.688349,23.02,23.059151,25.291651,24.038349,22.824151,23.3725,26.466651,28.425,27.915849,25.605,21.14,21.845,22.471651,23.881651,25.2525,28.464151,29.991651,27.485,26.075,24.5475,21.845,20.395849,21.453349,21.531651,22.510849,24.743349,24.860849,21.845,23.999151,26.466651,26.231651,26.035849,24.665,23.96,24.0775,26.975849,26.231651,24.743349,25.9575,26.701651,25.683349,26.153349,27.093349,25.84,27.25,25.330849,26.466651,27.1325,27.8375,29.325849,27.093349,23.999151,23.176651,24.273349,25.800849,27.093349,28.503349,28.111651,30.305,31.871651,31.910849,31.01,26.936651,28.268349,28.425,28.620849,31.401651,29.795849,29.874151,28.268349,28.816651,26.388349,25.37,25.409151,25.683349,26.8975,27.955,28.425,28.033349,25.7225,25.291651,24.234151,23.803349,23.294151,24.939151,25.996651,25.448349,24.195,24.7825,25.409151,24.508349,22.119151,23.646651,24.155849,24.9,23.96,25.231773,21.923349,22.040849,22.863349,22.785,22.236651,23.450849,25.330849,23.646651,17.38,20.160849,21.793911,22.55,23.02,22.706651,22.284356,22.589151,23.646651,19.1425,14.050849,15.108349,15.8525,17.810849,18.398349,19.965,21.531651,20.630849,20.513349,21.805849,22.510849,21.923349,21.845,20.983349,18.515849,11.27,8.763349,10.055849,14.755849,17.301651,15.225849,16.009151,16.518349,17.419151,18.829151,18.633349,17.536651,19.690849,17.889151,15.813349,16.048349,17.105849,17.0275,17.461733,14.364151,12.0925,11.8575,11.818349,13.776651,14.168349,14.755849,14.09,7.549151,3.945849,7.000849,7.98,10.839151,9.7425,11.191651,10.956651,7.353349,8.371651,10.565,11.191651,10.8,9.86,7.235849,8.763349,12.719151,16.91,16.91,13.463349,8.058349,4.885849,7.470849,13.776651,13.0325,11.583349,12.719151,9.546651,9.625,9.664151,13.580849,15.663466,13.541651,7.275,6.6875,6.765849,6.060849,7.549151,10.134151,13.7375,11.27,4.494151,5.669151,4.925,2.379151,3.201651,5.2775,6.9225,11.8575,9.625,4.141651,3.201651,5.003349,10.134151,12.131651,11.896651,9.546651,6.2175,4.914801,7.121733,7.275,6.05911,3.671651,6.648349,11.27,9.39,4.833021,-0.95,-2.9475,4.494151,7.705849,10.486651,7.8625,2.535849,6.508712,4.885849,9.9775,4.885849,0.459999999999999,-0.166651,0.93,9.533021,6.256651,0.93,2.2225,0.146650999999999,-0.3625,2.261651,8.0975,5.825849,8.058349,11.975,6.844151,5.2775,4.650849,10.33,14.050849,10.760849,6.726651,4.415849,4.494151,5.282623,8.645849,4.063349,4.455,5.199151,2.535849,-2.0075,2.4575,7.000849,8.371651,6.883349,8.136651,8.293349,5.16,5.16,5.527822,10.604151,13.345849,11.1525,5.669151,5.120849,9.233349,8.880849,8.184356,14.834151,8.606651,11.465849,7.314151,3.436651,4.141651,10.995849,16.7925,11.309151,5.5125,9.001733,13.933349,18.555,18.9075,18.2025,12.484151,16.165849,14.2075,17.615,18.359151,16.988349,18.045849,20.278349,15.6175,12.5625,12.954151,7.196651,14.755849,15.225849,9.39,11.935849,12.014151,12.393911,13.933349,17.458349,12.445,10.956651,12.5625,15.5,14.990849,12.993349,8.388712,10.6825,12.7975,15.265,20.513349,23.215849,20.591651,13.776651,15.421651,16.753349,18.79,10.643349,7.118349,11.426651,14.403349,15.421651,13.5025,9.703349,13.541651,13.815849,20.826651,18.515849,18.32,21.4925,21.218349,18.4375,17.2625,19.338349,19.025,15.774151,17.066651,18.515849,20.7875,18.946651,20.748349,21.923349,19.886651,18.515849,20.2,21.179151,20.121651,20.905,21.218349,22.785,23.96,24.5475,24.43,25.4875,25.9575,22.863349,23.96,22.745849,19.416651,20.3175,20.0825,17.419151,18.045849,20.3175,22.510849,25.409151,26.153349,25.879151,22.706651,22.824151,22.471651,22.040849,21.688349,19.8475,18.711651,24.351651,28.7775,29.874151,28.5425,26.388349,26.936651,25.644151,21.649151,24.7825,27.210849,31.205849,27.955,30.344151,28.738349,28.699151,29.090849,30.8925,30.931651,32.498349,30.6575,25.409151,25.879151,25.683349,25.644151,26.388349,25.056651,27.054151,27.876651,30.461651,29.286651,28.19,23.294151,20.004151,23.3725,26.858349,27.289151,26.035849,28.503349,28.738349,27.524151,25.918349,26.349151,25.526651,25.7225,27.3675,27.994151,29.286651,28.150849,27.3675,26.584151,27.25,27.524151,25.644151,24.5475,24.939151,25.879151,26.153349,25.213349,25.800849,25.996651,23.881651,21.884151,21.884151,22.510849,23.3725,24.704151,25.0175,23.098349,22.706651,25.056651,26.231651,24.195,25.213349,27.915849,27.406651,24.743349,25.2525,26.114151,26.623349,24.743349,25.056651,22.980849,20.67,19.416651,19.1425,20.160849,20.7875,21.766651,20.591651,19.26,19.299151,21.296651,17.9675,17.693349,20.160849,22.55,16.870849,16.165849,17.85,21.845,22.55,21.100849,17.4975,16.753349,16.479151,19.769151,22.9025,22.9025,20.905,18.045849,11.544151,10.016651,12.993349,16.165849,12.445,12.5625,10.486651,16.518349,18.398349,14.011651,13.424151,16.5575,18.476651,14.755849,13.815849,14.9125,17.575849,19.6125,17.85,17.654151,16.91,14.4425,12.68,6.954554,8.8025,9.194151,8.685,8.136651,7.314151,7.000849,5.199151,5.904151,8.552178,8.998349,10.290849,11.779151,14.795,8.136651,5.590849,7.118349,8.215,7.275,8.0975,9.899151,9.585849,8.606651,7.98,9.311651,5.081651,3.554151,6.726651,5.708349,5.943349,5.20089,6.021651,6.021651,8.3325,13.2675,14.364151,12.601651,4.024151,7.079151,9.938349,10.055849,12.484151,8.606651,5.9825,5.904151,5.238349,7.235849,9.0375,10.486651,11.309151,7.6275,7.51,7.353349,4.494151,3.554151,2.871288,5.691288,3.436651,3.945849,3.906651,3.906651,4.024151,2.144151],[80.5833,69.6087,43.7273,59.0435,43.6957,51.8261,49.8696,53.5833,43.4167,48.2917,68.6364,59.9545,47.0417,53.7826,49.875,48.375,53.75,86.1667,74.1739,53.8333,45.7083,40,43.6522,49.1739,61.6957,86.25,68.75,79.3043,65.1739,72.2174,60.375,82.9565,77.5417,43.7826,58.5217,92.9167,56.8333,73.8333,53.7917,49.4783,43.7391,50.6364,54.4167,45.7391,37.5833,31.4348,42.3478,50.5,51.6667,18.7917,40.7826,60.5,57.7778,42.3043,69.7391,71.2174,53.7917,68,87.6364,53.5,44.9583,31.8333,61.0417,78.9167,94.8261,55.1304,42.0833,77.5417,0,64.9565,59.4583,52.7391,49.6957,65.5652,77.6522,60.2917,52.5217,37.9167,47.375,73.7391,62.4583,83.9565,80.5833,49.5,39.4167,49.3913,30.2174,31.4167,64.6667,91.8333,68.625,65.375,48,42.625,64.2083,47.0833,60.2917,83.625,87.75,85.75,71.6956,73.9167,81.9167,54.0417,67.125,88.8333,47.9583,54.25,66.5833,61.4167,40.7083,72.9583,88.7917,81.0833,77.6667,72.9167,83.5417,70.0833,45.7083,50.3333,76.2083,73,69.7083,73.7083,44.4167,59,54.125,63.1667,58.875,48.9167,63.2917,74.75,86.3333,92.25,86.7083,78.7917,83.7917,87,82.9583,71.9583,62.6667,74.9583,81,74.0833,69.625,67.75,65.375,72.9583,81.875,68.5,63.6667,67.7083,30.5,35.4167,45.625,65.25,60,59.7917,62.2083,56.8333,60.5,65.4583,74.7917,49.4583,50.7083,47.1667,68.8333,73.5833,67.0417,66.6667,74.625,77.0417,70.75,70.3333,57.3333,48.3333,51.3333,65.8333,63.4167,49.7917,43.4167,39.625,44.4583,68.25,63.7917,59.0417,74.3333,65.125,75.7917,60.9167,57.8333,63.5833,55.9167,63.1667,47.625,59.125,58.5,60.4167,65.125,65.0417,70.7083,69.125,58.0417,50,55.0833,75.7083,54.0833,40.2917,58.3333,54.25,46.5833,48.0833,55.0833,49.125,65.75,75.75,63.0833,75.5,75.2917,59.2083,57.0417,42.4167,42.375,41.5,72.9583,81.75,71.2083,57.8333,57.5417,65.4583,72.2917,67.4167,77,47,45.5417,60.5,77.1667,76.125,85,56.1765,55.4583,54.8333,59.7917,63.9167,72.7083,71.6667,74.2083,79.0417,88.6957,91.7083,93.9565,89.7917,75.375,71.375,69.2174,71.25,69.7083,70.9167,59.0417,71.8333,69.5,69,88.125,90,90.2083,97.25,86.25,84.5,84.8333,88.5417,84.875,69.9167,64.75,75.375,79.1667,76.0833,71,64.7917,62.0833,68.4167,70.125,72.75,73.375,80.875,90.625,89.6667,71.625,48.3333,48.6667,57.9583,70.1667,89.5217,63.625,57.4167,62.9167,74.125,77.2083,62.2917,72.0417,81.2917,58.5833,88.25,62.375,70.3333,68.375,71.875,70.2083,62.25,51.9167,73.4583,75.875,72.1667,75.8333,81.3333,44.625,55.2917,45.8333,58.7083,68.875,93,57.5833,41,50.2083,68.4583,91,96.25,75.7917,54.9167,64.375,68.1667,69.8333,74.3043,83.0833,61.3333,52.4583,62.5833,61.2917,77.5833,82.7083,94.9583,97.0417,58,69.5833,50.75,49,67.0833,59,66.375,63.4167,50.0417,56.0833,58.625,63.75,59.5417,85.8333,75.75,68.625,54.25,68.1304,50.6957,76.25,50.3913,57.4167,63.6667,61.5833,69.25,38.1304,44.125,41.4583,52.4167,54.2083,53.1667,46.5,70.1667,64.6522,84.75,80.2917,50.75,45.75,41.9167,52.25,71.6087,44.3333,49.75,45,83.125,79.625,91.125,83.5833,64.375,76.9583,74.125,54.3333,31.125,40.0833,41.6667,50.7917,67.2917,52.6667,77.9583,68.7917,62.2174,49.625,72.2917,56.2083,54,73.125,46.4583,41.125,50.875,53.125,75.2917,63.4583,53.4583,51.5833,50.7826,59.4348,56.7917,55.4583,73.75,39.5833,41,49.0833,39.5833,80.4783,61.5417,65.7083,62.125,40.3333,50.625,45.6667,51.3333,56.75,40.7083,35.0417,47.6957,48.9167,61.75,50.7083,57.9583,84.2083,75.5833,81,72.875,80.7917,82.125,83.125,69.4167,88.5417,88.0833,47.7917,29,48.125,43.9167,58.0833,73.8333,67.625,50.4348,39.6667,46.9583,37.4167,37.7083,25.4167,27.5833,31.75,43.5,46.9565,46.625,40.8333,50.2917,50.7917,56.1667,39.0417,56.9167,61.25,69.4583,68.2917,83.5417,76.6667,45.4167,42.7917,75.6667,40.0833,48.9583,58.7083,57,65.9583,79.7083,76.8333,73.5417,75.6667,74,66.4167,68.5833,74.4167,55.2083,36.0417,48.0417,57.625,78.9583,79.4583,69.7917,52,52.3333,45.625,53.0417,81.125,76.5833,77.4583,71.6667,74.7083,73.25,69.7083,67.625,68.4583,67,49.2917,75.5417,54.9167,49.3333,48.7083,61.3333,61.125,56.7083,46.7917,43.7083,53.8333,58.7917,83.3333,58.2083,56.9583,58.9583,50.4167,59.875,77.7917,69,59.2083,56.7917,57.375,53.4583,47.9167,50.4167,37.3333,36,42.25,48.875,60.125,51.875,44.7083,49.2083,53.875,45.7917,45.0833,49.2083,57.375,68.3333,66.75,63.3333,52.9583,48.5833,69.9167,71.7917,64.5,50.5833,57.7083,60.0417,84.4167,86.5417,76.25,69.4167,65.5,45,59.6667,59.4583,61.3333,62.375,66.875,70.4167,67.75,65.9583,64.25,61.3333,65.25,65.4167,70.375,67.2917,62.0417,71.5833,73.2917,53.0417,54.5417,68.6667,61.9583,51.9167,57.0833,60.3333,71.1667,73.4167,67.375,67.7083,63.5833,61.5,71.2917,84.5833,73.0417,62,55.2083,59.0417,58.75,63.8333,81.5,79.0833,75.5,74.125,81.0417,73.625,79.9167,54.75,50.375,52,57.7083,63.7083,67.25,50.1667,57,73.4583,87.25,53.6667,61.8333,66.875,64.6667,46.7083,49.2917,57,63.0833,69.0833,69,54.2917,58.3333,64.9167,87.1667,79.375,72.2917,62.75,66.4167,70.8333,70.9583,76.1667,63.0833,46.3333,53.9167,49.4583,64.0417,70.75,55.8333,69.2917,72.8333,81.5,57.2917,51,56.8333,64.1667,63.625,80.0417,80.7083,72,69.4583,88,82.5455,66.6667,58.1667,52.2083,49.125,53.2917,49.4167,56.7083,54.75,33.3478,54.0833,64.5417,65.9167,74.1667,66.2917,55.2083,62.0417,52.4583,54.5417,69.2917,62.3333,68.5,61.375,58.0417,56.875,40.4583,46.8333,53.5417,78.6667,50.625,55.5652,64.9583,80.6667,82.3333,76.75,73.375,48.5,50.875,76.4167,91.125,90.5417,92.5,59.6667,53.8333,48.5833,64.2917,65.0417,83.875,90.7083,66.625,62.5417,66.7917,55.6667,44.125,51.5417,79.1304,73.4783,82.3333,65.2917,59,75.2917,48.3333,57.75],[10.749882,16.652113,16.636703,10.739832,12.5223,6.0008684,11.304642,17.875868,24.25065,14.958889,8.182844,20.410009,20.167,8.478716,10.583521,12.625011,12.999139,9.833925,13.957239,13.125568,23.667214,11.52199,16.5222,10.60811,8.696332,19.68795,7.627079,8.2611,9.739455,4.9568342,12.541864,3.565271,17.708636,18.609384,8.565213,10.792293,9.5006,3.0423561,24.25065,12.652213,14.869645,7.27285,13.625589,17.479161,27.999836,19.522058,16.869997,15.416968,17.749975,34.000021,14.956745,20.625682,13.110761,6.305571,16.783232,23.218113,12.500257,8.391616,19.408962,14.500475,20.624811,15.125518,13.624182,16.875357,23.000229,22.870584,8.08355,14.75005,17.545759,15.60899,14.791925,18.130468,9.174042,12.348703,13.608839,14.041793,15.478139,24.667189,13.917307,19.348461,15.12525,15.695487,16.333729,15.458575,14.041257,12.3481,14.217668,15.208732,11.583496,14.582282,17.333436,13.208782,12.208271,25.833257,26.000489,17.625221,10.874904,15.208464,8.916561,9.833389,21.739758,18.416893,16.791339,7.4169,15.167125,22.834136,20.334232,10.958989,10.584057,16.208975,21.792286,14.707907,15.458575,12.875725,12.417311,21.8755,20.9174,21.500836,16.084221,15.750025,7.125718,12.291418,22.958689,22.042732,19.791264,15.292482,10.75015,5.0007125,11.792,7.749957,8.083014,12.707689,12.041575,9.04165,10.249593,8.500357,18.582718,13.499964,7.250271,8.375871,8.08355,9.916536,15.667414,13.875164,10.333611,13.376014,16.125493,15.416164,14.333846,8.792075,7.459043,13.875164,19.583229,16.959107,8.250514,9.292364,8.167032,12.583136,9.166739,10.042161,9.417118,10.37495,10.958989,20.45845,18.041961,11.250104,13.833557,9.582943,8.000336,6.834,10.416825,11.458675,11.541554,15.999868,14.875675,14.041257,6.3337311,7.208396,9.666961,17.542007,12.415904,6.874736,7.709154,15.333486,5.4591064,8.459286,10.042161,10.6664,15.083643,11.250104,12.292557,18.916579,13.417018,9.790911,16.124689,12.249811,13.958914,16.417211,14.458868,8.7502,7.625739,14.875407,8.9177,8.791807,11.334457,6.0841561,13.417286,12.292021,11.958093,11.667246,11.291979,11.042471,10.500039,13.79195,9.084061,13.20905,12.374632,15.29275,13.499629,12.875725,10.125107,13.417286,11.041332,8.416607,14.167418,14.916411,13.999918,15.834043,9.625689,15.624936,9.333636,6.999289,16.666518,18.54225,9.833121,16.958236,14.125811,5.6254875,25.166339,20.412153,10.708275,8.375536,5.5833311,9.500332,9.375243,12.416775,13.833289,14.250632,23.044181,6.5003936,12.914116,8.333393,10.291736,7.708618,5.957171,9.500868,11.2091,18.166782,11.000261,12.708225,11.958361,10.166714,9.041918,6.4590814,8.584375,5.2505689,5.2516811,3.3754064,7.4169,7.917457,9.958143,11.583161,13.833825,19.583832,14.874871,5.5841686,13.792218,11.87575,9.041918,1.5002439,3.0420814,4.25115,2.8343814,9.583814,16.62605,9.499729,15.000161,17.291561,18.875039,11.750393,7.375829,16.303713,28.292425,14.833532,6.2086689,6.6673375,7.959064,11.166086,9.959014,13.250121,15.375093,23.541857,11.833339,7.12545,9.083257,5.5001439,9.166739,18.209193,12.667154,6.1676314,3.834075,4.6255125,4.1671186,12.667489,21.083225,14.208154,18.875307,20.541932,13.375411,9.167543,20.459254,11.291711,15.041232,12.45865,9.249618,7.959064,22.500275,11.209368,6.6260186,4.5841936,13.999918,9.522174,17.292164,18.167586,14.750586,6.750518,6.4174811,5.6252061,4.1679561,15.583061,17.833725,16.083886,5.5420189,15.625807,4.4582939,4.25115,9.41685,4.0842061,17.958814,17.458525,16.292189,11.375193,11.584032,4.1252436,14.8338,3.167425,18.374482,12.750368,10.391097,16.044155,12.62615,19.695387,8.000604,9.000579,14.750318,12.875189,22.087555,24.499957,12.3749,8.709129,11.249836,11.708786,12.833314,6.6263,12.565984,8.791807,12.124789,25.333236,12.541261,16.834286,15.500986,23.39171,27.833743,14.750586,13.58425,14.917014,13.375746,7.417436,8.292389,10.791757,4.9175186,22.958689,14.125543,16.08335,14.458064,17.541739,12.667489,12.541529,11.959232,8.167032,11.791732,10.3046,9.874393,8.959307,13.000479,7.834243,19.416332,27.417204,11.207961,9.458993,12.1672,6.125475,13.791682,12.792243,16.958504,15.348561,13.783039,15.709557,12.791171,15.916989,28.250014,13.750343,17.958211,12.958939,12.000839,15.208129,9.708568,10.792293,22.416257,15.333486,13.458625,23.167193,29.584721,27.7916,15.12525,14.913329,13.916771,15.87565,7.709154,10.042161,7.583864,7.417168,8.501161,10.875239,8.125157,6.0004061,7.876654,7.7921,12.916461,14.791925,25.917007,12.541864,19.541957,21.41655,9.250489,16.791339,11.541889,20.913313,6.708911,12.125325,14.708443,20.125996,18.416357,15.583932,23.999132,16.708125,19.783358,19.458743,10.416557,12.791439,15.083643,19.083543,18.333143,11.250104,4.4172564,10.041357,19.000329,23.084582,20.334232,16.708661,7.959064,11.833875,23.291411,8.708325,7.832836,11.499746,10.458432,9.249886,8.957632,10.916846,10.250464,10.041893,15.458307,19.833943,14.499604,21.042221,15.874779,8.249911,15.082839,14.250364,9.875264,8.208304,15.374825,9.166739,5.626325,17.042589,15.624668,7.917189,6.834,11.584032,9.41685,13.332464,14.416457,13.166907,19.7918,9.000043,13.083693,15.916721,12.499654,12.333829,19.083811,14.041525,5.167375,10.54245,11.750661,9.667229,8.959307,13.916771,14.374582,22.999693,17.000111,11.833339,11.166689,9.708568,11.707982,9.917139,7.625404,7.958729,12.250414,12.041307,9.750175,20.125661,23.292014,18.208925,11.50055,11.082939,10.791757,11.291443,13.082889,8.457879,9.04165,12.999943,9.791514,10.958118,8.417143,12.125325,10.166379,10.166111,9.833925,5.41695,9.626493,11.166689,11.000529,7.666743,9.208614,11.083743,14.000789,14.2911,6.2926936,9.291761,14.167418,11.0416,19.082471,10.250464,10.54245,11.416532,10.292339,11.083475,9.458993,8.666718,14.458064,17.249686,19.458207,8.666718,7.832836,7.4169,10.4587,16.000471,13.834093,8.208304,9.126204,11.333586,11.374657,9.500332,15.500718,11.917089,5.79215,8.708593,4.8756436,4.7089811,5.6679186,4.8337686,16.375336,15.333486,8.625111,12.791975,7.541654,5.1668189,11.291711,7.583529,4.2927436,10.125107,15.833507,12.583136,9.542207,11.500282,18.833968,15.041232,17.333771,6.1676314,8.833682,5.5422936,6.958821,16.583907,6.0422811,10.166714,23.958329,14.416725,7.917189,10.333343,19.000061,14.958286,9.541068,15.833507,16.3748,9.000914,10.999993,15.249468,9.042186,6.0838814,6.999825,4.4585686,7.875582,7.12545,17.957675,9.457854,12.708493,12.7501,12.584007,12.166932,15.751164,9.791514,18.667004,19.834479,12.208807,6.791857,15.874779,9.041918,7.874979,11.125618,5.4593811,6.3345686,4.8762064,8.333125,8.875289,15.791364,26.666536,23.9994,14.271603,11.166689,10.542182,17.833725,18.125443,12.000236,15.833775,11.625371,20.375236,23.304945,14.375386,3.8756686,8.5425,11.625639,22.917082,13.374875,10.250129,11.458675,12.041843,15.250004,15.749489,5.542575,6.917482,3.5423436,9.917407,25.250357,10.0835,3.12555,15.916654,14.125007,7.739974,3.9175436,4.0001814,8.333393,5.5422936,11.666643,21.709407,11.708518,8.7502,6.792393,10.584325,12.750636,19.834479,10.916779,11.666643,8.792343,7.12545,6.749714,6.5833061,14.834068,12.334164,8.875021,25.083661,27.292182,8.916561,5.1744368,11.304642,21.208582,23.458911,10.416557,8.333661,23.500518,10.374682],[985,801,1349,1562,1600,1606,1510,959,822,1321,1263,1162,1406,1421,1248,1204,1000,683,1650,1927,1543,981,986,1416,1985,506,431,1167,1098,1096,1501,1360,1526,1550,1708,1005,1623,1712,1530,1605,1538,1746,1472,1589,1913,1815,2115,2475,2927,1635,1812,1107,1450,1917,1807,1461,1969,2402,1446,1851,2134,1685,1944,2077,605,1872,2133,1891,623,1977,2132,2417,2046,2056,2192,2744,3239,3117,2471,2077,2703,2121,1865,2210,2496,1693,2028,2425,1536,1685,2227,2252,3249,3115,1795,2808,3141,1471,2455,2895,3348,2034,2162,3267,3126,795,3744,3429,3204,3944,4189,1683,4036,4191,4073,4400,3872,4058,4595,5312,3351,4401,4451,2633,4433,4608,4714,4333,4362,4803,4182,4864,4105,3409,4553,3958,4123,3855,4575,4917,5805,4660,4274,4492,4978,4677,4679,4758,4788,4098,3982,3974,4968,5312,5342,4906,4548,4833,4401,3915,4586,4966,4460,5020,4891,5180,3767,4844,5119,4744,4010,4835,4507,4790,4991,5202,5305,4708,4648,5225,5515,5362,5119,4649,6043,4665,4629,4592,4040,5336,4881,4086,4258,4342,5084,5538,5923,5302,4458,4541,4332,3784,3387,3285,3606,3840,4590,4656,4390,3846,4475,4302,4266,4845,3574,4576,4866,4294,3785,4326,4602,4780,4792,4905,4150,3820,4338,4725,4694,3805,4153,5191,3873,4758,5895,5130,3542,4661,1115,4334,4634,5204,5058,5115,4727,4484,4940,3351,2710,1996,1842,3544,5345,5046,4713,4763,4785,3659,4760,4511,4274,4539,3641,4352,4795,2395,5423,5010,4630,4120,3907,4839,5202,2429,2918,3570,4456,4826,4765,4985,5409,5511,5117,4563,2416,2913,3644,5217,5041,4570,4748,2424,4195,4304,4308,4381,4187,4687,3894,2659,3747,627,3331,3669,4068,4186,3974,4046,3926,3649,4035,4205,4109,2933,3368,4067,3717,4486,4195,1817,3053,3392,3663,3520,2765,1607,2566,1495,2792,3068,3071,3867,2914,3613,3727,3940,3614,3485,3811,2594,705,3322,3620,3190,2743,3310,3523,3740,3709,3577,2739,2431,3403,3750,2660,3068,2209,1011,754,1317,1162,2302,2423,2999,2485,2294,1951,2236,2368,3272,4098,4521,3425,2376,3598,2177,4097,3214,2493,2311,2298,2935,3376,3292,3163,1301,1977,2432,4339,4270,4075,3456,4023,3243,3624,4509,4579,3761,4151,2832,2947,3784,4375,2802,3830,3831,2169,1529,3422,3922,4169,3005,4154,4318,2689,3129,3777,4773,5062,3487,2732,3389,4322,4363,1834,4990,3194,4066,3423,3333,3956,4916,5382,4569,4118,4911,5298,5847,6312,6192,4378,7836,5892,6153,6093,6230,6871,8362,3372,4996,5558,5102,5698,6133,5459,6235,6041,5936,6772,6436,6457,6460,6857,5169,5585,5918,4862,5409,6398,7460,7132,6370,6691,4367,6565,7290,6624,1027,3214,5633,6196,5026,6233,4220,6304,5572,5740,6169,6421,6296,6883,6359,6273,5728,4717,6572,7030,7429,6118,2843,5115,7424,7384,7639,8294,7129,4359,6073,5260,6770,6734,6536,6591,6043,5743,6855,7338,4127,8120,7641,6998,7001,7055,7494,7736,7498,6598,6664,4972,7421,7363,7665,7702,6978,5099,6825,6211,5905,5823,7458,6891,6779,7442,7335,6879,5463,5687,5531,6227,6660,7403,6241,6207,4840,4672,6569,6290,7264,7446,7499,6969,6031,6830,6786,5713,6591,5870,4459,7410,6966,7592,8173,6861,6904,6685,6597,7105,7216,7580,7261,7175,6824,5464,7013,7273,7534,7286,5786,6299,6544,6883,6784,7347,7605,7148,7865,4549,6530,7006,7375,7765,7582,6053,5255,6917,7040,7697,7713,7350,6140,5810,6034,6864,7112,6203,7504,5976,8227,7525,7767,7870,7804,8009,8714,7333,6869,4073,7591,7720,8167,8395,7907,7436,7538,7733,7393,7415,8555,6889,6778,4639,7572,7328,8156,7965,3510,5478,6392,7691,7570,7282,7109,6639,5875,7534,7461,7509,5424,8090,6824,7058,7466,7693,7359,7444,7852,4459,22,1096,5566,5986,5847,5138,5107,5259,5686,5035,5315,5992,6536,6852,6269,4094,5495,5445,5698,5629,4669,5499,5634,5146,2425,3910,2277,2424,5087,3959,5260,5323,5668,5191,4649,6234,6606,5729,5375,5008,5582,3228,5170,5501,5319,5532,5611,5047,3786,4585,5557,5267,4128,3623,1749,1787,920,1013,441,2114,3095,1341,1796,2729],[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547,548,549,550,551,552,553,554,555,556,557,558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,620,621,622,623,624,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684,685,686,687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730]],"container":"<table class=\"cell-border stripe\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>season<\/th>\n      <th>yr<\/th>\n      <th>mnth<\/th>\n      <th>holiday<\/th>\n      <th>weekday<\/th>\n      <th>workingday<\/th>\n      <th>weathersit<\/th>\n      <th>temp<\/th>\n      <th>hum<\/th>\n      <th>windspeed<\/th>\n      <th>cnt<\/th>\n      <th>days_since_2011<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"tp","scrollX":true,"pageLength":3,"columnDefs":[{"className":"dt-right","targets":[8,9,10,11,12]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[3,10,25,50,100]}},"evals":[],"jsHooks":[]}</script>

---

# Model Agnostic Methods:

## .center[Prediction Visualizations]

---

# Partial Dependence Plots .tiny[[Friedman 2001](https://projecteuclid.org/euclid.aos/1013203451)]

**Purpose**: Visualize marginal relationship between one (or two) predictors and model predictions

**Estimated partial dependence function:**

`$$\hat{f}_{x_{int}}(x_{int})=\frac{1}{n}\sum_{i=1}^n\hat{f}(x_{int},x^{(i)}_{other})$$`

- `$\hat{f}=$` machine learning model fit using predictor variables 
- `$x_{int}=$` value of the predictor of interest
- `$x^{(i)}_{other}=$` vector of training data values of other predictors in the model for observation `$i$`

![](slides_files/figure-html/unnamed-chunk-11-1.png)

---

# Partial Dependence Plots in iml

```r
# Create a "predictor" object that 
# holds the model and the data
bike_pred = 
* Predictor$new(
    model = bike_mod,
    data = bike)

# Compute the partial dependence 
# function for temp and windspeed
pdp = 
* FeatureEffect$new(
    predictor = bike_pred, 
    feature = c("hum", "temp"), 
    method = "pdp")

# Create the partial dependence plot
*pdp$plot() +
  viridis::scale_fill_viridis(
    option = "D") + 
  labs(x = "Humidity", 
       y = "Temperature", 
       fill = "Prediction")
```
]

![](slides_files/figure-html/unnamed-chunk-13-1.png)
]

---

# Interactive PDPs .tiny[[Krause, Perer, and Ng 2016](https://josuakrause.github.io/info/prospector.html)]

---

# Individual Conditional Expectation .tiny[[Goldstein et. al. 2013](https://arxiv.org/pdf/1309.6392.pdf)]

**Purpose**: Similar to partial dependence plots, but consider each observation separately instead of taking an average.

**Estimated individual conditional expectation function:**

`$$\hat{f}^{(i)}_{x_{int}}(x_{int})=\hat{f}(x_{int},x^{(i)}_{other})$$`

![](slides_files/figure-html/unnamed-chunk-15-1.png)

---

# ICE Plots in iml

```r
# Compute the ICE function
ice = 
* FeatureEffect$new(
  predictor = bike_pred,
  feature = "temp",
  method = "ice")

# Create the plot
*plot(ice)
```

]

![](slides_files/figure-html/unnamed-chunk-17-1.png)
]

---

# Centered ICE Plots

"Sometimes it can be hard to tell whether the ICE curves differ between individuals because they start at different predictions. A simple solution is to center the curves at a certain point in the feature and display only the difference in the prediction to this point." - Molnar

```r
# Center the ICE function for 
# temperature at the 
# minimum temperature and 
# include the pdp
ice_centered = 
* FeatureEffect$new(
  predictor = bike_pred,
  feature = "temp",
  center.at = min(bike$temp),
  method = "pdp+ice")

# Create the plot
*plot(ice_centered)
```
]

---

# Addressing Correlation and Interactions

.pull-left[
**Accumulated Local Effects (ALE) Plots**  
[Apley and Zhu (2016)](https://arxiv.org/pdf/1612.08468.pdf)

- Similar to PDPs: consider how a feature influences predictions on average
- In contrast to PDPs: faster to create and account for correlation

![](slides_files/figure-html/unnamed-chunk-20-1.png)

]

- Metrics that measure effect of interactions on the predictions
  - H-statistic [Friedman and Popescu (2008)](https://projecteuclid.org/euclid.aoas/1223908046)
  - Variable Interaction Networks (VIN) [Hooker (2004)](http://faculty.bscb.cornell.edu/~hooker/VIN-kdd.pdf)
  - PDPs based on interactions [Greenwell et. al. (2018)](https://arxiv.org/pdf/1805.04755.pdf)
  
![](slides_files/figure-html/unnamed-chunk-21-1.png)

]

---

# ALE and Interaction Plots in iml

Code for the plots on the previous slide

```r
# Compute the ALEs
*ale = FeatureEffect$new(
  predictor = bike_pred,
  feature = c("hum", "temp"), 
  method = "ale", 
  grid.size = 40)

# Plot the ALEs
*plot(ale) +
  scale_x_continuous(
    "Relative Humidity") + 
  scale_y_continuous(
    "Temperature")+
  viridis::scale_fill_viridis(
    option = "D") + 
  labs(fill = "ALE")
```

]

```r
# Compute the interaction metrics
int = 
* Interaction$new(
    predictor = bike_pred,
    grid.size = 100, 
    feature = "season")

# Plot the interaction metrics
*plot(int) +
  scale_x_continuous(
    "2-way interaction strength")
```

]
---

# Parallel Coordinate Plots

Provide a nice overview of the predictions across

**PCP plot with bike data** (made with [ggpcp](https://github.com/yaweige/ggpcp))

![](slides_files/figure-html/unnamed-chunk-24-1.png)

---

# Parallel Coordinate Plots in ggpcp

Code for the plot on the previous slide

```r
# Determine order of features 
bike_ft_ordered = bike_vi %>% arrange(desc(IncNodePurity)) %>% pull(var)

# Create the pcp
bike %>%
  mutate(rf_pred = predict(bike_mod)) %>% 
  ggplot(aes(color = rf_pred)) + 
* ggpcp::geom_pcp(aes(vars = dplyr::vars(all_of(bike_ft_ordered))), alpha = 0.4) +
  viridis::scale_color_viridis(option = "D") + 
  labs(x = "Featured ordered by feature importance (left to right)",
       y = "Standardized Feature Value",
       color = "Random Forest Prediction") + 
  theme(legend.position = "bottom") + 
  guides(color = guide_colourbar(barwidth = 15))
```

---

# Interactive Parallel Coordinate Plots .tiny[[Beckett (2018)](https://digitalcommons.usu.edu/gradreports/1335/)]

R package [Rfviz] for interactive parallel coordinate plots with random forest models, but it could be extended to other machine learning models.

---

# Model Agnostic Methods:

## .center[Feature Importance]

---

# Permutation Feature Importance

**Background**

- Idea originally proposed by [Breiman (2001)](https://www.stat.berkeley.edu/~breiman/randomforest2001.pdf) for random forests

- Adapted to be used with all models by [Fisher, Rudin, and Dominici (2018)](https://arxiv.org/pdf/1801.01489v1.pdf)

- Call this measure the model class reliance

**Concept**

- Measure feature importance by seeing how much the prediction error is affected when a feature is permuted
  
  - important feature: one that affects the prediction error when changed
  
  - non-important feature: one that does not affect the prediction error when changed

---

# Permutation Feature Importance in iml

**Permutation feature importance of bike data random forest**
  
.left-code[

```r
# Create the predictor 
# (seemingly FeatureImp 
# requires y)
bike_pred = 
* Predictor$new(
    model = bike_mod, 
    data = bike, 
    y = bike$cnt)

# Compute the feature
# importance values
bike_imp = 
* FeatureImp$new(
    predictor = bike_pred, 
    loss = 'mae')

# Plot the feature
# importance values
*plot(bike_imp)
```
]

![](slides_files/figure-html/unnamed-chunk-28-1.png)

Point = median permutation importance  
Bars = 5th and 95th permutation importance quantiles

]

---

# Permutation FI with p-values .tiny[[Altmann et. al. (2010)](https://academic.oup.com/bioinformatics/article/26/10/1340/193348)]

- Permutation based feature importance method that returns p-values

- Example from the paper comparing Gini importance values to their permutation feature importance method with p-values

---

# More Feature Importance .tiny[[Casalicchio, Molnar, and Bischl (2019)](https://arxiv.org/pdf/1804.06620.pdf)]

**Individual Conditional Importance (ICI)**

- "local" permutation feature importance metric 
- similar to ICE plots but "visualize the expected (conditional) feature importance instead of the expected (conditional)" prediction"

**Partial Importance (PI)**

- aggregate of ICI values

**Shapley Feature Importance (SFIMP)**

- based on Shapley values

]

ICI and PI available in the [featureImportance](https://github.com/giuseppec/featureImportance) R package

]

---

# Model Agnostic Methods:

## .center[Surrogate Models]

---

# Global Surrogate Models

**Procedure**:

1. Train a black-box model

2. Obtain predictions from black-box model on a set of data (training data or other)

3. Fit an interpretable model (linear regression model, tree, etc)

.center[black-box predictions ~ predictor variables]
  
**Cautions**: How to know if the global surrogate is a good enough approximation of the complex model?
]

---

# Using a Tree as the Global Surrogate

Using a classification tree as the global surrogates for the random forest model fit to the sine data

---

# Local Surrogate Model: LIME .tiny[[Ribeiro, Singh, Guestrin (2016)](https://arxiv.org/pdf/1602.04938.pdf)]

**LIME** = Local Interpretable Model-Agnostic Explanations

- Consider one prediction of interest
- Use a surrogate model to explain the black-box model in a "local" region about a point of interest

---

# LIME in R

.pull-left[
**[lime](https://github.com/thomasp85/lime)**
  - written by Thomas Pedersen
  - package for implementing LIME
  
![](slides_files/figure-html/unnamed-chunk-33-1.png)

]

**[limeaid](https://github.com/goodekat/limeaid)**
  - written by me 😄
  - package for visually understanding and assessing LIME
  
![](slides_files/figure-html/unnamed-chunk-34-1.png)
]

---

# Model Agnostic Methods:

## .center[Game Theory Based Method]

---

# Shapley Values .tiny[[Štrumbelj and Kononenko (2014)](https://link.springer.com/article/10.1007/s10115-013-0679-x)]

**Idea**: Use game theory to determine contributions of predictor variables to one prediction of interest

**Game Theory Connection**: [Shapley values](https://www.rand.org/content/dam/rand/pubs/research_memoranda/2008/RM670.pdf) are "a method for assigning payouts to players depending on their contribution to the total payout."

| Game Theory Term | Machine Learning Meaning |
| ----------- | ---------------- |
| collaborative game | machine learning model prediction for one prediction |
| players | predictor variables | 
| payout | contribution of a predictor variable to the prediction | 
| gain | actual prediction - average prediction for all instances |

---

# Shapley Values in iml

**Interpretation**: "The value of the `$j$`th feature contributed `$\phi_j$` to the prediction of this particular instance compared to the average prediction for the dataset."

```r
# Select obs of interest and perpare data
x_int = bike[names(bike) != 'cnt'][285,]

# Compute prediction values
avg_pred = mean(predict(bike_mod))
actual_pred = predict(bike_mod, newdata = bike[names(bike) != 'cnt'][285,])
diff_pred = actual_pred - avg_pred

# Compute shapley values
predictor = 
  Predictor$new(model = bike_mod, data = bike[names(bike) != 'cnt'])
shapley = 
* Shapley$new(predictor = predictor,
*             x.interest = x_int)

# Create the plot
plot(shapley) +  
  scale_y_continuous("Feature value contribution") +
  ggtitle(sprintf("Actual prediction: %.0f\nAverage prediction: %.0f\nDifference: %.0f", actual_pred, avg_pred, diff_pred))
```
]

**Shapley values for one observation from the bike rental random forest**

![](slides_files/figure-html/unnamed-chunk-37-1.png)

]

---

# Random Forest Specific Techniques

---

# Quick intro to random forests

**Idea**: Aggregation of many trees (bootstrap data and randomly select predictors for each tree)

---

# Feature Importance Plot

**Mean decrease in impurity (gini importance)**: measures the average improvement in node purity for a predictor variable

```r
# Extract the importance values
bike_rfimp <- bike_mod$importance

# Put the feature importance in a df
bike_vi <-
  data.frame(var = rownames(bike_rfimp),
             bike_rfimp) %>%
  arrange(IncNodePurity)

# Create a feature importance plot
bike_vi %>%
  mutate(var = factor(x = var, levels = bike_vi$var)) %>%
  ggplot(aes(x = var, 
             y = IncNodePurity)) + 
  geom_col() + 
  coord_flip() + 
  labs(x = "Feature")
```
]

**Bike random forest feature importance plot**

![](slides_files/figure-html/unnamed-chunk-40-1.png)
]

---

# Visualizing Sets of Trees .tiny[[Simon Urbanek (2008)](https://haralick.org/DV/Handbook_of_Data_Visualization.pdf)]

**Cut points from all trees for two predictor variables**

---

# Visualizing Sets of Trees .tiny[[Simon Urbanek (2008)](https://haralick.org/DV/Handbook_of_Data_Visualization.pdf)]

**Trace plots of all trees in a random forest**

---

# ggRandomForests .tiny[[Ehrlinger (2015)](https://arxiv.org/pdf/1501.07196.pdf)]

R package for visually exploring random forests fit using randomForests or randomForest

```r
library(ggRandomForests)
```

```r
plot(gg_error(bike_mod)) + theme_gray()
```

![](slides_files/figure-html/unnamed-chunk-44-1.png)
]

```r
plot(gg_vimp(bike_mod)) + theme_gray()
```

![](slides_files/figure-html/unnamed-chunk-45-1.png)

]

---

# rfviz .tiny[[Beckett (2018)](https://chrisbeckett8.github.io/Rfviz.html)]

Previously mentioned...R package for interacting with parallel coordinate plots for random forests

```r
# Prepare data
rfprep <- rfviz::rf_prep(x = bike[names(bike) != "cnt"], y = bike$cnt)

# View plots
rfviz::rf_viz(rfprep, input = TRUE, imp = TRUE, cmd = TRUE, hl_color = 'black')
```

---

# Forest Floor Visualizations .tiny[[Welling et. al. (2016)](https://arxiv.org/pdf/1605.09196.pdf)]

- From the paper:

"*We suggest to first use feature contributions, a method to decompose trees by splitting features, and then subsequently perform projections. The advantages of forest floor over partial dependence plots is that interactions are not masked by averaging.*"
  
- R package: [`forestFloor`](https://github.com/sorhawell/forestFloor)
  - I struggled to get it to work...
]

**Forest floor plots** (figure from the paper)

]
---

# Neural Network Specific Techniques

---

# Quick intro to random forests

**Idea**: Combination of many non-linear regression models

.small[Image [source](http://www.datastuff.tech/machine-learning/convolutional-neural-networks-an-introduction-tensorflow-eager/)]

---

# Feature Visualization .tiny[[overview article by Olah, Mordvintsev, and Schubert (2017)](https://distill.pub/2017/feature-visualization/)]

**Idea**: Determine values of predictor variables that maximize activation functions at a specific "location" in the neural network

**Formula Version**: For a node in the network: `$\ \ \ \underset{x}{\arg\max} \ f_j(x, w_j)$`

- `$x$` = values of predictor variables
- `$w_j$` = estimated weights at node `$j$`
- `$f_j$` = activation function used at node `$j$`

<img src="./figures/feature-viz.png" width="90%" style="display: block; margin: auto;" />
.small[Image from [Olah, Mordvintsev, and Schubert (2017)](https://distill.pub/2017/feature-visualization/)]

---

# Saliency Maps .tiny[[Simonyan,  (2014)](https://arxiv.org/abs/1312.6034)]

.pull-left[
**Purpose**: To identify the features that are important for making a prediction for a single observation

**Concept**: Makes use of back-propagation algorithm to determine gradient values associated with a predictor variable which indicate how much a predictor variable influences the prediction

**In practice**:

- Commonly used with convolutional neural networks to identify important pixels in an image

- Many algorithms for creating saliency maps

]

.pull-right[
<img src="./figures/saliency.png" width="95%" style="display: block; margin: auto;" />
.center[.small[Image from [Simonyan, Vedaldi, and Zisserman (2014)](https://arxiv.org/abs/1312.6034)]]
]

---

# Grand Tours .tiny[[Li, Zhao, and Scheidegger (2020)](https://distill.pub/2020/grand-tour/)]

**Idea**: Make use of the Grand Tour to visualize behaviors of neural networks

---

# Additional Methods and Resources

---

# More Methods

- Scoped Rules (Anchors) [Ribeiro, Singh, Guestrin (2018)](https://homes.cs.washington.edu/~marcotcr/aaai18.pdf)

- SHAP [Lundberg and Lee (2017)](https://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions)

- Sensitivity Analyses [Cortez and Embrechts (2013)](https://www.sciencedirect.com/science/article/pii/S0020025512007098)

**Example-Based Explanations**

- Counterfactual examples

- Adversarial examples

- Prototypes and criticisms

- Influential instances

]

- Neural networks: See articles on [Distill](https://distill.pub/)

- Tree ensembles:
    - [da Silva, Cook, and Lee (2017)](https://arxiv.org/pdf/1704.02502.pdf)
  - [Hara and Hayashi (2016)](https://arxiv.org/pdf/1606.05390.pdf)

**General Model Viz**

- Visualizing statistical models: Removing the blindfold [Wickham, Cook, and Hofmann (2015)](https://onlinelibrary.wiley.com/doi/abs/10.1002/sam.11271)

<br>

Many more....

]

---

# Overviews

**Additional Resources for Overviews for Explainable Machine Learning**

[Gilpin et. al. (2019)](https://arxiv.org/pdf/1806.00069.pdf)
  - Explaining Explanations: An Overview of Interpretability of Machine Learning

[Mohseni, Zarei, and Ragan (2019)](https://arxiv.org/pdf/1811.11839.pdf)
  - A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
  
[Ming (2017)](http://www.cse.ust.hk/~huamin/explainable_AI_yao.pdf)
  - A Survey on Visualization for Explainable Classifiers
  
[Guidotti et. al. (2018)](https://arxiv.org/pdf/1802.01933.pdf)
  - A Survey Of Methods For Explaining Black Box Models

---

# A Cautionary Conclusion

---

# Some thoughts on EML

- model agnostic versus model specific

- global versus local explanations

- static versus interactive

- models versus metrics

**Good News**

- many methods to try out

- lots of research opportunities

- opportunity for creating useful visualizations

]

- this is a relatively new field

- unsure which are the most trusted methods

- a seemingly simple method may not be so simple

- model based methods

- add an additional layer of complexity to an already complex situation

- almost seems naive to expect a simple model to capture the complex relationship in a black-box model
]