Explainable AI, explained
Peter Flach, University of Bristol
Outline
- Setting the scene
- Explainability for time series
- Actionable counterfactuals
- XAI tools
- Multi-class explanations
Setting the scene
This is how Kacper Sokol describes explainability in his PhD thesis:
"which defines
A simple geometric view
- in black: decision boundary and weight vector
- in red: feature vector of instance currently on the negative side
- in blue: recourse/counterfactual (closest point on decision boundary)
What's special about time series?
- We need meaningful "super segments".
- We need to be able to "switch off" segments.
- We need to be able to define local neighbourhoods through similarity.
Segmenting time series
Segments and changepoints are constructed by considering similarity relations that preserve adjacency.
Background vs. foreground
Background components are present throughout.
Foreground components can then be "switched off" as required for LIME.
Similarity
With dynamic time warping the third perturbation is most similar to the original.
Case study on MIMIC
Counterfactuals
The recourse to be applied to the feature vector in order to change the predicted class.
This work was led by Rafael Poyiadzi.
FACE: Feasible and Actionable Counterfactual Explanations (AIES 2020)
Four counterfactuals for X:
- A has low margin;
- B is in a low-density region;
- C requires crossing a low-density region;
- D is the one found by FACE.
Example paths found by the FACE algorithm
From 0 to 8: MNIST example
XAI tools
Most of this work was led by Kacper Sokol
and funded by Thales UK.
XAI Fact Sheets
A framework for systematic assessment of
explainable approaches
Explaining multi-class predictions
This work was led by Paul-Gauthier Noé.
Explaining a probabilistic prediction on the simplex with Shapley compositions
Standard two-class Shapley values
True multi-class explanations
Per-feature histograms
Cumulative visualisation
Wrapping up
Take-home messages
- Approaches such as LIME (and SHAP) should be considered algorithm schemas.
- There are many devils in the details!
- Explainability approaches don't always generalise well across data types.
- What works for images doesn't necessarily work for tabular data, and vice versa.
- We need to take probabilities seriously.
- In particular, they are rarely additive.
With thanks to...
Kacper Sokol
Torty Sivill
Rafael Poyiadzi
Paul-Gauthier Noé
Miquel Perelló Nieto
Raul Santos-Rodriguez
