Interpretability methods comparison
What do different explanation methods reveal about the same model?
This experiment extends the California Housing project by comparing explanation methods applied to its selected LightGBM model.
The original project focuses on model selection and predictive evaluation. This extension examines how SHAP, LIME, partial dependence plots (PDP) and individual conditional expectation (ICE) explain the same model at global, local and contextual levels.
A model can predict accurately on average while remaining difficult to understand on individual observations. Feature rankings, average response curves and local explanations each reveal a different part of its behaviour. The objective here is to examine what each method contributes, where their interpretations differ, and what remains uncertain.
See the underlying modelling project →
One model, three levels of explanation
The study uses the California Housing dataset: 20,640 census block groups, eight numerical predictors and median house value as the target. Each observation describes an area rather than an individual property.
The selected LightGBM model provides a common reference for three questions:
- Global: which variables contribute most to predictions, and what shapes do the average responses take?
- Local: how do the methods explain a particular prediction?
- Contextual: how much can a model’s response vary between observations?
The comparison is qualitative. It examines complementary views of the same model rather than assigning an overall score to each explanation method.
Global patterns: importance and response shape
SHAP identifies the dominant contributions
Global SHAP importance aggregates the absolute feature contributions across the observations being explained. It describes how strongly a variable contributes to predictions, without indicating a single direction of effect.
In the reported results, latitude and longitude together account for approximately 50% of total mean absolute SHAP importance. Median income contributes approximately 24%, followed by average occupancy at approximately 12%.
These proportions describe the model’s attribution structure on the analysed sample. They are not shares of house prices or estimates of causal influence.
Figure 1 — SHAP summary plot
Graphique global montrant les huit variables et la distribution de leurs contributions SHAP.
interpretation/shap/summary_plot.png
The summary plot adds information that a ranking alone cannot provide: the spread of contributions, their direction and their relationship with feature values. Geography and median income dominate the global picture, but their contributions differ across observations.
PDP describes the average response
A partial dependence plot varies one feature while averaging model predictions over the remaining observed characteristics. It helps describe response shapes that an importance ranking leaves unresolved.
The exported median-income PDP is broadly increasing over the displayed range, with no plateau observed there. The report also describes a broadly decreasing average response for occupancy.
Figure 2 — Partial dependence plots
Figure des effets moyens ; conserver notamment le panneau MedInc. Vérifier les variables effectivement présentes avant de finaliser la légende.
interpretation/pdp/pdp_main_effects.png
SHAP and PDP therefore answer different questions. SHAP summarises feature contributions to predictions; PDP describes an average response as a feature changes.
That response requires care when predictors are dependent. Varying income or a geographic coordinate while holding other characteristics fixed can create combinations poorly represented in the data. A PDP should not be read as the expected outcome of changing a neighbourhood’s income or location.
Local explanations: understanding a particular prediction
Global patterns do not explain every observation equally well. The local analysis examines how the model arrives at a prediction and compares this explanation with the observed target.
SHAP and LIME offer different local descriptions
A SHAP waterfall starts from a baseline and adds feature contributions until it reaches the model prediction.
LIME instead fits a local surrogate around the observation. In this study, its explanations use feature intervals and associated weights to summarise that approximation.
For the case labelled median in the report, the prediction is approximately 1.716, compared with an observed target of 1.789 and a SHAP baseline of 2.071, in the reported target units. The combined SHAP contributions therefore move the prediction below the baseline.
Figure 3a — SHAP waterfall (median)
Première des deux explications du cas median, index 1679.
interpretation/shap/waterfall_median.png
Figure 3b — LIME (median)
Seconde explication du même cas median, index 1679. Les contributions numériques SHAP et LIME ne sont pas interchangeables.
interpretation/lime/lime_median.png
The useful comparison concerns the features highlighted and the type of explanation provided. A LIME rule weight and a SHAP contribution are different quantities, so their magnitudes should not be treated as equivalent.
The available material supports a qualitative comparison. It does not establish systematic agreement or stability across repeated LIME runs.
An explanation can accompany a large error
The case labelled high_error makes the distinction between explanation and accuracy explicit.
Its prediction is approximately 1.988, close to the SHAP baseline of 2.071, while the recorded target is 5.000. The absolute error is approximately 3.012 target units.
Figure 4 — SHAP waterfall (high-error case)
Waterfall du cas high_error, index 15311.
interpretation/shap/waterfall_high_error.png
The waterfall accounts for the model output. It does not, by itself, establish why the prediction differs so much from the observed value.
Target capping and missing predictors are relevant limitations, but this case alone cannot isolate the source of the error. The explanation identifies a prediction that needs further investigation rather than resolving its reliability.
Contextual variation: looking beyond the average
A PDP compresses many responses into one average curve. ICE retains a separate response curve for each observation, varying the selected feature while holding that observation’s other characteristics fixed.
This makes it possible to examine whether the average hides different response patterns.
Figure 5 — PDP and ICE for geographic features
Courbes individuelles et moyenne pour latitude et longitude. Des décalages verticaux seuls ne démontrent pas d’interactions.
interpretation/pdp/pdp_ice_curves.png
The report explores geographic heterogeneity through these curves. The interpretation remains qualitative: no aggregate measure of curve dispersion or systematic interaction test is presented.
SHAP dependence plots offer a complementary view by relating feature values to their attributed contributions. Variation in contributions at similar feature values can motivate further investigation, but it does not establish a causal geographic mechanism.
For this experiment, the practical lesson is to inspect variation before treating an average response as representative of every observation.
What each method contributes
The choice of explanation depends on the question:
- SHAP summary: which features have the largest attributed contributions across the analysed observations?
- PDP: what does the model’s average response look like over a feature range?
- SHAP waterfall: how do contributions combine to produce one prediction relative to a baseline?
- LIME: how can a local approximation summarise the model around an observation?
- ICE: does the response shape vary across observations?
Together, these methods connect global patterns with individual cases. Their differences matter because each explanation describes a particular aspect of the fitted model.
Limits and takeaways
The dataset reflects census conditions from 1990, aggregates observations by block group and caps the recorded target at the upper end. These properties limit the interpretation of individual cases and the transfer of findings to present-day housing decisions.
The comparison also has methodological limits. Per-feature local SHAP contributions were not exported as numerical tables, ICE heterogeneity was not summarised statistically, and the presented LIME explanations do not establish robustness to changes in sampling or neighbourhood settings.
Three findings remain useful:
- Geography and median income dominate the reported global attributions.
- Local explanations help inspect individual predictions, including predictions with large errors.
- Average responses need to be examined alongside variation across observations.
The experiment shows how several explanation methods can make a model easier to inspect. Their value lies in clarifying what the model uses, how it behaves and which predictions deserve closer examination.