SHAP (SHapley Additive exPlanations): Definition and Use in Compliance

What is SHAP (SHapley Additive exPlanations)?

SHAP is a method for explaining machine learning model predictions by assigning each input feature a numerical contribution score. A positive score means that feature pushed the prediction above the baseline. A negative score means it pulled the prediction down. The scores sum exactly to the difference between the model's output for a specific case and the average output across all cases.

The underlying mathematics come from cooperative game theory. Lloyd Shapley introduced Shapley values in 1953 to solve a specific allocation problem: how do you fairly distribute a coalition's total payoff among its members when different members contribute differently depending on who else is in the group? Applied to a machine learning model, the "players" are input features, the "game" is a single prediction, and the "payoff" is how much that prediction deviates from the expected baseline.

Scott Lundberg and Su-In Lee formalized SHAP for machine learning in "A Unified Approach to Interpreting Model Predictions," published at NeurIPS 2017. The core contribution was proving that SHAP is the only explanation method satisfying three mathematical properties simultaneously: local accuracy (contributions sum to the prediction gap), consistency (if a model changes so that a feature matters more, its SHAP score can't decrease), and missingness (features absent from a case score zero). Those guarantees make SHAP explanations auditable in ways that simpler attribution methods can't match.

In practice, financial institutions use SHAP alongside model validation processes to document why models produce specific outputs. A fraud score of 0.93 tells a regulator nothing. A ranked list showing that transaction velocity contributed +0.31, device fingerprint mismatch contributed +0.22, and account age contributed -0.08 is evidence. That's the operational difference between a defensible decision and a black box.

SHAP works across model types. The TreeSHAP algorithm, optimized for gradient-boosted trees and random forests, dominates in financial crime detection because it computes exact Shapley values in polynomial time. Kernel SHAP and Deep SHAP extend the method to neural networks and linear models, with higher computational cost.

How is SHAP (SHapley Additive exPlanations) used in practice?

We've seen compliance teams use SHAP at three distinct points in their workflow, and the productivity difference between teams that do and don't is measurable.

One mid-size regional bank cut analyst review time per alert from 18 minutes to 7 minutes after attaching SHAP summaries directly to transaction monitoring alerts. The analyst doesn't have to guess why a score is high. The top-five contributing features, with signed values, appear in the alert interface before the analyst has read the first line of the case.

The three main applications:

Alert triage. Each alert arrives with a pre-computed SHAP breakdown. Analysts assess whether the top contributors represent genuine risk or model artifacts, document their alert disposition accordingly, and move to the next case. This is faster and more auditable than the alternative: reading raw transaction data and reverse-engineering what the model might have seen.

SAR narrative support. When a case escalates to a Suspicious Activity Report (SAR), the SHAP values for the triggering transactions feed directly into the narrative. The analyst starts from a machine-generated ranked evidence list and adds investigative context. The result is consistent, documented reasoning rather than a summary written from memory after the fact.

Regulatory examination preparation. Examiners from the OCC, Federal Reserve, and FDIC reviewing a BSA/AML program ask how specific decisions were made. SHAP output for a sample of alerts is now standard documentation in exam preparation packages at several major U.S. banks. Examiners want contemporaneous explanations, not retroactive reconstructions.

One practical detail teams consistently miss: SHAP values should be stored at prediction time, not computed on demand. In a regulatory investigation, the explanation must reflect what was available when the decision was made. A retroactive SHAP computation on a retrained model gives different values. Store them with the original score record.

Aggregate SHAP visualizations, specifically beeswarm and dependence plots, also appear in board risk committee presentations where model behavior needs to be communicated without loss of technical accuracy.

SHAP (SHapley Additive exPlanations) in regulatory context

Regulators don't name SHAP in most published guidance, but the requirements they impose have made feature-level attribution operationally necessary at any institution using machine learning for credit, fraud, or AML decisions.

The Federal Reserve and OCC's SR 11-7 guidance (2011) established the framework: banks must be able to explain model outputs, validate that models perform as intended, and document model limitations. SR 11-7 predates modern gradient-boosted trees and neural networks, but examiners apply it to current architectures. "Explain model outputs" now means feature-level attribution, and SHAP is the standard implementation.

The EU AI Act, which entered into force in August 2024, classifies credit scoring and fraud detection systems as high-risk AI. Article 13 requires that high-risk AI systems be transparent enough for operators to interpret outputs and use them appropriately. Article 17 requires technical documentation covering data governance and model characteristics. Neither article names SHAP, but both create conditions where SHAP-based documentation is the practical response.

The European Banking Authority's guidelines on loan origination and monitoring (EBA/GL/2020/06) require institutions to document how input features affect model outputs, with particular attention to features that could act as proxies for protected characteristics. Running SHAP across the loan book and checking for high contribution weights on features correlated with race, gender, or national origin is now standard at EBA-supervised institutions.

AI governance frameworks reinforce this further. The NIST AI Risk Management Framework lists explainability and interpretability as explicit attributes in its Map function, and SHAP is the most cited implementation in financial services for meeting that requirement.

For AML specifically, FATF's 2021 Guidance on the Risk-Based Approach for the Banking Sector acknowledges technology-based transaction monitoring and notes that supervisors reviewing AI systems expect documented reasoning for alerts. SHAP provides exactly that, in a format that travels from analyst workstation to examiner review.

Common challenges and how to address them

Computational cost. TreeSHAP is fast compared to exact Shapley value computation, but adding it to every prediction in a high-volume environment adds latency. A system processing 10,000 transactions per second with a 2ms SHAP overhead per transaction will feel that addition. The standard solution: compute SHAP values asynchronously after scoring, store them alongside the model score, and retrieve them when a case is opened. Analysts get the explanation at case-open time, not at transaction time.

Correlated features. When two features are highly correlated, for example transaction amount and daily cumulative amount, SHAP distributes contribution between them in ways that can appear arbitrary. Both might show 0.11 when one is doing most of the work. The fix is upstream feature engineering to reduce collinearity, or using TreeSHAP's interventional path-dependent option, which handles correlated features more conservatively. Model monitoring should include correlation drift detection to catch new correlations introduced by data distribution shifts.

Baseline staleness. SHAP values are relative to a baseline, typically the expected model output across the training population. If your customer population has shifted since training, the baseline is stale and the explanations mislead. This is a subset of the general model drift problem. Re-establishing the baseline after model retraining or significant population shift is not optional.

Interpretability for non-technical audiences. SHAP values are numbers. A compliance officer reviewing a case isn't always equipped to interpret "+0.19 from device_fingerprint_mismatch." The practical fix is a translation layer: convert SHAP values into readable sentences. "This transaction scored high partly because the device has never been seen on this account before" is actionable. "+0.19" is not. Several AI governance frameworks, including NIST AI RMF, explicitly note that explainability must be useful to the intended audience, not just technically correct.

Adversarial gaming. If bad actors learn which features carry the highest SHAP weight, they may try to manipulate those features to reduce their scores. Treat SHAP output as sensitive operational data, limit its external exposure, and use it for internal analysis rather than customer-facing communication.

Related terms and concepts

SHAP belongs to a family of post-hoc explanation methods. The main alternative is LIME (Local Interpretable Model-Agnostic Explanations), which approximates a complex model locally with a simpler interpretable one. LIME is faster but doesn't satisfy SHAP's consistency and missingness properties. In financial services, where regulators may challenge the mathematical basis of an explanation, SHAP's guarantees matter. LIME has largely been displaced in production compliance applications as a result.

Both SHAP and LIME are implementations of the broader concept of explainability: the capacity to make AI model decisions understandable to humans. Explainability is a component of AI governance, the set of policies, controls, and accountability structures governing how AI systems are developed, deployed, and retired. Without explainability, governance is structural but not substantive.

Model Risk Management (MRM) is the discipline that governs how institutions identify, assess, and control risks from model use across the full lifecycle: development, validation, deployment, monitoring, and retirement. SHAP is now embedded in MRM workflows at most major banks, used during validation to verify that models are making decisions for the right reasons, and during ongoing monitoring to detect behavioral drift.

AI bias analysis is a direct application. If a credit or fraud model has high Shapley weights on features correlated with protected characteristics, that's evidence of proxy discrimination. Running SHAP population-wide and correlating feature importance with demographic attributes is standard practice in model validation and fair lending review at regulated institutions.

Transaction monitoring is the operational context where SHAP explanations are most visible in financial crime compliance. When a transaction monitoring model scores a transaction and generates an alert, the SHAP explanation is typically the first thing an analyst reads.

Champion-challenger model management is another common SHAP application. When evaluating whether to promote a challenger model to production, teams compare the SHAP rankings of both models on identical cases. If the challenger produces similar scores but different feature importance rankings, that's worth investigating before any promotion decision. Explanation stability is a signal of model stability.