# Model risk management for AI, not just spreadsheets
A mid-size US bank's fraud model quietly drifted for fourteen months. Nobody noticed until chargebacks spiked 40% in a single quarter, estimate based on typical post-incident disclosures, because the model was still scoring transactions the way it had at launch, while fraud rings had already adapted their patterns. The model wasn't broken. It was unmanaged. That distinction is the entire subject of this lesson.
Banks have run model risk management (MRM) programs for decades, mostly built around interest rate models, credit scoring formulas, and Excel-based valuation tools. Those models are static: build once, validate once, use for years with minor recalibration. AI models, especially machine learning ones, are not static. They learn from new data, degrade silently, and can be gamed by adversaries in real time. Treating them like spreadsheets is exactly how institutions end up in enforcement actions.
Model Risk Management is a formal discipline, most clearly codified in the US by the Federal Reserve and OCC (Office of the Comptroller of the Currency) under SR 11-7, a 2011 supervisory letter that remains the reference standard for how banks should govern models. It defines model risk as the potential for adverse consequences from decisions based on incorrect or misused model outputs.
SR 11-7 rests on three pillars:
In Europe, the equivalent expectations come from the European Banking Authority (EBA) guidelines on ICT and security risk management, and increasingly from the EU AI Act, which entered into force in 2024 and classifies many financial-sector AI uses (credit scoring, for instance) as "high-risk," triggering mandatory risk management systems, documentation, and human oversight requirements.
None of these frameworks were written with deep learning in mind. Applying them to AI requires stretching each pillar.
A logistic regression fraud model has maybe 15 coefficients. An analyst can read the equation and explain why a transaction was flagged. A gradient-boosted model or neural network scoring the same transaction might use hundreds of engineered features and produce a score no human can trace by inspection. This is the explainability gap, and it directly collides with regulatory expectations that banks be able to explain adverse decisions (in the US, under the Equal Credit Opportunity Act's adverse action notice requirements).
Three AI-specific risks deserve names:
Model drift: the statistical relationship between inputs and outcomes changes over time. Fraud patterns evolve; a model trained on 2023 fraud tactics degrades against 2026 tactics. Traditional MRM checks models annually or biannually. Drift can happen in weeks.
Adversarial gaming: fraudsters can probe a live model (via trial transactions) to reverse-engineer its blind spots, something almost impossible with a static underwriting formula.
Feedback loops: if a fraud model blocks certain transaction types, the training data for the next version never sees those patterns again, creating blind spots that compound over retraining cycles.
Walk a payments-fraud model through what a genuine MRM lifecycle looks like at a card issuer or payments processor.
1. Pre-deployment validation. Independent validators test the model on out-of-sample data, check for disparate impact across demographic groups (a fair-lending concern that extends to fraud false-positive rates too), and stress-test against synthetic adversarial transactions.
2. Challenger models. Before full deployment, the incumbent model runs in parallel with one or more "challenger" models on live (but non-decisioning) traffic. This is champion-challenger testing: the challenger's scores are logged but don't yet drive decisions. Over weeks, teams compare precision, recall, and false-positive cost. Visa and Mastercard's network-level fraud systems, and most large issuer programs, run these comparisons continuously, not just at launch, because a challenger that wins in January may lose by June.
3. Champion promotion or shadow mode. If the challenger consistently outperforms, it's promoted to full production, but frequently in "shadow mode" first, meaning it makes real-time decisions on a small traffic slice (say 5%) while the rest still runs on the old champion, limiting blast radius.
4. Ongoing monitoring. Post-deployment, teams track population stability index (PSI), a standard metric measuring how much the input data distribution has shifted from the training baseline. A PSI above roughly 0.25 is a common industry rule of thumb, an estimate rather than a legal standard, signaling material drift that should trigger review.
5. Kill switches. This is the piece spreadsheets never needed and AI absolutely does. A kill switch is a pre-authorized, fast mechanism to revert a model to a prior version or to a simpler rules-based fallback if monitoring detects a serious problem (false-positive spike, biased outcomes, a data pipelinedata pipelineETL (Extract, Transform, Load) is a data integration process that pulls data from sources, reshapes it into a consistent format, and writes it into a target system.Voir la définition complète → break). Without a kill switch, an institution's only option when a model misbehaves is manual intervention, which in fraud, where decisions happen in milliseconds, is far too slow.
A simplified monitoring check, the kind of thing a model risk team automates, looks like this:
import numpy as np
def population_stability_index(expected, actual, bins=10):
"""Estimate PSI between training (expected) and live (actual) score distributions."""
breakpoints = np.percentile(expected, np.linspace(0, 100, bins + 1))
breakpoints[0], breakpoints[-1] = -np.inf, np.inf
exp_pct = np.histogram(expected, breakpoints)[0] / len(expected)
act_pct = np.histogram(actual, breakpoints)[0] / len(actual)
exp_pct = np.clip(exp_pct, 1e-4, None)
act_pct = np.clip(act_pct, 1e-4, None)
return np.sum((act_pct - exp_pct) * np.log(act_pct / exp_pct))
# psi > 0.25 (industry rule of thumb, not a legal threshold) -> trigger reviewThis isn't production-grade code, but it's the actual logic sitting inside real-time drift dashboards at payments companies.
Vérification des acquis
1. The bank in the opening example experienced a fraud model failure primarily because of what underlying issue?
2. Why does the lesson argue that traditional MRM approaches built for tools like Excel-based valuation models are insufficient for AI/ML models?
3. Under SR 11-7's governance pillar, why must model validation be performed by a function independent from the model's builders?
4. Select ALL correct answers about the three pillars of SR 11-7 as described in the lesson.
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers about why AI/ML models pose distinct model risk management challenges compared to static models.
Sélectionnez toutes les réponses correctes.
Regulators don't typically penalize banks for having an imperfect model. They penalize banks for not knowing their model was imperfect, or knowing and not acting.
The US Consumer Financial Protection Bureau (CFPB) and the Federal Trade Commission (FTC) have both signaled, in public guidance and enforcement, that using "black box" AI is not a defense against fair-lending or unfair-practices claims; see the CFPB's circular on adverse action notices and complex algorithms for a concrete example of how this plays out for credit models, a close cousin of fraud scoring in governance terms.
In Europe, under the AI Act, high-risk AI systems used in creditworthiness assessment face explicit obligations: risk management systems, data governancedata governanceData governance is the set of policies, roles, and processes that ensure data is accurate, secure, well-defined, and used responsibly across an organization.Voir la définition complète → documentation, human oversight, and post-market monitoring, enforceable by national supervisory authorities with fines that can reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.Voir la définition complète → up to 7% of global annual turnover for the most serious violations (a statutory ceiling, not a typical penalty).
The common thread in enforcement narratives: a model that drifted, a validation team that was not independent enough, or a governance process that existed on paper but wasn't followed when the model actually failed. The 2011 SR 11-7 guidance predates deep learning by years, but its core insight, that model risk is organizational risk, not just a math problem, is exactly why it still applies.
🎬 [VIDEO: "Model Risk Management Explained" - youtube.com/results?search_query=model+risk+management+explained+banking - search for recent explainer content from risk consultancies or Federal Reserve education channels on SR 11-7 and MRM fundamentals]
A practical pre-deployment checklist for any AI model touching customer money or decisions: