# Running the pre-deployment gauntlet: checks that catch problems early
A model that scored 94% accuracy in the lab, passed fair-lending review, and impressed the credit committee still needs to survive one more thing before it touches a real customer: the gauntlet. At most large banks in 2026, that gauntlet is a formal, documented sequence of checks, and skipping a step is the single most common root cause cited in internal post-mortems when an AI system misbehaves in production.
This lesson walks through that checklist, in the order banks actually run it.
Banking regulators do not require a specific testing protocol for AI, but they require model risk management: the discipline of identifying, measuring, and controlling the risk that a model is wrong, misused, or degrades over time. The foundational US text is the Federal Reserve and OCC's SR 11-7 guidance, written in 2011 for statistical models but now stretched to cover machine learning and generative AI. In the EU, the EU AI Act (entered into force 2024, obligations phasing in through 2026-2027) classifies most credit scoring and creditworthiness models as "high-risk," triggering mandatory pre-market conformity assessments.
Neither law hands you a checklist. Banks built their own, borrowing from software engineering, clinical trials, and aviation safety culture. Here is the sequence.
Before a new model replaces an old one, it runs in parallel as a challenger model
Concretely: a bank replacing its credit card fraud model runs the new version on every transaction for 4 to 12 weeks, logging what it *would have* flagged. Analysts compare false positive rates (good transactions wrongly blocked) and false negative rates (fraud missed) against the champion.
A simple worked example: if the champion model on 1,000,000 monthly transactions produces 2,000 false positives (blocking legitimate purchases, a 0.2% rate) and the challenger produces 2,600 (0.26%), that's a 30% increase in customer friction, even if the challenger catches more fraud. The bank must decide if the fraud-catch improvement justifies that cost. This is the kind of tradeoff a champion-challenger test surfaces before anyone experiences it live.
Standard validation checks accuracy on historical data. Stress testing asks: what happens when the world looks nothing like the training data?
Banks build synthetic scenarios: a sudden rate shock, a regional recession, a novel fraud pattern, a data feed going dark. For a loan pricing model, testers might simulate a 2008-style unemployment spike and check whether the model's outputs stay within sane bounds rather than producing nonsensical approvals or blanket rejections.
For generative AI tools (a chatbot handling customer disputes, an LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → drafting credit memos), stress testing includes adversarial prompting: red teams try to make the system leak confidential data, hallucinate a false account balance, or produce advice that sounds like it's giving investment recommendations without a license.
US fair lending law (the Equal Credit Opportunity Act, ECOA, enforced by the CFPB and prudential regulators) prohibits discrimination on protected characteristics like race, sex, and age, and courts have applied it to model outcomes, not just model intent. A model can be legally blind to race and still produce racially disparate outcomes if it uses correlated proxies like zip code.
Standard pre-deployment checks include:
Every model entering production needs a model card: a standardized document covering intended use, training data provenancedata provenanceData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.View full definition →, known limitations, performance across subgroups, and who owns it. This isn't bureaucratic theater; it's what lets an auditor, a regulator, or next year's risk team understand a decision they didn't make.
Google's original model card framework has become an informal industry template, though banks add sections specific to SR 11-7: validation status, approval date, and re-review trigger conditions.
No high-stakes model deploys without a defined human checkpoint. The design question is *where*: before the decision (human approves every output), after (human reviews a sample), or only on exceptions (human reviews only low-confidence or high-value cases).
For a mortgage underwriting model, regulators generally expect human review of denials and high-value approvals. For a marketing next-best-offer model, a light-touch sampling review is proportionate. Getting this proportionality wrong in either direction (over-checking low-risk decisions, under-checking high-risk ones) is a common finding in supervisory exams.
A kill switch is a pre-built mechanism to instantly revert to the prior model or a manual process, without an emergency engineering scramble. Good kill-switch design means:
Banks that skip this step often discover, mid-incident, that reverting requires redeploying infrastructure that takes 48 hours, an unacceptable window when a broken model is actively denying loans or missing fraud.
Knowledge check
1. Why do banks run a formal pre-deployment gauntlet even though regulators like SR 11-7 don't specify an exact testing protocol?
2. What is the core purpose of challenger testing before replacing an incumbent model?
3. A bank's new fraud model shows a lower false negative rate than the incumbent during challenger testing, but a higher false positive rate. What does this trade-off illustrate about why both metrics must be examined together?
4. Select ALL correct answers about the role of model risk management (MRM) principles in AI deployment at banks.
Select all the correct answers.
5. Select ALL correct answers about why a strong lab accuracy score (e.g., 94%) is not sufficient to greenlight a model for production deployment.
Select all the correct answers.
Final deployment typically follows a staged pattern: internal employees first, then a small customer segment (often 1 to 5%), then gradual expansion while monitoring dashboards for drift. Sign-off requires distinct approvals from model risk management, compliance, and the business line, each testing a different question: is it statistically sound, is it legal, and does it work for the customer.
🎬 [VIDEO: "Model Risk Management Explained" - youtube.com - search for recent bank risk-management explainer channels covering SR 11-7 and model validation lifecycles for a visual walkthrough of this governance chain]
A lightweight pseudocode view of a rollout monitor:
if live_false_positive_rate > baseline * 1.3:
alert_risk_team()
if subgroup_approval_gap > 0.20: # 80% rule breach
pause_rollout()
if confidence_score < threshold:
route_to_human_reviewer()Simple rules like these, wired into production monitoring, are what actually catch drift in week one rather than in next quarter's audit.