A portfolio analytics team at a large asset manager builds a model that flags which corporate bonds to sell before a rating downgrade. In backtesting it looks brilliant. On the morning it goes live, a data feed glitches, the model sees stale prices, and it starts flagging healthy bonds for sale. The question that decides whether this becomes a headline: was there a kill switch, and did a human have to approve the trades?
This lesson is about the gate every AI model should pass before it touches a live fund workflow. Not the theory. The actual checklist.
Production-readiness is the formal state where a model is judged safe to run on real money, real client portfolios, or real regulatory filings. Getting there is a gate, not a suggestion. A gate means the model does not go live until named people sign off against named criteria.
Regulators expect this. In the US, the Federal Reserve and OCC (Office of the Comptroller of the Currency) guidance known as SR 11-7 sets the standard for model risk management: every model must be validated, monitored, and governed across its life. It predates modern AI but applies squarely to it. In Europe, the EU AI Act (in force from 2024, with obligations phasing in through 2026 and 2027) classifies many financial uses and imposes documentation, human oversight, and risk-management duties on higher-risk systems.
The four guardrails below are what an auditor will actually ask to see.
A kill switch is a mechanism to stop the model instantly, cleanly, and without a code deployment. If your only way to stop a misbehaving model is to page an engineer at 2am, you do not have a kill switch.
Concrete requirements:
Example: a robo-advice rebalancing engine at a wealth platform should be able to freeze into "no rebalancing" mode. Client portfolios sit still, which is safe, while humans investigate.
Human-in-the-loop (HITL) means a person reviews and approves the model's output before it takes effect. The key design choice is the threshold: at what point does a human have to step in?
You do not want a human approving every routine action; that destroys the value of automation. You want humans on the material and the unusual.
Set thresholds on:
Document the threshold and the reasoning. "We chose 0.5 percent of NAV because that is our existing manual trade-authorization limit" is a defensible answer. "It felt right" is not.
Drift is when the world changes so the model's assumptions no longer hold. Two kinds matter:
A drift monitor is an automated check that compares live data and live predictions against the baseline from validation, and alarms when they diverge.
A simple, common approach uses the Population Stability Index (PSI), a metric that scores how much a variable's distribution has shifted. A rough industry rule of thumb (treat as a convention, not a law): PSI below 0.1 is stable, 0.1 to 0.25 is moderate shift, above 0.25 is significant shift needing action.
import numpy as np
def psi(expected, actual, bins=10):
# expected: baseline sample from validation
# actual: recent live sample
breakpoints = np.quantile(expected, np.linspace(0, 1, bins + 1))
breakpoints[0], breakpoints[-1] = -np.inf, np.inf
e = np.histogram(expected, breakpoints)[0] / len(expected)
a = np.histogram(actual, breakpoints)[0] / len(actual)
e, a = np.clip(e, 1e-6, None), np.clip(a, 1e-6, None)
return np.sum((a - e) * np.log(a / e))
# score = psi(baseline_spreads, live_spreads)
# if score > 0.25: raise alert, route to reviewWorked example: your baseline credit-spread inputs give a monitored feature a PSI of 0.03 in week one. Rates spike, and in week six the same feature scores 0.31. That crosses the 0.25 line. The monitor fires, the model routes to HITL review, and a validator decides whether to retrain or pause. That trail (score, threshold, action, decision) is exactly what auditors want.
Everything above is worthless to an auditor if you cannot prove it happened. Sign-off evidence is the documented record that the gate was passed by the right people with the right information.
The core artifact is a model card or model documentation pack. At minimum it records:
For higher-risk EU AI Act systems, much of this maps directly to the technical documentation and human-oversight requirements the Act demands, so building one pack serves both regimes.
🎬 [VIDEO: "Model Risk Management (SR 11-7) Explained" - https://www.youtube.com/results?search_query=SR+11-7+model+risk+management - a concise walkthrough of the validation and governance expectations that anchor pre-deployment gates]
Knowledge check
1. In the opening scenario, a model begins flagging healthy bonds for sale after a data feed glitches. What does this example most directly illustrate about production-readiness?
2. Why does the lesson describe production-readiness as a 'gate, not a suggestion'?
3. An engineer says: 'We have a kill switch, if the model misbehaves, someone can page me at 2am to redeploy fixed code.' Why does this fail the lesson's definition of a kill switch?
4. Select ALL correct answers about what a proper kill switch requires according to the lesson.
Select all the correct answers.
5. Select ALL correct answers about the regulatory context for deploying AI models in finance.
Select all the correct answers.
Bring it together as a gate meeting. The model does not ship until every item is green and initialed.
1. Independent validation complete. Someone who did not build the model has tested it and signed off. Conflicts of interest are the classic audit finding.
2. Kill switch drilled. You physically stopped the model in staging and confirmed the safe default engaged.
3. HITL thresholds live and tested. You fired a synthetic oversized trade and confirmed it routed to a human, not to the market.
4. Drift monitors running with alert routing. Not just computing scores, but sending them to a named inbox with an owner.
5. Fallback defined. If the model is off, the workflow still functions (manual process or a simpler rules-based backup).
6. Documentation pack signed. Model owner, validator, risk, compliance, with dates.
7. Post-deployment review scheduled. A date to check the model in live conditions, typically within the first weeks.
A useful test of maturity: ask "who pulls the kill switch, and how fast?" If the room cannot answer in one sentence, the model is not ready.
Portfolio managers, COOs, and compliance officers do not need to read the code. They do need to own the thresholds and the sign-off. When the EU AI Act or an SEC (US Securities and Exchange Commission) examiner asks "who was accountable for this AI system," the answer must be a person, not "the data science team." Accountability does not delegate to an algorithm.