# Building pre-deployment guardrails and checks
In 2018, Amazon quietly scrapped an internal recruiting AI after discovering it penalized resumes containing the word "women's" (as in "women's chess club captain"). The model had learned from a decade of male-dominated hiring data. That failure never reached customers, because a human caught it before launch. That is the whole point of pre-deployment guardrails: catch the disaster in the lab, not on the runway.
Now picture your own stack. An AI recommends products to shoppers and ranks suppliers for your next season buy. What stops it from steering plus-size customers only toward "shapewear," or downgrading a supplier because of a data artifact? A checklist. Let's build one.
Fashion AI touches two high-stakes surfaces:
Both can cause real harm. A biased recommender can exclude body types or ethnicities. A biased supplier model can quietly defund small or minority-owned vendors. And both now sit inside a tightening regulatory net.
These are estimates of the regulatory state as of early 2026; verify current effective dates before you rely on them.
Treat deployment like a flight. No single person launches; the checklist does. Five gates.
Provenance means knowing where your training data came from and whether you had the right to use it.
Concrete checks:
A one-line rule for the sign-off: no dataset ships without a named owner and a documented right to use it.
Set the numeric bar *before* you see results, so you cannot rationalize a bad number later.
For a recommender, a common metric is demographic parity (do different groups get comparable exposure to the full catalog?) or equal opportunity (among shoppers who would buy premium coats, do all groups see them at similar rates?).
Worked example. Suppose you measure "share of users shown the premium outerwear collection":
Disparate impact ratio = 24% / 40% = 0.60.
A widely cited rule of thumb (from US employment law, the "four-fifths rule") flags anything below 0.80 as a red flag. 0.60 fails. That is a no-go until you investigate why Group B is being steered away from premium items.
# Simple disparate-impact check for a recommender
def disparate_impact(exposed, totals):
rates = {g: exposed[g] / totals[g] for g in exposed}
lowest = min(rates.values())
highest = max(rates.values())
ratio = lowest / highest
return round(ratio, 2), ("PASS" if ratio >= 0.80 else "NO-GO")
exposed = {"A": 4000, "B": 2400}
totals = {"A": 10000, "B": 10000}
print(disparate_impact(exposed, totals)) # (0.6, 'NO-GO')The 0.80 threshold is a starting convention, not a legal guarantee. Document why you chose your threshold.
Human-in-the-loop means a person reviews or approves AI output before it takes effect. Human-on-the-loop means a person monitors and can intervene, but the AI acts by default.
Match the review to the stakes:
Practical rule: the AI can *recommend* cutting a supplier's order by more than, say, 30%, but a human buyer must approve it. Log who approved and why. That log is also your EU AI Act evidence trail.
🎬 [VIDEO: "What Is Human-in-the-Loop Machine Learning?" - youtube.com - a short, plain-language explainer on where humans belong in the AI pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition →]
If the model misbehaves in production, how fast can you turn it off? A rollback plan answers three questions:
1. Trigger: what metric drop or complaint volume flips the switch? (Example: return rate on recommended items jumps 15% week over week, or fairness ratio drops below 0.80 in live monitoring.)
2. Fallback: what runs instead? Usually the previous model version or a simple rules-based ranking ("best sellers by category"). Never fall back to *nothing*.
3. Owner and clock: who can trigger it, and how fast? Target: revert within one hour, not one sprint.
Fashion tip: keep the last-known-good model warm during peak events (Black Friday, seasonal drops). That is exactly when a bad recommender costs the most and when engineers are least available.
Before launch, produce a model card: a short document describing what the model does, its training data, its known limits, and its intended use. Google popularized the format; it is now a governance staple.
Minimum contents for a supplier-ranking model:
The out-of-scope line matters. Most AI harm comes from using a model for something it was never validated to do.
Knowledge check
1. The Amazon recruiting AI example illustrates which core principle behind pre-deployment guardrails?
2. Why does a biased supplier-ranking model represent a distinct type of harm compared to a biased product recommender?
3. Under the EU AI Act's risk-based approach, why might a fashion company's AI fall into 'high risk' rather than 'limited risk'?
4. Select ALL correct answers about why fashion AI needs its own launch gate.
Select all the correct answers.
5. Select ALL correct answers about how GDPR and the EU AI Act apply to fashion AI systems.
Select all the correct answers.
A checklist is only real if it can say no. Assign a launch owner (often a product lead) and a small review group that includes someone outside the build team, so nobody is grading their own homework.
Run the gate at two moments:
Your team wants to launch an AI that reallocates reorder budget across 200 suppliers.
Investigation shows the model penalized suppliers with short data histories, which skewed against newer, smaller vendors. You add a rule: suppliers with under 6 months of data get a human review instead of an automated cut. Re-test: ratio rises to 0.83. Now it passes.
That single fix protected small vendors *and* reduced your legal exposure under emerging anti-discrimination rules. Governance and good buying pointed the same way.