Guardrails and pre-deployment checks before you ship AI

In the US there is no single federal AI-for-vehicles law. NHTSA (National Highway Traffic Safety Administration) governs through safety standards and recall authority, and it runs a Standing General Order requiring crash reporting for automated driving systems. State DMVs (California, Arizona, Texas) grant testing and deployment permits.

Translation: your checklist is not optional documentation. It is the evidence trail a regulator will subpoena.

The five gates of the go/no-go checklist

Gate 1: ODD boundary enforcement

ODD (Operational Design Domain) is the precise envelope where the system is designed to work: road types, speed range, weather, lighting, geography.

A gate check does not just ask "what is the ODD?" It asks: what happens the instant we exit it?

Concrete example. A highway pilot certified for divided highways, dry or light rain, above 5 degrees Celsius, up to 130 km/h. The check must prove that:

• Heavy fog detected by the camera and lidar confidence drop triggers a handover request within a defined time budget.
• Approaching an unmapped construction zone (outside the HD mapmapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.Voir la définition complète → ODD) degrades gracefully, it does not silently keep steering.

You want an explicit ODD monitor in the code path, not an assumption buried in a slide.

Gate 2: fallback and MRC behavior

MRC (Minimal Risk Condition) is the safe state the vehicle reaches when it cannot continue: typically slowing, pulling to the shoulder or stopping in-lane with hazards on. UN R157 explicitly requires a defined Minimum Risk Manoeuvre.

The gate check validates the full fallback ladder:

1. Driver available and responsive: hand back control.

2. Driver unresponsive: escalating alerts.

3. Still unresponsive or no driver (robotaxi): execute MRC.

Test the ugly cases. Driver has a medical emergency at 120 km/h in the middle lane. What does the MRC actually do? "Stop in lane" can be lethal on a highway. The check must show the MRC is context-appropriate, and that it was validated in simulation and on a closed track, not just written in a requirements doc.

Gate 3: Red-team results

Red-teaming means deliberately attacking your own system before someone else does.

For automotive AI, red-teams probe both the model and the pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.Voir la définition complète →:

• Adversarial perception: stickers on stop signs, projected phantom images, tuned to fool the classifier. The classic research example is small stickers making a model misread a stop sign. See NHTSA's broader safety framing in its Automated Vehicles work.
• Sensor spoofing and jamming: GPS spoofing pushing the vehicle off its mapped lane.
• Cybersecurity (UN R155 territory): can an attacker reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.Voir la définition complète → the inference system over the vehicle's connectivity stack?

The gate is not "did we red-team?" It is: what did red-team find, what is fixed, what residual risk are we accepting, and who signed the acceptance? An open critical finding with no owner is an automatic no-go.

Gate 4: Data lineageData lineageData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.Voir la définition complète → sign-off

Data lineageData lineageData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.Voir la définition complète → is the documented origin, transformation and rights history of every dataset used to train and validate the model.

Why a release gate cares:

• Bias and coverage gaps: if your pedestrian detection training data underrepresents wheelchair users or people in certain clothing at dusk, the model inherits that blind spot. The gate wants coverage evidence across your ODD conditions.
• Provenance and consent: were the images collected legally? Under GDPR (the EU's General Data Protection Regulation), faces and plates in European street data carry obligations.
• Reproducibility: can you rebuild the exact model version that is shipping? If a crash happens in month seven, you must recover the precise training set and weights.

A clean lineage sign-off names the dataset versions, the labeling vendor, the license, and confirms the validation set is disjoint from training. No leakage, no mystery data.

Gate 5: Monitoring hooks and the rollback trigger

Shipping is not the finish line. The gate requires that the fielded system is instrumented so you can see trouble and reverse it fleet-wide.

Define the trip wires before launch:

• Disengagement rate: interventions per 1,000 km. In California, disengagement data is publicly reported to the DMV, so this is a real, comparable metric.
• Near-miss and hard-braking events per vehicle-day.
• ODD exit frequency: sudden spikes suggest the world changed (new construction season) faster than the mapmapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.Voir la définition complète →.
• Model drift: input distribution moving away from training (for example a software update to a common competitor headlight changes the night-time image profile).

Then define the rollback: a documented, tested path to push all vehicles back to a previous validated software version over the air, plus who has authority to pull the trigger and how fast. Cruise's failure was partly that the response was slow and the fleet stayed live.

A worked example of a trigger threshold (illustrative, not a certified standard):

• Baseline validated hard-braking rate: 0.4 events per vehicle-day.
• Trigger: rate exceeds 2x baseline (0.8) sustained over 24 hours across the fleet.
• Action: auto-page the safety owner, freeze the rollout to new vehicles, prepare rollback.

That is 0.4 x 2 = 0.8, a simple multiplier you agree on and write down before launch, so no one negotiates it during a crisis.

Turning the checklist into a decision

A go/no-go is a binary meeting with named owners. Structure it so no single team marks its own homework.

| Gate | Owner | Go criterion |

|------|-------|--------------|

| ODD enforcement | Systems engineering | Exit behavior tested, monitor in code |

| Fallback / MRC | Safety engineering | MRC validated in sim + track, context-aware |

| Red-team | Security + safety | No open critical findings, residuals signed |

| Data lineageData lineageData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.Voir la définition complète → | Data / ML governance | Provenance clean, sets disjoint, reproducible |

| Monitoring + rollback | Operations | Triggers set, OTA rollback tested end to end |

The chair should be someone who can say no and survive it. If the safety owner reports to the person whose bonus depends on shipping, your governance is theater.

One more principle: a safety case. This is a structured, evidence-backed argument that the system is acceptably safe for its ODD. Standards like ISO 21448 (SOTIF, Safety Of The Intended Functionality) and ISO 26262 (functional safety) underpin it. SOTIF specifically targets the automotive AI problem: hazards from performance limitations even when nothing is broken (the model just does not recognize the situation). Your five gates are the evidence that feeds the safety case.

Key Takeaways

• ODD and MRC must be enforced in code, not just documented. The gate proves what happens at the boundary and in the worst-case fallback, tested in simulation and on track.
• Red-team findings are a release gate: ship only with zero open critical items and a signed, owned record of accepted residual risk.
• Data lineage buys you reproducibility and defensibility. You must be able to rebuild the exact shipped model and show your training data covers your ODD without leakage.
• Define rollback triggers and authority before launch, with concrete thresholds (for example 2x baseline hard-braking sustained 24 hours) and a tested over-the-air path. Cruise's slow response is the cautionary tale.
• Anchor the whole thing to real regimes: UN R157 and R155, the EU AI Act, NHTSA oversight, and a SOTIF/ISO 26262 safety case. The checklist is your regulatory evidence trail.