# AI Risk and the EU AI Act
In 2023, a Dutch tax authority scandal—the *toeslagenaffaire*—forced the resignation of an entire government cabinet. The trigger: an algorithm that flagged families for childcare-benefit fraud, disproportionately targeting dual-nationality households, and clawed back money from tens of thousands of innocent people. No CDO signed off on a "high-risk AI system." They signed off on a fraud-detection model. That gap—between how the business labels a system and how a regulator will—is exactly the terrain the EU AI Act now maps. Your job is not to memorize the Act. It is to build a triage function that tells you, on any Monday, which of your 200 models could become tomorrow's *toeslagenaffaire* and which are boring enough to leave alone.
The single most expensive mistake a CDO makes with the AI Act is governing at the wrong altitude. Teams instinctively ask "is this model risky?" The Act asks a different question: "what is this system *used for*, and on whom?"
The same gradient-boosted classifier is unregulated when it ranks marketing leads and high-risk when it screens job applicants. The technology is identical. The risk tier is determined entirely by deployment context. This is why model-centric governance fails—you cannot inspect a .pkl file and know its regulatory exposure. You have to trace it to a decision that affects a human's access to employment, credit, education, essential services, or liberty.
The Act sorts systems into four tiers, and your operating model should mirror them precisely:
The strategic insight: the Act is designed to concentrate scrutiny. If your governance program treats all 200 models with equal rigor, you have misread the entire regulatory intent. Proportionality is not a nice-to-have; it is the operating principle the legislators encoded.
There is a fifth track that cuts across the tiers: general-purpose AI (GPAI) models, including large language models. If you build on top of a foundation model, some obligations sit with the provider (OpenAI, Anthropic, Mistral). But if you fine-tune substantially, or if the model exceeds the systemic-risk compute threshold (10²⁵ FLOPs), obligations can flow to you. The practical CDO move: maintain a register of which foundation models you consume, whether you fine-tune them, and what documentation the provider gives you. When your legal team asks "are we a provider or a deployer of this GPAI system?", you need the answer in a spreadsheet, not a Slack archaeology dig.
Here is the Monday-morning problem. You have an AI inventory (if you don't, that's the prerequisite lesson). Now you need to classify each entry into a tier without hiring a legal team to review every model. The answer is a two-stage funnelfunnelThe customer journey from awareness to purchase, typically Awareness, Interest, Consideration, Decision, Action, with prospects narrowing at each stage.Voir la définition complète →: a cheap automated screen, then a human adjudication for anything that trips a wire.
Stage one—the screening questionnaire. Attach five questions to every system in your registry, answered by the product owner, not the data scientist:
1. Does the system make or materially inform a decision about a specific individual?
2. Does that decision affect their access to employment, credit, education, healthcare, public benefits, or legal standing?
3. Is the system used by law enforcement, migration, or the justice system?
4. Does it use biometric data or infer emotional/protected characteristics?
5. Does it interact directly with a person or generate synthetic media?
Any "yes" to Q3 or Q4 escalates immediately. A "yes" to Q1 *and* Q2 flags high-risk. A "yes" only to Q5 flags limited-risk transparency obligations. All-no lands in minimal risk. You can encode this as a decision table so classification is reproducible and auditable rather than a matter of opinion:
# ai_risk_triage_rules.yaml — deterministic first-pass classifier
rules:
- id: prohibited_check
if: uses_social_scoring OR realtime_biometric_public OR workplace_emotion_recognition
then: PROHIBITED # halt: legal review before any further work
- id: high_risk_domain
if: affects_individual AND domain in [employment, credit, education,
essential_services, critical_infra, law_enforcement, migration, justice]
then: HIGH_RISK
- id: transparency_only
if: interacts_with_human OR generates_synthetic_content
then: LIMITED_RISK
- id: default
then: MINIMAL_RISK
review:
HIGH_RISK: mandatory_human_adjudication # never auto-final on high-risk
PROHIBITED: mandatory_human_adjudicationThe point of the config isn't the syntax—it's the discipline. Classification becomes a policy artifact you can version, audit, and defend to a regulator, instead of a judgment that evaporates when the analyst who made it leaves.
Stage two—human adjudication. Never let the automated screen finalize a high-risk or prohibited label. The edge cases are where the money is. A credit model that only produces an internal risk score a human loan officer can override may or may not be high-risk depending on how much the human actually defers to it. This is the "meaningful human oversight" question, and it turns on operational reality, not org-chart theater. If your loan officers approve 99.3% of the model's recommendations, you do not have human oversight—you have a rubber stamp with a pulse, and a regulator will treat the system as high-risk.
When a system lands in the high-risk tier, the Act imposes a specific compliance stack. As a CDO, you need to know what each obligation demands of your teams:
The critical distinction most CDOs get wrong: are you a provider or a deployer? If you develop the high-risk system (or substantially modify one), you carry the provider's full obligations. If you merely use someone else's high-risk system, you're a deployer with a lighter—but real—set: ensure human oversight, monitor operation, keep logs, and use it according to instructions. Most enterprises are deployers of vendor tools and providers of their in-house builds simultaneously. Your registry must tag which hat you wear for each system, because the obligations and the liability differ sharply.
Vérification des acquis
1. According to the lesson, what is the fundamental determinant of an AI system's risk tier under the EU AI Act?
2. Why does the lesson argue that model-centric governance fails under the EU AI Act?
3. What lesson does the toeslagenaffaire (Dutch childcare-benefit scandal) illustrate about AI governance?
4. Select ALL correct answers. Which of the following are classified as 'unacceptable risk' (prohibited) practices under the EU AI Act as described in the lesson?
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers. What does the lesson identify as the CDO's actual job with respect to the EU AI Act?
Sélectionnez toutes les réponses correctes.
The reason this lesson exists is that fear produces bad governance. When the fines are cited—up to €35M or 7% of global turnover for prohibited-system violations—leadership's instinct is to lock everything down. That instinct will make you slower than competitors on the 90% of your portfolio that is minimal-risk, while giving you false comfort that you've "done AI governance."
The mature posture is tiered controls that map to tiered risk. Concretely:
Consider how Klarna approached its customer-service AI. When they deployed a large-scale assistant, the system interacts directly with customers—a limited-risk transparency trigger, not high-risk, because it handles service inquiries rather than credit decisions. Their credit-underwriting models, by contrast, sit squarely in high-risk. Same company, same AI investment, two entirely different governance regimes. A CDO who governs both identically has either strangled the chatbot or under-protected the underwriting engine. Proportionality means you get this split right.
The Act phases in, and your roadmap must sequence to it:
This staging is a gift. It tells you exactly where to spend first: confirm you have no prohibited systems (existential), then inventory your GPAI dependencies, then build the high-risk conformity stack ahead of the 2026 deadline. A CDO who front-loads the prohibited-systems audit and back-loads the high-risk documentation build is sequencing intelligently. One who tries to boil the ocean in Q1 is burning credibility.
The Act gives you tiers; it does not give you certainty at the boundaries. The genuine CDO judgment calls are:
1. Classify by use case, not by model. Build a deterministic triage screen tied to your AI registry that routes each system to a tier based on who it affects and what decision it drives—not on its technical sophistication.
2. Audit for prohibited systems first, this quarter. The bans are already in force and carry the largest fines. Everything else is a planning problem; this is an existential one.
3. Tag provider vs. deployer for every system. Your obligations and liability diverge sharply, and most enterprises are both simultaneously. Get this into the registry now.
4. Match control intensity to risk tier—and defend the minimal-risk 90% from over-governance. Proportionality is the Act's operating principle; violating it downward is illegal, violating it upward just makes you slow.
5. Wire a hard pre-deployment gate for high-risk systems into your release pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.Voir la définition complète → before the August 2026 deadline, and design logging and documentation in from the start—retrofitting them into live models is the most expensive path.