IA Responsable : biais, fairness, EU AI Act et processus de revue éthique
Responsible AI is not optional. Organizations that deploy AI without rigor on fairness, transparency, and accountability face regulatory action, reputational damage, and real harm to people. The CDO's job is to build AI governance that is rigorous but not so burdensome it paralyzes innovation.
Why AI goes wrong
AI systems fail in predictable ways:
Biased training data: A hiring algorithm trained on historical hiring data learns to prefer candidates similar to those historically hired, reinforcing existing biases. Amazon discovered their AI recruiting tool was penalizing resumes that included the word "women's" (e.g., "women's chess club") and shut it down before deployment.
Proxy discrimination: A model may not use protected attributes (race, gender) directly, but use correlated proxies (zip code, college name) that produce discriminatory outcomes. Legally and ethically, this is still discrimination.
Distributional shift: A loan default model trained during economic expansion may perform catastrophically during recession, a different economic regime.
Feedback loops: A predictive policing model trained on historical arrests (which reflect past policing patterns) directs police to historically over-policed areas, generating more arrests, validating the model's predictions. The model amplifies existing bias.
AI Fairness and Bias: A Practical Guide
Knowledge check
1. What is 'proxy discrimination' in the context of AI models?
2. Why does a predictive policing model trained on historical arrests create a feedback loop?
3. What key challenge does the lesson emphasize about the different mathematical definitions of AI fairness?
4. Select ALL statements that correctly describe fairness definitions discussed in the lesson.
Select all the correct answers.
5. Select ALL of the following that are recognized ways AI systems fail as described in the lesson.
Select all the correct answers.
The AI fairness framework
Fairness in AI is not a single concept, it's a set of mathematical definitions that can conflict with each other:
Demographic parity: The model's positive prediction rate is the same across groups. A loan approval model approves equal percentages of applicants regardless of race.
Equal opportunity: The true positive rate is the same across groups. Among actually creditworthy applicants, the approval rate is equal regardless of race.
Individual fairness: Similar individuals receive similar predictions. Two candidates with identical qualifications receive similar hiring scores.
These definitions can be mathematically incompatible. Satisfying demographic parity may require violating individual fairness. CDOs must decide which fairness concept applies to each use case, in collaboration with legal, compliance, and ethics stakeholders.
Practical approach: define the specific fairness requirements for each AI system before development, measure them before deployment, monitor them in production.
The EU AI Act: regulatory reality
The EU AI Act entered into force on 1 August 2024 and applies in phases. Some obligations already bind organizations today:
- Since 2 February 2025, the bans on unacceptable-risk practices apply, and companies must ensure staff who work with AI have basic AI literacy.
- Since 2 August 2025, the rules for general-purpose AI (GPAI) models and the main governance and penalty provisions apply. Providers of models like GPT-4o, Gemini, Claude and Llama must publish training-data summaries, technical documentation, and copyright policies. Models with "systemic risk" (very large compute, above roughly 10^25 FLOPs) carry extra obligations on evaluation and incident reporting.
- Most high-risk system rules apply from 2 August 2026, with a longer runway to 2 August 2027 for AI embedded in regulated products (medical devices, machinery, and similar).
The Act categorizes AI systems by risk:
Unacceptable risk (banned): Social scoring by public authorities, untargeted scraping of facial images to build recognition databases, emotion recognition in workplaces and schools, and most real-time remote biometric identification in public spaces. These are prohibited regardless of business case.
High risk (strict requirements): AI in hiring, credit scoring, medical diagnosis, critical infrastructure, and law enforcement. Requires conformity assessment, technical documentation, human oversight, transparency to affected individuals, and bias testing before deployment.
Limited risk (transparency obligations): Chatbots must disclose they're AI. AI-generated or manipulated content, including deepfakes, must be labeled as such.
Minimal risk: Most AI applications. No specific obligations.
Fines are steep: up to 35 million euros or 7% of global annual turnover for prohibited-practice breaches, and up to 15 million euros or 3% for most other violations. CDOs in EU-regulated markets must know which category each system falls into. High-risk obligations have to be built into development from the start, because retrofitting compliance is far more expensive.
Building an AI ethics review process
A lightweight but rigorous AI ethics review process:
Pre-development: For any new AI use case, complete a short "AI impact assessment": who is affected, what decisions does it influence, what are the failure modes, what protected groups could be impacted?
Pre-deployment: For high-impact systems: fairness testing across demographic groups, adversarial testing (what happens with malicious inputs?), human review of edge cases.
Post-deployment: Monitor for performance degradation across demographic groups, track appeals and complaints from affected individuals, schedule periodic re-evaluation.
Escalation path: Clear process for when to escalate concerns, from the data scientist who spots a bias issue to the AI ethics committee to the CDO and legal team.
This process adds 10-20% overhead to AI development timelines. It's worth it: the alternative is building systems that harm people and expose the organization to significant legal and reputational risk.
Key Takeaways
- AI fails in predictable ways: biased data, proxy discrimination, distributional shift, and feedback loops. Amazon's scrapped recruiting tool is a concrete example of historical bias baked into a model.
- Fairness is not one definition. Demographic parity, equal opportunity, and individual fairness can conflict, so pick the right one per use case with legal and ethics input.
- The EU AI Act is now partly in force: prohibited-practice bans and AI literacy since February 2025, GPAI and governance rules since August 2025, and most high-risk obligations from August 2026 (August 2027 for regulated products).
- Penalties reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → 35 million euros or 7% of global turnover, so build documentation, human oversight, and bias testing into development rather than retrofitting.
- A staged review process (impact assessment, pre-deployment testing, production monitoring, escalation path) adds 10-20% overhead and prevents far costlier harm.
Quiz Questions
- Pourquoi Amazon a-t-il arrarrAnnual Recurring Revenue (ARR) is the normalized, predictable revenue a subscription business expects to earn from active contracts over a single year.View full definition →êté son algorithme de recrutement IA ?
A) Il était trop coûteux à opérer
B) Il pénalisait les CV contenant des références à des activités féminines, un exemple de biais lié aux données historiques d'embauche
C) Il n'était pas assez précis pour prédire la performance
D) Il violait les réglementations sur la protection des données
Réponse: B
- Qu'est-ce qu'un "feedback loop" problématique dans l'IA, illustré par l'exemple du predictive policing ?
A) Un modèle qui s'améliore trop vite et devient imprévisible
B) Un mécanisme où les prédictions du modèle influencent les données futures qui valident ces mêmes prédictions, amplifiant les biais existants
C) Un problème de performance lié au volume de données
D) Une régression du modèle due au manque de réentraînement
Réponse: B
- Dans l'EU AI Act, à quelle date la plupart des obligations pour les systèmes à haut risque (recrutement, scoring de crcrThe percentage of visitors or prospects who complete a desired action (purchase, sign-up, contact form), calculated as conversions divided by total opportunities.View full definition →édit) commencent-elles à s'appliquer ?
A) 1 août 2024, dès l'entrée en vigueur du règlement
B) 2 février 2025, en même temps que les interdictions de pratiques inacceptables
C) 2 août 2026, avec un délai étendu au 2 août 2027 pour l'IA intégrée dans des produits réglementés
D) Aucune date fixée, l'application reste volontaire
Réponse: C