Consent, purpose limitation, and the AI feature trap

Feeding this corpus into a large language modellarge language modelA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → (LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition →) fine-tuningfine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition → job, or even using it for retrieval-augmented generation (RAG, where a model pulls live snippets from a database to answer questions), is a new processing purpose. If your original privacy notice said "to provide customer support," you likely do not have a lawful basis to also say "to train our AI features."

Why this is not just an EU problem

US companies often assume this is a European-only headache. It is not.

• The FTC (Federal Trade Commission) has explicitly warned companies against "silently" changing terms to allow AI training on previously collected data, calling it a potentially unfair or deceptive practice under Section 5 of the FTC Act. Their 2024 blog post on the topic is a useful primer: FTC: AI (In)security.
• California's CCPA/CPRA (California Consumer Privacy Act, amended by the California Privacy Rights Act) gives users a right to limit use of sensitive personal information and requires disclosure of new processing purposes.
• Sector rules stack on top: HIPAA (health data, US) and GLBA (financial data, US) restrict secondary use regardless of AI involvement.

The common thread across the EU, US, and most other frameworks: the problem is not AI itself, it is silent repurposing of data collected under a narrower promise.

What "purpose limitation" actually requires

Two lawful paths exist once you want to reuse data for a new purpose:

1. Compatibility test: Some regulators (notably under GDPR Article 6(4)) allow reuse if the new purpose is "compatible" with the original one, weighing factors like context, reasonable user expectations, and safeguards applied. Training an AI on aggregated, anonymized ticket categories to improve routing might pass. Training a generative copilot that could regurgitate a user's exact complaint text to another customer almost certainly does not.

2. Fresh consent or new lawful basis: If not compatible, you need a new legal basis, most commonly renewed, specific, informed consent, or a legitimate interest assessment (LIA) that is documented and defensible.

Anonymization matters here but is often overstated. True anonymization (irreversible, no re-identification possible) removes GDPR applicability entirely. Pseudonymization (replacing identifiers with tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition →) does not; pseudonymized data is still personal data under EU law if re-identification is feasible.

Building the re-permissioning workflow

When product wants to ship an AI feature on top of existing data, run this sequence before a single training job starts.

Step 1: Data mapping. Identify every dataset the feature will touch (tickets, chat transcripts, usage logs) and its original stated purpose. Tools like a data inventory or a Records of Processing Activities (RoPA, a GDPR-mandated log of what data you process and why) make this traceable.

Step 2: Compatibility assessment. Document, in writing, whether the AI use case is compatible with the original purpose. The UK Information Commissioner's Office (ICO) publishes a practical compatibility checklist worth adapting: ICO guidance on purpose limitation.

Step 3: Choose the basis.

• Compatible and low-risk (e.g., aggregated analytics) → proceed with a Data Protection Impact Assessment (DPIA) if risk is non-trivial.
• Incompatible or high-risk (e.g., raw ticket text into an LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition →) → trigger re-permissioning.

Step 4: Re-permissioning campaign. This is the operational core:

• Update the privacy notice with plain-language description of the AI use case.
• Present an explicit, granular opt-in (not a bundled "I agree to updated terms" checkbox). Regulators increasingly reject bundled consent as not "freely given."
• Offer a real opt-out that does not degrade core product function, since GDPR requires consent to be as easy to withdraw as to give.
• Log consent with timestamp, version of notice shown, and mechanism, and keep this log for audit purposes.

Step 5: Technical enforcement. Consent decisions must actually reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → the data pipelinedata pipelineETL (Extract, Transform, Load) is a data integration process that pulls data from sources, reshapes it into a consistent format, and writes it into a target system.View full definition →. A common failure: legal approves an opt-in flow, but engineering trains on the whole ticket table regardless because there is no flag propagated downstream.

A minimal schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → fix looks like this:

Without this kind of enforced filter, consent is a legal fiction: it exists in a policy document but not in the pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition →.

Audits to run before and after launch

• Pre-launch DPIA: mandatory under GDPR when processing is "likely to result in high risk," which training generative models on user content typically qualifies as.
• Data minimization check: does the model need raw ticket text, or would summaries/structured fields suffice?
• Leakage test: prompt the trained model to see if it reproduces verbatim customer text (a known LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → memorization risk).
• Consent coverage audit: quarterly check that the percentage of training data with valid, current consent matches what compliance believes it to be.
• Vendor flow-down: if a third-party AI vendor (OpenAI, Anthropic, Google) processes the data, verify your Data Processing Agreement (DPA) explicitly restricts their own model training on your customers' data.

The business tension, honestly stated

Re-permissioning is friction, and friction reduces the training data pool. Product teams often push back: "If we ask again, adoption drops and the copilot is worse." That tradeoff is real, but the alternative is regulatory exposure. Clearview AI, Meta, and Clarivate have all faced regulatory action or major fines tied to repurposing personal data without adequate basis; the amounts and rulings vary by jurisdiction and case, but the pattern is consistent enough to take seriously.

The better framing for product teams: build re-permissioning into the feature announcement itself. "We're launching an AI assistant trained on support patterns. Here's exactly what it uses, here's your control" converts a compliance obligation into a trust signal, which is itself a retention lever in enterprise SaaS deals where procurement teams now routinely ask about AI training data provenancedata provenanceData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.View full definition →.

🎬 [VIDEO: "GDPR and AI: Purpose Limitation Explained" - youtube.com - search for IAPP (International Association of Privacy Professionals) or ICO channel explainers on purpose limitation and AI training data, useful for a visual walkthrough of the compatibility test]

Key Takeaways

• Purpose limitation (GDPR Article 5(1)(b)) means data collected for support cannot automatically be reused to train AI features; this is a live enforcement area in the EU and increasingly at the FTC and under CCPA/CPRA in the US.
• Anonymization removes the legal problem only if it is irreversible; pseudonymized data is still regulated personal data.
• A real re-permissioning workflow has five steps: data mapping, compatibility assessment, choosing a lawful basis, running an explicit opt-in campaign, and enforcing consent flags in the actual data pipelinedata pipelineETL (Extract, Transform, Load) is a data integration process that pulls data from sources, reshapes it into a consistent format, and writes it into a target system.View full definition →, not just in policy documents.
• Run a DPIA before training, test for verbatim memorization after training, and audit consent coverage quarterly.
• Treat re-permissioning as a trust and sales asset, not just a legal cost, especially for enterprise SaaS where buyers now scrutinize AI training data provenancedata provenanceData lineage maps how data moves and transforms across systems, from origin to consumption, showing where it came from, what changed it, and where it goes.View full definition → during procurement.