LLM data privacy: the retention settings legal misses

September 3, 2026 · 5 min · Free, with full transcript

Does a written privacy policy actually protect a company once employees and systems start sending data to language models? The position here is no. Exposure sits in architecture and behavior, in retention settings, vendor training terms, and embeddings that researchers have reversed back into readable text. The episode works through the Samsung 2023 source code incident and the 15 million euro fine Italy's regulator issued to OpenAI in late 2024. Listeners walk away with a mapping exercise: list every place data reaches a model, and for each record where it is stored, for how long, and whether it trains the model. The empty cells are the risk register.

Listen to the podcast

4 min

Chapters

Key takeaways

  • Stop asking whether you are compliant and start asking where the data goes and whether you can prove it was deleted.
  • List every place employees or systems send data to a model, and record storage location, retention period, and whether it trains the model.
  • Treat the gaps in that grid as your risk register, since most companies cannot fill in half of it.
  • Check the actual retention configuration in the account for your five highest volume model integrations, not the vendor marketing page.
  • Remember that a vendor promising not to train on your data may still retain it in abuse monitoring logs for 30 days.
Read the full transcript

Host:Welcome to the Leaders Insights podcast. Today's episode, data privacy, when everything goes to a model, the blind spots your legal team isn't catching. Everyone's got a data privacy policy. So why do you keep saying most companies are exposed and don't know it?

Expert:Because a policy governs what you meant to do. It doesn't govern what your architecture actually does. Combined with language models, the systems that generate text by predicting the next word from everything they've seen, the architecture leaks in ways the policy never anticipated.

Host:Give me a concrete leak, not a hypothetical.

Expert:Samsung 2023. Engineers pasted proprietary source code into chat GPT to debug it. That code became part of a conversation history sitting on a third party's servers. No policy said, don't paste our chip designs into a public model. Nobody thought they had to say it. That's the blind spot. The harm happened in a behavior no framework had a line for.

Host:But that's a training mistake, right? Tell people not to paste secrets. Problem solved.

Expert:That's exactly the compliance checkbox thinking that gets you burned. You can ban the obvious paste. You can't ban the thousand quieter versions. Your customer support agent summarizes a ticket that contains a home address. Your sales tool enriches a lead by feeding it into a model. Every one of those is a data transfer. And most companies have zero visibility into where that data physically lands.

Host:So the lesson the good operators already learned, what is it?

Expert:Data privacy is a property of your plumbing, not your paperwork. The best team stopped asking, are we compliant? And started asking, what, where does this data actually go? And can I prove it came back deleted? Those are different questions. The first one a lawyer answers. The second one only an engineer can.

Host:That sounds like you're telling legal teams they're useless here.

Expert:Not useless, out-ranged. A privacy lawyer reads contracts. But the exposure lives in retention settings in whether your vendor uses your inputs for training in whether embeddings, the numerical fingerprints a model makes of your text, can be reversed back into the original data. Yes, that's a real attack. Vendors have reconstructed readable text from embeddings people assumed were anonymous. Your lawyer has never heard the word embedding, and that's the gap.

Host:Okay, but the vendors all swear they don't train on your data now. Open AI, anthropic, the enterprise tiers. Isn't this solved?

Expert:On the enterprise tiers, mostly, if you configure them right. And that if is doing enormous work. The default consumer product is a different animal. But here's the part vendors won't put on a slide. We don't train on it is not the same as we don't store it. Data can sit in logs for abuse monitoring for 30 days. In a regulated industry, 30 days of retained patient records you didn't authorize is a reportable breach waiting to happen.

Host:You mentioned regulated industries. Is anyone actually getting fined for this yet? Or is it all theoretical fear?

Expert:The company's regulator fined OpenAI 15 million euros in late 2024 over exactly this, processing personal data without a proper legal basis and weak age checks. That wasn't a leak. That was the architecture and the defaults being wrong from the start. The enforcement is here. It's just not evenly distributed, and companies are betting they won't be first. Bad bet.

Host:So a mid-sized company hears all this and panics. What do they actually do Monday morning?

Expert:They don't panic, they map. One exercise, list every place employees or systems send data to a model. And for each one, write down three things. Where it's stored, how long, and whether it trains the model. Most companies can't fill in that grid for even half their tools. That empty grid is your risk register.

Host:And if they only do one thing this week?

Expert:Find your five highest volume model integrations and check the data retention setting on each. Set the marketing page, the actual configuration in the account. I'd wage your money at least one is on a default you never chose, retaining data you never meant to hand over. Fix that one setting today, and you've closed more real exposure than a quarter of policy revisions ever will.

Host:Done. Want to know where you actually stand? Take the AI self-assessment at MBA-training.com.

Read and go deeper