Recommendation systems: architectures & ethical personalization
Every organization that serves more than a few hundred customers is already missing personalization opportunities. Manual segmentationsegmentationDividing a market into distinct groups of customers who share similar needs, characteristics or behaviours, so each group can be served with a tailored approach.View full definition → (5-10 customer segments, each receiving the same experience) leaves significant value on the table. AI-driven personalization, treating each customer as a segment of one, is the standard at leading consumer companies and increasingly accessible to all.
The personalization spectrum
Rule-based personalization: Explicit rules ("if customer is in segment X and has product Y, show message Z"). Transparent, explainable, but doesn't scale, you can't write rules for every situation.
ML-based personalization: Models learn patterns from behavior and context to predict the right content, product, or message for each individual. Scales to millions of users. Netflix, Amazon, and Spotify operate entirely in this tier.
Generative personalization: LLMs generate personalized content (emails, product descriptions, explanations) adapted to individual context. Still emerging but increasingly practical.
Most organizations should be targeting ML-based personalization. Rule-based is a starting point, not a destination.
Building Recommendation Systems
Knowledge check
1. What is the main limitation of rule-based personalization that makes it unsuitable as a final destination for most organizations?
2. A streaming service wants to recommend items to a brand-new user who has no interaction history. Which approach best addresses this challenge?
3. What best describes the core mechanism of a two-tower neural network for recommendation?
4. Select ALL correct statements about the personalization spectrum described in the lesson.
Select all the correct answers.
5. Select ALL correct statements distinguishing collaborative filtering from content-based filtering.
Select all the correct answers.
Recommendation system architectures
Collaborative filtering: Recommends based on what similar users liked. "Users like you also enjoyed X." Works well with abundant data, but suffers from the cold-start problem (no recommendations for new users with no history).
Content-based filtering: Recommends based on item attributes. "You liked documentaries about climate change, here are more." Requires rich item metadata. Doesn't require user history (solves cold-start). Less serendipitous.
Matrix factorization: Decomposes the user-item interaction matrix into latent factors representing user preferences and item characteristics. Efficient at scale. Used by Netflix (prize-winning approach), Spotify (Discover Weekly).
Two-tower neural networks: The current state of the art for large-scale recommendation. One "tower" encodes users, one encodes items. Items whose vectors are closest to the user's vector are recommended. Google, YouTube, Pinterest use this architecture.
Session-based recommendations: Recommends based on the current session context, not historical behavior. Critical for anonymous users or highly context-dependent recommendations (what to buy alongside the item currently in cart).
The e-commerce personalization case
Amazon attributes 35% of revenue to recommendation systems. Their approach covers every surface: homepage ("Based on your browsing"), product detail page ("Customers also bought"), email ("You might like"), and checkout ("Frequently bought together").
The infrastructure behind this: user behavior events (views, clicks, purchases) stream in real-time to a feature storefeature storeA centralised repository managing ML features, ensuring consistency between training and serving environments.View full definition →. Candidate items are retrieved using embeddings (approximate nearest neighbor search). Candidates are ranked using a real-time scoring model that factors in price, inventory, relevance, and margin. The top-KKThe average number of new users each existing user generates through referrals. Above 1.0, growth compounds on itself and becomes exponential.View full definition → items are displayed.
The entire pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → runs in under 100ms for each page load.
Personalization ethics and fairness
Personalization raises ethical questions that CDOs must address:
Filter bubbles: Recommending only what users already like reduces exposure to new ideas. In news and information contexts, this has societal implications. Build intentional diversity into recommendation systems.
Manipulation vs. relevance: Optimizing for engagement (time-on-site, clicks) can lead to addictive, harmful content promotion. Define the optimization objective carefully. At Netflix, the primary metric is long-term subscriber satisfaction, not immediate engagement.
Transparency: Users should understand why they're seeing recommendations. "Because you watched X" is a simple but powerful transparency mechanism that also builds trust.
Protected characteristics: Personalization must not discriminate based on race, gender, religion, or other protected characteristics. Systems that correlate proxy variables with protected characteristics can produce discriminatory outcomes without explicit discrimination.
Quiz Questions
- Quel est le "problème du cold-start" dans les systèmes de recommandation par filtrage collaboratif ?
A) Le système est trop lent au démarrage
B) L'impossibilité de générer des recommandations pertinentes pour les nouveaux utilisateurs sans historique comportemental
C) Le manque de données pour les nouveaux produits
D) Les recommandations sont trop similaires entre utilisateurs
Réponse: B
- Quelle architecture de recommendation est considérée comme l'état de l'art pour les systèmes à grande échelle comme YouTube ou Google ?
A) Filtrage collaboratif classique
B) Factorisation matricielle
C) Two-tower neural networks (un tower pour l'utilisateur, un pour l'item)
D) Filtrage basé sur le contenu
Réponse: C
- Pourquoi optimiser uniquement pour l'engagement (temps passé, clics) peut-il être problématique dans les systèmes de recommandation ?
A) Cela augmente trop les coûts de calcul
B) Cela peut favoriser du contenu addictif ou nuisible, crcrThe percentage of visitors or prospects who complete a desired action (purchase, sign-up, contact form), calculated as conversions divided by total opportunities.View full definition →éererThe ratio of interactions (likes, comments, shares) to reach for a given piece of content, used to gauge how well audiences respond relative to how many people saw it.View full definition → des bulles de filtre, et nuire à la satisfaction à long terme des utilisateurs
C) Cela rend le système moins précis
D) Cela viole automatiquement les réglementations RGPDRGPDEU regulation governing how organizations collect, store and use personal data, with fines tied to global revenue for breaches.View full definition →
Réponse: B
Related articles
Recent articles from the blog that build on this lesson.
- DataEveryone assumes the flywheel spins itself: Amazon's data advantage took twelve years of deliberate engineering to compoundAmazon's data flywheel is cited constantly as proof that more data automatically produces better outcomes. The reality is that the compounding happened because of specific architectural decisions, feedback loop designs, and organizational choices made over more than a decade.
- DataThe data flywheel field guide: who built compounding advantage and what they actually didThe data flywheel is one of the most cited concepts in data strategy, and one of the least examined in practice. This field guide cuts through the abstraction and names the companies and moments that show what compounding data advantage actually looks like when it works.