Every CDO has faced this question: "What should our data architecture look like?"
The answer has evolved dramatically over the last decade. The monolithic data warehousedata warehouseA central repository that consolidates data from many source systems into a structured, query-optimized store designed for analytics, reporting, and business intelligence.Voir la définition complète →, once the gold standard, is now just one option among many. To make the right architectural choices, you need to understand what's available, what trade-offs each option carries, and how to match architecture to business need.
The modern data stack has layered complexity on top of simplicity. Here's what exists today:
Data warehouse: Structured, SQLSQL-queryable storage optimized for analytics. Examples: Snowflake, BigQuery, Redshift. Best for: , dashboards, structured reporting.
Data lake: Raw storage at scale, structured, semi-structured, and unstructured data. Examples: AWS S3, Azure Data LakeData LakeA data lake is a centralized repository that stores large volumes of raw data in its native format, from structured tables to unstructured files, until needed.Voir la définition complète →, GCS. Best for: storing everything, enabling future use cases.
Data lakehouse: The hybrid that emerged from the friction between lakes and warehouses. ACID transactions, schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.Voir la définition complète → enforcement, and SQLSQLSales Qualified Lead: a prospect the sales team has validated as ready for direct outreach and a proposal, having passed clear qualification criteria.Voir la définition complète → queries on lake storage. Examples: Databricks Delta Lake, Apache Iceberg, Apache Hudi.
Data mesh: Not a technology, an organizational and architectural paradigm. Data ownership distributed to domain teams. Each domain produces data products. Federated governance. More on this in Module 3.2.
Data fabric: An architecture layer that connects disparate data sources through metadatametadataDonnées sur les données, informations décrivant le contexte, la structure, la provenance et les caractéristiques d'un asset de données (auteur, date, format, source, définition). and automated integration. Less about storage, more about connectivity and discovery.
Vérification des acquis
1. What fundamentally distinguishes a data lakehouse from a traditional data lake?
2. A retail company's primary need is fast, reliable BI dashboards with SLA commitments to business stakeholders, and their data is well-structured with a SQL-native team. Which architecture fits best?
3. Why is a data mesh described as fundamentally different from a warehouse, lake, or lakehouse?
4. Select ALL scenarios where choosing a data lake is the most appropriate decision.
Sélectionnez toutes les réponses correctes.
5. Select ALL statements that correctly describe when a lakehouse is the right architectural choice.
Sélectionnez toutes les réponses correctes.
The warehouse-lake-lakehouselakehouseA hybrid architecture combining the flexibility of a data lake with the analytical capabilities of a data warehouse, on a single storage layer.Voir la définition complète → debate is still active in every data team. Here's how to think about it:
Choose a data warehouse when: Your primary use case is BIBITechnologies and processes that turn raw data into actionable insights via reporting, dashboards and analysis, so teams can decide based on facts rather than intuition.Voir la définition complète → and reporting, your data is structured, your team is SQLSQLSales Qualified Lead: a prospect the sales team has validated as ready for direct outreach and a proposal, having passed clear qualification criteria.Voir la définition complète →-native, and you need guaranteed query performance with SLA commitments to business stakeholders.
Choose a data lake when: You have massive unstructured data (logs, clickstreams, images, documents), you want to preserve raw data for future ML use cases, or storage cost is a primary concern.
Choose a lakehouse when: You want the flexibility of a lake with the reliability of a warehouse. You're a data-intensive organization doing both ML and BIBITechnologies and processes that turn raw data into actionable insights via reporting, dashboards and analysis, so teams can decide based on facts rather than intuition.Voir la définition complète →. You can't afford to maintain two separate systems. This is increasingly the default for data-mature organizations.
The lakehouselakehouseA hybrid architecture combining the flexibility of a data lake with the analytical capabilities of a data warehouse, on a single storage layer.Voir la définition complète → pattern emerged from a specific pain: organizations built data lakes, discovered they were unusable swamps of unstructured data, and needed warehouse-like reliability without abandoning their lake investments.
Delta Lake (Databricks), Apache Iceberg, and Apache Hudi solved this by adding ACID transactions, schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.Voir la définition complète → evolution, and time-travel capabilities to object storage. The result: lake storage costs with warehouse query reliability.
Netflix migrated their entire data infrastructure to Apache Iceberg. They manage hundreds of petabytes of data with schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.Voir la définition complète → evolution, rollback capability, and concurrent read-write support, capabilities previously only available in expensive proprietary warehouses.
When evaluating architecture options, ask four questions:
There is no universally correct answer. A well-reasoned architecture that matches your actual workloads beats a sophisticated architecture that your team can't operate.
1. Quelle est la principale différence entre un data lakedata lakeA data lake is a centralized repository that stores large volumes of raw data in its native format, from structured tables to unstructured files, until needed.Voir la définition complète → et un data lakehousedata lakehouseA hybrid architecture combining the flexibility of a data lake with the analytical capabilities of a data warehouse, on a single storage layer.Voir la définition complète → ?
A) Le lakehouselakehouseA hybrid architecture combining the flexibility of a data lake with the analytical capabilities of a data warehouse, on a single storage layer.Voir la définition complète → est plus cher
B) Le lakehouselakehouseA hybrid architecture combining the flexibility of a data lake with the analytical capabilities of a data warehouse, on a single storage layer.Voir la définition complète → ajoute des transactions ACID et la fiabilité d'un warehouse sur un stockage de type lake
C) Le lakehouselakehouseA hybrid architecture combining the flexibility of a data lake with the analytical capabilities of a data warehouse, on a single storage layer.Voir la définition complète → ne supporte pas le SQLSQLSales Qualified Lead: a prospect the sales team has validated as ready for direct outreach and a proposal, having passed clear qualification criteria.Voir la définition complète →
D) Le lakehouselakehouseA hybrid architecture combining the flexibility of a data lake with the analytical capabilities of a data warehouse, on a single storage layer.Voir la définition complète → est uniquement pour les données non structurées
Réponse: B
2. Quelle architecture choisir en priorité pour une équipe BIBITechnologies and processes that turn raw data into actionable insights via reporting, dashboards and analysis, so teams can decide based on facts rather than intuition.Voir la définition complète → SQLSQLSales Qualified Lead: a prospect the sales team has validated as ready for direct outreach and a proposal, having passed clear qualification criteria.Voir la définition complète →-native avec des données structurées ?
A) Data lakeData lakeA data lake is a centralized repository that stores large volumes of raw data in its native format, from structured tables to unstructured files, until needed.Voir la définition complète →
B) Data meshData meshData Mesh is a decentralized approach to data architecture and organization where domain teams own and serve their data as products, governed by shared standards.Voir la définition complète →
C) Data warehouseData warehouseA central repository that consolidates data from many source systems into a structured, query-optimized store designed for analytics, reporting, and business intelligence.Voir la définition complète →
D) Data fabricData fabricCouche d'intégration unifiée reliant toutes les sources de données via des métadonnées intelligentes, alternative centralisée au data mesh.
Réponse: C
3. Quel projet open source a permis à Netflix de gérer des centaines de pétaoctets avec des capacités ACID ?
A) Apache Kafka
B) Apache Hudi
C) Apache Iceberg
D) Delta Lake
Réponse: C
Ces actions sont compilées dans le plan d'action du rôle.