Skip to content
Technology transformation expertise firm · ARDIANE Group Innovate, secure, succeed

Glossary

Lakehouse: definition and differences with the data warehouse and the data lake

Domain

Data

In brief

The lakehouse is a data platform architecture that combines, on a single foundation, the open and loosely constrained storage of the data lake with the guarantees of the data warehouse: reliable transactions, controlled schemas and high-performance queries. It aims to serve BI, data science and machine learning from a single copy of the data.

Page contents

Definition of the lakehouse

The lakehouse is a data platform architecture that combines, on a single foundation, the properties of the data warehouse and those of the data lake. The word is a contraction of the two terms. It spread from 2020 onwards.

The definition of the lakehouse comes down to two layers. Data is stored in open-format files, on object storage, as in a lake. A table management layer then brings what characterises a warehouse: reliable transactions, controlled schemas and high-performance queries.

What a lakehouse is for

Many organisations run two systems in parallel. A lake receives raw data and serves data science. A warehouse receives part of this data, reprocessed, and serves reporting. This two-tier architecture comes at a price: copies to synchronise, discrepancies between the figures of the two worlds, delays and a double bill for storage and compute.

The lakehouse aims to serve all uses from a single copy of the data: dashboards, exploratory analysis, model training, streaming processing. The expected benefit is consistency more than technical novelty.

How a lakehouse works

Four components are stacked.

  • Object storage, which keeps all files at low cost and with no practical volume limit.
  • An open columnar file format, such as Apache Parquet, suited to analytical queries.
  • An open table format, such as Delta Lake, Apache Iceberg or Apache Hudi. It keeps a transaction log on top of the files. It is this format that brings ACID transactions, schema evolution and the ability to read a table in an earlier state.
  • A catalogue, which lists the tables, holds access rights and, depending on the tools, data lineage.

Compute is separated from storage. Several engines (SQL, distributed processing, machine learning) read and write the same tables, and each is sized independently. Data is often organised in successive layers, from raw to ready to use, following a so-called medallion pattern.

Lakehouse, data warehouse, data lake: not to be confused

Data warehouse Data lake Lakehouse
Data accepted Structured All types, in raw form All types
Schema Enforced on write Applied on read Controlled by the table format
Reliable transactions Yes No Yes
Storage format Often vendor-specific Open Open
Main uses Reporting, BI Data science, archiving BI, data science, machine learning

The boundary is blurring. Data warehouses can now read and write open table formats, and lakehouse platforms have strengthened their SQL engines. The useful question is no longer the product label. It concerns three points: the format in which your data is stored, the engines able to read it, and the layer that carries governance.

The lakehouse is, moreover, a technical architecture. It says nothing about the allocation of responsibilities for data, which is the subject of the data mesh.

The limitations of the lakehouse

  • It does not solve governance. Without an up-to-date catalogue or quality control, a lakehouse becomes a disorderly data lake with an SQL engine.
  • It requires maintenance. Compacting small files, purging old versions, physical organisation of tables: these operations determine performance.
  • It assumes engineering skills broader than those needed for a fully managed warehouse.
  • The choice of table format and catalogue is a commitment. The promised openness depends on the actual compatibility between the engines you use.
  • It is oversized for a simple need. A moderate volume of structured data, intended for reporting alone, can make do with a warehouse.

A concrete example: a single source for BI and models

Take an industrial group whose financial control team works on a data warehouse, while the data science team trains its maintenance models on sensor readings stored in a lake. The two teams each calculate a machine availability rate, and the figures diverge, because they do not start from the same data and the same rules.

With a lakehouse, raw readings, cleaned data and reference tables reside in the same storage, under the same catalogue. The dashboard and the model read the same availability table. When a calculation rule changes, it changes for everyone. The version history also makes it possible to reconstruct the state of the data used to train a model, a point useful to MLOps practices.

For the choice of architecture and its implementation, see our page dedicated to the data platform.

FAQ

Frequently asked questions

Another question? Ask us

What is a lakehouse?

A lakehouse is a data architecture that stores data in open-format files, like a data lake, and adds a table management layer that brings the guarantees of a warehouse: transactions, controlled schemas and good query performance.

What is the difference between a lakehouse and a data warehouse?

The data warehouse stores structured data in a format that is most often specific to the vendor and mainly serves reporting. The lakehouse keeps all types of data in open formats, readable by several engines, and serves reporting, data science and machine learning alike.

What is the difference between a lakehouse and a data lake?

The data lake stores raw files with no guarantee of consistency and no schema control. The lakehouse adds a table format that keeps a transaction log, which enables reliable updates, controlled schema evolution and SQL querying with good performance.

Does a lakehouse always replace the data warehouse?

No. For moderate volumes of structured data intended for reporting, a data warehouse remains a simple and sufficient choice. The lakehouse becomes worthwhile when varied uses, data in diverse formats and a need for openness to several processing engines coexist.

A project to bring to completion?

A first conversation is enough to understand your challenge, identify the right people and tell you frankly what we can bring.