Definition of the data mesh
The data mesh is an organisational model for analytical data. Responsibility for data is entrusted to the business domains that produce it, not to a single central team. Formulated in 2019 by Zhamak Dehghani, it starts from an organisational observation more than a technological one.
The definition of the data mesh therefore refers neither to a tool nor to a ready-to-install architecture. It describes a distribution of responsibilities, supported by a common platform and shared rules.
What the data mesh is for
In the usual model, a central data team collects data from across the company, transforms it and serves it to users. This model works as long as sources and requests remain few. Beyond that, three symptoms appear.
- The queue grows longer: every new need goes through the same team.
- Business meaning is lost: those who transform the data are not those who understand it.
- No one is accountable for quality: producers consider that their data, once extracted, is no longer their concern.
The data mesh addresses this problem of scale by placing responsibility where the knowledge lies.
The four principles of the data mesh
1. Domain ownership. Each business domain (logistics, invoicing, customer relations) is responsible for its analytical data end to end: production, quality, availability.
2. Data as a product. The domain does not publish raw tables, but data products designed for their users. A data product can be found in a catalogue, is documented, comes with quality and freshness commitments, and is carried by an identified owner.
3. The self-service platform. A platform team provides the domains with common technical means: storage, processing, catalogue, access control, monitoring. The domains use them without having to build or operate the infrastructure. The logic is that of platform engineering, applied to data.
4. Federated governance. Common rules (shared identifiers, classification, security, interoperability) are decided by a body that brings together the domains and the platform. As far as possible, they are applied automatically by the platform rather than checked by hand.
These principles hold together. Without a platform, decentralisation multiplies costs. Without federated governance, it recreates silos.
Data mesh and centralised platform: not to be confused
| Centralised platform | Data mesh | |
|---|---|---|
| Owner of the data | The central data team | Each business domain |
| Role of the central team | Producing the datasets | Providing the platform and driving the rules |
| Unit of delivery | The pipeline, the table | The data product |
| Governance | Centralised | Federated, applied by the platform |
| Main risk | Bottleneck | Silos and inconsistencies |
Nor is the data mesh opposed to any given technical architecture. It can be implemented on a lakehouse as well as on a data warehouse: one describes who is responsible, the other how data is stored and queried.
The conditions for success of the data mesh
The model asks more of the organisation than of the technology. Five conditions weigh on the result.
- Sufficient scale. Several autonomous domains, many sources, a genuinely saturated central team. Below that, the cost of coordination exceeds the benefit.
- Equipped domains. Each domain must have data skills and a budget for its products.
- A mature platform before decentralisation. Decentralising without self-service amounts to asking each domain to rebuild the same infrastructure.
- Governance with real authority over common standards.
- Aligned objectives. The quality of data products appears in the objectives of the domains, not only in those of the data department.
The most common failure consists of renaming existing tables as “data products”, with no owner and no service commitment.
A concrete example: the “stock” data product
Take a retailer whose central data team feeds the sales and finance dashboards from the logistics applications. Every change to the applications breaks processes that the central team discovers after the fact.
In a data mesh organisation, the logistics domain itself publishes a “stock by warehouse” product: a documented schema, an announced freshness, quality checks, a named owner. When it changes its applications, it evolves its product and notifies its consumers. The central team no longer repairs the flows. It provides the platform on which this product is built, catalogued and secured.
For the technical foundation and the common rules, see our pages devoted to the data platform and to data governance.