A platform delivered is not a platform used
The scenario is familiar. An organisation invests in a modern foundation, loads most of its sources into it, then finds that the business teams carry on working from their own extracts. The data platform exists, it works, and it serves only a handful of users.
Three causes recur. The first is the order of operations: the foundation was built for “all the data”, with no priority use to set its requirements. The second is trust: loads that fail silently or tables without descriptions are enough to send users back to their spreadsheets. The third is operation: a platform delivered as a project, with no team to operate it, deteriorates from the first change to a source application.
A more recent trap adds to these. The vocabulary (lake, lakehouse, data mesh) is often used as an answer, whereas it refers to options that do not solve the same problems.
Warehouse, lake, lakehouse, data mesh: four options that are not mutually exclusive
The first three are technical architectures. The fourth is an organisational model, compatible with each of them.
| Option | Principle | Suitable when | Point of attention |
|---|---|---|---|
| Data warehouse | Structured data, modelled before being exposed, queried in SQL | Management and reporting dominate | Poorly suited to unstructured data and machine learning |
| Data lake | Raw files of any format on object storage, structure applied on read | Volumes are high and uses exploratory | Without a catalogue or rules, the lake becomes a dump that nobody can read |
| Lakehouse | Open table formats (Apache Iceberg, Delta Lake) on top of the lake’s storage, with transactions and schema management | The organisation wants to serve analysis and AI without maintaining two systems | Requires real engineering skills and discipline in table management |
| Data mesh | Each business domain publishes its data as products, on a self-service platform and under federated governance | A central team has become a bottleneck for numerous domains | Requires domain teams able to sustain their products over time |
The choice is not made on how modern the option is. It is made on four criteria: the nature of the uses (management reporting, exploration, models), the variety of the data, the skills available to operate the whole, and the degree of autonomy of the business domains. Our glossary entries detail the definition of the lakehouse and that of data mesh.
The four layers of a data platform
Whichever option is chosen, the whole is designed as a chain. A weakness in one layer cancels out the quality of the others.
Ingestion. Three modes coexist: batch loading, change data capture in the source databases and continuous event streaming. The right mode depends on the freshness the use really requires. Each source deserves an explicit contract: schema, frequency, owner, expected behaviour when it changes.
Storage and modelling. Common practice separates a raw layer, faithful to the source, a cleansed and conformed layer, and an exposed layer, modelled for the uses. This separation makes it possible to replay processing without calling on the source applications again.
Transformation. Business rules live in code that is versioned, reviewed and automatically tested, not in manual processing or scattered queries. This is the condition for a change to be traceable and reversible.
Exposure. Tables intended for reporting tools, interfaces for applications, datasets for data science teams: each access mode has its permissions, its documentation and its freshness commitments.
Our approach by use slices
We advise against delivering a complete foundation before the first use. Our consultants help your teams build a first vertical slice: one or two sources, the complete chain through to exposure, a real use and its users. This slice tests the architecture, sets the standards and serves as a model for the following ones.
Structuring decisions are taken early and recorded in writing: architecture model, environment layout, permissions management, modelling conventions. Each one mentions the options set aside. This record avoids reopening the same debates every time someone joins the team.
Automation is present from the first slice. Environments are described as infrastructure as code, transformations go through a continuous integration and deployment pipeline, tests run on every change. These are the practices of DevOps and platform engineering, applied to data.
Security and access management are designed together with the platform, based on the rules set by data governance: classification, role-based permissions, masking of sensitive data, access logging.
The mistakes we help you avoid
- Loading everything before serving a use. An exhaustive inventory of sources delays the first value and produces tables that nobody asked for.
- Choosing real time by default. A continuous stream is harder to build and operate than batch loading. It must meet a demonstrated need.
- Reproducing the old warehouse identically. A migration is an opportunity to remove what is no longer used. Taking over all existing processing carries the debt into the new foundation.
- Adopting data mesh without domain teams. Decentralising responsibility to business teams that have neither the time nor the skills moves the bottleneck instead of removing it.
- Discovering the cost after the fact. Without allocating spend by use, the bill grows with no possible trade-off. Our consultants help put this tracking in place from the start, following the principles of FinOps.
The signs of a platform that works
At the framing stage, we agree with you on a few observable indicators, tracked throughout the engagement.
- Usage: number of users and processes that are genuinely active, share of exposed tables that are actually consulted.
- Reliability: share of loads completed within the agreed time, time to detect and resolve a data incident.
- Delivery speed: time between the request for a new source or a new dataset and its availability.
- Cost clarity: share of spend attached to an identified use or team.
- Autonomy: the ability of your teams to add a source and fix an incident without our help.
The last indicator is the one that matters most at the end of the engagement. A platform that only the team that built it can evolve remains a project. It becomes a service when other teams use it, understand it and evolve it.