Skip to content
Technology transformation expertise firm · ARDIANE Group Innovate, secure, succeed

Data

Data provenance vs data lineage, two traces for two requests

In brief

Data provenance vs data lineage: provenance describes the origin and history of a piece of data (source, collection conditions, versions, owner), lineage describes its journey through the information system and the transformations it undergoes. Management Square compares the two notions here, whose separation is a matter of convention, and indicates which requests each one answers.

Page contents

Data provenance is the information that establishes where a piece of data comes from and what its history has been: its source, the date and conditions of its collection, its successive versions, the person or organisation accountable for it. Data lineage describes the path it travels through the information system: the processing jobs that read it, transform it and write it, from the source application to the report or the model. The comparison data provenance vs data lineage thus sets an origin against a journey.

The first says whether a piece of data can be trusted and on what terms it can be used, the second where it has been and what a change would affect. This separation is a working convention. The two words, like their French equivalents provenance and lignage, are also used interchangeably: the meaning each counterpart gives them needs checking.

Two words that frameworks and tools divide up differently

The W3C PROV model gives provenance a broad meaning. Its data model, PROV-DM, a recommendation of 30 April 2013, defines it as “information about entities, activities, and people involved in producing a piece of data or thing, which can be used to form assessments about its quality, reliability or trustworthiness”. It is organised around three notions, the entity, the activity and the agent. Among its relations, derivation covers the transformation of one entity into another and attribution links an entity to an agent. In the W3C sense, provenance therefore also encompasses transformations.

The OpenLineage standard, a project of the LF AI & Data foundation, presents itself as an open framework for collecting and analysing lineage. Its specification describes three objects, the dataset, the job and the run, enriched by “facets”, one of which concerns column-level lineage. Events are emitted when jobs run; others describe a job or a dataset from the moment it is designed.

The convention adopted in this article assigns origin and history to provenance, journey and transformations to lineage. It has a practical benefit: the two kinds of information are not built up in the same way and are not maintained by the same people.

Data provenance vs data lineage: the comparison table

Data provenance Data lineage
What is described The origin and history of a dataset or a record The journey between systems and the transformations applied
Typical request Where does this data come from, who produced it, on what terms can it be used? Which processing jobs has this figure gone through, and what does a change to this column affect?
Content Source, date and method of collection, original purpose, licence or contract, versions, owner Input and output datasets, jobs, runs, calculation rules
Granularity The dataset, sometimes the record The table, or the column when the tooling allows it
How it is built Declared at acquisition or collection, completed at each version Extracted from the transformation code or emitted by the jobs
Maintained by The data owner and the data steward The team running the platform, by means of the tooling
Limit Silent on what becomes of the data once loaded Silent on the reliability of the source and on the right of use

The “How it is built” row governs the organisation. Provenance is declared: no tool can guess the licence of a purchased file or the purpose for which personal data was collected. Lineage is calculated, from the code or from the events emitted by the jobs, which keeps it up to date without manual entry.

When the request concerns origin and the right of use

Provenance answers when a piece of data has to be justified to someone who has no need to know the information system.

For data purchased or received from a partner, the request concerns the contract, the permitted scope of use and the extraction date. Between two sources giving the same information, it concerns which one is the reference and the reason for that choice.

For personal data, the GDPR, Regulation (EU) 2016/679, makes origin a matter of information. Where the data has not been collected from the data subject, its Article 14(2)(f) requires the data subject to be told “from which source the personal data originate”. Its Article 15(1)(g) gives them access to “any available information as to their source”.

For high-risk AI systems, the AI Act, Regulation (EU) 2024/1689 as amended by Regulation (EU) 2026/1744, refers in Article 10(2)(b) to “data collection processes and the origin of data, and in the case of personal data, the original purpose of the data collection”. Its Annex IV, point 2(d), expects the technical documentation to include information on the provenance of training data sets. The application timetable for these provisions is set out in Article 113 and should be read in the consolidated version in force.

In these situations, a dependency graph is not enough. It shows that a table comes from a file, without indicating who supplied that file or on what terms.

When the request concerns the journey and the effect of a change

Lineage answers the teams that operate, modify or explain data.

To explain a disputed figure, it makes it possible to trace back from the indicator to the tables and calculation rules that produce it. This is one of the examinations in a data quality audit.

Before modifying a column or replacing a source application, it gives the list of reports, models and extracts that depend on it, and therefore of the people to notify. During a loading incident, the same downstream reading identifies the products affected.

For a request to erase or rectify personal data, it indicates the tables into which the data may have been copied. Before decommissioning a table, it shows what still consumes it.

A provenance record, however complete, says nothing about what happens after loading.

The uses that require both

Justifying a regulatory report to an auditor means showing the journey of the figure and, at the end of that journey, sources whose origin is established. Documenting an AI model means knowing the provenance of the training sets and the chain that prepared them.

The two kinds of information meet at one point: the entry node of the lineage graph, that is, the dataset as it arrives in the platform. This is where the provenance record attaches.

The division of work follows from this. The provenance record belongs to the catalogue. It is filled in at the entry of each dataset by a steward, under the authority of the owner, according to the roles described in our page data governance. Lineage is produced by the data platform, from the code of its transformations or from the events its jobs emit. According to the project’s documentation, OpenLineage integrations exist for tools such as Apache Airflow, Apache Spark or dbt.

Before choosing a tool

  • The request to be served is named: an auditor, a data protection officer, a team preparing a change.
  • The meaning given to “provenance” and “lineage” is written down, and compared with that of the vendor or integrator consulted.
  • Each dataset of external origin has an owner able to state its source, its terms of use and its date.
  • Transformations are written in versioned code or run by jobs that emit lineage events, failing which the graph cannot be built automatically.
  • The required granularity is decided: the table or the column.
  • One person checks, at a set interval, that the graph matches the jobs in service.

FAQ

Frequently asked questions

Another question? Ask us

What is the difference between data provenance and data lineage?

Data provenance describes the origin and history of a piece of data: its source, the conditions of its collection, its versions and the person accountable for it. Data lineage describes its journey through the information system and the transformations it undergoes on the way to the report or the model. The first is used to judge reliability and the right of use, the second to explain a figure and measure the effect of a change. This separation is a convention: the two words are also used interchangeably.

Are data provenance and data lineage synonyms?

The two expressions are sometimes used interchangeably. The W3C PROV model gives provenance a broad meaning, which includes transformations under the heading of derivation, whereas the OpenLineage standard is devoted to collecting the lineage of processing jobs. It is therefore wise to write down, in each project, the meaning adopted for each of the two words.

What is the W3C PROV model?

PROV is a family of W3C documents devoted to the exchange of provenance information. Its data model, PROV-DM, a recommendation of 30 April 2013, rests on three notions, the entity, the activity and the agent, linked by relations such as generation, derivation and attribution. The PROV-O ontology gives an expression of it in RDF.

Does the European AI regulation require data provenance to be documented?

For high-risk AI systems, Article 10(2)(b) of Regulation (EU) 2024/1689 lists the origin of data and the collection processes among the governance practices for training, validation and testing data sets. Annex IV, point 2(d), requires the technical documentation to include information on the provenance of training data sets. The application timetable can be read in Article 113 of the consolidated version, amended by Regulation (EU) 2026/1744.

Which should you start with, provenance or lineage?

With the one a pending request requires. An inspection on the origin of personal data or on a model's training sets calls first for provenance. Disputed figures or source changes with unforeseen effects call first for lineage, limited to the chains concerned.

A project to bring to completion?

A first conversation is enough to understand your challenge, identify the right people and tell you frankly what we can bring.