Skip to content
Technology transformation expertise firm · ARDIANE Group Innovate, secure, succeed

Data

A data quality audit traces symptoms back to their causes

In brief

A data quality audit is a bounded examination, commissioned by a business department, which measures the state of the data consumed by a use, traces each defect back to its cause and delivers a remediation plan ranked by effort. Management Square describes here the symptoms that justify it, the examinations it carries out in sequence and the decision with which it ends.

Page contents

When the revenue in a dashboard differs from the figure in the accounts, the discrepancy can be corrected once. If it comes back at every close, without anyone being able to explain it or say which of the two figures to trust, correcting the month’s discrepancy is no longer enough. The matter calls for a data quality audit.

A data quality audit is a bounded examination, commissioned by a business department, covering the data of a specific use. It measures their state, traces each defect found back to the point where it arises, then delivers a remediation plan ranked by effort. Permanent checks report defects as they occur; the audit looks for why they appear and why they come back after correction. It concludes with a decision by the sponsor on each remedy.

What leads a department to request a data quality audit

An isolated defect is corrected without an audit. The examination is justified when defects recur and nobody can say where they come from. Several situations can lead to it, alone or together.

  • Two reports give two values for the same indicator, and the meeting is about the figure rather than the decision.
  • Teams manually rework the same extracts at every deadline before using them.
  • A correction made in the data warehouse has to be redone at the next load.
  • Errors are reported by users or by customers: a duplicate invoice, a letter sent to the wrong address, a reminder issued wrongly.
  • A migration, master data or AI project is about to consume data whose state has never been measured.
  • A third party asks for a figure to be justified: a supervisory authority, a statutory auditor, a customer.

When the origin of the defect is already established, for example a field that the application leaves optional, the audit is not necessary: the remedy is funded directly.

The sponsor matters as much as the symptom. Commissioned by the department that suffers from the discrepancy, the audit starts from a use and leads to a decision that department can take. Commissioned by the data team on its own account, it risks describing tables without anyone having the authority to act on the source applications.

A scope set by a use and by its expectations

An audit that targeted all of the company’s data could not reach a conclusion. The scope comes down to three elements: a use (an indicator, a report, a processing job, a model), the data it consumes and the level that is sufficient for it.

This level is written down before any measurement. A rate of empty fields acceptable for a commercial segmentation can block a regulatory filing. Without an expectation formulated by the business, the audit counts anomalies without being able to say which ones matter. The dimensions on which an expectation is expressed, from accuracy to freshness, are described in our data quality page, whose “Profiling and assessment” service measures the actual state of critical data: this is the first of the examinations described below.

When the data is personal, the expectation also has a legal source. Article 5(1)(d) of the GDPR, Regulation (EU) 2016/679, requires data to be “accurate and, where necessary, kept up to date”. The data protection officer is then involved in the framing, and the audit’s extracts follow production access rules.

From the measured defect to the point where it arises

The audit carries out four examinations in sequence, each answering a different question.

Profiling establishes the state of the use’s critical data: missing values, duplicates, invalid formats, inconsistencies between systems. It shows what is defective and in what proportion, without giving the reason.

Examining the pipeline follows the data backwards, from the report to the source. Counts and totals are compared at each step, up to the one where the discrepancy appears: a filter, a join, a business rule, a partial load. Up-to-date lineage shortens this work, as explained in our article on data provenance and lineage.

Examining the source covers the application and the process that produce the data: screens and mandatory fields, lists of values, interfaces, past data migrations, instructions given to the people entering data. It is conducted at the workstation as much as in the documentation.

Examining responsibilities checks who is accountable for the data, who receives the alerts and who can have the source application modified. Data without an owner is a cause in itself: defects are observed without anyone having to deal with them.

Symptom Check carried out Causes to distinguish between
Two values for the same indicator Comparison of the definitions, filters and cut-off dates of the two pipelines Two business definitions, a different scope, a calendar offset
Duplicate customers or suppliers Matching of records, examination of the creation circuit Creation without prior search, several creating applications, an old data migration
Empty fields or default values Fill rate by entry channel and by period Optional field on screen, information unavailable at the time of entry, interface that does not pass the field on
Correction to be redone at each load Reading of the transformations and their order Correction made downstream while the source remains unchanged
Figure that varies with no business event Comparison of volumes and totals at each step Partial or doubled load, join that multiplies rows

A cause is adopted only if it is demonstrated: a reproduced case, a count before and after the suspect step, a screen shown to the sponsor. The report also mentions what the audit could not establish.

A remediation plan ranked by effort

The report links each remedy to a demonstrated cause and ranks them all from the least costly to the most demanding.

  1. Decision remedies: settling a definition, writing an expectation and its threshold, appointing an owner. They require no development.
  2. Configuration remedies in the source application: a field made mandatory, a list of values, a duplicate search at creation.
  3. Remedies in the processing pipeline: correcting a transformation, adding a check at the point where the defect becomes detectable.
  4. Reworking the stock: correcting existing records, once the source has been dealt with. Carried out before, it has to be started again.
  5. Structural remedies: redesigning an interface, master data management, replacing an application. These are projects, for which the audit merely establishes the need.

For each line, the plan states the cause addressed, the use that benefits, the intended owner and the dependencies. The effort ranking is combined with the expected effect: a heavy remedy moves to the top if a regulatory deadline depends on it.

Where the audit stops

The audit delivers a dated snapshot and a plan. Three neighbouring subjects remain outside its scope.

Permanent checks take over to monitor the defects the audit has named. Their choice and placement are covered in our article on data quality testing methods.

Carrying out the remedies is a separate workstream, with its own owners and follow-up. The audit itself corrects nothing: a correction made during the examination would distort the measurement and mask the cause.

Appointing data owners, where this is missing, is dealt with through data governance. The audit can observe the absence of an owner; it has no mandate to appoint one.

What remains is the decision. The report is handed to the sponsor with, for each remedy, three possible outcomes: commit to it, defer it to a stated date, or forgo it and accept the defect. The audit is closed when each line of the plan carries a choice, a name and a deadline.

FAQ

Frequently asked questions

Another question? Ask us

What is a data quality audit?

A data quality audit is an examination with a bounded scope, commissioned by a business department, which measures the state of the data consumed by a specific use and traces each defect back to its cause. It delivers demonstrated findings and a remediation plan ranked by effort. It is distinct from permanent checks, which report defects continuously without looking for their cause.

What is the difference between an audit and data quality tests?

Tests are automated checks, run at each load, which compare data against expectations already written down. The audit is a one-off: it has these expectations written down when they are missing, measures the actual state and looks for the origin of defects in the processing pipeline, the source application and the organisation. The checks to put in place are among the remedies it proposes.

Who should commission a data quality audit?

The department that suffers from the defect and can decide on the remedies: the one whose indicator, report or project depends on the data examined. The data team and the IT department provide access and expertise. Without a business sponsor, an audit can produce accurate findings without anyone having the authority to act on the source applications.

What does a data quality audit report contain?

The report contains the scope and the expectations adopted, the measured state of the critical data, the demonstrated causes with their evidence, and what could not be established. It ends with a remediation plan: for each remedy, the cause addressed, the effort, the use that benefits, the intended owner and the dependencies.

Should data be corrected during the audit?

No. A correction made during the examination alters the measured state and can mask the cause being sought. Reworking existing records is part of the remediation plan, after the source has been dealt with: carried out before, it does not hold.

A project to bring to completion?

A first conversation is enough to understand your challenge, identify the right people and tell you frankly what we can bring.