Skip to content
Technology transformation expertise firm · ARDIANE Group Innovate, secure, succeed

Engineering

Which journeys should be entrusted to an end-to-end test, and how do you keep it stable?

In brief

An end-to-end test (or E2E test) runs a complete user journey on the deployed application, from the interface to the database and neighbouring systems. Slow and costly to maintain, it is reserved for critical journeys: Management Square describes here how to choose them, prepare the data and environment, stabilise the suite and place it in the delivery pipeline.

Page contents

A regression reaches production despite green unit tests, and the development team immediately considers an end-to-end test for every feature. Far fewer are needed: one per critical journey.

An end-to-end test (abbreviated E2E) runs a complete user journey on the deployed application: it goes through the interface, the services, the database and the neighbouring systems, as a person would. Among automated tests, it alone checks that the assembled whole works. It is also the slowest test, the most costly to maintain and the most exposed to failures with no real cause. It is therefore reserved for journeys whose failure would halt the business. The method follows an order: choose these journeys, prepare the data and environment, stabilise the suite, then give it its place in the delivery pipeline.

The end-to-end test at the top of the pyramid: rare because expensive

The test pyramid, presented in our page software quality and testing, places the end-to-end test at its top, in small numbers. This position stems from three costs.

The first is execution. The test starts a browser, loads pages and waits for network responses: its duration is counted in seconds or minutes, whereas a unit test returns its verdict almost immediately. The second is diagnosis. A failure designates a journey, without saying which of the components traversed is at fault. The third is maintenance. A modified screen, missing data or an unavailable neighbouring service is enough to make a test fail whose subject has not changed: this is the source of its instability.

In return, it detects what no lower level sees, assembly defects: an environment-specific configuration, a wrongly declared route, authentication that does not propagate from one service to another, a database migration not applied. Each component passes its own tests, and the journey fails.

A journey can also be replayed through the programming interfaces alone, without a browser. The test remains end-to-end for the services, it gains in speed and stability, but it no longer checks the screen.

Choosing the journeys that deserve an end-to-end test

A journey goes on the list when it combines several of these characteristics:

  • its failure interrupts a sale, a payment, a subscription, the submission of an application or access to the service itself;
  • it goes through several systems, at least one of which the team does not control;
  • a large share of users take it;
  • no lower-level test can provide proof of it.

The splitting rule counts as much as the selection: one test per nominal journey. “Ordering with a valid card” justifies an end-to-end test. The many reasons for a card being declined are checked in the unit and integration tests of the payment service, where each one runs quickly and fails for a readable reason.

This step produces a short list of journeys, named in business vocabulary. The product owner answers for their criticality, the development team for their feasibility. Any later addition goes through this list.

Preparing the data and the environment

Each test creates the data it needs (an account, a basket, a contract) through an API call or a script, without going through the interface, and assumes nothing about the state left by another. Two tests that share the same customer account are likely to fail as soon as they run in parallel. Data sets are built from fictitious data. A production copy containing personal data raises the questions covered in our article on data anonymisation.

The environment receives the application in the version that will go to production, with a configuration as close as possible. Third-party services (payment, messaging, identity verification) are called in their provider’s sandbox where one exists, or replaced by a simulator. In the second case, the test no longer proves the real connection, and this limit is written down.

An environment created for each run, then destroyed, removes residual states. Providing it falls under the platform described in our page DevOps and platform engineering.

At the end of this step, the pipeline can create and reset, without intervention, a reproducible data set and a complete environment. The development team owns the data, the platform team the environment.

Stabilising the suite by addressing each intermittent failure at its cause

An unstable test (flaky test) fails then passes without the code having changed. Four causes come up regularly.

The first is the fixed-duration wait: a pause of a few seconds is too short on a slow day and useless the rest of the time. The test should instead wait for an observable state, such as a button that has become active or a message displayed. Tools such as Playwright or Cypress include this mechanism: they check that an element is ready before acting and retry up to a time limit.

Next come selectors tied to the layout. A test that designates an element by its position on the screen breaks at the first change of mock-up. The element’s role, its label or an attribute reserved for tests are more resilient.

The last two causes are dependency between tests, through execution order or shared data, and the environment itself: a slow third-party service, a background job, the clock.

Automatic rerunning of a failed test, offered notably by Playwright and Cypress, serves to distinguish an outright failure from an intermittent one. It fixes nothing: a test that passes only on the second attempt is counted as unstable and assigned to someone. If it is not fixed quickly, it leaves the blocking path and goes into quarantine, with an owner and a deadline, recorded in the same backlog as the rest of technical debt. Once the deadline has passed, it is repaired or deleted.

Screenshots, video recordings and the trace of network exchanges, kept at each failure, shorten diagnosis. The test code goes through the same code review as the application code. The step delivers a suite in which a failure signals a defect; the team developing the journey answers for it.

Placing the end-to-end test in the delivery pipeline

Moment What runs Effect of a failure
On each merge request The few vital journeys, on an environment created for the occasion The merge is blocked
After deployment to the integration environment The full suite, spread over several parallel runs Promotion to production is blocked
After the production release A few journeys with no side effects, replayed at regular intervals An alert is raised

Duration drives this distribution. A suite longer than the team is willing to wait for gets bypassed: it is parallelised, or it slims down. Blocking status must be earned too. It is granted only to stabilised tests, otherwise the pipeline stops for false alarms.

Journeys replayed in production use reserved accounts and write nothing that commits the organisation: no real order, no message to a customer.

This set-up checks that a journey works for a user. Whether the same journey holds up under high traffic falls under load testing, which follows other rules. The team in charge of the delivery pipeline answers for execution, the development team for the verdict.

A short suite is defended at every incident

Each defect that escapes to production suggests adding an end-to-end test. Before writing it, the team asks itself at which level the defect could have been detected. If it concerned a business rule or a contract between two services, the test is added at that level.

The suite is then reread at each major evolution of the product: a journey that has ceased to be critical loses its test, a journey that has become critical receives one, and the list validated by the product owner remains the reference.

FAQ

Frequently asked questions

Another question? Ask us

What is the difference between an end-to-end test and an integration test?

An integration test checks the collaboration of a few components, for example a service and its database, in a partial environment where the other dependencies are replaced. An end-to-end test runs a complete user journey on the deployed application with all its dependencies. The first locates the cause of a failure better; the second is the only one to prove that the assembled whole delivers the service.

How many end-to-end tests should be written?

There is no reference number. The count follows from the list of critical journeys: one test per nominal journey, with variants and error cases checked by unit and integration tests. A suite that the team no longer waits for before delivering, or whose failures it ignores, is too long or too unstable, whatever the number of tests it contains.

What is an unstable test, or flaky test?

An unstable test fails then passes without the application code having changed. Common causes are a fixed-duration wait, a selector tied to the layout, data shared between several tests or a slow third-party service. It is fixed at the cause: rerunning it until it passes masks the problem and gets the team used to no longer reading failures.

Which tools can automate an end-to-end test?

Tools such as Playwright, Cypress or Selenium drive a browser as a user would; Appium plays this role for mobile applications. The choice depends on the browsers and devices to cover, the languages the team uses and integration with the delivery pipeline. The tool weighs less on the outcome than the choice of journeys and the isolation of test data.

Should end-to-end tests be run on every code change?

Only if the suite remains short enough for the team to wait for its verdict. Common practice separates two rhythms: a few vital journeys on each merge request, then the full suite after deployment to an integration environment, before promotion to production. A suite that lengthens the feedback time too much ends up being bypassed.

A project to bring to completion?

A first conversation is enough to understand your challenge, identify the right people and tell you frankly what we can bring.