Definition of platform engineering
Platform engineering is the discipline of designing, building and evolving an internal platform that serves development teams. This platform provides them, on a self-service basis, with what they need to deliver and operate their applications: environments, deployment pipelines, monitoring, security.
The definition of platform engineering includes an organisational choice. The platform is managed as a product, with users (the organisation’s developers), a roadmap and a measure of adoption. It is not an infrastructure project delivered once and for all.
What platform engineering is for
The “you build it, you run it” principle made development teams responsible for their services in production. It also transferred a considerable load to them: containers, infrastructure described as code, pipelines, secrets management, monitoring, compliance. Each team ends up assembling its own solution.
The effects are well known. The same problems are solved several times, in different ways. The level of security varies from one team to another. A newcomer takes a long time to ship their first change. Platform engineering responds by pooling: reducing developers’ cognitive load, without taking away their responsibility for their services.
The internal developer platform
The internal developer platform (IDP) is the layer placed between developers and the infrastructure. It most often brings together the following elements.
- A developer portal, with the catalogue of services, their owners and their documentation. The open source software Backstage is one example.
- Project templates, which create a service already connected to its deployment pipeline.
- Shared continuous integration and deployment pipelines.
- Self-service provisioning of environments, databases and message queues, through infrastructure described as code.
- Identity and secrets management.
- Preconfigured monitoring: metrics, logs, dashboards.
- Security and compliance controls applied automatically.
The acronym IDP sometimes refers to the portal alone. The portal is only the interface. The platform is the set of capabilities it exposes.
Golden paths
A golden path is a recommended, documented and already tooled way of carrying out a common task: creating a new service, exposing an application programming interface, obtaining a database. Three properties characterise it.
- It is complete. It covers the task end to end, from creating the code repository to the production release.
- It is maintained. The platform team evolves it and provides support.
- It is optional. A team may depart from it, provided it takes responsibility itself for what it builds off the path.
This last point distinguishes the golden path from an imposed standard. The platform wins people over through its convenience. If teams bypass it, that is information about the product, not a fault on their part.
Platform engineering, DevOps, SRE: not to be confused
DevOps is a culture: development and operations share responsibility for the service. Platform engineering does not replace it. It makes it sustainable at scale, by sparing each team from having to master the whole technical chain.
Nor is it an operations team under a new name. A traditional operations team handles requests in a ticket queue. A platform team provides a product that developers use on their own. The test is simple: do you have to wait for someone to get an environment?
Finally, SRE focuses on the reliability of services in production, whereas platform engineering aims first at the efficiency of the delivery journey. The two complement each other: the platform embeds reliability practices by default.
Three mistakes recur in initiatives that disappoint: building the platform without consulting its future users, making it mandatory before it is useful, and trying to cover everything from the first version.
A concrete example: creating a new service
Consider an IT department where creating a service involves several teams. The developer opens a ticket to obtain a code repository, another for the deployment pipeline, a third for the environments, a fourth for access rights. Each step waits for the previous one.
With an internal platform, the developer chooses a template in the portal and enters the name of the service and their team. The platform creates the repository, the deployment pipeline, the environments and the dashboards, with the organisation’s security rules already applied. The service appears in the catalogue with its owner. The platform team, for its part, tracks the template’s adoption rate and gathers pain points to prepare the next version.
To design such a platform, see our page devoted to DevOps and platform engineering.