Platform engineering represents a paradigm shift in how companies approach software delivery. By building internal developer platforms (IDPs), organizations are streamlining their DevOps processes, enhancing collaboration, and accelerating time to market.

Central to the success of these IDPs is a myriad of API connections, webhooks, and asynchronous events that underpin the automated toolchains and services. Let’s dive into how platform engineering and IDPs came to be, how they impact the modern development ecosystem, and what this means for the industry.

The Evolution of Platform Engineering

In the early days of software development, code creation was somewhat straightforward. The monolithic structure that was commonplace meant that everyone was working on the same body of code, and iterations meant plugging your particular piece into the overall constraints and demands of the entire system. For a long time, this was the paradigm of software development, and though it was limited, it did provide a structure underneath development that constrained and guided team elements.

As the need for more complex systems gave way to microservices architectures, the experience of internal development changed drastically. With much less constraint and more variable pieces than ever before, developers were left to their own devices. This ultimately led to the development and implementation of disparate DevOps tools and SDLC toolchains, which increased complexity and reduced standardization. In essence, most common solutions were oriented for a specific use case, meaning those solutions weren’t applicable for much more than what they were designed for.

While traditional approaches often result in siloed development and operations activities, leading to inefficiencies and a slower response to market demands, the new techniques needed a much more supportive infrastructure that was variable, designed for self-service, and robust in implementation and extensibility.

IDPs emerged as a solution to these challenges, offering standardized and streamlined workflows for development teams. The promise of the IDP was simple — self-service capabilities with automated systems to allow for extensibility and scalability.

But what does this look like in practicality?

The Internal Structure of the IDP

IDPs are best thought of as a system of tools rather than as a monolithic singular tool. While the system has general benefits, the collection of tools under the hood almost mirrors the nature of microservices, requiring interconnection and standardization. IDPs utilize APIs in a very similar method to the modern web, connecting disparate tools, infrastructural elements, and internal systems via a series of standardized protocols.

Self-Service Portals

One of the hallmark features of a good IDP is a self-service portal. IDPs are meant to facilitate developer team independence while ensuring standardized access, and allowing the use of infrastructural deployment and tooling through a self-service portal does just that. APIs are the interface through which developers interact with these portals, often abstracting away the complex details of the underlying infrastructure.

Integration and Orchestration

IDPs surface different tools within the software development lifecycle —such as version control systems, build servers, and deployment tools —and facilitate integration through API services. This integration and automation allows developer teams to utilize the benefits of the platform without having to exhaustively understand or integrate the entire platform.

Automation of DevOps Workflows

Through APIs, routine and complex workflows can be automated across multiple projects and instances. For example, once code passes a series of automated tests, an API call can promote that code to a staging environment and notify developer teams about the state change. This reduces the need for manual oversight and accelerates the development process.

Unified Security

An IDP can also offer a unified security posture by connecting systems in a common, single configuration. By unifying the security posture and connecting systems to it, issues of misconfiguration, cross-communication, and differentiation can be deftly handled and substantially reduced.

IDP Reference Architecture for Azure by Humanitec

IDP as Internal Platform-as-a-Service

IDPs ultimately function as an internal Platform-as-a-Service (PaaS) solution. Think of all the benefits a PaaS delivers externally — the unified security, the cohesive base of code dependencies, the proven application of known entities within the ecosystem – and consider what it means to connect this internally, allowing disparate teams working on different solutions to all tie in.

This is the huge benefit of an IDP — teams can iterate and develop within a trusted system with a lot of plugin potential. Teams can iterate in their own way for their own goals without drifting away from the core promise and premise of the underlying service. Testing only has to be done once, and then you have a trusted solution that can be commoditized and deployed elsewhere. And all of this typically comes with a robust self-serve platform that allows developers to tie into systems with low overhead and time requirements.

APIs and IDPs

APIs are a huge part of what makes an IDP especially powerful. APIs are the backbone of the IDP and, more often than not, play a huge role in defining how good a solution the IDP actually is. APIs have several major roles to play in the IDP ecosystem, including:

• Extensibility: APIs provide the means to extend IDPs with new tools and services without overhauling existing services.
• Standardization: Proper standard deployment itself enforces organizational standardization, ensuring that all teams and services follow consistent practices by establishing the typical pattern and implementation across the board.
• Scalability: APIs facilitate scalability by allowing IDPs to handle increased loads by adding more services or tools into the ecosystem with minimal disruption.
• Flexibility: Variable technology choices, such as synchronous versus asynchronous APIs, allow for a more decoupled and flexible system where services can communicate and react to events as is necessary. An IDP can integrate multiple technologies depending on the need and ecosystem, allowing greater flexibility than single-source purpose-built solutions.

What Makes a Good IDP?

Because IDPs are essentially internal platforms, the general advice for an effective developer portal applies here. In summary, a good IDP should:

• Implement and communicate design standards: The underlying codebase should adopt good design standards, and this should be mirrored through the IDP. Employ good naming conventions, ensure consistency, and adopt radical transparency and clarity wherever possible.
• Be designed for people: Create a developer platform built for developers. Utilize proper design methods, ensure clarity in your communication, and adopt aesthetics that ensure humans can use the platform well.
• Provide intuitive documentation: Software lives and dies by its documentation, and developer portals are no exception. Ensure your documentation is comprehensive and useful, and make sure any idiosyncrasies are documented with enough detail to get developers primed for success.
• Consider the onboarding experience: Develop a good onboarding experience to ensure new users enter the system with everything they need to get started. Develop a flow for users so that someone trying to find a solution to their problem can easily find and implement it.
• Communicate versioning and change management: The portal should give context to the end user. Changes at the IDP level can have huge tack-on effects across multiple systems, and as such, version control should be handled with the utmost care and high communication standards.
• Provide sample code and implementations: Good IDPs should not just tell someone how something works — they should also show it. “Show, not tell” is a standard that should apply throughout the documentation. Providing systems such as playgrounds or live code testing environments can also go a huge way towards ensuring adequate support.
• Provide tooling and variations: For IDPs, the name of the game is standardization. Developer portals are your best bet for guiding implementation and design approach, and as such, IDPs should exist as a “one-stop shop” for your developer’s needs.

Conclusion

The rise of IDPs is promising, as it represents a unified approach towards cohesive development for a segment of the API space that benefits greatly from cohesion. The lessons learned from developing internal developer platforms can extend to platforms in general, exposing the massive value of adequate standardization, ample documentation, and a cohesive vision of what good development looks like.

What do you think of the rise of IDPs? Have we missed anything crucial that you’d love us to cover? Let us know in the comments below!