Matt Fellows, Principal Product Manager of PactFlow and core maintainer of Pact, SmartBear, shares what the future looks like for contract testing with Pact.

Application programming interfaces (APIs) are a double-edged sword.
Their convenience of having a set of definitions and protocols for building and integrating application software and creating rich user experiences is indisputable. As a result, they are continuing to proliferate as systems become ever more distributed.

Yet, with so many APIs — and the corresponding teams that build and maintain them — ensuring all the systems work together as expected can be challenging. This is why we test.

Adding to that challenge is that APIs use different protocols. SmartBear’s recent State of Software Quality API survey found 80% of organizations operate in a multi-protocol landscape. More than 60% of them manage three or more protocols. Running end-to-end (E2E) tests in a multi-protocol environment with many APIs can be onerous. E2E tests often involve running a battery of tests against a simulated production environment, which can be hard to replicate with high fidelity, requires high coordination between teams, is slow to execute, and tends to be unreliable or flaky.

Testing also requires an additional workforce to adequately maintain — debugging every failed test uses time and resources that development teams cannot afford. This can slow development speed and grind integration testing pipelines to a crippling halt. This is a key reason API contract testing is taking off.

Making of a Pact

Contract testing defines how two systems can communicate by agreeing on what interactions (conversations) can be sent between them and providing concrete examples to test behavior — this is referred to as specification by example. These interactions are usually captured from well-designed unit tests to be precise about what scenarios are covered. They are then stored as a contract used to ensure both parties can understand each other and work together as they evolve over time.

It should be noted that contract testing is different than schema testing. Schema testing relies on checking an API is compatible with a generalized, often declarative notation. It defines the data types and set of inputs/outputs that a single system supports (such as an OpenAPI Specification for an API producer) at a static point in time. In HTTP, for example, these are the syntactical rules that requests and responses must follow.

These types of schema checks may be necessary for a functional system and to ensure your API documentation is valid, but they are not sufficient. This is because schemas are abstract and introduce ambiguities when it comes to the semantic understanding of behavior. They don’t take into account their consumers, and it’s hard to track the evolution of their relationships over time.

Contract testing is also contrasted to integrated (or end-to-end) testing, which requires both parties and all their dependencies to be available in an environment to run a series of scenarios through the system, asserting on the visible output at the system level to determine correctness.

Contract testing allows you to test each system independently while still providing high confidence the systems are compatible and preserving visibility over the API surface area used by an APIs service community.

How consumer-driven contract testing works

Pact, an open-source project, takes that a step further by taming the chaos of manually creating contracts. It was originally conceived in 2013 at realestate.com.au to help with integration testing for its cloud-based Ruby microservices architecture.

After that, the open-source tool grew in importance. As the community grew, so did its uptake. Today, many development teams use Pact to only test those API parts that they will use and not the entire API. This further reduces effort and uncomplicates testing.

The code-first consumer-driven contract testing approach also allows testing to evolve safely and quickly. It means you no longer need to maintain a static contract — you can use automation to ensure it’s always up-to-date. This ensures your application ecosystem will work cohesively way before it reaches the final stages of end-to-end testing during development.

Since its inception ten years ago, Pact has supported RESTful microservices. In the years afterward, support for asynchronous and messaging use cases was added, allowing Pact to support a broader range of use cases and emerge as the de-facto contract testing solution for APIs.

Plugging Into Complexity

A lot has happened in software development in the last decade. And Pact strives to stay at the forefront.

When it was created, the environment was a lot simpler. Today, the number of protocols has mushroomed. New protocols like Protobufs and GraphQL transports such as gRPC, WebSockets, and MQTT, and event-driven architectures and data pipelines, or emerging standards such as AsyncAPI and CloudEvent, are creating a complex environment. This change in landscape means API testing is becoming a major challenge.

New versions of Pact (we’re at version 4) introduced new features and innovations. However, it became apparent that keeping up with all the protocols would be a challenge with the current design. This reduces the adoption, use, and impact of Pact. Besides deficiencies in some of the emerging modern protocols, testing legacy environments were becoming equally important, especially in large environments such as banking, and as the practice of contract testing continued to build momentum in the industry.

What we needed was a way for users to easily extend Pact with their specific needs without having to raise a feature request and wait for the maintainers to prioritize, implement, and roll it out across all of our 10+ languages to address all these use cases.

The Pact Plugin Framework was born.

Framing the Case for a Plugin

The Pact Plugin Framework introduces the idea of user-defined extensibility into the ecosystem, allowing anybody to extend Pact while retaining the benefits of the unified Pact tooling and workflows. Plugins allow users to create new types of protocols, transports, and content types, which can be mixed and matched to solve a wide variety of use cases. This allows teams to create plugins for their specific needs, such as a rich Kafka integration, or a gRPC (protocol) plugin with Protobuf support (content).

The Pact Plugin Framework

Plugins also were designed to address time-to-market, a key challenge that drove the idea of a plugin framework. Our intent is to free users from the limitations of the Pact maintainers allowing them to fit contract testing into any situation and take control over testing objectives.

Say you’re working in a bank with a lot of infrastructure that has been organically added over decades. You might be faced with integrating a mainframe system that communicates using very specific binary protocols (such as ISO 8583). Currently, Pact does not support this, and retrofitting such a standard into the existing design is no trivial pursuit (no pun intended).

The Pact Plugin Framework addresses this gap. It caters for closed, open, and commercial use cases and means you’re not limited by what the open-source community has done. You can now do your own testing for particular use cases, allowing you to apply contract testing to your tech stack.

The framework also adds a multiplier effect. Instead of having the most common use cases and allowing for very specific ones, it can quickly broaden the available use cases and grow our community of users and contributors. We hope to create and support a thriving plugin ecosystem, servicing the needs of various organizations and use cases.

Most importantly, the framework ensures Pact development does not remain too far behind the curve of current development needs. With the ability to extend Pact for new types of transports and protocols, you now have the freedom to do contract testing with Pact where you couldn’t before.

It’s a Multi-Protocol World

Development teams must face the fact that their world will be filled with new protocols. Already, IoT protocols are entering the vocabulary, making API testing more complex and requiring workarounds. A more flexible Pact Plugin Framework can help to bring some order to this chaos.

The Pact Plugin Framework development also speaks to the importance of contract testing. By removing many of the most problematic aspects of integration and E2E testing for APIs, it has given developers fast and reliable feedback on the API endpoints early in the software development lifecycle.

This contract between consumers and providers reduces the chance of API specification changes disrupting application performance. And if a problem is found, you can at least address it quickly during the development and not after when it then becomes a business concern in an increasingly application-driven world.