Contract testing is a powerful tool for modern API development. Using a simple set of tools and processes, developers can test the entirety of their system by testing only its output. But how does this work, and what benefit does it deliver to developers who adopt it? Below, we’ll define contract testing, look at its benefits, and suggest some tools to help incorporate contract testing into your workflows.

What is Contract Testing?

Contract testing is a method of integration testing across APIs and microservices that aim to test and verify requests and responses. It’s based on the idea of the contract, which is a concept wherein two entities essentially form an agreement around their technology consumption. Under a contract, both the server and the client are expecting a certain format and form of data exchange — the client expects to send a specific kind of format to a server, the server expects that request form to remain unchanged, and the response it provides is in a pre-determined format which the client is expecting.

When these contracts are violated, this signals a problem, either in terms of assumed form and function or in understanding the existing contract. Accordingly, when working with highly complex systems, it is often much faster to validate these contracts to find potential issues than diving into the entire codebase. Instead of testing the codebase as a whole or digging into the minutia of each endpoint, you can test against the contract itself, which is often specified in a specification file, and verify that the input and output resolves in an expected manner.

The Value of Contract Testing

Contract testing is important as it verifies that different services within a system are working together as expected in a way that is neither time nor cost intensive. The movement to microservices has meant that the testing environment is more complicated than ever. What was once a monolith has now become a collection of tens or hundreds of services working in concert backed by a codebase that is diffused and complicated.

In such a system, testing becomes much more expensive than in the past. In many cases, developers testing the entirety of the microservice collection are trying to find a needle in a haystack, and when this testing brings high expense with it, the value proposition often becomes murky. Accordingly, the fastest way to validate the existing system is to simply test the outcome of it — to test the contract.

Contract testing allows providers to test the complex interactions and the ultimate data feed out of the entire system. It provides for basic validation as a starting point, meaning you can potentially test hundreds of APIs with low cost and low time occupancy as a means to begin a more comprehensive process.

It should be noted that contract testing is not a full-scale replacement for more in-depth testing and tooling. Contract testing is a great way to validate that what you expect to happen is actually happening. But when errors result, more comprehensive testing may be required. Nevertheless, contract testing is a great way to increase testing performance and ensure proper functionality.

Why Should You Perform Contract Testing?

Contract testing has benefits that make it an excellent approach for many organizations. First and foremost, ensuring that performance is as high as possible through the enforcement of expected behavior and dynamics ensures that the contract is well-formed and meets the need of the consumer. Such testing ensures that production systems are well-rounded and well-designed, and early testing can catch issues before they become systemic in the production environment.

Contract testing does carry with it some specific benefits over other testing approaches. Firstly, it provides a starting point for more complex testing in a format that carries lower costs and increased manageability. Since the contract test is only on the outcome and assumptions of the system, you are testing the system as a whole by only testing the output, which dramatically reduces the cost of testing complex systems.

This cost is further driven down by the reality that much of the contract in the modern API space is well-defined. Open specifications exist for the identification and codification of the contract, and in such cases, the testing is made very easy and quick. While older contract testing carried the effort of creating scripts to test undefined contracts, the modern API space makes this a long-past issue.

This automation and open-source definition also means that much contract testing can be fully automated. Developers can plug in specification files and generate automated tests against every endpoint in minutes. This results in a massive reduction in overhead and complexity.

Finally, contract testing allows for isolation between the server and client side of the contract, which allows for more granularity than such a simple solution may typically offer. This isolation can narrow down the source of problems, thereby further reducing the complexity and cost of the overall testing solution.

How to Perform Contract Testing: A General Guide

Now that we know what contract testing is, how do you practically perform it?

Define the Contract

First, we must define the contract itself. Organizations can employ various methods to do this, but the simplest is perhaps the most common — utilizing specifications. Open specifications, including OpenAPI and Protobuff result in files that can be used to generate contracts.

Manual solutions do exist. When there are no specification files for an API, contracts can be defined as part of the creation of the tests themselves. That being said, an API file specification is a helpful step that developers should employ, as this allows for a clear definition of schema and functionality in an API, as well as bringing other benefits.

Regardless of the methodology employed, the contracts should include information such as the format of the request and response as expected, error conditions that may occur, the modality of data exchange, and failover states that are known. These contracts should be well-defined and agreed upon by all systems and teams interacting with the code.

Create the Test

Once the contracts have been defined, developers must now create the test. There are as many ways to do this as there are ways to define the API schema, but whatever the chosen build solution, the tests should cover the scenarios defined in the contract to be fully comprehensive. Solutions such as Pactflow or Joi can create tests from specification files.

Execute the Test

Now that the tests are created, they must be run. These tests should be run against the different services in the system, and should test all specified contracts. Microservices are a collection of a wide variety of systems, so ensure that your testing is truly comprehensive. Missing a single piece in the puzzle might seem like small potatoes, but when the overall contract depends on a microservice that might be buried in the wider collection, such misses can add up quickly.

Once the tests are run, the results should be compared to the expected results defined in the contracts. Any discrepancies between the actual results and the expected results should be investigated and resolved, as they represent a breakdown in the overall contract and could signal bugs, issues, misunderstandings, or poor design standards.

Automate the Process

Contract testing can be very time-consuming. Accordingly, automation turns this process from great to truly powerful. Automation using tools like Spring Cloud or Postman Collections can allow developers to continue testing over time to ensure that the contracts are as defined and expected.

Monitor and Maintain the Contracts

Finally, it’s very important that developers ensure these contracts and their testing approaches are maintained. Changes should be well-tracked, and when the API itself is altered, the specifications and the contracts based upon them should similarly be updated. When testing is automated, this becomes even more important, as both false negatives and false positives undermine the value of such automated testing.

Conclusion

Contract testing is one of the more powerful testing solutions currently on offer in the API realm. It carries with it a system that can validate interactions with stunningly low cost and complexity in most implementations. By simply defining the contract, creating the test, executing the test, automating the process, and maintaining the tests, API developers can create an ongoing testing environment that unlocks extreme value and ensures stability and reliability.

What do you think about contract testing? Let us know in the comments below!