Contracts, and more specifically contract testing, is becoming more and more a part of many modern WebSocket API implementations. The idea that content exchange can occur in a contracted form, and that these contracts can be controlled and tested, is key to many systems utilizing multi-factor backend, frontend, and push server setups.

Today, we’re going to look at such a setup, and check out how they specifically handle their WebSocket contract testing. We’ll do this by looking at Billie, a financial platform co-founded by Artem Demchenkov, who delivered the inspiration for this piece at the Nordic APIs 2018 Platform Summit. We’ll look at why their implementation is unique, and how they handle this complex testing in the production environment.

What is a Contract?

In the case of Billie’s communication model, a contract is essentially an expected process or series of processes between two microservice components. The contract states what each microservice does, how it communicates, and what the format of those interactions is. In other words, an API contract isn’t that different from a regular contract – deviations from each contract state a general problem in communication, and could signal a wide variety of failures.

In Billie’s case, the nature of these contracts is even more important. In order to understand how Billie handles its contract testing, we first need to understand the underlying communication model that is in use. Billie uses three basic components in their model – a frontend, a push server, and a backend. They prioritize pushing user data from the backend to the frontend, but they do so without the use of a traditional HTTP request model. How does it do this?

We can break down their communication model into five basic steps. In the first step, which Billie terms WebSocket Connection, the first connection from the user to the push server is made. Next, during WebSocket Authorization, the user data, specifically the User ID and the user’s API key, are sent from the frontend to the push server, and the user is authenticated. This data is then stored in memory until the client either closes the connection, or the connection fails. When a change to the internal data occurs, this data is then pushed via the third step, Change, to the backend. This is then propagated forward by the fourth step, API Endpoint Push, which the push server finalizes using the fifth step, the WebSocket Push, to update the client.

Accordingly, the nature of these contracts determine the most important elements of communication, and detecting deviations can help resolve most problems as they arise.

The Testing Problem

The problem with this type of approach, however, is that there is a very important, single point of failure. The push server is essentially the control point for communication between the user frontend and the provider backend – and with this comes the fact that lacking communication could be from any number of things, from the routine lack of connection to the catastrophic push server failure. What happens it that server fails and isn’t able to transfer data? What happens if it can’t accept frontend connections? How would the backend even know an error occurred?

Thus, Billie finds itself in a situation in which the process must be automatically tested in an efficient and effective way. Proper, automatic testing is required to ensure not only that the system is working, but that the overall communication patterns are healthy and as expected. While Billie does a variety of testing, the main approach focused on in this piece is the Contract testing methodology that they have termed Pact Contract Testing.

Pact Contract Testing

In Pact Contract Testing, there are three components, mirroring the three elements of the communication process. These three components, the consumer, the mock server, and the provider, mirror the frontend, the push server, and the backend in that order.

We have some expectations that we can use as a base from which to test deviations. The consumer expects a certain request to derive from the network, and expects to send a certain formatted response in return. The consumer expects a certain request as well from the network, and has its own strict response format and communication model. The push server sitting in the middle, known in this case as the mock server, knows both domains, and acts as a facilitator in this testing, managing all the interactions. The integration between those two services are defined in a written special data unit that Billie calls the “Interaction”, and deviations from this model are really what is being tested.

The testing itself leverages Erlang Worldview, specifically for its handling of testing. Everything in Erlang is a process, and that process either does what it is supposed to do, or it fails – this all or nothing situation makes sense in the Billie usecase, as the interaction is chiefly concerned with whether or not the communication occurs at all.

The contract testing scheme largely mirrors the regular communication plan, but uses a series of test processes in place of organic traffic to test each node in the communication path. First, the Test Process generates both the Consumer and the Push Server analogues in steps one and two, with the Test Process essentially functioning as an analogue for the backend. Once the Test Process requests user data from the client, this data is sent to the mock push server for authentication. From here, a second interaction is sent from the Test Process to the Consumer Process to mirror the data push that occurs when the backend is updated in the regular communication process.

The Test Process is first generated by the Provider, which starts the Mock Server for testing. Next, the Consumer Process is started, and a request is sent to the user to provide its authentication data to the Mock Server. With this data, the Mock Server produces the verification, mirroring the second step of the regular communication process.

Once this testing is completed, the code cleans itself up. The consumer process is ended, the push server ended, and the test process concludes the WebSocket connection with either a fail or a pass.

This testing methodology is primarily based upon the idea of separate processes, but it’s extensible to other solutions as well. In his presentation, Artem provides an example PHP implementation — the difference in this application was simply that the Consumer is no longer a process, but is instead an object initiated by the first Test Process.

What Makes This Testing Process Possible?

It should be noted that there are some elements of the Billie model that enable this type of testing. First and foremost, this type of testing is made possible by the relationship between each element of the communication model. The backend is a provider of data, the frontend is a consumer of data, and the push server functions as a communication unit.

These relationships form a heavily contracted communication pattern that can be tested. More specifically, the interactions between connection and authorization as well as the fundamental method of pushing from backend to frontend rely entirely on a contracted communication system that either fails or succeeds – there is very little ambiguity in such a system.

The use of Erlang Worldview is also a big enabling factor. When everything is a process and each process is uniquely identifiable, it’s much easier to test these interactions and to test each individual process.

Conclusion

While on its face, contract testing seems simple, the actual manner in which Billie tests its relationships is quite interesting, and shows the difficulties in testing these kinds of relationships.

Strict contracts on systems that are separate from one another make communications reliable, but they also make communication testing between them much more difficult to test – this is doubly so when these process and servers are removed from one another, and rely on the completion of each communication path in order to verify the relationship, authenticity, and validity of the communication path and contents.

What do you think about this process? Is there a more efficient or effective way of handling this type of testing? Let us know below.