Understanding Idempotency and Safety in API Design Kristopher Sandoval September 14, 2018 What does it mean in programming to be idempotent? Today, we’re going to look at the concepts of idempotence and safety, and identify what makes them so very important in the context of web API design. We’ll compare idempotence and safety to see how they’re different, and discover why this division is important. We’ll learn which HTTP methods are idempotent versus safe, and discover how to ultimately apply this knowledge toward building secure and predictable web APIs. What is Idempotence, Anyways? Simply put, an operation is idempotent if it produces the same result when called over and over. An identical request should return an identical result when done twice, two thousand, or two million times. The source of most confusion around this concept comes with the idea of identical results, however. What we expect to see is identical results in the return form rather than in the return value. When a call is issued, the response can be broken generally into two different camps – the return form, and the return value. If we were to make a GET request for a resource status update, for instance, we might get vastly different results depending on the resource that is being inspected and the time in which we do the inspection. Regardless of that change, which is simply a change in value, we would expect the format of that result (i.e. the package type, the way in which data is shown, the method in which it is encrypted, etc.) to be exactly the same. While the expected result should be identical, the value of that result can change. You can still make idempotent calls to resources that change, as long as the resource is returned in a known structure, with all known endpoints responding correctly. A good way to frame your understanding of this concept is to think of each functional call as asking a question in another language. When you ask someone what time it is in in Spanish – “¿que hora es?” — the answer to that question can change depending on the time of day. This would be a changing value. The way in which you expect the response, however, does not change – just because it is 4pm, the person responding is not going to start using “lemons” as a measurement of hours, or respond to you in French. Each response will be structured in the form, syntax, and grammar that you expect – and no matter how many times you ask, it will always be in Spanish. This is the function of the call. One can imagine the chaos that might ensue if a person constantly switched between Spanish, French, German, and English depending on the time of day and what is being asked – in this situation, it would be impossible to form any effective, coherent expectations. The same is true in the API space. The functional call should not be different with each request, but instead should return in a known, expected way, regardless of how different the response value might be. idempotence and safety, which is a related term also called “nullipotency”, are essentially guarantees of expectations, ensuring clients that their calls will be able to expect, parse, and understand the responses in a known way. Idempotence and Safety We should take a moment to address safety, as these terms are often used interchangeably. While they are definitely related, they differ somewhat significantly, making their colloquial use as synonyms an issue to be resolved. In the most simple terms, an HTTP method is said to be “safe” when it does not alter the state of the server – in other words, a “safe” method is one that functions in a “read-only” mode. Safe methods can be changed by the server itself, of course, but in those instances, it is not the request that is making the change, but the server interacting with the request. For instance, logs can update and change resources on the server, but the access to those resources are not causing the change, and will always (at least if they are programmed correctly) be “safe”. The reason these terms are often confused is because they fall into a cross-over definition with each other – this is basically a “square is a rectangle, but a rectangle is not a square” situation. Since a safe method will always result in the same form (if not the same value), even if the resources change, they are by definition idempotent – but something can be idempotent and still change a server resource or state, meaning that what is idempotent is not necessarily safe. Accordingly, the following table shows some common HTTP verbiage, and whether or not it is idempotent and safe. Why Not Make Everything Idempotent? The question should be asked, then – if being idempotent results in the same form day in and day out, why can’t we just make everything idempotent? The problem is that sometimes, in some significant special cases, it is desirable for both the value and the form to change. For example, POST is by definition something that results in a server state change and a change in expected form and value. If POST was idempotent, everything sent and accepted the web server would already have to exist on the server in some form to respond with the same codes and value response. For that reason, POST cannot be idempotent or safe – it defies the very purpose of the verbiage. The method itself hinges upon both changing resources and changing responses following each request. There may appear to be an exception to this rule with DELETE, but upon further inspection, we can see that it truly is idempotent. While DELETE removes something from the server, that is simply the change in value – whether or not the item exists or not. The response, however, is the same – an error code that either says the item exists or does not, and whether or not it was deleted. No matter what function is done with DELETE, it should result in the same response, even if the value itself is different. Thus, we can see that even something that appears to be non-idempotent can, itself, be idempotent, solidifying the importance of truly grasping this conceptual definition. Caveat Emptor Of course, all of this comes with one huge caveat – the theoretical is not always the practical. If something is idempotent, it may not be once it is invoked by the developer. An idempotent operation called in a non-idempotent way can result in many issues, and is fundamentally a non-idempotent operation. This is, in essence, bad design, and should be avoided at all times. One such example of misued idempotence can be seen in this Twitter thread. In this situation, an API was designed to toggle a relay switch, which in turn would operate the buttons for a garage door. The endpoint, however, was programmed to respond to GET requests. GET is supposed to be both idempotent and safe, but because GET was being used in this way to trigger the function instead of something like POST, an issue arose with the way Apple syncs content across multiple devices. Since the tab that held the GET function was synced, each time a new tab was opened, this syncing process would trigger the page, which would automatically launch the GET function. This, of course, resulted in the door opening each time a tab was created. For multiple nights, the user was confused as to why their garage door was opening in the middle of the night, and only discovered the issue after looking into the way the API handled the requests. In this case, a properly idempotent and safe setup would have negated this issue. Utilizing GET to alter the state of the garage door servo is fundamentally wrong – this is something that PUT should have been used for, while GET should have only been used to read the current state of the door servo. While this is a funny example, there are more serious concerns at play. Let’s say we’ve built a commerce app that allows users to submit a payment for a bill or other monetary transfer. In an idempotent and safe setting, the user will click the submit button once, automatically generating a POST function to update the order status. After this, a GET call will be used when the user clicks additional times, simply returning the order status or a warning that the order is already being processed. If we don’t implement this setup, however, each click of that button will, in theory, add an additional order to the backend. If a user clicks that button 100 times, and all 100 times are neither idempotent or safe, then that means 100 separate orders will be generated from the same, single customer. While this is much more a problem of design than of simple idempotence, we can begin to see how important this tool is in the API toolset, if used correctly. Failure to properly implement it can be expensive, dangerous, and ultimately result in your codebase being directly undermined by shoddy implementation. Contracts, Standardization, and Trust A major value here is the idea of having pre-existing contracts with your userbase and fostering an understanding of just how the system functions. Clients should be able to trust the APIs that they interact with, and conversely, the APIs should be standardized enough to support this. Not using idempotent and safe approaches is basically akin to asking your clients to not only trust you blindly, but to also be psychic – to know the limitations of the system, to know what is safe and what is not, and to expect the client knows the form of the response each time it changes. That is clearly a dangerous precedent to set, and one that complicates the relationship between user and client in a pretty significant way. The burden for contracted communication is on the side of the developer, not the client – failure to understand this is a significant problem. Conclusion In no uncertain terms, not functioning in both an idempotent and safe manner is dangerous, and is ultimately using things in a way for which they’re not designed. HTTP verbiage choices are designed to do very specific things, so forcing them to do other, sometimes counter-purpose things is very dangerous, and ultimately pointless. GET is not meant to do what POST does – that’s why they both exist. Establishing proper understanding and enforcing proper design improves user experience, and ultimately improves your total system strength and usability. Failure to adopt these simple concepts and design ethos is a recipe for confusion at best, and failure at worst.