As the growing number of consumer IoT devices make our lives easier, they are collecting vast amounts of useful data. This trend has just started making its way into the fitness and healthcare industries, with devices like smart watches, clothing sensors, and sociometric badges collecting millions of body temperatures, step counts, and heart rates. This sounds like incredibly valuable data, and it is! It’s priceless diagnostic information for healthcare professionals, offers cutting-edge self-monitoring capabilities to consumers, and creates a treasure trove of health information for researchers.

The caveat is that sensitive, confidential data in the medical domain must be treated carefully. From a legal perspective, there are strict regulations for how it should be stored, transferred, and used. Consumers too have their own privacy and security expectations. This means that working with medical data isn’t as straightforward as some would hope.

What’s more, medical data comes in all shapes and sizes, and from a number of sources. As a result, anyone who wants to work with medical data has the added challenges of having to collect it up in different ways, process a variety of data formats, and separate the sensitive material from the mundane, for the reasons above.

It’s no surprise, then, that working with medical data certainly isn’t easy. With competing standards, new devices, and a history of sluggish technological adoption, Nokia software engineer Vlad Stirbu has looked to API-powered microservices as a solution for making the most of invaluable medical data.

Regulations and Standards For Medical Data

With designing any product, the legal limitations must take priority. As such, it’s essential to be familiar with regulations for data protection that apply in your jurisdiction, which is the starting point for Vlad’s development process. Thankfully, current regulations are such that adhering to them ensures that you’ll handle sensitive data in a way that’s not only legal, but also considerate of the consumer — which is one less thing to worry about.

In the US, these regulations are described by HIPAA, or the Health Insurance Portability and Accountability Act — in particular Title II. HIPAA was introduced in 1996, defining what constitutes “protected health information” (PHI) and how it should be used and disclosed. Anyone and everyone who deals with medical data in the US needs to be HIPAA compliant, which is described further in this helpful article.

In Europe, the primary regulation at play is GDPR, or the General Data Protection Regulation. This is a universal framework for how consumer data should be handled across all industries — healthcare being no exception. Similarly, it defines what can be considered “personal data,” as well as how that should be handled. Under GDPR, all medical information is by nature considered sensitive and has special rules for how it should be treated.

In addition to these regulations, which can be thought of legal requirements, there are a number of popular standards that Vlad introduces in his presentation. An example is IEC 62304, which is harmonized in Europe and the States, and provides a good universal benchmark for what can be expected in the more specific regulations of either jurisdiction. In particular, IEC 62304 defines requirements for development of both medical devices (including the consumer electronics we mentioned earlier, among others) and medical software. It also defines the term “SOUP” — Software of Unknown Provenance — a pertinent topic considering how extensively microservice architectures are designed out-of-house.

There are also standards for effectively storing and exchanging healthcare information. Developed by HL7, FHIR is one of these; it suggests best practices for data transfer, authentication, and payload delivery. Finally, LOINC offers a standard for collecting and sharing measurements — including, for example, what units should be used; this way the data (in and of itself) follows a predictable format on the scientific level.

Other Difficulties of Collecting Medical Data

If navigating these regulations and standards wasn’t enough work, there are a few other glaring difficulties of dealing with medical data. Namely, these are the various formats data can come in and the sources they might originate from.

Data Formats in the Medical Domain

Medical data takes on all kinds of formats. Not only does this make for a muddle of units, with different units being attached to the same measurements, but it also gives rise to data of wildly variable densities. As the example in his presentation, Vlad offered a comparison between continuous ECG data and point measurements of an individual’s weight: 1 hour of ECG reading is as large as 1 million weight measurements.

The challenge here is building a system that can ingest this data smoothly while interpreting it correctly, and this problem is amplified when data is drawn from multiple sources, each with their own formatting standards…

Data Origins in the 21st Century

As Vlad points out, any data (medical data being no exception) can reach you in one of several ways. In some cases, you farm the data directly, but in others the data might pass through gateways or be stored with third parties first. Not only does this introduce additional security concerns (as every aspect of the more convoluted paths need to be secure, not to mention legally compliant), but it also means payloads are delivered in a variety of ways.

This is just taking consumer electronics and web apps into account; if you wanted to compound this data with, say, patient registries from local hospitals, there would be a whole host of other requirements for handling the data — in addition to yet another format to process.

Microservice Architectures as a Starting Point

In navigating these concerns, Vlad says that the logical approach is to adopt a microservice architecture. This way, data can be divided up and handled separately according to its nature — whether or not it is sensitive and needs to be treated specially, what format it takes, etc. This solution is inherently fast and flexible; just make sure to satisfy the local data protection regulations and do your best to adhere to FHIR and LOINC on a case-by-case basis for your data, right?

Unfortunately, things get complicated very quickly, as more and more microservices are added and changes have to be made. Vlad compares it to a monkey hanging on a tree: “it starts to weigh [down], and you don’t know how long it’s able to hang up there.” Instead of starting over, he pushed to make the microservice architecture work — and the secret lied in effective governance…

How to Maintain a Microservice Architecture

Vlad proposes a few simple but effective ideas for managing microservice platforms that keep growing in complexity. The advice applies to all industries as much as the medical one:

1. Make the most of standards

It might seem obvious, but Vlad promotes the use of standards for a reason: they make every aspect of development, from data formatting to microservice design, a whole lot more intuitive for developers old and new. In addition to the standards outlined earlier, Vlad suggests the use of standardized operating procedures — whether they’re written in code or plain English.

2. Group microservices based on commonality

Another suggestion is “capability sedimentation”: grouping microservices based on their commonality. By building common features into a platform layer, for example, developers can save time while also keeping multiple microservices connected. This prevents different branches of the platform from drifting too far apart.

3. Give developers a close-up view

In Vlad’s experience, giving developers a high-up view of the system is detrimental, or at best ineffective. Instead, a better move is to give developers a close-up view of only their direct dependencies, so they can focus on building the perfect microservice for the exact problem at hand (this was the primary advantage of adopting a microservice architecture in the first place). You can accomplish this with a variety of tools; Vlad uses Docker Compose.

4. Leverage tools whenever possible

Speaking of tools, there are many available to aid the microservice journey. Not only does using tools automatically standardize matters for you, but it also saves time. One such “tool” is the OpenAPI spec, which simplifies development along with a whole host of other benefits.

Taking Vlad’s Advice

Of all the data out there, medical data is some of the most difficult to handle: it comes in different formats, from different sources, and is strictly regulated. This makes it a prime candidate for a closely-managed microservice architecture, kept simple by following standards to the tee, grouping up features across microservices, setting aside the wholistic viewpoint, and using tools wherever possible.

Using Vlad’s approach to microservices with other high-stake, sensitive data, perhaps financial or legal data, is a sure-fire way of covering your responsibilities as a developer without overcomplicating things. Perhaps a similar approach can be used in today’s open banking movement, to expose powerful APIs while maintaining security and versatility.