The European financial services industry is on the cusp of a technological revolution. Reforms like PSD2 and the CMA initiative in the UK are forcing traditional banking institutions to enter the API Economy as active participants.

Great news for the FinTech scene, one would think; at last the banking platforms are being opened up ready for third-parties to securely access a customer’s data without the need to screen scrape online banking interfaces (also frequently referred to as ‘direct access’). However, recent events are showing otherwise…

Andrea Enria, chairperson of the European Banking Authority (EBA) stated in a speech to the Westminster Forum that “…the current practice of third party access without identification… referred to as ‘screen scraping’… will no longer be allowed once the transition period under the PSD2 has elapsed and the RTS (Regulatory Technical Standard) applies.” This speech was followed up with the final draft of the RTS confirming the intention to ban screen scraping 18 months after it comes into force.

Since that announcement there has been consternation amongst the ranks of European FinTechs including consortium forming, lobbying, and rallying in general against the banning of the practice of screen scraping. The EU commission itself has called on the EBA to rethink the ban. Given the number of perceived disadvantages of screen scraping one wonders why.

As with any debate there are always several different valid viewpoints. However, before getting into the debate it worth establishing the technical facts.

Brass Tacks: APIs vs Screen Scraping

Stripping the debate back to pure technicalities tends to lead to one conclusion: Web APIs offer the ability to create a more robust and secure integration with an organization’s platform than screen scraping.

A comparison of the features are shown in the table below:

Based on this comparison, an objective commentator would see Web APIs having significant advantages over screen scraping. However, the reasons to save screen scraping permeate beyond technology. Banning screen scraping has a number of business, cultural, and cost implications that qualify why a ban has significant implications for the European FinTech community.

Opportunity (And What It Costs)

In banking and financial services, screen scraping is a practice that has evolved over many years. The technique isn’t specific to the industry, but it has become so prevalent due to the value of the data and functionality that can be harvested from online banking. Organizations that employ screen scraping – whether they are classed as a “FinTech” or not – do so to take advantage of the opportunity this offers them.

One of the reasons this opportunity has arisen is due to the lack of APIs in banking in general. A commonly cited reason for using APIs rather than screen scraping is to offer a “standardized” interface for accessing an API provider’s platform; with a API specification in hand a consumer can build applications. However, what do you do as a consumer of a platform if no APIs exist? You use your initiative.

This is why screen scraping is so important to the FinTech community. The protagonists have taken advantage of what is available to them, and want to continue to do so. The FinTech scene is characterized by organizations that disrupt the incumbents by doing what they already do, but much better: More slick, less friction, more compelling propositions that grab the attention of the consumer.

The simple fact is this: If FinTechs had waited for the banking industry in general to create APIs, there would be no FinTech scene. Organizations like Yodlee, Sofort, Trustly and Figo would either not exist or have much less compelling offers. Consumers would be importing transactions from online banking into Xero or Quickbooks by doing CSV exports and then massaging the data into their chosen accounting package.

There is also an enormous sunk cost in developing the products these companies offer. Being forced to migrate to nascent web APIs would cause significant disruption to their products if the APIs allow fewer features and less functionality than their web interface counterparts. The time and effort to perform the R&D to move to new standards – in some cases in many EU member states – is considerable. While the majority of the companies this affects are fast moving FinTechs, the impact on their product development should not be underestimated. Moreover, the reticence to move to an API-only model in an industry where the incumbents are known for being glacial in their appetite to implement new technology is also understandable.

If the EBA closes the door on screen scraping without fully opening the API one the FinTech scene will be in serious trouble. Regulation has a habit of missing the mark after taking an excessive period of gestation — there is a PSD2 for a reason, after all — so the FinTechs calling for screen scraping to stay have every reason to oppose the ban.

Perception of Necessity

The other argument that raises it’s head is the one that is almost orthogonal to the reasons for using a web API: Why bother with web APIs when screen scraping is good enough? Many API practitioners or evangelists would reach for the sick bucket on hearing such heresy. In their view screen scraping is a necessary evil that needs to be exorcised from the ecosystem as quickly as possible.

A regular consumer, however — the kind without detailed technical knowledge of how data and services can be acquired from their back-end — won’t actually care. Consumers simply want to use a product that gets them want they want in as frictionless of a manner as possible. If web APIs are available to do this, then great. If not, screen scraping will do because to the regular consumer it’s just a technicality.

Technology providers of course have a duty to deliver a product to their customers as securely as possible. If a web API with delegated access is available they should use it. However, some of the debate and the regulations themselves are gravitating towards “when” screen scraping can be used rather a web API: For example, if an organization’s API is unavailable. A mix-and-match approach seems to indicate that even the policy setters acquiesce that screen scraping has a place. However, the practicality of swapping between screen scraping and web APIs as a technology approach is questionable. Both have different security requirements and entity models, making the integration for a consumer much more complex. Moreover, regulating such a mechanism and asserting when a consumer has the “right” to screen scrape rather than access a web API is challenging, especially in a distributed environment like the Internet.

There are, however, many technology providers who think screen scraping is “good enough” and their position is supported by the “don’t care” attitude of the regular consumer. This creates a culture where giving up your user credentials to a third-party is “OK” and no amount of persuasion (through a European Banking Federation YouTube video or otherwise) is going to make them change their mind. The fact that sharing your details with a third party breaks the terms and conditions of using many on-line banking offerings is clearly an insufficient deterrent.

Until this perception changes and consumers actually demand that only delegated authority will be used to access their bank account, technology providers will continue to employ screen scraping wherever they need to. Admittedly some providers have started to make inroads in this area. For example, Mint is integrating with Coinbase using personal API keys. However, and without trying to be condescending, it’s going to take some time for the average consumer themselves to understand what delegated access is and why it’s important.

Final Thoughts

Information technology is an oddly emotive subject. Generally speaking, computers do their work and it should all be just ones and zeros. However, add human beings to the mix – their perceptions, wants, and needs, and suddenly the calm is broken.

The rallying cry against banning screen scraping is a result of creating regulations based purely on technical correctness. A tech-only perception makes no concessions to the fact that it threatens current business models, innovation, a thriving sector of financial services, and ultimately people’s livelihoods. The technical truth – that web APIs coupled with a robust security protocol are more secure than screen scraping in its current form – is indefatigable. The human point – that banning screen scraping could cause serious harm to a valuable sector of the financial services industry – is equally so.

The EBA and the parties concerned must therefore address the non-technical concerns in a manner that works for the entire industry. The most sensible course for all participants seems to be to extend the deadline for a ban; nominally to three years from the date of the RTS coming into force. All participants then need to work towards a solution that either makes a modified form of screen scraping acceptable, or one that offers a workable API-based alternative (RESTful or otherwise) that is feasible to implement. Failing to reach such an acceptable solution could severely impact the continued growth of the European FinTech scene.