Cloudflare goes deep on API abuse detection

APIs (application programming interfaces) have emerged as the cornerstone of most modern, agile software companies, powering the shift from monolithic on-premise software to the cloud and microservices-based applications. Smaller, function-based components that connect via APIs are easier to maintain, with individual developers or teams taking charge of a single element.

There are many reasons why the API economy is booming, of course, but this proliferation potentially serves bad actors with unfettered access to companies’ internal systems and infrastructure. Many businesses have hundreds or even thousands of APIs to monitor, some of which they might not even know exist. And that is why web infrastructure and security company Cloudflare is introducing new ways to secure API endpoints beyond standard DDoS protection tools.

Adaptive

Cloudflare’s new API abuse detection toolset constitutes several elements. The first part relates to API discovery, with Cloudflare developing a system that builds a “trustworthy map of APIs” that gives businesses an accurate picture of their API landscape. With the APIs “discovered,” Cloudflare’s abuse detection smarts first target what it calls “volumetric anomalies,” which sets an API call threshold to manage abuse by guessing how often each path should be reached legitimately.

It’s worth noting that existing security tools can already set “rate limits” to prevent an API from becoming overwhelmed, which can help thwart automated bad actors from repeating the same breach tactic. But with so many potential unknown APIs in a company, it’s difficult to allocate realistic thresholds for each scenario automatically without causing problems. For example, it’s easy to set a threshold that blocks an IP after it exceeds 100 requests, but what if those requests are legitimate? Ultimately, it all boils down the purpose of the API. As Cloudflare notes, the problem “demands a more subjective arbiter,” which Cloudflare is attempting what it refers to as an “adaptive rate-limiting” technique.

Using unsupervised machine learning, Cloudflare can determine APIs that will likely require frequent calls from an end user and set an appropriate threshold. A sports betting website, for example, might have an API that serves real-time soccer score updates — this will likely have to refresh multiple times each minute to ensure that the information is up-to-date. But that same betting website might also have an API for resetting passwords, and it’s unlikely that a user would make nearly as many calls to that API as they would for soccer scores.

When Cloudflare maps out a company’s APIs, it establishes unique baselines for each one and predicts the intent of requests as they are made. “If we see 150 sudden attempts to reset a password, our systems immediately suspect an account takeover,” the company wrote in a blog post. Additionally, Cloudflare said that it can change thresholds if, for example, it detects that there should be a good reason for a sudden spike in traffic, such as a major sporting event is taking place.

In addition to detecting volumetric anomalies, Cloudflare is also applying an additional layer of security it refers to as “sequential anomaly detection,” where it figures out the most likely or common paths a user might take through a website, and flags any deviation from that. For example, it could be that a typical sequence involves a user logging in, verifying themselves, and then successfully entering the website. But if any steps in that typical process fall out of sync, e.g. if the “user” ends up directly at the third stage, then Cloudflare sounds the alarm.

Cloudflare’s new API abuse detection tools are available now through a request-only early access program for existing customers.