Protect your API: Twitter’s Denial of Service Attack and API Security »

Lots of news on the Twitter attack last week. There’s general consensus that it was a distributed denial-of-service attack (DDOS) that it targeted a particular account.

DDOS attacks are tricky things. There’s no one technique, product, or protocol that will stop them. That’s what makes them so nasty. To defend against one, a company needs to be able to quickly take countermeasures at all the different levels of the protocol stack, including firewalls, routers, load balancers, and even in the application itself. And even more importantly, you need to have experienced, well-trained operations people who know how to quickly identify an attack and come up with a way to minimize its effects.

When a DDOS attack affects a web service – and there’s an good chance that the Twitter attack made use of the Twitter API – then proper traffic management at the API level is an important part of this stack of protection.

For instance, if a DDOS attack targets a particular user account, or API key, it may be possible to block those particular requests before they get to the back-end application servers, so that the effect of the attack is limited. Or, if the attack always sends some of the same parameters in an HTTP query string, or in the body of an HTTP POST, it may be possible to detect those patterns and reject the requests, again before they reach the back-end application servers. In the worst case, it may even be necessary to shut off access to a particular web service or web service operation, allowing other services to function as best they can while the attack is neutralized by other means.

In other words, just as it may be possible to stop a DDOS attack by blocking a range of IP addresses, or redirecting certain URLs, or configuring traffic shaping in a router or an ADC, it may sometimes be necessary to stop a DDOS attack closer to the application level. The ability to inspect requests at the web services level and take action based on the request content gives an operations team one more weapon to use against an attack.

The Twitter API already has a usage quota on each user account, a rate limit on each IP address, and extensive caching. Imagine how much easier a DDOS attack might have been if they didn’t even have those things.

-Greg

(Greg Brail is the CTO of Sonoa and writes on API Security topics such as API Keys, OAuth, and API Security Recommendations.) 

Don’t have a meltdown: API Traffic Management »

(This is part 2 in our series on "Is your API naked?  10 API Roadmap Considerations")

Customers come up with far better sound-bites than we do. One asked us if we could protect them from “back-end melt down.”   They had opened an API for customers, but it was the same one their Web apps was using, so bursts of traffic from customer apps had their Web product running hot – and not in a good way.

A back-end melt down will lead to a front-end melt down in a hurry. That's a major risk to the business.

It's also one reason why traffic management is usually one of the first management features added to an open API. But usually there are both technical and business reasons for this.

Technical folks want throttling, or rate-limiting, to regulate traffic flow. Your customer's developers might inadvertently write some inefficient code. Someone might try to download all your data through your API. Rarely you might get some malicious use. Rate limiting or transaction throttling (X transactions per second usually) can be like a circuit breaker that protects your own apps and keeps your Web and API customers happy.

Business managers may need to measure consumption against a daily quota. This might be to find growing customers to upsell, or keep track of customer data usage (especially if you are paying for that data). Or they may want to create different or custom 'service tiers. '

Pitfalls to avoid with Rate limits and quotas

We see a few API providers substituting ‘quotas’ for rate limits – telling developers they can have X transactions per day. But just measuring against daily quotas can leave your ‘back end exposed’ if you get a short burst of traffic. And 'per second ' rate limits aren't a great way to create tiers of business-level consumption. You might need both.

Or, we see customers that define an SLA only on paper and say “call us if you need more. “ We had a SaaS customer that started this way but then found it was tough to tell important customers they needed to manage their API usage. They eventually found a good way to ease into the traffic management discussion - by first providing the customer with usage reports. ( “look how all your requests come in Monday at 9 am, we love you but can you help us?” )

And you may not consider all requests equal – you may find that some requests contain many transactions or big messages, so a simple “X requests per second” rate limit might not be enough.

So consider these questions for your open API roadmap:

What kinds of rate limiting do you need?

• Do you need to rate limit by developer, app, or customer organization?
• Can you rate limit a particular client (key and IP address)?
• Are all messages the same or do you need to adjust based on message size, or records per message, or bandwidth ?
• How do you throttle differently for your own web or internal apps vs. API traffic?

Does your business need quotas on API usage?

• Do you need to keep track of daily or monthly usage to measure business consumption?
• Do you need to define different consumption levels for different service tiers?
• If someone goes over the quota, what is the best business recourse? Call them up and upsell them? Or shut them off?
• If you are paying for the data are you measuring this for billing and pricing?

How do you monitor and respond to traffic management?

• How will you know when someone goes over a rate limit or quota?
• Are quota or rate limit overages a trend or a 'one time thing'?
• Can you define flexible actions? (i.e. drop requests, do nothing, send an alert?)
• Can you provide customers with data so they can help by metering themselves?

What about bad stuff like unauthorized access, Denial-of-Service, or XML specific attacks?

More on that in the next couple sections – Security and Data protection

(And here's one solution for help with API Traffic Management) 

So you want to open an API? »

Greg Brail, our CTO, took advantage of a break from O'Reilly Velocity and a new Flip HD ultra to record a series of four short whiteboard talks on issues many face when opening an API - visibility, security, traffic management, and more.  Here is the first clip and you can preview all of these and more API case studies on our Apigee youtube channel.).  

We love feedback and these are quick to do - so any more topics you'd like to see, please let us know..