API Scalability, part 2 - caching, rate limits, and offloading »

(Following from Tuesday's blog entry on API Scalability and Caching.

Last time we wrote about 3 things to think about when planning how to scale your API.

• Caching
• Rate limiting and threat protection
• Offloading expensive processing

and then talked about caching at length, so let's finish up with:

Rate Limiting and Threat Protection

Another aspect of scaling is just keeping unnecessary traffic away from your application servers and databases. Some of the techniques that we've discussed previously, such as rate limits and threat protection, apply here as well.

For instance, an API's performance can drop precipitously if a client, on purpose or by accident, sends too much traffic. A rate limit helps a lot here!

Bad requests can kill API performance too. XML threats, which we discussed in the last episode, are one example of a way that a bad request from a client can cause performance problems or even a crash on the server side. It's a lot easier to maintain scalability if you can stop these kinds of problems before they can hurt your servers.

Server Processing Offloading

Finally, consider the things that you can offload from your application server tier. The more you can offload to more efficient platforms, the less load your application servers have to handle. Plus, the more things you can offload, the simpler those application servers and their applications become, which means they're easier to manage and easier to scale.

For example:

SSL. Load balancers and ADCs like F5 and NetScaler products, not to mention web service proxies like Sonoa ServiceNet, can process SSL more efficiently than most application servers.

HTTP Connections. Those same products are highly optimized to handle tens of thousands of simultaneous connections from HTTP clients, and operate a smaller pool of connections to the back-end application servers. Offloading HTTP connection handling to another tier can free up a lot of server resources.

Authentication. If you perform authentication, a proxy like Sonoa ServiceNet can handle all your authentication for you, freeing your application servers to worry only about properly-authenticated requests. And if you're using SOAP, a product like ServiceNet can process many of the SOAP headers, such as WS-Security headers for authentication, then remove them so that the application server doesn't even need to see them.

Validation. If your API depends on XML input, it may run more efficiently if it only accepts valid XML requests. Turning on XML schema validation can hurt performance of most application servers - products like ServiceNet can do it more efficiently.

So to finishing up, key Questions to ask for your API scalability roadmap might include:

• What kind of volume are you expecting?
• Are you prepared if you get 10, 100, or 10,000 times that amount of volume with little warning?
• Do you have a way to shut a user off if they consume too much volume?
• Do you have a way to control API traffic in case you are unable to handle the volume (see Traffic Management)
• Are your back end servers capable of handling tens of thousands of concurrent connections?
• Are your back end services cacheable? Do you have a cache that you can use to reduce response times?
• Are you monitoring response times and tracking them to gauge customer satisfaction?

(next time:  API user management and oboarding)