Have an API roadmap? Here are 10 areas where you may need to CYA (cover your apps).
Greg recorded a few whiteboard talks last month - this one is a good summary of recent posts on API Keys, API security recommendations, and OAuth best practices.
(And here are some of our high-level API security features if you are looking into this.)
A lot has been happening with OAuth recently. Earlier this year a security hole was discovered in the protocol which exposed it to a potential “social engineering” attacks. However, the OAuth community is working on a revision to the spec that will eliminate this particular hole.
Last week we wrote a bit on OAuth as an option for API security. But today I wanted to bring up a related OAuth issue - how do you securely manage all those keys?
With traditional username / password authentication, good security practices require you don't just have a big database on the back end with a list of unencrypted passwords. Instead, a hash of the password is stored, preferably using a salt. So someone who can read the password file can verify they have the right password, but cannot see the actual password.
It is still critical to protect access to these encrypted passwords. Otherwise, an attacker can mount a dictionary attack to try and crack them. However, even if someone gains access to your entire database of encrypted passwords, they can still only easily gain access to lousy passwords. At least users who choose secure passwords are relatively safe. (It is also critical to protect access to the cleartext password, but at least this mechanism doesn’t require that it be stored in a database for all to see.)
As networking and middleware people, we spend a lot of time thinking about the security of our network protocols, and especially ensuring that someone eavesdropping on a network cannot grab our passwords and other sensitive data as they fly by. But how many times have we heard of a security breach caused by a stolen laptop? I would argue that protecting so-called “data at rest” is just as important, or maybe even more important, as protecting the data flying around your laptop.
Now, back to OAuth. Each “user” in OAuth holds something called an “access token,” which is like a username, and a “token secret,” which is like a password. When a request is sent over the network containing an OAuth authentication token, a bunch of data in the token is encrypted using the token secret, but the secret itself is never sent over the network. That way, regardless of whether SSL is in use, there is no way to gain access to the token secret by sniffing the network.
However, on the server side, in order to validate the OAuth token, the server must make the same calculation that the client made when it encrypted the data to put in the token. That means that both the client side and the server side in OAuth must be able to read the unencrypted token secret from some sort of database. Without it, OAuth doesn’t work. There’s no set of standard ways for storing those keys like there are for passwords, so presumably different implementations are storing them in different ways.
As a result, any client and any server that uses OAuth has to take extra-special care with all those token secrets. Otherwise, anyone who gets access to the database of tokens and secrets used by the back end servers immediately has access to all the OAuth-enabled accounts.
I am not suggesting a change to the OAuth protocol here — it solves an important problem. However, I am suggesting that anyone who implements either the “service provider” or “consumer” side of OAuth take very special care of those tokens!
For instance:
Come to think of it, perhaps the world needs a standard LDAP schema for storing OAuth secrets in a secure way. Anyone care to make a proposal?
Let's boil up the examples and common pitfalls from our last two entries on API Identity and Authorization and more API security choices.
Use API Keys for non-sensitive data (only):
If you have an “open” API - one that exposes data you’d make public on the Internet anyway - consider issuing non-sensitive API keys. These are easy to manipulate and still give you a way to identify users. Armed with an API key, you have the option of establishing a quota for your API, or at least monitoring usage by user. Without one, all you have to go on is an IP address, and those are a lot less reliable than you might think. (Why don’t web sites issue “web site keys?” Because they have cookies.)
For example, the Yahoo Maps Geocoding API issues API keys so it can track its users and establish a quota, but the data that it returns is not sensitive so it’s not critical to keep the key secret.
Use username / password authentication for APIs based on web sites:
If your API is based on a web site, support username/password authentication. That way, users can access your site using the API the same way that they access it using their web browser. For example, the Twitter API supports username/password authentication, so when you access it using a Twitter API client like Spaz or TweetDeck you simply enter the same password you use when you use the twitter.com web site.
However, if you do this, you may want to avoid session-based authentication, especially if you want people to be able to write API clients in lots of environments. It is very simple to use “curl,” for instance, to grab some data from the Twitter API because it uses HTTP Basic authentication. It is a lot more complex if I instead have to call “login,” and extract a session ID from some cookie or header, and then pass that to the real API call I want to make...
OAuth for server-to-server APIs that are based on web sites:
If your API is based on a web site (so you already have a username / password for each account) and the API will also be used by other sites, then support OAuth in addition to username / password authentication. This gives users a lot of peace of mind, because they can authorize another site to access their sensitive data without giving that site their password permanently.
Use SSL for everything sensitive:
Unless your API has only open and non-sensitive data, support SSL, and consider even enforcing it (that is, redirect any API traffic on the non-SSL port to the SSL port). It makes other authentication schemes more secure, and keeps your user’s private API data from prying eyes – and it’s not all that hard to do.
Don’t roll your own!
If the above suggestions still don’t apply to you, then keep looking – between OAuth, OpenID, SAML, HTTP authentication, and WS-Security, there are a lot of authentication schemes, and each has its pros and cons.
So wrapping up API security in our series on 10 API roadmap considerations. Here are some suggested questions you may want to ask when putting together your API security roadmap:
How valuable is the data exposed by your API?
What other schemes and websites will your API and users want to integrate with?
What other expectations might your customers have?
Up next: API Data Protection and thanks to Torbin H. for the photo.
(This entry continues our API Roadmap consideration series and is a continuation of last week's API Identity and Authorization.
Session based Authentication – cumbersome with RESTful APIs
Lots of APIs support session-based authentication. In these schemes, the user first has to call a “login” method which takes the username and password as input and returns a unique session key. The user must include the session key in each request, and call “logout” when they are done. This way, authentication is kept to a single pair of API calls and everything else is based on the session.
This type of authentication maps very naturally to a web site, because users are used to “logging in” when they start working with a particular site and “logging out” when they are done.
However, session-based authentication is much more complex when associated with an API. It requires that the API client keep track of state, and depending on the type of client that can be anything from painful to impossible. Session-based authentication, among other things, makes your API less “RESTful” - an API client can’t just make one HTTP request, but now a minimum of three.
The Role of OAuth
Today, many APIs also support OAuth authentication. OAuth was designed to solve the application-to-application security problem. The idea of OAuth is that it gives the user of a service or web site a way to conditionally grant access to another application. OAuth makes it possible for a human user to individually grant other APIs or sites access to their account without sharing their actual password. It works by giving a “token” to each API or site that will access the account, which the user may revoke at any time they wish.
For instance, if web site FooBar.com wants access to the Twitter API on behalf of John Smith, then OAuth specifies the protocol that FooBar.com must go through to get an OAuth token for the Twitter API. Part of this process requires John Smith to log in to his Twitter account using his normal username and password (which, in the OAuth protocol, are never seen by FooBar.com) and authorize FooBar to access his account. The result is that FooBar.com will have an OAuth token that gives it access to John Smith’s account. If, later on, John Smith decides he no longer trusts FooBar.com, he has the option to revoke that OAuth token without affecting his regular password or any other accounts.
This process makes OAuth the ideal for communication from one application to another – for instance, allowing MySpace to post photos to your Twitter account without requiring that you enter your Twitter password every time you want to do it – but it can be used for any kind of API communications as well.
Two-Way SSL, X.509, SAML, WS-Security...
Once we leave the world of “plain HTTP” we encounter many other ways of authentication, from SAML, X.509 certificates and two-way SSL, which are based on secure distribution of public keys to individual clients, to the various WS-Security specifications, which build upon SOAP to provide... well, just about everything.
An API that will primarily be used by “enterprise” customers – that is, big IT organizations – might consider these other authentication mechanisms such as an. X.509 certificate or SAML for more assurance over authentication process than a simple username/password. Also, a large enterprise may have an easier time accessing an API written to the SOAP standard because those tools can import a WSDL file and generate an API client in a few minutes.
The idea is to know your audience. If your audience is a fan of SOAP and WSDL, then consider some of the more SOAP-y authentication mechanisms like SAML and the various WS-Security specifications. (And if you have no idea what I’m talking about, then your audience probably isn’t in that category!)
Rolling Your Own
In between OAuth, HTTP Basic, and the basic API key are many alternatives. It seems that there are as many other API authentication schemes as there are APIs. Amazon Web Services, Facebook, and some Google APIs, for instance, are some big APIs that combine an API key with both public and secret data, usually through some sort of encryption code, to generate a secure token for each request.
The issue – every new authentication scheme requires API clients to implement it. On the other hand, OAuth and HTTP Basic authentication are already supported by many tools. The big guys may be able to get away with defining their own authentication standards but it’s tough for every API to do things its own way.
SSL
Most authentication parameters are useless, or even dangerous, without SSL. For instance, in “HTTP Basic” authentication the API must be able to see the password the client used, so the password is encoded – but not encrypted – in a format called “base 64.” It’s easy turn this back into the real password. The antidote to this is to use SSL to encrypt everything sent between client and server. This is usually easy to do and does not add as much of a performance penalty as people often think. (With Sonoa’s products, we see a 10-15 percent drop in throughput when we add SSL encryption.)
Some security schemes, such as OAuth, are designed to be resistant to eavesdropping. For instance, OAuth includes a “nonce” on each request, so that even if a client intercepts an OAuth token on one request, they cannot use it on the next. Still, there is likely other sensitive information in the request as well.
In general, if your API does not use SSL it also potentially exposes everything that your API does.
Next time, we'll make some recommendations among different options - or get our full paper here.
(This is part 3 in our series on "Is your API naked? 10 API Roadmap considerations")
We’ve seen very few API providers with a completely open API – almost all employ at least one of these:
Do you need them all? Maybe not. Some APIs only need only to establish identity and don’t really need to authenticate or authorize.
API Identity vs. Authentication - Compare Google Maps and Twitter
Take Yahoo and Google maps – they are fairly open. They want to know who you are but they aren’t concerned what address you are looking up. So they use an “API key” to establish identity, but don’t authenticate or authorize. So if you use someone else’s API key, it’s not good but not a serious security breach.
The API key lets them identify (most likely) who is making a API call so they can limit on the number of requests you can make. Identity is important here to keep service volume under control.
Then take Twitter’s API - open for looking up public information about a user, but other operations require authentication. So Twitter supports both username/password authentication as well as OAuth. Twitter also has authorization checks in its code, so that you cannot “tweet” on behalf of another user without either their password or an OAuth key to their account. This is an example of an API that implements identify, authentication and authorization.
The “API Key” – do you need one?
API keys originated with the first public web services, like Yahoo and Google APIs. The developers wanted to have some way to establish identity without having the complexity of actually authenticating users with a password, so they came up with the “API key,” which is often a UUID or unique string. If the API key doesn’t grant access to very sensitive data, it might not be critical to keep secret, so this use of the API key is easy for the consumers of the API to use however they invoke the API.
An API key gives the API provider a way to (most of the time) know the identity of each caller to maintain a log and establish quotas by user (see the last section).
Usernames and Passwords – again, see Twitter
With more sensitive data a simple, API key is not enough, unless you take measures to ensure users keep the key secret. An alternative is username/password authentication, like the authentication scheme supported by the vast majority of secure web sites.
It’s easiest to use “HTTP Basic” authentication that most websites use. The advantage of using this technology is that nearly all clients and servers support it. There is no special processing required, as long as the caller takes reasonable precautions to keep the password secret.
Twitter simplifies things for their users by using usernames and passwords for API authentication. Every time a user starts a Twitter client, it either prompts for the username and password to use, or it fetches them from the disk (where it is somehow scrambled or encrypted where possible). So here it makes a lot of sense to have the same username / password for the Twitter API that it used for the web site.
Usernames and passwords also work well for application-to-application communications. The trick - the password must be stored securely, and if it’s being used by a server, where do you store it? If you are running an application server that uses a database, you already have solved this same problem, because the database usually requires a password too. Better application server platforms include a “credential mapper” that can be used to store such passwords relatively securely.
There is a lot we'd like to write about around security so we'll split this up into a couple entries. Next time: Session-based authentication, OAuth, SSL and WS-Security, and rolling your own.
Today, Sonoa is joining the Cloud Security Alliance. Why are we doing this?
The first reason is because we’ve talked to hundreds of companies who are building APIs and web services both internally and externally, and for the most part they are using cloud services from other companies, or they are planning to expose their own web services to others on the Internet, or they are running their own infrastructure in the cloud – or all of the above. Cloud computing is a big part of what we do, and we want to make it succeed. The Cloud Security Alliance is a great group of experienced security architects working on solving the most vexing problem faced by companies hoping to take advantage of cloud computing – security.
We encounter security issues all the time when we talk to customers about their own experiences with APIs and cloud services. For instance, there seem to be as many ways to authenticate API users as there are companies publishing APIs. There are venerable standards like HTTP authentication, WS-Security, and two-way SSL, new ones like OAuth and OpenID, and the countless other schemes that API providers also come up with. How does a new API provider deal with all those standards? How does a company consuming some of all of these APIs deal with the proliferation of authentication mechanisms?
In this area, we are not necessarily looking for the CSA to define new standards, but to spend some time identifying best practices for producers and consumers of these APIs, and helping them choose when its necessary to make a choice.
We also encounter security issues when companies are looking to take advantage of cloud computing – especially when they are planning to run some or part of their infrastructure on a public cloud platform. What is the most effective way to connect services on a public cloud with services running behind the traditional corporate firewalls? What kinds of data can be sent to a public cloud platform and what data must remain in a corporate-owned data center? What are the best practices around data encryption, authentication, data retention, and the maze of legal requirements about all this? On these areas, the CSA has already shown itself to be leading the field, and we would like to help.
-Greg
