Why you need OAuth for your API or App »

I've been posting a bit on OAuth best practices recently. But I want to take a step back and talk about why OAuth is important in the first place, why it's difficult, and how to think about it for your API.

Why is OAuth important?
 
First, OAuth supports delegated authentication between web apps. That means that if web app X offers an API, and web site Y wants to use that API, then no one needs to type their "X" password into a screen controlled by "Y." This is a big deal and it's being used by a large and growing number of apps.
 
Second, OAuth does this using a security token called an "access token." An OAuth access token is a unique identifier that allows whoever has the token to get access from one and only one application to one and only one API. (The creators of OAuth describe this access token as a "valet  key," and that's a good description.)
 
An access token is different from a password, which allows whoever has the password to get access from any application to an API, because it is tied to a single application. Plus, access tokens are large and random, so they're not subject to a dictionary attack like a password, and they are hard to share. It's not going to be the case that your access token for Facebook is going to be the same one that you use for Foursquare, whereas if you're like most humans you might use the same password!
 
Third, in order to get access to an access token, an application itself must somehow be authenticated. That means that not only do users need security credentials in an OAuth world, but so do applications. This is a good thing because apps can't always be trusted, and with OAuth each API provider has the ability to revoke access to a rogue application if necessary. (Although there's no way to make it totally impossible for a determined hacker to extract the credentials from a mobile app given enough time and effort, the existence of these credentials adds a layer of security that wouldn't be there otherwise.)
 
Fourth, OAuth makes it possible to separate the assignment of API security credentials from the process of authenticating users to a web site. What does that mean? It means that if you are big company with a web app, and you want to offer an API, it's not necessary to wire the API into the web site infrastructure. Instead, the API can be implemented alongside the web site, and only link to one web page for the actual user authentication. (And even this can be done in other ways.) That means that companies of all sizes can build OAuth-enabled APIs without impacting their existing applications, and that's a good thing.
 
Getting all these things right is hard. For instance, the very first "production" version of OAuth 1.0 didn't actually do the first thing -- authentication delegation -- correctly and as a result the spec itself had to be patched (that's why we talk about OAuth 1.0a now). A lot of people looked at 1.0 without realizing this, which goes to show how tricky it is to create a new security standard from the ground up without making big mistakes.
 
Why has OAuth been difficult?
 
Right now, the OAuth world is sadly full and ambiguity. This is largely because of implementation work on the OAuth 2.0 spec, which offers the promise of an easy-to-use OAuth specification, but at the same time keeps changing. We fully expect the rate of change to slow or even stop in the next few months, but no one knows for sure, especially since it is an IETF process, which takes time.
 
However, it takes time to do a job right, and security standards that impact the whole Internet certainly deserve to be done right.
 
Unfortunately, that puts today's API implementors between a rock and a hard place. Until OAuth 2.0 is a little more stable, you have the choice of implementing the latest draft and being prepared for changes, and implementing OAuth 1.0a, which makes life more difficult for customers.
 
What should you do?
 
Be patient. It's possible to build an effective API right now using either a current draft of OAuth 2.0 (see Facebook) or OAuth 1.0a (see Twitter).
 
However, things are going to change a bit for the near future. Some would take this as a reason to give up, "sunset" OAuth, or declare it "dead" way before its time. I think that building a new security technology from the ground up that has all the advantages I enumerated at the top of this article would be a big mistake.  We have two pretty-good options today in OAuth 1.0a the latest drafts of 2.0 -- stick with them.

Best Practices for OAuth 2.0 vs. OAuth 1.0 - One year later »

Since we first wrote about OAuth 1.0 vs. 2.0 a while back, lots has been happening and lots has changed. The OAuth community has made progress and made changes, and an increasing number of API providers have deployed APIs that use OAuth 2.0. (Similarly, the number of new OAuth 1.0-enabled APIs doesn't seem to be growing.)

We've been busy helping our customers implement OAuth-based APIs, and we've also been watching the process develop. Here are some things that we've learned: 

OAuth is a solution -- not technology.

In order for an API provider to support OAuth, a number of things have to happen. There must be a place where the OAuth protocol is actually implemented -- all of the different URLs for requesting tokens, issuing tokens, redirecting browsers, and so on. This part is a matter of implementing the standard and doesn't change from implementation to implementation (so, it can be delegated to a third-party component like Apigee Enterprise).

In addition, there has to be somewhere to store OAuth tokens. There is no standard for this -- some use an RDBMS, some use NoSQL, and others use a cloud-hosted service.

Finally, the OAuth solution has to integrate with a provider's "login page," so that when a user wants to get an OAuth token, they are redirected to a legitimate "login" page with a proper SSL certificate,  look and feel, and so on. All of these kinds of things can be different from one provider to another.

At Apigee, we have been working with many companies to deploy APIs built using OAuth. What we've found is that OAuth is not just a technology that you buy or download -- it's a bit of technology and a bit of integration, and to implement it at all you need a platform that can flexibly integrate with what you already have, while adding what you might be missing. 

Be careful.

Since we last blogged about it, OAuth 2.0 has been improved a bit and has changed some basic things -- for instance the name on the HTTP "Authorization" header has changed from "OAuth" to "Bearer." This is a big change, and obviously if an API provider just switches from one value to another without warning, all the clients will break.

OAuth 2.0 is up to draft 13, and with every draft it gets closer to being "done" -- but it's not done yet and I haven't seen anyone say when it will be. That means that if you implement, say, draft 13, you will likely have to change, and find a way to maintain backward compatibility for existing clients.

If that's too much, then implement OAuth 1.0a -- it's as "done" as it's going to get at this point! 

Stick with "bearer tokens" for now in conjunction with SSL.

The latest draft includes support for various types of security tokens,  and a mechanism to plug in new types. The simplest are "bearer tokens." A bearer token is just a big, random string that a client must present on every API call. Bearer tokens are simple because there's no special signature or validation code required on either end. The client is responsible for storing the token in a safe place and sending it with every request. The server is responsible for looking up the token in a database and making sure it's a valid one -- that's it.

However, bearer tokens provide no security unless used in conjunction with SSL -- otherwise, any eavesdropper in a coffee shop can snatch OAuth tokens from  the air. (Fortunately, OAuth tokens can be individually revoked without requiring a password reset.) 

Get ready to change -- or stick with 1.0a.

We said this before but it bears repeating -- if you implement OAuth 2.0, it is likely that you will have to change your servers to support newer versions, and also maintain backward compatibility.

One way to ease the pain is to put version numbers in the URLs that you hand out for the various steps in the OAuth flow. So, instead of using:

http://api.foo.com/oauth/authorize

use:

http://api.foo.com/oauth/13/authorize

because you know that there will be future versions and that they will be different...

Get ready for more options.

 Part of the OAuth 2.0 work defines some new security token types other than "bearer." Right now only "bearer" seems to be sufficiently mature from a spec standard, but this will change over the next few months. Here are some of the options:

1.       Bearer -- as we mentioned before, this is just a long, random string. The advantage is that it's extremely easy to implement on both client and server. The disadvantage is that it requires SSL for all API calls. Also, if API traffic passes through multiple endpoints such as proxies, then the proxies will be able to see the token as it passes by.

2.       Mac -- this is equivalent to the token scheme in OAuth 1.0, in that it requires both a key and secret, and uses an "Hmac" algorithm to encrypt some data on each request. The result is that the request is only valid if both client and server have the same keys, and there is no way for an intermediary to re-create the original request without a password. In other words, it is secure even if SSL is not used or even if the request is intercepted by a proxy.

3.       SAML -- this uses SAML assertions on each request as a way to establish the client's identity. This is helpful if you already have SAML infrastructure for either the client or server in your environment.

Tech Talk: API Security and Threat Protection »

Greg recorded a few whiteboard talks last month - this one is a good summary of recent posts on API Keys, API security recommendations, and OAuth best practices. 

(And here are some of our high-level API security features if you are looking into this.)

OAuth — Take care with those keys! »

A lot has been happening with OAuth recently. Earlier this year a security hole was discovered in the protocol which exposed it to a potential “social engineering” attacks.  However, the OAuth community is working on a revision to the spec that will eliminate this particular hole.

Last week we wrote a bit on OAuth as an option for API security.  But today I wanted to bring up a related OAuth issue - how do you securely manage all those keys?

With traditional username / password authentication, good security practices require you don't just have a big database on the back end with a list of unencrypted passwords. Instead, a hash of the password is stored, preferably using a salt. So someone who can read the password file can verify they have the right password, but cannot see the actual password.

It is still critical to protect access to these encrypted passwords. Otherwise, an attacker can mount a dictionary attack to try and crack them. However, even if someone gains access to your entire database of encrypted passwords, they can still only easily gain access to lousy passwords. At least users who choose secure passwords are relatively safe. (It is also critical to protect access to the cleartext password, but at least this mechanism doesn’t require that it be stored in a database for all to see.)

As networking and middleware people, we spend a lot of time thinking about the security of our network protocols, and especially ensuring that someone eavesdropping on a network cannot grab our passwords and other sensitive data as they fly by. But how many times have we heard of a security breach caused by a stolen laptop? I would argue that protecting so-called “data at rest” is just as important, or maybe even more important, as protecting the data flying around your laptop.

Now, back to OAuth. Each “user” in OAuth holds something called an “access token,” which is like a username, and a “token secret,” which is like a password. When a request is sent over the network containing an OAuth authentication token, a bunch of data in the token is encrypted using the token secret, but the secret itself is never sent over the network. That way, regardless of whether SSL is in use, there is no way to gain access to the token secret by sniffing the network.

However, on the server side, in order to validate the OAuth token, the server must make the same calculation that the client made when it encrypted the data to put in the token. That means that both the client side and the server side in OAuth must be able to read the unencrypted token secret from some sort of database. Without it, OAuth doesn’t work. There’s no set of standard ways for storing those keys like there are for passwords, so presumably different implementations are storing them in different ways.

As a result, any client and any server that uses OAuth has to take extra-special care with all those token secrets. Otherwise, anyone who gets access to the database of tokens and secrets used by the back end servers immediately has access to all the OAuth-enabled accounts.

I am not suggesting a change to the OAuth protocol here — it solves an important problem. However, I am suggesting that anyone who implements either the “service provider” or “consumer” side of OAuth take very special care of those tokens!

For instance:

•     If they’re on a regular disk file, protect them using filesystem permissions, make sure that they’re encrypted, and hide the password well.
•     If they’re in a database, encrypt the fields, store the key well, and protect access to the database itself carefully.
•     If they’re in LDAP, do the same.

Come to think of it, perhaps the world needs a standard LDAP schema for storing OAuth secrets in a secure way. Anyone care to make a proposal?

Don’t roll your own: API Security Recommendations »

Let's boil up the examples and common pitfalls from our last two entries on API Identity and Authorization and more API security choices.   

Use API Keys for non-sensitive data (only): 

If you have an “open” API - one that exposes data you’d make public on the Internet anyway -  consider issuing non-sensitive API keys. These are easy to manipulate and still give you a way to identify users. Armed with an API key, you have the option of establishing a quota for your API, or at least monitoring usage by user. Without one, all you have to go on is an IP address, and those are a lot less reliable than you might think. (Why don’t web sites issue “web site keys?” Because they have cookies.)

For example, the Yahoo Maps Geocoding API issues API keys so it can track its users and establish a quota, but the data that it returns is not sensitive so it’s not critical to keep the key secret.

Use username / password authentication for APIs based on web sites: 

If your API is based on a web site, support username/password authentication. That way, users can access your site using the API the same way that they access it using their web browser. For example, the Twitter API supports username/password authentication, so when you access it using a Twitter API client like Spaz or TweetDeck you simply enter the same password you use when you use the twitter.com web site.

However, if you do this, you may want to avoid session-based authentication, especially if you want people to be able to write API clients in lots of environments. It is very simple to use “curl,” for instance, to grab some data from the Twitter API because it uses HTTP Basic authentication. It is a lot more complex if I instead have to call “login,” and extract a session ID from some cookie or header, and then pass that to the real API call I want to make...

OAuth for server-to-server APIs that are based on web sites:

If your API is based on a web site (so you already have a username / password for each account) and the API will also be used by other sites, then support OAuth in addition to username / password authentication. This gives users a lot of peace of mind, because they can authorize another site to access their sensitive data without giving that site their password permanently.

Use SSL for everything sensitive:

Unless your API has only open and non-sensitive data, support SSL, and consider even enforcing it (that is, redirect any API traffic on the non-SSL port to the SSL port). It makes other authentication schemes more secure, and keeps your user’s private API data from prying eyes – and it’s not all that hard to do.

Don’t roll your own!

If the above suggestions still don’t apply to you, then keep looking – between OAuth, OpenID, SAML, HTTP authentication, and WS-Security, there are a lot of authentication schemes, and each has its pros and cons.

So wrapping up API security in our series on 10 API roadmap considerations. Here are some suggested questions you may want to ask when putting together your API security roadmap:

How valuable is the data exposed by your API?

• If it’s not that valuable, or if you plan to make it as open as possible, is an API key enough to uniquely identify users?
• If it is valuable, can you reuse username and password scheme for each user?
• Are you using SSL? Many authentication schemes are vulnerable without it.

What other schemes and websites will your API and users want to integrate with?

• If your API will be called programmatically by other APIs, or if your API is linked to another web site that requires authentication, have you considered OAuth?
• If you have username/password authentication, have you considered OpenID?
• Can you make authorization decisions in a central place?

What other expectations might your customers have?

• If your customers are enterprise customers, would they feel better about SAML or X.509 certificates?
• Can you change or support more than one your authentication approach for diverse enterprise customers?
• Do you have an existing internal security infrastructure that you need your API to interact with?

Up next:  API Data Protection and thanks to Torbin H. for the photo.

More API Security Choices - OAuth, SSL, SAML, and rolling your own »

(This entry continues our API Roadmap consideration series and is a continuation of last week's API Identity and Authorization.

Session based Authentication – cumbersome with RESTful APIs

Lots of APIs  support session-based authentication. In these schemes, the user first has to call a “login” method which takes the username and password as input and returns a unique session key. The user must include the session key in each request, and call “logout” when they are done. This way, authentication is kept to a single pair of API calls and everything else is based on the session.
 
This type of authentication maps very naturally to a web site, because users are used to “logging in” when they start working with a particular site and “logging out” when they are done.
 
However, session-based authentication is much more complex when associated with an API. It requires that the API client keep track of state, and depending on the type of client that can be anything from painful to impossible. Session-based authentication, among other things, makes your API less “RESTful” - an API client can’t just make one HTTP request, but now a minimum of three.
 
The Role of OAuth

Today, many APIs also support OAuth authentication. OAuth was designed to solve the application-to-application security problem. The idea of OAuth is that it gives the user of a service or web site a way to conditionally grant access to another application. OAuth makes it possible for a human user to individually grant other APIs or sites access to their account without sharing their actual password. It works by giving a “token” to each API or site that will access the account, which the user may revoke at any time they wish.

For instance, if web site FooBar.com wants access to the Twitter API on behalf of John Smith, then OAuth specifies the protocol that FooBar.com must go through to get an OAuth token for the Twitter API. Part of this process requires John Smith to log in to his Twitter account using his normal username and password (which, in the OAuth protocol, are never seen by FooBar.com) and authorize FooBar to access his account. The result is that FooBar.com will have an OAuth token that gives it access to John Smith’s account. If, later on, John Smith decides he no longer trusts FooBar.com, he has the option to revoke that OAuth token without affecting his regular password or any other accounts.

This process makes OAuth the ideal for communication from one application to another – for instance, allowing MySpace to post photos to your Twitter account without requiring that you enter your Twitter password every time you want to do it – but it can be used for any kind of API communications as well.
 
Two-Way SSL, X.509, SAML, WS-Security...

Once we leave the world of “plain HTTP” we encounter many other ways of authentication, from SAML, X.509 certificates and two-way SSL, which are based on secure distribution of public keys to individual clients, to the various WS-Security specifications, which build upon SOAP to provide... well, just about everything.

An API that will primarily be used by “enterprise” customers – that is, big IT organizations – might consider these other authentication mechanisms such as an. X.509 certificate or SAML for more assurance over authentication process than a simple username/password. Also, a large enterprise may have an easier time accessing an API written to the SOAP standard because those tools can import a WSDL file and generate an API client in a few minutes.

The idea is to know your audience. If your audience is a fan of SOAP and WSDL, then consider some of the more SOAP-y authentication mechanisms like SAML and the various WS-Security specifications. (And if you have no idea what I’m talking about, then your audience probably isn’t in that category!)

 
Rolling Your Own

In between OAuth, HTTP Basic, and the basic API key are many alternatives. It seems that there are as many other API authentication schemes as there are APIs. Amazon Web Services, Facebook, and some Google APIs, for instance, are some big APIs that combine an API key with both public and secret data, usually through some sort of encryption code, to generate a secure token for each request.

The issue – every new authentication scheme requires API clients to implement it. On the other hand, OAuth and HTTP Basic authentication are already supported by many tools. The big guys may be able to get away with defining their own authentication standards but it’s tough for every API to do things its own way.
 
SSL

Most authentication parameters are useless, or even dangerous, without SSL. For instance, in “HTTP Basic” authentication the API must be able to see the password the client used, so the password is encoded – but not encrypted – in a format called “base 64.” It’s easy turn this back into the real password. The antidote to this is to use SSL to encrypt everything sent between client and server. This is usually easy to do and does not add as much of a performance penalty as people often think. (With Apigee’s products, we see a 10-15 percent drop in throughput when we add SSL encryption.)

Some security schemes, such as OAuth, are designed to be resistant to eavesdropping. For instance, OAuth includes a “nonce” on each request, so that even if a client intercepts an OAuth token on one request, they cannot use it on the next. Still, there is likely other sensitive information in the request as well.

In general, if your API does not use SSL it also potentially exposes everything that your API does.

Next time, we'll make some recommendations among different options - or get our full paper here.

Do you need API keys?  API Identity vs. Authorization »

(This is part 3 in our series on "Is your API naked? 10 API Roadmap considerations")

We’ve seen very few API providers with a completely open API – almost all employ at least one of these:

• Identity - who is making an API request?
• Authentication - are they really are who they say they are?
• Authorization – are they allowed to do what they are trying to do?

Do you need them all?  Maybe not.  Some APIs only need only to establish identity and don’t really need to authenticate or authorize.

API Identity vs. Authentication - Compare Google Maps and Twitter  

Take Yahoo and Google maps – they are fairly open.  They want to know who you are but they aren’t concerned what address you are looking up. So they use an “API key” to establish identity, but don’t authenticate or authorize. So if you use someone else’s API key, it’s not good but not a serious security breach.

The API key lets them identify (most likely) who is making a API call so they can limit on the number of requests you can make. Identity is important here to keep service volume under control.

Then take Twitter’s API -  open for looking up public information about a user, but other operations require authentication. So Twitter supports both username/password authentication as well as OAuth. Twitter also has authorization checks in its code, so that you cannot “tweet” on behalf of another user without either their password or an OAuth key to their account. This is an example of an API that implements identify, authentication and authorization.

The “API Key” – do you need one? 

API keys originated with the first public web services, like Yahoo and Google APIs.  The developers wanted to have some way to establish identity without having the complexity of actually authenticating users with a password, so they came up with the “API key,” which is often a UUID or unique string. If the API key doesn’t grant access to very sensitive data, it might not be critical to keep secret, so this use of the API key is easy for the consumers of the API to use however they invoke the API.

An API key gives the API provider a way to (most of the time) know the identity of each caller to maintain a log and establish quotas by user (see the last section). 

Usernames and Passwords – again, see Twitter

With more sensitive data a simple, API key is not enough, unless you take measures to ensure users keep the key secret. An alternative is username/password authentication, like the authentication scheme supported by the vast majority of secure web sites. 

It’s easiest to use “HTTP Basic” authentication that most websites use. The advantage of using this technology is that nearly all clients and servers support it. There is no special processing required, as long as the caller takes reasonable precautions to keep the password secret.

Twitter simplifies things for their users by using usernames and passwords for API authentication.  Every time a user starts a Twitter client, it either prompts for the username and password to use, or it fetches them from the disk (where it is somehow scrambled or encrypted where possible). So here it makes a lot of sense to have the same username / password for the Twitter API that it used for the web site.

Usernames and passwords also work well for application-to-application communications. The trick - the password must be stored securely, and if it’s being used by a server, where do you store it?  If you are running an application server that uses a database, you already have solved this same problem, because the database usually requires a password too. Better application server platforms include a “credential mapper” that can be used to store such passwords relatively securely.

There is a lot we'd like to write about around security so we'll split this up into a couple entries.  Next time:  Session-based authentication, OAuth, SSL and WS-Security, and rolling your own.

We’re joining the Cloud Security Alliance »

Today, Sonoa is joining the Cloud Security Alliance. Why are we doing this?

The first reason is because we’ve talked to hundreds of companies who are building APIs and web services both internally and externally, and for the most part they are using cloud services from other companies, or they are planning to expose their own web services to others on the Internet, or they are running their own infrastructure in the cloud – or all of the above. Cloud computing is a big part of what we do, and we want to make it succeed. The Cloud Security Alliance is a great group of experienced security architects working on solving the most vexing problem faced by companies hoping to take advantage of cloud computing – security.

We encounter security issues all the time when we talk to customers about their own experiences with APIs and cloud services. For instance, there seem to be as many ways to authenticate API users as there are companies publishing APIs. There are venerable standards like HTTP authentication, WS-Security, and two-way SSL, new ones like OAuth and OpenID, and the countless other schemes that API providers also come up with. How does a new API provider deal with all those standards? How does a company consuming some of all of these APIs deal with the proliferation of authentication mechanisms?

In this area, we are not necessarily looking for the CSA to define new standards, but to spend some time identifying best practices for producers and consumers of these APIs, and helping them choose when its necessary to make a choice.

We also encounter security issues when companies are looking to take advantage of cloud computing – especially when they are planning to run some or part of their infrastructure on a public cloud platform. What is the most effective way to connect services on a public cloud with services running behind the traditional corporate firewalls? What kinds of data can be sent to a public cloud platform and what data must remain in a corporate-owned data center? What are the best practices around data encryption, authentication, data retention, and the maze of legal requirements about all this? On these areas, the CSA has already shown itself to be leading the field, and we would like to help.

-Greg