RESTful API Design: authentication »

This time, in this series about pragmatic RESTful API Design, I'll discuss authentication.

There are many schools of thought - my colleagues at Apigee and I don't always agree on how to handle authentication - but overall here's my take.

Let's look at these three top services. See how each of these services handles things differently:

PayPal

Permissions Service API

Facebook

OAuth 2.0

Twitter

OAuth 1.0a

Note that PayPal's proprietary three-legged permissions API  was in place long before OAuth was conceived.

What should you do?

Use the latest and greatest OAuth - OAuth 2.0 (as of this writing)

Don't do something *like* OAuth, but different
It will be frustrating to app developers if they can't use an OAuth library in their language because of your variation.

Next time: Versions - how many? Meanwhile, I'd love to hear from you over on the API Craft Google Group.

“Bigger, Better Business with OAuth” API strategy webinar - video & slides »

Thanks to all that attended yesterday's webinar "Bigger, Better Business with OAuth."  

Unfortunately, we had a Webex outage which cut the webinar audio off after a few minutes, but thanks to @sramji and @landlessness for finishing out the webinar for the video (and slides below) after the outage.

OAuth: The next Big Thing in Security »

Today at #defragcon our @sramji gave this talk making the case for OAuth as a business imperative.

(As Sam mentions on the last slide, no developers were harmed in the production of this presentation.) 

Trust, Safety, and the OAuth API »

At Apigee, we've been developing a secure, robust API management platform since 2005 and have been running it in the cloud since 2008.  We're proud that some of the most demanding enterprise customers like Netflix, Comcast, and GameSpy have made Apigee technology part of their API platform.

We believe that it should be easy to start working with APIs, so last week we launched a new service called the OAuth API, which takes the pain out of getting up and running with APIs that use OAuth.

The story of the OAuth API is the story of yet another function that we used to think of as occurring on a device but that is now moving into the cloud.

It marks the beginning of a trend of APIs being manipulated before they get to the client, since some things are better handled as a service. Authentication mediation is one of those things.

Apigee has built a business of using our API Gateway to publicly expose backend APIs in a safe way, such as by adding OAuth as the mechanism by which applications can be granted permissions by end-users of applications.

The OAuth API takes that model and reverses it, taking something that is complicated for developers to implement (especially across multiple APIs) and simplifying it.

But is it secure?

Should you trust your OAuth keys to a cloud provider, and to Apigee in particular?

In many ways, the cloud might be more secure, since providers do things you might never do on your own. Far too many applications never bother with encrypting users' secrets in their own databases. When you use the OAuth API, you get that for free, in addition to the API normalization and long-term maintenance wins. 

If you do trust the cloud, should you trust Apigee? We respectfully submit that it comes down to three things: technology, processes, and people. 

Technology

Our technology processes billions of OAuth transactions a day. Our Apigee Free tools—including the OAuth API—are built upon this same platform that powers 250 enterprise customers.  It is a core requirement that our servers be both security hardened and highly available.

Processes

We encrypt the secrets entrusted to our database. We run security audits. We involve our security team members throughout the design and architecture process.We use the same software, made by the same people, working under the same processes behind the products that power the APIs of the most demanding Fortune 500 companies. We use industry standard security practices.

And in the end, it's still an OAuth token that is being stored, not a user's password. It can be revoked by a user at any time, and developers can still invalidate their consumer key and secret at any time.

People

Here at Apigee we have lots of experience in API security and operations in the cloud, again as a result of working with demanding enterprise customers for years, many of whom entrust us to protect their API traffic, their organizations, and their customers.

Q&A for OAuth and API Best Practices Webinar »

Thanks to all that attended last week's webinar:  OAuth: The Big Picture (slides and video here).

There were great questions  - here are thoughts on those we didn't have time to discuss.

Q: Would you recommend only using oauth for passwords and usernames?  What about more confidential items like card numbers? (John G)

 I'm not sure I completely understand - but are you asking - if a user can authenticate to OAuth using a credit card number rather than a username/password?

That's theoretically possible but I'm not sure it's a good idea. Also, keep in mind that most credit-card authorization systems use a unique ID to map your transaction to an actual credit card, and for PCI compliance the actual number itself is stored in a PCI-compliant card system. In other words,  I'm not sure that OAuth would be solving a new problem here.

Q: Can you talk about the practices around validity periods of tokens, given that their decoupled from password validity? (Eve Maler)

 Sure. First, why do OAuth tokens need to be refreshed at all?

For one, when an OAuth 1.0a or 2.0 "mac" signature is used without the benefit of SSL, it could theoretically be cracked given enough time and computing resources. However when SSL is in use, you can't do this by just sniffing the network, although if you can intercept a request some other way then you could theoretically do this. So, one reason to refresh the token is so that it can't be cracked.

Another reason is just because as users pass their computers around, phones, etc, it's possible that they won't realize that they contain OAuth tokens. This is a much more subtle and undefined reason than the first.

Some applications give out tokens that never expire, which leads to the theoretical "cracking" example if you are not using SSL. On the other hand, I've seen applications with tokens that expire in 15 minutes, which is not very user-friendly. I don't have a specific recommendation right now and I'm not sure if I have seen another as well.

I actually think that this is a good area for some new research if there's someone out there looking for a project -- I don't think that we've quantified this well.

Q: If the user is entering the user/pass on the browser, even though it may be a trusted website URL, it is still vulnerable for some client to sniff the credentials as it is on the machine itself!! (Kaushik)

 Sure -- nothing on a computer can be completely trusted, and the person operating the machinery can be trusted least of all!

What OAuth does for a web app, is keep the passwords from the server that you are using and prevents your password from being spread around the Internet from server to server. Given the number of security breaches in which servers are compromised and user accounts exposed,  this fixes a big problem.

What OAuth does for a mobile app is keep the password from the application that you are running. I think you can trust the browser embedded in your OS more than you can trust the application built by a developer that you don't know, but I know that others here disagree. (And yes, I realize that an app can put up a fake browser screen. I wonder if such an app can get through the Apple App Store? Does anyone have knowledge about how real a threat this is?)

Q: What options does Apigee support for server-to-server API calls where we want to verify the caller is a valid caller?  i.e. request signature? (Mark Visosky)

 We can support two-way SSL, WS-Security with UserNameToken profile or X.509 profile, SAML, XML encryption / digital signatures, and of course HTTP basic authentication.

Q: if i change my password on twitter, then try to use a previously working app, it makes me reauth (Daniel)

 I didn't know that about Twitter and should have tried before the call.

Q: Thanks for the webinar, you've clearly thought a lot about authentication and security issues.  However, I cannot grant you the point that opening up a web browser where you enter your password to the service provider keeps the user's password secure.  A malicious desktop or mobile application will be able to trick the user into divulging their password by spoofing the oAuth page in some way. oAuth sign in page, I mean (Kevin)

Thanks for the questions. There are certainly other ways to trick a user into divulging their password even beyond what you suggest... That doesn't mean that we have yet abandoned passwords completely.

However, an API provider that DOES want to abandon passwords and replace them with some sort of multi-factor authentication, VPN, SAML, hardware dongle, whatever, can do that by replacing the authentication screen in OAuth with one that requires whatever that API provider wants WITHOUT requiring any change at all on the client. This is a lot harder to do once there are clients out there that have hard-coded username/password authentication.

Q: if it again asks for reauth then does that mean it is using the changed password somehow..? (Kaushik)

An API has the option of revoking the OAuth tokens at any time, and some may choose to revoke them whenever the password is changed. However, OAuth doesn't require or specify that.

Q: Isn't the full OAuth dance only necessary if a user needs to authorize another application to perform actions on their behalf?  i.e. giving authorization to bit.ly to submit tweets through my twitter account.  What about API's where we just want to validate callers are who they say they are but no end-user is involved. (Mark V)

 For server-to-server communications, two-way SSL, or "regular" SSL with HTTP Basic and a long, random password are fine ways to solve this IMHO.

For applications that need to authenticate themselves as "the app" rather than as a particular user,  you can assign each app a unique and random "application key" and include that in each request. Use SSL as well if keeping the app key a secret is important to your business.

Q: Do services need to store/control which third-parties can retrieve your OAuth credentials? Or is it generally the rule that if you expose OAuth, any app can use it? (Kenneth)

 In order to build an application that uses an API via OAuth, the application needs a set of application credentials -- that's the case for OAuth 1.0 and 2.0.

Q: i want to write an OAuth2 *provider* - what libraries are available to me? (Kevin H)

There is a long list of open-source projects at oauth.net under "Code." It's not something that can usually be just "plugged in" to a server, however, because it depends on how you want to authenticate users, where you want to store credentials, etc. At this point in time I should mention that an API gateway product such as Apigee Enterprise is one great way to get it done quickly.

Q: If oAuth requires SSL in 2.0, the request signing step becomes useless, no? (Kevin W)

When OAuth 2.0 is used with a "bearer token," then there is no secret and no request signing -- you use SSL instead.  When OAuth 2.0 is used with a "Mac token" then it is just like 1.0a.

Q: Are the specs for SSL ,SAML assertion available to be used or implemented for my server..? (Kaushik R)

Absolutely. Check ietf.orf and search for "OAuth." There are active groups working on all these specs.

Q: If you're requiring SSL, why not just trade un/pw for a token set and be done with it?  This assumes that the oAuth sign in page is security theater, which I maintain that it is. (Kevin)

@Eve but you can use any RSA spec, as long as the client and server know what to do. (Dan)

un/pw/api key or whatever? (Kevin)

Even the father of OAuth, Eran Hammer-Lahav, says using SSL as a means of authentication is broken (ie, OAuth 2.0 is broken) - (Dan)

 Yes, he is wary of SSL.  That's why he personally ensured that OAuth 2.0 also includes the "mac token" option, which is a way to use OAuth 2.0 with a signature and without SSL, just like 1.0a. This particular spec is newer and changing more quickly but it'll be an important option soon. Given the amount of work he is doing on the spec I don't think he thinks it's broken.

Q: How complex it is to implement OAuth in our own servers..? (Kaushik)

 The hard part is integrating the OAuth flow with your existing authentication mechanisms, so that the browser is redirected and all that. There is also a bunch of work involved in implementing the various authentication flows, and there are quite a few of them. I think there are much more complex things to implement but OAuth does have its challenges.

Q: Where is the OAuth token stored? is it Server side in your example Twitter, or client side BIT.LY or BOTH? I think you mentioned in case of mobile apps it is stored on mobile. (Arpit)

 It depends who is accessing the API on your behalf. The bit.ly web site, for instance,  must store them on the server since it's a web site. The Twitter mobile apps, however,  store it on the mobile device itself.

Q: So are they saying that banks shouldn't use OAuth for banking transactions? (Kenneth K)

 If a bank has a fiduciary responsibility, for example, to sign every transaction with an RSA key that is stored on a hardware key store and managed by a PKI with tight policies and procedures around key distribution, then OAuth isn't going to be used in such an environment any more than plain old username / password combinations. That's the case for financial applications that deal with individual transactions worth millions or billions of dollars.

Seeing that many of us bank on our mobile phones or web browsers using just a username and password for authentication, I think that OAuth is just fine for a wide variety of other financial apps.

Q: The oAuth login screen is security theater - I can spoof it or otherwise trick the user if I am a malicious mobile or desktop app developer.  If you can't force me (the third party app developer) to NOT store the un/pw, why make me goof around with an embedded browser page in my desktop/mobile app?  Thoughts? (Kevin)

 I think I tried to answer this elsewhere, but in general if you really want to do this then OAuth 2.0 actually includes an authentication flow that lets you get a token by simply sending the username / password to the server.

Q: oAuth gives a false sense that your transactions is secure, but the spec really does not mandate any message level security?? (Praveen)

 Well the spec strongly recommends it but the spec can't force it -- it is up to each individual API to require message-level security in the right places.

Q: Advanced talk should have a topic on usage of the "SCOPE" parameter (Vladimir)

I think that the whole "scope" thing would be an interesting seminar,  yes. Let's consider it,  but we'll need a lot of good examples!

Thanks again to everyone and hope to see you on our next webinar:   "Boss, we need an API!"

Video and Slides - OAuth: The Big Picture »

If you weren't able to attend last week's API Best Practices Webinar #8 "OAuth for Your API: The Big Picture," or if you just want to review, here are the slides and video! 

OAuth is taking off as a standard way for apps and websites to handle authentication. But OAuth is a fast-moving spec that can be hard to pin down. In this webinar, our CTO Greg Brail and Sr. Architect Brian Pagano give the "big picture straight talk" on the business and operational benefits of OAuth, different OAuth flavors and which one you should choose. 

Questions we address: 

• What is OAuth (in plain English) and when should you use it for your API? 
• The Pros and Cons of different OAuth flavors (1.0a, 2.0, two-legged, three-legged) 
• "Big Picture" OAuth considerations, best practices, and decisions to make 

Are all OAuth implementations Created Equal? »

We've implemented a couple dozen OAuth implementations in the past months, and no two have been the same.

We've seen Oauth 1.0a (2-legged,3-legged), Oauth 2.0 (username password, user-agent, web-server flow) with custom token types (bearer,mac, saml).  Just to name a few.

We wanted to write some posts that discuss factors we see that drive these variances and their implications.  We also want to compare and contrast each of these variances along with our own lessons learned. 

First, why the variances?  In our experience, the primary cause of the variance is the type of application that the API is targeted for, which tend to break into one or more of 3 types.

Partner Apps - typically a "closed" API available only to a select group of partners with whom you collaborate very closely to build the apps. For instance - an health insurance provider who is exposing an API that a partner can use to build some health related activity tracing applications.

Open - Browser / Mobile Apps - APIs are exposed to a large community of 3rd party developer from whom you are expecting a large scale of applications to drive innovation. Here you do not control who, how many and what kind of applications get build using the APIs.   For example, mobile applications that integrate with the social networking sites.

Enterprise Apps -  when the API is exposed for use primarily by other enterprise customers - much like a B2B scenario - to allow the customers the freedom and flexibility of building in house Applications and services that mashup one of more of the applications from multiple vendors for internal optimizations.  A good example could be an enterprise that wants to use the Sales Force API along with one other internal apps to build an internal collaboration app.

Depending on the application type above, your approach to an OAuth implementation may vary:

An API for Partner Apps will typically follow a “User name / password” flow.  In this case the end user’s  user name and password will be passed from the user to the partner app and the partner app transmits this to the origin / backend API. The key consideration here is that the partner app will have direct access to the end users credentials and is trusted to treat this information sensibly.

In the Open Browser / Mobile apps scenario the app itself is not inherently trusted and is not allowed to handle the end user credentials directly. The onus of authenticating the end user falls squarely on the origin / backend API provider. The provider (after the end user authentication is successful) will simply issue an access token to the app – this token will correspond to the set of capabilities that the end user has authorized the application for. The sequence of steps followed in this case is commonly referred to as the “user-agent flow” model.

The Enterprise Apps scenario is by far the most robust flow (and understandably, the most complex one too). Here the access token is given to an intermediary server instead of the client device and this facilitates a more secure transmission of the access token. In this scenario, it is possible to restrict the source from which the web service is invoked. This sequence of steps is commonly referred to as the “web server flow” model.

Sounds complicated?  It is. We will try to simplify each of these variants and throw some more light in a series of follow-up entries. Stay tuned!

Shiny Shiny Monday: Two-legged OAuth, Custom Token, and more… »

We roll out big features with lots of fanfare, but the little ones roll out as they are ready. On Mondays we'll highlight recent changes here. For this week we've released:

1. Support for file attachments! Now you can attach a photo to your Twitter profile or Facebook via an API call (screenshot below). Note that you can attach via URL or by uploading from your local machine. We recommend the URL method, as that way it will be available to others if you choose to share a snapshot.
2. Custom Token - after launching Apigee To-Go, people clamored for support for custom tokens, so we added it!
3. Two legged OAuth - we've also added 2-legged OAuth, a very valid use case when people have APIs that aren't based around the concept of users granting 3rd parties permissions.

Why you need OAuth for your API or App »

I've been posting a bit on OAuth best practices recently. But I want to take a step back and talk about why OAuth is important in the first place, why it's difficult, and how to think about it for your API.

Why is OAuth important?
 
First, OAuth supports delegated authentication between web apps. That means that if web app X offers an API, and web site Y wants to use that API, then no one needs to type their "X" password into a screen controlled by "Y." This is a big deal and it's being used by a large and growing number of apps.
 
Second, OAuth does this using a security token called an "access token." An OAuth access token is a unique identifier that allows whoever has the token to get access from one and only one application to one and only one API. (The creators of OAuth describe this access token as a "valet  key," and that's a good description.)
 
An access token is different from a password, which allows whoever has the password to get access from any application to an API, because it is tied to a single application. Plus, access tokens are large and random, so they're not subject to a dictionary attack like a password, and they are hard to share. It's not going to be the case that your access token for Facebook is going to be the same one that you use for Foursquare, whereas if you're like most humans you might use the same password!
 
Third, in order to get access to an access token, an application itself must somehow be authenticated. That means that not only do users need security credentials in an OAuth world, but so do applications. This is a good thing because apps can't always be trusted, and with OAuth each API provider has the ability to revoke access to a rogue application if necessary. (Although there's no way to make it totally impossible for a determined hacker to extract the credentials from a mobile app given enough time and effort, the existence of these credentials adds a layer of security that wouldn't be there otherwise.)
 
Fourth, OAuth makes it possible to separate the assignment of API security credentials from the process of authenticating users to a web site. What does that mean? It means that if you are big company with a web app, and you want to offer an API, it's not necessary to wire the API into the web site infrastructure. Instead, the API can be implemented alongside the web site, and only link to one web page for the actual user authentication. (And even this can be done in other ways.) That means that companies of all sizes can build OAuth-enabled APIs without impacting their existing applications, and that's a good thing.
 
Getting all these things right is hard. For instance, the very first "production" version of OAuth 1.0 didn't actually do the first thing -- authentication delegation -- correctly and as a result the spec itself had to be patched (that's why we talk about OAuth 1.0a now). A lot of people looked at 1.0 without realizing this, which goes to show how tricky it is to create a new security standard from the ground up without making big mistakes.
 
Why has OAuth been difficult?
 
Right now, the OAuth world is sadly full and ambiguity. This is largely because of implementation work on the OAuth 2.0 spec, which offers the promise of an easy-to-use OAuth specification, but at the same time keeps changing. We fully expect the rate of change to slow or even stop in the next few months, but no one knows for sure, especially since it is an IETF process, which takes time.
 
However, it takes time to do a job right, and security standards that impact the whole Internet certainly deserve to be done right.
 
Unfortunately, that puts today's API implementors between a rock and a hard place. Until OAuth 2.0 is a little more stable, you have the choice of implementing the latest draft and being prepared for changes, and implementing OAuth 1.0a, which makes life more difficult for customers.
 
What should you do?
 
Be patient. It's possible to build an effective API right now using either a current draft of OAuth 2.0 (see Facebook) or OAuth 1.0a (see Twitter).
 
However, things are going to change a bit for the near future. Some would take this as a reason to give up, "sunset" OAuth, or declare it "dead" way before its time. I think that building a new security technology from the ground up that has all the advantages I enumerated at the top of this article would be a big mistake.  We have two pretty-good options today in OAuth 1.0a the latest drafts of 2.0 -- stick with them.

Best Practices for OAuth 2.0 vs. OAuth 1.0 - One year later »

Since we first wrote about OAuth 1.0 vs. 2.0 a while back, lots has been happening and lots has changed. The OAuth community has made progress and made changes, and an increasing number of API providers have deployed APIs that use OAuth 2.0. (Similarly, the number of new OAuth 1.0-enabled APIs doesn't seem to be growing.)

We've been busy helping our customers implement OAuth-based APIs, and we've also been watching the process develop. Here are some things that we've learned: 

OAuth is a solution -- not technology.

In order for an API provider to support OAuth, a number of things have to happen. There must be a place where the OAuth protocol is actually implemented -- all of the different URLs for requesting tokens, issuing tokens, redirecting browsers, and so on. This part is a matter of implementing the standard and doesn't change from implementation to implementation (so, it can be delegated to a third-party component like Apigee Enterprise).

In addition, there has to be somewhere to store OAuth tokens. There is no standard for this -- some use an RDBMS, some use NoSQL, and others use a cloud-hosted service.

Finally, the OAuth solution has to integrate with a provider's "login page," so that when a user wants to get an OAuth token, they are redirected to a legitimate "login" page with a proper SSL certificate,  look and feel, and so on. All of these kinds of things can be different from one provider to another.

At Apigee, we have been working with many companies to deploy APIs built using OAuth. What we've found is that OAuth is not just a technology that you buy or download -- it's a bit of technology and a bit of integration, and to implement it at all you need a platform that can flexibly integrate with what you already have, while adding what you might be missing. 

Be careful.

Since we last blogged about it, OAuth 2.0 has been improved a bit and has changed some basic things -- for instance the name on the HTTP "Authorization" header has changed from "OAuth" to "Bearer." This is a big change, and obviously if an API provider just switches from one value to another without warning, all the clients will break.

OAuth 2.0 is up to draft 13, and with every draft it gets closer to being "done" -- but it's not done yet and I haven't seen anyone say when it will be. That means that if you implement, say, draft 13, you will likely have to change, and find a way to maintain backward compatibility for existing clients.

If that's too much, then implement OAuth 1.0a -- it's as "done" as it's going to get at this point! 

Stick with "bearer tokens" for now in conjunction with SSL.

The latest draft includes support for various types of security tokens,  and a mechanism to plug in new types. The simplest are "bearer tokens." A bearer token is just a big, random string that a client must present on every API call. Bearer tokens are simple because there's no special signature or validation code required on either end. The client is responsible for storing the token in a safe place and sending it with every request. The server is responsible for looking up the token in a database and making sure it's a valid one -- that's it.

However, bearer tokens provide no security unless used in conjunction with SSL -- otherwise, any eavesdropper in a coffee shop can snatch OAuth tokens from  the air. (Fortunately, OAuth tokens can be individually revoked without requiring a password reset.) 

Get ready to change -- or stick with 1.0a.

We said this before but it bears repeating -- if you implement OAuth 2.0, it is likely that you will have to change your servers to support newer versions, and also maintain backward compatibility.

One way to ease the pain is to put version numbers in the URLs that you hand out for the various steps in the OAuth flow. So, instead of using:

http://api.foo.com/oauth/authorize

use:

http://api.foo.com/oauth/13/authorize

because you know that there will be future versions and that they will be different...

Get ready for more options.

 Part of the OAuth 2.0 work defines some new security token types other than "bearer." Right now only "bearer" seems to be sufficiently mature from a spec standard, but this will change over the next few months. Here are some of the options:

1.       Bearer -- as we mentioned before, this is just a long, random string. The advantage is that it's extremely easy to implement on both client and server. The disadvantage is that it requires SSL for all API calls. Also, if API traffic passes through multiple endpoints such as proxies, then the proxies will be able to see the token as it passes by.

2.       Mac -- this is equivalent to the token scheme in OAuth 1.0, in that it requires both a key and secret, and uses an "Hmac" algorithm to encrypt some data on each request. The result is that the request is only valid if both client and server have the same keys, and there is no way for an intermediary to re-create the original request without a password. In other words, it is secure even if SSL is not used or even if the request is intercepted by a proxy.

3.       SAML -- this uses SAML assertions on each request as a way to establish the client's identity. This is helpful if you already have SAML infrastructure for either the client or server in your environment.

A Short History of API Authentication (and where it’s going): from HTTP basic to OAuth 2.0 »

Part 1: The Web

In the beginning -- way back in the beginning -- the web was all about open access. Tim Berners-Lee and his colleagues focused on making information available, not on protecting it from unauthorized users.

But as time went on, and as Al Gore took the initiative in liberating the government-run Internet backbone for commercial use (really), the Web became about "e-commerce,"  and e-commerce required security. SSL matured to ensure that sensitive traffic was encrypted all the way from the client to the server and back, and various schemes emerged to allow user authentication.

The oldest and most common format for web authentication is HTTP Basic authentication. This is what you get when you visit a web site and a little browser window pops up requesting a username and password. Every web browser and every major web server supports some form of this.
 
From a web design perspective, HTTP Basic has a big disadvantage in that it's implemented entirely by the browser, and can't be customized for each site. As the quality of graphic design improved on the web, designers soon realized that they wanted not a generic little grey box on the screen, but a carefully-designed login page, with logos, disclaimers, and the like, or a discreet login button on the corner of a web page. The combination of HTTP Basic and HTML just didn't allow this.

The result was the rise of form-based authentication. This is what the vast majority of secure web sites use today. As a user, you visit a web page that prompts you for a username and password. If authentication is successful, then under the covers you are granted a unique cookie, which your web browser sends with each subsequent request. As the user you never see this -- it just looks like you logged in and now the site works -- but under the covers it is quite different from Basic.

Both Basic and form-based authentication rely on SSL to create an encrypted tunnel between the client and server. Without that encrypted tunnel, anyone snooping the Internet or the open Wifi at Starbucks can see the passwords go by in the clear. Fortunately, SSL protects against this very well, but sometimes developers neglect to use it, users neglect to ask for it (as most of us do with Facebook), and sometimes the traffic travels over unencrypted links behind the firewall of a large network.
 
The web community attempted to counter this using HTTP Digest authentication, which encrypts the password using a one-way hash so it's impossible to reverse-engineer even if SSL is not used -- but it still must be implemented by the browser and can't be designed in to a nice UI. It never took off.
 
For a higher level of security, SSL has long supported two-way authentication, which requires that individual end users request digital certificates for each site they plan to visit and install those signatures on their browsers. The overhead of issuing PKI credentials to end users was enormous and this never took off either.
 
Part 2: Early APIs
 
Some early APIs were built right on top of existing web sites built using form-based authentication. The easiest way to implement them was to use the same authentication mechanism, so API developers would create a method called "login" that returns a security token, another method called "logout," and require the security token on every API call.
 
This approach makes the API easy to tack on top of an existing web app, but it is more work on the client side and hurts API adoption. An administrator can't as easily drive the API from the script without logging in, extracting the security token from the response, making the call, and then logging out.
 
Other early APIs just use HTTP Basic authentication. It's simple, works with every client (and with every shell script based on "cURL"), but it requires SSL to be used, often leaving it up to the client to "remember" to use it. Still, it's effective as long as the user has the password for the API handy.
 
Yet others, especially Amazon, decided they wanted to avoid using SSL for performance reasons, but they also wanted to avoid using the uncommon HTTP Digest authentication. (Amazon's S3 is used to store multi-gigabyte files and SSL does make a difference there.) They chose to create their own access control mechanisms based on secret keys and in some cases, digital signatures. The result was a bit of programming for each developer starting out with AWS, but Amazon's services were so useful and cheap that it didn't matter. By now there are numerous libraries to make this process easier.
 
Part 3: APIs get formal
 
The first real access control mechanism aimed at the needs of APIs and API developers is OAuth. The idea came from a popular API (Twitter) and a defunct web site (Ma.gnolia). The goal was to make it possible for a Ma.gnolia user to access Twitter without requiring that each user give Ma.gnolia their Twitter password.
 
The result, OAuth 1.0, is like a "valet key" for an API -- it is a token that gives a single client or web site access to a particular API on behalf of one user. The client or web site can get an OAuth "access token" without ever seeing the user's password because the two web sites do a sort of "credential dance" to exchange the secret token. Once the token is issued, the user can see it or revoke it, thus taking away access from the client or site without requiring a password change.
 
Since then, OAuth 1.0 has seen a patch (it became 1.0a to fix a security flaw), and an IETF committee is working on OAuth 2.0. This new version includes some important simplifications and a wider range of use cases. OAuth has become the de facto standard for API authentication. (for more see my earlier post on OAuth 1.0 vs. OAuth 2.0)
 
The Future
 
OAuth is likely to dominate the world of APIs for a long time. With OAuth 2.0, soon API providers and users will have the option of using a simple "bearer token" in conjunction with SSL, or a signed token that can remain secure without further encryption. OAuth 2.0 is flexible enough to be used in the original "web site to API" use case that OAuth 1.0 was designed to handle, but it can also be used for access by mobile devices, where it can be important to be able to remotely revoke the access tokens that might be stored on your phone without going around and changing innumerable passwords.
 
Still, there remain problems to be solved. What about access for simple management APIs, for instance? Even OAuth 2.0 is cumbersome and when SSL is always in use sometimes a plain old password is sufficient. API providers can and should consider supporting good old Basic authentication alongside OAuth 2.0 if for no other reason than convenience -- as long as SSL and a strong password are required.
 
What about mobile apps? Is there a way for the server hosting an API for, say, a large bank to ensure that the request is coming from an official application, or from a rogue app that is attempting to "phish" passwords by pretending to be the real thing? Can we do something by combining signed applications with server-side validation, or is a secure app store the only way to protect against mobile malware?
 
The world of APIs is evolving and there's no doubt that security technology will continue to evolve along with it.

(For more on API security issues see our other entries on OAuth and API authorization, identity, and security - or get it all with our "Is your API Naked" whitepaper)

Do you need an SDK for your API? »

In our last couple API strategy workshops, we've had good discussions on a common question for API providers - do you need to offer an SDK (software dev kit) for your API?

In this context we defined an SDK as going beyond offering an API to include platform-specific code that developers use in their apps to invoke API operations, also including the source code, and documentation that developers might need. Here's what we came up with:

Reasons to consider an SDK:

Speed adoption on a specific platform - for example Objective C SDK for iPhone.  Lots of experienced developers are just starting off with objective C+ so an SDK might be helpful.

Simplify integration effort required to work with your API - If key use cases are complex or need to be complemented by standard on-client processing. 

An SDK can help reduce bad or inefficent code that might slow down service for everyone.

As a developer resource - Good SDKs are a forcing function to create good source code examples and documentation.  (See some good examples here and here. )

To market your API to a specific community - you upload the SDK to a samples or plug-in page on the platform's exisitng developer community.

Reasons you might not need an SDK:

If your API is designed for adoption -  standards-based, well documented, etc.. developers may be able to get rolling without a client SDK.

Resources - it's tough to do a SDK for each platform you target, you'll have to pick carefully. Also, it can be time consuming and expensive to create SDKs (tip: maybe packaging up internal samples or test cycles can kickstart things)

Maintenance and versioning - convincing clients to upgrade to the newest version can be rough. Also, capturing the "local flavor" of each language you support can be a challenge.

Complexity -On each platform there might be use cases you don't expect, like keeping application-level secrets off clients, debugging, etc.

More is not necessarily better - fewer, more well-documented, code-level samples can often be the best resource. Facebook provides Android, C#, iPhone, JavaScript, PHP, and Python libraries (all documented differently), but Twitter supplies none. Facebook has more resources but still struggles a bit at times.

Our view: whenever possible, forget the SDK, and spend the time making your API better and more useable, and give good API resources such as example code.  

Agree or disagree? 

Why basic auth and social security numbers both suck (and how OAuth helps for APIs) »

Eggs…

Around the time of the industrial revolution, there was a problem—too many ships were sinking. So in 1891, the Bulkhead Committee proposed a "Grade I subdivision" that, among other things, required passenger liners longer than 425 feet to be able to float even if two compartments were compromised. Accidents will always happen, but the committee wanted to minimize any loss of life when they did.

It was a simple idea: that a single hull breach should not sink a vessel.

Jump ahead 118 years to the month in which my son started kindergarten. It's going pretty well for him. But yesterday, a notice came home asking us to supply his social security number to identify him in a longitudinal data system in order to comply with federal funding requirements. While I appreciate the future value such data will have, I am not at all comfortable giving away that secret number.

If you are anything like me, you think twice before giving out your social security number. I've had mine compromised before (I still have the Secret Service letter describing how it was discovered by customs in a briefcase along with other people who had worked for the same employer, but that's another story). This totally sucked, and I now hesitate even more than I did before when presented with a rental application / insurance form / school form / etc.

• Every time I give out my social security number, I increase the chances of it being compromised.
• Every time I build ships without compartmentalization, I compromise the safety of its passengers.
• Every time I give out my username and password for a web service, I increase the chances of it being compromised.

… And Baskets

By now you can probably see where I'm going… because at Apigee we're obsessed with APIs (think Glenn Close in Fatal Attraction but without all the psychosis). Smart people are going to going to build web services. To grow those services, they're going to build APIs. And those APIs need to have some way to authenticate users.

Since the username/password is still the primary way in which people identify themselves to a web service, you might think that securing your API with usernames/passwords makes a lot of sense. But you are only as strong as your weakest link… and once that password is compromised, the whole ship sinks.

There are perfectly good ways to deal with this, of course, API tokens for example, usually a custom token passed as a query or header parameter. These make most sense when sent over SSL to keep them safe. Some providers, like 37signals's API for Highrise, expect them to be using HTTP basic authentication in lieu of the username or password.

The OAuthpocalypse & The Unsinkable

Of course there are other ways. OAuth 1, popularized by Twitter as much as anyone, is a great example. At it's best, it works like valet keys that users can give to applications to access their accounts on their behalf while retaining the option to revoke those permissions app by app. An additional benefit is that API analytics can be organized by application.

But OAuth has it's own problems. It's harder to understand conceptually. It's harder to implement. Perhaps worst of all, it can disenfranchise users who must use proxies to get around the censorship walls put in place by governments like China and Iran.

OAuth 2 is different in some important ways. Facebook is already using a version of it with their Graph API, despite the spec still being in draft. Twitter appears to be using OAuth 2 draft for @Anywhere. The best benefit is that it's simpler and easier for developers to start using, since, among other things, it does away with the signing of base strings.

So OAuth does away with the password anti-pattern. But transitioning away from an authentication system like basic authentication to one like OAuth isn't ever easy. Change is hard—just look how much complaining follows every Facebook redesign. And things will break… and with the OAuthpocalypse, they did.

The Titanic was famously built to exceed the Bulkhead Committee's Grade I requirements. But that alone wasn't enough to save it. The hubris of excessive speed for the conditions meant the ship couldn't turn in time, and following the subsequent impact, five compartments flooded. It was an unprecedented disaster.

Twitter, on the other hand, knew up front that things were going to break. And when it became obvious that the deadline was coming too fast, they slowed down, delaying the event for two months. It was certainly painful for developers—though we still don't know the full extent to which this broke applications (though apparently not their own, thanks to a basic auth back door).

Overall, Twitter did an excellent job of handling the migration. They gave developers lots of time, and then they gave them more. As the date approached, they shut off basic auth for short periods—an iceberg ahead alarm bell, if you will. They slowly lowered the rate limits over two weeks. And most importantly, they communicated and coordinated well—Developer Advocates Taylor Singletary and Matt Harris, and other members of the Twitter API team worked tirelessly to provide tools and support devs as they struggled with the migration.

So far, things are looking pretty good for their great experiment. But only history will tell the full extent of the damage.

OAuthpocalypse? Have No Fear, ApiGeeks! »

The Twitter API is one of the most popular APIs on our platform and with Twitter shutting off basic auth and moving to OAuth tomorrow, we wanted to reassure you that Apigee is ready for the OAuthpocalypse with full support for APIs which use OAuth.

Here's how to do it, using the Twitter API as an example:

1. Do the OAuth dance with the canonical API endpoint (e.g. api.twitter.com).

2. Once you have a user's token, use it to sign requests' base strings, which you would also build using the canonical endpoint from step 1.

3. Then send those signed requests to your Apigee URL, such as twitter.myusername.apigee.com.

In this way traffic can flow to and from Twitter via your Apigee API using OAuth.

Let us know if you run into any questions or issues- visit our support site or ping us on Twitter at www.twitter.com/apigee. Good luck with the big switch!

Top Differences between OAuth 1.0 and OAuth 2.0 for API Calls »

OAuth 1.0 (the current spec version is 1.0a, which fixes a security problem with 1.0) solves an important problem in the world of APIs -- how one web application can give another application API access without requiring that the user give out their password. OAuth 1.0 solves this problem in a clever way through a secure handshake, via API calls, between the two applications. This has allowed APIs to go in places where they could never go before.

OAuth 1.0 works by ensuring that the API client and server share a token, which is like a username, and a token secret, which is like a password. The client must generate a "signature" on every API call by encrypting a bunch of unique information using the token secret. The server must generate the same signature, and only grant access if both signatures match.

The advantage of this approach is that there is no way to find out the token secret, because it is always encrypted when it's sent over the network, and only the client and server have the keys. It doesn't matter if the data is eavesdropped on a WiFi network in Starbucks or captured by a proxy like Apigee -- there's no way to see the secret, so there's no way to impersonate the client based on what's sent over the network. All of this is done without requiring SSL, since SSL can slow down the client and server alike, and make deployment of the API server more complex.

However, both client and server developers found the complexity of generating and validating signatures to be too much. There are many tricky things that a developer must get right, down to exactly what type of "URL Encoding" is used (it's not exactly the same as it's used in other places). If the client or server makes a single tiny mistake in the signature, it's invalid and it's hard to figure out what went wrong.

OAuth 2.0 promises to simplify this stuff in a number of ways:

1. SSL is required for all the communications required to generate the token. This is a huge decrease in complexity because those complex signatures are no longer required.
2. Signatures are not required for the actual API calls once the token has been generated -- SSL is also strongly recommended here.
3. Once the token was generated, OAuth 1.0 required that the client send two security tokens on every API call, and use both to generate the signature. OAuth 2.0 has only one security token, and no signature is required.
4. It is clearly specified which parts of the protocol are implemented by the "resource owner," which is the actual server that implements the API, and which parts may be implemented by a separate "authorization server." That will make it easier for products like Apigee to offer OAuth 2.0 support to existing APIs.

For these reasons, OAuth 2.0 has already been adopted by companies like Facebook, which uses the draft spec in its Graph API. Of course, it's a new spec, which means there are new requirements and use cases that make it more complex. For instance, OAuth 2.0 also clearly lays out how to use OAuth entirely inside a browser using JavaScript that has no way to securely store a token, and it explains at a high level how to use it on a mobile phone or even on a device that has no web browser at all.

Finally, although the developers of the world will not miss generating OAuth 1.0 signatures, they served a purpose, because they allowed a client to send its token and secret securely to a server without requiring SSL. For APIs and devices that do not want to support SSL for performance or complexity reasons, signatures are still a good choice. Right now, signatures have been removed from the OAuth 2.0 spec, but they'll be added to a separate extension spec at some point.

So should you use OAuth 2.0 today? Here are a few things to consider: 

• Spec changes. OAuth 2.0 has not reached a stable IETF draft yet. If you implement it today, are you prepared to change your implementation every few weeks until the committee has agreed on a stable version? OAuth 1.0a, on the other hand, is already a well-defined standard that's not going to change any time soon.  
• Implementations. There are a number of code libraries for both client and server that support 1.0a today. There aren't as many for 2.0, so you're going to have to build more stuff on your own.
• Complexity. There's no doubt that 2.0 is easier to implement both on the client and server side.
• Performance. If you are unwilling or unable to use SSL for all of your API traffic, then OAuth 2.0 is not a good choice until some sort of signature extension is added to the spec. OAuth 1.0a already supports signatures, which are complex but allow you to securely exchange tokens without requiring the use of SSL.

You can also check out my blog on "When to Use OAuth" for more, and we'll continue to explore this issue as it evolves.  

When to Use OAuth »

In our daily talks with customers about their APIs, OAuth has become a major consideration. It's quickly replacing the various proprietary "authentication token" schemes used by early APIs (such as the Amazon Web Services APIs) with a flexible standard; and it solves for new use cases emerging from mashups and Web 2.0. Still, there is a lot of confusion about OAuth out there.

Based on the current state of the technology, we know that there are different API use cases. Depending on your security needs, sometimes OAuth is the only security technology an API can realistically support. In others cases, it may be overkill, or something to add alongside another scheme.

OAuth Overkill?        

OAuth is overkill for an API that doesn't require strong authentication. Many "catalog" and "search" APIs that access public data fall into this category. The goal of an API like this is to gain adoption. There's no reason for such an API to require individual users to authenticate to retrieve information from a product catalog, or search public data. (Unless, of course, your API's business model revolves around data that is so valuable that you can get away with charging for every data access. In today's world, that's not common.) A simple API key, however, is a great idea for these kinds of APIs, integrated into your API analytics so you and your developers can see what's going on.

The Case for OAuth

OAuth is the only realistic choice for a web application that itself uses another web application's API on behalf of the user. For instance, consider a web application that integrates with Twitter. (Perhaps it's a geolocation app like Foursquare that offers the ability to tweet where you are and what you're doing.) Today, it is unacceptable for such a web application to store its users' Twitter passwords. OAuth was designed precisely for this use case -- it gives the web app a secure way to get an access token for Twitter, which the user can revoke at any time, without ever revealing that Twitter password to the web app.

Basic Auth- Still Important

In most cases OAuth should be one of two or three security choices for most APIs. Again, consider Twitter. It makes perfect sense for OAuth to be used by other web apps as a way to access Twitter. But Twitter has long supported "Basic" HTTP authentication as well, using a username and a password. While this is bad news for a web application client, for a mobile or a desktop client that is used by an individual user, it's just fine as long as the client takes some care with the password.

And importantly, Basic authentication is easy. If your API requires secure authentication, and you want it to be easy to integrate and test, then offering Basic authentication means that the barrier to entry is low.

Now, once the decision to use OAuth has been made, the question is whether to use the current version 1.0a, or the still-under-development OAuth 2.0. We'll talk about that next.

When to Use OAuth »

In our daily talks with customers about their APIs, OAuth has become a major consideration. It's quickly replacing the various proprietary "authentication token" schemes used by early APIs (such as the Amazon Web Services APIs) with a flexible standard; and it solves for new use cases emerging from mashups and Web 2.0. Still, there is a lot of confusion about OAuth out there.

Based on the current state of the technology, we know that there are different API use cases. Depending on your security needs, sometimes OAuth is the only security technology an API can realistically support. In others cases, it may be overkill, or something to add alongside another scheme.

OAuth Overkill?        

OAuth is overkill for an API that doesn't require strong authentication. Many "catalog" and "search" APIs that access public data fall into this category. The goal of an API like this is to gain adoption. There's no reason for such an API to require individual users to authenticate to retrieve information from a product catalog, or search public data. (Unless, of course, your API's business model revolves around data that is so valuable that you can get away with charging for every data access. In today's world, that's not common.) A simple API key, however, is a great idea for these kinds of APIs, integrated into your API analytics so you and your developers can see what's going on.

The Case for OAuth

OAuth is the only realistic choice for a web application that itself uses another web application's API on behalf of the user. For instance, consider a web application that integrates with Twitter. (Perhaps it's a geolocation app like Foursquare that offers the ability to tweet where you are and what you're doing.) Today, it is unacceptable for such a web application to store its users' Twitter passwords. OAuth was designed precisely for this use case -- it gives the web app a secure way to get an access token for Twitter, which the user can revoke at any time, without ever revealing that Twitter password to the web app.

Basic Auth- Still Important

In most cases OAuth should be one of two or three security choices for most APIs. Again, consider Twitter. It makes perfect sense for OAuth to be used by other web apps as a way to access Twitter. But Twitter has long supported "Basic" HTTP authentication as well, using a username and a password. While this is bad news for a web application client, for a mobile or a desktop client that is used by an individual user, it's just fine as long as the client takes some care with the password.

And importantly, Basic authentication is easy. If your API requires secure authentication, and you want it to be easy to integrate and test, then offering Basic authentication means that the barrier to entry is low.

Now, once the decision to use OAuth has been made, the question is whether to use the current version 1.0a, or the still-under-development OAuth 2.0. We'll talk about that next.

Explore the Facebook API and Simplify OAuth 2.0 »

We launched a new tool at Apigee-  the API lifecycle management platform we offer free to the community. We've created a Facebook Graph API Console- a whole new way to interact with, learn and debug the Graph API that lets you easily view requests/responses to the API, share what you are seeing and dig into errors.

The console supports OAuth 2.0 so you can log in using your Facebook credentials- check out the video: 

InsideFacebook has some great coverage by Josh Constine out today on the console and how it works; you can also hop over to the Apigee blog for more details. 

This week in APIs- June 14-18 »

Here's the best of what happened this week in APIs!

Opera 10.6 Beta- The Opera 10.6 beta was announced- the new version implements W3C's Geolocation API. Is this a sign that the browser is moving away from plug-ins and into APIs for more functionality? Either way, we expect a lot more browser innovation with APIs.

Retail APIs- Over at InfoWorld, Neil McAllister wrote up a big new opportunity for retail: open APIs. Neil illustrates how companies like Amazon, Zappos, Tesco and Sears are opening up their APIs to reach customers in the new multi-device, multi-channel world. Big theme- the new emerging role of developers who can build APIs right. 

Twitter OAuth Switchover- The Twitter API team announced that they are moving back the date of the "OAuth switchover"- when they will no longer support basic auth. On August 16th Twitter will be ramping down rate limits on basic auth by about 10 calls/hour/day, ending on August 31 when basic auth won't be accepted.

New APIs- Every week there are new APIs! Here's what we're excited about right now:

• Plancast- a service for sharing future plans with your social network- announced an open read/write API. The world of social planning is heating up- and APIs are a critical part of the movement. 
• The State of California introduced 10 new APIs on their http://data.ca.gov/ site; including ones to get hospital locations, draw household and county population data, and find EDD office locations. Great signs that California is not only getting Gov 2.0, but also geolocation. We also like the clean way they lay out data sets- best practice for government agencies. 
• Infochimps- a data marketplace- just opened up an API which offers access to its Twitter and U.S. Census datasets. Lots of great information to build apps with and check out their business model- pricing to the API is tiered based on number of API calls per month- what do you think? 

Any major API news we missed? Leave a comment or tweet at us- http://www.twitter.com/apigee 

OAuth is improving, but still moving »

We've been following the fast-moving debate in the IETF regarding OAuth 2.0.  OAuth, for those of you who have not encountered it already, is a set of authentication technologies for the Internet designed around the concept of an access token.

Access tokens, in the words of Eran Hammer-Lahav, are like valet keys -- they give the holder access to a specific function, for a specific amount of time. For instance, you might use OAuth to give another web site the ability to read photos from your Flickr profile, but not to modify them. OAuth lets you do this, it lets you go back to Flickr and revoke the web site's permissions at any time, and it does it without requiring that you give the site your Flickr password.
 
The current spec, OAuth 1.0a, is implemented in lots of places, and it solves a lot of problems. However, implementing it is no picnic for either the API provider (the server) or for the developer who builds the client. (There are libraries, of course, not to mention products such as our own that simplify this process.)
 
OAuth 2.0 introduces many changes. The most important is that a client may now use a "bearer token." That's a fancy IETF way of saying that an access token can just be a string that the server gives you. On every request, the client passes that token back to the server, the server checks to see if the token is valid, and you're done. This is much simpler to implement than OAuth 1.0a, but it is only secure if you use SSL for every request. Applications that won't or can't use SSL may still use the old way of transmitting each token, which encrypts the token so that it is safe even if SSL is not used or even if it is intercepted by a proxy like Apigee.
 
However, OAuth 2.0 is far from complete. It is currently undergoing lots of discussion on the IETF mailing list, and the spec draft changes daily.
 
That's why I was surprised to read today that Facebook is using OAuth 2.0 to authenticate its own API. Now, some of the key players in OAuth work at  Facebook, and they have chosen to use only a part of the spec, and the part that's arguably the least complicated. I'm sure that they feel that taking this calculated risk now is in the best interest of Facebook and its developer community, but the possibility remains that the spec will change and Facebook will have to change its implementation to match.

(In fact, at the moment I write this, they do not -- the name of the query parameter that holds the token is "access_token" in the Facebook documentation and "oauth_token" in the latest version of the spec repository.)
 
In the meantime, developers building on top of these APIs may have to contend with OAuth 1.0a (the current spec), OAuth 1.0 (an older version that some sites may still use), the draft form of OAuth 2.0 as implemented by Facebook, and even "WRAP," which introduced some of the concepts used in OAuth 2.0.
 
So the good news is that there are lot of good standards being written that can make it easier to produce and consume powerful and secure APIs. The bad news is that those standards are still changing. So stay tuned, and be careful!

Tech Talk: API Security and Threat Protection »

Greg recorded a few whiteboard talks last month - this one is a good summary of recent posts on API Keys, API security recommendations, and OAuth best practices. 

(And here are some of our high-level API security features if you are looking into this.)

OAuth — Take care with those keys! »

A lot has been happening with OAuth recently. Earlier this year a security hole was discovered in the protocol which exposed it to a potential “social engineering” attacks.  However, the OAuth community is working on a revision to the spec that will eliminate this particular hole.

Last week we wrote a bit on OAuth as an option for API security.  But today I wanted to bring up a related OAuth issue - how do you securely manage all those keys?

With traditional username / password authentication, good security practices require you don't just have a big database on the back end with a list of unencrypted passwords. Instead, a hash of the password is stored, preferably using a salt. So someone who can read the password file can verify they have the right password, but cannot see the actual password.

It is still critical to protect access to these encrypted passwords. Otherwise, an attacker can mount a dictionary attack to try and crack them. However, even if someone gains access to your entire database of encrypted passwords, they can still only easily gain access to lousy passwords. At least users who choose secure passwords are relatively safe. (It is also critical to protect access to the cleartext password, but at least this mechanism doesn’t require that it be stored in a database for all to see.)

As networking and middleware people, we spend a lot of time thinking about the security of our network protocols, and especially ensuring that someone eavesdropping on a network cannot grab our passwords and other sensitive data as they fly by. But how many times have we heard of a security breach caused by a stolen laptop? I would argue that protecting so-called “data at rest” is just as important, or maybe even more important, as protecting the data flying around your laptop.

Now, back to OAuth. Each “user” in OAuth holds something called an “access token,” which is like a username, and a “token secret,” which is like a password. When a request is sent over the network containing an OAuth authentication token, a bunch of data in the token is encrypted using the token secret, but the secret itself is never sent over the network. That way, regardless of whether SSL is in use, there is no way to gain access to the token secret by sniffing the network.

However, on the server side, in order to validate the OAuth token, the server must make the same calculation that the client made when it encrypted the data to put in the token. That means that both the client side and the server side in OAuth must be able to read the unencrypted token secret from some sort of database. Without it, OAuth doesn’t work. There’s no set of standard ways for storing those keys like there are for passwords, so presumably different implementations are storing them in different ways.

As a result, any client and any server that uses OAuth has to take extra-special care with all those token secrets. Otherwise, anyone who gets access to the database of tokens and secrets used by the back end servers immediately has access to all the OAuth-enabled accounts.

I am not suggesting a change to the OAuth protocol here — it solves an important problem. However, I am suggesting that anyone who implements either the “service provider” or “consumer” side of OAuth take very special care of those tokens!

For instance:

•     If they’re on a regular disk file, protect them using filesystem permissions, make sure that they’re encrypted, and hide the password well.
•     If they’re in a database, encrypt the fields, store the key well, and protect access to the database itself carefully.
•     If they’re in LDAP, do the same.

Come to think of it, perhaps the world needs a standard LDAP schema for storing OAuth secrets in a secure way. Anyone care to make a proposal?

Don’t roll your own: API Security Recommendations »

Let's boil up the examples and common pitfalls from our last two entries on API Identity and Authorization and more API security choices.   

Use API Keys for non-sensitive data (only): 

If you have an “open” API - one that exposes data you’d make public on the Internet anyway -  consider issuing non-sensitive API keys. These are easy to manipulate and still give you a way to identify users. Armed with an API key, you have the option of establishing a quota for your API, or at least monitoring usage by user. Without one, all you have to go on is an IP address, and those are a lot less reliable than you might think. (Why don’t web sites issue “web site keys?” Because they have cookies.)

For example, the Yahoo Maps Geocoding API issues API keys so it can track its users and establish a quota, but the data that it returns is not sensitive so it’s not critical to keep the key secret.

Use username / password authentication for APIs based on web sites: 

If your API is based on a web site, support username/password authentication. That way, users can access your site using the API the same way that they access it using their web browser. For example, the Twitter API supports username/password authentication, so when you access it using a Twitter API client like Spaz or TweetDeck you simply enter the same password you use when you use the twitter.com web site.

However, if you do this, you may want to avoid session-based authentication, especially if you want people to be able to write API clients in lots of environments. It is very simple to use “curl,” for instance, to grab some data from the Twitter API because it uses HTTP Basic authentication. It is a lot more complex if I instead have to call “login,” and extract a session ID from some cookie or header, and then pass that to the real API call I want to make...

OAuth for server-to-server APIs that are based on web sites:

If your API is based on a web site (so you already have a username / password for each account) and the API will also be used by other sites, then support OAuth in addition to username / password authentication. This gives users a lot of peace of mind, because they can authorize another site to access their sensitive data without giving that site their password permanently.

Use SSL for everything sensitive:

Unless your API has only open and non-sensitive data, support SSL, and consider even enforcing it (that is, redirect any API traffic on the non-SSL port to the SSL port). It makes other authentication schemes more secure, and keeps your user’s private API data from prying eyes – and it’s not all that hard to do.

Don’t roll your own!

If the above suggestions still don’t apply to you, then keep looking – between OAuth, OpenID, SAML, HTTP authentication, and WS-Security, there are a lot of authentication schemes, and each has its pros and cons.

So wrapping up API security in our series on 10 API roadmap considerations. Here are some suggested questions you may want to ask when putting together your API security roadmap:

How valuable is the data exposed by your API?

• If it’s not that valuable, or if you plan to make it as open as possible, is an API key enough to uniquely identify users?
• If it is valuable, can you reuse username and password scheme for each user?
• Are you using SSL? Many authentication schemes are vulnerable without it.

What other schemes and websites will your API and users want to integrate with?

• If your API will be called programmatically by other APIs, or if your API is linked to another web site that requires authentication, have you considered OAuth?
• If you have username/password authentication, have you considered OpenID?
• Can you make authorization decisions in a central place?

What other expectations might your customers have?

• If your customers are enterprise customers, would they feel better about SAML or X.509 certificates?
• Can you change or support more than one your authentication approach for diverse enterprise customers?
• Do you have an existing internal security infrastructure that you need your API to interact with?

Up next:  API Data Protection and thanks to Torbin H. for the photo.

More API Security Choices - OAuth, SSL, SAML, and rolling your own »

(This entry continues our API Roadmap consideration series and is a continuation of last week's API Identity and Authorization.

Session based Authentication – cumbersome with RESTful APIs

Lots of APIs  support session-based authentication. In these schemes, the user first has to call a “login” method which takes the username and password as input and returns a unique session key. The user must include the session key in each request, and call “logout” when they are done. This way, authentication is kept to a single pair of API calls and everything else is based on the session.
 
This type of authentication maps very naturally to a web site, because users are used to “logging in” when they start working with a particular site and “logging out” when they are done.
 
However, session-based authentication is much more complex when associated with an API. It requires that the API client keep track of state, and depending on the type of client that can be anything from painful to impossible. Session-based authentication, among other things, makes your API less “RESTful” - an API client can’t just make one HTTP request, but now a minimum of three.
 
The Role of OAuth

Today, many APIs also support OAuth authentication. OAuth was designed to solve the application-to-application security problem. The idea of OAuth is that it gives the user of a service or web site a way to conditionally grant access to another application. OAuth makes it possible for a human user to individually grant other APIs or sites access to their account without sharing their actual password. It works by giving a “token” to each API or site that will access the account, which the user may revoke at any time they wish.

For instance, if web site FooBar.com wants access to the Twitter API on behalf of John Smith, then OAuth specifies the protocol that FooBar.com must go through to get an OAuth token for the Twitter API. Part of this process requires John Smith to log in to his Twitter account using his normal username and password (which, in the OAuth protocol, are never seen by FooBar.com) and authorize FooBar to access his account. The result is that FooBar.com will have an OAuth token that gives it access to John Smith’s account. If, later on, John Smith decides he no longer trusts FooBar.com, he has the option to revoke that OAuth token without affecting his regular password or any other accounts.

This process makes OAuth the ideal for communication from one application to another – for instance, allowing MySpace to post photos to your Twitter account without requiring that you enter your Twitter password every time you want to do it – but it can be used for any kind of API communications as well.
 
Two-Way SSL, X.509, SAML, WS-Security...

Once we leave the world of “plain HTTP” we encounter many other ways of authentication, from SAML, X.509 certificates and two-way SSL, which are based on secure distribution of public keys to individual clients, to the various WS-Security specifications, which build upon SOAP to provide... well, just about everything.

An API that will primarily be used by “enterprise” customers – that is, big IT organizations – might consider these other authentication mechanisms such as an. X.509 certificate or SAML for more assurance over authentication process than a simple username/password. Also, a large enterprise may have an easier time accessing an API written to the SOAP standard because those tools can import a WSDL file and generate an API client in a few minutes.

The idea is to know your audience. If your audience is a fan of SOAP and WSDL, then consider some of the more SOAP-y authentication mechanisms like SAML and the various WS-Security specifications. (And if you have no idea what I’m talking about, then your audience probably isn’t in that category!)

 
Rolling Your Own

In between OAuth, HTTP Basic, and the basic API key are many alternatives. It seems that there are as many other API authentication schemes as there are APIs. Amazon Web Services, Facebook, and some Google APIs, for instance, are some big APIs that combine an API key with both public and secret data, usually through some sort of encryption code, to generate a secure token for each request.

The issue – every new authentication scheme requires API clients to implement it. On the other hand, OAuth and HTTP Basic authentication are already supported by many tools. The big guys may be able to get away with defining their own authentication standards but it’s tough for every API to do things its own way.
 
SSL

Most authentication parameters are useless, or even dangerous, without SSL. For instance, in “HTTP Basic” authentication the API must be able to see the password the client used, so the password is encoded – but not encrypted – in a format called “base 64.” It’s easy turn this back into the real password. The antidote to this is to use SSL to encrypt everything sent between client and server. This is usually easy to do and does not add as much of a performance penalty as people often think. (With Apigee’s products, we see a 10-15 percent drop in throughput when we add SSL encryption.)

Some security schemes, such as OAuth, are designed to be resistant to eavesdropping. For instance, OAuth includes a “nonce” on each request, so that even if a client intercepts an OAuth token on one request, they cannot use it on the next. Still, there is likely other sensitive information in the request as well.

In general, if your API does not use SSL it also potentially exposes everything that your API does.

Next time, we'll make some recommendations among different options - or get our full paper here.