HUGE: Running an API at Scale »

Thanks to all who participated in last week's strategy webinar - HUGE: Running an API at Scale.

And thanks to our speakers @sramji, @brianpagano, and @edanuff.

Here are the slides and video. We'd love more of your thoughts, insights, or questions on the api-craft forum.  

API Trends: What to expect in 2012 »

Thanks to all who participated in last week's webinar:  "API Trends: What to expect in 2012" with Sam @sramji, Anant @jhingran, and Brian @brianpagno.

Here are the video and slides. Thanks for a great interactive session.

We'd love to hear more of your thoughts and questions; please join the conversation on the api-craft forum.

RESTful API Design: API Virtualization »

Last time, we looked at why you might consider complementing your API with an SDK or code libraries. In the series so far, we've covered a lot of tips and tricks for designing pragmatic RESTful APIs. 

You may be asking -  

How do I follow all these best practice guidelines and still maintain and iterate my APIs? 

What should I be thinking from an architectural perspective in terms of implementing these best practices?

Add an API virtualization layer 

I recommend you give yourself a buffer or virtual layer between the interface on top and the API implementation on the bottom.

The app that consumes the API is on top. The API virtualization layer isolates the application and the API. Make a clean design for the app on top and turn the problem into an integration problem.

 In other words, work at integrating your design on top of the key APIs through the virtualization layer.

Don't start in code, and try to build up from your business logic to a clean API interface.

RESTful API Design: complement with an SDK »

So far in the API design series, I've looked at best practices for designing pragmatic RESTful APIs.

This time, I'll talk about complementing APIs with code libraries and SDKs.

What to do when building a UI requires a lot of domain knowledge?

This image is an example of a twitter app displayed in a newsroom.

A developer created this cool visualization - a tweet referencing another twitter user.

The Twitter API is one of the simplest app domains imaginable: the primary object is 140 characters of text. Yet, the developer who created this needed quite a bit of domain knowledge - they needed to parse the tweet, color code it, and display it in a certain way.

The API team shouldn't try to solve presentation problems with the API. That's what code libraries do. Instead

Complement your API with code libraries and a software development kit (SDK)

That is, don't overburden your API design. A lot of what's needed in scenarios like this is on the client side and you can push that burden to an SDK.

The SDK can provide the platform-specific code that developers use in their apps to invoke API operations - meaning you keep your API clean.

Other reasons you might consider complementing your API with an SDK

• Speed adoption on a specific platform - for example Objective C SDK for iPhone. Lots of experienced developers are just starting off with objective C+ so an SDK might be helpful.
• Simplify integration effort required to work with your API - If key use cases are complex or need to be complemented by standard on-client processing.
• An SDK can help reduce bad or inefficient code that might slow down service for everyone.
• As a developer resource - Good SDKs are a forcing function to create good source code examples and documentation. Both Yahoo! and Paypal have good examples.
• To market your API to a specific community - you upload the SDK to a samples or plug-in page on the platform's existing developer community.

Next time: API Virtualization

RESTful API Design: chatty APIs »

In this post in the series, let's think about how app developers use that API you're designing and dealing with chatty APIs.

Imagine how developers will use your API

When designing your API and resources, try to imagine how  developers will use it to say construct a user interface, an iPhone app, or many other apps.

Some API designs become very chatty - meaning just to build a simple UI or app, you have dozens or hundreds of API calls back to the server.

The API team can sometimes decide not to deal with creating a nice, resource-oriented RESTful APIs, and just fall back to a mode where they create the 3 or 4 Java-style getter and setter methods they know they need to power a particular user interface.

I don't recommend this. You can design a RESTful API and still mitigate the chattiness.

Be complete and RESTful and provide shortcuts

First design your API and its resources according to pragmatic RESTful design principles (link the series) and then provide shortcuts.

What kind of shortcut?

Say you know that 80% of all your apps are going to need some sort of composite response, then build the kind of request that gives them what they need.

Just don't do the latter instead of the former. First design using good pragmatic RESTful principles!

Take advantage of the partial response syntax

The partial response syntax I discussed in a previous post (link) can help.

To avoid creating one-off base URLs, you can use the partial response syntax to drill down to dependent and associated resources.

In the case of our dogs API, the dogs have association with owners, who in turn have associations with veterinarians, and so on. Keep nesting the partial response syntax to get back just the information you need.

    /owners/5678?fields=name, dogs(name)

Next, SDKs and code libraries.

RESTful API Design: making requests »

We've covered singular vs. plural nouns to label your resources, tips for search, handling errors, and more.

Now lets take a look at what some API requests and responses look like for our dogs API.

Create a brown dog named Al

POST /dogs
name=Al&furColor=brown
Response
200 OK

{
"dog":{
"id:"1234",
"name": "Al",
"furColor": "brown"
}
} 

Rename Al to Rover - Update

PUT /dogs/1234
name=Rover
Response
200 OK

{
"dog":{
"id:"1234",
"name": "Rover",
"furColor": "brown"

}
} 

Tell me about a particular dog

GET /dogs/1234

Response
200 OK

{
"dog":{
"id:"1234",
"name": "Rover",
"furColor": "brown"
}
}


Tell me about all the dogs

GET /dogs
Response
200 OK

{
"dogs":
[{"dog:{
"id:"1233",
"name": "Fido",
"furColor": "white"}},
{"dog:{
"id:"1234",
"name": "Rover",
"furColor": "brown"}}]
}

 Delete Rover :-(

DELETE /dogs/1234
Response
200 OK

Next time: chatty APIs.

RESTful API Design: how many versions? »

Engage customers where they are - at the edge of your enterprise »

Wayne Gretzky has this great quote -
A good hockey player plays where the puck is. A great hockey player plays where the puck is going to be.

A few days ago I had a great conversation with a big retailer (and with many other customers since then) about engaging customers on the edge of the enterprise and think it's worth sharing.

Today our systems of record in enterprises are ERP and billing systems. That will continue to be the case but the problem is that all these systems of record represent where the puck has already been - what has already happened.

We'd like to predict where the puck is going to be. Today, lots of information can be gathered and analyzed quickly, and in real-time. So you can know what your customer is doing right now. Based on what they are doing now, or have just done, you can predict what they will do, or want to do next.

We're talking about figuring out how to play where the puck is going.
Imagine, you are a leading retailer. A customer walks into your retail location. If they check in (Foursquare, Facebook, Google+, . .  on their mobile device), and you have their credentials, you can know their profile and recent history.  

Where have they been? Did they just leave Walmart or Tiffany? Their spending patterns will obviously be different.

Can you know from their Twitter stream whether they were grocery shopping or anniversary shopping? Knowing this, and correlating it with what they've already bought from you, gives you a perspective and clues to their intent.

Imagine if you can make this information available to a sales person on the spot? Now you've provided that sales person a context in which to interact with the customer. They're equipped to provide a great and customized experience.

The only way to make this happen is if your enterprise is interacting with all of the different streams of data that are available and correlating this data with what you already have inside the enterprise.

Use it all - data in the core enterprise, from outside the enterprise and most importantly from the edge of the enterprise (mostly mobile apps) - to make context-sensitive interactions for customers.

That's what engaging customers on the edge is.

A look at the BlueVia Developer Platform »

BlueVia, a global developer platform initiative of Telefonica, has partnered with Apigee to provide the BlueVia API console for developers to explore and test their APIs.

So we asked them to sit down with us to tell us a little about BlueVia. In this video, BlueVia's Dan Appelquist and Jose Valles discuss the BlueVia Platform with Jeremy Perlman of Apigee.

Tune in to learn what BlueVia is, how the platform provides access to Telefonica's assets and helps developers take apps, web services, and ideas to market.

And it's BlueVia's first birthday and they're celebrating by giving developers a chance to win a MacBook Air and Samsung Galaxy Nexus. Check the BlueVia blog for details.

BlueVia Developer Platform

RESTful API Design: authentication »

This time, in this series about pragmatic RESTful API Design, I'll discuss authentication.

There are many schools of thought - my colleagues at Apigee and I don't always agree on how to handle authentication - but overall here's my take.

Let's look at these three top services. See how each of these services handles things differently:

PayPal

Permissions Service API

Facebook

OAuth 2.0

Twitter

OAuth 1.0a

Note that PayPal's proprietary three-legged permissions API  was in place long before OAuth was conceived.

What should you do?

Use the latest and greatest OAuth - OAuth 2.0 (as of this writing)

Don't do something *like* OAuth, but different
It will be frustrating to app developers if they can't use an OAuth library in their language because of your variation.

Next time: Versions - how many? Meanwhile, I'd love to hear from you over on the API Craft Google Group.

RESTful API Design: tips for handling exceptional behavior »

In the last post in this series, I talked about the top level domain (the stuff on the other side of the URL) and in this RESTful API design series so far, I've dealt with baseline, standard behaviors.

This time I'll explore some of the exceptions that can happen - when clients of Web APIs can't handle all the things we've discussed.

For example, sometimes clients intercept HTTP error codes, or support limited HTTP methods.

What are ways to handle these situations and work within the limitations of a specific client?

Client intercepts HTTP error code

One common thing in some versions of Adobe Flash - if you send an HTTP response that is anything other than HTTP 200 OK, the Flash container intercepts that response and puts the error code in front of the end user of the app.

Therefore, the app developer doesn't have an opportunity to intercept the error code. App developers need the API to support this in some way.

Twitter does an excellent job of handling this.

They have an optional  parameter suppress_response_codes. If set to true, the HTTP response is always 200.

/public_timelines.json?
suppress_response_codes=true
HTTP status code: 200{"error" : "Could not authenticate you."}

Notice that this parameter is a big verbose response code. (They could have used something like src, but they opted to be verbose.)

This is important because when you look at the URL, you need to see that the response codes are being suppressed as it has huge implications about how apps are going to respond to it.

Overall recommendations:

1 - Use suppress_response_codes = true

2 - The HTTP code is no longer just for the code

Our rules from earlier (what about errors?) change. In this context, the HTTP code is no longer just for the code - the program - it's now to be ignored. Client apps are never going to be checking the HTTP status code as it is always the same.

3 - Push any response code that we would have put in the HTTP response down into the response message

In my example below, the response code is 401 - see it in the response message.
Also include additional error codes and verbose information in that message.

Always return OK
/dogs?suppress_response_codes = true 

Code for  ignoring
200 - OK 

Message for people & code
{response_code" : "401", "message" : "Verbose, plain language description of the problem with hints about how to fix it."
"more_info" : "http://dev.tecachdogrest.com/errors/12345", "code" : 12345}

Client supports limited HTTP methods

It is common to see support for GET and POST and not PUT and DELETE.

To maintain the integrity of the four HTTP methods, I suggest you use the following methodology commonly used by Ruby on Rails developers:

Make the method an optional parameter in the URL.

Then the HTTP verb is always a GET but the developer can express rich HTTP verbs and still maintain a RESTful clean API.

Create
/dogs?method=post

Read
/dogs

Update
/dogs/1234?method=put&location=park

Delete
/dogs/1234?method=delete

WARNING:

It can be very dangerous to provide post or delete capabailities using a GET method because if the URL is in a web page then a web crawler like the Googlebot can create or destroy lots of content inadvertently. Be sure you understand the implications of supporting this approach for your applications' context.

Next time: authentication

RESTful API Design: consolidate API requests in one subdomain »

In the series so far, we've talked about everything that comes after the top level domain. This time, let's explore stuff on the other side of the URL.

Here's how Facebook, Foursquare, and Twitter handle this: 

Facebook provides two APIs.

They started with api.facebook.com, then modified it to orient around the social graph graph.facebook.com  

Foursquare has one API. 

Twitter has three APIs; two of them focused on search and streaming.

It's easy to understand how Facebook and Twitter ended up with more than one API.

It has a lot to do with timing and acquisition, and it's easy to reconfigure a CName entry in your DNS to point requests to different clusters.

But if we're making design decisions about what's in the best interest of app developer, I recommend following Foursquare's lead:

Consolidate all API requests under one API subdomain.

It's cleaner, easier and more intuitive for developers who you want to build cool apps using your API.

Facebook, Foursquare, and Twitter also all have dedicated developer portals.

developers.facebook.com
developers.foursquare.com
dev.twitter.com

How to organize all of this? 

Your API gateway should be the top-level domain

api.teachdogrest.com 

Your developer portal should be (in keeping with spirit of REST) developers.yourtopleveldomain.

developers.teachdogrest.com 

Do Web redirects

Then optionally, if you can sense from requests coming in from browser where the developer really needs to go, you can redirect.

Say a developer types api.teachdogrest.com in the browser but there's no other information for the GET request, you can probably safely redirect to your developer portal and help get the developer where he really needs to be.

• api -> developers (if from browser)
• dev -> developers
• developer -> developers

Next time: Exceptional behavior!

RESTful API Design: what about counts? »

What's the right way to do counts?

This question comes up a lot. This post talks about just one interesting way to approach it.

When I first kicked off the conversation in a recent RESTful API Design webinar, I asked developers to chime in with ideas about how best to design for counts, and the conversation over on our API Craft Google group is underway.

Check it out!

So here's where we started.

Say, you want a count of all of some resource (scoped based on whatever the requester has access to).

/dogs/count

In this way, you think about "count" as a reserved word.

This means that you can never have "count" as an identifier on a resource - doing so would return that specific resource.  Otherwise, this would get a count of the resource (dogs) available in the database.

But this is only one way to think about getting counts. I'd love to hear more of your ideas and approaches over on API Craft.

Next: your URL and the top level domain

RESTful API Design: tips for search »

In the most post in this series about Pragmatic REST API design, I talked about handling responses that don't involve resources. This time, a somewhat related topic - search. 

What about searching?

While a simple search could be modeled as a resourceful API (for example, dogs/?q=red), a more complex search across multiple resources requires a different design.

This will sound familiar if you've read the aforementioned API design tip about using verbs not nouns when results don't return a resource from the database - rather the result is some action or calculation. what about responses that don’t involve resources?

If you want to do a global search across resources, I suggest you follow the Google model

Global search

/search?q=fluffy+fur

Here, search is the verb; ?q represents the query.

Scoped search

To add scope to your search, you can prepend with the scope of the search. For example, search in dogs owned by resource ID 5678

/owners/5678/dogs/search?q=fluffy+fur

Formatted results

For search or for any of the action oriented (non-resource) responses, you can prepend with the format as follows:

/search.xml?q=fluffy+fur

Next: what about counts?

Check out the full series on Pragmatic REST API Design. Also we'd love to hear more of your comments and questions over on API Craft.

RESTful API Design: what about responses that don’t involve resources? »

In a related post in this series about Pragmatic REST API design, I talked about partial responses and pagination. Check out the full series. This time:

What's an API response that doesn't involve resources?

API calls that send a response that's not a resource per se are not uncommon depending on the domain. I've seen it in financial services, Telco, and the automotive domain to some extent.

Actions like the following are your clue that you might not be dealing with a "resource" response.

• Calculate
• Translate
• Convert

For example, you want to make a simple algorithmic calculation like how much tax someone should pay, or do a natural language translation (one language in request; another in response), or convert one currency to another. None involve resources returned from a database.

In these cases:

Use verbs not nouns

For example, an API to convert 100 euros to Chinese Yen

/convert?from=EUR&to=CNY&amount=100

Make it clear in your API documentation that these are different.

Simply separate out a section of documentation that makes it clear that you use verbs in cases like this because these calls are not resource related.

Next time: what about searching?

RESTful API Design: what about attribute names? »

In a recent post in this series about Pragmatic REST API design, I talked about formats - supporting multiple formats and working with JSON as the default. Check out the full series.

This time, let's talk about what happens when a response comes back.

You have an object with data attributes on it. How should you name the attributes?

Here are API responses from a few leading APIs:

Twitter

"created_at": Thu Nov 03 05:19;38 +0000 2011"

Bing

"DateTime": "2011-10-29T09:35:00Z"

Foursquare

"createdAt": 1320296464

They each use a different code convention. Although the Twitter approach is familiar to me as a Ruby on Rails developer, I think that Foursquare has the best approach.

How does the API response get back in the code? You parse the response (JSON parser); what comes back populates the Object. It looks like this

var myObject = JSON.parse(response);

If you chose the Twitter or Bing approach, your code looks like this. It's not JavaScript convention and looks weird - looks like the name of another object or class in the system, which is not correct.

timing = myObject.created_at;

timing - myObject.DateTime;

Recommendations

Use JSON as default

Follow JavaScript conventions for naming attributes

- Use medial capitalization (aka CamelCase)

- Use uppercase or lowercase depending on type of object

This results in code that looks like the following, allowing the JavaScript developer to write it in a way that makes sense for JavaScript.

"createdAt": 1320296464

timing = myObject.createdAt;

Next: What to do when your API returns something other than a resource.

RESTful API Design: support multiple formats »

In the most recent post in this series about Pragmatic REST API design, I talked about pagination and partial response. Check out the full series. This time:

What about formats? Should you support only one format or multiple formats?
I recommend that you support more than one format - that you push things out in one format and accept as many formats as necessary. You can usually automate the mapping from format to format.

Here's what the syntax looks like for a few key APIs

Google Data

?alt=json

Foursquare

/venue.json

Digg*

Accept: application/json
?type=json

* the type argument, if present, overrides the Accept header

Digg allows you to specify in two ways: in a pure RESTful way in the Accept header or in the type parameter in the URL. This can be confusing - at the very least you need to document what to do if there are conflicts.

I recommend the Foursquare approach

To get the JSON format from a collection or specific element.

dogs.json

/dogs/1234.json

Developers and even casual users of any file system are familiar to this dot notation. It also requires just one additional character (the period) to get the point across.

What about defaults?

In my opinion, JSON is winning out as the default format. JSON is the closest thing we have to universal language. Even if the back end is built in Ruby on Rails, PHP, Java, Python etc., most projects probably touch JavaScript for the front-end. It also has the advantage of being terse - less verbose than XML.

Next time: what about attribute names?

RESTful API Design: can your API give developers just the information they need? »

In a recent post in this series about Pragmatic REST API design, I talked about tips for versioning your APIs. Check out the full series.

This time - pagination and partial response - how do you give developers exactly the information they need?

Partial response

Take for example the Twitter API - a request for a tweet. You'll get much more than a typical twitter app often needs - including the name of person, the text of the tweet, a timestamp, how often the message was retweeted, and a lot of metadata.

Let's look at how several leading APIs handle giving developers just what they need in responses, including Google who pioneered the idea of partial response.

LinkedIn

/people:(id,first-name,last-name,industry)

This request on a person returns ID, first name, last name, and industry.

LinkedIn uses this terse :(...) syntax which isn't self-evident. Plus it's difficult for a developer to reverse engineer the meaning using a search engine. But this is the way that LinkedIn does partial selection.

Facebook

/joe.smith/friends?fields=id,name,picture

Google

?fields=title,media:group(media:thumbnail)

Google and Facebook have a similar approach, which works well.

They each have an optional parameter called fields after which you put the names of fields you want to be returned.

As you see in this example, you can also put sub-objects in responses to pull in other information from additional resources.

Add optional fields in a comma delimited list

The Google approach works extremely well.

Here's how to get just the information we need from our dogs API using this approach:

/dogs?fields=name,color,location

It's simple to read; a developer can select just the information an app needs at a given time; it cuts down on bandwidth issues, which is important for mobile apps.

The partial selection syntax can also be used to include associated resources cutting down on the number of requests needed to get the required information.

Make it easy for developers to paginate objects in a database

It's almost always a bad idea to return every resource in a database.

Let's look at how Facebook, Twitter, and LinkedIn handle pagination.

• Facebook uses offset and limit
• Twitter uses page and rpp (records per page)
• LinkedIn uses start and count

Semantically, Facebook and LinkedIn do the same thing. That is, the LinkedIn start & count is used in the same way as the Facebook offset & limit.

Say you want to get records 50 through 75 from each system. You would use:

• Facebook -  offset 50 and limit 25
  
• Twitter - page 3 and rpp 25 (records per page)
• LinkedIn - start 50 and count 25
  

Use limit and offset

I recommend limit and offset. It is more common, well understood in leading databases, and easy for developers.

/dogs?limit=25&offset=50

What about defaults?

My loose rule of thumb for default pagination is limit=10 with offset=0.

limit=10&offset=0

The pagination defaults are of course dependent on your data size. If your resources are large, you probably want to limit it to fewer than 10; if resources are small, it can make sense to choose a larger limit.

In summary:

Support partial response by adding optional fields in a comma delimited list

Use limit and offset to make it easy for developers to paginate objects.

Next time: dealing with multiple formats.

RESTful API Design: version & format in URLs or headers? »

In a recent tips for versioning post and on the RESTful API Design webinar, I described some best practices for versioning your APIs - move the v as far left in the URL as possible; place it at the highest scope (e.g. /v1/resources); and use a simple ordinal number, no dot notation.

I wanted to follow up with a little more on versioning, inspired by some great discussion on API craft (keep it coming!).

As I mentioned on the video, there is a strong school of thought about putting format and version in the header.

I think that sometimes people are forced to put the version in the header because they have multiple inter-dependent APIs. That is often a symptom of a bigger problem, namely, they are usually exposing their internal mess
instead of creating a usable API facade on top.

That's very different than saying putting the version in the header is a symptom of a problematic API design. It's not!

In fact, using headers is more correct for many reasons: it leverages existing HTTP standards, it's intellectually consistent with Fielding's vision, it solves some hard real-world problems related to inter-dependent APIs, and more.

However, I think the reason most of the popular APIs do not use it is because it's less fun to hack in a browser.

Simple rules I follow:

If it changes the logic I write to handle the response, put it in the URL so I can see it easily.

If it doesn't change the logic for each response, like OAuth stuff, put it in the header.

These for example, all represent the same resource:

dogs/1
Content-Type: application/json

dogs/1
Content-Type: application/xml

dogs/1
Content-Type: application/png

The code I would write to handle the responses would be very different.

There's no question the header is more correct and it is still a very strong API design.

Next in the series on RESTful API Design: pagination and partial response - how to return just what developers need.

Meanwhile, I'd love to hear more of your thoughts in the API Craft Google group.

Video demo: New updates to the API Console »

This week we released some updates to our Apigee API Console. Here is a short demo of:

• a new API provider page that lists all the API consoles
• a new API Resources page that lists all of the API operations 
• UI changes to the Console itself (split screen, awesome bar, and more)

For now these updates are limited to just the Twitter console, but we'll soon roll them out more broadly.

We'd love your feedback on these changes - you can drop us a line at .(JavaScript must be enabled to view this email address).  Thanks!

'

RESTful API Design: tips for versioning »

In the last post in this series about Pragmatic REST API design, I talked about designing error conditions in your APIs. Check out the full series here.

This time - versioning - one of the most important considerations when designing your pragmatic API.

Never release an API without a version and make the version mandatory.

Let's see how three top API providers handle versioning.

Twilio uses a timestamp in the URL (note the European format).

At compilation time, the developer includes the timestamp of the application when the code was compiled. That timestamp goes in all the HTTP requests.

When the request comes into Twilio, they do a look up. Based on the timestamp they identify the API that was valid when this code was created and route accordingly.

It's a very clever and interesting approach, although I think it is a bit complex. It can be confusing to understand whether the timestamp is compilation time or the timestamp when the API was released, for example.

Salesforce.com uses v20.0, placed somewhere in the middle of the URL.

I like the use of the v. notation. However, I don't like using the .0 because it implies that the interface might be changing more frequently than it should. The logic behind an interface can change rapidly but the interface itself shouldn't change frequently.

Facebook also uses the v. notation but makes the version an optional parameter.

This is problematic because as soon as Facebook forced the API up to the next version, all the apps that didn't include the version number broke and had to be pulled back and version number added.

How to think about version numbers in a pragmatic way with REST?

Specify the version with a 'v' prefix. Move it all the way to the left in the URL so that it has the highest scope (e.g. /v1/dogs).

Use a simple ordinal number - v1, v2, and so on. Don't use the dot notation like v1.2 because it implies a granularity of versioning that doesn't work well with APIs--it's an interface not an implementation.

Make the version mandatory.

How many versions to maintain?

Next time -Giving developers just the information they need -  pagination and partial response.

Please join the API Craft conversation on Google groups.

RESTful API Design: what about errors? »

In the previous posts in this series about Pragmatic REST API design, I talked about simplyfing associations, using the HTTP ? to hide complexities and optional parameters, choosing plural nouns and concrete names, and more. See the series here.

What about errors in the context of RESTful API best practices? Many software developers, including myself, don't always like to think about exceptions and error handling but it is a very important piece of the puzzle for any software developer, and especially for API designers.  

Why is good error design especially important for API designers?

Bottom line, it's about making your APIs intuitive and making developers successful.

First, developers learn to write code through errors.  The "test-first" concepts of the extreme programming model and the more recent "test driven development" models represent a body of best practices that have evolved because this is such an important and natural way for developers to work.

From the perspective of the developer consuming your Web API, everything at the other side of that interface is a black box. Errors therefore become a key tool providing context and visibility into how to use an API.

Secondly, in addition to when they're developing their applications, developers depend on well-designed errors at the critical times when they are troubleshooting and resolving issues after the applications they've built using your APIs are in the hands of their users.

How to think about errors in a pragmatic way with REST?

Let's take a look at how three top APIs approach it.

Facebook
No matter what happens on a Facebook request, you get back the 200 status code - everything is OK. Many error messages also push down into the HTTP response. Here they also throw an #803 error but with no information about what #803 is or how to react to it.

Twilio
Twilio does a great job aligning errors with HTTP status codes. Like Facebook, they provide a more granular error message but with a link that takes you to the documentation. Community commenting  and discussion on the documentation helps to build a body of information and adds context for developers experiencing these errors.

SimpleGeo
Provides error codes but with no additional value in the payload. 

A couple of best practices

Use HTTP status codes
Use HTTP status codes and try to map them cleanly to relevant standard-based codes.

There are over 70 HTTP status codes. However, most developers don't have all 70 memorized. So if you choose status codes that are not very common you will force application developers away from building their apps and over to wikipedia to figure out what you're trying to tell them. 

Therefore, most API providers use a small subset. For example, the Google GData API uses only 10 status codes, Netflix uses 9, and Digg, only 8.

How many status codes should you use for your API?

When you boil it down, there are really only 3 outcomes in the interaction between an app and an API:

• Everything worked
• The application did something wrong
• The API did something wrong

Start by using the following 3 codes. If you need more, add them. But you shouldn't go beyond 8.

• 200 - OK
• 404 - Not Found
• 500 - Internal Server Error

If you're not comfortable reducing all your error conditions to these 3, try picking among these additional 5:

• 201 - Created
• 304 - Not Modified
• 400 - Bad Request
• 401 - Unauthorized
• 403 - Forbidden

(Check out this good Wikipedia entry for all HTTP Status codes.)

It is important that the code that is returned can be consumed and acted upon by the application's business logic - for example, in an if-then-else, or a case statement. 

Make messages returned in the payload as verbose as possible

RESTful API Design: simplify associations, sweep complexities under the HTTP ? »

In the previous post in this series about Pragmatic REST API design, I talked about choosing plural versus singular nouns and concrete names over abstract. See RESTful API Design: plural nouns and concrete names.

This time we'll explore API design considerations when handling associations between resources and parameters like states and attributes.

Associations
Resources almost always have relationship to other resources. What's a simple way to express these relationships in a RESTful world?

Let's look again at the API we modeled in nouns are good, verbs are bad - the API that interacts with our dogs resource. Remember, we had two base URLs:

/dogs

/dogs/1234

We're using HTTP verbs to operate on the resources and collections. Our dogs belong to owners. To get all the dogs belonging to a specific owner, or to create a new dog for that owner, do a GET or a POST:

GET owners/5678/dogs

POST owners/5678/dogs

Now, the relationships can be complex. Owners have relationships with vetinerians, who have relationships with dogs, who have relationships with food, and so on. It's not uncommon to see people string these together making a URL 5 or 6 levels deep. Remember that once you have the primary key for one level, you don't need to include the levels above that because you've already got your specific object. In other words, you shouldn't need too many cases where a URL is deeper than what we have above - resource name/identifier/resource name.

Sweep complexity behind the HTTP ?
Most APIs have intricacies beyond the base level of a resource. Complexities can include many states that can be updated, changed, queried, as well as the attributes associated with a resource.

Make it simple for developers to use the base URL by putting optional states and attributes behind the HTTP question mark. To get all red dogs running in the park:

GET /dogs?color=red&state=running&location=park

In summary, keep your API intuitive by simplifying the associations between resources, and sweeping parameters and other complexities under the rug of the HTTP question mark.

Next time, what about errors?

RESTful API Design: plural nouns and concrete names »

In the first post in this series, Are you a Pragmatist or a RESTafarian?, I proposed that "pragmatic REST" is a design issue.

In the second post, RESTful API Design: nouns are good, verbs are bad, I outlined some of  the API design practices that work well: Nouns in URLs are good. Verbs are bad. Try to limit your API to 2 base URLs per resource.

This time, we'll explore how to pick the nouns for your URLs.

Plural nouns are better than singular
Should you choose singular or plural nouns? You'll see popular APIs use both. Let's look at a few examples  - key resources for these businesses are expressed as either singular or plural:

Foursquare
/checkins

GroupOn
/deals

Zappos
/Product

Given that the first thing most people probably do with a RESTful API is a GET, I think it reads more easily and is more intuitive to use plural nouns. But above all, avoid a mixed model in which you use singular for some resources, plural for others. Being consistent allows developers to predict and guess the method calls as they learn to work with your API.

Concrete names are better than abstract
Achieving pure abstraction is sometimes a goal of API architects. However, that abstraction is not always meaningful for developers.

Take for example an API that accesses content in various forms  - blogs, videos, news articles, and so on.

An API that models everything at the highest level of abstraction -  as /items or /assets in our example - loses the opportunity to paint a tangible picture for a developer to know what they can do with this API. It is more compelling and useful to see the resources listed as blogs, videos, and news articles.

The level of abstraction depends on your scenario. You also want to expose a manageable number of resources. Aim for concrete naming and to keep the number of resources between 12 and 24.

In summary, an intuitive API uses plural nouns and concrete names rather than abstract.

Next time, simplifying associations and hiding complexities using the HTTP ?.

RESTful API Design: nouns are good, verbs are bad »

In the first post in this series about pragmatic REST API design, I defined 'pragmatic REST' as looking at API design from the developer point of view. Now let's get into specific design practices we've seen work well.

The #1 principle in pragmatic RESTful design is: keep simple things simple.

Keep your base URL simple and intuitive

The base URL is the most important design affordance of your API.  A simple and intuitive base URL design will make using your API easy.

A design affordance is a design element that communicates how something should be used without requiring documentation.  A door handle's design should communicate whether you pull or push. But here's an example of a conflict between design affordance and documentation - not an intuitive interface!

My key litmus test for simple API design and pragmatic REST is:  only 2 base URLs per resource

Let's model an API around a simple object or resource (dogs) and create a RESTful API that interacts with dogs.

The first base URL is for collections; the second is for a specific element in the collection.

Boiling it down to this level will also force the verbs out of your base URLs.   

Keep verbs out of your base URLs

Many RESTful APIs start by using a method-driven approach to URL design. These method-based URLs sometimes contain verbs -  sometimes at the beginning, sometimes at the end.

For any resource that you model, like our dog, you can never consider one object in isolation. Rather, there are always related and interacting resources to account for - like owners, veterinarians, leashes, food, squirrels, and so on.

Think about the method calls required to address all the objects in the dogs world. The URLs for our resource might end up looking something like this:



It's a slippery slope - soon you have a huge list of URLs and no consistent pattern making it difficult for developers to learn how to use your APIs.
 
Use HTTP verbs to operate on the collections and elements

For our dog resources, we have two base URLs that use nouns as labels, and we can operate on them with HTTP verbs. Our HTTP verbs are POST, GET, PUT, and DELETE. (I think of them as mapping to the old metaphor of CRUD (Create-Read-Update-Delete).)  

With our two resources (/dogs and /dogs/1234) and the four HTTP verbs, we have a rich set of capability that's intuitive to the developer. Here is a chart that shows what I mean for our dogs.



The point is that a developer probably doesn't need the chart to grok how the API behaves. They can experiment with and learn the API without having to dig into the documentation.  

In summary: Use 2 base URLs per resource.  In your URLs - nouns are good; verbs are bad.

Next time:  how do you pick your nouns?

Please join the API Craft conversation on Google groups.

“Bigger, Better Business with OAuth” API strategy webinar - video & slides »

Thanks to all that attended yesterday's webinar "Bigger, Better Business with OAuth."  

Unfortunately, we had a Webex outage which cut the webinar audio off after a few minutes, but thanks to @sramji and @landlessness for finishing out the webinar for the video (and slides below) after the outage.

OAuth: The next Big Thing in Security »

Today at #defragcon our @sramji gave this talk making the case for OAuth as a business imperative.

(As Sam mentions on the last slide, no developers were harmed in the production of this presentation.) 

API Design: Are you a Pragmatist or a RESTafarian? »

I wanted to write some posts on RESTful API design best practices we've observed over the last year.   I covered a lot of these recently in a API design presentation but want to write more detail out.

First - let's start with our overall point of view on API design.

We advocate pragmatic, not dogmatic REST. What do I mean by dogmatic?   

You might have seen discussion threads on true REST - some of them can get pretty strict and wonky. 

 @mikeschinkel sums it up well - defining the RESTafarian. 

Our view: approach API design from the 'outside-in' perspective.

This means we start by asking - what are we trying to achieve with an API?

The API's job is to make the developer as successful as possible.   

Why? Look at the value chain below - the developer is the lynchpin of our entire API strategy.

So design the API to maximize developer productivity and success as the primary design principle.  

This is what we call pragmatic REST.

Pragmatic REST is a design problem.

You have to get the design right, because design communicates how it will be used.

So now the question is  - what is the design with optimal benefit for the app developer?

The design asethetic for APIs is to think about design choices from the developer point of view.

This is pretty much the guiding principle for all the specific tips that we'll dig into next.

Next time:   considerations in designing your base URLs.

“RESTful API Design: Second Edition”  Webinar - Video & slides »

Here are the slides and video for last week's RESTful API Design webinar, thanks again to everyone for joining.

Check out some additional tips and advice for designing Pragmatic RESTful APIs on the API Best Practices blog.

We'd love more thoughts or questions - there is a thread on the api-craft forum.  

Video and Slides - “Developer segmentation for your API”  strategy webinar »

Thanks to all that attended last week's API strategy webinar "Developer Segmentation for your API" on developer community and market size. (And thanks to our speakers @sramji  and @landlessness).

Here are the video and slides (and as always, you are free to use under Creative Commons).

I have an SDK - do I need an API? »

A frequent question from people who have an existing SDK is “Do I need an API?”  

First, what do we mean by “SDK” and “API”?

An SDK, software development kit, is usually compiled code that is distributed to developers to embed on local devices.

An API, application programming interface, is a set of operations exposed for invocation by a remote system.  (For the purposes of this discussion, we will keep the definition of an API at this high level and not delve into REST vs SOAP, good API design, etc.).

These days, SDKs are generally used when specialized processing will need to be done on the remote device.  Some examples of this might be speech recognition or 3D graphics.  Making these calls remotely could be expensive or impractical.  Some people distribute SDKs when there doesn’t seem to be a compelling reason (specialized processing).  In these situations, all would be better served by the existence of an exposed API.

An API provides a stable interface that, like any good software architecture, hides the ever changing back-end implementation from front-end developers.   API providers generally provide a dedicated web site containing the details of the API and all documentation.  Since nothing needs to be distributed, an API is logistically easier and also eases the burden of compilation against multiple OS and hardware platforms.

Even if you provide an SDK, you probably still need an API. 

(Remember, in this context, an API is a remotely exposed interface, not the set of functions in the SDK).

In today’s world, almost every application will need to communicate with remote servers, mash together other data, connect with social networks, etc…. 

While the SDK can be effective for building the application and performing specialized processing on the device, that application still needs to work with these remote services. 

Having a well managed API exposed between the applications and your back-end provides many benefits (logging, security, policy enforcement, caching, pagination, transformation,…).

There is no reason that an SDK and API need to be mutually exclusive.  While you should probably default to the use of an API whenever possible, SDKs and APIs can be friends when the situation calls for it.

If you're interested in this topic, we wrote more about this earlier in "Do you need an SDK for your API?" and we also just recently released Apigee SDKs for our own OAuth API, supporting Java, Objective C, and Javascript

“Your API is not a website!” - webinar video and slides »

Thanks to those that joined us for last week's API how to webinar presentation "Your API is not a website".   (and thanks to our speakers @gbrail and @brianpagano)

Video and slides are below!

Video and Slides - “Boss, we need an API!”  strategy webinar »

Thanks to all that attended last week's API strategy webinar "Boss, we need an API". (And thanks to our speakers @sramji, @brianpagano, and @landlessness).

Here are the video and slides (and as always, you are free to use under Creative Commons).

Join us this Thursday for our next API webinar "Your API is not a website!" by @gbrail - you can sign up here!

Trust, Safety, and the OAuth API »

At Apigee, we've been developing a secure, robust API management platform since 2005 and have been running it in the cloud since 2008.  We're proud that some of the most demanding enterprise customers like Netflix, Comcast, and GameSpy have made Apigee technology part of their API platform.

We believe that it should be easy to start working with APIs, so last week we launched a new service called the OAuth API, which takes the pain out of getting up and running with APIs that use OAuth.

The story of the OAuth API is the story of yet another function that we used to think of as occurring on a device but that is now moving into the cloud.

It marks the beginning of a trend of APIs being manipulated before they get to the client, since some things are better handled as a service. Authentication mediation is one of those things.

Apigee has built a business of using our API Gateway to publicly expose backend APIs in a safe way, such as by adding OAuth as the mechanism by which applications can be granted permissions by end-users of applications.

The OAuth API takes that model and reverses it, taking something that is complicated for developers to implement (especially across multiple APIs) and simplifying it.

But is it secure?

Should you trust your OAuth keys to a cloud provider, and to Apigee in particular?

In many ways, the cloud might be more secure, since providers do things you might never do on your own. Far too many applications never bother with encrypting users' secrets in their own databases. When you use the OAuth API, you get that for free, in addition to the API normalization and long-term maintenance wins. 

If you do trust the cloud, should you trust Apigee? We respectfully submit that it comes down to three things: technology, processes, and people. 

Technology

Our technology processes billions of OAuth transactions a day. Our Apigee Free tools—including the OAuth API—are built upon this same platform that powers 250 enterprise customers.  It is a core requirement that our servers be both security hardened and highly available.

Processes

We encrypt the secrets entrusted to our database. We run security audits. We involve our security team members throughout the design and architecture process.We use the same software, made by the same people, working under the same processes behind the products that power the APIs of the most demanding Fortune 500 companies. We use industry standard security practices.

And in the end, it's still an OAuth token that is being stored, not a user's password. It can be revoked by a user at any time, and developers can still invalidate their consumer key and secret at any time.

People

Here at Apigee we have lots of experience in API security and operations in the cloud, again as a result of working with demanding enterprise customers for years, many of whom entrust us to protect their API traffic, their organizations, and their customers.

Q&A for OAuth and API Best Practices Webinar »

Thanks to all that attended last week's webinar:  OAuth: The Big Picture (slides and video here).

There were great questions  - here are thoughts on those we didn't have time to discuss.

Q: Would you recommend only using oauth for passwords and usernames?  What about more confidential items like card numbers? (John G)

 I'm not sure I completely understand - but are you asking - if a user can authenticate to OAuth using a credit card number rather than a username/password?

That's theoretically possible but I'm not sure it's a good idea. Also, keep in mind that most credit-card authorization systems use a unique ID to map your transaction to an actual credit card, and for PCI compliance the actual number itself is stored in a PCI-compliant card system. In other words,  I'm not sure that OAuth would be solving a new problem here.

Q: Can you talk about the practices around validity periods of tokens, given that their decoupled from password validity? (Eve Maler)

 Sure. First, why do OAuth tokens need to be refreshed at all?

For one, when an OAuth 1.0a or 2.0 "mac" signature is used without the benefit of SSL, it could theoretically be cracked given enough time and computing resources. However when SSL is in use, you can't do this by just sniffing the network, although if you can intercept a request some other way then you could theoretically do this. So, one reason to refresh the token is so that it can't be cracked.

Another reason is just because as users pass their computers around, phones, etc, it's possible that they won't realize that they contain OAuth tokens. This is a much more subtle and undefined reason than the first.

Some applications give out tokens that never expire, which leads to the theoretical "cracking" example if you are not using SSL. On the other hand, I've seen applications with tokens that expire in 15 minutes, which is not very user-friendly. I don't have a specific recommendation right now and I'm not sure if I have seen another as well.

I actually think that this is a good area for some new research if there's someone out there looking for a project -- I don't think that we've quantified this well.

Q: If the user is entering the user/pass on the browser, even though it may be a trusted website URL, it is still vulnerable for some client to sniff the credentials as it is on the machine itself!! (Kaushik)

 Sure -- nothing on a computer can be completely trusted, and the person operating the machinery can be trusted least of all!

What OAuth does for a web app, is keep the passwords from the server that you are using and prevents your password from being spread around the Internet from server to server. Given the number of security breaches in which servers are compromised and user accounts exposed,  this fixes a big problem.

What OAuth does for a mobile app is keep the password from the application that you are running. I think you can trust the browser embedded in your OS more than you can trust the application built by a developer that you don't know, but I know that others here disagree. (And yes, I realize that an app can put up a fake browser screen. I wonder if such an app can get through the Apple App Store? Does anyone have knowledge about how real a threat this is?)

Q: What options does Apigee support for server-to-server API calls where we want to verify the caller is a valid caller?  i.e. request signature? (Mark Visosky)

 We can support two-way SSL, WS-Security with UserNameToken profile or X.509 profile, SAML, XML encryption / digital signatures, and of course HTTP basic authentication.

Q: if i change my password on twitter, then try to use a previously working app, it makes me reauth (Daniel)

 I didn't know that about Twitter and should have tried before the call.

Q: Thanks for the webinar, you've clearly thought a lot about authentication and security issues.  However, I cannot grant you the point that opening up a web browser where you enter your password to the service provider keeps the user's password secure.  A malicious desktop or mobile application will be able to trick the user into divulging their password by spoofing the oAuth page in some way. oAuth sign in page, I mean (Kevin)

Thanks for the questions. There are certainly other ways to trick a user into divulging their password even beyond what you suggest... That doesn't mean that we have yet abandoned passwords completely.

However, an API provider that DOES want to abandon passwords and replace them with some sort of multi-factor authentication, VPN, SAML, hardware dongle, whatever, can do that by replacing the authentication screen in OAuth with one that requires whatever that API provider wants WITHOUT requiring any change at all on the client. This is a lot harder to do once there are clients out there that have hard-coded username/password authentication.

Q: if it again asks for reauth then does that mean it is using the changed password somehow..? (Kaushik)

An API has the option of revoking the OAuth tokens at any time, and some may choose to revoke them whenever the password is changed. However, OAuth doesn't require or specify that.

Q: Isn't the full OAuth dance only necessary if a user needs to authorize another application to perform actions on their behalf?  i.e. giving authorization to bit.ly to submit tweets through my twitter account.  What about API's where we just want to validate callers are who they say they are but no end-user is involved. (Mark V)

 For server-to-server communications, two-way SSL, or "regular" SSL with HTTP Basic and a long, random password are fine ways to solve this IMHO.

For applications that need to authenticate themselves as "the app" rather than as a particular user,  you can assign each app a unique and random "application key" and include that in each request. Use SSL as well if keeping the app key a secret is important to your business.

Q: Do services need to store/control which third-parties can retrieve your OAuth credentials? Or is it generally the rule that if you expose OAuth, any app can use it? (Kenneth)

 In order to build an application that uses an API via OAuth, the application needs a set of application credentials -- that's the case for OAuth 1.0 and 2.0.

Q: i want to write an OAuth2 *provider* - what libraries are available to me? (Kevin H)

There is a long list of open-source projects at oauth.net under "Code." It's not something that can usually be just "plugged in" to a server, however, because it depends on how you want to authenticate users, where you want to store credentials, etc. At this point in time I should mention that an API gateway product such as Apigee Enterprise is one great way to get it done quickly.

Q: If oAuth requires SSL in 2.0, the request signing step becomes useless, no? (Kevin W)

When OAuth 2.0 is used with a "bearer token," then there is no secret and no request signing -- you use SSL instead.  When OAuth 2.0 is used with a "Mac token" then it is just like 1.0a.

Q: Are the specs for SSL ,SAML assertion available to be used or implemented for my server..? (Kaushik R)

Absolutely. Check ietf.orf and search for "OAuth." There are active groups working on all these specs.

Q: If you're requiring SSL, why not just trade un/pw for a token set and be done with it?  This assumes that the oAuth sign in page is security theater, which I maintain that it is. (Kevin)

@Eve but you can use any RSA spec, as long as the client and server know what to do. (Dan)

un/pw/api key or whatever? (Kevin)

Even the father of OAuth, Eran Hammer-Lahav, says using SSL as a means of authentication is broken (ie, OAuth 2.0 is broken) - (Dan)

 Yes, he is wary of SSL.  That's why he personally ensured that OAuth 2.0 also includes the "mac token" option, which is a way to use OAuth 2.0 with a signature and without SSL, just like 1.0a. This particular spec is newer and changing more quickly but it'll be an important option soon. Given the amount of work he is doing on the spec I don't think he thinks it's broken.

Q: How complex it is to implement OAuth in our own servers..? (Kaushik)

 The hard part is integrating the OAuth flow with your existing authentication mechanisms, so that the browser is redirected and all that. There is also a bunch of work involved in implementing the various authentication flows, and there are quite a few of them. I think there are much more complex things to implement but OAuth does have its challenges.

Q: Where is the OAuth token stored? is it Server side in your example Twitter, or client side BIT.LY or BOTH? I think you mentioned in case of mobile apps it is stored on mobile. (Arpit)

 It depends who is accessing the API on your behalf. The bit.ly web site, for instance,  must store them on the server since it's a web site. The Twitter mobile apps, however,  store it on the mobile device itself.

Q: So are they saying that banks shouldn't use OAuth for banking transactions? (Kenneth K)

 If a bank has a fiduciary responsibility, for example, to sign every transaction with an RSA key that is stored on a hardware key store and managed by a PKI with tight policies and procedures around key distribution, then OAuth isn't going to be used in such an environment any more than plain old username / password combinations. That's the case for financial applications that deal with individual transactions worth millions or billions of dollars.

Seeing that many of us bank on our mobile phones or web browsers using just a username and password for authentication, I think that OAuth is just fine for a wide variety of other financial apps.

Q: The oAuth login screen is security theater - I can spoof it or otherwise trick the user if I am a malicious mobile or desktop app developer.  If you can't force me (the third party app developer) to NOT store the un/pw, why make me goof around with an embedded browser page in my desktop/mobile app?  Thoughts? (Kevin)

 I think I tried to answer this elsewhere, but in general if you really want to do this then OAuth 2.0 actually includes an authentication flow that lets you get a token by simply sending the username / password to the server.

Q: oAuth gives a false sense that your transactions is secure, but the spec really does not mandate any message level security?? (Praveen)

 Well the spec strongly recommends it but the spec can't force it -- it is up to each individual API to require message-level security in the right places.

Q: Advanced talk should have a topic on usage of the "SCOPE" parameter (Vladimir)

I think that the whole "scope" thing would be an interesting seminar,  yes. Let's consider it,  but we'll need a lot of good examples!

Thanks again to everyone and hope to see you on our next webinar:   "Boss, we need an API!"

API Adoption Tip: One simple, open method »

Whether you are creating an API for internal use, partners or the wide open world of application developers -  here is a tip that could help with initial adoption of your API.

Create at least one simple, open API method.

What I mean by simple and open is an API satisfying these criteria:

• no signup required
• no authentication required
• uses HTTP GET
• returns JSON

A good example - is the the Twitter public timeline API method - no signup or authentication required.

Many API teams we work with are creating sophisticated, highly secure APIs with complex authentication and authorization requirements. The API's primary purpose is to access private, secure data.

Why would an API team working on such private stuff take the time to create at least one simple, open API method? Adoption.

If a developer can copy and paste a simple URL into his browser and see JSON coming back then he's already used your API. If he's already a user then getting him to sign up and figure out authentication will be easier for him to stomach.

How do you find an appropriate API method to create?

One place to start is your company's website. Maybe it has a listing of the products and services you sell? Or the people on the management team?

Find some information that is already publicly available and wrap an API around it. Your developers will thank you by using it, signing up and creating compelling apps.

For more on developer adoption - check out our new webinar video on Developer marketing and adoption How-Tos and best practices

Announcing the BlueVia API Console »

We're excited to announce that we've partnered with BlueVia, the new developer platform from Telefonica, to offer developers an Apigee API Console!

BlueVia's APIs gives developers access to Telefonica's network assets and data, including APIs for SMS, MSS, user contextual information, advertising and location.

The BlueVia API Console is built with Apigee To-Go, our free product to let you build, skin and embed your own API Console. With the BlueVia API console, developers will now be able to learn, explore and test the APIs more easily, and start building apps faster. Check out this video demo: 

One thing we love about the BlueVia platform is that it's committed to giving developers between 10-50% of the revenue generated by API transactions.

Additionally, it gives developers 70% of application sales and recurring subscription revenue. Helping developers make money is really important for platform providers, and Telefonica's BlueVia is also a strong example of telco innovation with open data, services and APIs. You can read more about telco APIs, the implications on network operations and frameworks for thinking about tools and technology in our whitepaper on Telco API Management. 

PCI and your API, part 3:  Know your Data »

(This is part 3 of our series on implications of PCI compliance for your API, including previous entries on Cloud computing and PCI and Pragmatic PCI. )

You’ve hardened your processes, separated environments, encrypted tables in your DBs, trained your developers and IT staff.  Then comes your audit and your auditors jump in and run a script against your DB.  Ding, ding, ding....  Left and right you start seeing things that look like Primary Account Numbers!  What? Where? How did this happen?

One of the challenges of PCI is that you’ll be focused on your cardholder environment and your payment processing tools and applications.  Meanwhile, you’ve got API’s, web forms, chat windows, log files, support tickets and any number of other places for data to hide.  Your customers, developers and employees will likely have innocently created a PCI Compliance risk. 

What other streams does data enter your system via?  Do you have a customer support or CRM tool?  Often customers, not so PCI saavy, will send you info they shouldn’t.  An email with a Credit Card number (aka PAN) in it asking for a refund, or a chat window or a comment in an API call.

You didn’t expect this channel of communication to be used to send credit card data, yet sure enough there are a magic 16 digits uncovered at the wrong time, during your assessment.

The antidote?  Know your data.  Know what information is flowing through your system, what information is stored and what might be masked on collection or display.  If you’re an API provider, this can mean watching your APIs for sensitive information passing through.  Not only PAN data, but other data can be useful to avoid storing unencrypted or storing at all.  Social Security Numbers are another sticky piece of data you’d like to avoid if possible.

Leveraging tools to help you discover your hidden vulnerabilities is one tactic as is encrypting vulnerable tables.  Eliminating those vulnerabilities is a better route.  As we shift towards an API Economy, knowing the data passing through your APIs grows ever important in achieving and maintaining PCI compliance.

Shameless Plug#1:  Ask us about Apigee PCI Gateway Policies and how we can help you know your data.

Shameless Plug #2:  For more on this topic, tomorrow we're hosting a live webinar on "Does your API need to be PCI Compliant"?   We'll send you the video if you can't make the live webinar.

PCI and APIs in the Cloud »

Previously we talked about Pragmatic PCI and applying just enough process to ensure you understand and execute your processes in a PCI compliant manner.  

What about when you inject “The Cloud” into the picture?  

PCI Compliance isn’t something that someone can sell you and even a PCI compliant environment can be misused - creating a hole in your assessment.

What is special about the cloud from a PCI perspective?  

First off, you don’t control the physical environment and therefore you are dependent upon your provider’s physical security measures to maintain compliance.  This doesn’t need to be a problem as there are numerous providers available now that can provide “bare metal” that is certified compliant for you to work from. 

You still are likely to have the responsibility of maintaining the virtual machine environment, updating operating systems, app servers, frameworks, applications and databases.   How do you offload that responsibility even further?  PCI is all about the cardholder environment and the protection of Primary Account Numbers (PANs).  

Cloud or on-premise, the guidelines are the same and keeping your exposure minimized is key to simplifying your PCI compliance.  

Isolating your cardholder environment and ensuring servers have a single purpose is key area of compliance and if you can leverage a cloud environment and a providers physical security to meet this goal, so much the better.

For more on PCI and how it might impact your API strategy, check out our live webinar this week - "Does your API need to be PCI compliant?"  If you can't catch the webinar we'll send you the video..

(And a shameless Plug: Apigee also offers a full managed PCI-compliant Cloud API management service. )

Next up, Knowing Your Data before the Auditors Do.

Does your API need PCI compliance? »

Many APIs start out serving content.   But if you eventually want to transact with your API and take credit card orders - you need to understand the implications of PCI DSS compliance.

PCI is a set of requirements that protects your customers and your business from the release of sensitive credit card information.   You don't buy technology to that makes you PCI compliant.  Instead, PCI is a process and checklist of standards that those accepting credit card data must adhere to (more on this here).   But it's important that the technology you use support and maintain your PCI compliance process.

Earlier this week we announced our Apigee Enterprise Cloud PCI - for companies that want to transact with their API, yet take advantage of the cloud for their API management and infrastructure.

On July 16 2011, we'll host a live webinar on the topic of PCI and considerations foryour API.   We'll also be writing a bit more on this topic over the next few days.

Have a great holiday.

Video and Slides: Is your API naked?  API Platform and Ops Considerations »

Thanks to all that attended last week's API Best Practices Webinar #5 "Is your API Naked? API Platform and Operations Considerations" (and thanks to our presenters @gbrail and @landlessness). Video and slides are below.

Our next API webinar, "Your API Sucks!  Why developers hang up and how to stop that" with @landlessness and @earth2marsh, is June 14th at 11am PST (sign up here!)

(And you can see all our API best practices webinars to date here)

Video and Slides: API Metrics - What to Measure »

Thanks to all that attended last week's API Best Practices Webinar #4, API Metrics - What to Measure (and thanks to our presenters @brianpagano and @landlessness). Video and slides are below.

Our next API webinar, "Is your API Naked? API Technology and Ops Considerations" with @landlessness and @gbrail, is June 14th at 11am PST (sign up here!)

Video and Slides: 10 Patterns in Successful API Programs »

Thanks to all that attended last week's API Best Practices webinar #3, 10 Patterns in Successful API Programs (and thanks to our presenters @gbrail and @landlessness). The video, slides, and Q&A is below.

Our next API webinar, "API Metrics: What to Measure" with @landlessness and @brianpagano, is June 2nd at 11am PST (sign up here!)

Questions and Answers from the Live Webinar 
 

Q: What is the best way to throttle/rate limit, software vs hardware?

A: If you’re talking about rate limiting at the API level, based on IP address, OAuth token, username, etc., then there are a bunch of good software solutions, including the Apigee Gateway. Some companies, like us, offer their software packaged inside a hardware appliance as well, but there’s nothing special that hardware can offer when it comes to this kind of rate limiting.

Q: Which OAuth do you recommend - 1.1, 2.0 or wait?

A: It’s hard to answer generally for all APIs, but the latest drafts of OAuth 2.0 are pretty mature, and include options that support a lot of use cases. A big advantage of OAuth 2.0 is that you can secure an API using just a security “bearer token” and SSL, or you can use a signature like OAuth 1.0 supported.

Q: What if I am in a big enterprise that has adopted SOAP as a standard for APIs, but I want to expose as REST.  How deadly, weird, etc... is protocol translation?

A: It’s not deadly or weird at all - lots of our customers are doing it. It’s very common to have internal services based on SOAP that meet the needs of the internal needs, but which don’t necessarily make sense to expose to the Internet in that form.  Putting a translation layer in front that can make use of protocol mediation and data transformation to make the SOAP API look like a REST API that supports JSON makes a lot of sense. The performance impact will be minimal unless you are talking about large data volumes (in the 10,000s of requests/second and above) and in that case you can horizontally scale if you need to throw CPU at the problem.

Q: Does it make sense to provide some kind of "notification" to users, which would  inform that certain part of the data is changing?

A I think it depends on the API! Twitter is a good example (I realize that we keep using them) -- they started with an API that requires you to poll, and they evolved to also offer a streaming API that pushes data out with less latency. But keep in mind that any kind of streaming or push API, including a full streaming API like Twitter, Webhooks (a little less complex), or Web Sockets, is going to be more work for a developer to adopt -- if you choose to offer this it should be in addition to something simpler to use that might offer slower performance.

Q: Does Apigee have insight to Cloud Computing Providers exposing API and the adoption of that by users (both individual consumers and businesses)?

A:  In general we’re seeing that every cloud computing provider is offering an API at some point, and that they’re getting used. I’m not sure that we have any more specifics...

Q: Do you have good example of content API ?

A:  We know of some great APIs in the works that are coming out from some leading media providers. In the UK, the Guardian newspaper is a good example of a content API.

Q: Do you recommend any framework (Python or PHP) for a content API ? 

A: There are many and we should do another webinar on this topic. I (Brian) personally like Ruby on Rails because you will get an excellent RESTful API design for free. There are frameworks for Java like JAX-RS. And for PHP there is FRAPI.

Q: Our API is a paid product. Does it make sense to make it free for developers?

A: You should at least expose some of the API methods for free - also known as a freemium model. It’s a really good way to promote adoption. Related to that it’s also good to have API methods that do not require any authentication - usually a simple read-only method.

Q: Is anyone familiar with open source rate limiting software?

A I know that some of the big guys like Twitter build frameworks on top of memcached -- it’s horizontally scalable and it has an atomic integer increment operation that works well.

Q: Do you recommend any developer community for mobile telecom services, like SMS, billing, etc?

A: Twilio has a good developer community with meetups. Also check out what AT&T is doing. They are starting to put together a solid developer community.

Video: RESTful API Design (Pragmatic, not Dogmatic) »

Recently Brian Mulloy (@landlessness) and Marsh Gardiner (@earth2marsh) hosted a webinar on API design and Pragmatic REST. The video of the recording and the slides are below.

Our next API webinar - 10 Patterns in Successful API Programs - is this Thursday, May 19th at 11am PST, with Brian and Greg Brail @gbrail. Interested in the topic? Then you should sign up now!

RESTful API Design:  Webinar slides and recording »

Thanks to everyone that attended yesterday's RESTful API Design Webinar (and thanks to our presenters @earth2marsh and @landlessness).

The slides are below and here is the full recorded webinar.   (We'll also post a video next week.)

Our next API webinar is "10 Patterns in Successful API Programs" with @landlessness and @gbrail, May 19th at 11am PST (sign up here!) 

API RESTful Design question:  Versioning number in the URL? »

We're starting to get some great questions from signups for next week's API design webinar:  "Pragmatic REST - API Design Fu". 

Here's one we'll comment on now and then talk about more next week.

Q: "For API Versioning.  Doesn't seem very RESTful to version a resource.  Can it be avoided by using implementation and specification versions described in the Sun API Cloud Documentation?"

A: Your question is an insightful one. Versioning is an open issue for REST API design in practice.

Clearly, an API will need to change it's behavior over time. As it changes application developers will need to change their code accordingly. So, versioning is inevitable.  

(caveat: Dan Jacobson of Netflix and NPR's API program has great points on what he calls a 'versionless API' in this presentation around slide 202.")

Our point of view here at Apigee is API design approach we call pragmatic REST.

The driving philosophy of our approach is: what's good for the application developer is good for the API. The Sun Cloud approach to versioning is interesting. The upside is it's powerful and will handle cases of inter-dependent API calls well.

The downside is it's complicated and as a developer starts using it, there is a lot to learn. Plus, because the versions are specified in the header, he won't be able to copy & paste it into the browser to learn it.

Our suggestion - even though it clearly violates pure REST theory - is to include the API version in the URL as a single digit number like /v1, /v2, etc. Whatever complexity is involved in making it "work" is on the shoulders of the API team.

More on this topic in our webinar - sign up here and hope to see you there.

Race conditions: Does your API TOCK TOO much? »

We've written on the topic of API threat detection before, and also outlined a "top 10 API threats" to guard against, but race conditions are another area in which APIs are also vulnerable.  

A race condition is a bug where the output is dependent on a sequence of timing of other events.   APIs are vulnerable to a type of race condition called TOCTTOU (pronounced "TOCK TOO") .   

During this crack in time, malicious users are using race conditions that have been exploited as security vulnerabilities in systems for almost 4 decades.

Historically TOCTTOU attacks have been used to take advantage of a weak filesystem API and limited security controls to OPEN, RENAME, CHANGE OWNER(chown), CHANGE MODE from READ ONLY to READ/WRITE (chmod). Applications such as vi, gedit, rpm, emacs, gdm, and many other GNU programs have been exploited to take advantage of TOCTTOU weaknesses. (See this even longer list at the NIST CVE CCE Vulnerability Database.)

What can you do to reduce race conditions in APIs?

Mediate the request and response with a single atomic event that collapses the check (authenticate) operation with the use (open) events.  If you are building atomic transactions as the design principle, then buffer the system with a checkpoint between the user zone and the service zone — aka, the DMZ.

The use of a mediation proxy in the DMZ to perform the authentication and authorization checks adds latency indeed, but it improves the quality of the system by adding load balancing, packet and message body introspection, event logging, and possibly a second authorization on the response to verify that the user is authorized to receive the object requested.

A second authorization check can also:
-    prevent accidental data leakage
-    makes it more difficult to create a race condition
-    detect if the user is no longer authorized to read the content, or if the user is attempting to access an object without execution, read or write entitlements.

So if you are concerned that your firewall is not doing enough for you, or you are worried that your DMZ has been compromised by the firewall TCP Split Handshake vulnerabilities, then consider using an extra layer of mediation between client side applications and server side APIs.

A policy enforcement point (like our Apigee API Gateway)  - behind the firewall and in front of the API service - reduces the risk of a man-in-the-middle attack.

Are all OAuth implementations Created Equal? »

We've implemented a couple dozen OAuth implementations in the past months, and no two have been the same.

We've seen Oauth 1.0a (2-legged,3-legged), Oauth 2.0 (username password, user-agent, web-server flow) with custom token types (bearer,mac, saml).  Just to name a few.

We wanted to write some posts that discuss factors we see that drive these variances and their implications.  We also want to compare and contrast each of these variances along with our own lessons learned. 

First, why the variances?  In our experience, the primary cause of the variance is the type of application that the API is targeted for, which tend to break into one or more of 3 types.

Partner Apps - typically a "closed" API available only to a select group of partners with whom you collaborate very closely to build the apps. For instance - an health insurance provider who is exposing an API that a partner can use to build some health related activity tracing applications.

Open - Browser / Mobile Apps - APIs are exposed to a large community of 3rd party developer from whom you are expecting a large scale of applications to drive innovation. Here you do not control who, how many and what kind of applications get build using the APIs.   For example, mobile applications that integrate with the social networking sites.

Enterprise Apps -  when the API is exposed for use primarily by other enterprise customers - much like a B2B scenario - to allow the customers the freedom and flexibility of building in house Applications and services that mashup one of more of the applications from multiple vendors for internal optimizations.  A good example could be an enterprise that wants to use the Sales Force API along with one other internal apps to build an internal collaboration app.

Depending on the application type above, your approach to an OAuth implementation may vary:

An API for Partner Apps will typically follow a “User name / password” flow.  In this case the end user’s  user name and password will be passed from the user to the partner app and the partner app transmits this to the origin / backend API. The key consideration here is that the partner app will have direct access to the end users credentials and is trusted to treat this information sensibly.

In the Open Browser / Mobile apps scenario the app itself is not inherently trusted and is not allowed to handle the end user credentials directly. The onus of authenticating the end user falls squarely on the origin / backend API provider. The provider (after the end user authentication is successful) will simply issue an access token to the app – this token will correspond to the set of capabilities that the end user has authorized the application for. The sequence of steps followed in this case is commonly referred to as the “user-agent flow” model.

The Enterprise Apps scenario is by far the most robust flow (and understandably, the most complex one too). Here the access token is given to an intermediary server instead of the client device and this facilitates a more secure transmission of the access token. In this scenario, it is possible to restrict the source from which the web service is invoked. This sequence of steps is commonly referred to as the “web server flow” model.

Sounds complicated?  It is. We will try to simplify each of these variants and throw some more light in a series of follow-up entries. Stay tuned!

Mapping out your API Strategy: Webinar slides and recording »

Thanks to everyone that attended yesterday's API Strategy Workshop Webinar (and thanks to our presenters @sramji and @landlessness)

The slides are below and here is the full recorded webinar. 

Join our next API webinar, "Pragmatic REST:API Design Fu", May 5th at 11am PST.  You can sign up here. 

Free Webinar: Mapping Out Your API Strategy »

Ever need to explain why APIs are so powerful to someone at your company?  Need an easier way to think about the different API strategy options?

Join us for a Webinar - “Mapping Out Your API Strategy” - this Wednesday, April 20 at 11 am PST / 2 pm EST.

It's free (of course) and you can sign up at this registration page.

Brian Mulloy and Sam Ramji will talk about the API market, and propose some ways of thinking (and cool frameworks) for how you can think about API strategy vs. your objectives.  

Punctuated Equilibrium, Celestial Navigation, and APIs - Web 2.0 2011 API Strategy talk »

Sam Ramji from Apigee, along with Dan Jacobson and Michael Hart from Netflix, recently gave this API strategy talk at the Web 2.0 Expo in San Francisco. 

It includes frameworks, best practices and lessons learned to help in thinking of your API strategy from a business model, architecture, and data perspective.   

Why Punctuated Equilibrium and Celestial Navigation? See the slides below and look out for the full video shortly. 

Shiny Shiny Monday: Two-legged OAuth, Custom Token, and more… »

We roll out big features with lots of fanfare, but the little ones roll out as they are ready. On Mondays we'll highlight recent changes here. For this week we've released:

1. Support for file attachments! Now you can attach a photo to your Twitter profile or Facebook via an API call (screenshot below). Note that you can attach via URL or by uploading from your local machine. We recommend the URL method, as that way it will be available to others if you choose to share a snapshot.
2. Custom Token - after launching Apigee To-Go, people clamored for support for custom tokens, so we added it!
3. Two legged OAuth - we've also added 2-legged OAuth, a very valid use case when people have APIs that aren't based around the concept of users granting 3rd parties permissions.

Why you need OAuth for your API or App »

I've been posting a bit on OAuth best practices recently. But I want to take a step back and talk about why OAuth is important in the first place, why it's difficult, and how to think about it for your API.

Why is OAuth important?
 
First, OAuth supports delegated authentication between web apps. That means that if web app X offers an API, and web site Y wants to use that API, then no one needs to type their "X" password into a screen controlled by "Y." This is a big deal and it's being used by a large and growing number of apps.
 
Second, OAuth does this using a security token called an "access token." An OAuth access token is a unique identifier that allows whoever has the token to get access from one and only one application to one and only one API. (The creators of OAuth describe this access token as a "valet  key," and that's a good description.)
 
An access token is different from a password, which allows whoever has the password to get access from any application to an API, because it is tied to a single application. Plus, access tokens are large and random, so they're not subject to a dictionary attack like a password, and they are hard to share. It's not going to be the case that your access token for Facebook is going to be the same one that you use for Foursquare, whereas if you're like most humans you might use the same password!
 
Third, in order to get access to an access token, an application itself must somehow be authenticated. That means that not only do users need security credentials in an OAuth world, but so do applications. This is a good thing because apps can't always be trusted, and with OAuth each API provider has the ability to revoke access to a rogue application if necessary. (Although there's no way to make it totally impossible for a determined hacker to extract the credentials from a mobile app given enough time and effort, the existence of these credentials adds a layer of security that wouldn't be there otherwise.)
 
Fourth, OAuth makes it possible to separate the assignment of API security credentials from the process of authenticating users to a web site. What does that mean? It means that if you are big company with a web app, and you want to offer an API, it's not necessary to wire the API into the web site infrastructure. Instead, the API can be implemented alongside the web site, and only link to one web page for the actual user authentication. (And even this can be done in other ways.) That means that companies of all sizes can build OAuth-enabled APIs without impacting their existing applications, and that's a good thing.
 
Getting all these things right is hard. For instance, the very first "production" version of OAuth 1.0 didn't actually do the first thing -- authentication delegation -- correctly and as a result the spec itself had to be patched (that's why we talk about OAuth 1.0a now). A lot of people looked at 1.0 without realizing this, which goes to show how tricky it is to create a new security standard from the ground up without making big mistakes.
 
Why has OAuth been difficult?
 
Right now, the OAuth world is sadly full and ambiguity. This is largely because of implementation work on the OAuth 2.0 spec, which offers the promise of an easy-to-use OAuth specification, but at the same time keeps changing. We fully expect the rate of change to slow or even stop in the next few months, but no one knows for sure, especially since it is an IETF process, which takes time.
 
However, it takes time to do a job right, and security standards that impact the whole Internet certainly deserve to be done right.
 
Unfortunately, that puts today's API implementors between a rock and a hard place. Until OAuth 2.0 is a little more stable, you have the choice of implementing the latest draft and being prepared for changes, and implementing OAuth 1.0a, which makes life more difficult for customers.
 
What should you do?
 
Be patient. It's possible to build an effective API right now using either a current draft of OAuth 2.0 (see Facebook) or OAuth 1.0a (see Twitter).
 
However, things are going to change a bit for the near future. Some would take this as a reason to give up, "sunset" OAuth, or declare it "dead" way before its time. I think that building a new security technology from the ground up that has all the advantages I enumerated at the top of this article would be a big mistake.  We have two pretty-good options today in OAuth 1.0a the latest drafts of 2.0 -- stick with them.

Pragmatic PCI Compliance and APIs: Just Enough Process »

“If the minimum wasn’t good enough, it wouldn’t be the minimum.”  - Keith W.

Wise words from one of my developers many years ago.  When it comes to tackling PCI Compliance, it is advice well worth taking.

With leaks of sensitive customer information in the news, there’s an increased focus on compliance as more services shift to cloud computing and APIs.  

If you are a merchant of any kind or deal with customer credit card information then you must be aware of PCI compliance regulations that are designed to protect consumer credit card information from exposure.

PCI compliance gets tricky as apps and services move to cloud services and APIs.   If you’re heading down the path of PCI compliance or just trying to position yourself, your APIs and your internal systems better for the future, keeping it simple will help you be successful.

First, Document your Process

The PCI Data Security Standards (PCI DSS) establish the “what” but not the “how” of achieving compliance. 

The how is up to you. But like most audit and process centric assessments, what is most important is being able to articulate what you do to support a particular DSS item - and then being able to show evidence to support that statement.

Identify all of the process standards that apply to you from the DSS and identify the proper owner of those processes as well.  Put together a simple Process Description Template that everyone uses to document their individual processes and adopt a naming convention that calls out the DSS section.  Centralize the storage of those documents and make sure everyone knows where they are. 

Just focus on capturing the “how” of your processes in as lean a manner as possible.  Your assessment team is not going to evaluate quality of the process or the documentation, only that it meets the requirements of the DSS.

With your processes documented "well enough" and easily mapped to the PCI DSS, you’ll discover gaps, strengths and you’ll make your assessors life easier and that makes your life easier.

Next, we'll talk about the special challenges with PCI in cloud computing and APIs, and practices that you can apply to reduce your risk.

Morgan Hall is a project manager with Apigee. Previously he was Director of Business Architecture at TransUnion.

Universal Design Principles Applied to APIs (Part 2 of 4) »

Following Part 1, here are the slides and a video for Part 2 of our series on applying universal principles of design to APIs.  In this part I cover 3 more universal design principles, including:

• Flexibility-Usability Tradeoff
• Hick’s Law
• 80/20 Rule
• Inverted Pyramid

And for more on API Design check out Teach a Dog to REST and REST for SQL Developers

Best Practices for OAuth 2.0 vs. OAuth 1.0 - One year later »

Since we first wrote about OAuth 1.0 vs. 2.0 a while back, lots has been happening and lots has changed. The OAuth community has made progress and made changes, and an increasing number of API providers have deployed APIs that use OAuth 2.0. (Similarly, the number of new OAuth 1.0-enabled APIs doesn't seem to be growing.)

We've been busy helping our customers implement OAuth-based APIs, and we've also been watching the process develop. Here are some things that we've learned: 

OAuth is a solution -- not technology.

In order for an API provider to support OAuth, a number of things have to happen. There must be a place where the OAuth protocol is actually implemented -- all of the different URLs for requesting tokens, issuing tokens, redirecting browsers, and so on. This part is a matter of implementing the standard and doesn't change from implementation to implementation (so, it can be delegated to a third-party component like Apigee Enterprise).

In addition, there has to be somewhere to store OAuth tokens. There is no standard for this -- some use an RDBMS, some use NoSQL, and others use a cloud-hosted service.

Finally, the OAuth solution has to integrate with a provider's "login page," so that when a user wants to get an OAuth token, they are redirected to a legitimate "login" page with a proper SSL certificate,  look and feel, and so on. All of these kinds of things can be different from one provider to another.

At Apigee, we have been working with many companies to deploy APIs built using OAuth. What we've found is that OAuth is not just a technology that you buy or download -- it's a bit of technology and a bit of integration, and to implement it at all you need a platform that can flexibly integrate with what you already have, while adding what you might be missing. 

Be careful.

Since we last blogged about it, OAuth 2.0 has been improved a bit and has changed some basic things -- for instance the name on the HTTP "Authorization" header has changed from "OAuth" to "Bearer." This is a big change, and obviously if an API provider just switches from one value to another without warning, all the clients will break.

OAuth 2.0 is up to draft 13, and with every draft it gets closer to being "done" -- but it's not done yet and I haven't seen anyone say when it will be. That means that if you implement, say, draft 13, you will likely have to change, and find a way to maintain backward compatibility for existing clients.

If that's too much, then implement OAuth 1.0a -- it's as "done" as it's going to get at this point! 

Stick with "bearer tokens" for now in conjunction with SSL.

The latest draft includes support for various types of security tokens,  and a mechanism to plug in new types. The simplest are "bearer tokens." A bearer token is just a big, random string that a client must present on every API call. Bearer tokens are simple because there's no special signature or validation code required on either end. The client is responsible for storing the token in a safe place and sending it with every request. The server is responsible for looking up the token in a database and making sure it's a valid one -- that's it.

However, bearer tokens provide no security unless used in conjunction with SSL -- otherwise, any eavesdropper in a coffee shop can snatch OAuth tokens from  the air. (Fortunately, OAuth tokens can be individually revoked without requiring a password reset.) 

Get ready to change -- or stick with 1.0a.

We said this before but it bears repeating -- if you implement OAuth 2.0, it is likely that you will have to change your servers to support newer versions, and also maintain backward compatibility.

One way to ease the pain is to put version numbers in the URLs that you hand out for the various steps in the OAuth flow. So, instead of using:

http://api.foo.com/oauth/authorize

use:

http://api.foo.com/oauth/13/authorize

because you know that there will be future versions and that they will be different...

Get ready for more options.

 Part of the OAuth 2.0 work defines some new security token types other than "bearer." Right now only "bearer" seems to be sufficiently mature from a spec standard, but this will change over the next few months. Here are some of the options:

1.       Bearer -- as we mentioned before, this is just a long, random string. The advantage is that it's extremely easy to implement on both client and server. The disadvantage is that it requires SSL for all API calls. Also, if API traffic passes through multiple endpoints such as proxies, then the proxies will be able to see the token as it passes by.

2.       Mac -- this is equivalent to the token scheme in OAuth 1.0, in that it requires both a key and secret, and uses an "Hmac" algorithm to encrypt some data on each request. The result is that the request is only valid if both client and server have the same keys, and there is no way for an intermediary to re-create the original request without a password. In other words, it is secure even if SSL is not used or even if the request is intercepted by a proxy.

3.       SAML -- this uses SAML assertions on each request as a way to establish the client's identity. This is helpful if you already have SAML infrastructure for either the client or server in your environment.

Build & Run a Web API for FREE »

With the proliferation of free, cloud services it's possible to build and run interesting mobile and web projects from end-to-end for free--including an awesome web API.

Here are 10 steps to building and running a web API for free.

1. Prepare

Get up to speed on APIs - from their economic and business model rationale to design best practices and principles to developer community practices.

 2. Design

Prioritize your resources (objects) in a Google spreadsheet and start to craft your URLs and HTTP methods. 
 

3. Build

Build your API quickly using Ruby on Rails scaffolding.  And here are some additional Rails API tips from @StefanSiebel.

Store your source code in GitHub.

 4. Deploy

After you create your Rails-based API, deploy it to Heroku.

5. Proxy

Use Apigee's Free service for API analytics, debugging and rate limiting.

6. Document

Document your API using GitHub pages.

7. Engage

Engage developers by embedding an API Console into your docs using Apigee To-Go.

8. Launch

Grab a domain name for your project from a DNS provider like GoDaddy.

Setup CNAME records for:

• GitHub docs: dev.mydomain.com
• Apigee proxy: api.mydomain.com

9. Promote

Use Twitter, Blogger and Facebook to promote your API.

10. Support

Support your community with GetSatisfaction and manage tickets with GitHub's Issues feature.

A couple of us (@earth2marsh & @landlessness) recently built an API with this set of products and had a lot of fun doing it. We'll talk more about that project in a future post.

A Short History of API Authentication (and where it’s going): from HTTP basic to OAuth 2.0 »

Part 1: The Web

In the beginning -- way back in the beginning -- the web was all about open access. Tim Berners-Lee and his colleagues focused on making information available, not on protecting it from unauthorized users.

But as time went on, and as Al Gore took the initiative in liberating the government-run Internet backbone for commercial use (really), the Web became about "e-commerce,"  and e-commerce required security. SSL matured to ensure that sensitive traffic was encrypted all the way from the client to the server and back, and various schemes emerged to allow user authentication.

The oldest and most common format for web authentication is HTTP Basic authentication. This is what you get when you visit a web site and a little browser window pops up requesting a username and password. Every web browser and every major web server supports some form of this.
 
From a web design perspective, HTTP Basic has a big disadvantage in that it's implemented entirely by the browser, and can't be customized for each site. As the quality of graphic design improved on the web, designers soon realized that they wanted not a generic little grey box on the screen, but a carefully-designed login page, with logos, disclaimers, and the like, or a discreet login button on the corner of a web page. The combination of HTTP Basic and HTML just didn't allow this.

The result was the rise of form-based authentication. This is what the vast majority of secure web sites use today. As a user, you visit a web page that prompts you for a username and password. If authentication is successful, then under the covers you are granted a unique cookie, which your web browser sends with each subsequent request. As the user you never see this -- it just looks like you logged in and now the site works -- but under the covers it is quite different from Basic.

Both Basic and form-based authentication rely on SSL to create an encrypted tunnel between the client and server. Without that encrypted tunnel, anyone snooping the Internet or the open Wifi at Starbucks can see the passwords go by in the clear. Fortunately, SSL protects against this very well, but sometimes developers neglect to use it, users neglect to ask for it (as most of us do with Facebook), and sometimes the traffic travels over unencrypted links behind the firewall of a large network.
 
The web community attempted to counter this using HTTP Digest authentication, which encrypts the password using a one-way hash so it's impossible to reverse-engineer even if SSL is not used -- but it still must be implemented by the browser and can't be designed in to a nice UI. It never took off.
 
For a higher level of security, SSL has long supported two-way authentication, which requires that individual end users request digital certificates for each site they plan to visit and install those signatures on their browsers. The overhead of issuing PKI credentials to end users was enormous and this never took off either.
 
Part 2: Early APIs
 
Some early APIs were built right on top of existing web sites built using form-based authentication. The easiest way to implement them was to use the same authentication mechanism, so API developers would create a method called "login" that returns a security token, another method called "logout," and require the security token on every API call.
 
This approach makes the API easy to tack on top of an existing web app, but it is more work on the client side and hurts API adoption. An administrator can't as easily drive the API from the script without logging in, extracting the security token from the response, making the call, and then logging out.
 
Other early APIs just use HTTP Basic authentication. It's simple, works with every client (and with every shell script based on "cURL"), but it requires SSL to be used, often leaving it up to the client to "remember" to use it. Still, it's effective as long as the user has the password for the API handy.
 
Yet others, especially Amazon, decided they wanted to avoid using SSL for performance reasons, but they also wanted to avoid using the uncommon HTTP Digest authentication. (Amazon's S3 is used to store multi-gigabyte files and SSL does make a difference there.) They chose to create their own access control mechanisms based on secret keys and in some cases, digital signatures. The result was a bit of programming for each developer starting out with AWS, but Amazon's services were so useful and cheap that it didn't matter. By now there are numerous libraries to make this process easier.
 
Part 3: APIs get formal
 
The first real access control mechanism aimed at the needs of APIs and API developers is OAuth. The idea came from a popular API (Twitter) and a defunct web site (Ma.gnolia). The goal was to make it possible for a Ma.gnolia user to access Twitter without requiring that each user give Ma.gnolia their Twitter password.
 
The result, OAuth 1.0, is like a "valet key" for an API -- it is a token that gives a single client or web site access to a particular API on behalf of one user. The client or web site can get an OAuth "access token" without ever seeing the user's password because the two web sites do a sort of "credential dance" to exchange the secret token. Once the token is issued, the user can see it or revoke it, thus taking away access from the client or site without requiring a password change.
 
Since then, OAuth 1.0 has seen a patch (it became 1.0a to fix a security flaw), and an IETF committee is working on OAuth 2.0. This new version includes some important simplifications and a wider range of use cases. OAuth has become the de facto standard for API authentication. (for more see my earlier post on OAuth 1.0 vs. OAuth 2.0)
 
The Future
 
OAuth is likely to dominate the world of APIs for a long time. With OAuth 2.0, soon API providers and users will have the option of using a simple "bearer token" in conjunction with SSL, or a signed token that can remain secure without further encryption. OAuth 2.0 is flexible enough to be used in the original "web site to API" use case that OAuth 1.0 was designed to handle, but it can also be used for access by mobile devices, where it can be important to be able to remotely revoke the access tokens that might be stored on your phone without going around and changing innumerable passwords.
 
Still, there remain problems to be solved. What about access for simple management APIs, for instance? Even OAuth 2.0 is cumbersome and when SSL is always in use sometimes a plain old password is sufficient. API providers can and should consider supporting good old Basic authentication alongside OAuth 2.0 if for no other reason than convenience -- as long as SSL and a strong password are required.
 
What about mobile apps? Is there a way for the server hosting an API for, say, a large bank to ensure that the request is coming from an official application, or from a rogue app that is attempting to "phish" passwords by pretending to be the real thing? Can we do something by combining signed applications with server-side validation, or is a secure app store the only way to protect against mobile malware?
 
The world of APIs is evolving and there's no doubt that security technology will continue to evolve along with it.

(For more on API security issues see our other entries on OAuth and API authorization, identity, and security - or get it all with our "Is your API Naked" whitepaper)

REST API design for SQL programmers »

@gbrail put together this short deck mapping SQL concepts to (pragmatic) RESTful API design.  And for more on API design - see Teach a Dog to REST and Universal Design Principles Applied to APIs

REST API Design for SQL developers

Why XML won’t die: XML vs. JSON for your API »

Last week I wrote that if you're API doesn't support JSON and JSONP - you're doing it wrong.   I don't think that's terribly controversial.

But is JSON (and JSONP) perfect for everything you need to support with your API?  Is XML dead?

JSON is especially good at representing programming-language objects. If you have a JavaScript or Java object, or even a C struct, the structure of the object and all its fields can be easily and quickly converted to JSON, sent over a network, and retrieved on the other end without too much difficulty and (usually) comes out the same on both ends.

But not everything in the world is a programming-language object. Sometimes to describe a complex real-world object we have to combine different descriptions and languages from different places, mash them up, and use them to describe even more complex things. The descriptions of these complex things need to be validated, they need to be commented on, they need to be shared and sometimes annotated with additional data that doesn't affect the original structure.

When the world gets complicated and open-ended like that, what's needed is not a programming-language-format object, but a open-ended, extensible -- umm -- markup language. That's what we have today with XML.

For instance, the travel industry (through the Open Axis Group), the insurance industry (through ACORD) and the financial services industry (through FpML) have all spent many person-years developing standards that describe what they do in XML format. Each standard comes complete with a schema, which means that any client or server can validate a document to ensure it is correct enough before starting to parse it, and which makes it easier to edit the document using one of the many of the mature tools that are available.

Sure, parsing and understanding these documents is not simple, but they do not represent simple things. The ability to represent a complex travel itinerary, a life insurance policy, or an interest-rate swap in a standards-based format is a big deal and a triumph of XML technology.

Similarly, look at HTML. (Most HTML is not XML but both come from SGML and are very similar.) HTML works because it can combine both structured and unstructured content in various ways and accept the ability to mash up different standards into one document.

In my opinion, XML will only be dead when the web has replaced HTML with JSON.

So for our APIs, let's embrace JSON -- it's small, simple, and easy to use. But when we have to collaborate on complex documents, pull information from different places, and define complex schemas to represent complex real-world concepts, let's also not forget about good old XML.

Universal Design Principles Applied to APIs (Part 1 of 4) »

The slides and video for Part 1 follow:

For more on API Design - see another presentation and video "Teach a Dog to REST: RESTful API Design Principles" 

We'd love to hear what you think - reach me at brian at apigee or @landlessness

Crossing the Streams:  Handling cross-site API requests (JSONP, CORS, UMP and more) »

My earlier post "Not using JSON and JSONP? You're doing it wrong!"  generated a lot of questions about the best ways to handle cross-site API requests from JavaScript running in the browser.

(Short intro -- the browser won't let you do it with the standard XmlHttpRequest API because of the "same-origin policy." This policy is in your browser's JavaScript engine so when you hit an application at http://www.foo.com, there's no way for http://www.badguys.com to return bad data to your application.)

This is an important topic, because if you are providing an API at api.yourapi.com, no one can use your API from inside a browser unless you implement one, and preferably more than one, of these techniques. It's also important because the state of the art is changing and changing fast.

Here's a run-down of the various techniques today, and how you might choose,  with the inevitable links to Wikipedia to help guide you: 

JSONP:

What it is: You return JSON from your API, and wrap it with a user-supplied "callback" identifier. This turns your JSON response from a JavaScript object into a function call or an assignment statement. Your user uses the DOM API to dynamically insert "script" tags into the web page to cause your API to be invoked, then executes the result as a JavaScript function call to create the object and start using it. 

Why it's good: Works on every browser, and asynchronously like every JavaScript network resource to boot. If users use a library like jQuery it is very easy to use.

Why it's bad: Only works with JSON,  although if you read my last entry then your API returns JSON anyway! It only works with GET, so you have to pollute your beautiful and philosophically-correct REST API by adding something like an "action" query parameter -- this tells your server that even though it just got a "GET" call, it should behave as if it were a POST, and you can't upload data with GET.

Finally, JSONP is really a hack around the security sandbox in your browser. As a result, it lets your JavaScript code download executable code from  another domain and run it inside the browser. If the user of your API trusts you, then that's great, but otherwise they're placing the security of their web app in your hands.

Aside from offering JSON as a response format with callbacks, you can also specify XML as the response with JSONP-X.

CORS: 

What it is: Cross Origin Resource Sharing is a W3C spec that allows a web service to specify that it's OK for it to be invoked from any domain. The API provider implements it by returning a special set of HTTP headers with every API response, and also by implementing an HTTP OPTIONS "pre-flight" request that lets the browser check first to see if a request is going to be OK. The actual spec is awfully complex but what you actually need to know in most cases isn't all that bad.

Why it's good: Because it lets the API consumer easily say, "it's OK, browser -- go ahead and make that API call." Once it's implemented, browsers that understand the CORS spec will automatically open a hole in the cross-origin policy as part of the standard XmlHttpRequest API -- the programmer of the web app doesn't have to do anything special. Plus, it can return any kind of data, not just JSON, and it works with all HTTP verbs. 

Why it's bad: It's pretty new and not all browsers implement it. Firefox 3.5, Chrome 3, and Safari 4 has it. IE8 does too but using a slightly different API. And support in many smaller mobile browsers (like Opera) is sketchy or nonexistent.

In other words, CORS is super-duper awesome as long as you don't care about people who run IE -- whether that's important to you is up to you

UMP:

What it is: UMP is a re-boot of CORS -- it handles 90 percent (or more) of the reasons an API would want to implement CORS without 90 percent of the complexity. It's also backward compatible in that it uses many of the same HTTP headers as CORS.

Why it's good: Like CORS, it allows an API to easily make itself available to web browser clients with a minimum of fuss. Better yet, all the API has to do is set a few HTTP headers, without also implementing an OPTIONS request.

Why it's bad: It's even newer than CORS. (New enough that it doesn't even have a Wikipedia page.) I have been told that recent versions of Chrome support it but it's not clear where else it can work at this point.

There's more to compare CORS and UMP here

Cross domain files

What they are: Flash and Silverlight also have a same-origin policy, but they do not work the same as the JavaScript engine and the techniques above do apply. What they do support are special files, namely crossdomain.xml (for Flash) and clientaccesspolicy.xml (for Silverlight). These function the same way as CORS and UMP headers in that they tell the client when it's OK to invoke the API across domains.

Why they're good: If you have Flash or Silverlight clients for your API, you need these too so that your API can be invoked from  those environments.

Why they're bad: If you don't want or need to support Flash and Silverlight clients then you don't need them.

So which one should I pick?

 If you want your API to be adopted in as many places as possible then you should implement all of them! Seriously:

1.       Always make it possible for your API to return JSON.

2.       Support JSONP as much as you can (since you can't use it to upload data that doesn't fit in the URL) and leave it up to the users of your API to assess the security risks

3.       Support CORS by returning the Access-Control-Allow-Origin header on all API responses and implementing the OPTIONS verb as specified in the CORS spec.

4.       If you do the above your API will also support UMP.

5.       Return a crossdomain.xml and clientaccesspolicy.xml file from the top level of your API domain for Flash and Silverlight clients.

Whew!

At Apigee, we have done the above for users of our Apigee Enterprise product, and since it's so flexible we can change what we support as the standards change.

Not serving JSON AND JSONP? Then you’re doing it wrong! »

If you've used an API recently, you've probably seen that the popular APIs out there support JSON. JavaScript Object Notation is a standard defined a while back by Douglas Crockford from Yahoo. It uses a subset of the JavaScript syntax to simply and effectively describe an object.

In the last few years, JSON has taken its place alongside XML as the de facto way to describe API data. Today's leading APIs support JSON in addition to XML, and an increasing number support only JSON.

JSON is popular because it's simple. Programming-language objects map to and from JSON in a straightforward way that everyone can understand. You need a "JSON parser" to convert JSON into an object (unless you're working in JavaScript) but you don't need to know much about it other than how to make it go.

If you are thinking of building an API, JSON support is critical. Here's why:

JavaScript. JSON is JavaScript. That is, a "JSON object" is literally a small fragment of JavaScript that represents an object and its sub-objects. That means that creating a real JavaScript object based on some JSON text is simple and fast. Web programmers love JSON.

JSONP. This lets JavaScript running inside the browser invoke APIs that reside on a different host on the Internet. This doesn't sound like a big deal but it is actually huge because all browsers implement a "same-origin policy" that otherwise makes this impossible. JSONP is hard to implement, but libraries like jQuery make it easy for the client if the server already supports it.  If you're not serving JSON AND JSONP you're doing it wrong!

Smaller. Smaller is better, especially in the mobile environment, and since JSON doesn't "say" every field name twice like XML does, JSON output is a lot smaller than XML.

Less complicated. JSON is free of namespaces, attributes, multiple "text" nodes, and other complexities of XML. The result is that JSON parsers exist for every language, they're small, and they're fast. Furthermore, if you need to write your own, it's not complicated. The same goes for security -- all that's necessary to prove that a JSON document is valid JSON is a simple regular expression check, which is easily available in nearly every programming environment.

Tools. An increasing number of tools support JSON. JSON support is not ubiquitous yet, but at the rate JSON is gaining it will be soon.

Many APIs now support XML and JSON - like the Twitter API, where JSON is the default. Some APIs support only JSON (like Foursquare's V2 API).

But JSON isn't  for everything. Next up: Why XML isn't dead yet!

Need API and developer adoption?  Try a Hack Day! »

Want to launch your API with a bang?  Or get more internal adoption?

Hack Days (or Hackathons) give developers a day off to build anything they can dream up. No rules. At the end of the day, developers demo for glory and free beer.

Check out the recent FourSquare Hack Day. LinkedIn had a great one. And there are independent Hack Days.

If your API is only available to developers inside the company - even more reason. Why?

- Get the word out - Hack Day is a high profile way to drive adoption and build grass-roots excitement internally.

- Showcase innovation - with great new Hacks and product ideas you'd never have thought of.

- Get those example apps! -  need example apps and code on your developer portal?  Don't DIY or pay someone -use the best Hacks!

- Eat your own dogfood -  Find bugs before releasing your API to partners.

-Get recruits! -   Need good hires for the API team but don't want to ask?

Here are a few tips:

- Dedicate time  - give developers a full day with no distractions.  

- Team up!   Encourage developers to team up and mix skillsets.  

- Demo for glory.  Get your CEO and execs to judge demos at day's end.   The Hack doesn't have to be finished.  Prizes are nice but recognition might be enough.

- Feed me!  Beer and free food can take a OK Hack Day and make it a *great* hack day.

- Start small, but rinse and repeat.  Your first Hack Day might be small and a little rough.  Buy the 3rd or 4th time - you might have Beck playing on your lawn.

- Productize!  Have a labs page or a way to turn great hacks into real product.  

- Provide tools.  Give your developers an API console to learn your new API and debug API problems.

(Thanks to SimpleGeo for the image above)

Two Ton API client:  creating a generation of car hackers »

Aside from curb weight and operating speed - there's not a lot of difference between an iPhone and a fine automobile.  Both are at their most innovative when people drive them with APIs.

Why? See Brian Mulloy's (@landlessness) O'Reilly Detroit Ignite session - "Two Ton API client."  Both a talk track and the slideshare are below. And drive carefully...

Using your existing systems to create new value with APIs »

Companies need to wring value out of their investments; find new ways to serve customers or create new value with their current systems, products, supply chains and partnerships. 

I keep going back to water analogies.  My house has a plumbing infrastructure that was paid for a long time ago.  I recently found new value in that infrastructure, an automatic sprinkler system.  Again the system reminds me of an API.  Through this one common interface I can connect many endpoints (sprinkler heads) and my yard has never looked better.  I have some endpoints that cover the lawn, some that concentrate on shrubs, others take care of the plant beds.  The point is that I tapped in to my existing water pipes and found new value in a 35 year old system. 

Can you do the same with your enterprise systems?

Let's take a common strategy like a mobile device strategy.  First we have to understand what information customers need when using a mobile device like a smartphone.  Certainly location and product come to mind.  Maybe information about their account or orders.  These little gems of information can be exposed through the API interface and carried to many different endpoints; today a smartphone, tomorrow a tablet PC or connected car.  Let's call these 'functional APIs" since they carry a function to the user through a mobile device. 

The next step to realizing a functional API is to map the data elements and functionality of the new API to their location in the existing systems.  These data elements may map to databases, ERP systems, other internal services, etc.  In fact, a single API may map to one or more internal systems.  Any holes that are discovered will have to be filled. 

For example, if you are designing an API and define features or data that aren't available through existing systems, you will have to provide that new system or new data set.  But, in the interest of embracing the legacy systems, you should be able to reuse the majority of what you have by extending it outward through the new abstract API.

Leveraging what you have through an API can reduce your development cost, give you speed to market and also give you more flexibility in the future to integrate with more devices and partners.  Leveraging my existing residential water system yielded a "yard of the month" award, perhaps this strategy can beautify your business as well.

Next we'll talk about the details of integrating with existing systems.   And ots on this in our recent whitepaper "APIs: A Profit Interface - using APIs to grow your business.

Developers Hate Marketing: Launching and marketing your API »

Once you build your API, will developers come?

We've picked up some good stuff from our customers on this topic, many of which we've posted as developer adoption best practices on this blog.

So today we've rolled these in a new whitepaper with our friends at Evolved Media  - Developers Hate Marketing:  Attracting Developers to your API.  

Topics include some thoughts on:

• what do developers expect?
• do's and don'ts in launching your API
• patterns in successful developer programs

You can find it here, we'd love to hear what you think.

Paypal API sandbox console demo interview with @marshacollier »

 Marsha Collier and Sam discuss the API console at the Paypal innovate show this week.  Check it out!

How to learn and explore PayPal’s Adaptive Payments APIs with Apigee Tools »

At PayPal's Innovate 2010 event yesterday, we launched our API Explorer for the PayPal Adaptive Payments APIs, including their new ones. Our own Sam Ramji gave a great demo on stage, but in case you couldn't be there, here's a quick overview of how to get started:

New API Console for Learning and Debugging the PayPal API »

In time for PayPal's Innovate conference, we've added the PayPal API to Apigee's API console. You can now easily view requests and responses to the PayPal API, test and debug, and share your results.

The PayPal API lets developers easily leverage its payment platform in apps, mobile and mashups. The Apigee PayPal console interacts with the PayPal sandbox environment to let you get up and running without needing to transact payments. 

We've made getting started on the API a single-click task by pre-populating custom tokens and sample values for all request parameters, but when you are ready to get your hands dirty, here's a few things to get you going: 

• The Apigee console supports the PayPal API's custom token authentication scheme. The PayPal API uses multiple tokens, so we let you specify them as a custom token so it is easy to stay authenticated--you only need to authenticate once for the entire session.  
•  You can easily configure the request parameters by using the Apigee parameters pop-up. This is designed to support simple experimentation to figure out what you'll need for your apps and integrations. 

Let us know your feedback and check out our other consoles for the Twitter, Facebook, and SimpleGeo APIs. And if you're hanging out at the Innovate conference, ping us on Twitter! We'll be hanging out at the hackathon.

Do you need an SDK for your API? »

In our last couple API strategy workshops, we've had good discussions on a common question for API providers - do you need to offer an SDK (software dev kit) for your API?

In this context we defined an SDK as going beyond offering an API to include platform-specific code that developers use in their apps to invoke API operations, also including the source code, and documentation that developers might need. Here's what we came up with:

Reasons to consider an SDK:

Speed adoption on a specific platform - for example Objective C SDK for iPhone.  Lots of experienced developers are just starting off with objective C+ so an SDK might be helpful.

Simplify integration effort required to work with your API - If key use cases are complex or need to be complemented by standard on-client processing. 

An SDK can help reduce bad or inefficent code that might slow down service for everyone.

As a developer resource - Good SDKs are a forcing function to create good source code examples and documentation.  (See some good examples here and here. )

To market your API to a specific community - you upload the SDK to a samples or plug-in page on the platform's exisitng developer community.

Reasons you might not need an SDK:

If your API is designed for adoption -  standards-based, well documented, etc.. developers may be able to get rolling without a client SDK.

Resources - it's tough to do a SDK for each platform you target, you'll have to pick carefully. Also, it can be time consuming and expensive to create SDKs (tip: maybe packaging up internal samples or test cycles can kickstart things)

Maintenance and versioning - convincing clients to upgrade to the newest version can be rough. Also, capturing the "local flavor" of each language you support can be a challenge.

Complexity -On each platform there might be use cases you don't expect, like keeping application-level secrets off clients, debugging, etc.

More is not necessarily better - fewer, more well-documented, code-level samples can often be the best resource. Facebook provides Android, C#, iPhone, JavaScript, PHP, and Python libraries (all documented differently), but Twitter supplies none. Facebook has more resources but still struggles a bit at times.

Our view: whenever possible, forget the SDK, and spend the time making your API better and more useable, and give good API resources such as example code.  

Agree or disagree? 

Apigee Enterprise 3.8: OAuth 2.0, streaming API, developer keys, and more. »

We're excited to announce the general availability of Apigee Enterprise 3.8!   Some highlights:

OAuth 2.0 – Apigee Enterprise 3.8 is Oauth 2.0, draft version 10, compliant.  If you are running our OAuth 1.0a solution, you can migrate to OAuth 2.0.

Developer Connect - a Drupal-based portal you brand with your look-and-feel and use for publishing API documentation, content, and community management.  Give your developers their own API testing console and other social publishing features such as blogs and forums.   See more developer portal details here. 

Key Manager Service - handle the complete lifecycle for API keys and security tokens (such as OAuth) -  including creation, validation, revocation and expiry of keys and tokens. This service now includes auditing capability and RBAC model for access control.

Streaming API  connections - proxy HTTP responses that use long standing connections. Enable at endpoint level or on-demand, using the context variable.

Distributed Quotas - enforce API usage limits, or quotas,  across cluster nodes. 

Cache and persist API response data on hard-disk, eliminating the dependency between cached data and available RAM. 

REST API for Apigee Analytics - we've updated this to support custom reports with various filters and date ranges. 

Citrix Xen Server 5.5 support -   if you choose to deploy Apigee on-premise, we've added support for another one of the most popular virtualization platforms.

And more..  enhancements to our API Debugging Console, Logging, Load Balancing, Virtual Host and more. 

Enterprise customers should expect us to be in touch soon about 3.8 and all the details are in the release notes - but email us at .(JavaScript must be enabled to view this email address) with any questions!

Check Out Facebook’s New Places API »

How to explore Facebook's new Places APIs in the Apigee Test Console:

Don't have a minute to watch the video? Start by viewing the checkins at the Cookie Jar.

This Week in APIs- July 24-30 »

We might be a little obsessed with APIs here at Apigee- here's what made API fanaticism great this week. 

The Khronos Group released version 4.1 of its OpenGL graphics API, which promises to better support different hardware and software configurations and provide a better 3D graphics experience. It's increasingly important for APIs to address the proliferation of devices and fragmentation of mobile phone OSes, so we're happy to see an open standard making progress- plus, it's 3D! 

Google announced that they will be working with developers on supporting applications with check-in functionality on its Places API, launched this spring. The check-in app space is increasingly competitive and we're excited to see what devs come up with.

Twitter has begun testing a new streaming API that lets desktop apps automatically update their streams in real-time. The User Streams API is being utilized by TweetDeck and Echofon. By transitioning to streaming, Twitter will cut down on rate limit and latency problems- major progress! We're looking forward to seeing this roll out more widely.  

Hot New APIs- 

• Video search engine blinkx unveiled a new mobile API to give access to its huge collection of MP4 videos- including those from the AP and Lonely Planet.
• TypeKit, a font startup, unwrapped a new API that lets developers generate kits from its library and build them into their applications. Very exciting for all you typography geeks! 
• Gamerang is launching a new site to focus on the social side of gaming. As Chris Crum over at WebProNews says, Gamerang is like Netflix for video games- and its new API makes it easy to incorporate the gaming news and social content into sites and apps. 

Did we miss anything? Shoot us an @ sign on Twitter!

This week in APIs-June 27th-July 2nd »

All the best info on APIs this week.

Mozilla's JetPack API Survey: Mozilla is getting ready to release Mozilla 4 beta, and their most anticipated feature is the JetPack SDK. Mozilla did a survey to ask which APIs developers want most. The winner? Page Mods, which allows developers to alter webpages. More browser innovation with APIs is on the way.

Alcatel-Lucent acquires Programmable web: Alcatel-Lucent acquired API news and directory site ProgrammableWeb this week. A smart move and we're exciting to keep seeing great intelligence on the API economy from one of our fav sites.

E*Trade Open API: Online trading giant E*trade just announced their first Open API for third party developers! E*trade is hoping their API will lead to innovative investing applications- we're eager to see what they come up with.

New APIs - Every week a new API is released! Here are three we <3 :

GroopBuy-GroopBuy is a company that hunts down local deals for people. It was launched this year, and they just released their GroopBuy API. Companies that look for local deals and coupons are becoming hot, and developers using their APIs is an important part of that.

Drumbone-Sunlight Labs has just released their Drumbone API for data about legislators, bills, and roll call votes. Just adds to mounting evidence that the government is moving towards Web 2.0!

NakdReality-NakdReality is a company specializing in instant on-location search and augmented reality service. They also just opened up their API for developers, adding to the Geolocation API craze!

API Tip of the Day: Reduce the Learning Curve »

Be consistent when building your API, and try to use standard conventions when possible. This will reduce the learning curve for 3rd party developers.

Hooks, Sockets and Firehoses: Streaming API Technologies Getting Us to REALLY Real-Time »

If you rely on Twitter to see what your friends are having for lunch, waiting 24 seconds to get an update is fine. If you're a stock trader who could have lost a few thousand dollars or more while reading this post, 24 seconds is a very long time to learn that your favorite company just got acquired.

Real-time runs on APIs as much as it runs inside a web browser. This means that the "real-time web" isn't always "real-time"- a web service API is typically a request-response technology, requiring that the client makes a request of the server, waits for a response, and then processes it. This can add critical seconds or milliseconds.

Earlier this month I attended the ReadWriteWeb Real-Time Summit in New York City. In an "unconference" session on streaming API technologies, we discussed 3 technologies bringing us closer to REALLY real time:

HTTP streaming- This technique uses the HTTP protocol, but instead of responding to a request with a single piece of data, the server responds with many small pieces of data that are published as soon as they are available. It's as if you're using HTTP to retrieve a very, very, large file -- the server just keeps pushing out data, one piece at a time. Twitter uses this technique in its Streaming API.

Since HTTP Streaming doesn't work quite the same way that most HTTP-based services work, the program that reads the stream has to be written to process the incoming data, handle reconnects, and so on. It's also difficult to use in a web browser without doing some JavaScript trickery; HTTP Streaming is mainly suitable for server-to-server communication.

WebHooks is a real-time tech with a simple concept: whenever something happens on server A, it makes an HTTP POST to server B and sends it the update directly. Like HTTP Streaming, it only works for server-to-server communications, but unlike HTTP Streaming the barrier to entry is very low for the programmer - anyone who can write a program or script that can process an HTTP POST can consume a WebHook. Unlike HTTP Streaming, however, a WebHook uses one HTTP request for each event, which means that with today's servers a WebHook is going to top out at several thousand events per second, whereas HTTP Streaming can theoretically process many more events.

WebSockets, unlike HTTP streaming and WebHooks, is designed for real-time streaming to the client- useful for creating real-time apps that run in the browser. WebSockets allows a client and server to use the HTTP protocol for some initial interchange, and then agree to communicate bidirectionally. The result is just like a TCP socket -- the client and server can send whatever data they'd like, in any order, in either direction, for as long as they want. WebSockets also supports JavaScript, ensuring you can easily build a high-fidelity, near-real-time application in a web browser using Internet-standard protocols. WebSockets is part of the HTML 5 family of standards, already implemented in Google Chrome and Apple Safari, and will eventually come to all the browsers.<;It was good to see Kaazing, who is leading the development of WebSockets, hanging out at the RRW Real Time Summit. With some existing technology out there and some new stuff on the way, the stage is set to build a real-time web that's a bit closer to real-time.

With some existing technology out there and some new stuff on the way, the stage is set to build a real time web that's a bit closer to real time.

This Week in APIs- June 7-11, 2010 »

Check out the top news in APIs this week!

S3 Import/Export API- Amazon Web Services added an Import/Export API for its Simple Storage Service (S3), an infrastructure service for storing data. The new API makes it easier to move large amounts of data in and out of Amazon- a great move for data portability.

Twitter API Meetup- Twitter held a meetup in its San Francisco offices this week to talk about the Twitter API, showcase some cool apps and celebrate Apple's WWDC. Twitter's Taylor Singletary presented on XAuth, a new authentication method for APIs. Twitter will no longer allow support for basic authentication at the end of the month so Twitter devs, it's time to brush up on the new method! The meetup made it very clear that the XAuth standard was developed to be used across the API community and promises to make authentication safer and easier, so keep an eye out for adoption.

APIs in the world of PaaS- Over at CloudAve, Krishnan Subramanian has a post and some love for Apigee on why APIs are the oxygen of Platform as a Service offerings- check out the article for some stark examples of how APIs can mean the difference between success and failure.

New APIs! A number of companies debuted new APIs- here are a few we thought were especially cool:

• HubSpot, which launched a Lead API to connect HubSpot's lead capture tools to other CRM software. Excellent example of how APIs can be used to connect tools and unleash new functionality. 
• The New York Times introduced a Most Popular API to track the most viewed and shared content- fodder for mashups here! 
• Microsoft introduced its Trial API that allows developers to include free trial versions of their apps in the Phone 7 App Market- more signs that the mobile API world is maturing. 
• As always, ProgrammableWeb's Newest APIs list is a great way to keep on the top of the all the new delicious APIs out there. 

Anything important we missed this week in APIs? Shoot me an email at .(JavaScript must be enabled to view this email address).

API Virtualization »

We've seen an architectural pattern emerge over the last two years that deals with the expanding variety of computing devices and the exploding number of APIs used in modern applications. This pattern is API Virtualization - applying a virtual layer above the API itself to deal with the different concerns that come into play when delivering an API to different device and machine endpoints as well as across a range of different classes of business relationships.

I gave this presentation at GlueCon as a Prezi. The prezi is available here, and I've put the audio with it in the screencast below.

One of the big shifts that has occurred in application development is the move from libraries to APIs for use of pre-written code. APIs enable access to not just logic but data and the the computational power required to render that data appropriately. The Facebook and Twitter APIs are good examples of this, but there are many more including Paypal's X.com, Ebay, Amazon, Netflix, Sears, Blockbuster, Best Buy, MTV and Google.

While applications like Siri which use 20 or more web APIs are still unusual, most applications use several APIs to deal with access to content, social networks, events, identification, payment, and other concerns. These APIs change frequently, as do the applications that consume them, resulting in a web of dependencies and causing chaos in the development lifecycle.

API Virtualization enables these different concerns and dependencies to be isolated from the application or API provider, resulting in a cleaner development process and higher-performing applications.

Where will your customers be this Christmas? »

Keith Morrow has a great article on TechTarget series on how an API can create a disruptive impact on a retailer's ability to reach customers beyond traditional web browser channels.

One key point is that an API can be driven by a relatively small team. 

Yet an API has a disproportionate impact on the business, and this impact scales with low marginal costs.  

This can be an especially effective strategy for retailers that might live in world of thin margins, limited resources, and the constant pressure of the peak holiday season. 

For more on Keith's retail API experience, we commissioned him to write a Retail 2.0 API Strategy whitepaper discussing Retail API opportunities, challenges of launching with tight resources, and best practices.

You can download the whitepaper here.

WWSJD- What Would Steve Jobs Do? Lessons for the Age of APIs »

API Scalability: Cache large ‘chunks’ in the API response »

Highscalability.com has an interesting piece on caching as "secret sauce" for good web performance.

Facebook and Twitter are cache assembly lines -- every web page and API request is served up by many calls to various caches at different levels - assembling the final result from many different chunks. At this scale there is almost no other way to deliver reasonable performance.
 
For APIs - what's the largest chunk of all? The entire API response.

APIs lend themselves nicely to caching responses because it is often easy to identify the cache key. If you follow the REST pattern each URL maps uniquely to a resource that has a well-defined lifecycle.

If you're working on the next Twitter, you will need many layers of caching to deliver great performance, including the filesystem and database caches, and then you will add additional caching layers using products like memcached or Coherence.  But a final tier of caching for the API responses themselves can only help performance, so consider:

• Can API analytics data show the most common or slowest API calls?
• Are there slow API calls that return the same data over and over?
• Can you design using a REST pattern to make it easy to identify individual cache items?

There are a number of ways to speed up calls that are caching candidates, such as memcached or something similar.

Another way is to use an API proxy. A proxy that is sitting in front of your API servers is an efficient and easy way to add an additional caching layer without making any changes to the server tier. We have helped a number of customers drastically improve the performance of their APIs by inserting such a caching tier without touching the clients or servers. 

And caching isn't just helpful if you are building an API. Applications that consume APIs, whether they are running on another web server or on a smartphone, can set up their own API proxy so that responses from many devices are cached in a central place, even if the APIs that the application is depending on can't be depended on themselves to return good performance.

(thanks to jules:stone for the photo!)

A Look at HipChat’s API »

While Twitter and other web 2.0 applications have changed public communications, companies have been clamoring for similar messaging systems for internal use. Used by over 1,000 companies and teams, HipChat is a great solution with deep features to help you supercharge your company's communications. You can create chat rooms for meetings, use it on your mobile phone, have a one-on-one chat with anybody in your company, and securely share files. HipChat recently released their very own API, which is well-designed and documented. Here's a quick demo of what makes HipChat great:

Since Garret Heaton of HipChat uses Apigee, we asked him to answer 5 questions for us:

What do you wish your API would do that it doesn't do today?

Garret:Obviously it'd be nice to have support for all the features users have access to when using HipChat: changing room topics, sending invites, uploading files, creating rooms, etc. It'd also be awesome to have a streaming API (and open up full access via XMPP) so users could build bots and other real-time services.

If you could send one message about building your API back in time to yourself, what would it be?

Garret:Keep it simple! The first versions of our API had a more complex interface for passing auth information, desired response format, and entity IDs. Our initial testers were often confused. It's important for new users to be able to test the API easily using cURL.

What insights into your app or other benefits have you gotten from working with Apigee?

Garret:The 'Response Time' data is really helpful for identifying slower requests. It's really nice not having to build this ourselves. The new debug console is also incredibly helpful when you need to see what your exact request/response data looks like.

What new Apigee feature would you most like to see from Apigee?

Garret:I know you're already working on it, but HTTPS support for mapped domains so our users can use HTTPS [NB: It's coming very soon, we promise!]. Also, a way to create a test console for our API. I think our users would find that very helpful.

How would you like to see developers make use of your API once it is exposed?

Garret:It'd be awesome to have plugins for sending HipChat messages available for lots of other software: Capistrano, Nagios, Subversion, Git, Perforce, Trac, Get Satisfaction, CoTweet, and many other cloud-based tools companies are using these days. We're also looking forward to seeing the creative things people will build that we haven't even thought of yet!

If you're looking for a great collaboration tool to improve your internal communications, be sure to check out HipChat. And keep telling us what Apigee features you'd like to see next so we can provide a better service for you!

API Versioning Hell? Think API Virtualization »

We've seen an architectural pattern emerge over the last two years that deals with the expanding variety of computing devices and the exploding number of APIs used in modern applications.  This pattern is API Virtualization - applying a virtual layer above the API itself to deal with the different concerns that come into play when delivering an API to different device and machine endpoints as well as across a range of different classes of business relationships.

Sam Ramji gave this presentation at GlueCon as a Prezi.  The prezi is available here, and we've put the audio with it in the screencast below.

One of the big shifts that has occurred in application development is the move from libraries to APIs for use of pre-written code.   APIs enable access to not just logic but data and the the computational power required to render that data appropriately.  The Facebook and Twitter APIs are good examples of this, but there are many more including Paypal's X.com, Ebay, Amazon, Netflix, Sears, Blockbuster, Best Buy, MTV and Google. 

While applications like Siri which use 20 or more web APIs are still unusual, most applications use several APIs to deal with access to content, social networks, events, identification, payment, and other concerns.  These APIs change frequently, as do the applications that consume them, resulting in a web of dependencies and causing chaos in the development lifecycle.

API Virtualization enables these different concerns and dependencies to be isolated from the application or API provider, resulting in a cleaner development process and higher-performing applications.

Vzaar: A Platform for Merchandising Video »

We all know YouTube is a famous platform for sharing videos. But the advertising that supports YouTube isn't always wanted when you, yourself, are a business who wants to be taken seriously. This is a problem that vzaar solves—hosting professional content and videos for businesses. They provide a professional video platform that allows you to serve high quality videos for your web applications, and many other businesses. Creating videos for your business is a great way to engage with customers and can be an important revenue generating tool. The best part about vzaar is the control they provide, like how your video gets presented and where. They also have a well-documented API that you can use to plug videos right into your application.

Since the CTO of Vzaar, Adrian Sevitz ,uses Apigee, we asked him to share some of his experiences.

 What insights into your app or other benefits have you gotten from working with Apigee?

 Adrian: Both insight into which calls are being accessed, and what the popular API calls are as well as how many hack attempts we get looking for open urls and holes.

 What new Apigee feature would you most like to see from Apigee?

 Adrian: Two things. The first is the ability to limit users by number of calls so we can sell different access limits. The second is to white or blacklist ulrs being access to provide a layer of protection at the api level.

 What mashup, app, or API do you admire?

 Adrian: App wise we love postmark for our emails and spreedly for our billing. Both of them make our life easier. And what could be better than that?

 When your trying to give your business the right impression on a video platform, vzaar is a great way to push your product or service out! Also we want to hear your ideas on www.apigee.com/support so we can provide a better service for all of our users.

Darwin’s Finches, 20th Century Business, and APIs: Evolve Your Business Model »

What do APIs have in comon with Darwin's observations on evolution, the 20th century garment district, and the Kobayashi Maru?

Sam Ramji makes the case for APIs in his much written about web 2.0 talk  - watch and listen to the full talk or just flip through the slides - both below.

Retailer Markets Moving to Open APIs »

When one of the largest, most successful retailers on the planet makes a move, you can definitely consider it a confirmation of a trend.

Last week, the WSJ reported that Wal-Mart is opening smaller-format stores in new urban locations.  Wal-Mart CEO Mike Duke stated in their annual report that their growth will be driven by “innovated new formats”, which includes the smaller stores and stores with drive-throughs for picking up internet purchases.

Essentially, Wal-Mart is going to where their customers are…. taking their brands to different chnnels where customers spend their time, to be there when those customers want to buy.

What does that have to do with APIs?  Everything.

Retailers as diverse as Sears, Netflix, and BestBuy are opening APIs to take their brands and their services to where their customers spend time.  Those customers are on Facebook, on Twitter, on enthusiast sites, on their iPhones, iPads, Xboxes, etc. 

As the image below from Sam's Web 2.0 talk depicts - you may have to connect to customers across many diverse channels, from smartphones to Xboxes to your car and kiosks.)

Getting your capabilities into these environments where customers are spending time requires that you fit into new formats… like an iPhone or Android app, or an iPad or Xbox app.  You can’t possibly be an expert in the dozens (hundreds?) of platforms, sites, and devices out there.  Developers can help you do that, if you have your catalogs and services exposed via web APIs.

By the way, we have a new whitepaper out on Retail 2.0 today, you can get a sneak preview here.

Working over vendors:  A buyers guide for API management »

The explosion of APIs is driving companies large and small to launch APIs for their customers and partners.  But both public and private APIs require management features well beyond their internal functions.    How should you approach the 'build vs. buy' analysis and evaluating API management solutions and vendors?

As a former CTO that has recently gone through this process, here are some lessons learned on how to scope out requirements, evaluate and select a solution.

Know the problem (and requirements) you are solving for

Many API product managers are initially tasked to spec out only a small slice of the problem (such as "get me a developer portal"  or "we need developer analytics."  ) Your requirements landscape is likely much wider than this.  Make sure your PM thinks about business requirements that have technology implications in areas such as:

1.    Security, Access Control - Authentication methods
2.    Reporting, Analytics -Usage by developer, organization, API, location
3.    Threat Protection - Denial of service attacks
4.    Traffic Management - Routing needs for versioning, operations management
5.    Mediation, translation - Protocol bridge between SOAP, REST, JMS etc
6.    Performance, Scale -     Clustering, caching
7.    Developer support-     Access key management, FAQs, wiki
8.    Operational support - Root cause analysis, logging

(For more on these topics, see this blog series)

Get everyone involved, early.

Because APIs touch partner relationships, they are critical to the business, and there are multiple stakeholders in your organization.  Walk through the lifecycle of an API, what roles support the release of an API, external access, operational support and management. These stakeholders should be represented in the overall product management strategy and accounted for in your requirements for API management.

Considering a vendor? Demand a POC!

The requirements, operational view and deployment strategy will provide an outline of scope for build vs. buy considerations. All of these functions should be outside of the core elements of the APIs themselves to ensure consistency without duplicating efforts for key policy areas that are needed for your APIs.

If you engage an API management technology vendor, these requirements can be prioritized and used to drive a demo of a solution to ensure the capabilities meet your needs. Insist on a POC (proof of concept) for a deeper dive and validation of more technical requirements.  This exercise will quickly allow you to determine commercial solutions that have the potential to meet your needs and provide a concrete plan for demonstrating capabilities that are of material value to your organization.

A POC will tell you a lot about a vendor -  both what they would be like to work with on a day by day basis and how they meet their commitments.  A POC will also tell you how well your own team has thought out what they need and how prepared they are to execute it.    Eliminate vendors that will not step up to a POC.

Also,  make sure your own team goes back and can repro the POC -  without the vendor.     

This diagram below (and a larger version here)  shows how we thought about our POC process.

Revisit the priorities and rollout strategy for API management

This evaluation process should expedite the short list of vendor options that will satisfy your needs and as well as gain internal consensus on what API management means to your organization. You may find the depth and breadth of your requirements will motivate you to find a commercial solution rather than build your own system that will have to be maintained in addition to your core APIs.  

A key consideration for the solution you select is handing ongoing changes after this system is deployed. Once your APIs are operational you’ll need a way to manage updates to numerous policy areas on a regular basis and most likely you will want to avoid any downtime to implement those changes.   For example, my previous role, we did an cost analysis of how many independent API codebase versions we could support at one time.  Our answer?   Two.   This drove us to select an API management technology to  provide a versioning and mediation layer above the API layer.

It is important to review the flexibility of the system to avoid limiting your API management capabilities and practices to a particular solution.

Operational awareness

Policy enforcement is typically the initial area focus for API management but operational visibility is also a key area to focus to include as well. How will you report on usage and do you have a consistent way to collect data to be aggregated? Do you have a common root cause analysis process to enable message level tracing and logging regardless of the API?

Conclusion

The topics around API management are numerous and not unlike the questions that were contemplated during web browser adoption. With a new delivery model of information and services there are multiple stakeholders to consider during a ongoing lifecycle of development and operations.  Developing an understanding of your needs up front will greatly streamline your selection process as well as get a common understanding of what API management entails within your organization.

Darwin’s Finches, 20th Century Business, and Open APIs: Evolve your Business Model »

Today @sramji was invited to speak on API business models and strategy at the Web 2.0 Expo in San Francisco.  

Thanks to our customers and many others for these API best practices and ideas.

Don’t try to market to developers.  Instead, solve their problems. »

Not long ago you could count the number of 'developer marketing' programs on one hand.  Now there are hundreds of programs as Web companies and enterprises open APIs.   These companies know that developer adoption will make their API strategy succeed or fail.

But Developer Marketing is an oxymoron.  Developers hate marketing.   

You cannot drive adoption by 'marketing to developers.'  Sure, you can send offers to your developers but your mileage may vary.

A better formula - understand what's important to developers and give them what they need to reach these goals. Developers want to:

•  build new skills that lead to the best projects and jobs.  This is why new or proprietary tools and programming models are tough to get off the ground - it's a small market of new projects for the developer.
•  increase their productivity.  With good tools and by connecting developers with decent resources and each other for help.  This is why sites like StackOverflow take off. 
•  be recognized for good work and see their products used.  Focus on showcasing their work, not your product.  It's not about you.
•  get paid.  Think App Store model, or affilate marketing networks.

Talk to the folks that made the big developer networks sucessful and you'll hear these points over and over. Some others:

• Developers are not buyers, but are very strong influencers.  There are superstars in the developer world - make them fans and that is the best marketing you'll ever get.
• You can't 'own' or 'use' developers because they have an account on your service.  Developers have lots of options and switching costs might be low from your API.
• Act on their feedback.  Developers are smart and listening and acting on their complaints and ideas is critical to your credibility.
• Developer communities are fragmented.  For example, there is no such thing as an "API developer', but instead there are Twitter or Facebook or Salesforce developers.

Once you have attracted a developer to use your service - they are like gold.   So treat them with respect - don't try to 'use' developers or you might lose them!

Apigee API Contest Winner at iPad Developers Camp - Netflix Actors »

The winners of the Apigee sponsored prize at the iPad developers camp was the "Netflix Actors" application. Check out a quick demo of the application below.

Sam Ramji discusses Apigee’s new Twitter API Tools with Robert Scoble. »

Robert Scoble of Rackspace caught up with Sam Ramji to discuss the latest release of Apigee and all the new Twitter centric features. Check out the full interview below!

OAuth is improving, but still moving »

We've been following the fast-moving debate in the IETF regarding OAuth 2.0.  OAuth, for those of you who have not encountered it already, is a set of authentication technologies for the Internet designed around the concept of an access token.

Access tokens, in the words of Eran Hammer-Lahav, are like valet keys -- they give the holder access to a specific function, for a specific amount of time. For instance, you might use OAuth to give another web site the ability to read photos from your Flickr profile, but not to modify them. OAuth lets you do this, it lets you go back to Flickr and revoke the web site's permissions at any time, and it does it without requiring that you give the site your Flickr password.
 
The current spec, OAuth 1.0a, is implemented in lots of places, and it solves a lot of problems. However, implementing it is no picnic for either the API provider (the server) or for the developer who builds the client. (There are libraries, of course, not to mention products such as our own that simplify this process.)
 
OAuth 2.0 introduces many changes. The most important is that a client may now use a "bearer token." That's a fancy IETF way of saying that an access token can just be a string that the server gives you. On every request, the client passes that token back to the server, the server checks to see if the token is valid, and you're done. This is much simpler to implement than OAuth 1.0a, but it is only secure if you use SSL for every request. Applications that won't or can't use SSL may still use the old way of transmitting each token, which encrypts the token so that it is safe even if SSL is not used or even if it is intercepted by a proxy like Apigee.
 
However, OAuth 2.0 is far from complete. It is currently undergoing lots of discussion on the IETF mailing list, and the spec draft changes daily.
 
That's why I was surprised to read today that Facebook is using OAuth 2.0 to authenticate its own API. Now, some of the key players in OAuth work at  Facebook, and they have chosen to use only a part of the spec, and the part that's arguably the least complicated. I'm sure that they feel that taking this calculated risk now is in the best interest of Facebook and its developer community, but the possibility remains that the spec will change and Facebook will have to change its implementation to match.

(In fact, at the moment I write this, they do not -- the name of the query parameter that holds the token is "access_token" in the Facebook documentation and "oauth_token" in the latest version of the spec repository.)
 
In the meantime, developers building on top of these APIs may have to contend with OAuth 1.0a (the current spec), OAuth 1.0 (an older version that some sites may still use), the draft form of OAuth 2.0 as implemented by Facebook, and even "WRAP," which introduced some of the concepts used in OAuth 2.0.
 
So the good news is that there are lot of good standards being written that can make it easier to produce and consume powerful and secure APIs. The bad news is that those standards are still changing. So stay tuned, and be careful!

Right product at the right time:  API Product Management »

Recently we were asked by a SaaS company exec "can't we just hire someone to come in here and build our API for us?"

Danger, Will Robinson.  Just like any other product in your stable, your API needs to go through your product management practice.  Successful APIs usually have a dedicated API product manager that creates the 'right product at the right time" by continually focusing helping the team stay on target by driving:

1. What is the vision for the API?      How do you go from an idea to great product?  Start by asking what is your vision?  if you were sitting around with your top 5 execs..would they agree?   One good PM framework we've seen really focus an effort is explicitly defining the "VMSO" or "Vision, Mission, Strategy, Objectives" before every major release.  For example:

• Vision - what is "the dream" (example: be the most widely used widget catalog on the planet)
• MIssion - what do we do every day to achieve the dream? (example: have the easiest catalog API to learn and use)
• Strategy - what is our unique approach for achieving our mission? (example: have the smoothest sign up, clearest REST API and best community support)
• Objectives - what are our 1-3 key API metrics to determine if the strategy is working? (example: developer apps, API transactions, API revenue)

2. What is the target customer segment for the API?  Mobile developers?  Your top 10 partners? Affliate marketers? Each segments may need different features, policies, or marketing approaches. Do a customer or developer segmentation analysis and force rank priority segments. 

And if you ask your API team who their target segment is and the answer is 'everybody' - get worried.

3. Develop use cases.  Ask how little, not how much, you can launch with your API.  Taking back functionality is difficult once it's out there.  Identify and prioritize the minimum set of use cases (or user scenarios - such as 'browse catalog information')  and consider throwing out anything outside what's needed for each use case.

4.  Iterate quickly.  It's rare to find a successful API program where the PM doesn't say something like "and after we launched our customers took us in a completely different direction." Consider agile development techniques to help your team iterate quickly. 

5.  Differentiate your API. How is your API or content different than competing APIs in your vertical?  Why should I drop what I'm doing now and use your API?  Using a well-worn PM 'positioning framework' can help the team agree on this beforehand. For example:

What other PM processes do you recommend?

(and thanks to respres for the great photo.)

@sramji demos @apigee @twitterapi features at #chirp »

Sam demoed Apigee's new Twitter API test and debug console at the Twitter Chirp conference yesterday.

There's both a video and the screenr screencast for a clearer view of the screen below. Enjoy!

Introducing new Apigee features for API testing and debugging »

Our Apigee team is delighted to announce powerful new features for API developers for the Twitter's Chirp conference.

Today, Twitter API developers have a new API Testing console that provides a way to explore and share Twitter API testing results via URL with other developers (see and example here).

And the new API Debugger gives developers API message capture, record, and playback for any API that is proxied with Apigee.  You can share your debug snapshots with other developers via URL.

A couple other changes:  we've removed our 'private preview' invitation requirement - anybody can sign up and get started now.   And we've increased our free traffic limit from 10K to 50K per hour.

Finally, if you've used Apigee before, check out our new proxy creation flow.

For a detailed perspective, check out the Apigee blog for more on the evolution of Apigee.

Beyond the Browser: The API web »

Maybe it was because I was reading it on my new iPad, but really enjoyed this post by John Doerr and partners on TechCrunch last week.

Was struck by the phraseology 'beyond the browser' and that we're quickly moving " beyond web sites limited by browsers… to interactive, connected applications with incredible simplicity, speed, and fluidity.... that will transform everything." 

Couldn't agree more, especially when we consider the next 5-10 billion devices sold (via Mary Meeker's amazing Web 2.0 deck below, slide 32) will not be PCs with browsers but connected devices - from the iPad to your Xbox or Tivo that don't use browsers but instead connect to content providers with APIs.

We are moving beyond the browser to the "API web", where much of your company’s web traffic will come from a non-browser source.  Enabling the API web is our mission.

 

Moving the needle: Example API metrics »

It's an old cliche, but it's been said that you can't move the