HATEOAS 101: Introduction to a REST API Style (video & slides) »

Thanks to all who participated in the HATEOAS 101 Webinar this week. 

The video (~30 min.) and slides are below. Check them out for an introduction to the core principles, examples, and a look at the value of the approach for API providers and app developers. Thanks @landlessness.

As always, we'd love more of your thoughts, insights, or questions on the api-craft forum.

Announcement: Free eBook on Web API Design »

We wrote an API design book! In 2009, a couple of my fellow Apigeeks (@earth2marsh & @gbrail) and I were frustrated with how many bad, inconsistent Web APIs we encountered in the world. So, we started cataloging the bad (and good) things we saw people doing to other people through API design. After three years of discussions, presentations and implementations with technologists all over the world (online & in real life), we are happy to announce the beta version of a new eBook.

Web API Design: Crafting Interfaces that Developers Love 

Although I did my best to chase down errors, there are likely mistakes and bugs in the book. Plus, it's shocking how poorly code and eBook publishing get along. Despite some blemishes though I think you'll enjoy kicking back and reading some of the world's best thinking about API design.

Big thanks to all the freelancers, corporate geeks, open source contributors, bloggers and doers who helped craft our thinking about API design. Please enjoy the book and please let us hear any feedback: @landlessness or .(JavaScript must be enabled to view this email address)

Armrest & REST API Design »

Last week I flew from Doha, Qatar to Brussels, Belgium. The fella in the aisle seat fell asleep before takeoff. The road warrior code of chivalry indicates one should never wake a fellow traveler.

So I stayed in my window seat for 8 hours. After a few minutes it became obvious that my armrest (singular, my neighbor had usurped the other one) was uncomfortable.

Here's a picture of the armrest.



The armrest has many problems. The biggest: you can't rest your arm on it.

If you try to relax, your elbow gets sucked by gravity into the cutout and pinched unpleasantly by hard plastic. If you deal with the pain and fall asleep you will wake to find your ears exploding because you inadvertently cranked up the volume, the flight attendant will be tapping your shoulder asking why you paged him and your reading light will be shining in your dilated eyes.

The Airbus A330-200 in which I was traveling holds 250 people. That's 2,000 man hours of discomfort per flight injected into the universe because of bad design.

So, why does such an obviously bad, pain-inflicting design make it into the world?

Because it cost-effectively solves other, non-customer-related problems! In this case, it is a clever way to store the remote controller while still keeping it accessible to passengers. It’s also done in a way that doesn't require an expensive reworking of the interior: cut off part of the armrest, put a hole in it and reattach on a hinge. Done! In-air multimedia problem solved!

What does this have to do with REST APIs?

Even with the best of intentions, API teams often stop short of doing the right things for their customers. Customers of APIs are application developers. Application developers are smart, busy, and born to suffer (especially Objective-C developers, NSThatsADifferentTopic). But they are also often musicians, gamers and aesthetes. They are attracted to things like truth and beauty. And they are attracted to good, self-consistent API design.

So, why do API teams often stop short of delighting developers? Because the time and cost of learning to do the right things and getting them done often seems prohibitive! At Apigee, we share everything we learn on our blog, twitter @Apigee, YouTube, API Craft on Google Group, and in our webinars.

There's no reason to walk the road to creating a fantastic API alone. If you know of an API or an API team that's not doing things right or is trying to go it alone, please send them to us. We'll get them straightened out.

RESTful API Design: API Virtualization »

Last time, we looked at why you might consider complementing your API with an SDK or code libraries. In the series so far, we've covered a lot of tips and tricks for designing pragmatic RESTful APIs. 

You may be asking -  

How do I follow all these best practice guidelines and still maintain and iterate my APIs? 

What should I be thinking from an architectural perspective in terms of implementing these best practices?

Add an API virtualization layer 

I recommend you give yourself a buffer or virtual layer between the interface on top and the API implementation on the bottom.

The app that consumes the API is on top. The API virtualization layer isolates the application and the API. Make a clean design for the app on top and turn the problem into an integration problem.

 In other words, work at integrating your design on top of the key APIs through the virtualization layer.

Don't start in code, and try to build up from your business logic to a clean API interface.

RESTful API Design: complement with an SDK »

So far in the API design series, I've looked at best practices for designing pragmatic RESTful APIs.

This time, I'll talk about complementing APIs with code libraries and SDKs.

What to do when building a UI requires a lot of domain knowledge?

This image is an example of a twitter app displayed in a newsroom.

A developer created this cool visualization - a tweet referencing another twitter user.

The Twitter API is one of the simplest app domains imaginable: the primary object is 140 characters of text. Yet, the developer who created this needed quite a bit of domain knowledge - they needed to parse the tweet, color code it, and display it in a certain way.

The API team shouldn't try to solve presentation problems with the API. That's what code libraries do. Instead

Complement your API with code libraries and a software development kit (SDK)

That is, don't overburden your API design. A lot of what's needed in scenarios like this is on the client side and you can push that burden to an SDK.

The SDK can provide the platform-specific code that developers use in their apps to invoke API operations - meaning you keep your API clean.

Other reasons you might consider complementing your API with an SDK

• Speed adoption on a specific platform - for example Objective C SDK for iPhone. Lots of experienced developers are just starting off with objective C+ so an SDK might be helpful.
• Simplify integration effort required to work with your API - If key use cases are complex or need to be complemented by standard on-client processing.
• An SDK can help reduce bad or inefficient code that might slow down service for everyone.
• As a developer resource - Good SDKs are a forcing function to create good source code examples and documentation. Both Yahoo! and Paypal have good examples.
• To market your API to a specific community - you upload the SDK to a samples or plug-in page on the platform's existing developer community.

Next time: API Virtualization

RESTful API Design: chatty APIs »

In this post in the series, let's think about how app developers use that API you're designing and dealing with chatty APIs.

Imagine how developers will use your API

When designing your API and resources, try to imagine how  developers will use it to say construct a user interface, an iPhone app, or many other apps.

Some API designs become very chatty - meaning just to build a simple UI or app, you have dozens or hundreds of API calls back to the server.

The API team can sometimes decide not to deal with creating a nice, resource-oriented RESTful APIs, and just fall back to a mode where they create the 3 or 4 Java-style getter and setter methods they know they need to power a particular user interface.

I don't recommend this. You can design a RESTful API and still mitigate the chattiness.

Be complete and RESTful and provide shortcuts

First design your API and its resources according to pragmatic RESTful design principles and then provide shortcuts.

What kind of shortcut?

Say you know that 80% of all your apps are going to need some sort of composite response, then build the kind of request that gives them what they need.

Just don't do the latter instead of the former. First design using good pragmatic RESTful principles!

Take advantage of the partial response syntax

The partial response syntax I discussed in a previous post can help.

To avoid creating one-off base URLs, you can use the partial response syntax to drill down to dependent and associated resources.

In the case of our dogs API, the dogs have association with owners, who in turn have associations with veterinarians, and so on. Keep nesting the partial response syntax to get back just the information you need.

    /owners/5678?fields=name, dogs(name)

Next, SDKs and code libraries.

RESTful API Design: making requests »

We've covered singular vs. plural nouns to label your resources, tips for search, handling errors, and more.

Now lets take a look at what some API requests and responses look like for our dogs API.

Create a brown dog named Al

POST /dogs
name=Al&furColor=brown
Response
200 OK

{
"dog":{
"id:"1234",
"name": "Al",
"furColor": "brown"
}
} 

Rename Al to Rover - Update

PUT /dogs/1234
name=Rover
Response
200 OK

{
"dog":{
"id:"1234",
"name": "Rover",
"furColor": "brown"

}
} 

Tell me about a particular dog

GET /dogs/1234

Response
200 OK

{
"dog":{
"id:"1234",
"name": "Rover",
"furColor": "brown"
}
}


Tell me about all the dogs

GET /dogs
Response
200 OK

{
"dogs":
[{"dog:{
"id:"1233",
"name": "Fido",
"furColor": "white"}},
{"dog:{
"id:"1234",
"name": "Rover",
"furColor": "brown"}}]
}

 Delete Rover :-(

DELETE /dogs/1234
Response
200 OK

Next time: chatty APIs.

RESTful API Design: how many versions? »

RESTful API Design: authentication »

This time, in this series about pragmatic RESTful API Design, I'll discuss authentication.

There are many schools of thought - my colleagues at Apigee and I don't always agree on how to handle authentication - but overall here's my take.

Let's look at these three top services. See how each of these services handles things differently:

PayPal

Permissions Service API

Facebook

OAuth 2.0

Twitter

OAuth 1.0a

Note that PayPal's proprietary three-legged permissions API  was in place long before OAuth was conceived.

What should you do?

Use the latest and greatest OAuth - OAuth 2.0 (as of this writing)

Don't do something *like* OAuth, but different
It will be frustrating to app developers if they can't use an OAuth library in their language because of your variation.

Next time: Versions - how many? Meanwhile, I'd love to hear from you over on the API Craft Google Group.

RESTful API Design: tips for handling exceptional behavior »

In the last post in this series, I talked about the top level domain (the stuff on the other side of the URL) and in this RESTful API design series so far, I've dealt with baseline, standard behaviors.

This time I'll explore some of the exceptions that can happen - when clients of Web APIs can't handle all the things we've discussed.

For example, sometimes clients intercept HTTP error codes, or support limited HTTP methods.

What are ways to handle these situations and work within the limitations of a specific client?

Client intercepts HTTP error code

One common thing in some versions of Adobe Flash - if you send an HTTP response that is anything other than HTTP 200 OK, the Flash container intercepts that response and puts the error code in front of the end user of the app.

Therefore, the app developer doesn't have an opportunity to intercept the error code. App developers need the API to support this in some way.

Twitter does an excellent job of handling this.

They have an optional  parameter suppress_response_codes. If set to true, the HTTP response is always 200.

/public_timelines.json?
suppress_response_codes=true
HTTP status code: 200{"error" : "Could not authenticate you."}

Notice that this parameter is a big verbose response code. (They could have used something like src, but they opted to be verbose.)

This is important because when you look at the URL, you need to see that the response codes are being suppressed as it has huge implications about how apps are going to respond to it.

Overall recommendations:

1 - Use suppress_response_codes = true

2 - The HTTP code is no longer just for the code

Our rules from earlier (what about errors?) change. In this context, the HTTP code is no longer just for the code - the program - it's now to be ignored. Client apps are never going to be checking the HTTP status code as it is always the same.

3 - Push any response code that we would have put in the HTTP response down into the response message

In my example below, the response code is 401 - see it in the response message.
Also include additional error codes and verbose information in that message.

Always return OK
/dogs?suppress_response_codes = true 

Code for  ignoring
200 - OK 

Message for people & code
{response_code" : "401", "message" : "Verbose, plain language description of the problem with hints about how to fix it."
"more_info" : "http://dev.tecachdogrest.com/errors/12345", "code" : 12345}

Client supports limited HTTP methods

It is common to see support for GET and POST and not PUT and DELETE.

To maintain the integrity of the four HTTP methods, I suggest you use the following methodology commonly used by Ruby on Rails developers:

Make the method an optional parameter in the URL.

Then the HTTP verb is always a GET but the developer can express rich HTTP verbs and still maintain a RESTful clean API.

Create
/dogs?method=post

Read
/dogs

Update
/dogs/1234?method=put&location=park

Delete
/dogs/1234?method=delete

WARNING:

It can be very dangerous to provide post or delete capabailities using a GET method because if the URL is in a web page then a web crawler like the Googlebot can create or destroy lots of content inadvertently. Be sure you understand the implications of supporting this approach for your applications' context.

Next time: authentication

RESTful API Design: consolidate API requests in one subdomain »

In the series so far, we've talked about everything that comes after the top level domain. This time, let's explore stuff on the other side of the URL.

Here's how Facebook, Foursquare, and Twitter handle this: 

Facebook provides two APIs.

They started with api.facebook.com, then modified it to orient around the social graph graph.facebook.com  

Foursquare has one API. 

Twitter has three APIs; two of them focused on search and streaming.

It's easy to understand how Facebook and Twitter ended up with more than one API.

It has a lot to do with timing and acquisition, and it's easy to reconfigure a CName entry in your DNS to point requests to different clusters.

But if we're making design decisions about what's in the best interest of app developer, I recommend following Foursquare's lead:

Consolidate all API requests under one API subdomain.

It's cleaner, easier and more intuitive for developers who you want to build cool apps using your API.

Facebook, Foursquare, and Twitter also all have dedicated developer portals.

developers.facebook.com
developers.foursquare.com
dev.twitter.com

How to organize all of this? 

Your API gateway should be the top-level domain

api.teachdogrest.com 

Your developer portal should be (in keeping with spirit of REST) developers.yourtopleveldomain.

developers.teachdogrest.com 

Do Web redirects

Then optionally, if you can sense from requests coming in from browser where the developer really needs to go, you can redirect.

Say a developer types api.teachdogrest.com in the browser but there's no other information for the GET request, you can probably safely redirect to your developer portal and help get the developer where he really needs to be.

• api -> developers (if from browser)
• dev -> developers
• developer -> developers

Next time: Exceptional behavior!

RESTful API Design: what about counts? »

What's the right way to do counts?

This question comes up a lot. This post talks about just one interesting way to approach it.

When I first kicked off the conversation in a recent RESTful API Design webinar, I asked developers to chime in with ideas about how best to design for counts, and the conversation over on our API Craft Google group is underway.

Check it out!

So here's where we started.

Say, you want a count of all of some resource (scoped based on whatever the requester has access to).

/dogs/count

In this way, you think about "count" as a reserved word.

This means that you can never have "count" as an identifier on a resource - doing so would return that specific resource.  Otherwise, this would get a count of the resource (dogs) available in the database.

But this is only one way to think about getting counts. I'd love to hear more of your ideas and approaches over on API Craft.

Next: your URL and the top level domain

RESTful API Design: tips for search »

In the most post in this series about Pragmatic REST API design, I talked about handling responses that don't involve resources. This time, a somewhat related topic - search. 

What about searching?

While a simple search could be modeled as a resourceful API (for example, dogs/?q=red), a more complex search across multiple resources requires a different design.

This will sound familiar if you've read the aforementioned API design tip about using verbs not nouns when results don't return a resource from the database - rather the result is some action or calculation. what about responses that don’t involve resources?

If you want to do a global search across resources, I suggest you follow the Google model

Global search

/search?q=fluffy+fur

Here, search is the verb; ?q represents the query.

Scoped search

To add scope to your search, you can prepend with the scope of the search. For example, search in dogs owned by resource ID 5678

/owners/5678/dogs/search?q=fluffy+fur

Formatted results

For search or for any of the action oriented (non-resource) responses, you can prepend with the format as follows:

/search.xml?q=fluffy+fur

Next: what about counts?

Check out the full series on Pragmatic REST API Design. Also we'd love to hear more of your comments and questions over on API Craft.

RESTful API Design: what about responses that don’t involve resources? »

In a related post in this series about Pragmatic REST API design, I talked about partial responses and pagination. Check out the full series. This time:

What's an API response that doesn't involve resources?

API calls that send a response that's not a resource per se are not uncommon depending on the domain. I've seen it in financial services, Telco, and the automotive domain to some extent.

Actions like the following are your clue that you might not be dealing with a "resource" response.

• Calculate
• Translate
• Convert

For example, you want to make a simple algorithmic calculation like how much tax someone should pay, or do a natural language translation (one language in request; another in response), or convert one currency to another. None involve resources returned from a database.

In these cases:

Use verbs not nouns

For example, an API to convert 100 euros to Chinese Yen

/convert?from=EUR&to=CNY&amount=100

Make it clear in your API documentation that these are different.

Simply separate out a section of documentation that makes it clear that you use verbs in cases like this because these calls are not resource related.

Next time: what about searching?

RESTful API Design: what about attribute names? »

In a recent post in this series about Pragmatic REST API design, I talked about formats - supporting multiple formats and working with JSON as the default. Check out the full series.

This time, let's talk about what happens when a response comes back.

You have an object with data attributes on it. How should you name the attributes?

Here are API responses from a few leading APIs:

Twitter

"created_at": Thu Nov 03 05:19;38 +0000 2011"

Bing

"DateTime": "2011-10-29T09:35:00Z"

Foursquare

"createdAt": 1320296464

They each use a different code convention. Although the Twitter approach is familiar to me as a Ruby on Rails developer, I think that Foursquare has the best approach.

How does the API response get back in the code? You parse the response (JSON parser); what comes back populates the Object. It looks like this

var myObject = JSON.parse(response);

If you chose the Twitter or Bing approach, your code looks like this. It's not JavaScript convention and looks weird - looks like the name of another object or class in the system, which is not correct.

timing = myObject.created_at;

timing - myObject.DateTime;

Recommendations

Use JSON as default

Follow JavaScript conventions for naming attributes

- Use medial capitalization (aka CamelCase)

- Use uppercase or lowercase depending on type of object

This results in code that looks like the following, allowing the JavaScript developer to write it in a way that makes sense for JavaScript.

"createdAt": 1320296464

timing = myObject.createdAt;

Next: What to do when your API returns something other than a resource.

RESTful API Design: support multiple formats »

In the most recent post in this series about Pragmatic REST API design, I talked about pagination and partial response. Check out the full series. This time:

What about formats? Should you support only one format or multiple formats?
I recommend that you support more than one format - that you push things out in one format and accept as many formats as necessary. You can usually automate the mapping from format to format.

Here's what the syntax looks like for a few key APIs

Google Data

?alt=json

Foursquare

/venue.json

Digg*

Accept: application/json
?type=json

* the type argument, if present, overrides the Accept header

Digg allows you to specify in two ways: in a pure RESTful way in the Accept header or in the type parameter in the URL. This can be confusing - at the very least you need to document what to do if there are conflicts.

I recommend the Foursquare approach

To get the JSON format from a collection or specific element.

dogs.json

/dogs/1234.json

Developers and even casual users of any file system are familiar to this dot notation. It also requires just one additional character (the period) to get the point across.

What about defaults?

In my opinion, JSON is winning out as the default format. JSON is the closest thing we have to universal language. Even if the back end is built in Ruby on Rails, PHP, Java, Python etc., most projects probably touch JavaScript for the front-end. It also has the advantage of being terse - less verbose than XML.

Next time: what about attribute names?

RESTful API Design: can your API give developers just the information they need? »

In a recent post in this series about Pragmatic REST API design, I talked about tips for versioning your APIs. Check out the full series.

This time - pagination and partial response - how do you give developers exactly the information they need?

Partial response

Take for example the Twitter API - a request for a tweet. You'll get much more than a typical twitter app often needs - including the name of person, the text of the tweet, a timestamp, how often the message was retweeted, and a lot of metadata.

Let's look at how several leading APIs handle giving developers just what they need in responses, including Google who pioneered the idea of partial response.

LinkedIn

/people:(id,first-name,last-name,industry)

This request on a person returns ID, first name, last name, and industry.

LinkedIn uses this terse :(...) syntax which isn't self-evident. Plus it's difficult for a developer to reverse engineer the meaning using a search engine. But this is the way that LinkedIn does partial selection.

Facebook

/joe.smith/friends?fields=id,name,picture

Google

?fields=title,media:group(media:thumbnail)

Google and Facebook have a similar approach, which works well.

They each have an optional parameter called fields after which you put the names of fields you want to be returned.

As you see in this example, you can also put sub-objects in responses to pull in other information from additional resources.

Add optional fields in a comma delimited list

The Google approach works extremely well.

Here's how to get just the information we need from our dogs API using this approach:

/dogs?fields=name,color,location

It's simple to read; a developer can select just the information an app needs at a given time; it cuts down on bandwidth issues, which is important for mobile apps.

The partial selection syntax can also be used to include associated resources cutting down on the number of requests needed to get the required information.

Make it easy for developers to paginate objects in a database

It's almost always a bad idea to return every resource in a database.

Let's look at how Facebook, Twitter, and LinkedIn handle pagination.

• Facebook uses offset and limit
• Twitter uses page and rpp (records per page)
• LinkedIn uses start and count

Semantically, Facebook and LinkedIn do the same thing. That is, the LinkedIn start & count is used in the same way as the Facebook offset & limit.

Say you want to get records 50 through 75 from each system. You would use:

• Facebook -  offset 50 and limit 25
  
• Twitter - page 3 and rpp 25 (records per page)
• LinkedIn - start 50 and count 25
  

Use limit and offset

I recommend limit and offset. It is more common, well understood in leading databases, and easy for developers.

/dogs?limit=25&offset=50

What about defaults?

My loose rule of thumb for default pagination is limit=10 with offset=0.

limit=10&offset=0

The pagination defaults are of course dependent on your data size. If your resources are large, you probably want to limit it to fewer than 10; if resources are small, it can make sense to choose a larger limit.

In summary:

Support partial response by adding optional fields in a comma delimited list

Use limit and offset to make it easy for developers to paginate objects.

Next time: dealing with multiple formats.

RESTful API Design: version & format in URLs or headers? »

In a recent tips for versioning post and on the RESTful API Design webinar, I described some best practices for versioning your APIs - move the v as far left in the URL as possible; place it at the highest scope (e.g. /v1/resources); and use a simple ordinal number, no dot notation.

I wanted to follow up with a little more on versioning, inspired by some great discussion on API craft (keep it coming!).

As I mentioned on the video, there is a strong school of thought about putting format and version in the header.

I think that sometimes people are forced to put the version in the header because they have multiple inter-dependent APIs. That is often a symptom of a bigger problem, namely, they are usually exposing their internal mess
instead of creating a usable API facade on top.

That's very different than saying putting the version in the header is a symptom of a problematic API design. It's not!

In fact, using headers is more correct for many reasons: it leverages existing HTTP standards, it's intellectually consistent with Fielding's vision, it solves some hard real-world problems related to inter-dependent APIs, and more.

However, I think the reason most of the popular APIs do not use it is because it's less fun to hack in a browser.

Simple rules I follow:

If it changes the logic I write to handle the response, put it in the URL so I can see it easily.

If it doesn't change the logic for each response, like OAuth stuff, put it in the header.

These for example, all represent the same resource:

dogs/1
Content-Type: application/json

dogs/1
Content-Type: application/xml

dogs/1
Content-Type: application/png

The code I would write to handle the responses would be very different.

There's no question the header is more correct and it is still a very strong API design.

Next in the series on RESTful API Design: pagination and partial response - how to return just what developers need.

Meanwhile, I'd love to hear more of your thoughts in the API Craft Google group.

RESTful API Design: tips for versioning »

In the last post in this series about Pragmatic REST API design, I talked about designing error conditions in your APIs. Check out the full series here.

This time - versioning - one of the most important considerations when designing your pragmatic API.

Never release an API without a version and make the version mandatory.

Let's see how three top API providers handle versioning.

Twilio uses a timestamp in the URL (note the European format).

At compilation time, the developer includes the timestamp of the application when the code was compiled. That timestamp goes in all the HTTP requests.

When the request comes into Twilio, they do a look up. Based on the timestamp they identify the API that was valid when this code was created and route accordingly.

It's a very clever and interesting approach, although I think it is a bit complex. It can be confusing to understand whether the timestamp is compilation time or the timestamp when the API was released, for example.

Salesforce.com uses v20.0, placed somewhere in the middle of the URL.

I like the use of the v. notation. However, I don't like using the .0 because it implies that the interface might be changing more frequently than it should. The logic behind an interface can change rapidly but the interface itself shouldn't change frequently.

Facebook also uses the v. notation but makes the version an optional parameter.

This is problematic because as soon as Facebook forced the API up to the next version, all the apps that didn't include the version number broke and had to be pulled back and version number added.

How to think about version numbers in a pragmatic way with REST?

Specify the version with a 'v' prefix. Move it all the way to the left in the URL so that it has the highest scope (e.g. /v1/dogs).

Use a simple ordinal number - v1, v2, and so on. Don't use the dot notation like v1.2 because it implies a granularity of versioning that doesn't work well with APIs--it's an interface not an implementation.

Make the version mandatory.

How many versions to maintain?

Next time -Giving developers just the information they need -  pagination and partial response.

Please join the API Craft conversation on Google groups.

RESTful API Design: what about errors? »

In the previous posts in this series about Pragmatic REST API design, I talked about simplyfing associations, using the HTTP ? to hide complexities and optional parameters, choosing plural nouns and concrete names, and more. See the series here.

What about errors in the context of RESTful API best practices? Many software developers, including myself, don't always like to think about exceptions and error handling but it is a very important piece of the puzzle for any software developer, and especially for API designers.  

Why is good error design especially important for API designers?

Bottom line, it's about making your APIs intuitive and making developers successful.

First, developers learn to write code through errors.  The "test-first" concepts of the extreme programming model and the more recent "test driven development" models represent a body of best practices that have evolved because this is such an important and natural way for developers to work.

From the perspective of the developer consuming your Web API, everything at the other side of that interface is a black box. Errors therefore become a key tool providing context and visibility into how to use an API.

Secondly, in addition to when they're developing their applications, developers depend on well-designed errors at the critical times when they are troubleshooting and resolving issues after the applications they've built using your APIs are in the hands of their users.

How to think about errors in a pragmatic way with REST?

Let's take a look at how three top APIs approach it.

Facebook
No matter what happens on a Facebook request, you get back the 200 status code - everything is OK. Many error messages also push down into the HTTP response. Here they also throw an #803 error but with no information about what #803 is or how to react to it.

Twilio
Twilio does a great job aligning errors with HTTP status codes. Like Facebook, they provide a more granular error message but with a link that takes you to the documentation. Community commenting  and discussion on the documentation helps to build a body of information and adds context for developers experiencing these errors.

SimpleGeo
Provides error codes but with no additional value in the payload. 

A couple of best practices

Use HTTP status codes
Use HTTP status codes and try to map them cleanly to relevant standard-based codes.

There are over 70 HTTP status codes. However, most developers don't have all 70 memorized. So if you choose status codes that are not very common you will force application developers away from building their apps and over to wikipedia to figure out what you're trying to tell them. 

Therefore, most API providers use a small subset. For example, the Google GData API uses only 10 status codes, Netflix uses 9, and Digg, only 8.

How many status codes should you use for your API?

When you boil it down, there are really only 3 outcomes in the interaction between an app and an API:

• Everything worked
• The application did something wrong
• The API did something wrong

Start by using the following 3 codes. If you need more, add them. But you shouldn't go beyond 8.

• 200 - OK
• 404 - Not Found
• 500 - Internal Server Error

If you're not comfortable reducing all your error conditions to these 3, try picking among these additional 5:

• 201 - Created
• 304 - Not Modified
• 400 - Bad Request
• 401 - Unauthorized
• 403 - Forbidden

(Check out this good Wikipedia entry for all HTTP Status codes.)

It is important that the code that is returned can be consumed and acted upon by the application's business logic - for example, in an if-then-else, or a case statement. 

Make messages returned in the payload as verbose as possible

RESTful API Design: simplify associations, sweep complexities under the HTTP ? »

In the previous post in this series about Pragmatic REST API design, I talked about choosing plural versus singular nouns and concrete names over abstract. See RESTful API Design: plural nouns and concrete names.

This time we'll explore API design considerations when handling associations between resources and parameters like states and attributes.

Associations
Resources almost always have relationship to other resources. What's a simple way to express these relationships in a RESTful world?

Let's look again at the API we modeled in nouns are good, verbs are bad - the API that interacts with our dogs resource. Remember, we had two base URLs:

/dogs

/dogs/1234

We're using HTTP verbs to operate on the resources and collections. Our dogs belong to owners. To get all the dogs belonging to a specific owner, or to create a new dog for that owner, do a GET or a POST:

GET owners/5678/dogs

POST owners/5678/dogs

Now, the relationships can be complex. Owners have relationships with vetinerians, who have relationships with dogs, who have relationships with food, and so on. It's not uncommon to see people string these together making a URL 5 or 6 levels deep. Remember that once you have the primary key for one level, you don't need to include the levels above that because you've already got your specific object. In other words, you shouldn't need too many cases where a URL is deeper than what we have above - resource name/identifier/resource name.

Sweep complexity behind the HTTP ?
Most APIs have intricacies beyond the base level of a resource. Complexities can include many states that can be updated, changed, queried, as well as the attributes associated with a resource.

Make it simple for developers to use the base URL by putting optional states and attributes behind the HTTP question mark. To get all red dogs running in the park:

GET /dogs?color=red&state=running&location=park

In summary, keep your API intuitive by simplifying the associations between resources, and sweeping parameters and other complexities under the rug of the HTTP question mark.

Next time, what about errors?

RESTful API Design: plural nouns and concrete names »

In the first post in this series, Are you a Pragmatist or a RESTafarian?, I proposed that "pragmatic REST" is a design issue.

In the second post, RESTful API Design: nouns are good, verbs are bad, I outlined some of  the API design practices that work well: Nouns in URLs are good. Verbs are bad. Try to limit your API to 2 base URLs per resource.

This time, we'll explore how to pick the nouns for your URLs.

Plural nouns are better than singular
Should you choose singular or plural nouns? You'll see popular APIs use both. Let's look at a few examples  - key resources for these businesses are expressed as either singular or plural:

Foursquare
/checkins

GroupOn
/deals

Zappos
/Product

Given that the first thing most people probably do with a RESTful API is a GET, I think it reads more easily and is more intuitive to use plural nouns. But above all, avoid a mixed model in which you use singular for some resources, plural for others. Being consistent allows developers to predict and guess the method calls as they learn to work with your API.

Concrete names are better than abstract
Achieving pure abstraction is sometimes a goal of API architects. However, that abstraction is not always meaningful for developers.

Take for example an API that accesses content in various forms  - blogs, videos, news articles, and so on.

An API that models everything at the highest level of abstraction -  as /items or /assets in our example - loses the opportunity to paint a tangible picture for a developer to know what they can do with this API. It is more compelling and useful to see the resources listed as blogs, videos, and news articles.

The level of abstraction depends on your scenario. You also want to expose a manageable number of resources. Aim for concrete naming and to keep the number of resources between 12 and 24.

In summary, an intuitive API uses plural nouns and concrete names rather than abstract.

Next time, simplifying associations and hiding complexities using the HTTP ?.

RESTful API Design: nouns are good, verbs are bad »

In the first post in this series about pragmatic REST API design, I defined 'pragmatic REST' as looking at API design from the developer point of view. Now let's get into specific design practices we've seen work well.

The #1 principle in pragmatic RESTful design is: keep simple things simple.

Keep your base URL simple and intuitive

The base URL is the most important design affordance of your API.  A simple and intuitive base URL design will make using your API easy.

A design affordance is a design element that communicates how something should be used without requiring documentation.  A door handle's design should communicate whether you pull or push. But here's an example of a conflict between design affordance and documentation - not an intuitive interface!

My key litmus test for simple API design and pragmatic REST is:  only 2 base URLs per resource

Let's model an API around a simple object or resource (dogs) and create a RESTful API that interacts with dogs.

The first base URL is for collections; the second is for a specific element in the collection.

Boiling it down to this level will also force the verbs out of your base URLs.   

Keep verbs out of your base URLs

Many RESTful APIs start by using a method-driven approach to URL design. These method-based URLs sometimes contain verbs -  sometimes at the beginning, sometimes at the end.

For any resource that you model, like our dog, you can never consider one object in isolation. Rather, there are always related and interacting resources to account for - like owners, veterinarians, leashes, food, squirrels, and so on.

Think about the method calls required to address all the objects in the dogs world. The URLs for our resource might end up looking something like this:



It's a slippery slope - soon you have a huge list of URLs and no consistent pattern making it difficult for developers to learn how to use your APIs.
 
Use HTTP verbs to operate on the collections and elements

For our dog resources, we have two base URLs that use nouns as labels, and we can operate on them with HTTP verbs. Our HTTP verbs are POST, GET, PUT, and DELETE. (I think of them as mapping to the old metaphor of CRUD (Create-Read-Update-Delete).)  

With our two resources (/dogs and /dogs/1234) and the four HTTP verbs, we have a rich set of capability that's intuitive to the developer. Here is a chart that shows what I mean for our dogs.



The point is that a developer probably doesn't need the chart to grok how the API behaves. They can experiment with and learn the API without having to dig into the documentation.  

In summary: Use 2 base URLs per resource.  In your URLs - nouns are good; verbs are bad.

Next time:  how do you pick your nouns?

Please join the API Craft conversation on Google groups.

API Design: Are you a Pragmatist or a RESTafarian? »

I wanted to write some posts on RESTful API design best practices we've observed over the last year.   I covered a lot of these recently in a API design presentation but want to write more detail out.

First - let's start with our overall point of view on API design.

We advocate pragmatic, not dogmatic REST. What do I mean by dogmatic?   

You might have seen discussion threads on true REST - some of them can get pretty strict and wonky. 

 @mikeschinkel sums it up well - defining the RESTafarian. 

Our view: approach API design from the 'outside-in' perspective.

This means we start by asking - what are we trying to achieve with an API?

The API's job is to make the developer as successful as possible.   

Why? Look at the value chain below - the developer is the lynchpin of our entire API strategy.

So design the API to maximize developer productivity and success as the primary design principle.  

This is what we call pragmatic REST.

Pragmatic REST is a design problem.

You have to get the design right, because design communicates how it will be used.

So now the question is  - what is the design with optimal benefit for the app developer?

The design asethetic for APIs is to think about design choices from the developer point of view.

This is pretty much the guiding principle for all the specific tips that we'll dig into next.

Next time:   considerations in designing your base URLs.

“RESTful API Design: Second Edition”  Webinar - Video & slides »

Here are the slides and video for last week's RESTful API Design webinar, thanks again to everyone for joining.

Check out some additional tips and advice for designing Pragmatic RESTful APIs on the API Best Practices blog.

We'd love more thoughts or questions - there is a thread on the api-craft forum.  

API RESTful Design question:  Versioning number in the URL? »

We're starting to get some great questions from signups for next week's API design webinar:  "Pragmatic REST - API Design Fu". 

Here's one we'll comment on now and then talk about more next week.

Q: "For API Versioning.  Doesn't seem very RESTful to version a resource.  Can it be avoided by using implementation and specification versions described in the Sun API Cloud Documentation?"

A: Your question is an insightful one. Versioning is an open issue for REST API design in practice.

Clearly, an API will need to change it's behavior over time. As it changes application developers will need to change their code accordingly. So, versioning is inevitable.  

(caveat: Dan Jacobson of Netflix and NPR's API program has great points on what he calls a 'versionless API' in this presentation around slide 202.")

Our point of view here at Apigee is API design approach we call pragmatic REST.

The driving philosophy of our approach is: what's good for the application developer is good for the API. The Sun Cloud approach to versioning is interesting. The upside is it's powerful and will handle cases of inter-dependent API calls well.

The downside is it's complicated and as a developer starts using it, there is a lot to learn. Plus, because the versions are specified in the header, he won't be able to copy & paste it into the browser to learn it.

Our suggestion - even though it clearly violates pure REST theory - is to include the API version in the URL as a single digit number like /v1, /v2, etc. Whatever complexity is involved in making it "work" is on the shoulders of the API team.

More on this topic in our webinar - sign up here and hope to see you there.

Universal Design Principles Applied to APIs (Part 2 of 4) »

Following Part 1, here are the slides and a video for Part 2 of our series on applying universal principles of design to APIs.  In this part I cover 3 more universal design principles, including:

• Flexibility-Usability Tradeoff
• Hick’s Law
• 80/20 Rule
• Inverted Pyramid

And for more on API Design check out Teach a Dog to REST and REST for SQL Developers

REST API design for SQL programmers »

@gbrail put together this short deck mapping SQL concepts to (pragmatic) RESTful API design.  And for more on API design - see Teach a Dog to REST and Universal Design Principles Applied to APIs

REST API Design for SQL developers

Design your API for adoption with ‘true REST’ »

"The only reason you'd have only a SOAP API is because you hate 80% of your addressable market." - @sramji

There's usually little argument that a REST API is easier to use than a SOAP API.

But how important is it to be 'truly' or 'strictly' RESTful? That is, adhering to standard HTTP operations or 'verbs' - GET, PUT, DELETE, POST - on well defined resources, as opposed to the common practice of embedding 'verbs' or operations as methods in a GET URL.

Typically, security is cited as the big advantage of 'true REST' (with some good discussions here and here).

However, a truly RESTful API may help you boost developer adoption. For example, imagine a 'shopping cart' API:

While the above API isn't 'truly RESTful', it's not that hard to use.  But you do have to learn the individual operations and this can get cumbersome if there are a lot of them or as the API evolves.

Instead, this 'true REST API' may be easier to learn and predict as you use more features.

What if we want to list all the shopping carts in the system at any one time? We would add that via an HTTP GET to:

http://api.shopping.com/carts.xml

Query parameters can still serve a purpose - making it possible to specify additional options. For instance, imagine a very large shopping cart, and you want to "paginate" the results. To look at items 20-29, you might use a URL like:

http://api.shopping.com/carts/X.xml?start=20&;count=10

Bonus: True REST makes analytics easier

It's a lot easier to build reports that segregate and analyze traffic by URL than to build logic that tries to do this by methods or combination of methods. Good API analytics helps you optimize features, debug problems, and weed out traffic that can slow down your service.

What if your 'non-strict' REST API is already out there?

It might not be that big a deal if your API is very simple or 'read-only' API with information that isn't too sensitive (such as a free search API). Or you can map a non-RESTful API into a 'truly RESTful" API with custom code or API management tools thatperform API transformations.

* A note on POST VS. PUT

* One way to insert an item in the shopping cart is to use POST to update the shopping cart by sending it a new item. In this case, we are using POST to send the server an instruction that essentially tells it to insert some new content to the existing resource. This is why we use POST -- it is like an "update" in a database.

Alternately, we could add the item by using PUT to a new URL, such as:

PUT http://api.shopping.com/carts/X/items/Y.xml

But if we do this, then we need to somehow give our item some sort of unique URL by picking a value for "Y". This is kind of a strange thing to do, so it may be more natural to use POST and have the response include the URL for the new item, so that we may retrieve it later using GET or delete it using DELETE. Still, sometimes using PUT like this makes sense.

This all comes down to the difference between PUT and POST in the HTTP spec. POST modifies something that already exists, and how that thing is modified is up to the server. PUT replaces the entire contents of the URL with new data. Plus, like GET, HEAD, and DELETE, PUT is idempotent, which means that if you call it more than once, it has the same result every time, whereas POST may keep doing what you ask it to do over and over again

Yes, REST security does exist »

A recent article "Why REST Security Doesn't Exist." postulates, "REST does not have predefined security methods, so developers define their own."

Some good points in here (such as 'don't roll your own')  but I might not completely agree with the premise.

One of the fundamental principles of REST is that it builds on the HTTP protocol -- and the HTTP protocol very much does have "predefined security methods."
 
The basic HTTP protocol supports a way to plug in different security schemes. It also supports OAuth, two-way SSL, and many other mechanisms. Not only does HTTP allow for many security schemes, many of which like HTTP basic are defined by IETF standards, but it also supports a mechanism that allows a server to identify when a request was rejected, if the request was rejected because the security credentials were invalid or because an authorization check failed, and whether the rejection was permanent or temporary. HTTP also includes a mechanism that allows the server to issue a "challenge" that asks the client to re-send a request with a particular type of credentials if it has them. This all adds up to a security method that has proven quite robust over the years and has been extended with new methods such as OAuth when new problems arise.
 
Also, don't forget that different APIs demand different security requirements. An API that offers product catalog information, for instance, with no way to update the information, does not require strong authentication if the owner of the data intends that data to be public anyway. A simple "developer key" that uniquely identifies the sender of the request -- yes, a username "without a password" -- is just fine for that type of API because it is used to identify the user for various tracking purposes, and is not designed to prevent unauthorized users from gaining access to the data.

Cool article and great to see some discussion on this!

Does open source still matter in cloud computing? »

Open source has a role in all types of computing.

It has always reduced barriers to entry to a particular platform or  programming model.

But in cloud computing open source plays a new role - driving the rate of evolution of the infrastructure to match pace with the rate of evolution of an early market.

Open Source in Cloud Platforms

Open source platforms like Xen, OpenNebula, and Eucalyptus enable universities and early-stage adopters to deploy the core platforms without financial risk.  No up-front commitment in a cash-poor environment like the current economy means that pilot projects can still move forward as companies and institutions explore the potential of cloud computing. 

Xen and OpenNebula enable development of on-premise cloud environments.  Eucalyptus goes a step beyond this to emulate Amazon's EC2 APIs for cloud infrastructure.  Eucalyptus effectively remedies a hole in Amazon's strategy - a zero-cost entry point for EC2 pilot projects - and adds weight to the de facto standard stature of the EC2 APis.

Open Source in Cloud Programming models

Cloud computing is expanding much more rapidly than service-oriented architectures, largely due to the accessibility of the model ; it can be understood with little effort by those who currently have skills in web programming, and is implemented in a range of popular programming platforms - Ruby on Rails, PHP Symfony, Apache Axis2 - as REST services. 

There are SOAP cloud services as well.  But the REST model is easier to pick up, implement and integrate.  Again, the ubiquity of open source through the entire cloud stack - from programming layers down to HTTP and TCP/IP - means that the time and cost barriers to entry to this market are very low, which has led to its rapid expansion.

Open Source in Evolution of the Cloud Computing Market

The most important aspect of open source in cloud computing is the open collaboration model which is a property of the communities developing both infrastructure and frameworks. 

There are fundamental differences in cloud-scale computing  - such as how to manage writes, reads and searches across all that data now that it's consolidated in a single place.  Hadoop and Cassandra represent two very different solutions to different aspects of this problem. 

As the largest cloud infrastructure platforms (Facebook, Digg, Yahoo, Microsoft, Google, and others) continue to demolish the envelope of scale, new approaches must emerge.  Hadoop is a "stand the problem on its head" approach to rapidly searching through huge repositories for particular pieces of data - evolving rapidly due to the multiple parties contributing to it as they solve for their individual business problems.  Cassandra is a reimagining of the nature of large-scale reads and writes outside of a traditional relational model that avoids the architectural bottlenecks that exist in the RDBMS genre.   

Programming models will also continue to evolve as the infrastructure and platforms evolve - and the crucial aspect of open source community development will continue to show its value as framework and application developers continue to codify their understanding of the new patterns and practices of cloud computing into existing and new application frameworks.

Commercial open source in cloud computing

Commercial opportunties surround these technologies - both in adapting to new de facto standards, and delivering faster or more efficient infrastructure. 

A few good examples of these include Heroku - an EC2-based fabric for Ruby on Rails, fully compliant with the evolving standards of the Rails community, yet providing an implementation of Rails functions that scale efficiently on Amazon's AWS - and Cloudera - again, an EC2-based packaging and support offering that makes it simple to deploy Hadoop, which is powerful but notoriously hard to get up and running.  I expect that savvy storage vendors will find ways to build Cassandra-compatible infrastructure that enhances scalable operations and analysis of next-gen cloud applications.

Does open source matter in cloud computing?

Some have said - including Tim O'Reilly - that open source doesn't matter in the cloud.  I think that what was meant was that once your computing workloads are hosted on a remote service, the most important things to you as a customer are reliability, uptime, scalability, interoperability and manageability - not whether or not the cloud infrastructure or application provider is running on open source.  I think this is true. 

Cloud computing does not make open source irrelevant.  The enormous value that cloud computing is generating, and the rate at which it is changing, demands that open source plays a ubiquitous role.  It will not always be a flashy role and may not seek the limelight, but it will be the glue that lets the whole system evolve and mature.