RESTful API Design: API Virtualization »

Last time, we looked at why you might consider complementing your API with an SDK or code libraries. In the series so far, we've covered a lot of tips and tricks for designing pragmatic RESTful APIs. 

You may be asking -  

How do I follow all these best practice guidelines and still maintain and iterate my APIs? 

What should I be thinking from an architectural perspective in terms of implementing these best practices?

Add an API virtualization layer 

I recommend you give yourself a buffer or virtual layer between the interface on top and the API implementation on the bottom.

The app that consumes the API is on top. The API virtualization layer isolates the application and the API. Make a clean design for the app on top and turn the problem into an integration problem.

 In other words, work at integrating your design on top of the key APIs through the virtualization layer.

Don't start in code, and try to build up from your business logic to a clean API interface.

RESTful API Design: complement with an SDK »

So far in the API design series, I've looked at best practices for designing pragmatic RESTful APIs.

This time, I'll talk about complementing APIs with code libraries and SDKs.

What to do when building a UI requires a lot of domain knowledge?

This image is an example of a twitter app displayed in a newsroom.

A developer created this cool visualization - a tweet referencing another twitter user.

The Twitter API is one of the simplest app domains imaginable: the primary object is 140 characters of text. Yet, the developer who created this needed quite a bit of domain knowledge - they needed to parse the tweet, color code it, and display it in a certain way.

The API team shouldn't try to solve presentation problems with the API. That's what code libraries do. Instead

Complement your API with code libraries and a software development kit (SDK)

That is, don't overburden your API design. A lot of what's needed in scenarios like this is on the client side and you can push that burden to an SDK.

The SDK can provide the platform-specific code that developers use in their apps to invoke API operations - meaning you keep your API clean.

Other reasons you might consider complementing your API with an SDK

• Speed adoption on a specific platform - for example Objective C SDK for iPhone. Lots of experienced developers are just starting off with objective C+ so an SDK might be helpful.
• Simplify integration effort required to work with your API - If key use cases are complex or need to be complemented by standard on-client processing.
• An SDK can help reduce bad or inefficient code that might slow down service for everyone.
• As a developer resource - Good SDKs are a forcing function to create good source code examples and documentation. Both Yahoo! and Paypal have good examples.
• To market your API to a specific community - you upload the SDK to a samples or plug-in page on the platform's existing developer community.

Next time: API Virtualization

RESTful API Design: chatty APIs »

In this post in the series, let's think about how app developers use that API you're designing and dealing with chatty APIs.

Imagine how developers will use your API

When designing your API and resources, try to imagine how  developers will use it to say construct a user interface, an iPhone app, or many other apps.

Some API designs become very chatty - meaning just to build a simple UI or app, you have dozens or hundreds of API calls back to the server.

The API team can sometimes decide not to deal with creating a nice, resource-oriented RESTful APIs, and just fall back to a mode where they create the 3 or 4 Java-style getter and setter methods they know they need to power a particular user interface.

I don't recommend this. You can design a RESTful API and still mitigate the chattiness.

Be complete and RESTful and provide shortcuts

First design your API and its resources according to pragmatic RESTful design principles (link the series) and then provide shortcuts.

What kind of shortcut?

Say you know that 80% of all your apps are going to need some sort of composite response, then build the kind of request that gives them what they need.

Just don't do the latter instead of the former. First design using good pragmatic RESTful principles!

Take advantage of the partial response syntax

The partial response syntax I discussed in a previous post (link) can help.

To avoid creating one-off base URLs, you can use the partial response syntax to drill down to dependent and associated resources.

In the case of our dogs API, the dogs have association with owners, who in turn have associations with veterinarians, and so on. Keep nesting the partial response syntax to get back just the information you need.

    /owners/5678?fields=name, dogs(name)

Next, SDKs and code libraries.

RESTful API Design: making requests »

We've covered singular vs. plural nouns to label your resources, tips for search, handling errors, and more.

Now lets take a look at what some API requests and responses look like for our dogs API.

Create a brown dog named Al

POST /dogs
name=Al&furColor=brown
Response
200 OK

{
"dog":{
"id:"1234",
"name": "Al",
"furColor": "brown"
}
} 

Rename Al to Rover - Update

PUT /dogs/1234
name=Rover
Response
200 OK

{
"dog":{
"id:"1234",
"name": "Rover",
"furColor": "brown"

}
} 

Tell me about a particular dog

GET /dogs/1234

Response
200 OK

{
"dog":{
"id:"1234",
"name": "Rover",
"furColor": "brown"
}
}


Tell me about all the dogs

GET /dogs
Response
200 OK

{
"dogs":
[{"dog:{
"id:"1233",
"name": "Fido",
"furColor": "white"}},
{"dog:{
"id:"1234",
"name": "Rover",
"furColor": "brown"}}]
}

 Delete Rover :-(

DELETE /dogs/1234
Response
200 OK

Next time: chatty APIs.

RESTful API Design: how many versions? »

RESTful API Design: authentication »

This time, in this series about pragmatic RESTful API Design, I'll discuss authentication.

There are many schools of thought - my colleagues at Apigee and I don't always agree on how to handle authentication - but overall here's my take.

Let's look at these three top services. See how each of these services handles things differently:

PayPal

Permissions Service API

Facebook

OAuth 2.0

Twitter

OAuth 1.0a

Note that PayPal's proprietary three-legged permissions API  was in place long before OAuth was conceived.

What should you do?

Use the latest and greatest OAuth - OAuth 2.0 (as of this writing)

Don't do something *like* OAuth, but different
It will be frustrating to app developers if they can't use an OAuth library in their language because of your variation.

Next time: Versions - how many? Meanwhile, I'd love to hear from you over on the API Craft Google Group.

RESTful API Design: tips for handling exceptional behavior »

In the last post in this series, I talked about the top level domain (the stuff on the other side of the URL) and in this RESTful API design series so far, I've dealt with baseline, standard behaviors.

This time I'll explore some of the exceptions that can happen - when clients of Web APIs can't handle all the things we've discussed.

For example, sometimes clients intercept HTTP error codes, or support limited HTTP methods.

What are ways to handle these situations and work within the limitations of a specific client?

Client intercepts HTTP error code

One common thing in some versions of Adobe Flash - if you send an HTTP response that is anything other than HTTP 200 OK, the Flash container intercepts that response and puts the error code in front of the end user of the app.

Therefore, the app developer doesn't have an opportunity to intercept the error code. App developers need the API to support this in some way.

Twitter does an excellent job of handling this.

They have an optional  parameter suppress_response_codes. If set to true, the HTTP response is always 200.

/public_timelines.json?
suppress_response_codes=true
HTTP status code: 200{"error" : "Could not authenticate you."}

Notice that this parameter is a big verbose response code. (They could have used something like src, but they opted to be verbose.)

This is important because when you look at the URL, you need to see that the response codes are being suppressed as it has huge implications about how apps are going to respond to it.

Overall recommendations:

1 - Use suppress_response_codes = true

2 - The HTTP code is no longer just for the code

Our rules from earlier (what about errors?) change. In this context, the HTTP code is no longer just for the code - the program - it's now to be ignored. Client apps are never going to be checking the HTTP status code as it is always the same.

3 - Push any response code that we would have put in the HTTP response down into the response message

In my example below, the response code is 401 - see it in the response message.
Also include additional error codes and verbose information in that message.

Always return OK
/dogs?suppress_response_codes = true 

Code for  ignoring
200 - OK 

Message for people & code
{response_code" : "401", "message" : "Verbose, plain language description of the problem with hints about how to fix it."
"more_info" : "http://dev.tecachdogrest.com/errors/12345", "code" : 12345}

Client supports limited HTTP methods

It is common to see support for GET and POST and not PUT and DELETE.

To maintain the integrity of the four HTTP methods, I suggest you use the following methodology commonly used by Ruby on Rails developers:

Make the method an optional parameter in the URL.

Then the HTTP verb is always a GET but the developer can express rich HTTP verbs and still maintain a RESTful clean API.

Create
/dogs?method=post

Read
/dogs

Update
/dogs/1234?method=put&location=park

Delete
/dogs/1234?method=delete

WARNING:

It can be very dangerous to provide post or delete capabailities using a GET method because if the URL is in a web page then a web crawler like the Googlebot can create or destroy lots of content inadvertently. Be sure you understand the implications of supporting this approach for your applications' context.

Next time: authentication

RESTful API Design: consolidate API requests in one subdomain »

In the series so far, we've talked about everything that comes after the top level domain. This time, let's explore stuff on the other side of the URL.

Here's how Facebook, Foursquare, and Twitter handle this: 

Facebook provides two APIs.

They started with api.facebook.com, then modified it to orient around the social graph graph.facebook.com  

Foursquare has one API. 

Twitter has three APIs; two of them focused on search and streaming.

It's easy to understand how Facebook and Twitter ended up with more than one API.

It has a lot to do with timing and acquisition, and it's easy to reconfigure a CName entry in your DNS to point requests to different clusters.

But if we're making design decisions about what's in the best interest of app developer, I recommend following Foursquare's lead:

Consolidate all API requests under one API subdomain.

It's cleaner, easier and more intuitive for developers who you want to build cool apps using your API.

Facebook, Foursquare, and Twitter also all have dedicated developer portals.

developers.facebook.com
developers.foursquare.com
dev.twitter.com

How to organize all of this? 

Your API gateway should be the top-level domain

api.teachdogrest.com 

Your developer portal should be (in keeping with spirit of REST) developers.yourtopleveldomain.

developers.teachdogrest.com 

Do Web redirects

Then optionally, if you can sense from requests coming in from browser where the developer really needs to go, you can redirect.

Say a developer types api.teachdogrest.com in the browser but there's no other information for the GET request, you can probably safely redirect to your developer portal and help get the developer where he really needs to be.

• api -> developers (if from browser)
• dev -> developers
• developer -> developers

Next time: Exceptional behavior!

RESTful API Design: what about counts? »

What's the right way to do counts?

This question comes up a lot. This post talks about just one interesting way to approach it.

When I first kicked off the conversation in a recent RESTful API Design webinar, I asked developers to chime in with ideas about how best to design for counts, and the conversation over on our API Craft Google group is underway.

Check it out!

So here's where we started.

Say, you want a count of all of some resource (scoped based on whatever the requester has access to).

/dogs/count

In this way, you think about "count" as a reserved word.

This means that you can never have "count" as an identifier on a resource - doing so would return that specific resource.  Otherwise, this would get a count of the resource (dogs) available in the database.

But this is only one way to think about getting counts. I'd love to hear more of your ideas and approaches over on API Craft.

Next: your URL and the top level domain

RESTful API Design: tips for search »

In the most post in this series about Pragmatic REST API design, I talked about handling responses that don't involve resources. This time, a somewhat related topic - search. 

What about searching?

While a simple search could be modeled as a resourceful API (for example, dogs/?q=red), a more complex search across multiple resources requires a different design.

This will sound familiar if you've read the aforementioned API design tip about using verbs not nouns when results don't return a resource from the database - rather the result is some action or calculation. what about responses that don’t involve resources?

If you want to do a global search across resources, I suggest you follow the Google model

Global search

/search?q=fluffy+fur

Here, search is the verb; ?q represents the query.

Scoped search

To add scope to your search, you can prepend with the scope of the search. For example, search in dogs owned by resource ID 5678

/owners/5678/dogs/search?q=fluffy+fur

Formatted results

For search or for any of the action oriented (non-resource) responses, you can prepend with the format as follows:

/search.xml?q=fluffy+fur

Next: what about counts?

Check out the full series on Pragmatic REST API Design. Also we'd love to hear more of your comments and questions over on API Craft.

RESTful API Design: what about responses that don’t involve resources? »

In a related post in this series about Pragmatic REST API design, I talked about partial responses and pagination. Check out the full series. This time:

What's an API response that doesn't involve resources?

API calls that send a response that's not a resource per se are not uncommon depending on the domain. I've seen it in financial services, Telco, and the automotive domain to some extent.

Actions like the following are your clue that you might not be dealing with a "resource" response.

• Calculate
• Translate
• Convert

For example, you want to make a simple algorithmic calculation like how much tax someone should pay, or do a natural language translation (one language in request; another in response), or convert one currency to another. None involve resources returned from a database.

In these cases:

Use verbs not nouns

For example, an API to convert 100 euros to Chinese Yen

/convert?from=EUR&to=CNY&amount=100

Make it clear in your API documentation that these are different.

Simply separate out a section of documentation that makes it clear that you use verbs in cases like this because these calls are not resource related.

Next time: what about searching?

RESTful API Design: what about attribute names? »

In a recent post in this series about Pragmatic REST API design, I talked about formats - supporting multiple formats and working with JSON as the default. Check out the full series.

This time, let's talk about what happens when a response comes back.

You have an object with data attributes on it. How should you name the attributes?

Here are API responses from a few leading APIs:

Twitter

"created_at": Thu Nov 03 05:19;38 +0000 2011"

Bing

"DateTime": "2011-10-29T09:35:00Z"

Foursquare

"createdAt": 1320296464

They each use a different code convention. Although the Twitter approach is familiar to me as a Ruby on Rails developer, I think that Foursquare has the best approach.

How does the API response get back in the code? You parse the response (JSON parser); what comes back populates the Object. It looks like this

var myObject = JSON.parse(response);

If you chose the Twitter or Bing approach, your code looks like this. It's not JavaScript convention and looks weird - looks like the name of another object or class in the system, which is not correct.

timing = myObject.created_at;

timing - myObject.DateTime;

Recommendations

Use JSON as default

Follow JavaScript conventions for naming attributes

- Use medial capitalization (aka CamelCase)

- Use uppercase or lowercase depending on type of object

This results in code that looks like the following, allowing the JavaScript developer to write it in a way that makes sense for JavaScript.

"createdAt": 1320296464

timing = myObject.createdAt;

Next: What to do when your API returns something other than a resource.

RESTful API Design: support multiple formats »

In the most recent post in this series about Pragmatic REST API design, I talked about pagination and partial response. Check out the full series. This time:

What about formats? Should you support only one format or multiple formats?
I recommend that you support more than one format - that you push things out in one format and accept as many formats as necessary. You can usually automate the mapping from format to format.

Here's what the syntax looks like for a few key APIs

Google Data

?alt=json

Foursquare

/venue.json

Digg*

Accept: application/json
?type=json

* the type argument, if present, overrides the Accept header

Digg allows you to specify in two ways: in a pure RESTful way in the Accept header or in the type parameter in the URL. This can be confusing - at the very least you need to document what to do if there are conflicts.

I recommend the Foursquare approach

To get the JSON format from a collection or specific element.

dogs.json

/dogs/1234.json

Developers and even casual users of any file system are familiar to this dot notation. It also requires just one additional character (the period) to get the point across.

What about defaults?

In my opinion, JSON is winning out as the default format. JSON is the closest thing we have to universal language. Even if the back end is built in Ruby on Rails, PHP, Java, Python etc., most projects probably touch JavaScript for the front-end. It also has the advantage of being terse - less verbose than XML.

Next time: what about attribute names?

RESTful API Design: can your API give developers just the information they need? »

In a recent post in this series about Pragmatic REST API design, I talked about tips for versioning your APIs. Check out the full series.

This time - pagination and partial response - how do you give developers exactly the information they need?

Partial response

Take for example the Twitter API - a request for a tweet. You'll get much more than a typical twitter app often needs - including the name of person, the text of the tweet, a timestamp, how often the message was retweeted, and a lot of metadata.

Let's look at how several leading APIs handle giving developers just what they need in responses, including Google who pioneered the idea of partial response.

LinkedIn

/people:(id,first-name,last-name,industry)

This request on a person returns ID, first name, last name, and industry.

LinkedIn uses this terse :(...) syntax which isn't self-evident. Plus it's difficult for a developer to reverse engineer the meaning using a search engine. But this is the way that LinkedIn does partial selection.

Facebook

/joe.smith/friends?fields=id,name,picture

Google

?fields=title,media:group(media:thumbnail)

Google and Facebook have a similar approach, which works well.

They each have an optional parameter called fields after which you put the names of fields you want to be returned.

As you see in this example, you can also put sub-objects in responses to pull in other information from additional resources.

Add optional fields in a comma delimited list

The Google approach works extremely well.

Here's how to get just the information we need from our dogs API using this approach:

/dogs?fields=name,color,location

It's simple to read; a developer can select just the information an app needs at a given time; it cuts down on bandwidth issues, which is important for mobile apps.

The partial selection syntax can also be used to include associated resources cutting down on the number of requests needed to get the required information.

Make it easy for developers to paginate objects in a database

It's almost always a bad idea to return every resource in a database.

Let's look at how Facebook, Twitter, and LinkedIn handle pagination.

• Facebook uses offset and limit
• Twitter uses page and rpp (records per page)
• LinkedIn uses start and count

Semantically, Facebook and LinkedIn do the same thing. That is, the LinkedIn start & count is used in the same way as the Facebook offset & limit.

Say you want to get records 50 through 75 from each system. You would use:

• Facebook -  offset 50 and limit 25
  
• Twitter - page 3 and rpp 25 (records per page)
• LinkedIn - start 50 and count 25
  

Use limit and offset

I recommend limit and offset. It is more common, well understood in leading databases, and easy for developers.

/dogs?limit=25&offset=50

What about defaults?

My loose rule of thumb for default pagination is limit=10 with offset=0.

limit=10&offset=0

The pagination defaults are of course dependent on your data size. If your resources are large, you probably want to limit it to fewer than 10; if resources are small, it can make sense to choose a larger limit.

In summary:

Support partial response by adding optional fields in a comma delimited list

Use limit and offset to make it easy for developers to paginate objects.

Next time: dealing with multiple formats.

RESTful API Design: version & format in URLs or headers? »

In a recent tips for versioning post and on the RESTful API Design webinar, I described some best practices for versioning your APIs - move the v as far left in the URL as possible; place it at the highest scope (e.g. /v1/resources); and use a simple ordinal number, no dot notation.

I wanted to follow up with a little more on versioning, inspired by some great discussion on API craft (keep it coming!).

As I mentioned on the video, there is a strong school of thought about putting format and version in the header.

I think that sometimes people are forced to put the version in the header because they have multiple inter-dependent APIs. That is often a symptom of a bigger problem, namely, they are usually exposing their internal mess
instead of creating a usable API facade on top.

That's very different than saying putting the version in the header is a symptom of a problematic API design. It's not!

In fact, using headers is more correct for many reasons: it leverages existing HTTP standards, it's intellectually consistent with Fielding's vision, it solves some hard real-world problems related to inter-dependent APIs, and more.

However, I think the reason most of the popular APIs do not use it is because it's less fun to hack in a browser.

Simple rules I follow:

If it changes the logic I write to handle the response, put it in the URL so I can see it easily.

If it doesn't change the logic for each response, like OAuth stuff, put it in the header.

These for example, all represent the same resource:

dogs/1
Content-Type: application/json

dogs/1
Content-Type: application/xml

dogs/1
Content-Type: application/png

The code I would write to handle the responses would be very different.

There's no question the header is more correct and it is still a very strong API design.

Next in the series on RESTful API Design: pagination and partial response - how to return just what developers need.

Meanwhile, I'd love to hear more of your thoughts in the API Craft Google group.

RESTful API Design: tips for versioning »

In the last post in this series about Pragmatic REST API design, I talked about designing error conditions in your APIs. Check out the full series here.

This time - versioning - one of the most important considerations when designing your pragmatic API.

Never release an API without a version and make the version mandatory.

Let's see how three top API providers handle versioning.

Twilio uses a timestamp in the URL (note the European format).

At compilation time, the developer includes the timestamp of the application when the code was compiled. That timestamp goes in all the HTTP requests.

When the request comes into Twilio, they do a look up. Based on the timestamp they identify the API that was valid when this code was created and route accordingly.

It's a very clever and interesting approach, although I think it is a bit complex. It can be confusing to understand whether the timestamp is compilation time or the timestamp when the API was released, for example.

Salesforce.com uses v20.0, placed somewhere in the middle of the URL.

I like the use of the v. notation. However, I don't like using the .0 because it implies that the interface might be changing more frequently than it should. The logic behind an interface can change rapidly but the interface itself shouldn't change frequently.

Facebook also uses the v. notation but makes the version an optional parameter.

This is problematic because as soon as Facebook forced the API up to the next version, all the apps that didn't include the version number broke and had to be pulled back and version number added.

How to think about version numbers in a pragmatic way with REST?

Specify the version with a 'v' prefix. Move it all the way to the left in the URL so that it has the highest scope (e.g. /v1/dogs).

Use a simple ordinal number - v1, v2, and so on. Don't use the dot notation like v1.2 because it implies a granularity of versioning that doesn't work well with APIs--it's an interface not an implementation.

Make the version mandatory.

How many versions to maintain?

Next time -Giving developers just the information they need -  pagination and partial response.

Please join the API Craft conversation on Google groups.

RESTful API Design: what about errors? »

In the previous posts in this series about Pragmatic REST API design, I talked about simplyfing associations, using the HTTP ? to hide complexities and optional parameters, choosing plural nouns and concrete names, and more. See the series here.

What about errors in the context of RESTful API best practices? Many software developers, including myself, don't always like to think about exceptions and error handling but it is a very important piece of the puzzle for any software developer, and especially for API designers.  

Why is good error design especially important for API designers?

Bottom line, it's about making your APIs intuitive and making developers successful.

First, developers learn to write code through errors.  The "test-first" concepts of the extreme programming model and the more recent "test driven development" models represent a body of best practices that have evolved because this is such an important and natural way for developers to work.

From the perspective of the developer consuming your Web API, everything at the other side of that interface is a black box. Errors therefore become a key tool providing context and visibility into how to use an API.

Secondly, in addition to when they're developing their applications, developers depend on well-designed errors at the critical times when they are troubleshooting and resolving issues after the applications they've built using your APIs are in the hands of their users.

How to think about errors in a pragmatic way with REST?

Let's take a look at how three top APIs approach it.

Facebook
No matter what happens on a Facebook request, you get back the 200 status code - everything is OK. Many error messages also push down into the HTTP response. Here they also throw an #803 error but with no information about what #803 is or how to react to it.

Twilio
Twilio does a great job aligning errors with HTTP status codes. Like Facebook, they provide a more granular error message but with a link that takes you to the documentation. Community commenting  and discussion on the documentation helps to build a body of information and adds context for developers experiencing these errors.

SimpleGeo
Provides error codes but with no additional value in the payload. 

A couple of best practices

Use HTTP status codes
Use HTTP status codes and try to map them cleanly to relevant standard-based codes.

There are over 70 HTTP status codes. However, most developers don't have all 70 memorized. So if you choose status codes that are not very common you will force application developers away from building their apps and over to wikipedia to figure out what you're trying to tell them. 

Therefore, most API providers use a small subset. For example, the Google GData API uses only 10 status codes, Netflix uses 9, and Digg, only 8.

How many status codes should you use for your API?

When you boil it down, there are really only 3 outcomes in the interaction between an app and an API:

• Everything worked
• The application did something wrong
• The API did something wrong

Start by using the following 3 codes. If you need more, add them. But you shouldn't go beyond 8.

• 200 - OK
• 404 - Not Found
• 500 - Internal Server Error

If you're not comfortable reducing all your error conditions to these 3, try picking among these additional 5:

• 201 - Created
• 304 - Not Modified
• 400 - Bad Request
• 401 - Unauthorized
• 403 - Forbidden

(Check out this good Wikipedia entry for all HTTP Status codes.)

It is important that the code that is returned can be consumed and acted upon by the application's business logic - for example, in an if-then-else, or a case statement. 

Make messages returned in the payload as verbose as possible

RESTful API Design: simplify associations, sweep complexities under the HTTP ? »

In the previous post in this series about Pragmatic REST API design, I talked about choosing plural versus singular nouns and concrete names over abstract. See RESTful API Design: plural nouns and concrete names.

This time we'll explore API design considerations when handling associations between resources and parameters like states and attributes.

Associations
Resources almost always have relationship to other resources. What's a simple way to express these relationships in a RESTful world?

Let's look again at the API we modeled in nouns are good, verbs are bad - the API that interacts with our dogs resource. Remember, we had two base URLs:

/dogs

/dogs/1234

We're using HTTP verbs to operate on the resources and collections. Our dogs belong to owners. To get all the dogs belonging to a specific owner, or to create a new dog for that owner, do a GET or a POST:

GET owners/5678/dogs

POST owners/5678/dogs

Now, the relationships can be complex. Owners have relationships with vetinerians, who have relationships with dogs, who have relationships with food, and so on. It's not uncommon to see people string these together making a URL 5 or 6 levels deep. Remember that once you have the primary key for one level, you don't need to include the levels above that because you've already got your specific object. In other words, you shouldn't need too many cases where a URL is deeper than what we have above - resource name/identifier/resource name.

Sweep complexity behind the HTTP ?
Most APIs have intricacies beyond the base level of a resource. Complexities can include many states that can be updated, changed, queried, as well as the attributes associated with a resource.

Make it simple for developers to use the base URL by putting optional states and attributes behind the HTTP question mark. To get all red dogs running in the park:

GET /dogs?color=red&state=running&location=park

In summary, keep your API intuitive by simplifying the associations between resources, and sweeping parameters and other complexities under the rug of the HTTP question mark.

Next time, what about errors?

RESTful API Design: plural nouns and concrete names »

In the first post in this series, Are you a Pragmatist or a RESTafarian?, I proposed that "pragmatic REST" is a design issue.

In the second post, RESTful API Design: nouns are good, verbs are bad, I outlined some of  the API design practices that work well: Nouns in URLs are good. Verbs are bad. Try to limit your API to 2 base URLs per resource.

This time, we'll explore how to pick the nouns for your URLs.

Plural nouns are better than singular
Should you choose singular or plural nouns? You'll see popular APIs use both. Let's look at a few examples  - key resources for these businesses are expressed as either singular or plural:

Foursquare
/checkins

GroupOn
/deals

Zappos
/Product

Given that the first thing most people probably do with a RESTful API is a GET, I think it reads more easily and is more intuitive to use plural nouns. But above all, avoid a mixed model in which you use singular for some resources, plural for others. Being consistent allows developers to predict and guess the method calls as they learn to work with your API.

Concrete names are better than abstract
Achieving pure abstraction is sometimes a goal of API architects. However, that abstraction is not always meaningful for developers.

Take for example an API that accesses content in various forms  - blogs, videos, news articles, and so on.

An API that models everything at the highest level of abstraction -  as /items or /assets in our example - loses the opportunity to paint a tangible picture for a developer to know what they can do with this API. It is more compelling and useful to see the resources listed as blogs, videos, and news articles.

The level of abstraction depends on your scenario. You also want to expose a manageable number of resources. Aim for concrete naming and to keep the number of resources between 12 and 24.

In summary, an intuitive API uses plural nouns and concrete names rather than abstract.

Next time, simplifying associations and hiding complexities using the HTTP ?.

RESTful API Design: nouns are good, verbs are bad »

In the first post in this series about pragmatic REST API design, I defined 'pragmatic REST' as looking at API design from the developer point of view. Now let's get into specific design practices we've seen work well.

The #1 principle in pragmatic RESTful design is: keep simple things simple.

Keep your base URL simple and intuitive

The base URL is the most important design affordance of your API.  A simple and intuitive base URL design will make using your API easy.

A design affordance is a design element that communicates how something should be used without requiring documentation.  A door handle's design should communicate whether you pull or push. But here's an example of a conflict between design affordance and documentation - not an intuitive interface!

My key litmus test for simple API design and pragmatic REST is:  only 2 base URLs per resource

Let's model an API around a simple object or resource (dogs) and create a RESTful API that interacts with dogs.

The first base URL is for collections; the second is for a specific element in the collection.

Boiling it down to this level will also force the verbs out of your base URLs.   

Keep verbs out of your base URLs

Many RESTful APIs start by using a method-driven approach to URL design. These method-based URLs sometimes contain verbs -  sometimes at the beginning, sometimes at the end.

For any resource that you model, like our dog, you can never consider one object in isolation. Rather, there are always related and interacting resources to account for - like owners, veterinarians, leashes, food, squirrels, and so on.

Think about the method calls required to address all the objects in the dogs world. The URLs for our resource might end up looking something like this:



It's a slippery slope - soon you have a huge list of URLs and no consistent pattern making it difficult for developers to learn how to use your APIs.
 
Use HTTP verbs to operate on the collections and elements

For our dog resources, we have two base URLs that use nouns as labels, and we can operate on them with HTTP verbs. Our HTTP verbs are POST, GET, PUT, and DELETE. (I think of them as mapping to the old metaphor of CRUD (Create-Read-Update-Delete).)  

With our two resources (/dogs and /dogs/1234) and the four HTTP verbs, we have a rich set of capability that's intuitive to the developer. Here is a chart that shows what I mean for our dogs.



The point is that a developer probably doesn't need the chart to grok how the API behaves. They can experiment with and learn the API without having to dig into the documentation.  

In summary: Use 2 base URLs per resource.  In your URLs - nouns are good; verbs are bad.

Next time:  how do you pick your nouns?

Please join the API Craft conversation on Google groups.

API Design: Are you a Pragmatist or a RESTafarian? »

I wanted to write some posts on RESTful API design best practices we've observed over the last year.   I covered a lot of these recently in a API design presentation but want to write more detail out.

First - let's start with our overall point of view on API design.

We advocate pragmatic, not dogmatic REST. What do I mean by dogmatic?   

You might have seen discussion threads on true REST - some of them can get pretty strict and wonky. 

 @mikeschinkel sums it up well - defining the RESTafarian. 

Our view: approach API design from the 'outside-in' perspective.

This means we start by asking - what are we trying to achieve with an API?

The API's job is to make the developer as successful as possible.   

Why? Look at the value chain below - the developer is the lynchpin of our entire API strategy.

So design the API to maximize developer productivity and success as the primary design principle.  

This is what we call pragmatic REST.

Pragmatic REST is a design problem.

You have to get the design right, because design communicates how it will be used.

So now the question is  - what is the design with optimal benefit for the app developer?

The design asethetic for APIs is to think about design choices from the developer point of view.

This is pretty much the guiding principle for all the specific tips that we'll dig into next.

Next time:   considerations in designing your base URLs.

“RESTful API Design: Second Edition”  Webinar - Video & slides »

Here are the slides and video for last week's RESTful API Design webinar, thanks again to everyone for joining.

Check out some additional tips and advice for designing Pragmatic RESTful APIs on the API Best Practices blog.

We'd love more thoughts or questions - there is a thread on the api-craft forum.  

Video and Slides: Your API Sucks!  Why Developers Hang Up and How to Avoid That »

Thanks to all that attended last week's API Best Practices Webinar #6 "Your API Sucks!: Why Developers Hang Up and How to Avoid That" (and thanks to our presenters @earth2marsh and @landlessness).

We apologize for the audio problems during the actual webinar - but Henry - our amazing filmaker - did a great job of cleaning it all up for the video below. 

Need to transact credit card information with your API?  Then check out our next API webinar, "Does your API to be PCI Compliant" with @brianpagano, our Sr. Architect, and @scottmetzger, our VP of engineering,  is July 14th at 11am PST (sign up here!)

(for our entire video series, check out the rest of our API best practices webinars)

Video and Slides: 10 Patterns in Successful API Programs »

Thanks to all that attended last week's API Best Practices webinar #3, 10 Patterns in Successful API Programs (and thanks to our presenters @gbrail and @landlessness). The video, slides, and Q&A is below.

Our next API webinar, "API Metrics: What to Measure" with @landlessness and @brianpagano, is June 2nd at 11am PST (sign up here!)

Questions and Answers from the Live Webinar 
 

Q: What is the best way to throttle/rate limit, software vs hardware?

A: If you’re talking about rate limiting at the API level, based on IP address, OAuth token, username, etc., then there are a bunch of good software solutions, including the Apigee Gateway. Some companies, like us, offer their software packaged inside a hardware appliance as well, but there’s nothing special that hardware can offer when it comes to this kind of rate limiting.

Q: Which OAuth do you recommend - 1.1, 2.0 or wait?

A: It’s hard to answer generally for all APIs, but the latest drafts of OAuth 2.0 are pretty mature, and include options that support a lot of use cases. A big advantage of OAuth 2.0 is that you can secure an API using just a security “bearer token” and SSL, or you can use a signature like OAuth 1.0 supported.

Q: What if I am in a big enterprise that has adopted SOAP as a standard for APIs, but I want to expose as REST.  How deadly, weird, etc... is protocol translation?

A: It’s not deadly or weird at all - lots of our customers are doing it. It’s very common to have internal services based on SOAP that meet the needs of the internal needs, but which don’t necessarily make sense to expose to the Internet in that form.  Putting a translation layer in front that can make use of protocol mediation and data transformation to make the SOAP API look like a REST API that supports JSON makes a lot of sense. The performance impact will be minimal unless you are talking about large data volumes (in the 10,000s of requests/second and above) and in that case you can horizontally scale if you need to throw CPU at the problem.

Q: Does it make sense to provide some kind of "notification" to users, which would  inform that certain part of the data is changing?

A I think it depends on the API! Twitter is a good example (I realize that we keep using them) -- they started with an API that requires you to poll, and they evolved to also offer a streaming API that pushes data out with less latency. But keep in mind that any kind of streaming or push API, including a full streaming API like Twitter, Webhooks (a little less complex), or Web Sockets, is going to be more work for a developer to adopt -- if you choose to offer this it should be in addition to something simpler to use that might offer slower performance.

Q: Does Apigee have insight to Cloud Computing Providers exposing API and the adoption of that by users (both individual consumers and businesses)?

A:  In general we’re seeing that every cloud computing provider is offering an API at some point, and that they’re getting used. I’m not sure that we have any more specifics...

Q: Do you have good example of content API ?

A:  We know of some great APIs in the works that are coming out from some leading media providers. In the UK, the Guardian newspaper is a good example of a content API.

Q: Do you recommend any framework (Python or PHP) for a content API ? 

A: There are many and we should do another webinar on this topic. I (Brian) personally like Ruby on Rails because you will get an excellent RESTful API design for free. There are frameworks for Java like JAX-RS. And for PHP there is FRAPI.

Q: Our API is a paid product. Does it make sense to make it free for developers?

A: You should at least expose some of the API methods for free - also known as a freemium model. It’s a really good way to promote adoption. Related to that it’s also good to have API methods that do not require any authentication - usually a simple read-only method.

Q: Is anyone familiar with open source rate limiting software?

A I know that some of the big guys like Twitter build frameworks on top of memcached -- it’s horizontally scalable and it has an atomic integer increment operation that works well.

Q: Do you recommend any developer community for mobile telecom services, like SMS, billing, etc?

A: Twilio has a good developer community with meetups. Also check out what AT&T is doing. They are starting to put together a solid developer community.

Video: RESTful API Design (Pragmatic, not Dogmatic) »

Recently Brian Mulloy (@landlessness) and Marsh Gardiner (@earth2marsh) hosted a webinar on API design and Pragmatic REST. The video of the recording and the slides are below.

Our next API webinar - 10 Patterns in Successful API Programs - is this Thursday, May 19th at 11am PST, with Brian and Greg Brail @gbrail. Interested in the topic? Then you should sign up now!

RESTful API Design:  Webinar slides and recording »

Thanks to everyone that attended yesterday's RESTful API Design Webinar (and thanks to our presenters @earth2marsh and @landlessness).

The slides are below and here is the full recorded webinar.   (We'll also post a video next week.)

Our next API webinar is "10 Patterns in Successful API Programs" with @landlessness and @gbrail, May 19th at 11am PST (sign up here!) 

API RESTful Design question:  Versioning number in the URL? »

We're starting to get some great questions from signups for next week's API design webinar:  "Pragmatic REST - API Design Fu". 

Here's one we'll comment on now and then talk about more next week.

Q: "For API Versioning.  Doesn't seem very RESTful to version a resource.  Can it be avoided by using implementation and specification versions described in the Sun API Cloud Documentation?"

A: Your question is an insightful one. Versioning is an open issue for REST API design in practice.

Clearly, an API will need to change it's behavior over time. As it changes application developers will need to change their code accordingly. So, versioning is inevitable.  

(caveat: Dan Jacobson of Netflix and NPR's API program has great points on what he calls a 'versionless API' in this presentation around slide 202.")

Our point of view here at Apigee is API design approach we call pragmatic REST.

The driving philosophy of our approach is: what's good for the application developer is good for the API. The Sun Cloud approach to versioning is interesting. The upside is it's powerful and will handle cases of inter-dependent API calls well.

The downside is it's complicated and as a developer starts using it, there is a lot to learn. Plus, because the versions are specified in the header, he won't be able to copy & paste it into the browser to learn it.

Our suggestion - even though it clearly violates pure REST theory - is to include the API version in the URL as a single digit number like /v1, /v2, etc. Whatever complexity is involved in making it "work" is on the shoulders of the API team.

More on this topic in our webinar - sign up here and hope to see you there.

Universal Design Principles Applied to APIs (Part 2 of 4) »

Following Part 1, here are the slides and a video for Part 2 of our series on applying universal principles of design to APIs.  In this part I cover 3 more universal design principles, including:

• Flexibility-Usability Tradeoff
• Hick’s Law
• 80/20 Rule
• Inverted Pyramid

And for more on API Design check out Teach a Dog to REST and REST for SQL Developers

Build & Run a Web API for FREE »

With the proliferation of free, cloud services it's possible to build and run interesting mobile and web projects from end-to-end for free--including an awesome web API.

Here are 10 steps to building and running a web API for free.

1. Prepare

Get up to speed on APIs - from their economic and business model rationale to design best practices and principles to developer community practices.

 2. Design

Prioritize your resources (objects) in a Google spreadsheet and start to craft your URLs and HTTP methods. 
 

3. Build

Build your API quickly using Ruby on Rails scaffolding.  And here are some additional Rails API tips from @StefanSiebel.

Store your source code in GitHub.

 4. Deploy

After you create your Rails-based API, deploy it to Heroku.

5. Proxy

Use Apigee's Free service for API analytics, debugging and rate limiting.

6. Document

Document your API using GitHub pages.

7. Engage

Engage developers by embedding an API Console into your docs using Apigee To-Go.

8. Launch

Grab a domain name for your project from a DNS provider like GoDaddy.

Setup CNAME records for:

• GitHub docs: dev.mydomain.com
• Apigee proxy: api.mydomain.com

9. Promote

Use Twitter, Blogger and Facebook to promote your API.

10. Support

Support your community with GetSatisfaction and manage tickets with GitHub's Issues feature.

A couple of us (@earth2marsh & @landlessness) recently built an API with this set of products and had a lot of fun doing it. We'll talk more about that project in a future post.

How APIs encourage the separation of content and presentation »

From the dawn of the web, we've been entangling presentation with content. Which is, overall, a big suck. Why? Well, on one hand, it's fine for people to consume. But it's really rude to computers. And computers, after all, are doing all the work to keep the Internet running.

If computers can't easily parse data, they can't do anything meaningful with it. However, once computers can easily disentangle the layers of data, watch out: innovation happens like crazy.

In a series of blog posts I'm going to talk about all the beauty of separating content from presentation. And how that simple idea when amplified with proven, existing design patterns makes things really interesting. Let's start with Twitter as an example...

When you view tweets on twitter.com, you see shared links, hashtags, and @mentions, all of which are clickable. But if you query the Twitter API you get back plain text. Why?

Because Twitter understands the importance of separating the content layer from the presentation layer. Imagine seeing this on a phone:

Ran into <a href="http://twitter.com/shanley">@shanley</a> at <a href="http://twitter.com/#!/search?q=%23sxsw">#sxsw</a>!

… because the SMS app on that old phone didn't understand HTML! Furthermore, when every character counts, like in a 160 character SMS message, markup would end up dominating the content. You shouldn't even assume that the consumer device will even know what to do with it.

So instead, Twitter keeps statuses as simple as possible—text only. This is plain, vanilla content. No markup, just data (well, unless you count the "organic" markup conventions like hashtags and @mentions).

Everything else, all the rest of the CSS and HTML that make up the twitter.com experience—that's the presentation layer. It takes all that content and makes it look good.

APIs, at their most basic, are carriers of content. They are perfectly evolved for getting data from one place to another.

And this is super-important in today's world, where we consume content on multiple devices. One size no longer fits all. There is no single form-factor. Each device offers a different experience of the same content.

This is just one example why APIs are changing the way we develop, by demanding the best-practice approach of separating the content and presentation layers. Modern web applications use the same APIs that their official mobile apps do. And those are the same APIs that get exposed to 3rd party developers who innovate around the edges.

REST API design for SQL programmers »

@gbrail put together this short deck mapping SQL concepts to (pragmatic) RESTful API design.  And for more on API design - see Teach a Dog to REST and Universal Design Principles Applied to APIs

REST API Design for SQL developers

Why XML won’t die: XML vs. JSON for your API »

Last week I wrote that if you're API doesn't support JSON and JSONP - you're doing it wrong.   I don't think that's terribly controversial.

But is JSON (and JSONP) perfect for everything you need to support with your API?  Is XML dead?

JSON is especially good at representing programming-language objects. If you have a JavaScript or Java object, or even a C struct, the structure of the object and all its fields can be easily and quickly converted to JSON, sent over a network, and retrieved on the other end without too much difficulty and (usually) comes out the same on both ends.

But not everything in the world is a programming-language object. Sometimes to describe a complex real-world object we have to combine different descriptions and languages from different places, mash them up, and use them to describe even more complex things. The descriptions of these complex things need to be validated, they need to be commented on, they need to be shared and sometimes annotated with additional data that doesn't affect the original structure.

When the world gets complicated and open-ended like that, what's needed is not a programming-language-format object, but a open-ended, extensible -- umm -- markup language. That's what we have today with XML.

For instance, the travel industry (through the Open Axis Group), the insurance industry (through ACORD) and the financial services industry (through FpML) have all spent many person-years developing standards that describe what they do in XML format. Each standard comes complete with a schema, which means that any client or server can validate a document to ensure it is correct enough before starting to parse it, and which makes it easier to edit the document using one of the many of the mature tools that are available.

Sure, parsing and understanding these documents is not simple, but they do not represent simple things. The ability to represent a complex travel itinerary, a life insurance policy, or an interest-rate swap in a standards-based format is a big deal and a triumph of XML technology.

Similarly, look at HTML. (Most HTML is not XML but both come from SGML and are very similar.) HTML works because it can combine both structured and unstructured content in various ways and accept the ability to mash up different standards into one document.

In my opinion, XML will only be dead when the web has replaced HTML with JSON.

So for our APIs, let's embrace JSON -- it's small, simple, and easy to use. But when we have to collaborate on complex documents, pull information from different places, and define complex schemas to represent complex real-world concepts, let's also not forget about good old XML.

Universal Design Principles Applied to APIs (Part 1 of 4) »

The slides and video for Part 1 follow:

For more on API Design - see another presentation and video "Teach a Dog to REST: RESTful API Design Principles" 

We'd love to hear what you think - reach me at brian at apigee or @landlessness

Crossing the Streams:  Handling cross-site API requests (JSONP, CORS, UMP and more) »

My earlier post "Not using JSON and JSONP? You're doing it wrong!"  generated a lot of questions about the best ways to handle cross-site API requests from JavaScript running in the browser.

(Short intro -- the browser won't let you do it with the standard XmlHttpRequest API because of the "same-origin policy." This policy is in your browser's JavaScript engine so when you hit an application at http://www.foo.com, there's no way for http://www.badguys.com to return bad data to your application.)

This is an important topic, because if you are providing an API at api.yourapi.com, no one can use your API from inside a browser unless you implement one, and preferably more than one, of these techniques. It's also important because the state of the art is changing and changing fast.

Here's a run-down of the various techniques today, and how you might choose,  with the inevitable links to Wikipedia to help guide you: 

JSONP:

What it is: You return JSON from your API, and wrap it with a user-supplied "callback" identifier. This turns your JSON response from a JavaScript object into a function call or an assignment statement. Your user uses the DOM API to dynamically insert "script" tags into the web page to cause your API to be invoked, then executes the result as a JavaScript function call to create the object and start using it. 

Why it's good: Works on every browser, and asynchronously like every JavaScript network resource to boot. If users use a library like jQuery it is very easy to use.

Why it's bad: Only works with JSON,  although if you read my last entry then your API returns JSON anyway! It only works with GET, so you have to pollute your beautiful and philosophically-correct REST API by adding something like an "action" query parameter -- this tells your server that even though it just got a "GET" call, it should behave as if it were a POST, and you can't upload data with GET.

Finally, JSONP is really a hack around the security sandbox in your browser. As a result, it lets your JavaScript code download executable code from  another domain and run it inside the browser. If the user of your API trusts you, then that's great, but otherwise they're placing the security of their web app in your hands.

Aside from offering JSON as a response format with callbacks, you can also specify XML as the response with JSONP-X.

CORS: 

What it is: Cross Origin Resource Sharing is a W3C spec that allows a web service to specify that it's OK for it to be invoked from any domain. The API provider implements it by returning a special set of HTTP headers with every API response, and also by implementing an HTTP OPTIONS "pre-flight" request that lets the browser check first to see if a request is going to be OK. The actual spec is awfully complex but what you actually need to know in most cases isn't all that bad.

Why it's good: Because it lets the API consumer easily say, "it's OK, browser -- go ahead and make that API call." Once it's implemented, browsers that understand the CORS spec will automatically open a hole in the cross-origin policy as part of the standard XmlHttpRequest API -- the programmer of the web app doesn't have to do anything special. Plus, it can return any kind of data, not just JSON, and it works with all HTTP verbs. 

Why it's bad: It's pretty new and not all browsers implement it. Firefox 3.5, Chrome 3, and Safari 4 has it. IE8 does too but using a slightly different API. And support in many smaller mobile browsers (like Opera) is sketchy or nonexistent.

In other words, CORS is super-duper awesome as long as you don't care about people who run IE -- whether that's important to you is up to you

UMP:

What it is: UMP is a re-boot of CORS -- it handles 90 percent (or more) of the reasons an API would want to implement CORS without 90 percent of the complexity. It's also backward compatible in that it uses many of the same HTTP headers as CORS.

Why it's good: Like CORS, it allows an API to easily make itself available to web browser clients with a minimum of fuss. Better yet, all the API has to do is set a few HTTP headers, without also implementing an OPTIONS request.

Why it's bad: It's even newer than CORS. (New enough that it doesn't even have a Wikipedia page.) I have been told that recent versions of Chrome support it but it's not clear where else it can work at this point.

There's more to compare CORS and UMP here

Cross domain files

What they are: Flash and Silverlight also have a same-origin policy, but they do not work the same as the JavaScript engine and the techniques above do apply. What they do support are special files, namely crossdomain.xml (for Flash) and clientaccesspolicy.xml (for Silverlight). These function the same way as CORS and UMP headers in that they tell the client when it's OK to invoke the API across domains.

Why they're good: If you have Flash or Silverlight clients for your API, you need these too so that your API can be invoked from  those environments.

Why they're bad: If you don't want or need to support Flash and Silverlight clients then you don't need them.

So which one should I pick?

 If you want your API to be adopted in as many places as possible then you should implement all of them! Seriously:

1.       Always make it possible for your API to return JSON.

2.       Support JSONP as much as you can (since you can't use it to upload data that doesn't fit in the URL) and leave it up to the users of your API to assess the security risks

3.       Support CORS by returning the Access-Control-Allow-Origin header on all API responses and implementing the OPTIONS verb as specified in the CORS spec.

4.       If you do the above your API will also support UMP.

5.       Return a crossdomain.xml and clientaccesspolicy.xml file from the top level of your API domain for Flash and Silverlight clients.

Whew!

At Apigee, we have done the above for users of our Apigee Enterprise product, and since it's so flexible we can change what we support as the standards change.

Not serving JSON AND JSONP? Then you’re doing it wrong! »

If you've used an API recently, you've probably seen that the popular APIs out there support JSON. JavaScript Object Notation is a standard defined a while back by Douglas Crockford from Yahoo. It uses a subset of the JavaScript syntax to simply and effectively describe an object.

In the last few years, JSON has taken its place alongside XML as the de facto way to describe API data. Today's leading APIs support JSON in addition to XML, and an increasing number support only JSON.

JSON is popular because it's simple. Programming-language objects map to and from JSON in a straightforward way that everyone can understand. You need a "JSON parser" to convert JSON into an object (unless you're working in JavaScript) but you don't need to know much about it other than how to make it go.

If you are thinking of building an API, JSON support is critical. Here's why:

JavaScript. JSON is JavaScript. That is, a "JSON object" is literally a small fragment of JavaScript that represents an object and its sub-objects. That means that creating a real JavaScript object based on some JSON text is simple and fast. Web programmers love JSON.

JSONP. This lets JavaScript running inside the browser invoke APIs that reside on a different host on the Internet. This doesn't sound like a big deal but it is actually huge because all browsers implement a "same-origin policy" that otherwise makes this impossible. JSONP is hard to implement, but libraries like jQuery make it easy for the client if the server already supports it.  If you're not serving JSON AND JSONP you're doing it wrong!

Smaller. Smaller is better, especially in the mobile environment, and since JSON doesn't "say" every field name twice like XML does, JSON output is a lot smaller than XML.

Less complicated. JSON is free of namespaces, attributes, multiple "text" nodes, and other complexities of XML. The result is that JSON parsers exist for every language, they're small, and they're fast. Furthermore, if you need to write your own, it's not complicated. The same goes for security -- all that's necessary to prove that a JSON document is valid JSON is a simple regular expression check, which is easily available in nearly every programming environment.

Tools. An increasing number of tools support JSON. JSON support is not ubiquitous yet, but at the rate JSON is gaining it will be soon.

Many APIs now support XML and JSON - like the Twitter API, where JSON is the default. Some APIs support only JSON (like Foursquare's V2 API).

But JSON isn't  for everything. Next up: Why XML isn't dead yet!

How do content and transactional APIs differ? »

Recently, during one of our our RAW (Rapid API Workshops) with a retail customer, a great question came up - what are the major differences between a content and transactional API?

Probably not a complete list, but in general:

Content APIs are more likely to be open, without sensitive information. Think of a search, media, or mapping API.    While the provider might want to track identity through API keys, these APIs often need no authentication, authorization, or encryption.    Search results may be highly cachable, which might be helpful to support high concurrency for bursts of demand for popular content.    Content APIs are also more likely to need throttling to protect the back-end and quotas to measure consumption - think about that grad student downloading your entire database one API call at a time.   Users might have some tolerance of downtime for free content that can easily be requested again.      Success for content APIs might be measured in terms of usage or engagement. (usage per consumer), so having  API usage analytics might be important.   If you can, make content APIs simple and easy to adopt with standards like REST.   

Transactional APIs  have sensitive data and therefore security needs go beyond identity and developer key level tracking  to include API authentication and authorization.   The data might need encryption and XML or API specific threat protection.    Instead of quotas, the back-end business logic might already contain all the controls you need to measure consumption and monetization.    There is probably no tolerance for downtime or lost transactions.   And of course success for transactional APIs can be measured in existing financial terms.

What's your experience in the difference between content and transactional APis?

Why your sweet new feature needs an API. Now. »

Over the weekend, Twitter developer Alex Payne posted a tweet (since deleted) which sent such waves of speculation rippling through the Twitterverse that even Techcrunch felt the need to comment. Of course I'm interested in seeing the features that Alex wrote about, but I'm even more interested in Al3x's follow up tweet that noted, "We still release most everything API-first, of course."

Twitter, as much as anyone, has depended upon their API to drive adoption. This is the future of web services. The web IS the common ground, but as the web moves beyond browsers into devices like mobile phone apps, APIs are what allows it to happen. When you consider that 60% of all eBay listings are made via their API, then you have to appreciate how important APIs are.

If you want your developers to succeed, you have to empower them. Until you release most everything API-first, you aren't giving your developers the tools they need to push your platform forward. Take LinkedIn, for example. Their search API docs suggest that you cannot search LinkedIn by email address or phone number, even as an authenticated user. 

An email address is critical for disambiguating users. Imagine that I use a CRM system, like Highrise, and I want other users to be able to authenticate with LinkedIn to see who in the company might have a LinkedIn connection with a contact. What if that contact has a common name, and the only other information entered is an email address? The ability to identify specific people to see how you're connected is a primary use of the LinkedIn network. And yet third parties don't have access to these fundamental methods. Without this, their digital hands are tied. 

What's hot: releasing most everything API-first
What's not: leaving core functions out of any API

An API at Launch: Reflecting on the Google Buzz API »

Remember when videogames used to be released on different platforms at different times? One platform like the PlayStation version would lead, while other versions trailed behind. Then at some point, publishers realized they could make a bigger splash by landing all versions in stores at once—the simultaneous release.

Today on the web we expect an API to lag the launch of a new webservice. So it was interesting to see Google Buzz tout its API on day 1. On one hand the shiny new Buzz API is not much more than a stub, though on the other it is a promise to deliver a robust API.

But at this time, you can't even post an update to Buzz via the API. You also can't mute a buzz. You can't even search. It's read-only; all you can do with the API is get the Atom feed for a user's Buzz, which doesn't include important metadata like location information. You could argue, as Fred Wilson has, that any read-only API shouldn't even qualify as an API.

There ARE things to like about the promise of the Buzz API. It's a stake-in-the-ground to support existing standards like PortableContacts, the Social Graph API, and OAuth. Important protocols like PubSubHubbub, Salmon, and WebFinger should get a good boost too. Yay for open standards!

But there are some concerns too. In order to build on top of the Buzz API, you may have to become familiar with the disparate set of technologies listed above. And if you prefer JSON or raw XML responses to Atom, you'll have to wait (FWIW, format conversion is one of the many tricks that Sonoa's ServiceNet does right out-of-the-box).

What could Google do to encourage Buzz innovation? After all, isn't that a big reason why APIs get released? They could wrap the various standards they plan to support with a look and feel similar to the full Buzz API (whenever it lands). They could emulate the Twitter API, like Wordpress and Tumblr have done, but that brings up issues of licensing—something that Googler Dewitt Clinton has pursued with Twitter's Ryan Sarver. Still, that kind of familiarity would only drive Buzz adoption with developers already versed in Twitter. In some ways Buzz resembles FriendFeed more than Twitter. So why not map the FriendFeed API pattern onto the Buzz API, as Dave Winer has suggested (again not without issues)?

Not having API in 2010 is like a not having a website in 1997. How long will it be until everyone launches both of them at the same time?

(logo by Alex Gillis)