RESTful API Design: API Virtualization »

Last time, we looked at why you might consider complementing your API with an SDK or code libraries. In the series so far, we've covered a lot of tips and tricks for designing pragmatic RESTful APIs. 

You may be asking -  

How do I follow all these best practice guidelines and still maintain and iterate my APIs? 

What should I be thinking from an architectural perspective in terms of implementing these best practices?

Add an API virtualization layer 

I recommend you give yourself a buffer or virtual layer between the interface on top and the API implementation on the bottom.

The app that consumes the API is on top. The API virtualization layer isolates the application and the API. Make a clean design for the app on top and turn the problem into an integration problem.

 In other words, work at integrating your design on top of the key APIs through the virtualization layer.

Don't start in code, and try to build up from your business logic to a clean API interface.

API Virtualization »

We've seen an architectural pattern emerge over the last two years that deals with the expanding variety of computing devices and the exploding number of APIs used in modern applications. This pattern is API Virtualization - applying a virtual layer above the API itself to deal with the different concerns that come into play when delivering an API to different device and machine endpoints as well as across a range of different classes of business relationships.

I gave this presentation at GlueCon as a Prezi. The prezi is available here, and I've put the audio with it in the screencast below.

One of the big shifts that has occurred in application development is the move from libraries to APIs for use of pre-written code. APIs enable access to not just logic but data and the the computational power required to render that data appropriately. The Facebook and Twitter APIs are good examples of this, but there are many more including Paypal's X.com, Ebay, Amazon, Netflix, Sears, Blockbuster, Best Buy, MTV and Google.

While applications like Siri which use 20 or more web APIs are still unusual, most applications use several APIs to deal with access to content, social networks, events, identification, payment, and other concerns. These APIs change frequently, as do the applications that consume them, resulting in a web of dependencies and causing chaos in the development lifecycle.

API Virtualization enables these different concerns and dependencies to be isolated from the application or API provider, resulting in a cleaner development process and higher-performing applications.

API Versioning Hell? Think API Virtualization »

We've seen an architectural pattern emerge over the last two years that deals with the expanding variety of computing devices and the exploding number of APIs used in modern applications.  This pattern is API Virtualization - applying a virtual layer above the API itself to deal with the different concerns that come into play when delivering an API to different device and machine endpoints as well as across a range of different classes of business relationships.

Sam Ramji gave this presentation at GlueCon as a Prezi.  The prezi is available here, and we've put the audio with it in the screencast below.

One of the big shifts that has occurred in application development is the move from libraries to APIs for use of pre-written code.   APIs enable access to not just logic but data and the the computational power required to render that data appropriately.  The Facebook and Twitter APIs are good examples of this, but there are many more including Paypal's X.com, Ebay, Amazon, Netflix, Sears, Blockbuster, Best Buy, MTV and Google. 

While applications like Siri which use 20 or more web APIs are still unusual, most applications use several APIs to deal with access to content, social networks, events, identification, payment, and other concerns.  These APIs change frequently, as do the applications that consume them, resulting in a web of dependencies and causing chaos in the development lifecycle.

API Virtualization enables these different concerns and dependencies to be isolated from the application or API provider, resulting in a cleaner development process and higher-performing applications.

Tradeoffs in XML data transformations »

Daniel Jacobson of NPR posted a fascinating piece about how NPR tackles a common problem – what’s the best way to render content on a variety of devices, from modern web browsers with top-notch CSS implementations that look almost like typesetting (like Safari) to mobile phones using WAP to low-end devices like HD Radio receivers that don’t understand anything but plain ASCII text.

NPR’s clever solution is to strip markup out of the text and store it in a database table, indexed by position in the text document. To re-generate the content for a particular device, their software queries the database and re-applies the markup tags to the content according to what device it is rendering to.

This takes me back to the original reason SGML was invented and made an ISO standard in 1986. The idea was to describe the semantic meaning of text, and then to let a computer program figure out how to render it for human consumption.

SGML was a little over-engineered for that purpose, however, so a bunch of smart people got together in 1996 and invented XML. XML then begat technologies like HTML, XSLT, and CSS.

So today, instead of writing something like:

<h1 class=”headline”>This is a headline</h1><p class=”byline”><b>By I.M.A. Reporter</b></p><p class=”paragraph”>And here is my first paragraph with something in <i>italics</i>.</p>

XML lets us write:

<main_headline>This is a headline</main_headline><byline>By I.M.A. Reporter</byline><p>And here is my first paragraph with something in <i>italics</i>.</p>

The difference is that my second example isn’t HTML – it’s part of a document that uses an XML schema that’s up to me, and when writing it I don’t care if I’m coding for an HTML browser or for a car radio – I just have to identify when I’m writing a headline, or a byline, or a caption, and so on. I can now use XSLT or another transformation technology to transform this XML into very simple HTML for a simple browser, or into very complex HTML with links to a CSS stylesheet for a more sophisticated browser, or just into plain text. And if I decide that part of my XML schema should look just like HTML (like I did above with the “p” and “i” tags) then that’s fine too.

Other approaches and tradeoffs

NPR’s approach has a lot of benefts. Depending on your business and situation, this might mean lot of database processing, which could to be expensive to scale in either licenses or capacity.  Caching helps a lot in this case, since once is content there’s no need to do it again.

You could also solve this problem by writing the original content in very simple HTML or XML (in whatever schema one desires) and then by using something like XSLT to transform the content for each input device. This solution might be CPU-intensive but might compare favorably vs. database operations depending on what you are doing. Plus, XSLT processing can be easily scaled across thousands of parallel nodes if necessary without buying any more database licenses.  

If development resources and cycles are the constraint, a dedicated policy layer can help.  In the case of our Sonoa ServiceNet technology - you could configure transformation policies that leverage XPath or XSLT from within our proxy.   This might also make it easier to add and validate 3rd party APIs or feeds from outside your own database.   You can also handle other types of mediation such as versioning or protocol transformations, if that is in your use case, such as some of our Sonoa media and consumer web services customers do.

TrueCredit.com API case study »

Scott Metzger, CTO of TrueCredit.com was kind enough to take some time to talk about their Consumer Connect API program and some of the technical challenges that they have addressed using Apigee's API Gateway.

Scott wanted to make life easier on his development team as they ramped up their number of APIs, partners and traffic volumes.  Here, he describes how he uses the technology as a 'policy layer' to provide API analytics, fine-grained data protection, and caching in an API Gateway. In this case, Apigee Enterprise is deployed on-premise virtualized software.

We're very excited to be working with Scott and TrueCredit, and check out the full TrueCredit Case study.