We've seen an architectural pattern emerge over the last two years that deals with the expanding variety of computing devices and the exploding number of APIs used in modern applications. This pattern is API Virtualization - applying a virtual layer above the API itself to deal with the different concerns that come into play when delivering an API to different device and machine endpoints as well as across a range of different classes of business relationships.
Sam Ramji gave this presentation at GlueCon as a Prezi. The prezi is available here, and we've put the audio with it in the screencast below.
One of the big shifts that has occurred in application development is the move from libraries to APIs for use of pre-written code. APIs enable access to not just logic but data and the the computational power required to render that data appropriately. The Facebook and Twitter APIs are good examples of this, but there are many more including Paypal's X.com, Ebay, Amazon, Netflix, Sears, Blockbuster, Best Buy, MTV and Google.
While applications like Siri which use 20 or more web APIs are still unusual, most applications use several APIs to deal with access to content, social networks, events, identification, payment, and other concerns. These APIs change frequently, as do the applications that consume them, resulting in a web of dependencies and causing chaos in the development lifecycle.
API Virtualization enables these different concerns and dependencies to be isolated from the application or API provider, resulting in a cleaner development process and higher-performing applications.
At some point most API product managers are on the hook to demonstrate how the API results in downstream revenue.
And even if you never intend to charge for your API, you may be surprised by unexpected opportunities.
I used to be responsible for a number of ‘free open, take-it-or-leave-it APIs’ at a large Web portal. After a few months, the team was surprised to get a ton of calls from big companies that wanted to pay (a lot) for use of the API within their commercial products.
One small detail - they expected that the API could support some of the basic business terms of any commercial deal. For example – could we:
Meter service and give them a bill every month?
Enforce a special rate limit give them an “SLA” (service level agreement)?
Insert extra data fields if they were willing to pay for more licensed content?
We also needed some other basics:
SOX compliant audit logs for each deal.
Reporting on the cost of the data served by the API. (because we were licensing it from another company)
The means to enforce similar terms on a ‘package’ of multiple APIs together in one deal
Unfortunately, all of these needed to be coded as they came up, so it was a big strain on development, PM and BD to tough to lock down and execute deals in time.
So – even if your API business model is already covered and baked in your API – you may want at least the option to support a slightly tailored or customized relationship. You may need to quickly tailor the APIs syntax, protocol, priority, or content to a specific opportunity.
Once you have partners using an API, your business managers and partners will put your team under a lot of pressure - dollars are lost each day a business policy change takes to make.
So consider if you’ll ever need to abstract these policies – the difference between the API content and features that and the commercial business and operational terms of each deal – in an a separate API policy layer that you can manage quickly and independently of development cycles. Or to make a single change across multiple APIs. (here's a case study on an API policy layer.)
Therefore, some questions to ask for your API roadmap might include:
General business and partner model questions:
How is your business and revenue model supported by your API? (are you looking for distribution and awareness only? Does the API drive a monetized transaction? Will you ever charge for ‘pay by the API drink’?)
How are your costs reflected in your API? Do you pay for any of the data you are serving up? If so, how do you keep track of this for the business and partner?
Will large partners or deals surface where your team will need to change the API content, SLA, protocol or content? Is there a partner that might want to pay enough and who is large enough to drive your team to move a way from “one size fits all?”
Will you need to create ‘service tiers”?
Enforcing unique business and operational terms
How will you meter service like a utility, so that you can bill partners and report data costs?
What regulatory compliance will you need to support? Do you need SOX-compliant audit trails by partner? HIPPA? PCI?
How would you create and enforce a partner specific SLA, rate limit, or offer ‘priority access’ to a priority partner?
If the partner wants any change in the API protocol or content – do you need to support a different API code-base? Is there a way to create a transformation layer to handle and scale this?
Do you need to buffer up or queue incoming requests?
TrueCredit.com tells a story of calculating they would need thousands of IP addresses for all the different versions and flavors of their open API - to account for different variations and versions needed by partners.
Even if you have a ‘one sized fits all’ API - you might need to be able to transform data, mediate terms or customize SLAs without coding each change or a creating a new version of the API. Reasons could include:
• Protocol needs - A SaaS customer with a REST API had an important deal on the table with a bank, but the Bank insisted on a SOAP API with WS-Security. Some SOAP shops want to offer a RESTful API because it’s easier for developers to work with. And you might need to transform between different syntaxes of SOAP, REST, or REST/JSON, etc... • Monetization – You might want to sell a premium version of your ‘one size fits all’ free API. For example, a search API provider wanted to do a BD deal with it’s free API and needed to insert extra data ('enrich the API' as they called it) the partner wanted to pay for. • Standardization – A customer of ours grew from offering 1 API to 40, and needed to add some standard fields to each API - enforcing some consistency without needing to coordinate a bunch of teams to write code. • Versioning - Ever used an API where you get an email every month asking you to upgrade to a new version? TrueCredit wanted to provide API upgrades to customers that needed it while holding the API fixed for everybody else longer, to reduce versioning headaches.
So you may need to figure out how you to provide and manage different flavors or versions of the same API – or ‘mediate’ (or transform) API content and syntax.
Alternatives might be to support multiple APIs (painful), hold off as long as possible and push back on customers to snap to a ‘one size fits all’ model (more painful), or create a ‘mediation’ capability or layer that can transform between different ‘shapes of the API – protocol, data, version, credentials, etc.
(And going back to TrueCredit’s story at the top, this is what led them to think about an API gateway for mediation, caching, load balancing, and more.)
So ask if and when any of these issues might apply to your roadmap:
Will you need to mediate protocols?
Will you need to offer more than one protocol or a different protocol? (SOAP for enterprise customers? REST or JSON for developer adoption? )
Would you ever need to map across different security or credential schemes? (ex: from simple HTTP auth to WS-Security)
Will you need to handle syntax issues across a particular protocol (SOAP 1.1 vs. 1.2, etc.)
How important will it be to minimize API versions?
How important is version management?
How often will you need to release upgrades to the API? What is your process for asking customers to upgrade and how long will it take to sunset a version?
If you offer more than one API, any need to standardize elements of the API (header or payload)? Do different teams need to do this or does it make sense to put this capability at one point?
Will there be a need for payload transformations?
Will you ever need to enrich an API for a particular customer or class of service? (such as a big customer that licenses more data..)
Will you need to remove or clip certain fields for certain customers or classes of service?
How fast will you need to turnaround these requests for the business vs. your dev or product cycle?
A mediation layer (see more flavors here) can be important to handle complexity so you can focus development on business specific API capabilities.
(continuing our series on