Today, Twitter API developers have a new API Testing console that provides a way to explore and share Twitter API testing results via URL with other developers (see and example here).
And the new API Debugger gives developers API message capture, record, and playback for any API that is proxied with Apigee. You can share your debug snapshots with other developers via URL.
A couple other changes: we've removed our 'private preview' invitation requirement - anybody can sign up and get started now. And we've increased our free traffic limit from 10K to 50K per hour.
Finally, if you've used Apigee before, check out our new proxy creation flow.
"Open and Free" are often used interchangeably when talking about APIs. But if you look at in terms of the "API Economy" - open and free API are orthogonal.
'Freedom' in the cloud is more typically discussed as your rights to move data from one service to another. Application portability has also been raised as important but current application models are both early in their maturity curve and vary greatly.
Open and free are both crucial attributes in order for a market economy to grow. There are many aspects of cloud computing but for the developers and users of cloud services, the atomic unit of the cloud is the API.
The link between openness and economic growth is a deep subject that may provide clues for how to build a better cloud – this paper makes me think a company’s APIs might represent the “export goods in which it has a comparative advantage”.
In the software industry open platforms have typically outperformed closed platforms in the long run due to the economies that develop on top of them, cementing those platforms’ place in a range of markets.
Open APIs are:
1) Openly documented 2) Available via self-service (i.e. developers can sign up and get a key on a website) 3) Using open technologies (SOAP, REST, RSS)
A good example is bit.ly - a simple URL shortening service that also lets you see how many people have clicked on your shortened version of that URL. It’s really useful if you want to both project important articles on the web and understand the reach of your projection. Is it an Open API?
Test 1: Bit.ly's API is openly documented (here) Test 2: It's available via self-service. You can get an account and a key right away. Test 3: It uses open technologies: It's a REST API (granted, it is a mix of verbs and nouns)
Open APIs lead to 3rd party innovation
Tweetdeck users put their bit.ly API key into Tweetdeck, which automatically uses their bit.ly account (indicated by the API key) to shorten URLs that are typed into tweets. It made Tweetdeck better and probably increases the traffic to bit.ly.
This could also work on the iPhone application of Tweetdeck but it’s not yet implemented in the version I have. Many iPhone applications work use one or more cloud APIs that provide access to services in a clean, machine-friendly way.
From the efficiency of innovation perspective, keep in mind that Twitter most likely never contacted Tweetdeck to use their API, nor did bit.ly (as far as I know). The Tweetdeck guys simply built a killer application that uses those services via APIs rather than scraping their web sites. In the last month, Tweetdeck also added Facebook and MySpace support via their APIs.
Open APIs lead to innovation, efficiency and reach through designing your core business service to be “remixed” is found through APIs – Tweetdeck users got new value through the app they like and Facebook and MySpace got a new stream of user-driven content, all without sales or business development teams engaging at the outset.
Next time: Free vs. Paid APIs...
(Sam Ramji is the VP of Strategy for Sonoa Systems. Previously, Sam drove many of Microsoft's Open Source Initiatives)
Last time we showed how Apigee calculates API response time. This time we want to discuss how Apigee calculates API error rates.
Apigee provides charts and tables with API error rates by day, hour or minute with drill downs by API URL.
The 'error rate' calculation itself is simply # of errors divided by # of requests. Errors include both any Apigee error (including those that you might configure) or any target error (such as a rate limit error from the Twitter API).
Let's show this with Brian's Twitter Growl mashup. Twitter Growl provides alerts on topics using Mac's Growl alerts. For example, Brian gets alerts for Tweets on Detroit and Michigan news.
Brian monitors Twitter Growl API activity using Apigee. You can see he had some stretches in mid August and early September with high error rates.
You can also drill into the days or hours where these spikes in errors occurred, such as this 3 day period in September - high error rates on a single IP that was being rate limited by the Twitter API.
What was happening over this time? The Detroit Tigers - in a pennant race for the AL Central - were swept in a 3 game series with the Royals, resulting in rate-limit busting volumes of angst-ridden tweets from the Detroit Twitterati.
So there you have it. The Tigers are behind all the errors. (not sure we needed Apigee to tell us that.) And they still have a 4 and 1/2 game lead, FWIW.
If you'd like Twitter Growl to keep you up to date about the Tigers or any other topics on Twitter, thanks to Brian - his version is a fork of a project by visnup on github
Lots of news on the Twitter attack last week. There’s general consensus that it was a distributed denial-of-service attack (DDOS) that it targeted a particular account.
DDOS attacks are tricky things. There’s no one technique, product, or protocol that will stop them. That’s what makes them so nasty. To defend against one, a company needs to be able to quickly take countermeasures at all the different levels of the protocol stack, including firewalls, routers, load balancers, and even in the application itself. And even more importantly, you need to have experienced, well-trained operations people who know how to quickly identify an attack and come up with a way to minimize its effects.
When a DDOS attack affects a web service – and there’s an good chance that the Twitter attack made use of the Twitter API – then proper traffic management at the API level is an important part of this stack of protection.
For instance, if a DDOS attack targets a particular user account, or API key, it may be possible to block those particular requests before they get to the back-end application servers, so that the effect of the attack is limited. Or, if the attack always sends some of the same parameters in an HTTP query string, or in the body of an HTTP POST, it may be possible to detect those patterns and reject the requests, again before they reach the back-end application servers. In the worst case, it may even be necessary to shut off access to a particular web service or web service operation, allowing other services to function as best they can while the attack is neutralized by other means.
In other words, just as it may be possible to stop a DDOS attack by blocking a range of IP addresses, or redirecting certain URLs, or configuring traffic shaping in a router or an ADC, it may sometimes be necessary to stop a DDOS attack closer to the application level. The ability to inspect requests at the web services level and take action based on the request content gives an operations team one more weapon to use against an attack.
The Twitter API already has a usage quota on each user account, a rate limit on each IP address, and extensive caching. Imagine how much easier a DDOS attack might have been if they didn’t even have those things.
If you have an “open” API - one that exposes data you’d make public on the Internet anyway - consider issuing non-sensitive API keys. These are easy to manipulate and still give you a way to identify users. Armed with an API key, you have the option of establishing a quota for your API, or at least monitoring usage by user. Without one, all you have to go on is an IP address, and those are a lot less reliable than you might think. (Why don’t web sites issue “web site keys?” Because they have cookies.)
For example, the Yahoo Maps Geocoding API issues API keys so it can track its users and establish a quota, but the data that it returns is not sensitive so it’s not critical to keep the key secret.
Use username / password authentication for APIs based on web sites:
If your API is based on a web site, support username/password authentication. That way, users can access your site using the API the same way that they access it using their web browser. For example, the Twitter API supports username/password authentication, so when you access it using a Twitter API client like Spaz or TweetDeck you simply enter the same password you use when you use the twitter.com web site.
However, if you do this, you may want to avoid session-based authentication, especially if you want people to be able to write API clients in lots of environments. It is very simple to use “curl,” for instance, to grab some data from the Twitter API because it uses HTTP Basic authentication. It is a lot more complex if I instead have to call “login,” and extract a session ID from some cookie or header, and then pass that to the real API call I want to make...
OAuth for server-to-server APIs that are based on web sites:
If your API is based on a web site (so you already have a username / password for each account) and the API will also be used by other sites, then support OAuth in addition to username / password authentication. This gives users a lot of peace of mind, because they can authorize another site to access their sensitive data without giving that site their password permanently.
Use SSL for everything sensitive:
Unless your API has only open and non-sensitive data, support SSL, and consider even enforcing it (that is, redirect any API traffic on the non-SSL port to the SSL port). It makes other authentication schemes more secure, and keeps your user’s private API data from prying eyes – and it’s not all that hard to do.
Don’t roll your own!
If the above suggestions still don’t apply to you, then keep looking – between OAuth, OpenID, SAML, HTTP authentication, and WS-Security, there are a lot of authentication schemes, and each has its pros and cons.
So wrapping up API security in our series on 10 API roadmap considerations. Here are some suggested questions you may want to ask when putting together your API security roadmap:
How valuable is the data exposed by your API?
If it’s not that valuable, or if you plan to make it as open as possible, is an API key enough to uniquely identify users?
If it is valuable, can you reuse username and password scheme for each user?
Are you using SSL? Many authentication schemes are vulnerable without it.
What other schemes and websites will your API and users want to integrate with?
If your API will be called programmatically by other APIs, or if your API is linked to another web site that requires authentication, have you considered OAuth?
If you have username/password authentication, have you considered OpenID?
Can you make authorization decisions in a central place?
What other expectations might your customers have?
If your customers are enterprise customers, would they feel better about SAML or X.509 certificates?
Can you change or support more than one your authentication approach for diverse enterprise customers?
Do you have an existing internal security infrastructure that you need your API to interact with?
Up next: API Data Protection and thanks to Torbin H. for the photo.
Last week Twitter added a new rate limit to the “verify_credentials” method on their API. As a result, a number of applications that expected to be able to call “verify_credentials” more than 15 times per hour stopped working. The regular Twitter rate limit is 150 requests/hour, so Twitter is essentially saying that “verify_credentials” can only be called once for every 10 other API method calls that are made.
According to Jesse Stay, Twitter made this change without any notice to their developer community because they “assumed (apparently incorrectly) that people are only using this method occasionally.”
Twitter does a great job with their API – they make it easy to use and handle huge volumes.
But this is a great example of the need for API analytics. It should be fast and easy for any API product or engineering manager to drill down into “how often is verify_credentials called,” or better yet, “how often does the average API user call verify_credentials,” or even, “who are the top users of this API method?”
We all have Web analytics to help us make decisions for our websites - we need the same decision support for APIs.
We’ve seen very few API providers with a completely open API – almost all employ at least one of these:
Identity - who is making an API request?
Authentication - are they really are who they say they are?
Authorization – are they allowed to do what they are trying to do?
Do you need them all? Maybe not. Some APIs only need only to establish identity and don’t really need to authenticate or authorize.
API Identity vs. Authentication - Compare Google Maps and Twitter
Take Yahoo and Google maps – they are fairly open. They want to know who you are but they aren’t concerned what address you are looking up. So they use an “API key” to establish identity, but don’t authenticate or authorize. So if you use someone else’s API key, it’s not good but not a serious security breach.
The API key lets them identify (most likely) who is making a API call so they can limit on the number of requests you can make. Identity is important here to keep service volume under control.
Then take Twitter’s API - open for looking up public information about a user, but other operations require authentication. So Twitter supports both username/password authentication as well as OAuth. Twitter also has authorization checks in its code, so that you cannot “tweet” on behalf of another user without either their password or an OAuth key to their account. This is an example of an API that implements identify, authentication and authorization.
The “API Key” – do you need one?
API keys originated with the first public web services, like Yahoo and Google APIs. The developers wanted to have some way to establish identity without having the complexity of actually authenticating users with a password, so they came up with the “API key,” which is often a UUID or unique string. If the API key doesn’t grant access to very sensitive data, it might not be critical to keep secret, so this use of the API key is easy for the consumers of the API to use however they invoke the API.
An API key gives the API provider a way to (most of the time) know the identity of each caller to maintain a log and establish quotas by user (see the last section).
Usernames and Passwords – again, see Twitter
With more sensitive data a simple, API key is not enough, unless you take measures to ensure users keep the key secret. An alternative is username/password authentication, like the authentication scheme supported by the vast majority of secure web sites.
It’s easiest to use “HTTP Basic” authentication that most websites use. The advantage of using this technology is that nearly all clients and servers support it. There is no special processing required, as long as the caller takes reasonable precautions to keep the password secret.
Twitter simplifies things for their users by using usernames and passwords for API authentication. Every time a user starts a Twitter client, it either prompts for the username and password to use, or it fetches them from the disk (where it is somehow scrambled or encrypted where possible). So here it makes a lot of sense to have the same username / password for the Twitter API that it used for the web site.
Usernames and passwords also work well for application-to-application communications. The trick - the password must be stored securely, and if it’s being used by a server, where do you store it? If you are running an application server that uses a database, you already have solved this same problem, because the database usually requires a password too. Better application server platforms include a “credential mapper” that can be used to store such passwords relatively securely.
There is a lot we'd like to write about around security so we'll split this up into a couple entries. Next time: Session-based authentication, OAuth, SSL and WS-Security, and rolling your own.
"Open and Free" are often used interchangeably when talking about APIs. But if you look at in terms of the "API Economy" - open and free API are orthogonal. 
Last time we showed how Apigee 
Let's boil up the examples and common pitfalls from our last two entries on 
(This is part 3 in our series on