Tech Talk: API Visibility and Metrics »

Earlier this week, Greg speculated that Twitter might have benefited from digging deeper into API metrics and usage patterns, so we thought it would be a good time to put him on the spot with a tech talk he recorded on API visibility a couple weeks ago. 

For more, here are some sample API metrics considerations and a demo of our own API Analytics solution.

Is your API naked?  10 API roadmap considerations (part 1: visibility) »

We'd like to run a short series on product and technical requirements you might consider for an API roadmap or strategy.    We’ll base this on trends we see in talking to hundreds of product and engineering managers that are either opening or consuming APIs (or aggregating and publishing large numbers of RSS feeds.)

Instead of talking about your APIs functionality, this is about what's *around* the API features and data.  Many APIs start out as just raw naked back end features. And there is often a big gap between a raw feature and a full-fledged service, which is your API might eventually become.

So this series is about what's needed to "monetize", "productize", or "operationalize" an API.  And not just if you are providing an API to customers – much of this applies if you are consuming APIs as well.

Topic #1: API Visibility

We're always surprised how almost every company we talk to says how little they know about their API traffic and usage.   We see lots sifting through web server logs to understand usage. As the API becomes more strategic and you want to make the case for more investment - this gets more painful.

This happens a lot when an API starts as an experiment, launched by the 'ask for forgiveness, not permission' types (you know who you are).  Things like metrics or analytics are back burner until an API either gets off the ground or doesn't.

But most APIs usually end up getting more important more quickly than expected, and as a product and engineering manager you may start asking:

Who is using the API and how much are they using?

• How many clients, apps, developers are out there?
• How do they break down by type, region, class of service?
• How does usage map to existing customer or partner organizations. Or how do developers map to applications map to customers?  (This can be tough with only key or IP based tracking.)
• What parts of the API and service are they using - on a method or operation level?   
• How does traffic breakdown between the your own products and 3rd party products?  (If they use the same API.)
• What the aggregate and per developer/app/customer transaction and data volumes?

How fast and 'good' is your service? 

• What latency are customers experiencing, by developer, customer, region, or operation?
• Where are errors and user experienced bugs happening and how often? 
• How is the API delivering vs. any formal SLA have agreed to or paid for?
• How can you find out if a customer is having a problem (before they call you)?
• How is the API usage impacting the rest of the platform or web products that also use the same API?
• Can you quickly trap and debug based on a specific message? Based on what is in a cache? streaming right now?

How does the API impact the business?

• Who are the top priority customers?  Developers?   Partners?  Who should you call to up sell to a higher service tier or do a deal with?
• What do you need to show general management to make product strategy (or tactical) decisions?
• Will you need to create audit trails or metering reports for partners that are paying for API access?
• Do you need to create metrics based on a certain data value in the payload? (Such as a specific product SKU)
• What is the cost of the data that you are serving up? (if you are licensing this data)
• Are you in line with all the compliance standards that IT enforces for the on-premise apps?

Knowing this stuff is really important when opening an internal service as an API, because now customers, contract terms, and compliance regulations come into play.  Analytics and metrics help you get proactive with customers and partners, and are critical when you need to make the business case for an APIs to executives.  You probably use web analytics to help you improve your Web UI - at some point you need this for APIs as well to see where to invest.

What's your experience?  We’d love to hear what you think.  Next up: traffic management.

So you want to open an API? »

Greg Brail, our CTO, took advantage of a break from O'Reilly Velocity and a new Flip HD ultra to record a series of four short whiteboard talks on issues many face when opening an API - visibility, security, traffic management, and more.  Here is the first clip and you can preview all of these and more API case studies on our Apigee youtube channel.).  

We love feedback and these are quick to do - so any more topics you'd like to see, please let us know..