New API Management Offering from IBM – Give it a try!

To steal the words of Alex Williams, we “quietly launched” a new API Management service today. I’m very excited about this new offering. It is visually elegant, and provides some fantastic capabilities covering all aspects of API management. It offers the ability to easily socialize APIs through a developer portal and manage API consumption, including some really nice use of advanced big data analytics. Unlike other API Management offerings in the market, it also allows customers to easily create new APIs, using Cast Iron’s intuitive integration capabilities under the covers.

This offering allows our customers to combine on-premise DataPower appliances for security and policy enforcement, including two and three-legged OAuth and traffic management, with cloud-hosted developer outreach and support, API analytics, operational monitoring and control, and life cycle management.

I’m convinced it is the most complete API Management offering in the market: the combination of the market leading security gateway functionality of DataPower with these new cloud-based visual analytics, developer outreach, and life cycle management capabilities, with embedded API creation and integration capabilities of Cast Iron. It is a truly everything an organization needs to get started in publishing enterprise-class APIs.

And best of all, you can start using it for free! See for yourself:
http://webapi.castiron.com/

API Management: On Premise or Off Premise?

I have been having more and more customer conversations recently about API Management. Interestingly there are a range of discussions, from customers wanting to publish their own API & Mobile “Stores,” to customers simply wanting to open up key service interfaces over the Internet.

One thing that I continuously get asked is whether API Management functions should be housed on premise or off premise. In some ways, the decision of where to manage APIs is no different than any other on premise vs. off premise discussion. However, when security and system protection are paramount concerns, there is no question that those functions are best hosted on site, typically in a service gateway appliance within your DMZ. At the same time, there are distinct advantages to placing things like Developer Awareness and Support in the Cloud.

Ideally, you should look at each of the main functions of API Management you are deploying and determine where each one is optimal to run. Good API Management products will give you the flexibility to run things where they make the most sense.

The API Explosion

For years, I have been tracking the trend of the “information explosion” (as evidenced by the title of this blog). The basic premise of this is that the amount and availability of information is growing at an unprecedented rate as we digitize, collect, and create information from more and more sources. The “explosion” represents both an opportunity and a threat to organizations, as they must learn to understand, control, and ultimately make use of it to drive new opportunities. This trend has been well-documented and has driven a large amount of investment by organizations of all sizes.

However, I think a more interesting emerging trend may be the API Explosion. Data, without a way of organizing it and accessing it, does not have much value. However, when information can be made available in context, it suddenly multiplies in value. This is where APIs come in. APIs provide a self-describing way to get to information – offering a programmatic gateway to specific slices of information that abstracts the complexity of where it might actually reside. APIs also provide the advantage of acting as a controlled gatekeeper for the information (presuming they are deployed this way) that can limit who can see what, as well as acting as a quality of service governor that can ensure that the things accessing the data are able to rely on a predictable response.

When you think about it, APIs are largely the drivers of some of the upstart darlings of the high tech world. What are Google, Facebook, and Linkedin but a set of APIs and UIs sitting on a complex foundation of distributed information? In addition, much of the investment that has been attributed to the information explosion – including much of the “Big Data” phenomenon, has actually been focused on providing APIs that sit on top of non-traditional data sources, so that organizations can actually make use of what is valuable inside that data.

To illustrate my point, I created this quick Many Eyes graph of the number of APIs published on ProgrammableWeb per day over the past 6 years (I would have preferred to have published this as a mashup, since ProgrammableWeb has an API, but frustratingly Many Eyes doesn’t). As you can see, the recent growth is extraordinary. This is only one data point, and it is clearly biased toward technology companies rather than traditional businesses, but it is hard to argue against this trend.

So there is an information explosion, but on top of that it is the proliferation of APIs making that information accessible that is driving the change in how organizations operate.

API Spotlight: Twilio

One of the cool companies that has emerged in this new API economy is Twilio. Twilio provides APIs for building voice, voip, and SMS applications. Essentially, Twilio acts as an intermediary between phones (voice & SMS) and back end applications, allowing verbal and phone-based commands to be translated into back end API calls on their customers’ sites.

This allows organizations to quickly provide phone-based APIs to allow them to build IVR-like interactions into any application, or even add phone functionality into any application (for live support or other purposes).

It seems like a pretty simple idea, but Twilio has built this into a decent business, with flagship customers like Intuit, Salesforce.com, and eBay. It is one of a few good examples of the API service economy that has emerged around this growing trend.

Managing Your APIs

Copyright Lara Jade Photography

One of the more interesting side effects of the democratization of the API is the rise of the faceless application and the new challenges these applications pose. Whereas organizations used to publish just web pages and Web apps externally, they are now compelled to publish APIs. These APIs can’t take advantage of all the infrastructure that was developed over the years to support Web pages. For example, the browser itself handles a lot of the heavy lifting for client authentication, and there are specialized tools for lifecycle management of web content, optimizing web content delivery, and other common tasks.

All of this changes when you move to APIs. Let’s look at 5 areas of requirements for managing APIs that require a new approach.

1. Security: Security is one of the major changes as you move into APIs. In the web app world, authentication is managed by passing security credentials from the browser. Since a human is involved to enter and manage the credentials, and the browser handles the secure channel, it all works pretty well. However with APIs, you’ve got a client application on the other side, and no browser infrastructure to manage secure credential passing. That is why the OAuth standard was created, which provides an easy way to authenticate API requests and works easily with non-website services. Google, Yahoo, Amazon and others had previously created their own protocols for this, but OAuth seems to be the best of all worlds and is emerging to be a more broadly accepted standard.
2. Lifecycle Management: In the Web app world, lifecycle management is important to ensure orchestrated rollout of content (and avoid broken links, etc), but it is even more important with APIs. Since applications are the things accessing the APIs, any change will likely break them and potentially cause further reaching damage. Therefore, you need to be even more careful in managing the rollout of API changes. This includes things like making multiple versions of the API available simultaneously to support applications that haven’t “upgraded” to the latest version. While there aren’t many great standards here yet, service registries often provide this capability, and some support RESTful services in addition to the Web services for which they were originally designed.
3. Developer Awareness & Support: What good is having an API if nobody knows about it… and how do developers get documentation, authorization keys, and notification of new versions? Unlike Web pages, there is no Google API search (though I suspect that there will be soon – and according to Kin Lane, Swagger may provide the answer). Also, since many companies publishing APIs these days are not software companies, how do they provide support for developers wanting to use the APIs? Online developer resources need to be provided to support these requirements. Though many companies choose to build their own infrastructure for this, increasingly they are turning to API Management solutions like Mashery to fast track these capabilities.
4. Operational Monitoring & Control: This is a big area for Websites also, but APIs can’t use the same tools that Websites use. When deploying APIs, companies want to know who is using them and how they are being used. They also want to understand volume and distribution of API requests. This is also critical for billing, when companies want to bill for API usage (which also requires integration with billing applications). In addition, they want to be able to control and throttle access based on the user – potentially even doing things like limiting the number or rate of calls per user, or the times the API is available to specific users. Much of this can be provided from a combination of service registries and service monitoring tools, but many API Management tools also provide some of these functions.
5. Performance & Availability Optimization: Performance is always an issue, whether you are dealing with websites or APIs, but often applications are even more sensitive to bottlenecks, and they are certainly more sensitive to outages. Unfortunately, many of the tools and techniques used by Web apps don’t translate directly to APIs. For example, Web page caching is different than data caching for applications, both at the application layer and across the Internet with services like Akamai. So, the appropriate caching and high availability technologies need to be used when you move to APIs.

So clearly as you move into the brave new world of publishing APIs, you need to think differently about your technology, and make some additional investments to ensure that you are doing it right.

API Spotlight: Google APIs

If anyone has mastered the art of accessible APIs, it is Google. In fact, they are even creative in how they publish them. They’ve laid them out in a periodic table format, complete with clickable links to details.

Here is a screenshot:

More on API proliferation

In reviewing the feedback on my last post        (the bulk of which can be summarized as: “you silly silly man, have you only just now comprehended the basics of SOA? Where have you been for the past decade?“), I realized what I was really describing is the current state of SOA, and while the technique of architecting things as loosely-coupled services that can be strung together in new ways is not new, the number of easy ways there are to do this (what I called “democratization”) is relatively new – and the number of service-oriented things available to enterprises (both in their internal systems and in the cloud) that can be strung together in this easy way is reaching critical mass. In fact there’s an entire industry of new companies that are built exclusively on this new opportunity. In short – the sheer number and availability of interesting APIs is multiplying at an exponential pace. It is the impact this is having on businesses that I find so interesting.

So what does this all mean to the average organization? Well, I think it means that organizations need to understand the possibilities this brave new world offers, and have a plan for investing in those possibilities. For some organizations, this will simply be a better way to do things, like market to customers or design products. For others, it will include actually producing and publishing their own APIs – as a way to directly produce revenue, or as a way to reach new customers (hell, even NPR has an API!). Either way, organizations will need to adapt to this trend, or risk getting left behind by competitors who are discovering ways to use this to their advantage.

The Democratization of the API

Image copyright Scientific American 2011

It wasn’t too long ago that if someone brought up the term “API” (Application Programming Interface) in a meeting, it was a surefire sign that they were a geek. APIs were complex proprietary things that required specialized knowledge of applications and advanced programming skills in Java & C++. Mention of “APIs” would elicit approving nods and smarmy smiles from those elite holders of knowledge in the room who could claim dominance over them, but everyone else could only conceptualize what that meant, and turn hopefully to the geeks for reassurance.

These days, everything has changed. APIs have gone mainstream. They are no longer restricted to those elite few who speak in brackets and parenthesis. It started with Web services, which lowered the bar by opening up the number of programming languages that could be used to interface, but still required mastery of SOAP headers and parsing skills. But what really changed things for good was REST – which is really much more like simple commands issued over HTTP. Suddenly, anybody could call an API, and those complex barriers between applications went away. Non-programmers could connect Facebook and Google Analytics into Excel spreadsheets and actually accomplish something. As enterprise applications have added these types of interfaces (and organizations have adopted replacement applications built from the ground up on this approach), this phenomenon has spread. Now connecting data from Salesforce.com with data from Demandware to understand buying patterns, or even building a complete retail storefront in Facebook – projects that previously would have taken months or years – can be done relatively quickly and easily.

This really isn’t new news, I realize, but it struck me as I was out for a beer with my friend who works for Demandware. He referred to what they sell as a set of APIs (and he’s a Marketing guy!). Initially I thought this strange that he would refer to it this way, because I would have called them an application. But then I realized, the APIs are the application. This is the way of the new world – people don’t hold to the same application boundaries they once did – they may mix and match information from across many applications and use it in any variety of new ways. The things that facilitate this the best are the things that are beginning to win in the marketplace. This is why Google is so formidable… they have access to an enormous trove of information (all the public information in the world, in fact) – and they have the ability to provide access to it through any number of APIs, and recast it in new ways through analytics, recombination, or summarization to provide value on top of it (an emerging practice Stephen O’Grady describes well here).

This capability is critical to my “murmuration of systems” concept. When organizations begin to see their systems as simply a set of APIs, they can do the sort of recombination of this system DNA that allows them to evolve more quickly. Organizations are no longer bound to how a system is designed – they can simply pull what they want and use it how they want. Pretty cool if you ask me.