Power up your website with the might of APIs
2nd Aug 2009 | 07:00
Harness the best API tech to create richer sites and apps
Get creative with the most popular APIs
The popularity of application programming interfaces (APIs) is now such that they're considered just another part of a web developer's arsenal.
Reasons for API usage are many, but mostly relate to accessibility, in the sense that you gain access to masses of data and functionality that you wouldn't be able to afford or develop on your own.
Developer Tantek Çelik likens APIs to building blocks, enabling savvy developers to rapidly fashion richer applications, sites, tools and services. Web editor Patrick H Lauke concurs, adding that a key benefit of APIs is taking advantage of services that provide narrow functionality very well.
"There's Flickr for photo management, Amazon S3 for storage, Twitter for microblogging," he says. "In many cases, it's cost-effective to use these rather than building bespoke or separate equivalent functionality." In other words, as developer Shaun Inman says: "You offload your content management and related maintenance, but then aggregate said content on your own sites."
Old-fashioned means of embedding 'foreign' content, such as frames, become rarer when APIs are used and, short of APIs being amended, content arrives in a predictable and an often styleable manner, enabling you to repurpose content as you see fit.
Another benefit, says Yahoo developer evangelist Christian Heilmann, is in API usage potentially enabling your data to spread beyond the sites it's being aggregated on: "If I upload photos to Flickr to include on my site, all the conversion is done for me for free. Furthermore, I tap into a community of people interested in tagging and commenting on my data. This can be pulled back to add keywords and make my data more findable by search engines. And in the event of my server going down, the information will still be available."
IN THE CHARTS:Twitter plus the Google Chart API enabled Christian Heilmann to create 'Twitter type' charts
Of course, with the current popularity of APIs and their relative ease of use, there's a danger of going 'API crazy' – using as many as possible. "You know you're not using APIs properly when, rather than thinking about something you want to achieve and searching for an API to assist, you instead find cool APIs and wonder where you can use them," warns developer David Dixon.
It's also arguable that although 'typical' API implementation is fine for some sites, better designers should go further. "Creativity versus 'lists of data' is my favourite thing to rant about," claims interface developer Amy Hoy. "Generic designers suffer from 'it just lies there' syndrome, because that's what the data does: lies there, limp, lifeless and unappetising."
Hoy notes that no one wastes time figuring out why they should care about your 'thing', and so you must engage them and hold on: API use must be relevant, offer visual interest and provide a reason for someone to care about your data.
"Information doesn't speak for itself," continues Hoy. "We must give it a voice. To do that we must understand the data but also human nature – data isn't information without a brain and personality to interpret it."
This doesn't mean API usage should veer towards art and away from the practical, merely that more thought needs to go into why APIs are used and relevant visual design. Lauke talks of sites that "integrate disparate sets of data to reveal emergent behaviours, unexpected correlations or serendipity, or provide an experience to the end user that's richer and more engaging", and cites Dopplr's display of a user's past trips.
This mixes Dopplr trip detail data with Flickr photographs the user took during a trip's timeframe, cross referencing calendar events and relevant city guides. Each component might resemble Hoy's 'lifeless' data, but the combination is compelling.
Of course, APIs aren't just about mash-ups and data visualisation. "They can be used in subtle ways that create better user experiences," explains Çelik.
By way of example Brendan Dawes recalls that a client of his previously had to log into Brightcove, upload a video, copy the URL, move to their CMS and add the content. "Using Brightcove's API, we integrated the video upload part of the workflow within the CMS. In using an API, the workflow was simplified and more user-friendly," he says.
Dawes also takes 'subtle' further, with coloured wires on brendandawes.com created from found palettes using the colorlovers.com API. "Why? Because the palettes are created by human beings, so I can be sure that using top palettes from a search for the word 'love' will return colours that will pretty much always work," he explains.
"Integrating the API added so much to the aesthetic, and the palettes grow over time, crafted by human hands, so I'm constantly surprised by colours that appear on my own site."
Exploring APIs Developer Andrew 'Spode' Miller says experience with APIs has taught him that they mostly have an underlying similarity: you request information, get XML back and process it. But even the most widely used APIs differ enormously in terms of functionality.
So popular as to be almost ubiquitous, working with Google Maps hardly feels like you're using an API. "But don't forget Google Maps was responsible for the popularity of web APIs," says Cameron Adams. "Before, we hardly knew how important it was to access geo information. Now we can't live without it."
Although tools exist that mean you rarely need to go near the API, it remains handy for dynamically adding data to a map: in Adams's workshops, he takes the latest CNN news stories, converted from RSS to JSON via Yahoo Pipes, geocodes their location (with Maps), and draws them onto a map.
Engage Interactive developer Neil Charlton thinks Google Maps is the perfect example of how users can benefit from the use of APIs: "It provides simple maps, complex route planning and a real-time geocoded data display, utilising satellites, countless cameras, and a massive index of places and names. If it didn't exist, few users would be able to write something similar."
SMART CMS:Engage Interactive CMS offers headline 'e-marketing' campaign stats and web stats on the dashboard page
Still, Google Maps isn't faultless, but the nature of APIs is that they're extensible. Irked by the lack of keyboard accessibility provided by default Google Maps controls, Lauke used API methods to craft his own set that hooks into the standard methods. "An API can deal with situations where a site or service doesn't by default provide an accessible interface," he adds.
Similarly, accessibility concerns inspired Heilmann to create Easy YouTube, a YouTube player for people with learning disabilities: "A conference speaker complained about the lack of a video player for people with learning disabilities. Since YouTube had released its 'chromeless' player API earlier that month, I considered that a perfect opportunity to have a go."
After signing up for a key, Heilmann worked with the demo page, which provided external controls for 'play', 'stop' and loading a movie. The API got around the Flash accessibility problem of only IE enabling getting into a movie with a keyboard. With feedback from mailing lists, the player went through several iterations, and the final version won funding for the Scripting Enabled conference.
Heilmann notes that despite initially merely wanting a YouTube player with bigger controls, he inadvertently created an accessible player that got him thanks from blind users – and it all started with an API.
ADDRESSING A NEED:Accessibility concerns inspired Christian Heilmann's Easy YouTube, a player for people with learning disabilities
The dark side of APIs
A tweet in time
Recently, Twitter has become a media darling and developer favourite, its simple concept providing basic building blocks ripe for repurposing. Pulling tweets into a blog maintains life signs, and Twitter is unparalleled as an early warning news source. "Twitter's also new to the layperson, so providing your latest tweets can show someone how Twitter can benefit them," says Miller.
"At a more advanced scale, enabling users to add tweets to an article that's also added to their Twitter feed is excellent advertising for your site and simple for the user, encouraging engagement." Twitter's relative simplicity enables radical reworking, too, such as accessibletwitter.com's alternative interface for users with specific access requirements, and 'visualisation', which Heilmann interprets as "using the power of APIs to make information more relevant by showing it in a different light".
By example, he mentions his script that displays your 'Twitter type' – a pie chart detailing whether you're a 'follower' or 'being followed' – which also uses the Google Chart API. Heilmann prefers the likes of Twitter to "massive Facebook-style catch-all APIs. I like seeing the web as a collection of data and helpers for me to make sense of it," he explains.
"A good API does one job and does it well, but developer programs like Facebook lock me in. When using them, I build for Facebook and not the web as a whole. This is when an API becomes a developer program or network. In a perfect set-up, you could use parts of a framework without having to subscribe fully to the network."
Heilmann says this is the case with Yahoo Pipes, which he uses to filter tweets: "Every tweet ending with § goes to my blog, ensuring I don't flood it with information nobody needs there." Adams also recommends Pipes, calling it "one of the most interesting web application interfaces and the most appropriate use of visual programming I've seen".
Although Pipes enables complex interaction between disparate data sources, Adams says he often uses it for relatively simple transformations, such as filtering items from RSS feeds, or accessing RSS via Ajax by transforming any RSS feed into a JSON-P object. "As with many useful APIs," he says, "Pipes fills a need you never knew you had, acting as the blender of the web."
The dark side of APIs
It's not all sunshine in the world of APIs, though. Newcomers must beware of many things before taking the plunge, not least that over-reliance on APIs can spell disaster if a service fails; and vendors can vanish or amend APIs on a whim, giving you no option but to rewrite custom code.
"Another issue is getting access to APIs," explains Heilmann, who recalls signing up for the TFL developer program to create a mashup of UK traffic cameras, only to discover a complex sign-up process asking for the range of IPs the hack would run on and requiring his email be part of the API URL: "That's needlessly hard for me to do and my email shouldn't have to be readable only to access data."
Worse is the proprietary nature of APIs and the variance in how they work. "Each and every one expects different parameters, returns data in different formats and has different terms and conditions," complains Heilmann. The first two issues can be overcome using YQL Open Data Tables, he says, but terms remain a thorny issue.
"My official Guardian hack violated their terms, due to me adding paragraph tags to the content to make it more readable – this meant I'd modified the content." You're also limited by what an API enables, unless you're willing to stretch what's officially allowed via hacks or workarounds.
Adams says most of his frustrations with APIs are summed up by one example: "I wanted to get a user's second level Flickr contacts. This is possible, but you must make a separate API call for each contact, and this quickly multiplies the calls you have to make, ultimately making it unfeasible to do such operations unless you're prepared to pre-crawl the data."
Adams notes that if you had access to the Flickr database, this operation would be trivial, but access via the API is subject to network latency restrictions, API rate limiting and various callbacks, and so not everything can be done with APIs.
Dixon suggests this is just as well, as he thinks APIs are overused: "Too many developers don't consider what a site really needs. This reflects badly on the developer and results in sites becoming interchangeable." He cites countless mashup sites that "do little more than aggregate information from other websites, and give a bad name to APIs".
Drawbacks aside, it's clear that APIs are now a cornerstone of web development, and even with the most popular of APIs, Dixon concedes there's hope: "When businesses think of innovative uses for the Twitter API to benefit them, that's when it will become successful long-term – not with these 10-a-day sites that tell me how often someone mentions my name on a tweet."
Undoubtedly, this consideration (once the 'new toys' mentality wears thin) mixed with existing and new technology should confirm rapid escalation of API use. In technology terms, it's OAuth that's currently getting many developers excited. Çelik describes it as "an essential authorisation building block for the continued growth and development of the web," adding that it "enables different websites to perform various functions for a user, while allowing that user to share private data across those websites with specific authorisation".
Miller thinks along similar lines, suggesting OAuth has the potential to eradicate the main barrier in encouraging online interaction: the hassle of signing up. Some developers remain negative, citing OAuth's awkwardness and difficulty, along with logistical concerns.
Inman, for example, comments: "There's redundant programming overhead, since you must implement traditional login for those without an OAuth provider; and as a developer and user, OAuth introduces a third-party whose reliability and longevity cannot be ascertained." But OAuth does hint at a near future where developers are relieved of figuring out how to handle sensitive information, yet can create increasingly elaborate mashups by pulling data from a mixture of different social networking applications.
Standardisation elsewhere should be of further help, enabling services to increasingly be 'glued together', while the likes of YQL Open Data Tables will become more popular, according to Heilmann, owing to them "providing an easy way to have an API without having to create your own back-end or document your output formats".
Organic evolution, with APIs increasingly worming their way into basic workflow, should also cement their purpose and popularity. Charlton notes that "APIs increasingly aid in more than adding functionality – they're used to provide the very infrastructure sites run on". He considers Amazon Web Services a great example. "You can store all your website assets (S3) and data (SimpleDB), and through an API add more computing power (Amazon EC2) to your application," he says.
Ultimately, what we're seeing at the moment is just the beginning. As Amy Hoy concludes: "I don't think we've gone near the limit of what can be done with Twitter or any other API. These things look simple, but they run deep."
First published in .net Issue 191
Sign up for TechRadar's free Weird Week in Tech newsletterGet the oddest tech stories of the week, plus the most popular news and reviews delivered straight to your inbox. Sign up at http://www.techradar.com/register