Not so long ago there were only a few internet giants – portals such as AltaVista, Excite, Lycos – all fighting to win our attention, with a whole mix of data covering everything from news and weather right down to theatre and restaurant listings. Each site had its own set of information, its own databases and editorial staff, regurgitating anything new that surfaced but plastered with own branding and links to other in-house content. Servers across the world were quickly filling up with repeated instances of the same junk. Search engines at the time were confusing and biased, preferring to return results hosted on sister sites and partner networks, followed by hundreds of completely unrelated sites whose authors had learnt how to successfully spam the meta-data. Something needed to change before the web went stale, and in September 1998 that something came along as the brainchild of two Stanford students, Larry Page and Sergey Brin.

Ever since Google launched, streamlined web applications and services have been the key to growing a successful site. Some of the biggest online brands today – YouTube, Flickr, eBay, Wikipedia – have all become giants in themselves by following in Google’s footsteps of specialising in one particular service, and emphasising it by using minimalist layouts – often on a clean white background. A mass of new data has found its way onto the internet, increasingly more personal and spurring on the growth of user-generated content. Things have got really interesting in the last couple of years, however, with most of the major players jumping on the open source bandwagon and opening up their frameworks, giving developers free access to their application programming interfaces (APIs). The result? Thousands of new sites mashing data together to provide the same information in a different context. For example, a news article with an embedded video streamed directly from YouTube, an image gallery fed by Flickr and a list of related articles straight from CNN. As sites become more like containers for content rather than hard-coded pages, Tim Berners-Lee’s vision of the Semantic Web is suddenly sitting right on the doorstep.

But it’s more than that. Our perception of the web and what we can do with it is beginning to gain solid ground, as is our attitude towards it: share and share alike. Even though it’s still very much in its infancy – akin to a library that’s only just been fully stocked with books – we’ve still got so many more ways in which to sculpt the information becoming available to us, and we’re only just discovering them. Admittedly, the amount of personal information stored on servers across the world is a little bit frightening, but measures are being taken to protect our privacy

The world’s most respected broadcaster has become a formidable online force, placing instantly updated content into the ether as soon as the stories break. With a commitment to journalistic quality that more than measures up to its other broadcasts, it rightly takes centre stage when it comes to delivering freshly squeezed internet news. We’ve placed it within our ultimate mashup project to supply relevant news and current affairs stories applicable to the example data scenario – in our chosen case, U2. With a vocalist who is never shy of seeking world press, particularly political, we would expect plenty of associated articles to be fetched by the main Google query. This would largely be performed by RSS feeds that could be built on the fly to include the chosen search terms, displayed accordingly and then followed to read the content.

Although BBC Online can pride itself on being admirably open to developers, encouraging independent creativity with its content, we are primarily concerned with its news content. Like iTunes, the Beeb has made it really easy to pick chunks of data out and add it to your pages, with customisable queries made possible by appending search keywords to a URL. The example below will search the News and Sport archives for article entries containing example references to U2, Bono and the Edge. Simply change or replace the keywords after the ‘/%7B’ prefix, using addition signs to attach extra search terms.

More information on the provision and use of BBC feeds can be found at http://news.bbc.co.uk/1/hi/help/3223484.stm#my feeds.

Apple Inc. found its way into our ‘major sites’ list not because of the information it offers or because of having a clever Web 2.0 service – instead, we’ve given it a spot because of the major branding power the company has built up for itself. Their customer base has an emotional attachment to the brand – they’re buying an idea rather than a product. John Sculley, former CEO, told the Guardian in 1997, “People talk about technology, but Apple was a marketing company. It was the marketing company of the decade.”

A huge part of the Web 2.0 phenomenon, Apple’s graphic style – introduced in 2001 with the launch of OS X – pretty much laid down the rules for the look and feel of all successful sites since: simple, reflective gradients on top of clean, white, minimalist layouts. It has become so popular, in fact, that the term Web 2.0 is commonly associated with the style rather than the idea of usergenerated content. As such, we’re going to take a leaf out of the Apple tree and fashion our fantasy mashup in the same vein.

Another website that absolutely exploded in the user-generated content arena, Flickr was launched in 2004 and is considered now as one of the earliest Web 2.0 applications. Although primarily a photo-sharing website, a massive, hugely devoted community has built up around it with over an incredible two billion photos hosted. Flickr’s inspired ability to tag and browse through images by folksonomic means – in other words, freely chosen keywords that completely ignore hierarchy – along with AJAX-driven organisational tools have been key to its success. Considering that it is a free service to a certain point, it has been quickly adopted by the blog community for easy photo management – but bloggers aren’t the only ones who can benefit from Flickr’s functionality. More than three million images are geotagged, in turn enabling the crossover into travel and tourism, for example. All of Flickr’s public data is open to use in custom applications – including photos, videos, tags, profiles and groups, – and because it is possible to actually post to Flickr from a custom application, the possibilities really are truly endless.

Amazon’s inclusion on our hit list is due to its hugely successful site navigation. By separating product categories with different tabs at the top of the page, it was very easy to browse straight to the item in question. After the expansion that saw Amazon start to distribute consumer electronics, CDs, DVDs and even food, the tabs took on different colours and another hierarchical level – but essentially remained the same. Competitor and affiliate sites quickly adopted virtually identical systems, often a direct replica – bolstering Amazon’s position as a first mover. But what really blew the competition out of the water was the user-submitted reviews. Letting even negative comments about the product to be published was one of the defining movements of the Web 2.0 generation, Amazon’s founder Jeff Bezos explaining that, “We want to make every book available – the good, the bad and the ugly… to let truth loose”.

Amazon’s product library, complete with submitted reviews, are fully available to plug into your own sites – visit Amazon Web Services (link above) to find out more.

With the help of a certain portable media player, Apple has managed to align itself as one of the premier Web 2.0 brands. Via iTunes, it has become synonymous with online audio and MP3 in the same way that YouTube and Flickr respectively represent video and images, ensuring a market share that has reinvigorated the firm completely. By securing the rights to distribute key artist and album recordings old or new, it boasts an archive that can stream track previews seamlessly to your site thanks to RSS technology. Other than that, there aren’t any more developer APIs available as yet for performing sophisticated mashups, while it might take a hack or two in order to get the specific RSS results you may desire. In our mashup scenario, iTunes would be used to pull audio – either music tracks or related podcasts – into our results, which would add previews of corresponding multimedia you could then reference.

A great point of reference for implementing the basic RSS services of iTunes into your projects is the generator that Apple has linked to the store. Head over to:

Here you’ll get a simple drop-down list box where the most popular feed URLs are displayed based on the options you pick. For example, to embed the top five iTunes songs for the UK store across all genres and explicit content, you’d use:

Therefore, it would seem feasible that more specific queries could be used to build custom RSS feeds, or certainly create the XML needed to do so. You’ll find more information on how to do so at the main Apple developer URL (http://developer.apple.com), although taking a gander at the browser page source of these feeds yields some good clues. We found a form query that could be hacked to deliver band search results, although we’d like to hear from anyone who can display the results in the browser as opposed to iTunes!

YouTube is one of the most successful websites of all time, a leader of the usergenerated pack. Despite the fact that it’s only been around for three years, it has quickly grown into the third most visited site – only slightly behind Yahoo! and Google. Part of this may be due to Google’s US $1.65bn takeover of YouTube in November 2006, and consequentially videos being displayed at the top of the larger company’s search results, but it goes without saying that the popularity is largely down to giving anyone with a webcam five seconds of fame and a place to have their voice heard. There are, at time of press, 84 million videos hosted on the site with four million registered users. The bandwidth usage alone is estimated to cost $1m a day. Luckily for developers, the entire YouTube video repository is open for access via the GoogleData (GData) API, along with all the relevant community features. When the video-overlay advertising system is introduced later this year – structured in a similar way to the Google AdWords and AdSense programs – the potential revenue stream for content and site owners will be lucrative. It’s a good time to jump on the bandwagon…

In many respects the most ‘next-gen’ of the current web brands, Second Life is less of a site and more of a world. Like the leading MMORPGs, it commands a huge online audience but has gone a step beyond by restyling itself as a virtual community or communication channel – hosting live web events, conferences and interviews. This is Facebook for the future and could well suggest a much more human internet experience that could influence the way our other big sites could behave one day. Within the context of our mashup, it would provide a direct link to personalities, significant virtual happenings and related lectures, etc, offering immersive 3D renditions of locations, museums, landmarks, galleries and even shops. Wishful thinking perhaps, but it isn’t that far off, particularly with creator Linden Labs beginning to offer up developer APIs that can embed Google-style maps of the fictional locales it inhabits.

Although Second Life still has some way to go in terms of opening itself to mashup developers, the beta technique for adding maps of varying complexity can be found at:

As a taster, here are the first few steps you need for embedding a basic map into your page. Start by placing the line below between the head tag of those pages using the API:

Next, add a div element which will contain your map. Setting the position attribute of this element in the CSS will cause it not to work, so watch for that:

Then add the following JavaScript to the page to instantiate the map and centre it:

This is best invoked when added to a onload() event handler, while the positioning and zoom parameters can be tweaked too.

Somehow it seems unnecessary to explain what Facebook’s all about – if you’re not a registered user, we’re sure that you will have heard of it or even disowned your partner for spending too much time on it. One of those applications that sort of fits into the Web 2.0 bracket, Facebook’s part in the social networking phenomenon is – without a doubt – completely down to its community, and thus user-generated content. Being able to use its massive data set is of huge social and commercial benefit, and Facebook duly launched a ‘platform’ for us to play with. However, they’ve been heavily criticised by the developer community for their ‘half-open’ API; they do give direct access to user content in order to build custom applications, but only if the app runs on the Facebook site. It’s exactly this sort of tight control that the internet is moving away from, and has seen several major players lose their market control for doing the same thing.

Last year, a developer called Christian Flickinger wrote a script that pulled Twitter posts and pushed them to Facebook as status updates, externally via the Facebook mobile platform. They quickly shut down his account and insinuated legal action, with any other developers who used the code following suit – their terms state that interacting with Facebook externally is prohibited. With all the industry pressure, we’re pretty sure they’ll give in soon, hopefully realising that opening up the community will lead to even greater brand awareness. In the meantime, you can pull your friends’ status updates, notes and posted items via an RSS feed. Log in to your account and open up your profile page, and under the Mini Feed heading click on See All. Click on the feed you want to pull (we’ll use Status Updates as an example), then right-click on My Status and copy the link. There’s your feed, let’s just hope they allow more in the future.

Founded in 1995 by programmer Pierre Omidyar, eBay was originally set up under the name ‘Auction Web’ to sell a broken laser pointer. Within three years eBay made its IPO and both Omidyar and company president Jeffrey Skoll became instant billionaires. True, it’s redefined the way in which goods are traded; however, its success comes not from the potential buyer discounts, but instead from connecting buyers and sellers from all four corners of the world. If you’ve completed a transaction on the site before, you’ll know what we mean when we say that doing business on eBay is fun. Even though it was launched before user-generated content became a buzzword, the online auction giant could easily be considered part of the Web 2.0 movement just because of the community that takes part – some even making a full-time living from it. As a result, eBay has become integral to the buying habits of most Westerners, and has seen the benefits of opening up its application layer to developers. Tailored apps can be built to return completely customised search results, as well as manage listings externally.

Google was the first of the streamlined web services, offering nothing more than the nowfamous search facility. Like any successful brand in the commercial world, its strength and sticking power has come from focusing on providing just one product – a move that not even ten years later has seen the company grow to over 19,000 employees and raking in nearly $17bn revenue in 2007 alone.

Google quickly dominated (and pretty much defined) the search market, with ever-more users setting it as their browser’s start page – at the end of last year, 50 per cent of all online searches went through Google.

A number of tools are available for developers to work with, providing access to everything from their advertising systems to their user data. In particular, they give open application layer access to the main search function, as well as Calendars, Charts, Maps, Blogger, FeedBurner and Earth to name but a few – it’s well worth giving the developers site a visit to see what’s possible.

With reference to our monster mashup, there was no way on earth that we would ever leave Google out. Although there are plenty of other search engines out there, Google was our definitive choice.

It’s reasonable to say that Wikipedia is the quintessential site of the Web 2.0 era – a massive knowledge base containing ten million articles on just about everything. Completely usergenerated, every article has been written by volunteers. It’s the largest, fastest-growing collaborative reference work in existence – and still growing. Completely free to use, its nonprofit, no-advertising approach has helped its growth immensely, embodying the open source and democratic spirit of the internet, earning a loyal volunteer base.

Currently there isn’t any API or even RSS access available. However, there are several workarounds. One is to use the special export function, which returns page content inside a wiki-formatted XML wrapper:

Another great option to tapping Wiki content is the community-built DBpedia API. Visit http://dbpedia.org for more information.